Field wheat three-dimensional character analysis method based on unmanned aerial vehicle multi-view three-dimensional point cloud

Abstract

The invention discloses a field wheat three-dimensional character analysis method based on unmanned aerial vehicle multi-view three-dimensional point cloud, and the method comprises the following steps: S1, shooting a multi-view image and carrying out the three-dimensional reconstruction, and obtaining an original point cloud; performing interpolation segmentation and secondary segmentation, and outputting a plurality of cell point clouds; s2, performing data enhancement, starting model training, and obtaining each 3D bounding box; s3, extracting point cloud data of each planting row from each 3D bounding box by using a row segmentation algorithm, and calculating each static phenotype parameter; and S4, calculating five dynamic phenotypic parameters of the field wheat. According to the invention, based on the 3D wheat plot detection network and the row segmentation algorithm, the point cloud data and the 3D bounding box of the wheat are rapidly extracted, and accurate monitoring and accurate segmentation of the wheat are realized; based on accurate analysis of static phenotypic parameters, a new system of dynamic phenotypic parameters is provided, and based on the dynamic phenotypic parameters, the growth vigor and yield of wheat can be effectively predicted.

Description

Technical Field

[0001] The present invention relates to the field of agricultural technology, and in particular to a method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point clouds of unmanned aerial vehicles. Background Art

[0002] From the jointing stage to the maturity stage of wheat, the ear formation and filling process are the key growth stages that determine the final yield. During this period, the demand for water, nutrients and light increases significantly, which directly affects the grain filling efficiency and development quality. Therefore, by accurately monitoring the growth dynamics of this growth stage, water and fertilizer management strategies can be optimized to achieve yield increases. Plant phenotype, as an observable representation of the interaction between genotype and environment, covers multi-dimensional characteristics such as morphology, structure, and physiology; among them, morphological parameters such as plant height, canopy area and leaf area index can directly reflect the growth status of wheat. When phenotypic abnormalities occur in specific varieties, these indicators can provide a scientific basis for precision agricultural decision-making, thereby maximizing yield.

[0003] Traditional wheat phenotyping mainly relies on manual sampling, but most studies only randomly select sampling points for local measurements. This random sampling method has drawbacks such as ignoring differences between planting rows and lacking continuity in time series data, which can easily lead to statistical bias. With the development of computer vision technology, sensor-based image acquisition can effectively overcome human errors and, in conjunction with specific algorithms, can quickly extract phenotypic parameters. Unmanned aerial vehicle systems, with their multi-sensor integration, have demonstrated outstanding advantages in collecting phenotypic data such as crop morphology and structure. UAV systems can be used to collect RGB two-dimensional and three-dimensional images of wheat at different growth stages and perform trait analysis. Although RGB images can extract important phenotypic parameters such as canopy height and canopy area, their two-dimensional nature makes it difficult to identify row boundaries under dense planting conditions, limiting the fine demarcation of planting plots. In contrast, three-dimensional images have an advantage because they contain depth information and are more easily used for automated processing of high-density field plots.

[0004] However, extracting the point cloud containing crop plots typically uses algorithms such as pass-through filtering and plane detection. These conventional 3D processing algorithms often require numerous input parameters, resulting in cumbersome processing and limited generalizability. Utilizing a 3D detection network to extract object bounding boxes offers higher computational efficiency than segmentation networks that focus on point-by-point labels. However, there is currently no effective 3D object detection network for wheat plots. Furthermore, the characteristics of the rows in each plot vary, and algorithms for plot-row segmentation are lacking. Summary of the Invention

[0005] In view of the above problems, the purpose of the present application is to provide a field wheat three-dimensional trait analysis method based on multi-view three-dimensional point cloud of unmanned aerial vehicle, a 3D wheat plot detection network specially designed for multiple planting plots of field wheat is proposed, combined with multiple flight modes of unmanned aerial vehicle and photos taken at specific angles, the three-dimensional features of field wheat are accurately obtained by using candidate box generation system and candidate box optimization system, and the performance is significantly better than that of existing mainstream three-dimensional detection network; based on the features of point cloud data, three-dimensional point cloud segmentation at planting row level is realized, and the problem of dense planting row segmentation that two-dimensional image cannot handle is successfully solved; in addition, combined with accurate analysis of static phenotype parameters, a new index system containing five time dynamic phenotypes is further proposed, and based on these dynamic phenotype parameters, the growth and yield of field wheat can be effectively predicted, and technical support is provided for precision agriculture cultivation decision.

[0006] The technical scheme is realized by the following: A field wheat three-dimensional trait analysis method based on multi-view three-dimensional point cloud of unmanned aerial vehicle, comprising the following steps: S1, using unmanned aerial vehicle to shoot multi-view images of field wheat and perform three-dimensional reconstruction to obtain original point cloud, three dimensions including x, y and z; performing interpolation segmentation on the original point cloud to obtain multiple nitrogen gradient point clouds, and then performing secondary segmentation on each nitrogen gradient point cloud to output multiple plot point clouds as total samples; S2, performing data enhancement on the total samples in step S1, dividing the data enhancement result into a training set, a test set and a validation set, and starting model training of 3D wheat plot detection network using the training set; wherein the 3D wheat plot detection network comprises a candidate box generation system and a candidate box optimization system, during model training, setting various parameters of model training, then outputting each 3D candidate box using the candidate box generation system, and then obtaining each optimized 3D bounding box based on a region attention module, a feature extraction module and an optimization module using the candidate box optimization system; S3, based on each 3D bounding box, using a row segmentation algorithm to extract point cloud data of field wheat at each planting row from each 3D bounding box and calculate each static phenotype parameter; wherein the static phenotype parameters include plot height, canopy area, leaf volume and leaf area; S4, according to each static phenotype parameter in step S3, calculating five dynamic phenotype parameters of field wheat, including trait rapid change period, trait slow change period, trait reduction period, maximum height difference or maximum area difference, height change rate or area change rate.

[0007] Preferably, the wheat in the field includes multiple varieties, and each variety is provided with three nitrogen fertilizer gradients, including a zero nitrogen fertilizer gradient, a medium nitrogen fertilizer gradient, and a high nitrogen fertilizer gradient; the field is provided with multiple planting plots, each planting plot has a specification of 1.5 meters*1.5 meters, adopts a planting mode of a row spacing of 25 centimeters, 100 plants per row, and 6 rows of planting rows, and wheat of a corresponding variety is planted in each planting plot. Through the planting mode of multiple varieties and multiple nitrogen fertilizer gradients, the diversity of the test data can be effectively improved, and the universality of the subsequently calculated phenotype parameter results can be enhanced.

[0008] Preferably, in the step S1, when the interpolation segmentation is performed, the maximum value and the minimum value of the original point cloud in the x direction are calculated to determine the corresponding interval in the x direction, and the maximum value and the minimum value of the original point cloud in the y direction are calculated to determine the corresponding interval in the y direction; then the original point cloud is divided into six nitrogen gradient point clouds by performing three equal-division interpolation segmentations along the corresponding intervals in the x direction and two equal-division interpolation segmentations along the corresponding intervals in the y direction. Through interpolation segmentation, accurate spatial division of the nitrogen gradient point cloud is realized.

[0009] Preferably, in the step S1, when the unmanned aerial vehicle shoots the multi-view images of the wheat in the field, the tilt photography parameters of the unmanned aerial vehicle are set as: an orthographic ground resolution of 0.32 centimeters, a tilt resolution of 0.33 centimeters, and a gimbal pitch angle of -80 degrees; five flight modes of a straight flight line, a forward-leaning flight line, a transverse-leaning flight line, a multi-angle cross flight line, and a reverse flight line are randomly adopted for flight shooting to obtain the multi-view images, and a three-dimensional reconstruction is performed by using a reconstruction mode of a measurement software. Through the multi-flight-line mode and the setting of the specific projection angle, the accuracy in the three-dimensional reconstruction is ensured.

[0010] Preferably, in the step S2, the parameters for model training include an initial learning rate, a batch size, and a training round number; and the AdamW optimizer is used to update the parameters during the model training. The setting of the parameters for model training and the adoption of the AdamW optimizer can effectively accelerate the training process and improve the generalization ability of the model.

[0011] Preferably, the candidate box generation system in step S2 comprises a set abstraction module, a cross-feature pyramid transformation 3D module, a feature propagation module, a point-by-point module and a bounding box prediction module; the set abstraction module comprises four sequentially arranged abstraction layers, and uses a matrix in the form of N x 3 as input for each cell point cloud in the test set based on the coordinates of three directions of corresponding x, y and z, and the number of points of each cell point cloud is represented by N, and four feature maps corresponding to each cell point cloud in the test set are obtained layer by layer, which are used to form four feature matrices (B x Ci x Hi x Wi, i ∈ [1, 4]) corresponding to each cell point cloud in the test set and serve as input data of the cross-feature pyramid transformation 3D module, B represents batch size, C corresponds to a value vector, H corresponds to a query vector, and W corresponds to a key vector; the four feature maps comprise three local feature maps and one final feature map; the cross-feature pyramid transformation 3D module comprises a cross-layer attention unit and a shared detection head unit, the cross-layer attention unit performs point multiplication and scaling on each query vector and each corresponding key vector to obtain each corresponding attention weight; each attention weight is weighted and summed with each corresponding value vector to obtain each preliminary feature; each preliminary feature is processed based on a convolution operation and an activation function to obtain each spatial-channel hybrid attention feature n SC ; the shared detection head unit uses a shared detection head to horizontally connect the four abstraction layers to obtain each output matrix of B x L x (4 + K) corresponding to each cell point cloud, L is the number of 3D candidate boxes corresponding to each cell point cloud, and 4 in each output matrix is position information, any position information comprises center coordinates of a corresponding 3D candidate box and distances from the center coordinates of the 3D candidate box to vertices of the 3D candidate box; the feature propagation module performs detail recovery on each output matrix of the shared detection head unit through four feature propagation layers to output each corresponding (N x D') matrix, N represents the number of points of the cell point cloud, and D' represents the feature dimension; the point-by-point module adopts a binary classification segmentation head, predicts a probability that each point belongs to a foreground based on each (N x D') matrix output by the feature propagation module, and obtains each foreground point set through threshold screening; the bounding box prediction module receives each foreground point set, predicts an offset of each foreground point relative to a corresponding target center and a size of the target, and generates each corresponding 3D candidate box through clustering to output a matrix of K x 7, K in K x 7 represents the number of 3D candidate boxes and 7 represents x coordinates, y coordinates, z coordinates, size length, width, height and direction angle of each 3D candidate box. Through cross-layer feature fusion, the accuracy of feature extraction can be enhanced, and the adverse effects of field interference objects and the like can be reduced.

[0012] Preferably, when the candidate frame optimization system obtains each 3D bounding box in step S2, the area attention module divides each of the four feature maps corresponding to each small plot point cloud in the test set into four segments respectively, and obtains sixteen groups of receptive fields respectively; the feature extraction module retransmits each receptive field into four abstract layers to obtain each corresponding feature information and perform corresponding detail recovery, and then synthesizes each final feature information; the optimization module receives each final feature information, and calculates each correction matrix of the final optimized frame corresponding to each final feature information using the regression head, and applies each correction matrix to the corresponding 3D candidate frame to obtain each 3D bounding box; wherein each correction matrix includes the offset of the center coordinate, the scaling of the size, and the adjustment of the direction angle. Through the compression of the area attention receptive field, efficient bounding box optimization can be achieved.

[0013] Preferably, based on each 3D bounding box in step S3, the feature difference corresponding to each 3D bounding box in the width axis direction is extracted, linear interpolation is performed at a fixed interval in each feature difference to form a plurality of corresponding interpolation intervals, and the point cloud data in each interpolation interval corresponding to each 3D bounding box is counted to construct a distribution curve; wherein the feature interpolation = maximum value - minimum value, and the fixed interval = 5 cm. By performing linear interpolation in the feature difference, the corresponding distribution curve can be effectively constructed, and then the data of the wheat in all aspects can be analyzed.

[0014] Preferably, after counting the point cloud data in each interpolation interval corresponding to each 3D bounding box, the distribution curve is constructed, and in the case that the distribution curve presents a six-peak and five-valley characteristic, each valley minimum point in the distribution curve is determined as the inter-row interval center position and is extended by 5 cm in the positive and negative directions of the width axis to form a plurality of interval row regions with a width of 10 cm, wherein each small plot point cloud includes five corresponding interval row regions; then, each three-dimensional minimum bounding box corresponding to each small plot point cloud is constructed, and each static phenotype parameter is quantified; wherein when each static phenotype parameter corresponding to any small plot point cloud is quantified, the average height of the six bounding boxes corresponding to the six planting rows is taken as the corresponding plot height, the cumulative length-width product of the six bounding boxes is taken as the canopy area, the length-width-height product of the six bounding boxes is taken as the leaf volume, and the total area of the six planting rows is divided by the area of the 3D bounding box corresponding to the small plot point cloud to obtain the leaf area. Using the bounding box for calculation can reduce the information loss of each small plot point cloud and accurately calculate each static phenotype parameter.

[0015] Preferably, before the five dynamic phenotype parameters of field wheat are calculated in step S4, a dynamic change broken line I of height of each plot, a dynamic change broken line II of each crown area, and a dynamic change broken line III of each leaf area are constructed, and the slope of each dynamic change broken line represents the corresponding height change rate or the corresponding area change rate; then, the points with the slope greater than 1 in the dynamic change broken lines I, II and III are taken as the starting points of the rapid change period of the traits, the points with the slope less than 1 are taken as the ending points of the rapid change period of the traits and the starting points of the slow change period of the traits, and the points with the slope less than 0 are taken as the ending points of the slow change period of the traits and the starting points of the reduction period of the traits; then, the maximum height difference and the maximum area difference are defined along the time axis of the dynamic change broken line I and the dynamic change broken line II, respectively. According to the dynamic change broken line constructed based on the plurality of static phenotype parameters, the growth of the wheat can be effectively and objectively quantified, so that the corresponding dynamic phenotype parameters of the wheat can be accurately located, and the generation of the wheat can be accurately represented.

[0016] Compared with the prior art, the present application has the following beneficial effects: The technical scheme of the present application firstly proposes a 3D wheat plot detection network specially designed for a plurality of planting plots of field wheat, combines a plurality of modes of an unmanned aerial vehicle and a specific angle of a photographed picture, accurately obtains three-dimensional features of field wheat by using a candidate box generation system and a candidate box optimization system, and has a performance significantly better than that of a mainstream three-dimensional detection network; further, based on the features of point cloud data, three-dimensional point cloud segmentation at a planting row level is realized, and the problem of dense planting row segmentation that cannot be processed by a two-dimensional image is successfully solved; in addition, combined with accurate analysis of static phenotype parameters, a new index system containing five time dynamic phenotypes is further proposed, and based on the dynamic phenotype parameters, the growth and yield of field wheat can be effectively predicted, thereby providing technical support for precision agriculture cultivation decision-making. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 It is a flowchart of a three-dimensional trait analysis method for field wheat based on multi-view three-dimensional point cloud of an unmanned aerial vehicle; Figure 2 It is a schematic diagram of unmanned aerial vehicle data acquisition; Figure 3 It is a schematic diagram of a point cloud data processing process; Figure 4 It is a schematic diagram of a network architecture of a 3D wheat plot detection network; Figure 5 It is a schematic diagram of row segmentation and trait analysis; Figure 6 It is a schematic diagram of a dynamic phenotype parameter. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described in detail below. Figures 1 to 6 The technical solutions in the embodiments of the present application will be described in detail below.

[0019] As Figure 1 As shown in the figure, it is a flowchart of a field wheat three-dimensional trait analysis method based on a multi-view three-dimensional point cloud of a UAV, which is divided into four stages. In the first stage, the DJI UAV 3M industry version tilt photography mode is used to collect three-dimensional point cloud data of wheat at each growth period. Meanwhile, a set of wheat plot preprocessing method is developed to obtain pure plot point cloud data without land and noise points, so as to improve the efficiency and accuracy of labeling. In the second stage, a 3D wheat plot detection network specially used for wheat three-dimensional plot detection is proposed. The model is optimized for special challenges in the complex field environment, especially solving the depth direction detection error problem caused by interference such as sensors. In the third stage, based on the 3D bounding box obtained by detection, an accurate row segmentation algorithm is developed to extract the point cloud data of each plot planting row, so as to realize accurate positioning and segmentation of each three-dimensional plot planting row. In the fourth stage, according to the static phenotype parameters, five dynamic time phenotypes are proposed, including trait rapid change period, trait slow change period, trait reduction period, maximum height difference or maximum area difference, height change rate or area change rate.

[0020] In the first stage, first, the field wheat is planted, data is collected, and preliminary preprocessing is performed to obtain the total sample for calculation and analysis.

[0021] The present application is a field test carried out in a university teaching and scientific research base at North Latitude 31°36'57.8"N, East Longitude 119°10'46.1"E. For each variety of wheat, three nitrogen fertilizer gradient treatments are set: zero nitrogen fertilizer N0, medium nitrogen fertilizer amount N180 (nitrogen fertilizer 180 kg / ha), and high nitrogen fertilizer amount N240 (nitrogen fertilizer 240 kg / ha). The three nitrogen fertilizer gradients use the same nitrogen fertilizer raw material, and two biological replicates are set, that is, two groups of the same treatment are set for each variety, and a total of 960 wheat test planting plots are planted.

[0022] Each planting plot is 1.5 meters x 1.5 meters, with a row spacing of 25 cm, and 100 plants are uniformly planted in each row, with 6 rows of planting rows. After sowing, irrigation is carried out according to the water amount of 20L per square meter of land, and the irrigation water is reduced by 5%-15% according to the actual natural precipitation.

[0023] As Figure 2The figure shows a schematic diagram of drone data collection. During data collection, a DJI 3M Industrial Edition drone was used to collect data during the wheat growing period. Oblique photography parameters were set, including an orthographic ground resolution of 0.32 cm, an oblique resolution of 0.33 cm, and a gimbal pitch angle of -80 degrees. By executing five flight modes (forward, forward, banked, multi-angle crossing, and reverse), the drone was reconstructed using the high-precision reconstruction mode of the surveying software for multi-view 3D reconstruction. This generated a raw point cloud, also known as a 3D point cloud, with three dimensions in the x, y, and z directions. To fully understand the dynamic phenotypic changes of wheat during its growing period, regular aerial surveys were conducted every six days for all wheat plots in the field from jointing to maturity. During the two critical growth stages, after jointing fertilizer application and heading, the survey frequency was increased to every three days. Furthermore, multi-view data of the base surface of the planting plots was collected and stored immediately after wheat sowing. The resulting base point cloud also provided an important reference for subsequent point cloud data preprocessing and static phenotypic parameter extraction.

[0024] The measurement software that can be used is Pix4D Enterprise, which is a professional photogrammetry software designed to convert images taken by drones or aerial photography into accurate two-dimensional maps and three-dimensional models. It is widely used in surveying and mapping, construction, agriculture, disaster response, land management and other fields.

[0025] like Figure 3 The figure shows a schematic diagram of the point cloud data processing process. Figure 1 and Figure 3 As shown, preprocessing can be performed on each planting plot to obtain pure plot point cloud data with plots and noise points removed.

[0026] Taking the preprocessing of any one original point cloud as an example, first, the maximum and minimum values in the x direction and the maximum and minimum values in the y direction are calculated, then interpolation segmentation is performed along the x direction for three equal parts and along the y direction for two equal parts, that is, 3x2, so as to divide the original point cloud into six nitrogen gradient point clouds, each gradient point cloud containing the overall three-dimensional data of the planting plot of 160 varieties under the corresponding nitrogen level. According to the distribution characteristics of the varieties, corresponding interpolation segmentation methods can be set respectively, for example, setting the interpolation segmentation of the areas where varieties 1-128 are located to be 16x4, and the interpolation segmentation of the areas where varieties 129-160 are located to be 8x4. Therefore, in the same nitrogen gradient point cloud, the areas where varieties 1-64 and 65-128 are located are interpolated and segmented in quantities of 16 and 4, and the areas where varieties 129-160 are located are interpolated and segmented in quantities of 8 and 4, finally each nitrogen gradient point cloud is accurately segmented into 160 independent plot point clouds corresponding to the planting plots of 160 varieties. Through this process repeatedly, since there are three nitrogen fertilizer gradients and two biological repeats, 160x3x2=960, that is, a total of 960 plot point clouds are obtained.

[0027] For the 960 plot point clouds, the ground point clouds need to be removed to avoid interference with the center coordinates of the bounding box. A plane fitting method can be used to remove the point clouds in the ground part, and the point clouds where the wheat canopy is located may also be misclassified as a plane, therefore, a straight-through filtering algorithm along the z direction is used, first, the lower half of the point cloud is extracted for plane fitting to identify the ground point cloud; then, the ground information under the canopy is obtained by difference operation between the reference surface point cloud and the detected ground, and then it is fused with the processed plot point cloud, after radius filtering and denoising, finally each plot point cloud containing the complete land reference surface is obtained, which is used as the total sample for subsequent calculation and processing.

[0028] In the second stage, target detection needs to be performed on the three-dimensional plot where the wheat is located, a 3D wheat plot detection network is used for training to obtain each 3D bounding box. To improve the generalization ability of the model, comprehensive data enhancement processing is performed on the original data set. Specifically, it includes multiple transformation methods such as random rotation in three-dimensional space, random translation along the coordinate axis direction, random scaling by different proportions, adding Gaussian random noise, and randomly discarding point clouds according to probability. After these enhancement treatments, the sample size is expanded from the original 960 to 9600, and finally the data set is divided into a training set (6720 samples), a test set (1920 samples), and a validation set (960 samples) according to the standard ratio of 7:2:1. During model training, the initial learning rate is set to 1x10⁻³, the batch size is 32, the number of training rounds is 100, the accuracy is verified after each round of training, and the AdamW optimizer is used to update the parameters.

[0029] AsFigure 4 As shown, it is a network architecture diagram of a 3D wheat plot detection network, and the 3D wheat plot detection network, namely a 3D Wheat Plot Detection Net model, comprises a 3D candidate box generation system and a candidate box optimization system.

[0030] The 3D candidate box generation system comprises five modules, namely a set abstraction module, a cross-feature pyramid transformation 3D module, a feature propagation module, a point-by-point module, and a bounding box prediction module. The set abstraction module comprises four abstract layers arranged in sequence. N represents the number of plot point clouds based on the coordinates of three directions of x, y, and z. Each plot point cloud in the test set is input in the form of a matrix of N×3, and four feature maps corresponding to each plot point cloud in the test set are obtained layer by layer. The four abstract layers can gradually extract local features of the point cloud, and the final feature is extracted by maximum pooling operation in the last layer, which is used to compose four feature matrices (B×Ci×Hi×Wi, i∈[1, 4]) corresponding to each plot point cloud in the test set and as input data of the cross-feature pyramid transformation 3D module. C corresponds to a value vector, H corresponds to a query vector, and W corresponds to a key vector. The four feature maps comprise three local feature maps and one final feature map. B is the abbreviation of batch size, which represents the number of sample plots of the plot point cloud sent into the 3D wheat plot detection network for processing at a time.

[0031] The cross-feature pyramid transformation 3D module comprises a cross-layer attention unit and a shared detection head unit. The cross-layer attention unit performs point multiplication and scaling on each query vector and each corresponding key vector to obtain each corresponding attention weight. Scaling means dividing by , , which represents the square root of the dimension of the key vector or the query vector. Then, each attention weight is weighted and summed with each corresponding value vector to obtain each preliminary feature; each preliminary feature is processed based on a convolution operation and a set activation function to obtain each spatial-channel hybrid attention feature n SC ; the shared detection head unit horizontally connects the four abstract layers by using a shared detection head to obtain each B×L×(4+K) output matrix corresponding to each plot point cloud. L is the number of 3D candidate boxes corresponding to each plot point cloud. The 4 in each output matrix is position information, and any position information comprises the center coordinates (i.e. coordinates made of specific numerical values of x, y, and z) of the corresponding 3D candidate box and the distance from the center coordinates of the 3D candidate box to each vertex of the 3D candidate box.

[0032] Then, the feature propagation module performs detail recovery on the output matrix of the shared detection head unit through four feature propagation layers, outputting the corresponding (N×D') matrix, where N represents the number of points in the point cloud and D' represents the feature dimension. Subsequently, the point-by-point module uses a binary classification segmentation head to predict the probability of each corresponding point belonging to the foreground based on each (N×D') matrix output by the feature propagation module, and then obtains each corresponding foreground point set through threshold screening. Finally, the bounding box prediction module receives each foreground point set, predicts the offset of each foreground point relative to the center of the target and the size of the target, and then generates each 3D candidate box through clustering, outputting a K×7 matrix, where K in K×7 represents the number of 3D candidate boxes and 7 represents the x-coordinate, y-coordinate, z-coordinate, size length, width, height and direction angle of each 3D candidate box.

[0033] The candidate box optimization system consists of a region attention module, a feature extraction module, and an optimization module. After the region attention module obtains each 3D candidate box, it divides the four feature maps corresponding to each cell point cloud in the test set into four segments, reducing the receptive field to 1 / 4 of its original size. Sixteen corresponding receptive fields are obtained for each segment. The receptive field refers to the spatial range that a point on any feature map can affect in the original point cloud. This effectively extracts features from localized, relevant regions, significantly reducing the computational overhead of the subsequent candidate box feature extraction module and improving model training performance and speed. The feature extraction module then retransmits each receptive field through four abstraction layers to retrieve the corresponding feature information and restore the corresponding details. The sixteen feature sets are finally merged into the final feature information. Finally, the optimization module receives each final feature information and uses a regression head to calculate a correction matrix from each 3D candidate box to the final optimized box corresponding to each final feature information. Each correction matrix is ​​applied to each 3D candidate box to correct it, resulting in a 3D bounding box. Each correction matrix includes a center coordinate offset, size scaling, and orientation adjustment.

[0034] In the third phase, an accurate row segmentation algorithm was developed to extract point cloud data for each planting row. Based on the segmented plot and planting row point cloud data, the system calculated plot height, canopy area, leaf volume, and leaf area, a total of four key static phenotypic parameters.

[0035] like Figure 5 The following is a schematic diagram of row analysis and trait analysis, combined with Figure 5As shown, the algorithm first extracts all the point clouds within the bounding box, analyzes the spatial distribution along the planting row direction (the wide axis), calculates the difference between the maximum and minimum values in the wide axis direction, performs linear interpolation at intervals of 5 cm, counts the number of point clouds in each interpolation interval, and constructs a distance-point number distribution curve. The distribution curve presents obvious six-peak and five-valley alternating characteristics, six peaks refer to the existence of six local high points in the curve, and five valleys refer to the existence of five local low points in the curve, which completely coincides with the six-row and five-interval planting mode designed in the experiment. When analyzing, it can also be judged whether the previous operation is wrong according to whether it has the six-peak and five-valley alternating characteristics, so as to decide whether to recalculate.

[0036] Combined with the characteristic rule of the six-peak and five-valley alternating characteristics, the lowest point of the distribution curve is determined as the center position of the inter-row interval, and each extends 5 cm in the positive and negative directions of the wide axis to form a 10 cm wide interval row area (five interval rows for each plot). By traversing all the bounding boxes and applying the segmentation method, the precise positioning and segmentation of the planting rows of each three-dimensional plot are finally realized. During the calculation process, after constructing a three-dimensional minimum bounding box for each of the six planting rows of the plot, the following methods are used to quantify the indicators: the average height of the six bounding boxes is used as the height estimate of the plot; the cumulative length-width product of the six bounding boxes is used as the crown area; the length-width-height product of the six bounding boxes is used to obtain the plot volume; the leaf area of the plot is obtained by calculating the area of each planting row (i.e. the product of the length and width of each planting row), and then adding up the areas of all six planting rows and dividing the total area by the area of the 3D bounding box of the plot.

[0037] In the fourth stage, according to the static phenotypic parameters, five dynamic time phenotypes including the rapid change period of traits, the slow change period of traits, the reduction period of traits, the maximum height / area difference, and the height area change rate are proposed.

[0038] As Figure 6 shown, is a schematic diagram of a dynamic phenotypic parameter, which shows the curves of static phenotypic parameters under three different nitrogen fertilizer gradients. From the curve, the corresponding each dynamic phenotypic parameter can be obtained. Combined Figure 6As shown, to obtain multiple dynamic phenotype parameters, it is necessary to first construct a dynamic change broken line I for each plot height, a dynamic change broken line II for each crown area, and a dynamic change broken line III for each leaf area based on the aforementioned multiple static phenotype parameters, and the slope of each dynamic change broken line represents the corresponding height change rate or the corresponding area change rate. Then, the points with a slope greater than 1 in the dynamic change broken lines I, II and III are taken as the starting points of the rapid change of the traits, the points with a slope less than 1 are taken as the end points of the rapid change period of the traits and the starting points of the slow change period of the traits, and the points with a slope less than 0 are taken as the end points of the slow change period of the traits and the starting points of the reduction period of the traits; then, the maximum difference in the vertical direction of the ordinate perpendicular to the time axis is defined as the maximum height difference and the maximum area difference, respectively, along the positive direction of the time axis of the dynamic change broken line I and the dynamic change broken line II.

[0039] In addition, for the use of the 3D wheat plot detection network, an application program can also be designed, which includes a functional part and a display part. The functional part has a total of six modules, namely, a point cloud reading module, a plot detection module, a row segmentation module, a trait analysis module, a chart generation module and a data export module, which are six functional modules for supporting the operation of the aforementioned field wheat three-dimensional trait analysis method; the display interface includes a point cloud visualization interface and an analysis chart display area, which are used to display the results of the analysis.

[0040] The point cloud reading module supports single or batch file reading in multiple formats (.pcd,.ply, etc.,.pcd is a point cloud data format, and.ply is a polygon file format), and realizes multi-angle interactive display in the visualization interface. Users can observe the details of the point cloud through translation, rotation and scaling operations, and intuitively evaluate the detection effect. The plot detection module integrates the pre-trained model of the 3D wheat layout detection network, and can automatically calculate and visualize the three-dimensional bounding box of all plots. The row segmentation module allows users to customize the number of planting rows, and the algorithm automatically performs accurate segmentation according to the set parameters. The trait analysis module provides four key static phenotype parameter calculations: plot height, plot crown area, plot volume and plot leaf area. The system marks the current analysis plot in red in real time to ensure operation visualization. The data analysis module includes two special functions: through the radar chart and the heat map, the trait comparison analysis between varieties is realized, and the difference in the growth of different varieties at the same time is intuitively displayed; through the time series growth change broken line chart, the dynamic phenotype parameter changes in the key growth period are revealed, providing decision support for field management. The data export module can save the phenotype data of all plots and their corresponding plot point cloud files to a specified folder by clicking the save button.

[0041] In summary, the present application proposes a 3D wheat plot detection network specially designed for multiple planting plots of field wheat, which combines multiple flight modes of unmanned aerial vehicles and photos taken at specific angles, accurately obtains three-dimensional features of field wheat by using a candidate box generation system and a candidate box optimization system, and has significantly better performance than existing mainstream three-dimensional detection networks; based on the features of point cloud data, three-dimensional point cloud segmentation at the planting row level is realized, and the problem of dense planting row segmentation that two-dimensional images cannot handle is successfully solved; in addition, combined with accurate analysis of static phenotypic parameters, a new index system containing five time dynamic phenotypes is further proposed, and based on these dynamic phenotypic parameters, the growth and yield of field wheat can be effectively predicted, providing technical support for precision agriculture cultivation decision-making, and having significant progress.

[0042] The above examples only illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical scheme falls within the protection scope of the present application.

Claims

1. A method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point clouds of drones, characterized in that: The steps include: S1. Use a drone to capture multi-view images of wheat in the field and perform 3D reconstruction to obtain an original point cloud. The 3D images include x, y, and z directions. Interpolate and segment the original point cloud to obtain multiple nitrogen gradient point clouds. Then, perform secondary segmentation on each nitrogen gradient point cloud to output multiple small-area point clouds as the total sample. S2. Perform data augmentation on the total samples in step S1, divide the data augmentation results into a training set, a test set, and a validation set, and use the training set to start model training of the 3D wheat cell detection network; wherein, the 3D wheat cell detection network includes a candidate box generation system and a candidate box optimization system. During model training, various parameters of the model training are first set, and then the candidate box generation system is used to output each 3D candidate box. Then, the candidate box optimization system is used to obtain each optimized 3D bounding box based on the regional attention module, the feature extraction module, and the optimization module; S3. Based on each 3D bounding box, using a row segmentation algorithm, extract the point cloud data of each planting row of field wheat from each 3D bounding box and calculate each static phenotypic parameter; wherein the static phenotypic parameters include plot height, canopy area, leaf volume, and leaf area; S4. Based on each static phenotypic parameter in step S3, five dynamic phenotypic parameters of field wheat are calculated, including the period of rapid phenotypic change, the period of slow phenotypic change, the period of phenotypic reduction, the maximum height difference or the maximum area difference, and the height change rate or the area change rate.

2. The method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point cloud of drone according to claim 1, characterized in that: There are multiple varieties of wheat in the field, and each variety is set with three nitrogen fertilizer gradients, including zero nitrogen fertilizer gradient, medium nitrogen fertilizer gradient and high nitrogen fertilizer gradient; there are multiple planting plots in the field, each planting plot is 1.5 meters × 1.5 meters in size, with a row spacing of 25 cm, 100 plants evenly planted in each row, and 6 planting rows. Each variety of wheat is planted in each planting plot.

3. The method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point cloud of drone according to claim 2, characterized in that: When performing interpolation segmentation in step S1, first calculate the maximum and minimum values ​​of the original point cloud in the x direction to determine the corresponding interval in the x direction, and at the same time calculate the maximum and minimum values ​​of the original point cloud in the y direction to determine the corresponding interval in the y direction; Then, interpolation segmentation is performed along the corresponding interval in the x direction into three equal parts, and interpolation segmentation is performed along the corresponding interval in the y direction into two equal parts to obtain six nitrogen gradient point clouds.

4. The method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point cloud of drone according to claim 1, characterized in that: In step S1, when the drone takes multi-perspective images of wheat in the field, the oblique photography parameters of the drone are set as follows: orthographic ground resolution 0.32 cm, oblique resolution 0.33 cm, and gimbal pitch angle -80 degrees; during shooting, five flight modes are randomly adopted, namely, forward route, forward route, tilt route, multi-angle cross route, and reverse route, to obtain multi-perspective images, and then three-dimensional reconstruction is performed using the reconstruction mode of the measurement software.

5. The method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point cloud of drone according to claim 1, characterized in that: In step S2, various parameters for model training are set, including: initial learning rate, batch size, and number of training rounds; at the same time, during model training, the AdamW optimizer is used to update various parameters.

6. The method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point clouds of drones according to claim 1, characterized in that: The candidate box generation system in step S2 includes a set abstraction module, a cross-feature pyramid transformation 3D module, a feature propagation module, a point-by-point module, and a bounding box prediction module; The set abstraction module includes four sequentially arranged abstraction layers, with N representing the number of points in each cell point cloud and based on the corresponding x, y, and z coordinates, each cell point cloud in the test set is input in the form of an N×3 matrix. The four feature maps corresponding to each cell point cloud in the test set are obtained layer by layer to form the four feature matrices (B×Ci×Hi×Wi, i∈[1, 4]) corresponding to each cell point cloud in the test set and serve as the input data of the cross-feature pyramid transformation 3D module. B represents the batch size, C corresponds to the value vector, H corresponds to the query vector, and W corresponds to the key vector. The four feature maps include three local feature maps and one final feature map. The cross-feature pyramid transformation 3D module includes a cross-layer attention unit and a shared detection head unit. The cross-layer attention unit performs dot product and scaling on each query vector and each corresponding key vector to obtain the corresponding attention weight. The weighted sum of each attention weight and each corresponding value vector is used as each preliminary feature; each preliminary feature is then processed based on the convolution operation and activation function to obtain each spatial-channel mixed attention feature n SC The shared detection head unit uses the shared detection head to connect the four abstraction layers horizontally to obtain each B×L×(4+K) output matrix corresponding to each cell point cloud. L is the number of 3D candidate boxes corresponding to each cell point cloud. The 4 in each output matrix is ​​the position information. Any position information includes the center coordinates of the corresponding 3D candidate box and the distance from the center coordinates of the 3D candidate box to each vertex of the 3D candidate box. The feature propagation module recovers details of each output matrix of the shared detection head unit through four feature propagation layers and outputs each corresponding (N×D') matrix, where N represents the number of points in the cell point cloud and D' represents the feature dimension. The point-by-point module uses a binary classification segmentation head. Based on each (N×D') matrix output by the feature propagation module, it predicts the probability of each corresponding point belonging to the foreground. Then, it uses threshold filtering to obtain each foreground point set. The bounding box prediction module receives each set of foreground points, predicts the offset of each foreground point relative to the corresponding target center and the size of the target, and then generates each corresponding 3D candidate box through clustering, outputting a K×7 matrix, where K represents the number of 3D candidate boxes and 7 represents the x-coordinate, y-coordinate, z-coordinate, length, width, height and direction angle of each 3D candidate box.

7. The method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point cloud of drone according to claim 6, characterized in that: In step S2, when the candidate box optimization system obtains each optimized 3D bounding box, the four feature maps corresponding to each cell point cloud in the test set are divided into four segments through the regional attention module, and the corresponding sixteen groups of receptive fields are obtained respectively; The feature extraction module is used to re-transmit each receptive field into the four abstraction layers to re-acquire the corresponding feature information and perform corresponding detail recovery, and then synthesize the corresponding final feature information; The optimization module then receives each final feature information. The optimization module uses the regression head to calculate each correction matrix from each 3D candidate box to the final optimized box corresponding to each final feature information, and applies each correction matrix to each corresponding 3D candidate box for correction to obtain each 3D bounding box; wherein each correction matrix includes the offset of the center coordinates, the scaling of the size, and the adjustment of the direction angle.

8. The method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point clouds of drones according to claim 3, characterized in that: In step S3, based on each 3D bounding box, each feature difference corresponding to each 3D bounding box in the width direction is first extracted. Linear interpolation is then performed at fixed intervals in each feature difference to form multiple corresponding interpolation intervals. The point cloud data within each interpolation interval corresponding to each 3D bounding box is then counted to construct a distribution curve; where feature interpolation = maximum value - minimum value, and fixed interval = 5 cm.

9. The method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point clouds of drones according to claim 8, characterized in that: After counting the point cloud data within each interpolation interval corresponding to each 3D bounding box, a distribution curve was constructed. When the distribution curve showed the characteristics of six peaks and five valleys, the lowest point of each valley in the distribution curve was determined as the center position of the inter-row interval and extended 5 cm in both the positive and negative directions along the width axis to form multiple 10 cm wide inter-row regions, where each small cell point cloud contained the corresponding five inter-row regions. Then, each corresponding three-dimensional minimum bounding box was constructed for each small cell point cloud and each static phenotypic parameter was quantified. Among them, when quantifying each static phenotypic parameter corresponding to any plot point cloud, the average height of the six bounding boxes corresponding to the six planting rows is used as the corresponding plot height, the cumulative length and width product of the six bounding boxes is used as the canopy area, the length, width and height product of the six bounding boxes is summarized to obtain the leaf volume, and the total area of ​​the six planting rows is divided by the area of ​​the 3D bounding box corresponding to the plot point cloud as the leaf area.

10. The method for analyzing three-dimensional traits of field wheat based on multi-view three-dimensional point cloud of drone according to claim 1, characterized in that: Before calculating the five dynamic phenotypic parameters of field wheat in step S4, a dynamic change line I of the height of each plot, a dynamic change line II of the canopy area, and a dynamic change line III of the leaf area are first constructed. The slope of each dynamic change line represents the corresponding height change rate or the corresponding area change rate. Then, the points with slopes greater than 1 in the dynamic change lines I, II and III are taken as the starting points of the rapid change of traits, the points with slopes less than 1 are taken as the end points of the rapid change period of traits and the starting points of the slow change period of traits, and the points with slopes less than 0 are taken as the end points of the slow change period of traits and the starting points of the trait reduction period; then, the time axes of the dynamic change lines I and II are traversed respectively, and the maximum difference in the vertical coordinate direction perpendicular to the time axis is defined as the maximum height difference and the maximum area difference respectively.

Images