Method and device for determining morphological structure of rapeseed population

Abstract

The present application discloses a method and device for determining the morphological structure of a rapeseed population, relating to the field of data processing; the method comprises: obtaining residual point cloud data of a target rapeseed population; inputting the residual point cloud data into a rapeseed population point cloud completion model, and outputting complete point cloud data of the rapeseed population; the complete point cloud data of the rapeseed population is used to determine the three-dimensional phenotypic information of the rapeseed population and determine the morphological structure of the rapeseed population; the rapeseed population point cloud completion model is obtained by iteratively updating and training a constructed rapeseed population point cloud completion network using adaptive moment estimation and back propagation algorithm; the rapeseed population point cloud completion network is constructed based on the idea of ​​adversarial generative network; the rapeseed population point cloud completion network comprises a generator and a discriminator connected in sequence; the generator comprises: a multi-resolution encoder and a pyramid point cloud decoder; the present invention can achieve the acquisition of the complete morphological structure of the rapeseed population.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method and device for determining the morphological structure of a rapeseed population. Background Art

[0002] Three-dimensional morphological structure is a crucial component of plant phenomics research, directly related to plant growth and development, physiological function, genetic expression, and adaptability to environmental changes. Plant 3D information not only helps scientists better understand the relationship between genotype and phenotype, improving breeding efficiency, but also guides agricultural practices such as irrigation, fertilization, and pest and disease management in precision agriculture. As my country's most important oilseed crop, rapeseed urgently requires accurate and efficient methods for acquiring population 3D point clouds.

[0003] Single-plant-scale point cloud data can be collected using imaging devices based on principles such as structured light or stereo vision. However, their high lighting requirements and small acquisition range preclude their use in large field environments. With the development of LiDAR scanning equipment, it has become a mainstream technology for 3D point cloud data collection, providing high-precision, high-resolution data suitable for modeling from single plants to populations. However, due to current technological limitations, higher-precision equipment is more expensive, making it unsuitable for agricultural research. Therefore, the development of high-precision 3D reconstruction algorithms based on multi-stereo vision is expected to address these issues.

[0004] After acquiring the point cloud data, complex post-processing is still required to truly realize its value. However, compared to two-dimensional images, the disorder of the point cloud itself makes its processing much more difficult, especially for the application of deep learning technology. Although deep learning has achieved good results in many tasks, high-quality three-dimensional point cloud datasets are often difficult to obtain, especially those with detailed annotations, which makes it difficult for the currently constructed models to fully learn and generalize. In the field of plants, the above problems will be more prominent. It is almost impossible for existing equipment to collect complete point cloud data at the population scale, which greatly limits the ability to extract specific phenotypic information. Today, with computer digital simulation technology constantly approaching the real world, it has become possible to create large quantities of point cloud data.

[0005] Digital simulation technology plays an important role in simulating the interaction between plant growth and the environment, providing a method for research to quickly verify hypotheses without the need for field experiments. However, although digital simulation technology can help solve the problem of insufficient deep learning point cloud datasets to a certain extent, it also has some significant limitations when simulating real-world conditions. For example, digital simulations usually require simplifications and assumptions about complex biological processes, which may not fully capture the complexity of actual conditions, resulting in deviations between simulation results and the real world. Therefore, existing research cannot rely entirely on simulation methods to construct point cloud datasets, especially for plant populations. An ideal method is to simulate population-scale scenarios based on real single-plant-scale data. Summary of the Invention

[0006] The purpose of this application is to provide a method and device for determining the morphological structure of a rapeseed population, which can achieve the acquisition of the complete morphological structure of a rapeseed population.

[0007] To achieve the above objectives, this application provides the following solutions:

[0008] In a first aspect, the present application provides a method for determining the morphological structure of a rapeseed population, the method comprising:

[0009] Obtain residual cloud data of the target rapeseed population;

[0010] The incomplete point cloud data is input into a rapeseed population point cloud completion model, and complete point cloud data of the rapeseed population is output; the complete point cloud data of the rapeseed population is three-dimensional point cloud data; the complete point cloud data of the rapeseed population is used to determine the three-dimensional phenotypic information of the rapeseed population and determine the morphological structure of the rapeseed population; the rapeseed population point cloud completion model is obtained by iteratively updating and training the constructed rapeseed population point cloud completion network using adaptive moment estimation and back propagation algorithm; the rapeseed population point cloud completion network is constructed based on the idea of ​​adversarial generative network; the rapeseed population point cloud completion network includes a generator and a discriminator connected in sequence; the generator includes: a multi-resolution encoder and a pyramid point cloud decoder.

[0011] Optionally, the method for determining the rapeseed population point cloud completion model includes:

[0012] Acquire collected data; the collected data includes: complete point cloud data of a training single rapeseed plant and residual point cloud data of a training rapeseed group;

[0013] Using normal distribution transformation and iterative closest point algorithm, the training data is registered to obtain a point cloud completion data set, and the point cloud completion data set is determined as training data;

[0014] Constructing a rapeseed population point cloud completion network;

[0015] Inputting residual point cloud data of a training rapeseed population into the rapeseed population point cloud completion network, with the goal of meeting a set number of training rounds or minimizing a loss function, adopting adaptive moment estimation and back propagation algorithms to iteratively update and train the parameters of the rapeseed population point cloud completion network to obtain a trained rapeseed population point cloud completion network; the loss function is determined based on the training data and data output by the rapeseed population point cloud completion network; the loss function includes a multi-resolution completion loss function and an adversarial loss function; and the parameters include weights;

[0016] The trained rapeseed population point cloud completion network is determined as the rapeseed population point cloud completion model.

[0017] Optionally, the acquiring of collected data specifically includes:

[0018] Based on the set number of circles and the set angle, video data of a single rapeseed plant for training is obtained;

[0019] Determining sequence image data; the sequence image data is a series of static images generated by extracting video frames from the video data;

[0020] Using a three-dimensional point cloud acquisition method, training acquisition information is determined based on the sequence image data; the training acquisition information includes: a three-dimensional Gaussian ellipsoid group;

[0021] The ellipsoid center coordinates are extracted and processed based on the training information to obtain the complete point cloud data of the training single rapeseed plant;

[0022] Based on the UE software, according to the set imaging path and the set imaging parameters, a rapeseed population simulation sequence image is determined according to the training acquired information;

[0023] uniformly extracting a set number of static images from the rapeseed population simulation sequence image to obtain extracted sequence image data;

[0024] Using a three-dimensional point cloud acquisition method, the simulated training acquisition information is determined based on the extracted sequence image data;

[0025] The ellipsoid center coordinates are extracted and processed based on the simulated training information to obtain the residual cloud data of the training rapeseed population.

[0026] Optionally, a three-dimensional point cloud acquisition method is used to determine the training acquisition information based on the sequence image data, specifically including:

[0027] Using motion structure recovery and multi-view stereo vision algorithms, determining processing data based on the sequence image data; the processing data includes: sparse point cloud data and camera pose parameters;

[0028] A three-dimensional Gaussian scattering algorithm is used to determine an ellipsoid group in three-dimensional space based on the processed data; the ellipsoid group includes a plurality of ellipsoids; each ellipsoid is a three-dimensional Gaussian distribution of attribute parameters; the attribute parameters include: coordinates of the ellipsoid center, opacity, a three-dimensional covariance matrix, and spherical harmonics;

[0029] Projecting each ellipsoid in the ellipsoid group onto a projection plane corresponding to the posture parameter according to a preset angle, and rendering to obtain a rendered image;

[0030] Calculating the pixel color on the rendered image by using transparency synthesis; the pixel color is a pixel color value; the pixel color includes: the color values ​​of the three primary colors R, G, and B;

[0031] Using a back propagation algorithm, the attribute parameters are optimized with the goal of minimizing a pixel color loss function to obtain optimized attribute parameters; the pixel color loss function is determined based on the error between the pixel color after synthetic calculation and the actual pixel color;

[0032] The training acquisition information is determined according to the optimized attribute parameters and the ellipsoid group.

[0033] Optionally, the loss function is expressed as:

[0034] L=λ com L com +λ adv L adv ;

[0035] Among them, L is the loss function; λ com and λ adv are all weight hyperparameters; L com is the multi-resolution completion loss function; L adv is the adversarial loss function.

[0036] Optionally, the expression of the multi-resolution completion loss function is:

[0037] L com =d CD1 (Y Pre ,Y GT )+αd CD2 (Y′ Pre ,Y′ GT +2αd CD3 (Y″ Pre ,Y″ GT );

[0038] Among them, L com is the multi-resolution completion loss function; d CD1Y is the value of the similarity evaluation index CD between the fine completion point cloud data generated by the three fully connected layers in the generator and the actual point cloud; Pre Y is the point cloud data for generating detailed completion by three fully connected layers in the generator; GT is the actual fine point cloud data output by the expected model; α is the weighted hyperparameter; d CD2 Y' is the value of the similarity evaluation index CD between the medium-density completed point cloud data generated by the two fully connected layers in the generator and the actual medium-density point cloud data; Pre Y' is the medium-density completed point cloud data generated by two fully connected layers in the generator; GT is the medium-density point cloud data obtained by downsampling the actual fine point cloud output by the expected model; d CD3 Y” is the value of the similarity evaluation index CD between the rough point cloud data corresponding to the completed point cloud data generated by a fully connected layer in the generator and the actual rough point cloud data; Pre Y″ is the rough result corresponding to the completed point cloud data generated by a fully connected layer in the generator; GT It is the coarse point cloud data output by the expected model, which is obtained by downsampling the actual fine point cloud data.

[0039] Optionally, the adversarial loss function is expressed as:

[0040]

[0041] Among them, L adv is the adversarial loss function; S is the number of samples of residual cloud data for training the rapeseed population; i is the sample number; D() is the function corresponding to the discriminator; F() is the function corresponding to the generator; y i is the actual point cloud data; x i is the point cloud data predicted by the model.

[0042] In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned method for determining the morphological structure of a rapeseed population.

[0043] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0044] This application provides a method and device for determining the morphological structure of a rapeseed plant. This method uses neural radiation field technology to reconstruct complete point cloud data of a single rapeseed plant from multi-view images. This method then simulates actual rapeseed planting scenes and generates images from different viewpoints. The same method is used to reconstruct an incomplete group point cloud from the simulated images. After alignment with the complete point cloud, a large amount of training data required for a point cloud completion model is obtained. Ultimately, the trained model is used to complete the incomplete point cloud, achieving the complete morphological structure of the rapeseed plant. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 This is a flow chart of a method for determining the morphological structure of a rapeseed population in one embodiment of the present application;

[0047] Figure 2 A schematic diagram of the structure of a loading platform provided in one embodiment of the present application;

[0048] Figure 3 Schematic diagram of the shooting path of the simulated image;

[0049] Figure 4 This is a schematic diagram of the structure of the rapeseed population point cloud completion network;

[0050] Figure 5 A flow chart for the design of a method for determining the morphological structure of rapeseed populations in practical applications;

[0051] Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0053] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0054] In an exemplary embodiment, Figure 1 As shown, a method for determining the morphological structure of rapeseed population, the method comprising:

[0055] Step 100: Obtain residual defect cloud data of the target rapeseed population.

[0056] Step 200: Input the incomplete point cloud data into the rapeseed population point cloud completion model, outputting complete rapeseed population point cloud data. The complete rapeseed population point cloud data is 3D point cloud data; it is used to determine the 3D phenotypic information and morphological structure of the rapeseed population. The rapeseed population point cloud completion model is constructed by iteratively updating and training a rapeseed population point cloud completion network using adaptive moment estimation and backpropagation algorithms. The rapeseed population point cloud completion network is constructed based on the concept of generative adversarial networks and comprises a generator and a discriminator connected in sequence. The generator includes a multi-resolution encoder and a pyramid point cloud decoder.

[0057] As an optional implementation method, a method for determining a rapeseed population point cloud completion model includes:

[0058] Acquire collected data; the collected data includes: complete point cloud data of a training single rapeseed plant and residual point cloud data of a training rapeseed group.

[0059] The normal distribution transformation and iterative closest point algorithm are used to align the training data to obtain a point cloud completion dataset, and the point cloud completion dataset is determined as the training data.

[0060] Constructing a rapeseed population point cloud completion network.

[0061] The residual point cloud data of the training rapeseed population is input into the rapeseed population point cloud completion network. With the goal of meeting the set training rounds or minimizing the loss function, the adaptive moment estimation and back propagation algorithm are used to iteratively update the parameters of the rapeseed population point cloud completion network to obtain a trained rapeseed population point cloud completion network; the loss function is determined based on the training data and the data output by the rapeseed population point cloud completion network; the loss function includes: multi-resolution completion loss function and adversarial loss function; the parameters include: weights.

[0062] The trained rapeseed group point cloud completion network is determined as the rapeseed group point cloud completion model.

[0063] The expression of the loss function is:

[0064] L=λ com L com +λ adv L adv ;

[0065] Among them, L is the loss function; λ com and λadv are all weight hyperparameters; L com is the multi-resolution completion loss function; L adv is the adversarial loss function.

[0066] The expression of the multi-resolution completion loss function is:

[0067] L com =d CD1 (Y Pre ,Y GT )+α d CD2 (Y′ Pre ,Y′ GT )+2αd CD3 (Y″x re ,Y″ GT );

[0068] Among them, L com is the multi-resolution completion loss function; d CD1 is the value of the similarity evaluation index (Chamfer Distance, CD) between the fine completion point cloud data generated by the three fully connected layers in the generator and the actual point cloud; YPre is the completion point cloud data with details generated by the three fully connected layers in the generator; Y GT is the actual fine point cloud data output by the expected model; α is the weighted hyperparameter; d CD2 Y' is the value of the similarity evaluation index CD between the medium-density completed point cloud data generated by the two fully connected layers in the generator and the actual medium-density point cloud data; Pre Y' is the medium-density completed point cloud data generated by two fully connected layers in the generator; GT is the medium-density point cloud data obtained by downsampling the actual fine point cloud output by the expected model; d CD3 Y″ is the value of the similarity evaluation index CD between the rough point cloud data corresponding to the completed point cloud data generated by a fully connected layer in the generator and the actual rough point cloud data; Pre Y″ is the rough result corresponding to the completed point cloud data generated by a fully connected layer in the generator; GT It is the coarse point cloud data output by the expected model, which is obtained by downsampling the actual fine point cloud data.

[0069] The expression of the adversarial loss function is:

[0070]

[0071] Among them, L adv is the adversarial loss function; S is the number of samples of residual cloud data for training the rapeseed population; i is the sample number; D() is the function corresponding to the discriminator; F() is the function corresponding to the generator; yi is the actual point cloud data; x i is the point cloud data predicted by the model.

[0072] The acquisition of collected data specifically includes:

[0073] Based on the set number of circles and the set angle, video data of a single rapeseed plant for training is obtained.

[0074] Determine the sequence image data; the sequence image data is a series of static images generated by extracting video frames from video data.

[0075] A three-dimensional point cloud acquisition method is adopted to determine training acquisition information according to sequence image data; the training acquisition information includes: a three-dimensional Gaussian ellipsoid group.

[0076] The ellipsoid center coordinates are extracted and processed based on the information obtained during training to obtain the complete point cloud data of the training single rapeseed plant.

[0077] Based on the UE software, the rapeseed population simulation sequence images were determined according to the set imaging path and imaging parameters and the training information.

[0078] A set number of static images are uniformly extracted from a rapeseed population simulation sequence image to obtain extracted sequence image data.

[0079] A three-dimensional point cloud acquisition method is used to determine the simulated training acquisition information based on the extracted sequence image data.

[0080] The ellipsoid center coordinates are extracted and processed based on the simulated training information to obtain the residual cloud data of the training rapeseed population.

[0081] As an optional implementation, a three-dimensional point cloud acquisition method is used to determine training acquisition information based on sequence image data, specifically including:

[0082] The motion structure recovery and multi-view stereo vision algorithms are used to determine the processing data based on the sequence image data; the processing data includes: sparse point cloud data and camera pose parameters.

[0083] A three-dimensional Gaussian scattering algorithm is used to determine an ellipsoid group in three-dimensional space based on the processed data. The ellipsoid group includes multiple ellipsoids. Each ellipsoid is a three-dimensional Gaussian distribution of attribute parameters. The attribute parameters include the coordinates of the ellipsoid center, opacity, three-dimensional covariance matrix, and spherical harmonics.

[0084] Each ellipsoid in the ellipsoid group is projected onto the projection plane of the corresponding posture parameter according to a preset angle, and rendered to obtain a rendered image.

[0085] The pixel color on the rendered image is calculated using transparency synthesis; the pixel color is the pixel color value; the pixel color includes: the color values ​​of the three primary colors R, G, and B.

[0086] The back propagation algorithm is used to optimize the attribute parameters with the goal of minimizing the pixel color loss function to obtain the optimized attribute parameters; the pixel color loss function is determined based on the error between the pixel color after synthetic calculation and the actual pixel color.

[0087] The training acquisition information is determined based on the optimized attribute parameters and ellipsoid group.

[0088] The present application also provides an application scenario that applies the above-mentioned method for determining the morphological structure of rapeseed populations. Specifically, the method for determining the morphological structure of rapeseed populations provided in this embodiment can be applied in plant phenotyping research scenarios. The plant phenotyping research scenario includes residual point cloud acquisition, point cloud processing, and three-dimensional morphology acquisition. The residual point cloud enters the point cloud processing from the point cloud acquisition stage, and the complete point cloud data of the rapeseed population is obtained through the rapeseed population point cloud completion model, and then enters the downstream three-dimensional morphology acquisition stage.

[0089] In practical applications, the various operation processes of the method mentioned in this application, such as Figure 5 As shown, they correspond to the contents of three technical solutions respectively.

[0090] 1. Acquisition of 3D point cloud of a single rapeseed plant.

[0091] The present invention uses Neural Radiance Fields (NeRF) technology to reconstruct 3D point clouds, so it is necessary to collect multi-view images of individual rapeseed plants. Since field and indoor planting have a significant impact on the morphological structure of rapeseed, in order to better restore the actual production situation, the rapeseed needs to be uprooted from the field and placed in a Figure 2 On the stage shown, a mobile phone, drone, or other imaging device was used to capture rapeseed in video mode. Three rotations were performed, with the camera angled at -60°, -30°, and 10° relative to the horizontal.

[0092] After the recording is completed, a series of static images are generated in the computer by extracting key frames. Then, based on the sequence images, the sparse point cloud and camera pose parameters are obtained using the Structure from Motion (SfM) and Multi-View Stereo (MVS) algorithms.

[0093] The sparse point cloud is input into the three-dimensional Gaussian Splatting (3DGS) algorithm to obtain a dense group of ellipsoids in three-dimensional space. Each ellipsoid represents a three-dimensional Gaussian distribution, including the point position (coordinates of the ellipsoid center) μ, opacity β, three-dimensional covariance matrix ∑, and color (spherical harmonic function) C. These attribute parameters are all learnable and optimized through backpropagation.

[0094] These ellipsoids are projected along a specific angle onto the projection plane of the corresponding pose for rendering. Given the view transformation W and the three-dimensional covariance matrix ∑, the projected two-dimensional covariance matrix ∑' can be calculated using the following formula:

[0095] ∑'=JW∑W T J T

[0096] Where J is a matrix that converts a 3D vector into a 2D vector. The center position and color of the projected 2D Gaussian can be directly obtained from the parameters of the 3D Gaussian. The opacity β' of the projected 2D Gaussian needs to be adjusted based on the opacity and covariance matrix of the 3D Gaussian. The specific formula is as follows:

[0097]

[0098] For a given pixel position x, through the view transformation W, the distance between the point and all overlapping Gaussians can be calculated, that is, the depth of these Gaussians, forming a sorted Gaussian list The final color of this pixel is then calculated using transparency compositing:

[0099]

[0100] Among them C i is the learned color, and the final opacity β' i is the learned opacity β i The product of and Gaussian distribution is:

[0101]

[0102] where x' and μ' i are coordinates in the projected space.

[0103] By calculating all pixels, we get the entire picture, then compare it with the true value and calculate the loss function. The formula is as follows:

[0104]

[0105] Where λ is a weight factor, Loss and The term is a standard metric. The back propagation algorithm is used to optimize the parameters contained in the above three-dimensional Gaussian distribution, continuously reducing the loss function, and finally obtaining a three-dimensional Gaussian ellipsoid group containing realistic attribute parameters.

[0106] Based on the above results, by setting different observation perspectives, corresponding two-dimensional images can be obtained through projection, which can be used for subsequent simulation of rapeseed group scenes. Secondly, the center coordinates of the above ellipsoid can be extracted to form a complete three-dimensional point cloud of rapeseed plants.

[0107] 2. Rapeseed group scene simulation and point cloud completion dataset construction.

[0108] The scene simulation requires the use of the computer software Unreal Engine (UE). After installing the plug-in Luma AI, the results obtained in the first step are imported into the software to obtain a single rapeseed plant scene that can truly restore the real scene when viewed from different angles.

[0109] Duplicate the above scene 15 times to obtain 16 rapeseed plants. Set their coordinates to the absolute origin (x=0, y=0, z=0). Based on the commonly used planting density (row spacing 15 cm, plant spacing 10 cm), and the fact that one unit of the absolute coordinate system in UE corresponds to a length of 1 cm, set the coordinates of each rapeseed plant to the following format:

[0110]

[0111] After setting the coordinates, you can see the generated rapeseed group simulation scene in the software. Then, use the simulation camera and guide rail function of the UE software to set the imaging path and imaging parameters to generate a simulated image of the rapeseed group. Figure 3 As shown, three shooting paths are set: one with the camera pointing parallel to the horizontal plane, simulating the imaging results of a ground platform; the other two with the camera pointing at angles of -45° and -60° to the horizontal, respectively, simulating the results of drone photography. The camera imaging parameters include three: the video frame rate, set to 30 frames; the output format, selected as PNG; and the video duration, set to 1 minute. After completing the settings, select Export Simulated Imaging Results, which will result in three folders, each containing 1,800 images.

[0112] After acquiring simulated images of a rapeseed plant population, 200 images were evenly sampled from each folder and stored in three new folders to expedite processing. For each folder, the 3D point cloud acquisition method designed in the first step was applied to obtain three incomplete point cloud data sets due to inter-plant occlusion. The incomplete point clouds were then accurately registered with the complete point cloud used in the simulation using the Normal Distribution Transform (NDT) and Iterative Closest Point (ICP) algorithms, successfully constructing the training data required for the three-pair point cloud completion model. Specifically, the NDT algorithm divides the reference point cloud into multiple small voxels and approximates the point cloud distribution within each voxel using a normal distribution, thereby establishing a probability density function. The algorithm then matches the probability density functions of the target and reference point clouds and achieves preliminary registration by optimizing the transformation parameters. The ICP algorithm then refines the NDT registration, first finding the closest point in the reference point cloud for each point in the target point cloud. By minimizing the sum of the distances from each point in the target point cloud to its nearest corresponding point, the optimal rotation and translation are calculated, achieving precise point cloud registration. By setting different combinations of individual plants, varying planting spacing, and varying growth directions, the above method can be applied to generate diverse population morphological structures. This in turn constructs a point cloud completion dataset with sufficiently broad feature coverage, providing a solid foundation for training deep learning models with sufficient generalization capabilities.

[0113] 3. Train the point cloud completion model to obtain the complete morphological structure of rapeseed plants.

[0114] After obtaining enough training data, we adopt the idea of ​​adversarial generative network to build a point cloud completion model, which consists of two parts: generator and discriminator. Figure 4As shown in the figure, the generator consists of a multi-resolution encoder and a pyramid point cloud decoder. After the residual point cloud is input, the original point cloud is downsampled to half and one-quarter of the original number through iterative farthest point sampling (IFPS), and three resolution point cloud inputs (X, X', X") are obtained. The input point cloud is continuously improved in feature dimension through the multi-layer perceptron (MLP) method. A total of 5 layers of network are set up. After each layer is calculated, a maximum pooling operation is performed to obtain vectors with dimensions of 64, 128, 256, 512, and 1024. The vectors of the last four layers are concatenated to obtain a 1920-dimensional feature vector, which contains both low-level and high-level feature information. The three feature vectors extracted from the three point cloud inputs are then concatenated to obtain the final feature extraction result (V). In order to correspond to the input of three resolutions, the feature vector (V) undergoes two convolution operations to generate three fully connected layers (FC1, FC2, FC3). The deepest FC3 is used to generate the rough result of the completed point cloud, i.e. Y" Pre , then add FC3 and FC2 to generate a medium-density completed point cloud, namely Y' Pre Finally, add FC1, FC2, and FC3 together to generate the details of the completed point cloud, namely Y Pre After obtaining a complete and dense completed point cloud, it is input into the discriminator to determine the accuracy of the generated point cloud.

[0115] The discriminator is a multi-layer perceptron network with 4 layers. After the point cloud is input, each layer (MLP) calculates and obtains feature vectors of size 64, 64, 128, and 256. The features extracted by the last three layers are fused to obtain a vector of size 448. The discrimination result is then obtained using the Sigmoid classifier through the fully connected layer.

[0116] The point cloud completion effect predicted by the model is evaluated using the chamfer distance (CD), which is formulated as follows:

[0117]

[0118] Among them, S1 and S2 represent two groups of 3D point clouds. The first item represents the sum of the minimum distances from any point x in S1 to S2, and the second item represents the sum of the minimum distances from any point y in S2 to S1. The smaller the distance, the better the reconstruction effect.

[0119] Since the point pyramid decoder will predict three point clouds with different resolutions, the multi-resolution completion loss is given by d CD1 d CD2 and d CD3 It is composed of weighted hyperparameter α. The specific calculation formula is as follows:

[0120] L com =d CD1 (Y Pre ,Y GT )+αd CD2 (Y′ Pre ,Y′ GT )+2αd CD3 (Y″ Pre ,Y″ GT )

[0121] Each of these items calculates the chamfer distance between the predicted result and the real point cloud, and the hyperparameter α is set to a different constant as the model is trained.

[0122] The discriminator part also requires a loss function for training. Here, the generator part is defined as the function F(), and the discriminator part is defined as the function D(). The adversarial loss formula is as follows:

[0123]

[0124] where x i ∈X, i=1,…,S, where S is the number of samples in the dataset. X is the point cloud dataset predicted by the model, and Y is the actual point cloud dataset.

[0125] Finally, the loss function is divided into two parts: multi-resolution completion loss and adversarial loss. The formula is as follows:

[0126] L=λ com L com +λ adv L adv

[0127] Among them, λ com and λ adv Is a weight hyperparameter that satisfies the following conditions: com +λ adv =1.

[0128] The model training process uses Adaptive Moment Estimation (Adam) to optimize the network with an initial learning rate of 0.0001. After each round of network training, Adam updates the network weights by calculating the first-order moment (i.e., the mean of the gradient) and the second-order moment (i.e., the uncentered variance of the gradient) based on the backpropagation algorithm. The loss function is then calculated until the loss function is less than the set value (0.5) or the number of training rounds (200 rounds) reaches the set value. The final network weights are saved to obtain the point cloud completion model.

[0129] In research related to the three-dimensional morphological structure of rapeseed populations, no matter what method is used, after collecting the incomplete point cloud of the population, the trained model can be input to complete the complete three-dimensional point cloud data that is difficult to obtain in practice, thereby improving the accuracy and completeness of the three-dimensional phenotypic information analysis.

[0130] Compared to existing 3D point cloud acquisition solutions, the method for acquiring a complete point cloud of a single rapeseed plant proposed in this application offers the advantages of low cost and fast reconstruction speed. The low cost is reflected in the fact that no specialized acquisition equipment is required; only a simple platform supporting the plant is required, and then any imaging device is used to capture the surrounding image. The fast reconstruction speed is reflected in the fact that this patent uses the 3DGS algorithm based on NeRF principles to reconstruct the 3D point cloud, which is faster than existing methods.

[0131] This application proposes for the first time a solution to construct a rapeseed group point cloud completion dataset based on real single rapeseed plant point clouds through computer simulation methods. Currently, there is no relevant method to achieve this goal.

[0132] After the point cloud completion model mentioned in this application is trained, it can be directly used to complete the residual point cloud of the rapeseed group actually collected to obtain a complete morphological structure, which for the first time solves the problem that existing solutions cannot obtain complete point clouds of plant groups.

[0133] In addition, in the complete point cloud collection plan for a single rapeseed plant, if factors such as cost and efficiency are not considered, imaging equipment based on principles such as structured light or stereo vision, as well as depth cameras, lidar scanners, etc. can be used. These devices can complete the collection of point cloud data.

[0134] In terms of the selection of computer simulation software, in addition to the UE used in this application, the same effect can also be achieved through other software with the same functions, such as Blender.

[0135] Currently, deep learning is developing rapidly, and point cloud completion models are also iterating rapidly. Therefore, in addition to the model used in this application, other point cloud completion models can be replaced and the training data can be used to train different models.

[0136] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store point cloud data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for determining the morphological structure of a rapeseed population is implemented.

[0137] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0138] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0140] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0141] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0142] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for determining the morphological structure of rapeseed populations, characterized in that: The method for determining the morphological structure of rapeseed population comprises: Obtain residual cloud data of the target rapeseed population; The incomplete point cloud data is input into a rapeseed population point cloud completion model, and complete point cloud data of the rapeseed population is output; the complete point cloud data of the rapeseed population is three-dimensional point cloud data; the complete point cloud data of the rapeseed population is used to determine the three-dimensional phenotypic information of the rapeseed population and the morphological structure of the rapeseed population; the rapeseed population point cloud completion model is obtained by iteratively updating and training a constructed rapeseed population point cloud completion network using adaptive moment estimation and back propagation algorithms; the rapeseed population point cloud completion network is constructed based on the concept of generative adversarial networks; the rapeseed population point cloud completion network includes a generator and a discriminator connected in sequence; the generator includes: a multi-resolution encoder and a pyramid point cloud decoder; Acquiring collected data; the acquiring collected data specifically includes: Based on the set number of circles and the set angle, video data of a single rapeseed plant for training is obtained; Determining sequence image data; the sequence image data is a series of static images generated by extracting video frames from the video data; Using a three-dimensional point cloud acquisition method, training acquisition information is determined based on the sequence image data; the training acquisition information includes: a three-dimensional Gaussian ellipsoid group; The ellipsoid center coordinates are extracted and processed based on the training information to obtain the complete point cloud data of the training single rapeseed plant; Based on the UE software, according to the set imaging path and the set imaging parameters, a rapeseed population simulation sequence image is determined according to the training acquired information; uniformly extracting a set number of static images from the rapeseed population simulation sequence image to obtain extracted sequence image data; Using a three-dimensional point cloud acquisition method, the simulated training acquisition information is determined based on the extracted sequence image data; The ellipsoid center coordinates are extracted and processed based on the simulated training information to obtain the residual cloud data of the training rapeseed population.

2. The method for determining the morphological structure of rapeseed population according to claim 1, characterized in that: The method for determining the rapeseed population point cloud completion model includes: Acquire collected data; the collected data includes: complete point cloud data of a training single rapeseed plant and residual point cloud data of a training rapeseed group; Using normal distribution transformation and iterative closest point algorithm, the training data is registered to obtain a point cloud completion data set, and the point cloud completion data set is determined as training data; Constructing a rapeseed population point cloud completion network; Inputting residual point cloud data of a training rapeseed population into the rapeseed population point cloud completion network, with the goal of meeting a set number of training rounds or minimizing a loss function, adopting adaptive moment estimation and back propagation algorithms to iteratively update and train the parameters of the rapeseed population point cloud completion network to obtain a trained rapeseed population point cloud completion network; the loss function is determined based on the training data and data output by the rapeseed population point cloud completion network; the loss function includes a multi-resolution completion loss function and an adversarial loss function; and the parameters include weights; The trained rapeseed population point cloud completion network is determined as the rapeseed population point cloud completion model.

3. The method for determining the morphological structure of rapeseed population according to claim 1, characterized in that: Using a three-dimensional point cloud acquisition method, determining training acquisition information based on the sequence image data specifically includes: Using motion structure recovery and multi-view stereo vision algorithms, determining processing data based on the sequence image data; the processing data includes: sparse point cloud data and camera pose parameters; A three-dimensional Gaussian scattering algorithm is used to determine an ellipsoid group in three-dimensional space based on the processed data; the ellipsoid group includes a plurality of ellipsoids; each ellipsoid is a three-dimensional Gaussian distribution of attribute parameters; the attribute parameters include: coordinates of the ellipsoid center, opacity, a three-dimensional covariance matrix, and spherical harmonics; Projecting each ellipsoid in the ellipsoid group onto a projection plane corresponding to the posture parameter according to a preset angle, and rendering to obtain a rendered image; Calculating the pixel color on the rendered image by using transparency synthesis; the pixel color is a pixel color value; the pixel color includes: the color values ​​of the three primary colors R, G, and B; Using a back propagation algorithm, the attribute parameters are optimized with the goal of minimizing a pixel color loss function to obtain optimized attribute parameters; the pixel color loss function is determined based on the error between the pixel color after synthetic calculation and the actual pixel color; The training acquisition information is determined according to the optimized attribute parameters and the ellipsoid group.

4. The method for determining the morphological structure of rapeseed population according to claim 2, characterized in that: The expression of the loss function is: L=λ com L com +λ adv L adv ; Among them, L is the loss function; λ com and λ adv are all weight hyperparameters; L com is the multi-resolution completion loss function; L adv is the adversarial loss function.

5. The method for determining the morphological structure of rapeseed population according to claim 4, characterized in that: The expression of the multi-resolution completion loss function is: L com =d CD1 (Y Pre ,Y GT )+αd CD2 (Y′ Pre ,Y′ GT )+2αd CD3 (Y″ Pre ,Y″ GT ); Among them, L com is the multi-resolution completion loss function; d CD1 Y is the value of the similarity evaluation index CD between the fine completion point cloud data generated by the three fully connected layers in the generator and the actual point cloud; Pre Y is the point cloud data for generating detailed completion by three fully connected layers in the generator; GT is the actual fine point cloud data output by the expected model; α is the weighted hyperparameter; d CD2 Y' is the value of the similarity evaluation index CD between the medium-density completed point cloud data generated by the two fully connected layers in the generator and the actual medium-density point cloud data; Pre Y' is the medium-density completed point cloud data generated by two fully connected layers in the generator; GT is the medium-density point cloud data obtained by downsampling the actual fine point cloud output by the expected model; d CD3 Y″ is the value of the similarity evaluation index CD between the rough point cloud data corresponding to the completed point cloud data generated by a fully connected layer in the generator and the actual rough point cloud data; Pre Y″ is the rough result corresponding to the completed point cloud data generated by a fully connected layer in the generator; GT It is the coarse point cloud data output by the expected model, which is obtained by downsampling the actual fine point cloud data.

6. The method for determining the morphological structure of rapeseed population according to claim 4, characterized in that: The expression of the adversarial loss function is: Among them, L adv is the adversarial loss function; S is the number of samples of residual cloud data for training the rapeseed population; i is the sample number; D( ) is the function corresponding to the discriminator; F( ) is the function corresponding to the generator; y i is the actual point cloud data; x i is the point cloud data predicted by the model.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for determining the morphological structure of a rapeseed population according to any one of claims 1 to 6.

Images