Point cloud segmentation method and system for crop population

Abstract

The invention discloses a crop population point cloud segmentation method and system, and relates to the field of agricultural engineering, and the method comprises the steps: inputting a to-be-detected corn population point cloud into a feature vector model, and obtaining a feature vector of the corn population point cloud; the multilayer perceptron uses an L1 loss function and a direction loss function as loss functions, the feature vector of the single-plant corn point cloud is used for training the multilayer perceptron to generate an offset vector of the single-plant corn point cloud, and an offset model is obtained; inputting the feature vector of the corn group point cloud into an offset model to obtain a constrained offset vector; according to the direction of the constrained offset vector of the point cloud of each plant of corn in the corn group, moving each point cloud of each plant of corn to the central axis of the single plant of corn to which the point cloud belongs, and obtaining a new coordinate set of the point cloud of the corn group; and segmenting the new coordinate set of the corn group point clouds into a plurality of single-plant corn aggregation point clouds by using a median normalization vector growth method to finish segmentation. According to the invention, complex and dense corn groups can be accurately segmented.

Description

Technical Field

[0001] The present invention relates to the field of agricultural engineering, and in particular to a point cloud segmentation method and system for a crop group. Background Art

[0002] Corn phenotyping is of great significance to global food supply. Although LiDAR technology provides a powerful tool for 3D corn phenotyping indoors and in the field, the point cloud segmentation of individual plants and the extraction of their phenotypic parameters in complex field environments still face many challenges.

[0003] The existing technology uses a two-stage segmentation strategy to perform point cloud segmentation on individual plants. First, the stem point cloud is extracted using unsupervised point cloud segmentation technology or deep learning methods. The leaf point cloud is then further classified using the extracted stem point cloud, and the dense corn population is segmented into individual corn plants for corn phenotype analysis.

[0004] However, the above method only focuses on the point cloud feature extraction of corn stalks during the point cloud segmentation process of dense corn groups, but lacks the point cloud feature extraction of parts other than corn stalks. In the process of segmenting the dense corn group into single corn plants, all point clouds of the single corn plant cannot be retained, resulting in inaccurate segmentation results. Summary of the invention

[0005] The embodiments of the present invention provide a point cloud segmentation method and system for a crop group, which can solve the problem in the prior art that when a dense corn group is segmented into individual corn plants, all point clouds of the corn plant cannot be retained, resulting in inaccurate segmentation results.

[0006] The embodiment of the present invention provides a point cloud segmentation method for a crop group, comprising the following steps: using point clouds to train a point cloud segmentation network Point Transformer V3 to generate feature vectors and obtain a feature vector model; inputting a corn group point cloud to be detected into the feature vector model to obtain a feature vector of the corn group point cloud; using the feature vector of a single corn point cloud to train a multilayer perceptron to generate an offset vector of the single corn point cloud, and constraining the size of the offset vector of the single corn point cloud through an absolute error loss function L1, and constraining the direction of the offset vector of the single corn point cloud to a projection direction through a direction loss function, so that each point of the single corn point cloud can be moved toward the central axis of the single corn along the projection direction, and obtaining an offset model; inputting the feature vector of the corn group point cloud into the offset model to obtain a point cloud of each corn in the corn group. The invention relates to a constrained offset vector of a rice point cloud; adding the coordinates of each corn point cloud in the corn group point cloud to the constrained offset vector of the point cloud to obtain the new coordinates of each corn point cloud in the corn group point cloud, wherein the new coordinates represent the projection position of each point cloud moved along the projection direction to the central axis of the single corn to which the point cloud belongs; obtaining a new coordinate set of the corn group point cloud according to the new coordinates of each corn point cloud in the corn group point cloud; using the median normalized vector growing method to segment the new coordinate set of the corn group point cloud into a plurality of single corn aggregated point clouds, wherein each of the single corn aggregated point clouds is a point cloud aggregated on the central axis of the single corn, and obtaining the point cloud segmentation result of the corn group.

[0007] Furthermore, the step of obtaining the point cloud of the corn group in the virtual planting area specifically includes: applying forces of different displacements, directions and strengths to various organs of a single corn plant in the planting area to guide the morphology of each organ to change, and obtaining point clouds of single corn plants of different morphologies; constructing a virtual planting area according to the row spacing, plant spacing, number of rows and number of plants of the corn plants in the planting area, and adjusting the position of each corn plant by random offset to simulate the irregularity of a real planting scene.

[0008] Furthermore, the absolute error loss function L1 is used to constrain the size of the offset vector of the single corn point cloud, and the direction of the offset vector of the single corn point cloud is constrained to be the projection direction by the direction loss function. The specific steps include:

[0009] Use the L1 loss function to constrain the size of the offset vector;

[0010] The L1 loss function L reg , the formula is:

[0011]

[0012] Among them, i For each point cloud, the vector moving toward the central axis is:

[0013] o i =(b i ·a i )·a i -b i

[0014] b i =p i -s i

[0015] Among them, v i Represents the offset vector of the i-th point, s i is the lowest point of the corn plant's stem, a i The corn plant is formed by the stem bottom i Starting direction vector along the stem, p i represents a point in the corn population point cloud, b i Indicates that the stem is s i Departure to point p i A vector of

[0016] Use the direction loss function to constrain the direction of the offset vector;

[0017] The direction loss function L dir , the formula is:

[0018]

[0019] Furthermore, the new coordinate set of the corn group point cloud is divided into multiple single corn clustered point clouds using the median normalized vector growth method, and the specific steps include: finding the lowest point of each corn point cloud in the corn group point cloud, and constructing an initial circle with a radius of r from the lowest point of each corn point cloud, and obtaining the median vector within the initial circle; along the direction of the median vector, constructing a circle every r from the center of the initial circle, and obtaining the median vector within the corresponding circle until the highest point of each corn point cloud is reached; wherein, the process of obtaining the median vector is to perform central axis clustering on the corn group point cloud.

[0020] An embodiment of the present invention provides a point cloud segmentation system for a crop group, comprising:

[0021] The region construction module is used to use the point cloud to train the point cloud segmentation network Poi nt Transformer V3 to generate feature vectors and obtain a feature vector model; the corn group point cloud to be detected is input into the feature vector model to obtain the feature vector of the corn group point cloud; the central axis aggregation network construction module is used to use the feature vector of the single corn point cloud to train the multilayer perceptron to generate the offset vector of the single corn point cloud, and constrain the size of the offset vector of the single corn point cloud through the absolute error loss function L1, and constrain the direction of the offset vector of the single corn point cloud to the projection direction through the direction loss function, so that each point of the single corn point cloud can be moved along the projection direction toward the central axis of the single corn to obtain the offset model; the feature vector of the corn group point cloud is input into the offset model to obtain the offset vector of each corn point cloud in the corn group. The constrained offset vector of the corn group point cloud is added; the coordinates of each corn point cloud in the corn group point cloud are added to the constrained offset vector of the point cloud to obtain the new coordinates of each corn point cloud in the corn group point cloud, and the new coordinates represent the projection position of each point cloud moved along the projection direction to the central axis of the single corn to which the point cloud belongs; according to the new coordinates of each corn point cloud in the corn group point cloud, a new coordinate set of the corn group point cloud is obtained; a point value normalized bias vector growth construction module is used to use the median normalized vector growth method to segment the new coordinate set of the corn group point cloud into multiple single corn aggregated point clouds, each of the single corn aggregated point clouds is a point cloud aggregated on the central axis of the single corn, and the point cloud segmentation result of the corn group is obtained.

[0022] The embodiment of the present invention provides a point cloud segmentation method and system for a crop group. Compared with the prior art, the beneficial effects thereof are as follows:

[0023] The corn group point cloud to be detected is input into the feature vector model to obtain the feature vector of the corn group point cloud; the feature vector of the corn group point cloud is input into the offset model to obtain the constrained offset vector of the point cloud of each corn plant in the corn group. The offset model constrains the size and direction of the offset vector through the L1 loss function and the direction loss function. At this time, the offset vector of the point cloud of a single corn plant points to the central axis of the single corn plant to which the point cloud belongs along the projection direction; then, according to the direction of the constrained offset vector of the point cloud of each corn plant in the corn group, each point cloud of each corn plant is moved along the projection direction to the projection position on the central axis of the single corn plant to which the point cloud belongs, and the new coordinate set of the corn group point cloud is obtained according to the new coordinates of each corn point cloud in the corn group point cloud.

[0024] The point cloud gathered at the central axis not only includes the point cloud of the corn stalk, but also includes the point cloud of the rest of the corn stalk. All the point clouds of each corn plant are gathered at the central axis of each corn plant by central axis clustering. During the segmentation process, the median normalized vector growing method is used to segment the new coordinate set of the corn group point cloud into multiple single corn clustered point clouds. After the segmentation, all the point clouds of each corn plant are gathered at the central axis, making the segmentation result more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A technical solution diagram provided for an embodiment of the present invention;

[0026] Figure 2 A visualization diagram of the predicted column vector provided by an embodiment of the present invention;

[0027] Figure 3 A result diagram of the projection-based central axis aggregation strategy provided by an embodiment of the present invention;

[0028] Figure 4 The simulation results of various densities of corn populations at the seedling stage and mature stage provided by the embodiments of the present invention;

[0029] Figure 5 The corn group point cloud segmentation result in 17 group data provided by the embodiment of the present invention;

[0030] Figure 6 A flow chart of a method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0032] See also Figure 1 , an embodiment of the present invention provides a point cloud segmentation method for a crop group, comprising the following steps:

[0033] Step 1: Obtain the point cloud of the corn population in the virtual planting area.

[0034] The specific construction method is as follows:

[0035] The point cloud enhancement method based on physical deformation is used to induce changes in organ morphology by applying forces of different displacements, directions and strengths to the organs of individual plants, thereby enhancing the diversity of plant morphology and generating a large number of point clouds of individual plants with different morphologies as the simulation data set C. Subsequently, according to the row spacing d row 、Plant spacing d plant , number of rows n row and number of plants n plant Constructing virtual planting area U original , and adjust the position of each plant by random offset ΔU = (Δ x ,Δ y ,0), to simulate the irregularities in real planting scenes.

[0036] U new =U original +ΔU

[0037] The point cloud data C of each plant is transformed by affine transformations such as rotation R and scaling S to simulate its natural posture in three-dimensional space. new Move to the specified position U new , and finally a corn population point cloud with high realism and diversity is generated.

[0038] C new =C.R.S

[0039] Step 2: Use the point cloud to train the point cloud segmentation network Point Transformer V3 to generate feature vectors and obtain the feature vector model; input the corn group point cloud to be detected into the feature vector model to obtain the feature vector of the corn group point cloud. Use the feature vector of the single corn point cloud to train the multilayer perceptron to generate the offset vector of the single corn point cloud, and constrain the size of the offset vector of the single corn point cloud through the absolute error loss function L1, and constrain the direction of the offset vector of the single corn point cloud to the projection direction through the direction loss function, so that each point of the single corn point cloud can move along the projection direction toward the central axis of the single corn, and obtain the offset model. The characteristic vector of the corn group point cloud is input into the offset model to obtain the constrained offset vector of the point cloud of each corn plant in the corn group; the coordinates of the point cloud of each corn plant in the corn group point cloud are added to the constrained offset vector of the point cloud to obtain the new coordinates of each corn point cloud in the corn group point cloud, and the new coordinates represent the projection position of each point cloud moved along the projection direction to the central axis of the single corn plant to which the point cloud belongs; and the new coordinate set of the corn group point cloud is obtained according to the new coordinates of each corn point cloud in the corn group point cloud.

[0040] The specific operation process is as follows:

[0041] Point TransformerV3 is selected as the backbone network. First, the point cloud set P input to the backbone network contains N points, each with coordinates p i and the normal vector f i After being processed by the backbone network, the feature vector of each point is generated These features are then mapped into offset vectors through a multi-layer perceptron (MLP) where v i Represents the offset vector of the i-th point. Aiming at the point cloud characteristics of dense plant groups, this paper designs a projection-based central axis aggregation method, aiming to aggregate the point clouds of individual plants to their central axis. To achieve this goal, we constrain the size and direction of the offset vector.

[0042] The design of loss function plays a crucial role in deep learning networks. For points belonging to the same corn plant, we use L1 loss to constrain the size of the learned offset vector, and the loss function is:

[0043]

[0044] The vector o i As shown below.

[0045] o i =(b i ·a i )·a i -b i

[0046] b i =p i -s i

[0047] Among them, point s i is the lowest point of the stem of the jth plant instance, where a i is the number of stem bases s in the jth plant instance i Starting from the direction vector of the first principal component of the stem, in order to intuitively understand the meaning of each vector, we provide the vector o i Visualization, such as Figure 2 As shown. We use direction loss to constrain the direction of the offset vector:

[0048]

[0049] This loss is independent of the magnitude of the vector and can ensure that the point moves toward the corresponding plant axis along the projection direction.

[0050] Through the learned offset vector v i , each point pi Move to the projection position of the corresponding corn central axis, the projection method is as follows Figure 3 As shown, the adjusted coordinates are This approach makes the point clouds of corn plants belonging to the same instance cluster around the central axis of the individual corn, rather than being randomly distributed in space.

[0051] Step 3: Use the median normalized vector growing method to segment the new coordinate set of the corn group point cloud into multiple individual corn plant clustered point clouds. Each individual corn plant clustered point cloud is a point cloud clustered on the central axis of the individual corn plant, and the point cloud segmentation result of the corn group is obtained.

[0052] The specific steps are as follows:

[0053] The adjusted coordinate set Q = {q i} to form a clustered point cloud, and then generate different cluster instances. Finally, the instance labels are mapped back to the original point cloud to obtain the segmentation result G of the corn population. This method effectively achieves instance segmentation in dense and complex agricultural environments, significantly improving the accuracy and reliability of phenotyping and agricultural monitoring.

[0054] In order to solve the problem of separating individual corn point clouds from a group, this paper proposes a 3D point cloud instance segmentation network based on medial axis aggregation. The method mainly consists of three parts: a backbone network, a projection-based medial axis aggregation strategy, and a point value normalized bias vector growth method.

[0055] The present invention can achieve the following effects:

[0056] 1) The corn population point cloud simulation method proposed in the present invention can efficiently simulate corn population point clouds of various densities and arbitrary scales, thereby providing a rich and diverse data set for the training of population segmentation networks. The method is simple and fast, greatly reducing the time and cost of data acquisition.

[0057] 2) The 3D point cloud medial axis aggregation network provided by the present invention can achieve good segmentation effect in real corn point clouds collected by different sensors without additional training. This not only promotes the development of corn population point cloud segmentation technology, but also provides an important reference for point cloud segmentation of other crops. This method is crucial for achieving rapid and high-throughput phenotyping and has broad application prospects in the fields of plant phenomics and intelligent breeding.

[0058] An embodiment of the present invention provides a point cloud segmentation system for a crop group, comprising:

[0059] The region construction module is used to obtain the point cloud of the corn population in the virtual planting area.

[0060] The backbone network building module is used to use the point cloud to train the point cloud segmentation network Point Transformer V3 to generate feature vectors and obtain a feature vector model; the point cloud of the corn group to be detected is input into the feature vector model to obtain the feature vector of the corn group point cloud.

[0061] The central axis aggregation network construction module is used to train a multilayer perceptron using the feature vector of a single corn point cloud to generate an offset vector of the single corn point cloud, and constrain the size of the offset vector of the single corn point cloud through the absolute error loss function L1, and constrain the direction of the offset vector of the single corn point cloud to the projection direction through the direction loss function, so that each point of the single corn point cloud can be moved toward the central axis of the single corn along the projection direction to obtain an offset model; the feature vector of the corn group point cloud is input into the offset model to obtain the constrained offset vector of each corn point cloud in the corn group; the coordinates of each corn point cloud in the corn group point cloud are added to the constrained offset vector of the point cloud to obtain the new coordinates of each corn point cloud in the corn group point cloud, and the new coordinates represent the projection position of each point cloud moving along the projection direction to the central axis of the single corn to which the point cloud belongs; and a new coordinate set of the corn group point cloud is obtained according to the new coordinates of each corn point cloud in the corn group point cloud.

[0062] The point value normalized biased vector growth construction module is used to segment the new coordinate set of the corn group point cloud into multiple single corn plant clustered point clouds using the median normalized vector growth method. Each single corn plant clustered point cloud is a point cloud clustered on the central axis of the single corn plant, so as to obtain the point cloud segmentation result of the corn group.

[0063] A specific embodiment is as follows:

[0064] This paper proposes a simplified and efficient simulation data-driven end-to-end instance segmentation framework for crop population point clouds, focusing on the individual plant segmentation and phenotyping measurement of high-density crop population point clouds. The technical solution mainly includes two parts: a simulation method for corn population point clouds and an end-to-end point cloud median axis aggregation network. The overall process is as follows: Figure 1 shown.

[0065] First, the present invention develops a corn population point cloud simulation tool for generating labeled synthetic point clouds to solve the problem of scarce labeled data. The tool uses a small amount of real labeled plant point clouds (including 40 mature plants and 40 seedling plants) as the basic data source, and uses physical deformation technology to expand and generate any number of labeled plant point clouds of different forms. Subsequently, through simple affine transformation, corn population point clouds of different planting densities and quantities are simulated. The simulation results are shown in Figure 2. Figure 4 shown.

[0066] In addition, the present invention designs an end-to-end point cloud central axis aggregation network for group instance segmentation. The network structure is simple, with PointTransformerV3 as the backbone network (Backbone Network), and then connected to an MLP module to map the point cloud features into aggregation offset vectors (OffsetVector), thereby shrinking the point cloud into easy-to-segment clusters. Based on the morphological characteristics of crops and their planting layout characteristics in agricultural scenarios, this study adopted a central axis aggregation strategy so that the corn plant point cloud of each instance is contracted on its central axis, rather than aggregated at the center point like traditional clustering methods. This method can directly process corn group point clouds from different data sources without the need for additional training. The segmentation results are shown in the figure. Figure 5 As shown, the method flow chart of the present invention is as follows Figure 6 shown.

[0067] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A point cloud segmentation method for a crop group, characterized in that: The following steps are involved: Obtaining a point cloud of corn colonies in a virtual planting area; Use the point cloud to train the point cloud segmentation network Point Transformer V3 to generate a feature vector and obtain a feature vector model; input the corn group point cloud to be detected into the feature vector model to obtain the feature vector of the corn group point cloud; The feature vector of a single corn point cloud is used to train a multilayer perceptron to generate an offset vector of the single corn point cloud, and the size of the offset vector of the single corn point cloud is constrained by the absolute error loss function L1, and the direction of the offset vector of the single corn point cloud is constrained to the projection direction by the direction loss function, so that each point of the single corn point cloud can be moved toward the central axis of the single corn along the projection direction to obtain an offset model; Input the feature vector of the corn group point cloud into the offset model to obtain the constrained offset vector of each corn point cloud in the corn group; add the coordinates of each corn point cloud in the corn group point cloud to the constrained offset vector of the point cloud to obtain the new coordinates of each corn point cloud in the corn group point cloud, wherein the new coordinates represent the projection position of each point cloud moved along the projection direction to the central axis of the single corn to which the point cloud belongs; obtain a new coordinate set of the corn group point cloud according to the new coordinates of each corn point cloud in the corn group point cloud; The new coordinate set of the corn group point cloud is segmented into a plurality of single corn plant clustered point clouds using the median normalized vector growing method, each of the single corn plant clustered point clouds being a point cloud clustered on the central axis of the single corn plant, thereby obtaining the point cloud segmentation result of the corn group.

2. The point cloud segmentation method for a crop group according to claim 1, characterized in that: The step of obtaining the point cloud of the corn group in the virtual planting area specifically includes: In the planting area, different displacements, directions and strengths of forces are applied to various organs of a single corn plant to guide the morphology of each organ to change, and point clouds of single corn plants with different morphologies are obtained; A virtual planting area is constructed according to the row spacing, plant spacing, number of rows and number of plants of corn plants in the planting area, and the position of each corn plant is adjusted by random offset to simulate the irregularity of the real planting scene.

3. The point cloud segmentation method for a crop group according to claim 1, characterized in that: The steps of constraining the size of the offset vector of the single corn point cloud by the absolute error loss function L1 and constraining the direction of the offset vector of the single corn point cloud to be the projection direction by the direction loss function include: Use the L1 loss function to constrain the size of the offset vector; The L1 loss function L reg , the formula is: Among them, i For each point cloud, the vector moving toward the central axis is: o i =(b i ·a i )·a i -b i b i =p i -s i Among them, v i Represents the offset vector of the i-th point, s i is the lowest point of the corn plant's stem, a i The corn plant is formed by the stem base i Starting direction vector along the stem, p i represents a point in the corn population point cloud, b i Indicates that the stem is s i Departure to point p i A vector of Use the direction loss function to constrain the direction of the offset vector; The direction loss function L dir , the formula is:

4. The point cloud segmentation method for a crop group according to claim 1, characterized in that: The method of using the median normalized vector growth method to divide the new coordinate set of the corn group point cloud into a plurality of individual corn cluster point clouds specifically comprises the following steps: Find the lowest point of each corn point cloud in the corn group point cloud, and construct an initial circle with a radius of r starting from the lowest point of each corn point cloud, and obtain the median vector within the initial circle; Along the direction of the median vector, a circle is constructed from the center of the initial circle at a distance r, and the median vector is obtained in the corresponding circle until the highest point of each corn point cloud; The process of obtaining the median vector is to perform median clustering on the corn group point cloud.

5. A point cloud segmentation system for crop groups, characterized in that: include: The region construction module is used to obtain the point cloud of the corn population in the virtual planting area; The backbone network building module is used to train the point cloud segmentation network Point Transformer V3 using the point cloud to generate a feature vector and obtain a feature vector model; the point cloud of the corn group to be detected is input into the feature vector model to obtain the feature vector of the corn group point cloud; A central axis aggregation network construction module is used to train a multilayer perceptron using the feature vector of a single corn point cloud to generate an offset vector of the single corn point cloud, and constrain the size of the offset vector of the single corn point cloud through the absolute error loss function L1, and constrain the direction of the offset vector of the single corn point cloud to the projection direction through the direction loss function, so that each point of the single corn point cloud can be moved toward the central axis of the single corn along the projection direction to obtain an offset model; the feature vector of the corn group point cloud is input into the offset model to obtain a constrained offset vector of each corn point cloud in the corn group; the coordinates of each corn point cloud in the corn group point cloud are added to the constrained offset vector of the point cloud to obtain new coordinates of each corn point cloud in the corn group point cloud, and the new coordinates represent the projection position of each point cloud along the projection direction onto the central axis of the single corn to which the point cloud belongs; a new coordinate set of the corn group point cloud is obtained according to the new coordinates of each corn point cloud in the corn group point cloud; The point value normalized biased vector growth construction module is used to use the median normalized vector growth method to segment the new coordinate set of the corn group point cloud into multiple single corn plant clustered point clouds, each of the single corn plant clustered point clouds is a point cloud clustered on the central axis of the single corn plant, and obtain the point cloud segmentation result of the corn group.

Images