Corn plant population canopy three-dimensional model construction method and system

Abstract

The invention provides a corn plant population canopy three-dimensional model construction method and system. The method comprises the steps of obtaining three-dimensional point cloud instance data of a target corn plant; based on the growth stage information of the target corn plant, determining a corresponding corn plant point cloud semantic segmentation model; inputting the three-dimensional point cloud instance data into a corn plant point cloud semantic segmentation model to obtain a three-dimensional point cloud semantic segmentation result; performing semantic segmentation according to the three-dimensional point cloud; constructing a corn organ grid corresponding to each organ point cloud of the target corn plant, and constructing a corn plant grid model corresponding to the target corn plant according to each corn organ grid; and based on preset corn plant region parameters and the corn plant grid model, constructing a corn plant population canopy three-dimensional model. According to the method, the canopy spatial heterogeneity presented by variety plant types and density differences is reflected more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method and system for constructing a three-dimensional model of a maize plant population canopy. Background Art

[0002] The plant type of crops consists of traits that affect crop photosynthesis, growth and development, and grain yield, which can maximize the light energy utilization rate, increase the biological yield and improve the economic coefficient. The research on crop plant type and population structure has an important impact on crop resource utilization efficiency, yield, stress resistance, etc.

[0003] At present, the three-dimensional models of maize plants and population canopies are mainly constructed by methods such as three-dimensional digitizers, assembled three-dimensional modeling, and three-dimensional point cloud reconstruction. However, the three-dimensional models of maize plant population canopies constructed by these methods cannot accurately reflect the canopy spatial heterogeneity presented by variety plant type and density differences.

[0004] Therefore, there is an urgent need for a method and system for constructing a three-dimensional model of a maize plant population canopy to solve the above problems. Summary of the Invention

[0005] In view of the problems existing in the prior art, the present invention provides a method and system for constructing a three-dimensional model of a maize plant population canopy.

[0006] The present invention provides a method for constructing a three-dimensional model of a maize plant population canopy, including: Obtaining three-dimensional point cloud instance data of a target maize plant; Determining a corresponding maize plant point cloud semantic segmentation model based on the growth stage information of the target maize plant, wherein the maize plant point cloud semantic segmentation model is trained based on three-dimensional point cloud sample data of sample maize plants at different growth stages and sample semantic labels corresponding to the three-dimensional point cloud sample data; Inputting the three-dimensional point cloud instance data into the maize plant point cloud semantic segmentation model to obtain a three-dimensional point cloud semantic segmentation result output by the target maize plant point cloud semantic segmentation model; According to the three-dimensional point cloud semantic segmentation result, constructing a maize organ grid corresponding to each organ point cloud of the target maize plant, and constructing a maize plant grid model corresponding to the target maize plant according to each maize organ grid; wherein, when the maize organ grid is a maize leaf organ grid, the maize leaf organ grid is constructed based on leaf point cloud data, and the leaf point cloud data is obtained by performing instance segmentation on the three-dimensional semantic segmentation result that is the maize leaf point cloud segmentation result; Based on preset maize plant area parameters and the maize plant grid model, constructing a three-dimensional model of a maize plant population canopy.

[0007] A method for constructing a three-dimensional model of the canopy of a corn plant population provided by the present invention, wherein the corn plant point cloud semantic segmentation model is trained through the following steps: Obtain first three-dimensional point cloud sample data, second three-dimensional point cloud sample data, and third three-dimensional point cloud sample data. Among them, the first three-dimensional point cloud sample data is the three-dimensional point cloud sample data of the sample corn plants in the seedling stage to the jointing stage; the second three-dimensional point cloud sample data is the three-dimensional point cloud sample data of the sample corn plants in the large flare stage; the third three-dimensional point cloud sample data is the three-dimensional point cloud sample data of the sample corn plants in the period after silking is completed. Mark the corresponding sample semantic labels for the corn stalk point cloud and the corn leaf point cloud in the first three-dimensional point cloud sample data to obtain a first training sample set. Mark the corresponding sample semantic labels for the corn stalk point cloud and the corn leaf point cloud in the second three-dimensional point cloud sample data to obtain a second training sample set. Mark the corresponding sample semantic labels for the corn stalk point cloud, the corn leaf point cloud, the corn tassel point cloud, and the corn ear point cloud in the third three-dimensional point cloud sample data to obtain a third training sample set. Based on the first training sample set, train a deep learning model to obtain the corn plant point cloud semantic segmentation model for semantic segmentation of the corn plant point cloud in the seedling stage to the jointing stage. Based on the second training sample set, train the deep learning model to obtain the corn plant point cloud semantic segmentation model for semantic segmentation of the corn plant point cloud in the large flare stage. Based on the third training sample set, train the deep learning model to obtain the corn plant point cloud semantic segmentation model for semantic segmentation of the corn plant point cloud in the period after silking is completed.

[0008] A method for constructing a three-dimensional model of the canopy of a corn plant population provided by the present invention. After inputting the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model and obtaining the three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model, the method further includes: Obtain the maximum size information of the three-dimensional point cloud instance data in the three-dimensional rectangular coordinate system. Based on the maximum size information, perform size reduction processing on the three-dimensional point cloud semantic segmentation result to obtain a size-reduced three-dimensional point cloud semantic segmentation result, wherein the size-reduced three-dimensional point cloud semantic segmentation result is the same as the size of the three-dimensional point cloud instance data. Traverse each point in the three-dimensional point cloud instance data, and determine the nearest point corresponding to each point in the three-dimensional point cloud instance data, where the nearest point is the point with the shortest distance between the point in the three-dimensional point cloud instance data and the semantic segmentation result of the three-dimensional point cloud after size reduction; Assign the semantic label of the nearest point to the corresponding point in the three-dimensional point cloud instance data. After determining that all points in the three-dimensional point cloud instance data have been traversed, obtain the target three-dimensional point cloud semantic segmentation result; The constructing the corn organ meshes corresponding to the organ point clouds of the target corn plant according to the three-dimensional point cloud semantic segmentation result includes: Construct the corn organ meshes corresponding to the organ point clouds of the target corn plant according to the target three-dimensional point cloud semantic segmentation result.

[0009] According to a method for constructing a three-dimensional model of a corn plant population canopy provided by the present invention, the method further includes: Based on the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result, obtain the corn leaf point cloud segmentation result and the corn stalk point cloud segmentation result; Calculate the Euclidean distance between the leaf and stalk points between each point in the corn leaf point cloud segmentation result and the target point in the corn stalk point cloud segmentation result, where the target point is the point at the bottom of the corn plant stalk in the corn stalk point cloud segmentation result; Based on the improved QuickShift++ algorithm, perform leaf point cloud segmentation on the corn leaf point cloud segmentation result to obtain the leaf point cloud data of each leaf in the corn leaf point cloud segmentation result, so as to construct the corresponding corn leaf organ mesh according to the leaf point cloud data, where the k-nearest neighbor radius in the improved QuickShift++ algorithm is calculated by the reciprocal of the Euclidean distance between the leaf and stalk points.

[0010] According to a method for constructing a three-dimensional model of a corn plant population canopy provided by the present invention, when it is determined that the development stage of the target corn plant is in the period after silking is completed, the method further includes: Obtain the distance between the lowest point in the leaf point cloud data and the target point in the corn stalk point cloud segmentation result, and sort the leaf point cloud data in ascending order of distance to obtain the sorting result of the leaf point cloud data; Based on the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result, determine the corn ear point cloud segmentation result; Based on the clustering algorithm, obtain the largest cluster point set in the segmentation result of the maize ear point cloud, and determine the ear leaf positioning point according to the heights of all points in the largest cluster point set, where the ear leaf positioning point represents the first quartile in the sorting result of the heights of all points in the largest cluster point set from low to high; the heights of all points in the largest cluster point set represent the heights between the points in the largest cluster point set and the target point; Based on the sorting result of the leaf point cloud data, determine the first target leaf point cloud data and the second target leaf point cloud data, where the first target leaf point cloud data is determined based on the point closest to the ear leaf positioning point in the leaf point cloud data; the second target leaf point cloud data is determined based on the point with the second closest distance to the ear leaf positioning point in the leaf point cloud data; According to the first target leaf point cloud data and the second target leaf point cloud data, determine the ear leaf point cloud data, where the ear leaf point cloud data is the first target leaf point cloud data or the second target leaf point cloud data with the smallest difference between the leaf azimuth angle and the ear azimuth angle; the ear azimuth angle is obtained based on the segmentation result of the maize ear point cloud.

[0011] According to a method for constructing a three-dimensional model of a maize plant population canopy provided by the present invention, the constructing of the maize organ grid corresponding to each organ point cloud of the target maize plant according to the three-dimensional point cloud semantic segmentation result includes: Perform voxel downsampling on the leaf point cloud data and / or the ear leaf point cloud data, and generate the maize leaf organ grid based on the leaf point cloud data after voxel downsampling; According to the radius of each section of the maize stem in the maize stem point cloud segmentation result, construct a multi-section frustum geometric model to obtain the maize stem organ grid; According to the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result, obtain the maize tassel point cloud segmentation result, and construct a frustum grid model according to the tassel point cloud skeleton extracted from the maize tassel point cloud segmentation result to obtain the maize tassel organ grid; Perform voxel downsampling on the maize ear point cloud segmentation result, and generate the maize ear organ grid based on the maize ear point cloud segmentation result after voxel downsampling; The constructing of the maize plant grid model corresponding to the target maize plant according to each of the maize organ grids includes: Based on different preset development stages, select the corresponding grid of the corn leaf apparatus, the grid of the corn stalk apparatus, the grid of the corn tassel apparatus, and the grid of the corn ear apparatus for combination to obtain the corn plant grid model corresponding to the target corn plant.

[0012] According to a method for constructing a three-dimensional model of a corn plant population canopy provided by the present invention, based on the preset corn plant area parameters and the corn plant grid model, constructing a three-dimensional model of a corn plant population canopy includes: According to the preset corn plant area parameters, obtain a set of seed plant models constructed from the corn plant grid models of different corn varieties and development stages, wherein the preset corn plant area parameters at least include corn variety, planting density, growth period information, planting row spacing, planting plant spacing, number of planting rows, number of plants per row, and planting row direction parameters; Based on the planting density within the corn plant area to be constructed, determine the plant azimuth plane angle of the target seed plant grid model; wherein the corn plant area to be constructed is constructed based on the planting row spacing, the planting plant spacing, the number of planting rows, the number of plants per row, and the planting row direction parameters; the target seed plant grid model is a seed plant grid model randomly selected from the set of seed plant models; According to the plant azimuth plane angle, rotate the target seed plant grid model along the Z-axis of the three-dimensional space coordinate system, and translate the rotated target seed plant grid model to the coordinate position of the corresponding planting point within the corn plant area to be constructed, wherein the coordinate position of the planting point is determined based on the planting row spacing and the planting plant spacing; After determining that the target seed plant grid model is set at all planting points within the corn plant area to be constructed, obtain a to-be-completed three-dimensional model of the corn plant population canopy, and based on the planting row direction parameters, rotate the to-be-completed three-dimensional model of the corn plant population canopy along the Z-axis of the three-dimensional space coordinate system to obtain the three-dimensional model of the corn plant population canopy.

[0013] The present invention also provides a system for constructing a three-dimensional model of a corn plant population canopy, including: A point cloud data acquisition module for acquiring three-dimensional point cloud instance data of a target corn plant; A point cloud semantic segmentation model determination module for determining a corresponding corn plant point cloud semantic segmentation model based on the growth stage information of the target corn plant, wherein the corn plant point cloud semantic segmentation model is trained based on the three-dimensional point cloud sample data of sample corn plants at different growth stages and the sample semantic labels corresponding to the three-dimensional point cloud sample data; A point cloud semantic segmentation module, configured to input the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model, and obtain a three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model; A corn plant model construction module, configured to construct a corn organ grid corresponding to each organ point cloud of the target corn plant according to the three-dimensional point cloud semantic segmentation result, and construct a corn plant grid model corresponding to the target corn plant according to each corn organ grid; wherein, when the corn organ grid is a corn leaf organ grid, the corn leaf organ grid is constructed based on leaf point cloud data, and the leaf point cloud data is obtained by instance segmentation of the three-dimensional semantic segmentation result which is the corn leaf point cloud segmentation result; A plant population canopy model construction module, configured to construct a three-dimensional model of the corn plant population canopy based on preset corn plant area parameters and the corn plant grid model.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the corn plant population canopy three-dimensional model construction method as described in any one of the above.

[0015] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the corn plant population canopy three-dimensional model construction method as described in any one of the above.

[0016] The corn plant population canopy three-dimensional model construction method and system provided by the present invention obtain the three-dimensional point cloud instance data of the target corn plant, select a corresponding point cloud semantic segmentation model according to its growth stage, then input the instance data into the model to obtain a three-dimensional point cloud semantic segmentation result, construct a point cloud grid of each organ of the target corn plant according to the segmentation result, and further construct a grid model of the whole corn plant. Finally, combined with the preset plant area parameters and the grid model, a three-dimensional model of the corn plant population canopy is constructed, so as to more accurately reflect the canopy spatial heterogeneity presented by the variety plant type and density differences. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1Schematic flowchart of the method for constructing a three-dimensional model of the maize plant population canopy provided by the present invention; Figure 2 Schematic structural diagram of the system for constructing a three-dimensional model of the maize plant population canopy provided by the present invention; Figure 3 Schematic structural diagram of the electronic device provided by the present invention. Detailed implementation manners

[0019] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the scope of protection of the present invention.

[0020] The plant type and population structure of maize crops are the core contents of maize crop cultivation, phenomics and plant functional-structure model research, and are crucial for plant type breeding and optimizing cultivation management measures. There are mainly three approaches to constructing existing three-dimensional models of maize plants and population canopies: One is the method based on a three-dimensional digitizer. By manually obtaining the three-dimensional coordinate point set of maize in the field, an in-situ three-dimensional model is constructed. This method has high precision and contains semantic information, but has low work efficiency and is difficult to apply on a large scale.

[0021] The second is the method based on assembled three-dimensional modeling. At the plant scale, first construct three-dimensional models of each organ, and then assemble them into a complete plant; at the population scale, construct a three-dimensional model of the population canopy by replicating plants and translating them according to the row and plant spacing. This method has a relatively high degree of freedom, but requires a large number of three-dimensional organ templates for support.

[0022] The third is the method based on three-dimensional point cloud reconstruction. Use multi-view images or lidar scanning to obtain the three-dimensional point cloud data of maize plants, and construct a three-dimensional model of maize plants through steps such as segmentation, skeleton extraction, mesh deformation or reconstruction. Currently, point cloud segmentation mainly relies on the combination of stem and leaf semantic segmentation and clustering algorithms, and most are only for stems and leaves, lacking four-class semantic segmentation of stems, leaves, tassels and ears. At the population canopy scale, although three-dimensional reconstruction can be carried out using unmanned aerial vehicles or lidar, it has requirements for the size and planting density of different types of planting plots, and it is difficult to solve the problem of missing point clouds inside the canopy or in the middle and lower parts.

[0023] The present invention mainly aims at the problems existing in the construction process of the three-dimensional models of existing maize plants and population canopies. By integrating the three-dimensional reconstruction at the plant scale and the construction of the three-dimensional model of the population canopy, it can reduce the dependence on the data of the population canopy, ensure the plant type characteristics within the population, and better reflect the spatial heterogeneity of the three-dimensional model of the population canopy due to differences such as variety, canopy density, and row spacing.

[0024] Figure 1 FIG. is a schematic flow chart of the method for constructing a three-dimensional model of a maize plant population canopy provided by the present invention. As Figure 1 shown, the present invention provides a method for constructing a three-dimensional model of a maize plant population canopy, including: Step 101, obtaining three-dimensional point cloud instance data of a target maize plant.

[0025] In the present invention, the target maize plant is a selected specific maize plant with specific variety, growth stage or phenotypic characteristics, which is the basis for subsequent analysis, modeling or research. The three-dimensional point cloud instance data is the point cloud data of the target maize plant obtained by three-dimensional scanning or measurement techniques. The point cloud data is a set composed of a large number of discrete three-dimensional points, and each point contains its coordinate information in the three-dimensional space (such as X, Y, Z coordinates), as well as possible other attribute information (such as color, reflectivity, etc.).

[0026] In order to obtain the three-dimensional point cloud instance data of the target maize plant, professional three-dimensional scanning equipment can be used, such as lidar (LiDAR), structured light scanners or stereo camera systems, etc. These devices can scan the target maize plant from multiple angles and capture the three-dimensional shape and details of the plant surface.

[0027] Step 102, determining a corresponding maize plant point cloud semantic segmentation model based on the growth stage information of the target maize plant, wherein the maize plant point cloud semantic segmentation model is trained based on the three-dimensional point cloud sample data of sample maize plants at different growth stages and the sample semantic labels corresponding to the three-dimensional point cloud sample data.

[0028] In the present invention, a large amount of three-dimensional point cloud data of sample corn plants at different growth stages is collected in the early stage. These sample three-dimensional point cloud data are obtained by scanning actual corn plants with three-dimensional scanning devices (such as lidar, structured light scanners, etc.). The point cloud data of each sample corn plant contains detailed shape and structure information of the plant in three-dimensional space. For the point cloud data of each sample corn plant, semantic labels also need to be assigned to it. The semantic label refers to classifying each point or each region in the point cloud data to indicate which organ of the plant they belong to (such as the stem, leaves, tassel, and ear, etc.). Then, by using the sample point cloud data and the corresponding semantic labels collected at each growth stage, the corn plant point cloud semantic segmentation models corresponding to each growth stage can be trained.

[0029] For target corn plants at different growth stages, corn plant point cloud semantic segmentation models at different growth stages need to be selected to achieve the best segmentation effect. The selected corn plant point cloud semantic segmentation model can perform semantic segmentation on the point cloud data of the target corn plant, so as to more accurately extract the information of each organ of the plant at the current growth stage.

[0030] Step 103: Input the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model to obtain the three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model.

[0031] In the present invention, the three-dimensional point cloud instance data of the target corn plant is used as the input and passed to the corn plant point cloud semantic segmentation model. The corn plant point cloud semantic segmentation model will analyze and judge each point or each region in the point cloud data by using the knowledge and rules learned in the previous training process. In this process, the corn plant point cloud semantic segmentation model will consider the shape, structure, context information, and possible other features of the point cloud data to determine which organ of the plant each point or region belongs to. After being processed by the corn plant point cloud semantic segmentation model, the three-dimensional point cloud semantic segmentation result is obtained.

[0032] The three-dimensional point cloud semantic segmentation result is a new point cloud data corresponding to the input point cloud data, but each point or region in it has been marked with semantic information, that is, which organ of the plant they belong to. For example, the points of the stem may be marked as green, the points of the leaves may be marked as blue, and the points of the tassel and ear may be marked as red and yellow respectively (the actual output may be digital labels or other forms). In the present invention, for the corn leaf point cloud segmentation result, since the leaves generally present a dense form, the obtained corn leaf point cloud segmentation result can actually be understood as a leaf group, and all the leaves have the same semantics (that is, the semantics is leaf group), and further instance segmentation processing needs to be performed on this segmentation result to obtain more accurate individual leaves.

[0033] Step 104: According to the three-dimensional point cloud semantic segmentation result, construct a maize organ mesh corresponding to each organ point cloud of the target maize plant, and construct a maize plant mesh model corresponding to the target maize plant based on each of the maize organ meshes; wherein, when the maize organ mesh is a maize leaf organ mesh, the maize leaf organ mesh is constructed based on leaf point cloud data, and the leaf point cloud data is obtained by instance segmentation of the three-dimensional semantic segmentation result that is the maize leaf point cloud segmentation result.

[0034] In the present invention, for each separated organ (such as a stalk, a leaf, a tassel, an ear, etc.), a three-dimensional organ mesh needs to be constructed according to the corresponding point cloud data. The organ mesh is a data structure representing a three-dimensional shape, which is composed of a series of vertices, edges, and faces and can accurately describe the three-dimensional morphology of the maize organ.

[0035] In the present invention, the process of constructing the maize organ mesh includes preprocessing of the point cloud data (such as denoising, simplification, etc.), generation of the mesh (such as by triangulation, etc.), and optimization of the mesh (such as smoothing processing, detail enhancement, etc.). For each organ of the target maize plant, an independent organ mesh will be obtained, and these organ meshes together constitute the complete three-dimensional morphology of the target maize plant.

[0036] After obtaining the organ meshes of each maize organ, they need to be integrated together to construct a complete maize plant mesh model. This process includes arranging and combining each organ mesh according to their actual positions on the plant, as well as dealing with the joints and overlapping parts between them. By integrating each organ mesh, a complete maize plant mesh model can be obtained, and this model can accurately represent the three-dimensional morphology and structure of the target maize plant, including the shapes, positions, and mutual relationships of its various organs.

[0037] Step 105: Based on the preset maize plant area parameters and the maize plant mesh model, construct a three-dimensional model of the maize plant population canopy.

[0038] In the present invention, in the process of constructing the three-dimensional model of the corn plant population canopy, it is necessary to combine the preset corn plant area parameters with the single corn plant grid model, that is, according to the information such as plant spacing, number of rows, and number of columns defined in the area parameters, reasonably arrange and layout the single corn plant grid model in three-dimensional space. By combining the area parameters and the grid model, the construction of the three-dimensional model of the corn plant population canopy can be started. This process includes copying the single corn plant grid model, adjusting their positions and orientations according to the area parameters, and dealing with the occlusion, overlap, and interaction relationships between them. In order to more realistically simulate the canopy structure of the corn plant population, the present invention can also consider factors such as growth competition between plants, mutual occlusion of leaves, light, and ventilation conditions.

[0039] In the construction process, it is necessary to optimize and adjust the model to ensure its accuracy and authenticity, including adjusting the plant spacing and arrangement, modifying the leaf angles and distributions, and adding or deleting certain organs or details, etc., so that the final three-dimensional model of the corn plant population canopy reflects the spatial heterogeneity presented due to differences in variety, density, row spacing, etc. while ensuring the plant type characteristics within the population.

[0040] The method for constructing the three-dimensional model of the corn plant population canopy provided by the present invention obtains the three-dimensional point cloud instance data of the target corn plant, selects the corresponding point cloud semantic segmentation model according to its growth stage, and then inputs the instance data into the model to obtain the three-dimensional point cloud semantic segmentation result. According to the segmentation result, the point cloud grid of each organ of the target corn plant is constructed, and further the grid model of the whole corn plant is constructed. Finally, combined with the preset plant area parameters and the grid model, the three-dimensional model of the corn plant population canopy is constructed, so as to more accurately reflect the canopy spatial heterogeneity presented by the variety plant type and density differences.

[0041] On the basis of the above embodiments, the corn plant point cloud semantic segmentation model is trained through the following steps: Obtain the first three-dimensional point cloud sample data, the second three-dimensional point cloud sample data, and the third three-dimensional point cloud sample data. Among them, the first three-dimensional point cloud sample data is the three-dimensional point cloud sample data of the sample corn plant at the growth stage from the seedling stage to the jointing stage; the second three-dimensional point cloud sample data is the three-dimensional point cloud sample data of the sample corn plant at the big flare stage; the third three-dimensional point cloud sample data is the three-dimensional point cloud sample data of the sample corn plant at the stage after silking is completed; Label the corresponding sample semantic tags for the corn stalk point cloud and the corn leaf point cloud in the first three-dimensional point cloud sample data to obtain the first training sample set; Label the corresponding sample semantic tags for the corn stalk point cloud and the corn leaf point cloud in the second three-dimensional point cloud sample data to obtain the second training sample set; Mark the corresponding sample semantic labels for the corn stalk point cloud, corn leaf point cloud, corn tassel point cloud, and corn ear point cloud in the third 3D point cloud sample data to obtain a third training sample set; Based on the first training sample set, train a deep learning model to obtain the corn plant point cloud semantic segmentation model for semantic segmentation of the corn plant point cloud at the seedling stage to jointing stage; Based on the second training sample set, train the deep learning model to obtain the corn plant point cloud semantic segmentation model for semantic segmentation of the corn plant point cloud at the large flare stage; Based on the third training sample set, train the deep learning model to obtain the corn plant point cloud semantic segmentation model for semantic segmentation of the corn plant point cloud at the stage after silking completion.

[0042] In the present invention, the first 3D point cloud sample data refers to the 3D point cloud data collected from a sample corn plant at the seedling stage to jointing stage. At this stage, the main characteristics of the corn plant are that the stalks are very short and mainly consist of leaves. Therefore, the 3D point cloud data at this stage mainly contains the point cloud information of the stalks and leaves of the corn plant.

[0043] The second 3D point cloud sample data refers to the 3D point cloud data collected from a sample corn plant at the large flare stage. At this stage, only the stalks and leaves are visible in the plant point cloud, and there are more upper leaves on the plant. Therefore, the 3D point cloud data at this stage mainly contains the point cloud information of more obvious stalks and dense leaves.

[0044] The third 3D point cloud sample data refers to the 3D point cloud data collected from a sample corn plant at the stage after silking completion. At this stage, in addition to the stalks and leaves, it also includes tassels and ears, and the spacing between adjacent leaves is relatively large, and the stalks are relatively clear. Therefore, the 3D point cloud data at this stage contains the complete point cloud information of the stalks, leaves, tassels, and ears.

[0045] Furthermore, perform semantic annotation on the corn stalk point cloud and corn leaf point cloud in the first 3D point cloud sample data. During the annotation process, assign corresponding sample semantic labels to the stalk and leaf point clouds. After the annotation is completed, a first training sample set is obtained for training the corn plant point cloud semantic segmentation model for the seedling stage to jointing stage.

[0046] Perform semantic annotation on the corn stalk point cloud and corn leaf point cloud in the second 3D point cloud sample data. Similarly, assign corresponding sample semantic labels to the stalk and leaf point clouds. After the annotation is completed, a second training sample set is obtained for training the corn plant point cloud semantic segmentation model for the large flare stage.

[0047] Semantically annotate the corn stalk point cloud, corn leaf point cloud, corn tassel point cloud, and corn ear point cloud in the third three-dimensional point cloud sample data, and assign corresponding sample semantic labels to the point clouds of each organ. After completing the annotation, a third training sample set is obtained for training a corn plant point cloud semantic segmentation model for the period after silking completion.

[0048] Furthermore, use the first training sample set to train the deep learning model. During the training process, the model learns how to identify and segment the stalk and leaf point clouds of corn plants from the seedling stage to the jointing stage. After training is completed, a corn plant point cloud semantic segmentation model is obtained for semantic segmentation of the point clouds of corn plants at the developmental stage from the seedling stage to the jointing stage.

[0049] Use the second training sample set to train the deep learning model. During the training process, the model learns how to identify and segment the stalk and leaf point clouds of corn plants at the large bell stage. After training is completed, a corn plant point cloud semantic segmentation model is obtained for semantic segmentation of the point clouds of corn plants at the developmental stage at the large bell stage.

[0050] Use the third training sample set to train the deep learning model. During the training process, the model learns how to identify and segment the stalks, leaves, tassels, and ear point clouds of corn plants in the period after silking completion. After training is completed, a corn plant point cloud semantic segmentation model is obtained for semantic segmentation of the point clouds of corn plants at the developmental stage in the period after silking completion.

[0051] In the present invention, based on the training sample sets constructed in the above embodiments, three sets of deep learning models for semantic segmentation of corn plant point clouds for different growth stages are respectively constructed. Among them, the deep learning model can be a model for point clouds, such as PointNet, PointNet++, PointTransformer V3, PlantNet, and PSegNet, etc.

[0052] Based on the above embodiments, after inputting the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model and obtaining the three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model, the method further includes: Obtain the maximum size information of the three-dimensional point cloud instance data in the three-dimensional rectangular coordinate system; Based on the maximum size information, perform size reduction processing on the three-dimensional point cloud semantic segmentation result to obtain a size-reduced three-dimensional point cloud semantic segmentation result, where the size of the size-reduced three-dimensional point cloud semantic segmentation result is the same as the size of the three-dimensional point cloud instance data; Traverse each point in the three-dimensional point cloud instance data, and determine the nearest point corresponding to each point in the three-dimensional point cloud instance data, where the nearest point is the point with the shortest distance between the three-dimensional point cloud semantic segmentation result after size reduction and the point in the three-dimensional point cloud instance data; Assign the semantic label of the nearest point to the corresponding point in the three-dimensional point cloud instance data. After determining that all points in the three-dimensional point cloud instance data have been traversed, obtain the target three-dimensional point cloud semantic segmentation result; The constructing the corn organ grid corresponding to each organ point cloud of the target corn plant according to the three-dimensional point cloud semantic segmentation result includes: Construct the corn organ grid corresponding to each organ point cloud of the target corn plant according to the target three-dimensional point cloud semantic segmentation result.

[0053] In the present invention, since the result after segmentation by the deep learning model has been downsampled and scaled, the semantic segmentation result needs to be processed accordingly before being returned to the original three-dimensional point cloud data (i.e., the three-dimensional point cloud instance data). The maximum size information refers to the size of the original three-dimensional point cloud data before scaling, or a reference size used to restore the scaled point cloud to the original size.

[0054] Specifically, before the original three-dimensional point cloud data is input into the corn plant point cloud semantic segmentation model, the present invention calculates the maximum size information of the original three-dimensional point cloud data on the three coordinate axes X, Y, and Z in the three-dimensional rectangular coordinate system, denoted as L box and then standardizes the original three-dimensional point cloud data according to L box to the range of [-1.0, 1.0] and inputs it into the corn plant point cloud semantic segmentation model.

[0055] Furthermore, for the obtained three-dimensional point cloud semantic segmentation result, first use L box to reversely restore it to the original plant size; then, according to the downsampled segmentation result, traverse each point in the original three-dimensional point cloud data, calculate and determine the nearest point between each point in the original three-dimensional point cloud data and the restored three-dimensional point cloud semantic segmentation result, and assign the semantic segmentation label of the nearest point to the point in the currently traversed original three-dimensional point cloud data. After the traversal is completed, semantic segmentation in the original three-dimensional point cloud data is achieved. In the present invention, the method for determining the nearest point can be to calculate the Euclidean distance or other distance metrics between two points and select the point with the minimum distance as the nearest point. Finally, by assigning the semantic label of the nearest point to the corresponding point in the instance data, a target three-dimensional point cloud semantic segmentation result with the same size as the original three-dimensional point cloud instance data and each point carrying the correct semantic label can be obtained.

[0056] Based on the above embodiments, the method further includes: Based on the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result, obtain the corn leaf point cloud segmentation result and the corn stalk point cloud segmentation result; Calculate the Euclidean distance between the leaf and stalk points between each point in the corn leaf point cloud segmentation result and the target point in the corn stalk point cloud segmentation result, where the target point is the point at the bottom of the corn plant stalk in the corn stalk point cloud segmentation result; Based on the improved QuickShift++ algorithm, perform leaf point cloud segmentation on the corn leaf point cloud segmentation result to obtain the leaf point cloud data of each leaf in the corn leaf point cloud segmentation result, so as to construct the corresponding corn leaf organ grid according to the leaf point cloud data, where the k-nearest neighbor radius in the improved QuickShift++ algorithm is calculated through the reciprocal of the Euclidean distance between the leaf and stalk points.

[0057] In the present invention, the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result refers to that the three-dimensional point cloud data has been subjected to semantic segmentation, that is, each point is assigned a specific semantic label, such as "leaf", "stalk", etc.

[0058] From the semantic segmentation result, all the points labeled as "leaf" can be extracted to form the corn leaf point cloud segmentation result. Similarly, all the points labeled as "stalk" can also be extracted to form the corn stalk point cloud segmentation result. Since the leaf point cloud is relatively dense, the corn leaf point cloud segmentation result reflects the semantic segmentation result of the corn leaf group points, and the semantic segmentation result of individual leaves needs to be further segmented.

[0059] Furthermore, calculate the Euclidean distance between the leaf and stalk points between each point in the corn leaf point cloud segmentation result and the target point in the corn stalk point cloud segmentation result. In the present invention, the target point is the point at the bottom of the corn plant stalk in the corn stalk point cloud segmentation result. Usually, this point is the lowest point of the stalk in the stalk segmentation result and can represent the growth point or the bottom of the plant.

[0060] For each point in the corn leaf point cloud segmentation result, calculate the Euclidean distance between it and the target point, that is, the Euclidean distance between the leaf and stalk points. The Euclidean distance is a measure of the straight-line distance between two points and can be calculated through the coordinates of two points in three-dimensional space. The present invention is based on the improved QuickShift++ algorithm to perform leaf point cloud segmentation on the corn leaf point cloud segmentation result to obtain the leaf point cloud data of each leaf in the corn leaf point cloud segmentation result.

[0061] In the present invention, in the improved QuickShift++ algorithm, the k-nearest neighbor radius is set to the reciprocal of the Euclidean distance between leaf-stem points, which means that for points closer to the bottom of the stem, their k-nearest neighbor radius is larger; for points farther away, the k-nearest neighbor radius is smaller. The improved QuickShift++ algorithm dynamically adjusts the k-nearest neighbor radius according to the reciprocal of the Euclidean distance between leaf-stem points, so as to more accurately segment the point cloud data of each leaf, and finally construct the organ grid of the leaf according to the segmentation result.

[0062] Based on the above embodiments, when it is determined that the development stage of the target corn plant is in the period after silking is completed, the method further includes: Obtain the distance between the lowest point in the leaf point cloud data and the target point in the corn stem point cloud segmentation result, and sort each piece of the leaf point cloud data in ascending order of the distance to obtain the sorting result of the leaf point cloud data; Based on the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result, determine the corn ear point cloud segmentation result; Based on the clustering algorithm, obtain the largest clustering point set in the corn ear point cloud segmentation result, and determine the ear leaf positioning point according to the heights of all points in the largest clustering point set, where the ear leaf positioning point represents the first quartile in the sorting result of the heights of all points in the largest clustering point set in ascending order; the heights of all points in the largest clustering point set represent the heights between the points in the largest clustering point set and the target point; Based on the sorting result of the leaf point cloud data, determine the first target leaf point cloud data and the second target leaf point cloud data, where the first target leaf point cloud data is determined based on the point in the leaf point cloud data that is closest to the ear leaf positioning point; the second target leaf point cloud data is determined based on the point in the leaf point cloud data that is the second closest to the ear leaf positioning point; According to the first target leaf point cloud data and the second target leaf point cloud data, determine the ear leaf point cloud data, where the ear leaf point cloud data is the first target leaf point cloud data or the second target leaf point cloud data with the smallest difference between the leaf azimuth angle and the ear azimuth angle; the ear azimuth angle is obtained based on the corn ear point cloud segmentation result.

[0063] In the present invention, the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result contains various parts of the corn plant, such as leaves, stems, ears, etc. When it is determined that the development stage of the target corn plant is in the period after silking is completed, the points marked as "ear" are extracted from the segmentation result to form the corn ear point cloud segmentation result.

[0064] The lowest point in the leaf point cloud data refers to the point with the lowest position in the vertical direction (usually the Z-axis) in each leaf point cloud. The target point is the plant growth point bottom in the corn stalk point cloud segmentation result, that is, the lowest point of the stalk.

[0065] For each leaf point cloud, calculate the Euclidean distance (or other suitable distance metric) between its lowest point and the target point. According to the calculated distances, sort all the leaf point cloud data from smallest to largest to obtain the sorted result of the leaf point cloud data.

[0066] In the present invention, apply a clustering algorithm, such as K-means, DBSCAN, etc., to the corn ear point cloud segmentation result to identify different parts or clusters of the ear. Then, select the cluster with the largest number of points from the clustering result as the set of maximum cluster points, and this set represents the main part of the ear.

[0067] Furthermore, calculate the heights of all points in the set of maximum cluster points (i.e., their positions in the vertical direction). Sort these heights in ascending order and select the first quartile (i.e., the point at the 25% position in the height sorting result) as the ear leaf positioning point. In the present invention, the heights of all points within the set of maximum cluster points represent the distances in the vertical direction between the points within the set of maximum cluster points and the target point, and are used to describe the relative position of the ear in the plant.

[0068] Furthermore, in the sorted result of the leaf point cloud data, find the two leaf point cloud data closest to the ear leaf positioning point, and use them as the first target leaf point cloud data and the second target leaf point cloud data respectively. Calculate the azimuth angles of the first target leaf point cloud data, the second target leaf point cloud data, and the ear (obtained based on the corn ear point cloud segmentation result), and then compare the differences between the azimuth angles of the first target leaf point cloud data and the second target leaf point cloud data respectively and the azimuth angle of the ear. Select the leaf point cloud data with the smallest difference as the ear leaf point cloud data, that is, it is considered that this leaf is closest to the orientation of the ear. In the present invention, the calculation of the ear azimuth angle depends on the corn ear point cloud segmentation result, and the orientation of the ear can be determined by analyzing the distribution, shape, or main direction of the corn ear point cloud.

[0069] Based on the above embodiments, constructing the corn organ grid corresponding to the point clouds of each organ of the target corn plant according to the three-dimensional point cloud semantic segmentation result includes: Perform voxel downsampling processing on the leaf point cloud data and / or the ear leaf point cloud data, and generate the corn leaf organ grid based on the leaf point cloud data after voxel downsampling processing; Construct a geometric model of multiple frustums based on the radius of each section of the corn stalk in the corn stalk point cloud segmentation result to obtain the official grid of the corn stalk; According to the 3D point cloud semantic segmentation result or the target 3D point cloud semantic segmentation result, obtain the corn tassel point cloud segmentation result, and construct a frustum grid model based on the tassel point cloud skeleton extracted from the corn tassel point cloud segmentation result to obtain the official grid of the corn tassel; Perform voxel downsampling on the corn ear point cloud segmentation result, and generate the official grid of the corn ear based on the corn ear point cloud segmentation result after voxel downsampling; The constructing the corn plant grid model corresponding to the target corn plant according to each of the official grids of the corn includes: Based on different preset development stages, select the corresponding official grids of the corn leaf, the corn stalk, the corn tassel, and the corn ear for combination to obtain the corn plant grid model corresponding to the target corn plant.

[0070] In the present invention, the Alpha shape algorithm is used to process the leaf point cloud data and / or the ear leaf point cloud data after voxel downsampling to generate a grid model of the leaf. Voxel downsampling is a point cloud simplification method that divides the point cloud data into a series of small cubes (voxels) and selects a representative point in each voxel to represent all the points within the voxel. Performing voxel downsampling on the leaf point cloud data (or ear leaf point cloud data) can reduce the amount of point cloud data and improve the efficiency of subsequent processing. The Alpha shape algorithm can generate a grid model that fits the shape of the point cloud according to the distribution and density of the point cloud data, thereby accurately representing the geometric shape of the leaf.

[0071] For the generation process of the official grid of the corn stalk, the present invention first processes the corn stalk point cloud segmentation result, divides the stalk into sections at a certain distance interval from bottom to top, and calculates the radius of each section of the stalk. Then, with the skeleton points of the stalk point cloud as the center and the calculated radii of multiple sections of the stalk as the radii, construct a geometric model of multiple frustums. The frustum model can accurately represent the thickness change and overall shape of the stalk. By combining the geometric models of multiple frustums, a complete official grid of the corn stalk is formed.

[0072] For the generation process of the grid of the maize tassel, the present invention extracts the point cloud data of the maize tassel from the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result. Then, the skeleton of the tassel point cloud data is extracted to obtain the skeleton point cloud of the tassel, which can represent the main structure and morphology of the tassel. Further, a radius gradient change from the base to the end is set, and a frustum grid model is constructed according to the tassel point cloud skeleton, which can accurately represent the shape and size change of the tassel. The frustum grid model generated through the above steps is the grid of the maize tassel. In the present invention, for the tassel point cloud with obvious compact branches, the Alpha shape algorithm can be directly used to generate the grid of the maize tassel.

[0073] For the generation process of the grid of the maize ear, the present invention first performs voxel downsampling on the segmentation result of the maize ear point cloud to reduce the quantity of the point cloud data. Then, the Alpha shape algorithm is used to process the ear point cloud data after voxel downsampling to generate the grid model of the ear.

[0074] Further, based on different preset development stages, the corresponding grids of maize leaves, maize stalks, maize tassels, and maize ears are selected. Different development stages may correspond to different organ morphologies and sizes, so it is necessary to select the corresponding organ grids for combination. For example, the grids of maize tassels and maize ears are only selected during the construction process of the grid model of the maize plant at the stage after silking. Finally, the selected maize organ grids are combined to form a complete grid model of the maize plant, which can accurately represent the overall morphology and structure of the target maize plant.

[0075] Based on the above embodiments, the construction of the three-dimensional model of the canopy of the maize plant population based on the preset maize plant area parameters and the maize plant grid model includes: According to the preset maize plant area parameters, a set of seed plants obtained by constructing the grid models of maize plants of different varieties and development stages is acquired, where the preset maize plant area parameters at least include maize variety, planting density, growth period information, planting row spacing, planting plant spacing, number of planting rows, number of plants per row, and planting row direction parameters; Based on the planting density within the maize plant area to be constructed, the plant azimuth plane angle of the target seed plant grid model is determined; where the maize plant area to be constructed is constructed based on the planting row spacing, the planting plant spacing, the number of planting rows, the number of plants per row, and the planting row direction parameters; the target seed plant grid model is a seed plant grid model randomly selected from the set of seed plants; According to the plant azimuth plane angle, rotate the target seed plant grid model around the Z-axis of the three-dimensional space coordinate system, and translate the rotated target seed plant grid model to the coordinate position of the corresponding planting point within the to-be-constructed maize plant area, where the coordinate position of the planting point is determined based on the planting row spacing and the planting plant spacing; After setting the target seed plant grid model at all the planting points within the to-be-constructed maize plant area, a to-be-completed three-dimensional model of the maize plant population canopy is obtained, and based on the planting row direction parameter, rotate the to-be-completed three-dimensional model of the maize plant population canopy around the Z-axis of the three-dimensional space coordinate system to obtain the three-dimensional model of the maize plant population canopy.

[0076] In the present invention, the preset maize plant area parameters at least include maize variety, planting density, growth period information, planting row spacing, planting plant spacing, number of planting rows, number of plants per row, and planting row direction parameter, and these parameters are used to define the basic characteristics and layout of the to-be-constructed maize plant population.

[0077] Further, according to the maize variety, planting density, and growth period information within the to-be-constructed maize plant area, obtain the corresponding three-dimensional point cloud data of the plants, and for the maize varieties in different planting plots, select the corresponding number of plants (for example, at least 3 plants per plot), and generate a maize plant grid model. These maize plant grid models serve as the seed plants (seed plants) of the current ecological point, variety, density (or other measures), and growth period, forming a set of seed plants, where each seed plant needs to be standardized, that is, its growth point is located at the origin, and the plant azimuth plane is rotated to 0°.

[0078] The to-be-constructed maize plant area is a virtual area constructed based on the planting row spacing, planting plant spacing, number of planting rows, number of plants per row, and planting row direction parameter, and is used to simulate the actual maize planting situation. In the present invention, according to the number of planting rows n row 、the number of plants per row n plant parameters, the number of plants within the maize population canopy can be determined n=n row ×n plant ; then, according to the planting row spacing d row and the planting plant spacing d plant , the growth x and y coordinates of each plant within the population can be determined. For example, the growth i and j coordinates of the plant in the x row and yThe coordinates (growing position coordinates) are: ( d row × ( i -1), d plant × ( j -1)). (It can also be extended to the wide-narrow row planting mode).

[0079] Further, traverse the n plants in the population canopy: Each time, randomly select a plant from all the seed plant sets (i.e., the target seed plant grid model), and then randomly generate its plant azimuth plane angle α , rotate the plant around the Z-axis of the three-dimensional space coordinate system by α , and then translate it to the growth position coordinates calculated in the above embodiment, that is, complete the model construction of the current plant in the population canopy. In the present invention, the method for randomly generating the plant azimuth plane angle α is: According to the population planting density, set the deflection threshold angle θ (0 < θ < 90°), the higher the planting density, the θ is smaller. In the present invention, θ <20°, α is a random number generated in the interval (- θ , θ ) and (180° - θ , 180° + θ ).

[0080] Finally, after setting the target seed plant grid model at all the planting points within the area of the corn plants to be constructed, obtain the three-dimensional model of the corn plant population canopy to be completed. Rotate the three-dimensional model of the corn plant population canopy to be completed as a whole around the Z-axis of the three-dimensional space coordinate system according to the row direction (i.e., the angle between the row and the due north direction) parameter, and complete the construction of the three-dimensional model of the corn population canopy.

[0081] The three-dimensional model of the corn population canopy constructed by the present invention through the above process contains clear organ grid semantic information, and can reflect the canopy spatial heterogeneity of the variety plant type and density differences, and can be used for further structural and functional analyses such as canopy light distribution calculation.

[0082] Next, the three-dimensional model construction system of the corn plant population canopy provided by the present invention will be described. The three-dimensional model construction system of the corn plant population canopy described below can be mutually corresponding and referred to the three-dimensional model construction method of the corn plant population canopy described above.

[0083] Figure 2 is the structural schematic diagram of the three-dimensional model construction system of the corn plant population canopy provided by the present invention, as shown in Figure 2As shown in the figure, the present invention provides a system for constructing a three-dimensional model of the canopy of a corn plant population, including a point cloud data acquisition module 201, a point cloud semantic segmentation model determination module 202, a point cloud semantic segmentation module 203, a corn plant model construction module 204, and a plant population canopy model construction module 205. Among them, the point cloud data acquisition module 201 is used to obtain the three-dimensional point cloud instance data of the target corn plant; the point cloud semantic segmentation model determination module 202 is used to determine the corresponding corn plant point cloud semantic segmentation model based on the growth stage information of the target corn plant, where the corn plant point cloud semantic segmentation model is trained based on the three-dimensional point cloud sample data of sample corn plants at different growth stages and the sample semantic labels corresponding to the three-dimensional point cloud sample data; the point cloud semantic segmentation module 203 is used to input the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model to obtain the three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model; the corn plant model construction module 204 is used to construct the corn organ mesh corresponding to each organ point cloud of the target corn plant according to the three-dimensional point cloud semantic segmentation result, and construct the corn plant mesh model corresponding to the target corn plant according to each corn organ mesh; among them, when the corn organ mesh is the corn leaf organ mesh, the corn leaf organ mesh is constructed based on the leaf point cloud data, and the leaf point cloud data is obtained by performing instance segmentation on the three-dimensional semantic segmentation result that is the corn leaf point cloud segmentation result; the plant population canopy model construction module 205 is used to construct a three-dimensional model of the corn plant population canopy based on the preset corn plant area parameters and the corn plant mesh model.

[0084] The system for constructing a three-dimensional model of the canopy of a corn plant population provided by the present invention obtains the three-dimensional point cloud instance data of the target corn plant, selects the corresponding point cloud semantic segmentation model according to its growth stage, then inputs the instance data into the model to obtain the three-dimensional point cloud semantic segmentation result, constructs the point cloud meshes of each organ of the target corn plant according to the segmentation result, and further constructs the mesh model of the whole corn plant. Finally, combined with the preset plant area parameters and the mesh model, a three-dimensional model of the corn plant population canopy is constructed, so as to more accurately reflect the canopy spatial heterogeneity presented by the variety plant type and density differences.

[0085] The system provided by the embodiments of the present invention is used to execute the above method embodiments. For the specific process and detailed content, please refer to the above embodiments and will not be elaborated here.

[0086] Figure 3 It is a schematic structural diagram of the electronic device provided by the present invention, as Figure 3As shown in the figure, the electronic device may include: a processor 301, a communications interface 302, a memory 303, and a communication bus 304. Among them, the processor 301, the communications interface 302, and the memory 303 complete communication with each other through the communication bus 304. The processor 301 may call the logical instructions in the memory 303 to execute a method for constructing a three-dimensional model of the canopy of a corn plant population. The method includes: obtaining three-dimensional point cloud instance data of a target corn plant; determining a corresponding corn plant point cloud semantic segmentation model based on the growth stage information of the target corn plant, where the corn plant point cloud semantic segmentation model is trained based on the three-dimensional point cloud sample data of sample corn plants at different growth stages and the sample semantic labels corresponding to the three-dimensional point cloud sample data; inputting the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model to obtain a three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model; constructing a corn organ mesh corresponding to each organ point cloud of the target corn plant according to the three-dimensional point cloud semantic segmentation result, and constructing a corn plant mesh model corresponding to the target corn plant according to each of the corn organ meshes; where when the corn organ mesh is a corn leaf organ mesh, the corn leaf organ mesh is constructed based on leaf point cloud data, and the leaf point cloud data is obtained by performing instance segmentation on the three-dimensional semantic segmentation result of the corn leaf point cloud segmentation result; constructing a three-dimensional model of the canopy of the corn plant population based on preset corn plant area parameters and the corn plant mesh model.

[0087] In addition, when the logical instructions in the above-mentioned memory 303 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0088] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the method for constructing a three-dimensional model of the canopy of a corn plant population provided by the above-mentioned various methods. The method includes: obtaining three-dimensional point cloud instance data of a target corn plant; determining a corresponding corn plant point cloud semantic segmentation model based on the growth stage information of the target corn plant, wherein the corn plant point cloud semantic segmentation model is trained based on three-dimensional point cloud sample data of sample corn plants at different growth stages and sample semantic labels corresponding to the three-dimensional point cloud sample data; inputting the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model to obtain a three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model; constructing a corn organ mesh corresponding to each organ point cloud of the target corn plant according to the three-dimensional point cloud semantic segmentation result, and constructing a corn plant mesh model corresponding to the target corn plant according to each corn organ mesh; wherein, when the corn organ mesh is a corn leaf organ mesh, the corn leaf organ mesh is constructed based on leaf point cloud data, and the leaf point cloud data is obtained by performing instance segmentation on the three-dimensional semantic segmentation result that is the corn leaf point cloud segmentation result; constructing a three-dimensional model of the canopy of the corn plant population based on preset corn plant area parameters and the corn plant mesh model.

[0089] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the method for constructing a three-dimensional model of a corn plant population canopy provided in the above embodiments. The method includes: obtaining three-dimensional point cloud instance data of a target corn plant; determining a corresponding corn plant point cloud semantic segmentation model based on the growth stage information of the target corn plant, where the corn plant point cloud semantic segmentation model is trained based on three-dimensional point cloud sample data of sample corn plants at different growth stages and sample semantic labels corresponding to the three-dimensional point cloud sample data; inputting the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model to obtain a three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model; constructing a corn organ mesh corresponding to each organ point cloud of the target corn plant according to the three-dimensional point cloud semantic segmentation result, and constructing a corn plant mesh model corresponding to the target corn plant according to each corn organ mesh; where when the corn organ mesh is a corn leaf organ mesh, the corn leaf organ mesh is constructed based on leaf point cloud data, and the leaf point cloud data is obtained by performing instance segmentation on the three-dimensional semantic segmentation result that is the corn leaf point cloud segmentation result; constructing a three-dimensional model of the corn plant population canopy based on preset corn plant area parameters and the corn plant mesh model.

[0090] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0091] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a three-dimensional model of the canopy of a corn plant population, characterized in that, Including: Obtaining three-dimensional point cloud instance data of a target corn plant; Based on the growth stage information of the target corn plant, determining a corresponding corn plant point cloud semantic segmentation model, wherein the corn plant point cloud semantic segmentation model is trained based on three-dimensional point cloud sample data of sample corn plants at different growth stages and sample semantic labels corresponding to the three-dimensional point cloud sample data; Inputting the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model to obtain a three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model; According to the three-dimensional point cloud semantic segmentation result, constructing a corn organ grid corresponding to each organ point cloud of the target corn plant, and constructing a corn plant grid model corresponding to the target corn plant according to each corn organ grid; wherein, when the corn organ grid is a corn leaf organ grid, the corn leaf organ grid is constructed based on leaf point cloud data, and the leaf point cloud data is obtained by performing instance segmentation on the three-dimensional semantic segmentation result of the corn leaf point cloud segmentation result; Based on preset corn plant area parameters and the corn plant grid model, constructing a three-dimensional model of the corn plant population canopy.

2. The method for constructing a three-dimensional model of the canopy of a corn plant population according to claim 1, wherein The corn plant point cloud semantic segmentation model is trained through the following steps: Obtaining first three-dimensional point cloud sample data, second three-dimensional point cloud sample data, and third three-dimensional point cloud sample data, wherein the first three-dimensional point cloud sample data is three-dimensional point cloud sample data of the sample corn plant at the growth stage from seedling stage to jointing stage; the second three-dimensional point cloud sample data is three-dimensional point cloud sample data of the sample corn plant at the large flare stage; the third three-dimensional point cloud sample data is three-dimensional point cloud sample data of the sample corn plant at the stage after silking is completed; Marking the corresponding sample semantic labels for the corn stalk point cloud and corn leaf point cloud in the first three-dimensional point cloud sample data to obtain a first training sample set; Marking the corresponding sample semantic labels for the corn stalk point cloud and corn leaf point cloud in the second three-dimensional point cloud sample data to obtain a second training sample set; Marking the corresponding sample semantic labels for the corn stalk point cloud, corn leaf point cloud, corn tassel point cloud, and corn ear point cloud in the third three-dimensional point cloud sample data to obtain a third training sample set; Based on the first training sample set, training a deep learning model to obtain the corn plant point cloud semantic segmentation model for semantic segmentation of the corn plant point cloud at the growth stage from seedling stage to jointing stage; Based on the second training sample set, training the deep learning model to obtain the corn plant point cloud semantic segmentation model for semantic segmentation of the corn plant point cloud at the large flare stage; Based on the third training sample set, training the deep learning model to obtain the corn plant point cloud semantic segmentation model for semantic segmentation of the corn plant point cloud at the stage after silking is completed.

3. The method for constructing a three-dimensional model of the corn plant population canopy according to claim 2, wherein After inputting the three-dimensional point cloud instance data into the corn plant point cloud semantic segmentation model and obtaining the three-dimensional point cloud semantic segmentation result output by the target corn plant point cloud semantic segmentation model, the method further includes: Obtain the maximum dimension information of the three-dimensional point cloud instance data in a three-dimensional rectangular coordinate system; Based on the maximum dimension information, perform dimension restoration processing on the three-dimensional point cloud semantic segmentation result to obtain a dimension-restored three-dimensional point cloud semantic segmentation result, where the dimension-restored three-dimensional point cloud semantic segmentation result has the same size as the three-dimensional point cloud instance data; Traverse each point in the three-dimensional point cloud instance data and determine the nearest point corresponding to each point in the three-dimensional point cloud instance data, where the nearest point is the point with the shortest distance between the dimension-restored three-dimensional point cloud semantic segmentation result and the point in the three-dimensional point cloud instance data; Assign the semantic label of the nearest point to the corresponding point in the three-dimensional point cloud instance data. After determining that all points in the three-dimensional point cloud instance data have been traversed, obtain the target three-dimensional point cloud semantic segmentation result; The constructing the corn organ grid corresponding to each organ point cloud of the target corn plant according to the three-dimensional point cloud semantic segmentation result includes: Construct the corn organ grid corresponding to each organ point cloud of the target corn plant according to the target three-dimensional point cloud semantic segmentation result.

4. The method for constructing a three-dimensional model of the corn plant population canopy according to claim 3, wherein The method further includes: Based on the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result, obtain the corn leaf point cloud segmentation result and the corn stalk point cloud segmentation result; Calculate the Euclidean distance between the leaf and stalk points between each point in the corn leaf point cloud segmentation result and the target point in the corn stalk point cloud segmentation result, where the target point is the point at the bottom of the corn plant stalk in the corn stalk point cloud segmentation result; Based on the improved QuickShift++ algorithm, perform leaf point cloud segmentation on the corn leaf point cloud segmentation result to obtain each leaf point cloud data in the corn leaf point cloud segmentation result, so as to construct the corresponding corn leaf organ grid according to the leaf point cloud data, where the k-nearest neighbor radius in the improved QuickShift++ algorithm is calculated by the reciprocal of the Euclidean distance between the leaf and stalk points; 5. The method for constructing a three-dimensional model of the corn plant population canopy according to claim 4, wherein When it is determined that the development stage of the target corn plant is in the period after silking is completed, the method further includes: Obtain the distance between the lowest point in the leaf point cloud data and the target point in the corn stalk point cloud segmentation result, and sort each leaf point cloud data in ascending order of the distance to obtain the sorting result of the leaf point cloud data; Based on the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result, determine the corn ear point cloud segmentation result; Based on the clustering algorithm, obtain the largest cluster point set in the segmentation result of the maize ear point cloud, and determine the ear leaf positioning point according to the heights of all points in the largest cluster point set, where the ear leaf positioning point represents the first quartile in the sorting result of the heights of all points in the largest cluster point set from low to high; the heights of all points in the largest cluster point set represent the heights between the points in the largest cluster point set and the target point; Based on the sorting result of the leaf point cloud data, determine the first target leaf point cloud data and the second target leaf point cloud data, where the first target leaf point cloud data is determined based on the point closest to the ear leaf positioning point in the leaf point cloud data; the second target leaf point cloud data is determined based on the point with the second-closest distance to the ear leaf positioning point in the leaf point cloud data; According to the first target leaf point cloud data and the second target leaf point cloud data, determine the ear leaf point cloud data, where the ear leaf point cloud data is the first target leaf point cloud data or the second target leaf point cloud data with the smallest difference between the leaf azimuth angle and the ear azimuth angle; the ear azimuth angle is obtained based on the segmentation result of the maize ear point cloud; 6. The method for constructing a three-dimensional model of the corn plant population canopy according to claim 5, characterized in that, The construction of the maize organ grid corresponding to each organ point cloud of the target maize plant according to the three-dimensional point cloud semantic segmentation result includes: Perform voxel downsampling on the leaf point cloud data and / or the ear leaf point cloud data, and generate the maize leaf organ grid based on the leaf point cloud data after voxel downsampling; According to the radius of each section of the stem in the maize stem point cloud segmentation result, construct a multi-section frustum geometric model to obtain the maize stem organ grid; According to the three-dimensional point cloud semantic segmentation result or the target three-dimensional point cloud semantic segmentation result, obtain the maize tassel point cloud segmentation result, and construct a frustum grid model based on the tassel point cloud skeleton extracted from the maize tassel point cloud segmentation result to obtain the maize tassel organ grid; Perform voxel downsampling on the maize ear point cloud segmentation result, and generate the maize ear organ grid based on the maize ear point cloud segmentation result after voxel downsampling; The construction of the maize plant grid model corresponding to the target maize plant according to each of the maize organ grids includes: Based on different preset development stages, select the corresponding maize leaf organ grid, maize stem organ grid, maize tassel organ grid, and maize ear organ grid for combination to obtain the maize plant grid model corresponding to the target maize plant.

7. The method for constructing a three-dimensional model of the canopy of a corn plant population according to claim 5, characterized in that, The construction of the three-dimensional model of the maize plant population canopy based on the preset maize plant area parameters and the maize plant grid model includes: Obtain a set of seed plant models constructed from the grid models of maize plants of different maize varieties and development stages according to the preset maize plant area parameters, where the preset maize plant area parameters at least include maize variety, planting density, growth period information, planting row spacing, planting plant spacing, number of planting rows, number of plants per row, and planting row direction parameters; Based on the planting density within the maize plant area to be constructed, determine the plant azimuth plane angle of the target seed plant grid model; wherein, the maize plant area to be constructed is constructed based on the planting row spacing, the planting plant spacing, the number of planting rows, the number of plants per row, and the planting row direction parameters; the target seed plant grid model is a randomly selected seed plant grid model from the set of seed plant models; According to the plant azimuth plane angle, rotate the target seed plant grid model along the Z-axis of the three-dimensional space coordinate system, and translate the rotated target seed plant grid model to the coordinate position of the corresponding planting point within the maize plant area to be constructed, where the coordinate position of the planting point is determined based on the planting row spacing and the planting plant spacing; After determining that the target seed plant grid model is set at all planting points within the maize plant area to be constructed, obtain the three-dimensional model of the maize plant population canopy to be completed, and rotate the three-dimensional model of the maize plant population canopy to be completed along the Z-axis of the three-dimensional space coordinate system based on the planting row direction parameters to obtain the three-dimensional model of the maize plant population canopy.

8. A three-dimensional model construction system for the canopy of a maize plant population, characterized in that, Including: A point cloud data acquisition module for obtaining three-dimensional point cloud instance data of a target maize plant; A point cloud semantic segmentation model determination module for determining a corresponding maize plant point cloud semantic segmentation model based on the growth stage information of the target maize plant, where the maize plant point cloud semantic segmentation model is trained based on the three-dimensional point cloud sample data of sample maize plants at different growth stages and the sample semantic labels corresponding to the three-dimensional point cloud sample data; A point cloud semantic segmentation module for inputting the three-dimensional point cloud instance data into the maize plant point cloud semantic segmentation model to obtain a three-dimensional point cloud semantic segmentation result output by the target maize plant point cloud semantic segmentation model; A maize plant model construction module for constructing maize organ grids corresponding to the point clouds of each organ of the target maize plant according to the three-dimensional point cloud semantic segmentation result, and constructing a maize plant grid model corresponding to the target maize plant according to each maize organ grid; wherein, when the maize organ grid is a maize leaf organ grid, the maize leaf organ grid is constructed based on leaf point cloud data, and the leaf point cloud data is obtained by performing instance segmentation on the three-dimensional semantic segmentation result that is the maize leaf point cloud segmentation result; A plant population canopy model construction module for constructing a three-dimensional model of the maize plant population canopy based on the preset maize plant area parameters and the maize plant grid model.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method for constructing a three-dimensional model of a maize plant population canopy according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for constructing a three-dimensional model of the canopy of a corn plant population according to any one of claims 1 to 7.

Images