Plant leaf skeleton extraction method, system and device

Abstract

The present application provides a plant leaf skeleton extraction method, system and device, the method comprising: obtaining a plant point cloud and preprocessing and point cloud segmentation to obtain a segmented plant point cloud; constructing a three-dimensional point cloud coordinate system, projecting the segmented plant point cloud to the three-dimensional point cloud coordinate system to obtain point cloud projection results under different viewing angles, and obtaining point cloud projection results under remaining viewing angles through point cloud rejection; respectively performing skeleton extraction and inverse mapping on the point cloud projection results under the remaining viewing angles to obtain a three-dimensional skeleton point set, and obtaining a dense skeleton point set through skeleton extension; projecting the dense skeleton point set under the remaining viewing angles and obtaining a skeleton point set through point cloud screening, and then obtaining leaf pillow points, and performing segmentation through the leaf pillow points to obtain a leaf skeleton and a leaf point cloud. The present application solves the problem of skeleton extraction error caused by different forms due to different growth periods and insect pests of leaves, and realizes accurate extraction of the leaf skeleton.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a method, system, and apparatus for extracting the skeleton of plant leaves. Background Technology

[0002] In the field of agricultural production, the extraction of point cloud skeletons of plant leaves is one of the important means of plant leaf observation. It is of great significance for scientific research and breeding, gene sequencing and phenotypic research. Naturally grown plant leaves may have differences in size and irregular leaf curling. The manual measurement method has problems of operation difficulty and subjective error. With the development and application of machine vision in the field of smart agriculture, the defects of manual measurement can be made up to a certain extent.

[0003] Existing machine vision-based skeleton extraction methods include: 1. Extracting the skeleton using a preset leaf shape; 2. Using an isotropic shrinkage method, shrinking the skeleton in all directions using a sphere with a neighborhood radius; 3. Using the L1 median skeleton extraction method to address the issue of noise affecting the extraction effect; 4. Smoothing and shrinking the point cloud using the Laplacian operator, causing surface points to gradually shrink inwards and eventually converge to the skeleton position. In real-world scenarios, plant leaves exhibit varying degrees of curling at different growth stages. If pests or diseases are present, the plant leaves may be incomplete, affecting the calculation of the plant leaf skeleton position. Existing methods have the following problems: 1. They are limited to a certain leaf shape and cannot accommodate leaves of different shapes; 2. They cannot handle scenarios with missing or unevenly dense point clouds. Summary of the Invention

[0004] This invention addresses the shortcomings of existing leaf skeleton extraction methods, which are limited to certain leaf morphologies and cannot accommodate different point cloud scenarios. It proposes a method, system, and device for extracting plant leaf skeletons.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A method for extracting the skeleton of a plant leaf includes the following steps: Acquire plant point clouds and perform preprocessing and point cloud segmentation to obtain segmented plant point clouds; Obtain several bounding rectangles of the segmented plant point cloud, construct a three-dimensional point cloud coordinate system based on the bounding rectangles, obtain the point cloud projection results of the segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtain the point cloud projection results under the remaining viewpoints by removing the point cloud. Skeleton extraction and inverse mapping are performed on the point cloud projection results under the remaining viewpoints to obtain a 3D skeleton point set. The 3D skeleton point set is then extended to obtain a dense skeleton point set. The dense skeleton point set is projected under the remaining view and the skeleton point set is obtained by filtering the point cloud. The leaf pulvinus points are obtained based on the skeleton point set. The dense skeleton point set and the segmented plant point cloud are segmented by the leaf pulvinus points to obtain the leaf skeleton and leaf point cloud.

[0007] As one possible implementation, the process of acquiring plant point clouds, performing preprocessing and point cloud segmentation to obtain segmented plant point clouds includes the following steps: Obtain the point cloud of the entire plant and perform preprocessing to obtain the preprocessed plant point cloud; The preprocessed plant point cloud is segmented using a point cloud segmentation method to obtain an initial plant point cloud. The point cloud segmentation method includes a method for point cloud segmentation based on region normal vectors. Obtain the bounding rectangle of the initial plant point cloud. Search according to the preset search radius. Add the preprocessed plant point cloud that is different from the initial plant point cloud within the bounding rectangle to the initial plant point cloud until the number of initial plant point clouds and preprocessed plant point clouds within the bounding rectangle are the same, and obtain the segmented plant point cloud.

[0008] As one possible implementation, the process of obtaining several bounding rectangles of the segmented plant point cloud, constructing a three-dimensional point cloud coordinate system based on the bounding rectangles, obtaining the point cloud projection results of the segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtaining the point cloud projection results under the remaining viewpoints through point cloud culling includes the following steps: Several bounding rectangles of the segmented plant point cloud are obtained respectively. A three-dimensional point cloud coordinate system is established with a vertex of the bounding rectangle as the origin. The segmented plant point cloud in the current bounding rectangle is projected to different viewpoints of the three-dimensional point cloud coordinate system to obtain the point cloud projection results under different viewpoints. Based on the number of points and the size of the plant point cloud in the point cloud projection results, the effective pixel ratio of the point cloud projection results under different viewpoints is obtained. Based on the effective pixel ratio, point cloud culling is performed on the point cloud projection results under different viewpoints to obtain the point cloud projection results of the remaining viewpoints after point cloud culling.

[0009] As one possible implementation, the process of extracting and inversely mapping the point cloud projection results from the remaining viewpoints to obtain a 3D skeleton point set includes the following steps: The initial number of clusters is preset, and the point cloud projection results of the remaining views are clustered according to the initial number of clusters to obtain the corresponding point cloud clustering results. The point cloud clustering results include the cluster centers and the corresponding cluster point clouds. The cluster centers in the point cloud clustering results are sorted using a sorting algorithm to minimize the path after sorting, thereby obtaining the corresponding two-dimensional skeleton result of the point cloud projection result under the remaining view. The sorting algorithm includes the ant colony algorithm. The two-dimensional skeleton result is back-mapped to three-dimensional space to obtain a three-dimensional skeleton point set.

[0010] As one possible implementation, the method of extending the three-dimensional skeleton point set to obtain a dense skeleton point set includes the following steps: Denoise the 3D skeleton point set to obtain the denoised 3D skeleton point set; The skeleton endpoints in the three-dimensional skeleton point set are obtained from the cluster centers in the point cloud clustering results. The skeleton endpoints include the skeleton start point, the start point adjacent points, the skeleton end point, and the end point adjacent points. Based on the skeleton endpoints, construct endpoint vectors, and project the cluster point clouds corresponding to the skeleton start point and skeleton end point onto the corresponding endpoint vectors to obtain the cluster point cloud projection point set; Calculate the distance between the projected points of the cluster point cloud and the endpoints of the skeleton, obtain the skeleton extension points through the distance, and add the skeleton extension points to the 3D skeleton point set to obtain the extended skeleton point set; Based on a preset distance threshold, the extended skeleton point set is interpolated to obtain a dense skeleton point set.

[0011] As one possible implementation, the step of projecting a dense skeleton point set onto the remaining viewpoint and obtaining a skeleton point set through point cloud filtering includes the following steps: Obtain the center position of the plant point cloud, project the dense skeleton point set under the remaining view to obtain the projected point cloud under the remaining view, and calculate the sum of the center distances between the first and last projected points and the center position in the projected point cloud. The projection point cloud under the remaining viewpoints is filtered based on the sum of the corresponding center distances under the remaining viewpoints to obtain the skeleton point set.

[0012] As one possible implementation, the step of obtaining leaf pulvinus points based on the skeleton point set, and then segmenting the dense skeleton point set and the segmented plant point cloud using the leaf pulvinus points to obtain the leaf skeleton and leaf point cloud, includes the following steps: Based on the plant point cloud, obtain the point cloud width value of the skeleton point position in the skeleton point set; If the point cloud width value meets the preset leaf pillow segmentation condition, then the corresponding skeleton point position is the leaf pillow point; Based on the location of the leaf pulvinus point, the dense skeleton point set and segmented plant point cloud are vertically segmented to obtain the corresponding leaf skeleton and leaf point cloud.

[0013] A plant leaf skeleton extraction system, comprising: Point cloud processing module: used to acquire plant point clouds and perform preprocessing and point cloud segmentation to obtain segmented plant point clouds; Point cloud projection module: used to obtain several bounding rectangles of segmented plant point cloud, construct a three-dimensional point cloud coordinate system based on the bounding rectangles, obtain the point cloud projection results of segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtain the point cloud projection results under the remaining viewpoints by point cloud culling. Point cloud mapping module: used to extract and inverse map the point cloud projection results under the remaining viewpoints to obtain a 3D skeleton point set, and extend the 3D skeleton point set to obtain a dense skeleton point set; The filtering and segmentation module is used to project the dense skeleton point set under the remaining view and obtain the skeleton point set through point cloud filtering. Based on the skeleton point set, leaf pulvinus points are obtained. The dense skeleton point set and the segmented plant point cloud are segmented by leaf pulvinus points to obtain the leaf skeleton and leaf point cloud.

[0014] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of the following ways: Acquire plant point clouds and perform preprocessing and point cloud segmentation to obtain segmented plant point clouds; Obtain several bounding rectangles of the segmented plant point cloud, construct a three-dimensional point cloud coordinate system based on the bounding rectangles, obtain the point cloud projection results of the segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtain the point cloud projection results under the remaining viewpoints by removing the point cloud. Skeleton extraction and inverse mapping are performed on the point cloud projection results under the remaining viewpoints to obtain a 3D skeleton point set. The 3D skeleton point set is then extended to obtain a dense skeleton point set. The dense skeleton point set is projected under the remaining view and the skeleton point set is obtained by filtering the point cloud. The leaf pulvinus points are obtained based on the skeleton point set. The dense skeleton point set and the segmented plant point cloud are segmented by the leaf pulvinus points to obtain the leaf skeleton and leaf point cloud.

[0015] A plant leaf skeleton extraction device includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the method described in any one of the following ways: Acquire plant point clouds and perform preprocessing and point cloud segmentation to obtain segmented plant point clouds; Obtain several bounding rectangles of the segmented plant point cloud, construct a three-dimensional point cloud coordinate system based on the bounding rectangles, obtain the point cloud projection results of the segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtain the point cloud projection results under the remaining viewpoints by removing the point cloud. Skeleton extraction and inverse mapping are performed on the point cloud projection results under the remaining viewpoints to obtain a 3D skeleton point set. The 3D skeleton point set is then extended to obtain a dense skeleton point set. The dense skeleton point set is projected under the remaining view and the skeleton point set is obtained by filtering the point cloud. The leaf pulvinus points are obtained based on the skeleton point set. The dense skeleton point set and the segmented plant point cloud are segmented by the leaf pulvinus points to obtain the leaf skeleton and leaf point cloud.

[0016] This invention, by adopting the above technical solutions, has significant technical effects: The method of this invention projects the segmented point cloud of a plant from different perspectives, removes the remaining point cloud projections, and then maps them back to three-dimensional space to obtain a three-dimensional skeleton point set. Further filtering based on the remaining perspectives yields the skeleton point set. Leaf pulvinus points are obtained from the point cloud width, and segmentation is performed to obtain the leaf skeleton and leaf point cloud. This method solves the problem of morphological and incomplete morphology issues caused by different plant growth stages and pest / disease conditions, which affect the effectiveness of leaf skeleton extraction in existing methods. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the method of the present invention; Figure 2 This is a schematic diagram of the modules of the system of the present invention; Figure 3 This is a schematic diagram of plant point clouds according to the present invention; Figure 4 This is a schematic diagram of the initial plant point cloud of the present invention; Figure 5 This is a schematic diagram of the segmented plant point cloud of the present invention; Figure 6 This is a schematic diagram of the construction of the three-dimensional point cloud coordinate system of the present invention; Figure 7 These are schematic diagrams of point cloud projection from different perspectives according to the present invention; Figure 8 This is a schematic diagram of the remaining view frame of the present invention; Figure 9 This is a schematic diagram of the three-dimensional skeleton point set expansion method of the present invention; Figure 10 This is a schematic diagram of the dense skeleton point set of the present invention; Figure 11 This is a schematic diagram of the leaf bolster point of the present invention; Figure 12This is a schematic diagram illustrating the leaf skeleton extraction effect of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] Example 1: A method for extracting plant leaf skeletons, such as Figure 1 As shown, it includes the following steps: S100: Acquire plant point clouds and perform preprocessing and point cloud segmentation to obtain segmented plant point clouds; S200. Obtain several bounding rectangles of the segmented plant point cloud, construct a three-dimensional point cloud coordinate system based on the bounding rectangles, obtain the point cloud projection results of the segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtain the point cloud projection results under the remaining viewpoints by removing the point cloud. S300: Extract the skeleton and inverse map the point cloud projection results under the remaining viewpoints to obtain a three-dimensional skeleton point set. Extend the skeleton of the three-dimensional skeleton point set to obtain a dense skeleton point set. S400. Project the dense skeleton point set under the remaining view and obtain the skeleton point set by filtering the point cloud. Obtain the leaf pulvinus point based on the skeleton point set. Use the leaf pulvinus point to segment the dense skeleton point set and the segmented plant point cloud to obtain the leaf skeleton and leaf point cloud.

[0024] This embodiment takes the extraction of the leaf skeleton of *Chicken Feather Spinach* as an example. At the same time, the method of this application can be applied to other plants and has good scalability. By preprocessing and segmenting the point cloud of the whole plant, multi-view projection is performed based on the segmented plant point cloud, and the skeleton is extracted and filtered based on the projection results. The multi-view projection results are analyzed and filtered by the method of this application, thereby accurately locating the leaf pulvinus point and realizing accurate segmentation of the leaf point cloud and leaf skeleton.

[0025] S100. Obtain plant point clouds and perform preprocessing and point cloud segmentation to obtain segmented plant point clouds, including the following steps: Step S1001: In this embodiment, 3DGS technology is used to reconstruct the point cloud of the plant from all directions, obtaining the point cloud of the entire plant, such as... Figure 3 As shown; Step S1002: Preprocess the point cloud of the whole plant to obtain a preprocessed plant point cloud. In this embodiment, the preprocessing method includes one or more of denoising, point cloud smoothing and downsampling. Step S1003: In this embodiment, the preprocessed plant point cloud is initially segmented using the point cloud segmentation method based on the region normal vector to obtain the initial plant point cloud. However, due to the problem that the normal vector gradient changes too much in some areas of the leaf point cloud, there are missing segments in the leaf segmentation. Step S1004: To address the missing information in step S1003, first obtain the bounding rectangles of different leaves from the initial plant point cloud, such as... Figure 4 As shown, the preprocessed plant point cloud within the bounding rectangle is extracted according to the position information of the bounding rectangle. According to the preset search radius, the preprocessed segmented point cloud within the bounding rectangle is traversed, and points that are different from the initial plant point cloud are added to the initial plant point cloud until the number of the initial plant point cloud and the preprocessed plant point cloud within the bounding rectangle are the same, thus obtaining the segmented plant point cloud.

[0026] S200. Obtain several bounding rectangles of the segmented plant point cloud, construct a three-dimensional point cloud coordinate system based on the bounding rectangles, obtain the point cloud projection results of the segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtain the point cloud projection results under the remaining viewpoints by point cloud culling, including the following steps: Step S2001: Obtain the bounding rectangles of different leaves in the segmented plant point cloud, such as... Figure 5 As shown, taking one leaf as an example, a three-dimensional point cloud coordinate system is established with a vertex of the leaf's bounding rectangle as the origin, as follows: Figure 6 As shown; Step S2002: Project the segmented plant point cloud within the current bounding rectangle to different viewpoints in the 3D point cloud coordinate system to obtain the point cloud projection results from different viewpoints, such as... Figure 7 As shown, the different perspectives in this embodiment include Perspective, Perspective and Perspective; Step S2003: Calculate the effective pixel ratio within the point cloud projection results from different viewpoints, wherein the effective pixel ratio is:

[0027] Step S2004: Remove the point cloud projection results from the corresponding viewpoint with the largest effective pixel ratio to obtain the point cloud projection results from the remaining viewpoints. In this embodiment... The effective pixel ratio for the viewing angle is 0.61. The effective pixel ratio for the viewing angle is 0.23. The effective pixel ratio of the viewing angle is 0.57, and the viewing angle with the largest effective pixel ratio is... The remaining viewpoint after point cloud culling is Perspective and Perspective.

[0028] S300. The point cloud projection results under the remaining viewpoints are subjected to skeleton extraction and inverse mapping to obtain a 3D skeleton point set. The 3D skeleton point set is then extended to obtain a dense skeleton point set, including the following steps: Step S3001: Based on the preset cluster coefficients and image size, obtain the initial number of clusters. Based on the initial number of clusters, cluster the point cloud projection results under the remaining viewpoints to obtain the corresponding point cloud clustering results. The clustering algorithm includes the K-means algorithm, and the point cloud clustering results include cluster centers and corresponding clustered point clouds. Step S3002: Sort the cluster centers in the point cloud clustering results using a sorting algorithm, and select the sorted results with the shortest paths to form the corresponding two-dimensional skeleton results of the point cloud clustering results under the remaining viewpoints, such as... Figure 8 As shown; Step S3003: Map the two-dimensional skeleton result back to the three-dimensional space of the three-dimensional point cloud coordinate system to obtain the three-dimensional skeleton point set, and perform smoothing processing on the three-dimensional skeleton point set. The smoothing processing method includes median filtering. Step S3004: Since the obtained 3D skeleton point set originates from the center of the cluster obtained by clustering and is not located at the boundary of the plant's point cloud, this embodiment extends the 3D skeleton point set. First, the skeleton endpoints of the 3D skeleton point set are obtained, such as... Figure 9 As shown, the skeleton endpoints in this embodiment include the skeleton starting point. Adjacent points of the starting point skeletal endpoint and adjacent points to the endpoint ; Step S3005: Construct endpoint vectors , ,like Figure 9 The middle arrow points to the starting point of the skeleton. and skeletal endpoint The corresponding cluster point cloud is projected onto the endpoint vector. , The direction yields the projected point set of the clustered point cloud. ; Step S3006: Calculate the projection points of the cluster point cloud in the cluster point cloud projection point set respectively. The distance between the skeleton endpoints, with the skeleton end point and adjacent points to the endpoint For example, the distance is:

[0029] Step S3007: Obtain the cluster point cloud projection point corresponding to the maximum distance. Take the point with the largest distance between the cluster point cloud projection points corresponding to the maximum distance as the skeleton extension point endpoint2. Obtain the skeleton extension point endpoint1 through the above method. Perform back projection on the skeleton extension point and add it to the three-dimensional skeleton point set to obtain the extended skeleton point set. Step 3008: Calculate the length distance between adjacent 3D skeleton points in the 3D skeleton point set. If the length distance is less than a preset distance threshold, no skeleton points need to be added between the current adjacent 3D skeleton points. Otherwise, obtain the midpoint of the adjacent 3D skeleton points and add the plant point cloud with the closest midpoint to the 3D skeleton point set. Through iterative processing, until the length distance between all adjacent 3D skeleton points is less than the preset distance threshold, a dense skeleton point set is obtained. Figure 10 As shown; in, Represents the first point in the projection point set of the clustered point cloud. The distance between the projection points of each cluster point cloud and the endpoints of the skeleton. Indicates the first Projected points of each cluster point cloud.

[0030] S400. Project the dense skeleton point set under the remaining view and obtain the skeleton point set through point cloud filtering. Obtain the leaf pulvinus points based on the skeleton point set. Use the leaf pulvinus points to segment the dense skeleton point set and the segmented plant point cloud to obtain the leaf skeleton and leaf point cloud. This includes the following steps: Step S4001: Project the dense skeleton point set into the remaining viewpoints respectively. In this embodiment, the dense skeleton point cloud is projected according to the three-dimensional point cloud coordinate system. Perspective and Projecting the viewpoints yields the projected point cloud for the remaining viewpoints; Step S4002: Obtain the center position of the plant point cloud, and calculate the distances between the first and last projection points of the projected point cloud under the remaining view and the center position, to obtain the center distance sum, which is:

[0031] Step S4003: Select the projected point cloud with the maximum center distance and the corresponding viewing angle as the skeleton point set. In this embodiment, select... The projected point cloud from the viewpoint serves as the skeleton point set; Step S4004: Based on the segmented plant point cloud, obtain the point cloud width value corresponding to the position of each skeleton point in the skeleton point set. If the point cloud width value satisfies... or If one of them is true, then the current skeleton point is the pulvinus point, such as... Figure 11 As shown; Step S4005: At the leaf pulvinus point position, vertically segment the dense skeleton point set and divide the plant point cloud along the skeleton path direction to obtain the corresponding leaf skeleton and leaf point cloud. in, Indicates the first The center distance from each perspective, Indicates the center position of the plant point cloud. Indicates the first The first projection point of the projection point cloud from each viewpoint. Indicates the first Tail projection points of the projection point cloud from each viewpoint Indicates the index of the projected point cloud.

[0032] The leaf pulvinus points of different leaves in the segmented plant point cloud are obtained separately, and segmentation is performed based on the leaf pulvinus points to obtain the skeleton of all leaves and the leaf point cloud of the plant, such as... Figure 12 As shown in the figure. The leaf skeleton extraction method proposed in this embodiment can be applied to the calculation of plant leaf length. Using the plant leaf skeleton extraction method of this embodiment, the comparison between the calculated and measured leaf length values ​​in the calculation of the leaf length of *Chicken Feather Cabbage* is shown in Table 1 below.

[0033] Table 1 Comparison of blade length measurement results

[0034] Example 2: A plant leaf skeleton extraction system, such as Figure 2 As shown, it includes: Point cloud processing module 100: used to acquire plant point clouds and perform preprocessing and point cloud segmentation to obtain segmented plant point clouds; Point cloud projection module 200: used to obtain several bounding rectangles of segmented plant point cloud, construct a three-dimensional point cloud coordinate system based on the bounding rectangles, obtain the point cloud projection results of segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtain the point cloud projection results under the remaining viewpoints by point cloud culling. Point cloud mapping module 300: used to extract and inverse map the point cloud projection results under the remaining viewpoints to obtain a three-dimensional skeleton point set, and extend the three-dimensional skeleton point set to obtain a dense skeleton point set. The filtering and segmentation module 400 is used to project the dense skeleton point set under the remaining view and obtain the skeleton point set through point cloud filtering. Based on the skeleton point set, leaf pulvinus points are obtained. The dense skeleton point set and the segmented plant point cloud are segmented by leaf pulvinus points to obtain the leaf skeleton and leaf point cloud.

[0035] Various changes and modifications made without departing from the spirit and scope of this invention, and all equivalent technical solutions, also fall within the scope of this invention.

[0036] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0037] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0038] This invention is described with reference to flowchart illustrations and / or block diagrams of the method, terminal device (system), and computer program product according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0039] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0040] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0041] It should be noted that: The phrase "an embodiment" or "an embodiment" used in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Therefore, the phrase "an embodiment" or "an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0042] Furthermore, it should be noted that the shapes and names of the parts and components described in the specific embodiments described in this specification may differ. All equivalent or simple variations made to the structure, features, and principles described in this patent concept are included within the protection scope of this patent. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to replace them, as long as they do not depart from the structure of this invention or exceed the scope defined in these claims, they should all fall within the protection scope of this invention.

Claims

1. A method for extracting the skeleton of plant leaves, characterized in that, Includes the following steps: Acquire plant point clouds and perform preprocessing and point cloud segmentation to obtain segmented plant point clouds; Obtain several bounding rectangles of the segmented plant point cloud, construct a three-dimensional point cloud coordinate system based on the bounding rectangles, obtain the point cloud projection results of the segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtain the point cloud projection results under the remaining viewpoints by removing the point cloud. Specifically, obtaining the point cloud projection result from the remaining viewpoint through point cloud culling is as follows: Calculate the percentage of effective pixels in the point cloud projection results from different viewpoints, and remove the point cloud projection results from the viewpoint with the largest percentage of effective pixels to obtain the point cloud projection results from the remaining viewpoints. Skeleton extraction and inverse mapping are performed on the point cloud projection results under the remaining viewpoints to obtain a 3D skeleton point set. The 3D skeleton point set is then extended to obtain a dense skeleton point set. The dense skeleton point set is projected under the remaining view and the skeleton point set is obtained by filtering the point cloud. The leaf pulvinus points are obtained based on the skeleton point set. The dense skeleton point set and the segmented plant point cloud are segmented by the leaf pulvinus points to obtain the leaf skeleton and leaf point cloud.

2. The method for extracting plant leaf skeletons according to claim 1, characterized in that, The process of acquiring plant point clouds, performing preprocessing and point cloud segmentation to obtain segmented plant point clouds includes the following steps: Obtain the point cloud of the entire plant and perform preprocessing to obtain the preprocessed plant point cloud; The preprocessed plant point cloud is segmented using a point cloud segmentation method to obtain an initial plant point cloud. The point cloud segmentation method includes a method for point cloud segmentation based on region normal vectors. Obtain the bounding rectangle of the initial plant point cloud. Search according to the preset search radius. Add the preprocessed plant point cloud that is different from the initial plant point cloud within the bounding rectangle to the initial plant point cloud until the number of initial plant point clouds and preprocessed plant point clouds within the bounding rectangle are the same, and obtain the segmented plant point cloud.

3. The method for extracting plant leaf skeletons according to claim 1, characterized in that, The process of obtaining several bounding rectangles for segmenting plant point clouds, constructing a three-dimensional point cloud coordinate system based on the bounding rectangles, obtaining the point cloud projection results of the segmented plant point clouds within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtaining the point cloud projection results under the remaining viewpoints through point cloud culling includes the following steps: Several bounding rectangles of the segmented plant point cloud are obtained respectively. A three-dimensional point cloud coordinate system is established with a vertex of the bounding rectangle as the origin. The segmented plant point cloud in the current bounding rectangle is projected to different viewpoints of the three-dimensional point cloud coordinate system to obtain the point cloud projection results under different viewpoints. Based on the number of points and the size of the plant point cloud in the point cloud projection results, the effective pixel ratio of the point cloud projection results under different viewpoints is obtained. Based on the effective pixel ratio, point cloud culling is performed on the point cloud projection results under different viewpoints to obtain the point cloud projection results of the remaining viewpoints after point cloud culling.

4. The method for extracting plant leaf skeletons according to claim 1, characterized in that, The process of extracting and inverse mapping the skeleton from the point cloud projection results under the remaining viewpoints to obtain a 3D skeleton point set includes the following steps: The initial number of clusters is preset, and the point cloud projection results under the remaining views are clustered according to the initial number of clusters to obtain the corresponding point cloud clustering results. The point cloud clustering results include the cluster centers and the corresponding cluster point clouds. The cluster centers in the point cloud clustering results are sorted using a sorting algorithm to minimize the path after sorting, thereby obtaining the corresponding two-dimensional skeleton result of the point cloud projection result under the remaining view. The sorting algorithm includes the ant colony algorithm. The two-dimensional skeleton result is back-mapped to three-dimensional space to obtain a three-dimensional skeleton point set.

5. The method for extracting plant leaf skeletons according to claim 1, characterized in that, The process of extending the three-dimensional skeleton point set to obtain a dense skeleton point set includes the following steps: Denoise the 3D skeleton point set to obtain the denoised 3D skeleton point set; The skeleton endpoints in the three-dimensional skeleton point set are obtained from the cluster centers in the point cloud clustering results. The skeleton endpoints include the skeleton start point, the start point adjacent points, the skeleton end point, and the end point adjacent points. Based on the skeleton endpoints, construct endpoint vectors, and project the cluster point clouds corresponding to the skeleton start point and skeleton end point onto the corresponding endpoint vectors to obtain the cluster point cloud projection point set; Calculate the distance between the projected points of the cluster point cloud and the endpoints of the skeleton, obtain the skeleton extension points through the distance, and add the skeleton extension points to the 3D skeleton point set to obtain the extended skeleton point set; Based on a preset distance threshold, the extended skeleton point set is interpolated to obtain a dense skeleton point set.

6. The method for extracting plant leaf skeletons according to claim 1, characterized in that, The process of projecting a dense skeleton point set onto the remaining viewpoint and obtaining a skeleton point set through point cloud filtering includes the following steps: Obtain the center position of the plant point cloud, project the dense skeleton point set under the remaining view to obtain the projected point cloud under the remaining view, and calculate the sum of the center distances between the first and last projected points and the center position in the projected point cloud. The projection point cloud under the remaining viewpoints is filtered based on the sum of the corresponding center distances under the remaining viewpoints to obtain the skeleton point set.

7. The method for extracting plant leaf skeletons according to claim 1, characterized in that, The process of obtaining leaf pulvinus points based on a skeleton point set, and then segmenting the dense skeleton point set and the plant point cloud using these leaf pulvinus points to obtain the leaf skeleton and leaf point cloud, includes the following steps: Based on the plant point cloud, obtain the point cloud width value of the skeleton point position in the skeleton point set; If the point cloud width value meets the preset leaf pillow segmentation condition, then the corresponding skeleton point position is the leaf pillow point; Based on the location of the leaf pulvinus point, the dense skeleton point set and segmented plant point cloud are vertically segmented to obtain the corresponding leaf skeleton and leaf point cloud.

8. A system for extracting plant leaf skeletons, characterized in that, include: Point cloud processing module: used to acquire plant point clouds and perform preprocessing and point cloud segmentation to obtain segmented plant point clouds; Point cloud projection module: used to obtain several bounding rectangles of segmented plant point cloud, construct a three-dimensional point cloud coordinate system based on the bounding rectangles, obtain the point cloud projection results of segmented plant point cloud within the bounding rectangles under different viewpoints in the three-dimensional point cloud coordinate system, and obtain the point cloud projection results under the remaining viewpoints by point cloud culling. Specifically, obtaining the point cloud projection result from the remaining viewpoint through point cloud culling is as follows: Calculate the percentage of effective pixels in the point cloud projection results from different viewpoints, and remove the point cloud projection results from the viewpoint with the largest percentage of effective pixels to obtain the point cloud projection results from the remaining viewpoints. Point cloud mapping module: used to extract and inverse map the point cloud projection results under the remaining viewpoints to obtain a 3D skeleton point set, and extend the 3D skeleton point set to obtain a dense skeleton point set; The filtering and segmentation module is used to project the dense skeleton point set under the remaining view and obtain the skeleton point set through point cloud filtering. Based on the skeleton point set, leaf pulvinus points are obtained. The dense skeleton point set and the segmented plant point cloud are segmented by leaf pulvinus points to obtain the leaf skeleton and leaf point cloud.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. A plant leaf skeleton extraction device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

Images