Plant point cloud extraction method, device and equipment

Abstract

The present application provides a method, device and equipment for extracting plant point clouds. The method can remove noise from the initial plant three-dimensional point cloud to obtain a plant three-dimensional point cloud to be processed. After determining its main direction, the plant three-dimensional point cloud to be processed is rotated based on the main direction and a minimum bounding box is constructed to determine the directions of each coordinate axis of the point cloud, and then the ground part thereof is removed, and the corresponding two-dimensional projection is determined. Then, the hexagonal histogram is used for discretization processing to obtain an undirected graph. Since there are various connected components in the undirected graph, the maximum connected component can be determined, and the hexagonal vertices of the maximum connected component are mapped to the coordinate plane of the plant three-dimensional point cloud to be processed to determine the corresponding boundary. The plant three-dimensional point cloud is extracted according to the boundary. The plant three-dimensional point cloud obtained in this way can remove some complex backgrounds and improve the accuracy of plant three-dimensional point cloud extraction.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and equipment for extracting plant point clouds. Background Art

[0002] With the development of modern agriculture, point cloud technology has been widely used in plant phenotyping and crop breeding. The demand for analysis of crop growth and phenotypic characteristics is increasing in agricultural research, especially the accurate analysis of individual plants in field environments. Point cloud data, as an expression of three-dimensional structural information, can fully reflect the morphological characteristics of plants, such as height, canopy width, and biomass. These characteristics are of great reference value for crop breeding, planting management, and growth monitoring.

[0003] However, in complex field environments, point cloud data processing faces many challenges, such as noise interference, weed occlusion, and ground reflection. These interferences not only affect the extraction accuracy of plant point clouds, but also greatly increase the complexity and computational workload of data processing. Traditional point cloud processing methods are difficult to effectively solve these problems and often require manual intervention or rely on complex algorithm adjustments, resulting in low efficiency.

[0004] Therefore, how to accurately and automatically extract the point cloud of the target plant in a complex environment has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, the purpose of this application is to propose a plant point cloud extraction method, device and equipment to solve or partially solve the above-mentioned technical problems.

[0006] Based on the above purpose, the present application provides a plant point cloud extraction method, comprising:

[0007] Acquire an initial plant three-dimensional point cloud, and perform denoising on the initial plant three-dimensional point cloud to obtain a plant three-dimensional point cloud to be processed;

[0008] Determine the main direction of the three-dimensional point cloud of the plant to be processed;

[0009] According to the main direction, the three-dimensional point cloud of the plant to be processed is rotated and a minimum bounding box is constructed, and the minimum bounding box is rotated back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed to form a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud, wherein the original coordinate system includes: X, Y and Z axes;

[0010] Removing the ground point cloud portion in the minimum enclosing point cloud to obtain a three-dimensional point cloud with the ground removed;

[0011] Determine a two-dimensional projection of the three-dimensional point cloud without the ground, and discretize the two-dimensional projection using a hexagonal histogram to obtain an undirected graph;

[0012] Determining a maximum connected component according to the undirected graph;

[0013] Determine the vertex coordinates of the hexagon corresponding to the maximum connected component, map the vertex coordinates to the coordinate plane of the three-dimensional point cloud of the plant to be processed and determine the boundary, remove the points outside the boundary in the three-dimensional point cloud of the plant to be processed, and obtain the extracted three-dimensional point cloud of the plant.

[0014] Based on the same inventive concept, the present application also provides a plant point cloud extraction device, comprising:

[0015] A denoising module is configured to obtain an initial plant three-dimensional point cloud, and perform denoising on the initial plant three-dimensional point cloud to obtain a to-be-processed plant three-dimensional point cloud;

[0016] A main direction determination module is configured to determine the main direction of the three-dimensional point cloud of the plant to be processed;

[0017] The bounding box processing module is configured to rotate the three-dimensional point cloud of the plant to be processed according to the main direction and construct a minimum bounding box, rotate the minimum bounding box back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed, and form a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud, wherein the original coordinate system includes: X, Y and Z axes;

[0018] A ground part removal module is configured to remove the ground point cloud part in the minimum enclosing point cloud to obtain a three-dimensional point cloud with the ground removed;

[0019] an undirected graph determination module, configured to determine a two-dimensional projection of the three-dimensional point cloud with the ground removed, and to obtain an undirected graph by discretizing the two-dimensional projection using a hexagonal histogram;

[0020] A connected component processing module, configured to determine a maximum connected component according to the undirected graph;

[0021] The plant point cloud extraction module is configured to determine the vertex coordinates of the hexagon corresponding to the maximum connected component, map the vertex coordinates to the coordinate plane of the three-dimensional point cloud of the plant to be processed and determine the boundary, remove the points outside the boundary in the three-dimensional point cloud of the plant to be processed, and obtain the extracted three-dimensional point cloud of the plant.

[0022] Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0023] Based on the same inventive concept, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method as described above.

[0024] From the above, it can be seen that the plant point cloud extraction method, device and equipment provided by the present application can obtain the three-dimensional point cloud of the plant to be processed after denoising the initial plant three-dimensional point cloud, so that after determining its main direction, the three-dimensional point cloud of the plant to be processed can be rotated according to the main direction and a minimum bounding box can be constructed. In order to make the minimum bounding box correspond to the coordinates of the plant point cloud to be processed for point cloud extraction, it is necessary to correspond the minimum bounding box to the original coordinate system (X, Y and Z axes) of the plant point cloud to be processed to obtain the minimum enclosing point cloud. In this way, the ground part can be removed according to the minimum enclosing point cloud, and then the corresponding two-dimensional projection is determined. The two-dimensional projection is discretized using a hexagonal histogram to obtain an undirected graph. Since there are various connected components in the undirected graph, the maximum connected component can be determined, and the hexagonal vertices of the maximum connected component are mapped to the coordinate plane of the three-dimensional point cloud of the plant to be processed to determine the corresponding boundary. The three-dimensional point cloud of the plant is extracted according to the boundary. The three-dimensional point cloud of the plant obtained in this way can remove some complex environmental images and improve the accuracy of the three-dimensional point cloud extraction of the plant. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A flow chart of a plant point cloud extraction method according to an embodiment of the present application;

[0027] Figure 2 A schematic diagram of an initial plant three-dimensional point cloud of a soybean plant according to an embodiment of the present application;

[0028] Figure 3 A schematic diagram of a histogram in the vertical direction of a horizontal plane according to an embodiment of the present application;

[0029] Figure 4 A schematic diagram of a two-dimensional histogram according to an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the maximum connected component of an embodiment of the present application;

[0031] Figure 6The extracted three-dimensional point cloud of the plant in the embodiment of the present application;

[0032] Figure 7 This is a structural block diagram of a plant point cloud extraction device according to an embodiment of the present application;

[0033] Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0035] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0036] The point cloud processing method based on histogram information can combine statistical analysis, principal component analysis and undirected graph algorithm to improve the accuracy of point cloud processing from the aspects of spatial distribution and structural characteristics of plants, remove irrelevant data and retain the target area. This method can not only effectively reduce noise, but also ensure the automation and efficiency of the processing process, providing strong technical support for precision breeding and crop management in the agricultural field.

[0037] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0038] The plant point cloud extraction method (abbreviated as HH3DMesh) provided in the embodiments of the present application is as follows: Figure 1 As shown, including:

[0039] Step 101, obtain the initial plant three-dimensional point cloud (such as Figure 2 The initial plant three-dimensional point cloud of the soybean plant shown in FIG. 1 is denoised to obtain the plant three-dimensional point cloud to be processed.

[0040] In specific implementation, the initial plant 3D point cloud is obtained by collecting the 3D coordinate information of the field plants by a 3D laser scanning camera. The initial plant 3D point cloud contains the target plants that need to be collected as well as the ground, debris, weeds, etc. The statistical filtering algorithm is used to perform preliminary denoising on the point cloud according to the preset filtering parameters to reduce the amount of calculation and improve the accuracy of subsequent calculations.

[0041] Step 102: determine the main direction of the three-dimensional point cloud of the plant to be processed.

[0042] In the specific implementation, the principal component analysis (PCA) algorithm is used to process the three-dimensional point cloud of the plant to be processed to obtain the main direction. The main direction represents the main direction of the target plant to be extracted.

[0043] Step 103, according to the main direction, rotate the three-dimensional point cloud of the plant to be processed and construct a minimum bounding box, rotate the minimum bounding box back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed, and form a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud, wherein the original coordinate system includes: X, Y and Z axes.

[0044] In the specific implementation, in order to ensure that the determined minimum bounding box is the most appropriate, the 3D point cloud of the plant to be processed will be rotated to the main direction, so that the minimum bounding box can be determined according to the rotated plant point cloud, thereby making the determined minimum bounding box more accurate. Then the minimum bounding box is rotated back to the X, Y and Z axes of the original coordinate system corresponding to the 3D point cloud of the plant to be processed to obtain the minimum bounding point cloud. The minimum bounding point cloud includes the target plant.

[0045] Step 104 , removing the ground point cloud portion in the minimum enclosing point cloud to obtain a three-dimensional point cloud with the ground removed.

[0046] In specific implementation, in addition to the target plant, the minimum enclosing point cloud also contains some noise such as ground, weeds or other debris. Since the ground is relatively low, the ground point cloud can be removed according to the distance to obtain a three-dimensional point cloud without the ground.

[0047] Step 105 , determining a two-dimensional projection of the three-dimensional point cloud without the ground, and performing discretization processing based on the two-dimensional projection using a hexagonal histogram to obtain an undirected graph.

[0048] In specific implementation, in order to facilitate calculation, it is necessary to perform dimensionality reduction processing on the three-dimensional point cloud after removing the ground to obtain a two-dimensional projection, and then discretize it using a hexagonal histogram to obtain an undirected graph.

[0049] Step 106, determining the maximum connected component according to the undirected graph.

[0050] In specific implementation, since there are many connected components corresponding to the undirected graph, in order to accurately extract them, it is necessary to determine the largest connected component therein.

[0051] Step 107, determine the vertex coordinates of the hexagon corresponding to the maximum connected component, map the vertex coordinates to the coordinate plane of the three-dimensional point cloud of the plant to be processed and determine the boundary, remove the points outside the boundary in the three-dimensional point cloud of the plant to be processed, and obtain the extracted three-dimensional point cloud of the plant.

[0052] In specific implementation, the target plant is included in the maximum connected component, so the boundary of the target plant is determined according to its corresponding hexagonal vertices. After the boundary is determined, other point clouds outside the boundary can be removed to obtain the plant three-dimensional point cloud corresponding to the target plant.

[0053] Through the above scheme, the initial plant three-dimensional point cloud can be denoised to obtain the three-dimensional point cloud of the plant to be processed, so that after determining its main direction, the three-dimensional point cloud of the plant to be processed can be rotated according to the main direction and a minimum bounding box can be constructed. In order to make the minimum bounding box correspond to the coordinates of the plant point cloud to be processed for point cloud extraction, it is necessary to correspond the minimum bounding box to the original coordinate system (X, Y and Z axes) of the plant point cloud to be processed to obtain the minimum bounding point cloud. In this way, the ground part can be removed according to the minimum bounding point cloud, and then the corresponding two-dimensional projection is determined. The two-dimensional projection is discretized using a hexagonal histogram to obtain an undirected graph. Since there are various connected components in the undirected graph, the maximum connected component can be determined, and the hexagonal vertices of the maximum connected component are mapped to the coordinate plane of the three-dimensional point cloud of the plant to be processed to determine the corresponding boundary. The three-dimensional point cloud of the plant is extracted according to the boundary. The three-dimensional point cloud of the plant obtained in this way can remove some complex environmental images and improve the accuracy of the three-dimensional point cloud extraction of the plant.

[0054] In some embodiments, step 102 includes:

[0055] Step 1021, determining the mean point according to the coordinates corresponding to the three-dimensional point cloud of the plant to be processed.

[0056] In specific implementation, the coordinates of the three-dimensional point cloud of the plant to be processed are determined as for i=1,2,…, n . Where n is the total number of points. The corresponding mean point for:

[0057] .

[0058] Step 1022: determine the covariance matrix C of the three-dimensional point cloud of the plant to be processed based on the mean point.

[0059] In specific implementation, the corresponding calculation formula is: Where T represents the matrix transpose.

[0060] Step 1023, performing eigenvalue decomposition on the covariance matrix to obtain eigenvectors and eigenvalues, and determining the eigenvector corresponding to the largest eigenvalue as the main direction, wherein the eigenvector represents the eigendirection, and the eigenvalue represents the data variance in the eigendirection corresponding to the eigenvector.

[0061] Through the above solution, it can be ensured that the determined main direction is more consistent with the directional characteristics of the target plant to be extracted, and the determined main direction is more accurate.

[0062] In some embodiments, step 103 includes:

[0063] Step 1031, determine the rotation matrix R according to the eigenvector corresponding to the main direction, and based on the rotation matrix R, rotate the coordinate points corresponding to the three-dimensional point cloud of the plant to be processed to the coordinate system corresponding to the main direction to obtain the rotated three-dimensional point cloud of the plant.

[0064] In specific implementation, the rotation matrix R is composed of the eigenvectors of the main directions, and the coordinates of the three-dimensional point cloud of the rotated plant are for: ,in, p=p i .

[0065] Step 1032, determining the maximum coordinate value and the minimum coordinate value in the coordinates corresponding to the rotated three-dimensional point cloud of the plant, and determining a minimum bounding box according to the maximum coordinate value and the minimum coordinate value.

[0066] In specific implementation, Find the maximum coordinate value in and the minimum coordinate value , determine the minimum bounding box The dimensions are (length, width, height) = .

[0067] Step 1033, rotating the minimum bounding box back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed, forming a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud.

[0068] In specific implementation, the coordinates of the corresponding minimum bounding box The formula for the minimum enclosing point cloud P obtained after conversion is: , where the coordinate values ​​of the X, Y, and Z axes of the minimum bounding point cloud P correspond to the length, width, and height of the minimum bounding box, respectively.

[0069] Through the above scheme, an accurate minimum enclosing point cloud for the target plant can be obtained, which facilitates the subsequent point cloud extraction for the target plant.

[0070] In some embodiments, step 104 includes:

[0071] Step 1041, draw a histogram in a direction perpendicular to the horizontal plane according to the minimum enclosing point cloud (such as Figure 3 as shown).

[0072] In specific implementation, the coordinate axes of the minimum enclosing point cloud include: the X-axis and Y-axis of the two-dimensional horizontal plane, and the Z-axis in the vertical direction.

[0073] Extract the Z-axis coordinate values ​​of all points in the minimum enclosing point cloud, obtain the distribution data of the Z-axis, use a certain interval (bin) width to perform segmented statistics on the Z-axis coordinate values, calculate the number of points in each interval, and draw a histogram about the Z-axis direction. Assume that the Z-axis coordinate value is , then the number of each interval in the histogram can be expressed as: ,in, Indicates the Z-axis coordinate value The number of points in the interval corresponds to the number of points in each height interval.

[0074] Step 1042 , smoothing the histogram, and using a Gaussian kernel density estimation (KDE) algorithm to generate a Gaussian kernel density curve based on the smoothed histogram.

[0075] In specific implementation, the Gaussian kernel density estimation algorithm is a non-parametric algorithm used to estimate the probability density function, and the corresponding formula is: Where z is the coordinate point of the Gaussian kernel density curve, n is the total number of points, and h is the bandwidth parameter. This curve represents the point distribution density along the Z axis, helping to identify peaks (maximum values) and troughs (minimum values).

[0076] Step 1043: Calculate the slope of the Gaussian kernel density curve, and determine at least one waveform corresponding to the Gaussian kernel density curve according to the slope.

[0077] In specific implementation, the first-order derivative operation is performed on the Gaussian density curve to obtain the slope , the slope It represents the rate of change of the Gaussian kernel density curve at each point. The specific formula is: .

[0078] Step 1044, determining the difference between the maximum value and the minimum value in each waveform as the slope difference, and taking the waveform with the largest slope difference as the target waveform.

[0079] In specific implementation, the maximum value of each waveform is determined according to the slope (For example, Figure 3 The red point Max Slope) and the minimum (For example, Figure 3 The blue point Min Slope is used as the slope difference. , and then select the one with the largest slope difference (for example, Figure 3 In the figure, the waveform of the yellow point Max Curvature is taken as the most representative target waveform.

[0080] Step 1045, determining a trough point according to the target waveform, determining a dividing line according to the trough point, removing the point cloud smaller than the dividing line in the minimum enclosing point cloud as a ground point cloud portion, and obtaining a three-dimensional point cloud with the ground removed.

[0081] In specific implementation, the minimum value of the target waveform, that is, the point with the smallest point cloud density, is determined as the trough point. The trough point usually represents the valley area between two waves, which may be the boundary of different segmented point clouds. Then, the Z value corresponding to the corresponding trough point is used as the dividing line, and the point cloud below the dividing line in the minimum enclosing point cloud is removed, thereby obtaining a three-dimensional point cloud without the ground.

[0082] Through the above solution, the point cloud of the ground in the minimum enclosing point cloud can be accurately removed, thereby obtaining a three-dimensional point cloud without the ground.

[0083] In some embodiments, step 105 includes:

[0084] Step 1051, perform a two-dimensional projection on the horizontal plane on the three-dimensional point cloud without the ground, and determine a two-dimensional histogram (such as Figure 4 as shown).

[0085] In the specific implementation, the coordinates of the X and Y axes are extracted from the three-dimensional point cloud without the ground to form a two-dimensional projection. , and draw a two-dimensional projection on the XY plane The corresponding two-dimensional histogram.

[0086] Step 1052: divide the two-dimensional histogram into hexagons, discretize the two-dimensional histogram, and obtain a plurality of hexagonal histograms.

[0087] In the specific implementation, the standard hexagonal division is used to divide the two-dimensional histogram, where each hexagon represents a unit area. The corresponding hexagonal histogram of the multiple discretized areas is ,in, If the point If it falls within the hexagon, the function value is 1, otherwise it is 0, where N is the total number of points in the point cloud in the two-dimensional histogram.

[0088] Step 1053, establish an undirected edge between each group of two adjacent hexagonal histograms.

[0089] Step 1054: Take each hexagonal histogram as a node, and construct all the hexagonal histograms into an undirected graph according to the undirected edges.

[0090] In the specific implementation, if two adjacent hexagonal histograms have points, an undirected edge will be established between them. In this way, each hexagonal histogram is regarded as a node in the undirected graph. Each node is represented by a hexagonal histogram coordinate, and the relationship of the corresponding undirected edges can be represented by the adjacency list or adjacency matrix of the undirected graph, thereby constructing an undirected graph.

[0091] In some embodiments, step 106 includes:

[0092] Step 1061, traverse at least one node in the undirected graph, and for each node, use a depth-first traversal algorithm to determine all nodes connected to the node, and use all nodes connected to the node as connected components of the node, wherein each node corresponds to a connected component.

[0093] In specific implementation, the process of the depth-first traversal algorithm is: randomly select an unvisited node to start, traverse, mark the node as visited, and recursively visit all nodes connected to the node (i.e. adjacent nodes). When all adjacent nodes have been visited, backtrack using the depth-first algorithm until all connected components have been traversed.

[0094] Specifically: ,in is the starting node in the graph, is with The connected components are composed of all connected nodes.

[0095] Step 1062, calculate the number of nodes corresponding to each connected component, and determine the connected component with the largest number of nodes as the maximum connected component (such as Figure 5 as shown).

[0096] In specific implementation, if the set of connected components is , then the maximum connected component , Represents the number of nodes contained in the i-th connected component.

[0097] Through the above solution, the maximum connected component can be found more accurately.

[0098] In some embodiments, step 107 includes:

[0099] Step 1071, obtaining the vertex coordinate set of all hexagons in the maximum connected component.

[0100] In a specific implementation, the above description states that each hexagonal node of each connected component corresponds to a set of vertex coordinates, and the set of vertices corresponding to the maximum connected component.

[0101] For each hexagon there are 6 vertices , the vertex set of the hexagon is: , for all hexagonal nodes in the maximum connected component, extract their corresponding vertex coordinates.

[0102] The corresponding vertex coordinate set is .

[0103] Step 1072, mapping the vertices of each hexagon in the vertex coordinate set to the coordinate plane of the three-dimensional point cloud of the plant to be processed to determine the boundary.

[0104] In a specific implementation, the vertices of each hexagon in the vertex coordinate set are connected, mapped to the three-dimensional point cloud of the plant to be processed, and the boundary covered by the maximum connected area is constructed.

[0105] Step 1073, determining whether each point in the three-dimensional point cloud of the plant to be processed is within the boundary, if yes, retain the point, otherwise delete the point.

[0106] In the specific implementation, each point in the plant 3D point cloud is processed , to determine whether it is within the boundary, it is determined as:

[0107] .

[0108] Step 1074, obtaining the extracted plant three-dimensional point cloud (such as Figure 6 as shown).

[0109] Point set for the 3D point cloud of the plant to be processed ,reserve The corresponding points are recorded as:

[0110] .

[0111] The resulting set of points As the extracted plant 3D point cloud.

[0112] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0113] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a plant point cloud extraction device.

[0115] refer to Figure 7 , the device comprises:

[0116] The denoising module 201 is configured to obtain an initial plant 3D point cloud, and perform denoising on the initial plant 3D point cloud to obtain a to-be-processed plant 3D point cloud;

[0117] A main direction determination module 202 is configured to determine the main direction of the three-dimensional point cloud of the plant to be processed;

[0118] The bounding box processing module 203 is configured to rotate the three-dimensional point cloud of the plant to be processed according to the main direction and construct a minimum bounding box, rotate the minimum bounding box back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed, and form a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud, wherein the original coordinate system includes: X, Y and Z axes;

[0119] A ground part removal module 204 is configured to remove the ground point cloud part in the minimum enclosing point cloud to obtain a three-dimensional point cloud with the ground removed;

[0120] The undirected graph determination module 205 is configured to determine a two-dimensional projection of the three-dimensional point cloud without the ground, and obtain an undirected graph by discretizing the two-dimensional projection using a hexagonal histogram;

[0121] A connected component processing module 206 is configured to determine a maximum connected component according to the undirected graph;

[0122] The plant point cloud extraction module 207 is configured to determine the vertex coordinates of the hexagon corresponding to the maximum connected component, map the vertex coordinates to the coordinate plane of the three-dimensional point cloud of the plant to be processed and determine the boundary, remove the points outside the boundary in the three-dimensional point cloud of the plant to be processed, and obtain the extracted three-dimensional point cloud of the plant.

[0123] In some embodiments, the main direction determination module 202 is specifically configured to:

[0124] Determine the mean point according to the coordinates corresponding to the three-dimensional point cloud of the plant to be processed;

[0125] Determine the covariance matrix of the three-dimensional point cloud of the plant to be processed based on the mean point;

[0126] Perform eigenvalue decomposition on the covariance matrix to obtain eigenvectors and eigenvalues, and determine the eigenvector corresponding to the largest eigenvalue as the main direction, wherein the eigenvector represents the eigendirection, and the eigenvalue represents the data variance in the eigendirection corresponding to the eigenvector.

[0127] In some embodiments, the bounding box processing module 203 is specifically configured to:

[0128] Determine a rotation matrix according to the eigenvector corresponding to the main direction, and based on the rotation matrix, rotate the coordinate points corresponding to the three-dimensional point cloud of the plant to be processed to the coordinate system corresponding to the main direction to obtain a rotated three-dimensional point cloud of the plant;

[0129] Determine the maximum coordinate value and the minimum coordinate value in the coordinates corresponding to the rotated three-dimensional point cloud of the plant, and determine the minimum bounding box according to the maximum coordinate value and the minimum coordinate value;

[0130] The minimum bounding box is rotated back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed to form a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud.

[0131] In some embodiments, the ground portion removal module 204 is specifically configured to:

[0132] Draw a histogram in a direction perpendicular to the horizontal plane according to the minimum enclosing point cloud;

[0133] The histogram is smoothed, and a Gaussian kernel density estimation algorithm is used to generate a Gaussian kernel density curve based on the smoothed histogram;

[0134] Calculating the slope of the Gaussian kernel density curve, and determining at least one waveform corresponding to the Gaussian kernel density curve according to the slope;

[0135] Determine the difference between the maximum value and the minimum value in each waveform as the slope difference, and take the waveform with the largest slope difference as the target waveform;

[0136] A trough point is determined according to the target waveform, a dividing line is determined according to the trough point, and a point cloud smaller than the dividing line in the minimum enclosing point cloud is removed as a ground point cloud portion to obtain a three-dimensional point cloud with the ground removed.

[0137] In some embodiments, the undirected graph determination module 205 is specifically configured to:

[0138] Performing a two-dimensional projection on the horizontal plane on the three-dimensional point cloud without the ground, and determining a two-dimensional histogram according to the two-dimensional projection;

[0139] Dividing the two-dimensional histogram by hexagons, and discretizing the two-dimensional histogram to obtain a plurality of hexagonal histograms;

[0140] Create an undirected edge between each set of two adjacent hexagonal histograms;

[0141] Each hexagonal histogram is taken as a node, and all the hexagonal histograms are constructed into an undirected graph according to the undirected edges.

[0142] In some embodiments, the connected component processing module 206 is specifically configured to:

[0143] Traversing at least one node in the undirected graph, for each node, using a depth-first traversal algorithm to determine all nodes connected to the node, and taking all nodes connected to the node as connected components of the node, wherein each node corresponds to one connected component;

[0144] The number of nodes corresponding to each connected component is calculated, and the connected component with the largest number of nodes is determined as the maximum connected component.

[0145] In some embodiments, the plant point cloud extraction module 207 is specifically configured to:

[0146] Obtaining a vertex coordinate set of all hexagons in the maximum connected component;

[0147] Mapping the vertices of each hexagon in the vertex coordinate set to the coordinate plane of the three-dimensional point cloud of the plant to be processed to determine the boundary;

[0148] Determine whether each point in the three-dimensional point cloud of the plant to be processed is within the boundary, if yes, keep the point, otherwise delete the point;

[0149] The extracted plant 3D point cloud is obtained from the set of all retained points.

[0150] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0151] The device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0152] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the above embodiments when executing the program.

[0153] Figure 8 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0154] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0155] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0156] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0157] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0158] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0159] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0160] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0161] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0162] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0163] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0164] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0165] It is understandable that before using the technical solutions of each embodiment of the present application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0166] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present application according to the prompt message.

[0167] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0168] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of this application. Other methods that meet relevant laws and regulations may also be applied to the implementation method of this application.

[0169] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0170] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device may be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0171] Although the present application has been described in conjunction with specific embodiments of the present application, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the discussed embodiments.

[0172] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A plant point cloud extraction method, characterized in that: include: Acquire an initial plant three-dimensional point cloud, and perform denoising on the initial plant three-dimensional point cloud to obtain a plant three-dimensional point cloud to be processed; Determine the main direction of the three-dimensional point cloud of the plant to be processed; According to the main direction, the three-dimensional point cloud of the plant to be processed is rotated and a minimum bounding box is constructed, and the minimum bounding box is rotated back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed to form a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud, wherein the original coordinate system includes: X, Y and Z axes; Removing the ground point cloud portion in the minimum enclosing point cloud to obtain a three-dimensional point cloud with the ground removed; Determine a two-dimensional projection of the three-dimensional point cloud without the ground, and discretize the two-dimensional projection using a hexagonal histogram to obtain an undirected graph; Determining a maximum connected component according to the undirected graph; Determine the vertex coordinates of the hexagon corresponding to the maximum connected component, map the vertex coordinates to the coordinate plane of the three-dimensional point cloud of the plant to be processed and determine the boundary, remove the points outside the boundary in the three-dimensional point cloud of the plant to be processed, and obtain the extracted three-dimensional point cloud of the plant.

2. The method according to claim 1, characterized in that Determining the main direction of the three-dimensional point cloud of the plant to be processed includes: Determine the mean point according to the coordinates corresponding to the three-dimensional point cloud of the plant to be processed; Determine the covariance matrix of the three-dimensional point cloud of the plant to be processed based on the mean point; Perform eigenvalue decomposition on the covariance matrix to obtain eigenvectors and eigenvalues, and determine the eigenvector corresponding to the largest eigenvalue as the main direction, wherein the eigenvector represents the eigendirection, and the eigenvalue represents the data variance in the eigendirection corresponding to the eigenvector.

3. The method according to claim 1, characterized in that According to the main direction, the three-dimensional point cloud of the plant to be processed is rotated and a minimum bounding box is constructed, and the minimum bounding box is rotated back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed to form a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud, including: Determine a rotation matrix according to the eigenvector corresponding to the main direction, and based on the rotation matrix, rotate the coordinate points corresponding to the three-dimensional point cloud of the plant to be processed to the coordinate system corresponding to the main direction to obtain a rotated three-dimensional point cloud of the plant; Determine the maximum coordinate value and the minimum coordinate value in the coordinates corresponding to the rotated three-dimensional point cloud of the plant, and determine the minimum bounding box according to the maximum coordinate value and the minimum coordinate value; The minimum bounding box is rotated back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed to form a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud.

4. The method according to claim 1, characterized in that: The step of removing the ground point cloud portion in the minimum enclosing point cloud to obtain a three-dimensional point cloud without the ground includes: Draw a histogram in a direction perpendicular to the horizontal plane according to the minimum enclosing point cloud; The histogram is smoothed, and a Gaussian kernel density estimation algorithm is used to generate a Gaussian kernel density curve based on the smoothed histogram; Calculating the slope of the Gaussian kernel density curve, and determining at least one waveform corresponding to the Gaussian kernel density curve according to the slope; Determine the difference between the maximum value and the minimum value in each waveform as the slope difference, and take the waveform with the largest slope difference as the target waveform; A trough point is determined according to the target waveform, a dividing line is determined according to the trough point, and a point cloud smaller than the dividing line in the minimum enclosing point cloud is removed as a ground point cloud portion to obtain a three-dimensional point cloud with the ground removed.

5. The method according to claim 1, characterized in that The determining of the two-dimensional projection of the three-dimensional point cloud with the ground removed, and discretizing the two-dimensional projection using a hexagonal histogram to obtain an undirected graph includes: Performing a two-dimensional projection on the horizontal plane on the three-dimensional point cloud without the ground, and determining a two-dimensional histogram according to the two-dimensional projection; Dividing the two-dimensional histogram by hexagons, and discretizing the two-dimensional histogram to obtain a plurality of hexagonal histograms; Create an undirected edge between each set of two adjacent hexagonal histograms; Each hexagonal histogram is taken as a node, and all the hexagonal histograms are constructed into an undirected graph according to the undirected edges.

6. The method according to claim 1, characterized in that Determining the maximum connected component according to the undirected graph comprises: Traversing at least one node in the undirected graph, for each node, using a depth-first traversal algorithm to determine all nodes connected to the node, and taking all nodes connected to the node as connected components of the node, wherein each node corresponds to one connected component; The number of nodes corresponding to each connected component is calculated, and the connected component with the largest number of nodes is determined as the maximum connected component.

7. The method according to claim 1, characterized in that The step of determining the vertex coordinates of the hexagon corresponding to the maximum connected component, mapping the vertex coordinates to the coordinate plane of the three-dimensional point cloud of the plant to be processed and determining the boundary, and removing points outside the boundary in the three-dimensional point cloud of the plant to be processed to obtain the extracted three-dimensional point cloud of the plant includes: Obtaining a vertex coordinate set of all hexagons in the maximum connected component; Mapping the vertices of each hexagon in the vertex coordinate set to the coordinate plane of the three-dimensional point cloud of the plant to be processed to determine the boundary; Determine whether each point in the three-dimensional point cloud of the plant to be processed is within the boundary, if yes, keep the point, otherwise delete the point; The extracted plant 3D point cloud is obtained from the set of all retained points.

8. A plant point cloud extraction device, characterized in that: include: A denoising module is configured to obtain an initial plant three-dimensional point cloud, and perform denoising on the initial plant three-dimensional point cloud to obtain a to-be-processed plant three-dimensional point cloud; A main direction determination module is configured to determine the main direction of the three-dimensional point cloud of the plant to be processed; The bounding box processing module is configured to rotate the three-dimensional point cloud of the plant to be processed according to the main direction and construct a minimum bounding box, rotate the minimum bounding box back to the original coordinate system corresponding to the three-dimensional point cloud of the plant to be processed, and form a minimum bounding point cloud with the length, width and height of the minimum bounding box as the X, Y and Z axes of the point cloud, wherein the original coordinate system includes: X, Y and Z axes; A ground part removal module is configured to remove the ground point cloud part in the minimum enclosing point cloud to obtain a three-dimensional point cloud with the ground removed; an undirected graph determination module, configured to determine a two-dimensional projection of the three-dimensional point cloud with the ground removed, and to obtain an undirected graph by discretizing the two-dimensional projection using a hexagonal histogram; A connected component processing module, configured to determine a maximum connected component according to the undirected graph; The plant point cloud extraction module is configured to determine the vertex coordinates of the hexagon corresponding to the maximum connected component, map the vertex coordinates to the coordinate plane of the three-dimensional point cloud of the plant to be processed and determine the boundary, remove the points outside the boundary in the three-dimensional point cloud of the plant to be processed, and obtain the extracted three-dimensional point cloud of the plant.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images