A flue-cured tobacco plant instance segmentation and 3D model reconstruction method based on TLS point cloud

Abstract

The application relates to the technical field of tobacco 3D modeling, in particular to a flue-cured tobacco plant instance segmentation and 3D model reconstruction method based on TLS point clouds, which comprises the following steps: obtaining ground-based laser radar point cloud data of a target area; denoising the point cloud data based on an SOR algorithm; thinning the point cloud data based on 3D voxels; performing tobacco plant instance segmentation by separately segmenting and identifying tobacco plant leaves and stalks; and reconstructing a 3D model of the tobacco plant based on the results of the tobacco plant instance segmentation. The method can nondestructively, quickly and accurately obtain complete three-dimensional morphological information of flue-cured tobacco by taking advantage of the wide scanning range, high precision and strong anti-interference capability of TLS point clouds, and has great theoretical and practical significance for deepening the research on the relationship between flue-cured tobacco plant type and yield and quality, guiding the breeding of high-light-efficiency plant types and optimizing cultivation measures, and is suitable for complex field environments and has excellent simulation effects.

Description

Technical Field

[0001] This application relates to the field of tobacco 3D modeling technology, specifically to a method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds. Background Technology

[0002] As one of the world's most important economic crops, the quality and yield of flue-cured tobacco leaves directly determine its economic value. Plant architecture, that is, the geometric shape and topological relationship of the three-dimensional spatial structure of its above-ground parts, is a core factor determining the crop's photosynthetic efficiency, nutrient distribution, ventilation, and the environment for pests and diseases. For flue-cured tobacco, an ideal plant architecture should have a reasonable leaf spatial arrangement to maximize the light interception efficiency of the canopy, reduce mutual shading of lower leaves, thereby promoting the uniform accumulation of dry matter and improving the coordination of the tobacco leaves' internal chemical composition.

[0003] However, traditional research on flue-cured tobacco plant type relies heavily on two-dimensional measurement tools, such as rulers, tape measures, and protractors, to measure a limited number of parameters, including plant height, stem diameter, maximum leaf length, maximum leaf width, and leaf angle. These methods are not only cumbersome and inefficient, but also prone to causing physical damage to the plants during measurement. More importantly, they fail to capture the complex three-dimensional morphology of leaves, such as leaf curling, wrinkling, drooping posture, and the precise azimuth and tilt angles of the leaves in space—key information determining the light environment and microclimate within the canopy. Therefore, developing a method that can non-destructively, rapidly, and accurately acquire complete three-dimensional morphological information of flue-cured tobacco is of significant theoretical and practical importance for deepening research on the relationship between plant type and function, guiding the breeding of high-light-efficiency plant types, and optimizing cultivation practices. Summary of the Invention

[0004] This application provides a method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds, in order to solve or partially solve the problems mentioned in the background art.

[0005] This application provides a method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds, including the following steps: S100: Acquire point cloud data of ground-based lidar in the target area; S200: Denoising point cloud data based on the SOR algorithm; S300: Point cloud data thinning based on 3D voxels; S400: Tobacco plant instance segmentation is performed by separately segmenting and identifying tobacco plant leaves and stems; S500: Based on the results of tobacco plant instance segmentation, a 3D model of the tobacco plant is reconstructed. In step S400, the thinned point cloud data is first processed using a surface growth segmentation algorithm to segment the tobacco plant leaves into instances. Then, the RANSAC shape detection algorithm is used to process the point set that was not identified as belonging to a certain leaf during the surface growth process to identify the tobacco plant stems. Finally, the number of tobacco plant stems is used as the number of tobacco plants, and the tobacco plant leaves are assigned to the corresponding tobacco plant stems to complete the instance segmentation of the tobacco plants.

[0006] Preferably, in step S200, the specific method for denoising point cloud data based on the SOR algorithm is as follows: S201: Neighborhood definition: Define the neighborhood for each point in the point cloud; S202: Distance calculation, calculate the distance from each point to all points in its neighborhood, and calculate the mean μ and standard deviation σ of these distances; S203: Noise detection, if the average distance from a point to its neighborhood... satisfy ,in If the value is the standard deviation multiplier, then the point is considered noise and removed. S204: Output the result, retaining the points that meet the conditions to form a de-noising point cloud.

[0007] Preferably, in step S300, the specific method for thinning point cloud data based on 3D voxels is as follows: S301: Find the bounding box of the point cloud; S302: 3D voxel partitioning, for bounding boxes, the bounding box space is partitioned into 3D space according to the voxel resolution d; S304: Voxelization, traversing each point in the ground-based LiDAR point cloud and assigning the point to a corresponding voxel; S305: Discrimination: If a voxel contains only one point, then retain that point; if a voxel contains multiple points, then retain only the point closest to the center in each voxel. S306: Output the result, retaining the points that meet the conditions to form a thinned point cloud.

[0008] Preferably, in step S400, the specific steps for segmenting tobacco leaves using the surface growth segmentation algorithm include: first, estimating the normal vector and residual; then, performing specific region growth; The process for estimating the normal vector and residuals is as follows: S401: Load point cloud data and mark all points as "1" for category and "unsegmented" for status. Assume there are a total of One point; S402: Create a 3D kd-tree spatial index for the point cloud; S403: Process each point one by one, for the... For each point, first use a kd-tree to find its value. Find the nearest neighbor points, and then use the eigenvalue method to find the current point and its nearest neighbor. The fitted plane equation of the point set consisting of the nth neighboring points can then be used to determine the nth... The normal vector of each point and its residuals .

[0009] Preferably, the surface growth process requires setting two parameters, including: a threshold angle sα between normal vectors and a distance threshold r; The rules for using sα are as follows: For the seed point to be expanded, use a kd-tree to query one of its neighboring points as the points to be processed. Let the normal vector of the current seed point be... The normal vector of the adjacent point to be processed is s ,and and All are unit normal vectors; If the dot product of the normal vectors of the adjacent point to be processed and the seed point... Less than the threshold If the adjacent point to be processed is found to be a seed point, it will be added to the seed point queue and used as a new seed point to continue to identify other unprocessed points; otherwise, the adjacent point to be processed will be skipped and other unprocessed points will be processed. The rules for using r are as follows: If the distance between the neighboring point to be processed and the seed plane determined by the seed point is less than the given distance threshold r, then the neighboring point to be processed is added to the seed point queue and used as a new seed point to continue to identify other unprocessed points; otherwise, the neighboring point to be processed is skipped, and other unprocessed points are processed. During the surface growth process, the similarity between the current seed point and its neighboring points is determined according to the usage rules of the angle threshold sα between normal vectors and the distance threshold r.

[0010] Preferably, in step S400, the specific method for identifying tobacco plant stems using the RANSAC shape detection algorithm includes: RANSAC shape detection was used to identify tobacco plant stem points, perform stem discrimination, and cluster two-dimensional spatially connected regions based on Euclidean distance.

[0011] Preferably, the RANSAC shape detection algorithm is used to verify whether the cylinder is a stem based on three criteria, including: The angle between the cylinder's axial direction and the z-axis is less than 10 degrees; The diameter of the cylinders ranges from 2.0 to 4.0 cm; The length of the cylinder is greater than 10cm; Cylinders that simultaneously meet the above three conditions and their corresponding point sets are retained for the next step of clustering. For candidate stalk points, spatially connected region clustering based on Euclidean distance is performed. Each cluster represents the stalk of a tobacco plant. During clustering, the distance threshold is set to 0.168m.

[0012] Preferably, the specific method for segmenting tobacco plants into instances is as follows: using the number of tobacco plant stems as the number of tobacco plants, and allocating tobacco plant leaves to the corresponding tobacco plants. S431: Calculate the two-dimensional center point of the tobacco plant stem; For each tobacco plant stem cluster, calculate the horizontal geometric center point of all points contained in the cluster, and calculate the geometric centroid point of each tobacco plant stem one by one to form a set of stem geometric centroid points; S432: Calculate the two-dimensional center point of tobacco plant leaves; For each tobacco plant leaf cluster, calculate the horizontal geometric centroid of all points contained in the cluster, calculate the geometric centroid of each tobacco plant leaf one by one, and form a set of leaf geometric centroids. S433: Nearest neighbor query; For each leaf, find the nearest geometric centroid of the stem to the geometric centroid of the leaf, associate the leaf with the corresponding stem, establish the relationship that the leaf belongs to the tobacco plant corresponding to the stem, establish the association and relationship between the leaf and the stem one by one, and finally complete the tobacco plant instance segmentation.

[0013] Preferably, in step S500, based on the result of tobacco plant instance segmentation, a tobacco plant instance is decomposed into a stem and several leaves, and a three-dimensional model of a tobacco plant is constructed by modeling the stem and leaves respectively. For the stem, a cylindrical model of the stem is established using least squares fitting. For the blade, the 3D reconstruction of the blade surface based on least-squares cylinder fitting, unfolding, and TIN generation includes the following steps: S501: For a 3D point set of data of a blade, fit a cylinder using least squares. S502: Project the 3D point cloud of the blade onto the surface of the cylinder, unfold the projected point cloud onto the cylindrical surface, transform the 3D point set into a 2D point set, construct a Delaunay triangulation for the 2D point set, and establish the spatial connection relationship between the 2D point sets. S503: Based on the one-to-one correspondence between points in the 3D point set and the 2D point set, the spatial connection relationship within the Delaunay triangulation is converted into the spatial connection relationship between points in the 3D point set, and a 3D Mesh is constructed. This Mesh model is the 3D model of the blade.

[0014] Compared with the prior art, the beneficial effects of this application are as follows: (1) This application utilizes the characteristics of TLS point cloud scanning, such as wide range, high precision and strong anti-interference ability, to construct a method that can obtain complete three-dimensional morphological information of flue-cured tobacco in a non-destructive, fast and accurate manner. It has great theoretical and practical significance for deepening the research on the relationship between flue-cured tobacco plant type and yield and quality, guiding the selection of high light efficiency plant type and optimizing cultivation measures. It is adaptable to complex field environment and has excellent simulation effect.

[0015] (2) This application first uses the SOR algorithm and 3D voxels to denoise and thin the point cloud, ensuring the accuracy and effectiveness of the point cloud data. Then, the surface growth segmentation algorithm and RANSAC shape detection algorithm are used to process the point cloud data in sequence, and the leaves and stems are accurately segmented into instances respectively. Finally, the instance segmentation of tobacco plants is completed. The algorithm is adapted to the characteristics of leaves and stems, and the recognition and segmentation effect is excellent. Attached Figure Description

[0016] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0017] Figure 1 For TLS point cloud data of tobacco plants, Figure 2 To remove noise from the TLS point cloud data, Figure 3 This is a 3D voxel diagram of a point cloud. Figure 4 This is a schematic diagram of the surface growth process. Figure 5 To illustrate the blade segmentation effect based on the surface growth segmentation algorithm, Figure 6 This is a schematic diagram of the unclustered point set in the surface growth segmentation algorithm. Figure 7 The segmentation effect for detecting the shape of a cylinder. Figure 8 To determine the effectiveness of two-dimensional connected region clustering of stem point sets, Figure 9 This is a schematic diagram showing the results of tobacco plant instance segmentation. Figure 10 To achieve the effect of three-dimensional reconstruction of tobacco plants, Figure 11 The data used in this application is point cloud data of flue-cured tobacco plants obtained on June 5, 2024. Figure 12 The data used in this application is point cloud data of flue-cured tobacco plants obtained on June 29, 2024. Figure 13 The image shows the segmentation effect of tobacco plant instances from point cloud data dated June 5, 2024, in an embodiment of this application. Figure 14 The image shows the segmentation effect of tobacco plant instances from point cloud data dated June 29, 2024, in an embodiment of this application. Figure 15 The image shows the reconstruction effect of a 3D model of tobacco plants based on point cloud data from June 5, 2024, in an embodiment of this application. Figure 16 The image shows the reconstruction effect of a 3D model of tobacco plants based on point cloud data from June 29, 2024, in an embodiment of this application. Detailed Implementation

[0018] The specification and claims use certain terms to refer to specific components. Those skilled in the art will understand that hardware manufacturers may use different names to refer to the same component. This specification and claims do not distinguish components based on differences in name, but rather on differences in function. The term "comprising" throughout the specification and claims is an open-ended term and should be interpreted as "comprising but not limited to." "Approximately" means that within an acceptable margin of error, those skilled in the art can solve the technical problem and substantially achieve the technical effect within a certain margin of error.

[0019] In the description of this application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "horizontal", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0020] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0021] With the development of computer graphics, machine vision and 3D sensing technology, plant 3D digitization has become a cutting-edge hot topic in plant phenomics research. Currently, plant 3D modeling methods are mainly divided into two categories: rule-based virtual modeling and reconstruction based on real structural data.

[0022] Rule-based virtual modeling primarily utilizes methods such as Lindenmayer-Systems (L-Systems) and Functional-Structural Plant Models (FSPM) to simulate the growth and development of plants using a set of predefined growth rules and parameters. For example, Guo Yan et al. (2012) explored virtual modeling of flue-cured tobacco plants by defining growth units of organs and topological rules to construct virtual tobacco plants. The advantage of this type of method is that it can simulate the dynamic growth process of plants and is suitable for studying the mechanism of environmental factors on plant morphogenesis. However, its main disadvantage is that the model is an abstract expression of the growth law of a class of plants, and it is difficult to accurately reproduce the complex morphology of a real, independent plant under a specific environment. Especially for flue-cured tobacco, a crop with variable leaf morphology and susceptible to environmental influences, the biological realism of its model is limited.

[0023] Three-dimensional reconstruction based on real structural data is a method that directly measures the surface geometry of plants using 3D sensors to reconstruct their three-dimensional models in a data-driven manner, achieving high fidelity. Handheld 3D scanners based on structured light technology (such as Artec Eva) and Structure from Motion (SfM) technology based on multi-view images have been widely used in recent years. Hu Xinyu et al. (2022) successfully constructed a flue-cured tobacco model using the Artec Eva scanner. Its advantages are portability, flexible operation, and the ability to acquire high-precision local details and texture information. However, handheld scanners are prone to cumulative errors in the overall scanning of large-sized plants (such as mature tobacco plants), and structured light technology is greatly affected by strong light environments. SfM technology is inexpensive, but for flue-cured tobacco leaves with similar textures and smooth surfaces, feature point matching is difficult, and the reconstruction accuracy and integrity are hard to guarantee.

[0024] Based on the above analysis, TLS (Ground-based LiDAR Point Cloud) has become an optional solution for acquiring large-scale, high-precision plant point cloud data. By emitting laser pulses and receiving echoes, TLS can quickly acquire millions of three-dimensional coordinate points on the target surface to form a "point cloud". The advantages of TLS are its wide scanning range, high accuracy, and strong anti-interference ability, making it particularly suitable for operations in complex environments such as fields or indoors. Its main challenge is that, due to the straight-line propagation of the laser, the occlusion of the target itself ("shadow effect") will lead to incomplete point cloud data.

[0025] like Figures 1 to 10 As shown, this application provides a method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds, specifically including the following steps: S100: Acquire point cloud data of ground-based lidar in the target area; S200: Denoising point cloud data based on the SOR algorithm; S300: Point cloud data thinning based on 3D voxels; S400: Tobacco plant instance segmentation is performed by separately segmenting and identifying tobacco plant leaves and stems; S500: Based on the results of tobacco plant instance segmentation, a 3D model of the tobacco plant is reconstructed.

[0026] like Figure 1 As shown, the raw point cloud data obtained by TLS often contains noise (the points within the red ellipse are all noise). The presence of noise may significantly reduce the effectiveness of subsequent point cloud processing algorithms. Therefore, these noise points need to be removed before point cloud processing. Statistical Outlier Removal (SOR) is a point cloud noise removal algorithm based on statistical principles. It is mainly used to remove noise from point cloud data. Its core idea is to analyze the neighborhood distance distribution of each point, identify and remove outliers that are significantly different from the distribution of surrounding points, thereby improving data quality.

[0027] Specifically, in step S200, the specific method for denoising point cloud data based on the SOR algorithm is as follows: S201: Neighborhood definition: For each point in the point cloud, define its neighborhood (a fixed number of neighboring points k); S202: Distance calculation, calculate the distance from each point to all points in its neighborhood, and calculate the mean μ and standard deviation σ of these distances; S203: Noise detection, if the average distance from a point to its neighborhood... satisfy (in If the standard deviation multiplier is used, then the point is considered noise and removed. S204: Output the result, retaining the points that meet the conditions to form a de-noising point cloud.

[0028] like Figure 2 As shown, Figure 1 The effect of noise removal from point cloud data, preferably in this application, =10, =4.0.

[0029] Ground-based LiDAR point clouds inherently exhibit density inconsistencies, and multi-site stitched LiDAR point clouds also exhibit data redundancy in overlapping areas. Point cloud thinning can ensure relatively consistent point cloud density and reduce data redundancy.

[0030] Specifically, in step S300, the specific method for thinning point cloud data based on 3D voxels is as follows: S301: Find the bounding box of the point cloud; S302: 3D voxel partitioning. For the bounding box, the bounding box space is partitioned into 3D spaces according to the voxel resolution d, such as... Figure 3 As shown; S304: Voxelization, traversing each point in the ground-based LiDAR point cloud and assigning the point to a corresponding voxel; S305: Discrimination: If a voxel contains only one point, then retain that point; if a voxel contains multiple points, then retain only the point closest to the center in each voxel. S306: Output the result, retaining the points that meet the conditions to form a thinned point cloud.

[0031] Specifically, in step S301, a typical method for obtaining the bounding box of the point cloud can be an axis-aligned bounding box (AABB). The bounding box is a cube parallel to the X / Y / Z coordinate axes, and its boundary is directly determined by the maximum and minimum values ​​of the point cloud in the three axes. Other bounding boxes or improved AABB bounding boxes can also be used, and this application does not impose any restrictions.

[0032] Preferably, the voxel resolution of this application is d=0.0025 meters.

[0033] Point cloud segmentation is the process of labeling each point in point cloud data according to certain discrimination rules. After segmentation, point sets that satisfy the same rule are assigned the same label, and each point set is called an object. The discrimination rule of the point cloud segmentation method in this application is that the points are adjacent and share a smooth surface in 3D space. In addition, isolated points that do not satisfy the above discrimination rules, points with insufficient number of neighboring points, and points with poor coplanarity will also be labeled. Given that the surface growing algorithm has the characteristics of requiring few parameters, good segmentation effect, and good universality, this application uses it to segment point clouds.

[0034] Specifically, in step S400, the tobacco plant leaves are first segmented into instances using a surface growth segmentation algorithm. The specific steps include: first, estimating the normal vector and residual; and then performing specific region growth.

[0035] Specifically, the process for estimating the normal vector and residuals is as follows: S401: Load point cloud data and mark all points as "1" for category and "unsegmented" for status. Assume there are a total of One point; S402: Create a 3D kd-tree spatial index for the point cloud; S403: Process each point one by one, for the... For each point, first use a kd-tree to find its value. Find the nearest neighbor points, and then use the eigenvalue method to find the current point and its nearest neighbor. The fitted plane equation of the point set consisting of the nth neighboring points can then be used to determine the nth... The normal vector of each point and its residuals .

[0036] Specifically, such as Figure 4 The image shows the surface growth process, which requires setting two parameters: the angle threshold sα between normal vectors and the distance threshold r.

[0037] The rules for using sα are as follows: For the seed point to be expanded, use a kd-tree to query one of its neighboring points as the points to be processed. Let the normal vector of the current seed point be... The normal vector of the adjacent point to be processed is s ,and and All are unit normal vectors; If the dot product of the normal vectors of the adjacent point to be processed and the seed point... Less than the threshold If the condition is met, the adjacent point to be processed is added to the seed point queue and used as a new seed point to continue to identify other unprocessed points; otherwise, the adjacent point to be processed is skipped and other unprocessed points are processed.

[0038] The rules for using r are as follows: If the distance between the neighboring point to be processed and the seed plane determined by the seed point is less than the given distance threshold r, then the neighboring point to be processed is added to the seed point queue and used as a new seed point to continue to identify other unprocessed points; otherwise, the neighboring point to be processed is skipped and other unprocessed points are processed.

[0039] Specifically, the surface growth treatment method of this application includes: S411: The "area label number" starts from 0; S412: Check the number of points in the "undivided" point set. If the number is 0, proceed to step S416. Otherwise, the residuals are then found from the set of "unsegmented" points. The smallest point is selected as the seed point and pushed into a queue of seed points. The processing status of this point is marked as "unprocessed", and region growing begins. S413: Take the first "unprocessed" seed point in the seed point queue, and use a kd-tree to find the k nearest neighbors of the seed point; S414: Determine each neighboring point one by one: If neighboring points have already been assigned a separator, no further processing is required; If a neighboring point is already in the seed point's queue, it will not be processed. Next (if not in either of the above two states), the similarity between the current seed point and its neighboring points is determined according to the rules for using the angle threshold sα between normal vectors and the distance threshold r. If a neighboring point satisfies both similarity conditions, then add the neighboring point to the seed point queue. Conversely, if a neighboring point does not simultaneously satisfy both similarity conditions, it will not be processed. After neighboring points are identified, the processing status of the seed point is marked as "processed". S415: Sequentially check if there are any "unprocessed" points in the seed point queue. If so, return to step S413; Otherwise, record the label of the point set in the seed point queue as "region label number" and the status as "segmented". At the same time, increment the "region label number" by 1, clear the seed point queue and return to step S412. S416: End.

[0040] After the above segmentation, any point is assigned to an object, but some objects have fewer points, and in some cases, an object contains only one point. Figure 2 The segmentation effect of a certain TLS point cloud data shown in the image is as follows: Figure 5 As shown, the point cloud of a single tobacco leaf is likely to be identified as one cluster, while the point clouds of different tobacco leaves are identified as different clusters, and the clustering effect is quite ideal.

[0041] After processing the above-mentioned regional growth process, we found that points belonging to tobacco leaves were most likely assigned to a certain cluster, while points not clustered were often tobacco plant stems, such as... Figure 6 As shown; ideally, the stem of a tobacco plant can be abstracted as a cylinder. We found that, despite missing or interfering points, the set of unclustered points in a certain local area can be fitted into a cylinder with a high probability. The set of points of a tobacco plant's stem is clustered into several interconnected cylinders. Therefore, the classic RANSAC shape detection can be used to identify tobacco plant stems. The process is further divided into two stages, which specifically include: identifying tobacco plant stem points using RANSAC shape detection, stem discrimination, and two-dimensional connected region clustering based on Euclidean distance.

[0042] This application uses the RANSAC shape detection algorithm for tobacco plant stem point identification. This algorithm involves four key parameters: the maximum allowable distance from the point to the shape. , normal vector angle threshold α, minimum sampling resolution e, minimum number of points required to construct the support shape The point set that was not identified as belonging to a certain leaf in steps S411 to S416 is denoted as a new point set S, which contains N points. The specific algorithm execution flow is as follows: S421: Normal vector calculation, using the PCA algorithm to calculate the normal vector of each point; S422: Construct an octree, where the level of the octree is denoted as . ; S423: Transfer the entire point cloud Randomly divided into A subset { , , ..., In this application ; S424: Shape candidate, using two points , and its corresponding normal vector , To determine a candidate cylinder, the specific process is as follows: First, for a new candidate shape, a point is randomly selected from the unlabeled points in the point cloud data as the first candidate point of the new shape. ,choose Then, select a unit at any level of the octree. (Included in the cell), in which another candidate point is selected from the unlabeled points. ; Secondly, establish an axis Its calculation formula is Then, the two parametric lines along this axis are... , Projected to On a plane, the intersection of the two lines obtained after projection is denoted as the center of the circle. Next, the center of the circle With point The Euclidean distance between them is denoted as the radius of the candidate cylinder; Finally, by using the distance threshold Threshold of the angle between the normal vector and the normal vector Two conditions are used to check whether the candidate cylinder meets the conditions; if both conditions are met, proceed to the next step; otherwise, perform shape candidate selection again. S425: Determine candidate shapes, As mentioned earlier, the entire point cloud has been... Divided into A subset First, regarding subsets within Evaluation was conducted at several points, selecting those points whose distance from the candidate shape was less than [a certain value]. And the angle between the normal vectors is less than of Given a number of points, calculate the following probabilities: in, The number of shapes that have been generated. if If yes, proceed to the next step; otherwise, continue evaluating the next subset. The cumulative points of the subset. To satisfy the requirement that the distance to the candidate shape is less than And the angle between the normal vectors is less than The probability calculation formula is based on the cumulative number of qualified points under each condition: Once satisfied If the condition is met, proceed to the next step; otherwise, continue evaluating the next subset until all subsets have been evaluated. If the condition is still not met, return to step S424. S426: Candidate point determination based on connected components Find the connected components in the bitmap located in the shape parameter domain. Project a point into the bitmap, and it corresponds to a pixel in the bitmap. Ideally, the size of a pixel in the bitmap should correspond to the distance between adjacent points in the data, i.e., the sampling resolution. If the data is irregularly sampled, the minimum sampling resolution that is satisfied everywhere in the data should be selected. In this patent, the pixel size is set as the input parameter threshold "minimum sampling resolution". ”; S427: Shape growth, For connected points, if they simultaneously satisfy the condition that the distance to the candidate shape is less than 1 / 3... And the angle between the normal vectors is less than If the condition is met, then add the point set of the shape. After growth is complete, if the point set... The number of points included is greater than the input parameter threshold "minimum number of points to support the shape construction". ", then the shape point set From point cloud Remove points from the list and save them separately. Delete all candidate shapes that contain the removed points, update the candidate set and the point cloud octree, and merge sparse nodes. Continue to the next iteration, selecting other shape point sets from the point cloud data; S428: Termination Condition Judgment. The shape detection algorithm terminates when one of the following conditions is met: ① The number of remaining points in the point cloud is <= 400; ②In the already generated Given τ candidate shapes, the probability that a shape containing τ points is not detected. , The calculation formula is: .

[0043] Figure 7 Showing Figure 6 The cylindrical segmentation effect of the midpoint cloud, where the red rectangle marks one of the 3D cylinders corresponding to the segmented point cloud. Figure 7 The partial loss of the stem point cloud of China Tobacco Plant was severe, and a large number of fragmented cylinders were detected in the cylinder segmentation results.

[0044] Preferably, in this application, the maximum permissible distance from a point to a shape is... =0.01m, threshold of the angle between normal vectors =25°, minimum sampling resolution =0.01m, minimum number of points required to construct the supporting shape =50.

[0045] Specifically, tobacco plant stems are generally perpendicular to the ground, and their diameter typically ranges from 2.0 to 4.0 cm. Therefore, three criteria are set to determine whether the detected cylindrical shape is a stem, including: (1) The angle between the cylinder axis and the z-axis is less than 10 degrees; (2) The diameter of the cylinder is between 2.0 and 4.0 cm; (3) The length of the cylinder is greater than 10cm; Cylinders that simultaneously meet the above three conditions and their corresponding point sets will be retained for the next step of clustering.

[0046] For candidate stalk points, perform two-dimensional spatial connectivity region clustering based on Euclidean distance, where each cluster represents the stalk of a tobacco plant; preferably, during clustering, the minimum threshold of the Euclidean distance between two neighboring points in the two-dimensional space is set to 0.168m.

[0047] 0.168m is an empirical threshold value, which is set by comprehensively considering factors such as plant height, maximum stem inclination, row spacing, and plant spacing. Figure 8 Showing Figure 7 The results of two-dimensional connected region clustering of stem point sets were evaluated, with a total of 5 clusters, corresponding to the stems of 5 real tobacco plants.

[0048] In S400, tobacco plant leaves are first obtained based on a surface growth segmentation algorithm, then tobacco plant stems are obtained based on a RANSAC shape detection algorithm. In the final stage, the number of tobacco plant stems is used as the number of tobacco plants, and tobacco plant leaves are assigned to the corresponding tobacco plant stems to complete the instance segmentation of tobacco plants. The specific method is as follows: S431: Calculate the two-dimensional center point of the tobacco plant stem; For each tobacco plant stem cluster, calculate the horizontal geometric center point of all points contained in the cluster, and calculate the geometric centroid point of each tobacco plant stem one by one to form a set of stem geometric centroid points; S432: Calculate the two-dimensional center point of tobacco plant leaves; For each tobacco plant leaf cluster, calculate the horizontal geometric centroid of all points contained in the cluster, calculate the geometric centroid of each tobacco plant leaf one by one, and form a set of leaf geometric centroids. S433: Nearest neighbor query; For each leaf, find the nearest geometric centroid of the stem to the geometric centroid of the leaf, associate the leaf with the corresponding stem, establish the relationship that the leaf belongs to the tobacco plant corresponding to the stem, establish the association and relationship between the leaf and the stem one by one, and finally complete the tobacco plant instance segmentation.

[0049] like Figure 9 As shown, it demonstrates Figure 2 The tobacco plant instance segmentation results of the pilot test data consist of 5 instances, each corresponding to a real tobacco plant.

[0050] Specifically, in step S500, based on the results of tobacco plant instance segmentation, a tobacco plant instance is decomposed into a stem and several leaves, and a three-dimensional model of a tobacco plant is constructed by modeling the stem and leaves separately.

[0051] Specifically, for the stem, a cylindrical model of the stem is established using least squares fitting.

[0052] Specifically, for the blade, the 3D reconstruction of the blade surface based on least-squares cylinder fitting, unfolding, and TIN (Triangulated Irregular Network) generation includes the following steps: S501: For a 3D point set of data of a blade, fit a cylinder using least squares. S502: Project the 3D point cloud of the blade onto the surface of the cylinder, unfold the projected point cloud onto the cylindrical surface, transform the 3D point set into a 2D point set, construct a Delaunay triangulation for the 2D point set, and establish the spatial connection relationship between the 2D point sets. S503: Based on the one-to-one correspondence between points in the 3D point set and the 2D point set, the spatial connection relationship within the Delaunay triangulation is converted into the spatial connection relationship between points in the 3D point set, and a 3D Mesh is constructed. This Mesh model is the 3D model of the blade.

[0053] like Figure 10 As shown, it demonstrates Figure 9 The 3D reconstruction results of 5 tobacco plant examples.

[0054] Example 1 Based on Example 1, this example uses a FARO FocusM 70 scanner to acquire point cloud data of flue-cured tobacco in the indoor plot of the greenhouse of the Yishui Tobacco Monopoly Bureau (Branch) Tobacco Experimental Station in Shandong Province. The Yishui Tobacco Experimental Station (N35.85843, E 118.63433) is located in Chaoyang Guanzhuang, Yishui County, Linyi City, Shandong Province, at an altitude of 191m. It belongs to the warm temperate continental monsoon climate with an average annual temperature of 14.1°C, precipitation of 849mm, and a frost-free period of more than 200 days per year.

[0055] The FARO FocusM 70 is a powerful 3D laser scanner designed for indoor and outdoor applications with scanning distances up to 70 meters. Measuring only 230 x 183 x 103 mm and weighing just 4.2 kg, it integrates a 165MP color camera. It offers a measurement range from 0.6 meters to 70 meters, a measurement rate of up to 488,000 points / second, a distance accuracy of ±3 mm, an IP54 protection rating, and an operating temperature range of -20°C to 55°C. The laser is Class 1. The device comes with a waterproof and ergonomic carrying case for enhanced portability. This ultra-portable device enables rapid, direct, and accurate measurements of construction sites, small building facades, complex structures, production / supply facilities, and accident and crime scenes.

[0056] The experiment used two phases of TLS point cloud data collected on June 5, 2024, and June 29, 2024, respectively. The point cloud data from the two phases are as follows: Figure 11 , Figure 12 As shown, there are 137 tobacco plants in this plot.

[0057] Figure 13 , Figure 14 The results of tobacco plant instance segmentation corresponding to point cloud data collected on June 5th and June 29th are shown respectively. Figure 13 and Figure 14 The tobacco plant examples shown are numbered from 0 to 136, totaling 137 examples, which were examined one by one. Figure 13 and Figure 14Each instance in the study was found to correspond to a real tobacco plant, and each leaf of each tobacco plant was correctly identified, with no oversegmentation or undersegmentation.

[0058] Figure 15 , Figure 16 The results of 3D reconstruction of tobacco plants corresponding to point cloud data collected on June 5th and June 29th are presented respectively. The 137 tobacco plants in the two reconstruction results were checked one by one, and it was found that all tobacco plants were correctly reconstructed. It can be seen that the method of this application has excellent instance segmentation and 3D model reconstruction effects.

[0059] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds, characterized in that, Includes the following steps: S100: Acquire point cloud data of ground-based lidar in the target area; S200: Denoising point cloud data based on the SOR algorithm; S300: Point cloud data thinning based on 3D voxels; S400: Tobacco plant instance segmentation is performed by separately segmenting and identifying tobacco plant leaves and stems; S500: Based on the results of tobacco plant instance segmentation, a 3D model of the tobacco plant is reconstructed. In step S400, the thinned point cloud data is first processed using a surface growth segmentation algorithm to segment the tobacco plant leaves into instances. Then, the RANSAC shape detection algorithm is used to process the point set that was not identified as belonging to a certain leaf during the surface growth process to identify the tobacco plant stems. Finally, the number of tobacco plant stems is used as the number of tobacco plants, and the tobacco plant leaves are assigned to the corresponding tobacco plant stems to complete the instance segmentation of the tobacco plants.

2. The method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds according to claim 1, characterized in that: In step S200, the specific method for denoising point cloud data based on the SOR algorithm is as follows: S201: Neighborhood definition: Define the neighborhood for each point in the point cloud; S202: Distance calculation, calculate the distance from each point to all points in its neighborhood, and calculate the mean μ and standard deviation σ of these distances; S203: Noise detection, if the average distance from a point to its neighborhood... satisfy ,in If the value is the standard deviation multiplier, then the point is considered noise and removed. S204: Output the result, retaining the points that meet the conditions to form a de-noising point cloud.

3. The method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds according to claim 1, characterized in that: In step S300, the specific method for thinning point cloud data based on 3D voxels is as follows: S301: Find the bounding box of the point cloud; S302: 3D voxel partitioning, for bounding boxes, the bounding box space is partitioned into 3D space according to the voxel resolution d; S304: Voxelization, traversing every point in the ground-based LiDAR point cloud and assigning the point to a corresponding voxel; S305: Discrimination: If a voxel contains only one point, then retain that point; if a voxel contains multiple points, then retain only the point closest to the center in each voxel. S306: Output the result, retaining the points that meet the conditions to form a thinned point cloud.

4. The method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds according to claim 1, characterized in that: In step S400, the specific steps for segmenting tobacco leaves using the surface growth segmentation algorithm include: first, estimating the normal vector and residual; then performing specific region growth; The process for estimating the normal vector and residuals is as follows: S401: Load point cloud data and mark all points as "1" for category and "unsegmented" for status. Assume there are a total of One point; S402: Create a 3D kd-tree spatial index for the point cloud; S403: Process each point one by one, for the... For each point, first use a kd-tree to find its value. Find the nearest neighbor points, and then use the eigenvalue method to find the current point and its nearest neighbor. The fitted plane equation of the point set consisting of the nth neighboring points can then be used to determine the nth... The normal vector of each point and its residuals .

5. The method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds according to claim 4, characterized in that: The surface growth process requires setting two parameters: the threshold angle sα between normal vectors and the distance threshold r; The rules for using sα are as follows: For the seed point to be expanded, use a kd-tree to query one of its neighboring points as the points to be processed. Let the normal vector of the current seed point be... The normal vector of the adjacent point to be processed is s ,and and All are unit normal vectors; If the dot product of the normal vectors of the adjacent point to be processed and the seed point... Less than the threshold If the adjacent point to be processed is found to be a seed point, it will be added to the seed point queue and used as a new seed point to continue to identify other unprocessed points; otherwise, the adjacent point to be processed will be skipped and other unprocessed points will be processed. The rules for using r are as follows: If the distance between the neighboring point to be processed and the seed plane determined by the seed point is less than the given distance threshold r, then the neighboring point to be processed is added to the seed point queue and used as a new seed point to continue to identify other unprocessed points; otherwise, the neighboring point to be processed is skipped, and other unprocessed points are processed. During the surface growth process, the similarity between the current seed point and its neighboring points is determined according to the usage rules of the angle threshold sα between normal vectors and the distance threshold r.

6. The method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds according to claim 1, characterized in that: In step S400, the specific method for identifying tobacco plant stems using the RANSAC shape detection algorithm includes: RANSAC shape detection was used to identify tobacco plant stem points, perform stem discrimination, and cluster two-dimensional spatially connected regions based on Euclidean distance.

7. The method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds according to claim 6, characterized in that: The RANSAC shape detection algorithm is used to determine whether a cylinder is a stem based on three criteria: The angle between the cylinder's axial direction and the z-axis is less than 10 degrees; The diameter of the cylinders ranges from 2.0 to 4.0 cm; The length of the cylinder is greater than 10cm; Cylinders that simultaneously meet the above three conditions and their corresponding point sets are retained for the next step of clustering. For candidate stalk points, perform two-dimensional spatial connectivity region clustering based on Euclidean distance. Each cluster represents the stalk of a tobacco plant. During clustering, the distance threshold is set to 0.168m.

8. The method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds according to claim 1, characterized in that: The specific method for segmenting tobacco plants into instances, using the number of tobacco plant stems as the number of tobacco plants and allocating tobacco plant leaves to the corresponding tobacco plants, is as follows: S431: Calculate the two-dimensional center point of the tobacco plant stem; For each tobacco plant stem cluster, calculate the horizontal geometric center point of all points contained in the cluster, and calculate the geometric centroid point of each tobacco plant stem one by one to form a set of stem geometric centroid points; S432: Calculate the two-dimensional center point of tobacco plant leaves; For each tobacco plant leaf cluster, calculate the horizontal geometric centroid of all points contained in the cluster, calculate the geometric centroid of each tobacco plant leaf one by one, and form a set of leaf geometric centroids. S433: Nearest neighbor query; For each leaf, find the nearest geometric centroid of the stem to the geometric centroid of the leaf, associate the leaf with the corresponding stem, establish the relationship that the leaf belongs to the tobacco plant corresponding to the stem, establish the association and relationship between the leaf and the stem one by one, and finally complete the tobacco plant instance segmentation.

9. The method for segmenting flue-cured tobacco plant instances and reconstructing 3D models based on TLS point clouds according to claim 1, characterized in that: In step S500, based on the result of tobacco plant instance segmentation, a tobacco plant instance is decomposed into a stem and several leaves, and a three-dimensional model of a tobacco plant is constructed by modeling the stem and leaves respectively. For the stem, a cylindrical model of the stem is established using least squares fitting. For the blade, the 3D reconstruction of the blade surface based on least-squares cylinder fitting, unfolding, and TIN generation includes the following steps: S501: For a 3D point set of data of a blade, fit a cylinder using least squares. S502: Project the 3D point cloud of the blade onto the surface of the cylinder, unfold the projected point cloud onto the cylindrical surface, transform the 3D point set into a 2D point set, construct a Delaunay triangulation for the 2D point set, and establish the spatial connection relationship between the 2D point sets. S503: Based on the one-to-one correspondence between points in the 3D point set and the 2D point set, the spatial connection relationship within the Delaunay triangulation is converted into the spatial connection relationship between points in the 3D point set, and a 3D Mesh is constructed. This Mesh model is the 3D model of the blade.

Images