Single-view tea bud 3D skeleton rapid extraction method based on point cloud analysis

Abstract

The application provides a single-view tea bud 3D skeleton rapid extraction method based on point cloud analysis, relates to the technical field of machine vision and image processing, and acquires all leaf preliminary skeletons of a tea tree surface according to point cloud data in a depth map, then selects sampling points through the position calibrated by the skeleton, acquires a plurality of feature maps according to the color of the sampling points, respectively acquires the skeleton in each feature map through the same method, compares the skeleton in all feature maps with the skeleton in the depth map, and thus the real tea bud skeleton is acquired. The application constructs a comprehensive comparison mode through point cloud data and color characteristics of tea buds, and guarantees the accuracy and comprehensiveness of skeleton extraction.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and image processing technology, specifically a method for rapid extraction of 3D skeletons of tea buds from a single perspective based on point cloud analysis. Background Technology

[0002] With the development of the tea market, the demand for high-quality tea is increasing. Tea buds are the most important material in tea production. Traditionally, tea bud picking is done by hand by tea farmers, a process that requires a large amount of labor and is time-consuming. Therefore, automated mechanized picking has become a necessity. Tea bud identification is the first step in automated picking, making machine vision-based tea bud identification crucial for automated harvesting. To avoid wasting computational resources and improve picking efficiency, identifying only the skeletal structure of the tea buds is sufficient to provide data support for subsequent picking actions. Therefore, a visual picking method based on image processing is needed.

[0003] In the prior art, CN117078926A discloses a method for locating tea shoots based on 3D point cloud topology. First, RGB images and point clouds of tea shoots are acquired using a depth camera. An improved target detection network is then used to identify and coarsely locate the tea shoots. Next, target point clouds are extracted based on the original point cloud set and the target region, and preprocessing is performed, including point cloud denoising and downsampling. Then, the preprocessed 3D point cloud is clustered and segmented, and skeleton points are extracted. Connecting these skeleton points yields the shoot skeleton. Finally, a nearest neighbor search is performed on the skeleton points of the tea shoot skeleton structure to obtain the grade of the tea shoot and the location of the picking position.

[0004] While the publicly available technical documents have achieved skeleton recognition of tea buds, they obtain skeleton information by recognizing the overall appearance of the tea buds. Furthermore, in tea trees, the leaves are dense and there is a lot of occlusion. Relying solely on point cloud data to obtain the skeleton has low robustness and affects the recognition success rate.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a method for rapid extraction of 3D skeleton of tea buds from a single perspective based on point cloud analysis, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A rapid single-view 3D skeleton extraction method for tea buds based on point cloud analysis, the specific steps of which include:

[0009] Step 1: Simultaneously acquire the depth image and grayscale image generated from the RGB image of the tea tree, align the pixel blocks of the two images, obtain the gradient value of each pixel block in the depth image, and divide them into groups according to the similarity and closure of the gradient values ​​of different pixel blocks to form several pixel groups.

[0010] Step 2: Binarize each pixel group, detect the two boundary lines of each pixel group, and use parallel propagation to simultaneously launch a spreading wave inward from the two boundary lines. The line where the two waves meet is marked as the skeleton.

[0011] Step 3: Take a sampling point on each skeleton in the depth map, record the pixel value of the sampling point in the grayscale image through the coordinate index of the sampling point, and construct the pixel range with the pixel value. Retain the pixel blocks in the grayscale image within the pixel range to form a feature map. A feature map is generated for each sampling point.

[0012] Step 4: Label each discrete pixel block in each feature map as a pixel group, obtain several pixel groups in the feature map, and obtain the skeleton of each pixel group in the feature map again through Step 2. Compare the skeleton in each feature map with the skeleton in the depth map. According to the number of identical pixel blocks occupied by the skeletons in the two maps, label the skeleton to be confirmed in the feature map and the corresponding skeleton in the depth map.

[0013] Step 5: Generate a similarity score by the number of skeletons to be confirmed in each feature map and the similarity between the skeletons to be confirmed and the corresponding skeletons. The feature map with the highest similarity score is the tea bud map, and the skeletons to be confirmed in the tea bud map are the tea bud skeletons.

[0014] Furthermore, tea tree images are acquired using an RGB-D camera. These images include RGB images and depth images. The RGB images are converted into grayscale images. Each pixel block in the grayscale image records the pixel value at that location, and each pixel block in the depth image records the depth value at that location. The pixel blocks of the grayscale image and the depth image are then aligned.

[0015] The grayscale image and the depth image were processed separately using median filtering.

[0016] Furthermore, the horizontal and vertical neighborhood gradients of each pixel block in the depth map are obtained by using a Sobel convolution kernel and gradient values ​​are formed. The size of the Sobel convolution kernel is 3x3.

[0017] A gradient array of a depth image is obtained, wherein the number of rows of the gradient array is equal to the number of rows of pixel blocks inside the depth image, the number of columns of the gradient array is equal to the number of columns of pixel blocks inside the depth image, and the two-dimensional index of each element in the gradient array corresponds one-to-one with the two-dimensional index of each pixel block in the depth image, and each element in the gradient array is the gradient value of the corresponding pixel block.

[0018] Furthermore, the pixel blocks in the depth image are grouped using a gradient array, and the grouping logic is as follows:

[0019] Set a gradient difference threshold. If there is a gradient value in the eight adjacent squares of this pixel block that is less than the gradient value of this pixel block and the difference is greater than the gradient difference threshold, then this pixel block is identified as an edge pixel block. Otherwise, this pixel block is not identified as an edge pixel block. Use the contour search method to determine all continuous edge pixel blocks, identify and obtain the edge pixel groups that form a closed space, connect the pixel blocks in the closed space and merge them into a pixel group, and discard the edge pixel blocks that do not form a closed space.

[0020] Determine the number of pixel blocks in each pixel group, set an element count threshold, remove pixel groups with fewer pixel blocks than the element count threshold, and use only the remaining pixel groups to form several pixel groups.

[0021] Furthermore, the skeleton is extracted for each pixel group separately, as follows:

[0022] Each pixel block within a pixel group is marked with a pixel value of 255, and each pixel block outside the pixel group is marked with a pixel value of 0. The boundary line of the pixel group is obtained through a contour tracing algorithm. At the same time, the endpoints of the pixel group are obtained through simple endpoint detection. The number of endpoints is at least two. If there are more than two endpoints in the pixel group, two endpoints are randomly saved, and the remaining endpoints are treated as points in the boundary line. Two independent boundary lines are formed based on the two endpoints. Starting from the two boundary lines, waves are simultaneously initiated from the boundary lines and spread into the pixel group according to the parallel wave propagation algorithm. The line where the two waves meet is the skeleton.

[0023] Furthermore, all skeletons in the depth image are obtained. Taking the top direction of the depth image as the top, the point N mm away from the top endpoint of each skeleton is taken as the sampling point. The coordinate index of each sampling point in the pixel block of the depth image is recorded, and this coordinate index is compared with the grayscale image to obtain the pixel value of the sampling point corresponding to each skeleton in the grayscale image.

[0024] Furthermore, a pixel value fluctuation range is set, centered on the pixel value obtained from each skeleton. Within this range, similarity extraction is performed on the grayscale image. The logic for this similarity extraction is as follows:

[0025] After a skeleton forms a corresponding sampling point pixel value, a pixel range is formed with this pixel value as the center. The upper limit of this range is the sampling point pixel value plus the pixel value fluctuation range, and the lower limit is the sampling point pixel value minus the pixel value fluctuation range. Pixel blocks in the grayscale image whose pixel values ​​are within the pixel range are retained and their pixel values ​​are marked as 255, while the pixel values ​​of other pixel blocks are set to 0, forming a feature map.

[0026] Obtain feature maps with the same number of skeletons.

[0027] Furthermore, the skeleton in each feature map is compared with the skeleton in the depth map to obtain the skeleton that may be a tea bud for confirmation. The comparison logic is as follows:

[0028] Each skeleton in the feature map is numbered, and the pixel blocks traversed by each skeleton and the coordinate index of each pixel block are recorded. Similarly, the pixel blocks traversed by each skeleton and the coordinate index of each pixel block are recorded in the depth map. Within the same feature map, the coordinate indices of each skeleton are compared with those of each skeleton in the depth map in numerical order to check for duplicates and the number of duplicate pixel blocks. A duplication threshold is set, representing the number of identical pixel blocks occupied by two skeletons. Skeletons in the feature map that exceed the duplication threshold are marked as skeletons to be confirmed. Each skeleton to be confirmed has a corresponding skeleton in the depth map.

[0029] Furthermore, tea bud feature maps are selected based on the number and similarity of skeletons to be identified in each feature map. The selection logic is as follows:

[0030] The proportion of the number of skeletons to be confirmed in this feature map to the total number of skeletons in this feature map is obtained. Simultaneously, the number of duplicate pixel blocks between the skeletons to be confirmed and their corresponding skeletons in this feature map is obtained. A similarity score is then constructed, with the following logic:

[0031] The proportion and the number of repeated pixel blocks are independent variables, and the similarity score is the dependent variable. The similarity score is formed by weighted summation of the proportion and the number of repeated pixel blocks. The weight of the proportion is greater than the weight of the number of repeated pixel blocks. The higher the proportion or the more repeated pixel blocks, the larger the similarity score. Moreover, the proportion has a greater impact on the similarity score than the number of repeated pixel blocks.

[0032] The feature map with the highest similarity score is obtained and labeled as the tea bud map. The skeleton to be confirmed in the tea bud map is the tea bud skeleton.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] This invention obtains a preliminary skeleton of all leaves on the surface of the tea tree based on point cloud data in the depth map. Then, sampling points are selected according to the positions marked by the skeleton, and multiple feature maps are obtained according to the color of the sampling points. The skeleton in each feature map is then obtained in the same way. The skeletons in all feature maps are compared with the skeletons in the depth map to obtain the real skeleton of the tea bud. This invention constructs a comprehensive comparison method by using point cloud data and the color characteristics of tea buds to ensure the accuracy and comprehensiveness of skeleton extraction. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall method flow of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0037] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0038] Example

[0039] Please see Figure 1 The present invention provides a technical solution:

[0040] A rapid single-view 3D skeleton extraction method for tea buds based on point cloud analysis, the specific steps of which include:

[0041] Step 1: Simultaneously acquire the depth image and grayscale image generated from the RGB image of the tea tree, align the pixel blocks of the two images, obtain the gradient value of each pixel block in the depth image, and divide them into groups according to the similarity and closure of the gradient values ​​of different pixel blocks to form several pixel groups.

[0042] Step 1 includes the following:

[0043] Tea tree images are acquired using an RGB-D camera. The tea tree images include RGB images and depth images. The RGB images are converted into grayscale images. Each pixel block in the grayscale image records the pixel value at that location, and each pixel block in the depth image records the depth value at that location. The pixel blocks of the grayscale image and the depth image are aligned.

[0044] The grayscale image and the depth image were processed separately using median filtering.

[0045] In a preferred embodiment, the median filtering method uses a 3x3 grid as the window size, with the center cell as the base cell. The window slides through the pixel blocks in the grayscale and depth images, allowing the base cell to traverse each pixel block. The values ​​in the 3x3 grid are sorted from high to low, and the values ​​in the base cell are replaced with the value of the fifth element in the sort. When the base cell is an edge pixel block of the grayscale or depth image, the missing elements in the 3x3 grid are filled with the constant 0. This achieves filtering on each pixel block in the grayscale and depth images, thereby removing noise.

[0046] The camera intrinsic parameter alignment pixel block data processing logic, by establishing a precise spatial mapping mechanism between modes, profoundly enhances the inherent consistency of multi-source data. This logic not only fundamentally curbs the accumulation of cross-modal bias caused by parallax or distortion, but also significantly improves the accuracy and stability of tea tree structure analysis in practical applications, avoiding systematic errors caused by position mismatch in downstream analysis. The local statistical replacement strategy of median filtering profoundly filters random noise impulses in a non-parametric manner, ensuring the smooth transition of data distribution. This noise suppression logic lays a robust foundation upstream of the data flow, not only connecting the sensitive input stability of subsequent gradient calculations, but also injecting an efficient preprocessing barrier into the overall solution, reducing unnecessary consumption of computing resources. At the same time, it has profound practical significance in agricultural scenarios, ensuring the reliability of tea bud extraction under real-time imaging.

[0047] Step 102: Obtain the horizontal and vertical neighborhood gradients of each pixel block in the depth map using a Sobel convolution kernel and form gradient values. The Sobel convolution kernel size is 3x3.

[0048] As a preferred embodiment, when the convolution kernel is located at the edge of the depth image, the missing elements in the convolution kernel are supplemented by zero-value padding.

[0049] In a preferred embodiment, the gradient value is equal to the square root sum of the horizontal and vertical neighborhood gradients.

[0050] A gradient array of a depth image is obtained, wherein the number of rows of the gradient array is equal to the number of rows of pixel blocks inside the depth image, the number of columns of the gradient array is equal to the number of columns of pixel blocks inside the depth image, and the two-dimensional index of each element in the gradient array corresponds one-to-one with the two-dimensional index of each pixel block in the depth image, and each element in the gradient array is the gradient value of the corresponding pixel block.

[0051] As a preferred embodiment, the formula used for gradient alignment is as follows:

[0052] ,in, Represents a gradient array. These respectively represent the first row and first column, and the first row and first column. Column, No. row, first column and second Line 1 The gradient value of the column, This indicates the total number of columns, which is the same as the number of columns in the pixel blocks of the depth map. This indicates the total number of rows, which is the same as the number of rows in the depth map summary pixel block;

[0053] The local difference operation logic of the Sobel convolution kernel deeply captures the intensity of directional changes in the depth field within a small window. This edge enhancement mechanism extracts a quantitative description of structural continuity from the microscopic level of data texture, which not only lays the computational foundation for segmentation in the overall process, but also significantly reduces the complexity of global search. In actual tea tree imaging, it effectively isolates the blurred boundaries caused by overlapping leaves. The gradient array index correspondence construction strategy organizes multi-dimensional gradient information in a matrix manner, realizing efficient memory access and parallel query. This structured storage logic profoundly optimizes the spatiotemporal efficiency of data processing, not only connecting the similarity comparison link of subsequent grouping, but also providing low-latency input for iterative algorithms such as wave propagation. In agricultural automation, its practical significance lies in accelerating the response time of single-view processing and significantly improving the adaptability to the field environment and system throughput.

[0054] Step 103: Group the pixel blocks in the depth image using a gradient array. The grouping logic is as follows:

[0055] Set a gradient difference threshold. If there is a gradient value in the eight adjacent squares of this pixel block that is less than the gradient value of this pixel block and the difference is greater than the gradient difference threshold, then this pixel block is identified as an edge pixel block. Otherwise, this pixel block is not identified as an edge pixel block. Use the contour search method to determine all continuous edge pixel blocks, identify and obtain the edge pixel groups that form a closed space, connect the pixel blocks in the closed space and merge them into a pixel group, and discard the edge pixel blocks that do not form a closed space.

[0056] Determine the number of pixel blocks in each pixel group, set an element count threshold, remove pixel groups with fewer pixel blocks than the element count threshold, and use only the remaining pixel groups to form several pixel groups.

[0057] The clustering judgment logic of omnidirectional gradient matching of adjacent elements profoundly transforms noise dispersion into ordered clustering of connected components. This region growth mechanism profoundly purifies the semantic boundaries of the depth map from the perspective of data topology, and acts as an efficient pre-segmentation engine in the overall scheme, filtering out isolated artifacts. The scale filtering strategy of element number threshold profoundly eliminates low-confidence clusters with statistical significance, ensuring the representativeness and computational value of the retained groups. This threshold constraint logic not only connects the purity of the regional input of skeleton extraction, but also profoundly avoids the diffusion deviation caused by small clusters in downstream wave propagation. In the actual scenario of tea garden monitoring, its significance lies in balancing the dual requirements of accuracy and efficiency, and promoting a smooth transition from coarse-grained to fine-grained processing.

[0058] Step 2: Binarize each pixel group, detect the two boundary lines of each pixel group, and use parallel propagation to simultaneously launch a spreading wave inward from the two boundary lines. The line where the two waves meet is marked as the skeleton.

[0059] Step 2 includes the following:

[0060] The skeleton is extracted for each group separately, according to the following logic:

[0061] The skeleton is extracted for each pixel group separately, as follows:

[0062] Each pixel block within a pixel group is marked with a pixel value of 255, and each pixel block outside the pixel group is marked with a pixel value of 0. The boundary line of the pixel group is obtained through a contour tracing algorithm. At the same time, the endpoints of the pixel group are obtained through simple endpoint detection. The number of endpoints is at least two. If there are more than two endpoints in the pixel group, two endpoints are randomly saved, and the remaining endpoints are treated as points in the boundary line. Two independent boundary lines are formed based on the two endpoints. Starting from the two boundary lines, waves are simultaneously initiated from the boundary lines and spread into the pixel group according to the parallel wave propagation algorithm. The line where the two waves meet is the skeleton.

[0063] In a preferred embodiment, the boundary line is obtained using the Moore neighborhood algorithm, and the spreading wave is a BFS wave.

[0064] The boundary extraction logic of binarization marking and contour tracing profoundly transforms complex regions into vector descriptions through topological simplification. This binary abstraction mechanism deeply bridges the segmentation output and topological analysis at the data representation level, not only strengthening the centerline of the local structure in the whole, but also effectively capturing linear continuity in the actual tea bud morphology. Endpoint detection combined with a randomized and simplified boundary division strategy, as well as the encounter equilibrium logic of parallel wave propagation, profoundly generates the skeleton axis from geometric symmetry. This dual-source diffusion suppression mechanism not only connects the stable anchoring of sampling points, but also significantly reduces the interference of branch noise on the main trunk extraction.

[0065] Step 3: Take a sampling point on each skeleton in the depth map, record the pixel value of the sampling point in the grayscale image through the coordinate index of the sampling point, and construct the pixel range with the pixel value. Retain the pixel blocks in the grayscale image within the pixel range to form a feature map. A feature map is generated for each sampling point.

[0066] Step 3 includes the following:

[0067] Step 3: Take a sampling point on each skeleton in the depth map, record the pixel value of the sampling point in the grayscale image through the coordinate index of the sampling point, and construct the pixel range with the pixel value. Retain the pixel blocks in the grayscale image within the pixel range to form a feature map. A feature map is generated for each sampling point.

[0068] As a preferred embodiment, the distance between the sampling point and the upper end point of the skeleton can be set to 3mm, where 3mm is the length in the real coordinate system. If the coordinate index of the sampling point in the pixel array of the depth map is (132,253), then the pixel value of the pixel block with coordinate index (132,253) in the grayscale map is obtained.

[0069] The sampling point positioning logic with a fixed offset at the upper endpoint ensures the representativeness and repeatability of the location through standardized anchor point selection. This anchoring mechanism deeply injects color semantics into modal mapping from geometric consistency, which not only expands the attribute dimension of the skeleton in the overall scheme, but also effectively avoids the pollution of feature sampling by endpoint noise in actual tea bud grading. The cross-image comparison and recording strategy of coordinate indexing deeply realizes depth-color alignment with pixel-level accuracy. This data association logic not only connects the baseline construction of the pixel range, but also deeply enhances the traceability capability of multimodal fusion. In the scenario of varying outdoor lighting, its practical significance lies in improving feature robustness and supporting the injection of color constraints for subsequent verification.

[0070] Step 302: Set a pixel value fluctuation range. Centered on the pixel value obtained for each skeleton, perform similarity extraction on the grayscale image within the pixel value fluctuation range. The logic for the similarity extraction is as follows:

[0071] After a skeleton forms a corresponding sampling point pixel value, a pixel range is formed with this pixel value as the center. The upper limit of this range is the sampling point pixel value plus the pixel value fluctuation range, and the lower limit is the sampling point pixel value minus the pixel value fluctuation range. Pixel blocks in the grayscale image whose pixel values ​​are within the pixel range are retained and their pixel values ​​are marked as 255, while the pixel values ​​of other pixel blocks are set to 0, forming a feature map.

[0072] As a preferred embodiment, the formula for determining the pixel range is as follows:

[0073] ,in, Indicates the pixel range. This represents the pixel value at the sampling point. Indicates the range of pixel value fluctuation;

[0074] The larger the range of pixel value fluctuation, the larger the interval included in this pixel range, and the pixel value at the sampling point determines the location of the pixel range.

[0075] Obtain feature maps with the same number of skeletons.

[0076] The similar region binarization logic based on the floating threshold of the center pixel value profoundly refines the semantic consistency of color clustering through symmetrical interval filtering. This threshold-driven mask generation mechanism profoundly extends the texture description of the skeleton in the data spectral domain. It not only acts as an auxiliary channel for color guidance in the whole, but also effectively suppresses the destruction of regional connectivity by spectral noise in actual tea bud identification. The parallel strategy of calibration and batch feature map generation profoundly optimizes the representation efficiency of multiple instances from the perspective of batch processing. This binary abstraction logic not only connects the input scale of color block grouping, but also profoundly provides a benchmark map set for quantifying color matching. In the practical application of tea tree variety differentiation, its significance lies in deepening modal complementarity and promoting the deep integration from geometric skeleton to visual semantics.

[0077] Step 4: Label each discrete pixel block in each feature map as a pixel group, obtain several pixel groups in the feature map, and obtain the skeleton of each pixel group in the feature map again through Step 2. Compare the skeleton in each feature map with the skeleton in the depth map. According to the number of identical pixel blocks occupied by the skeletons in the two maps, label the skeleton to be confirmed in the feature map and the corresponding skeleton in the depth map.

[0078] Step 4 includes the following:

[0079] The skeleton in each feature map is compared with the skeleton in the depth map to obtain the skeleton that may be a tea bud for confirmation. The comparison logic is as follows:

[0080] Each skeleton in the feature map is numbered, and the pixel blocks traversed by each skeleton and the coordinate index of each pixel block are recorded. Similarly, the pixel blocks traversed by each skeleton and the coordinate index of each pixel block are recorded in the depth map. Within the same feature map, the coordinate indices of each skeleton are compared with those of each skeleton in the depth map in numerical order to check for duplicates and the number of duplicate pixel blocks. A duplication threshold is set, representing the number of identical pixel blocks occupied by two skeletons. Skeletons in the feature map that exceed the duplication threshold are marked as skeletons to be confirmed. Each skeleton to be confirmed has a corresponding skeleton in the depth map.

[0081] The reuse grouping and skeleton extraction logic of discrete color block sets profoundly realizes the structural mirroring within the color domain through iterative topological simplification. This cross-domain reproduction mechanism profoundly strengthens the cross-consistency of multimodal data verification. It not only bridges feature generation and confirmation filtering in the overall scheme, but also effectively eliminates geometric offsets caused by color artifacts in actual tea bud positioning. The coordinate index comparison of numbering order combined with the repetition threshold calibration strategy profoundly quantifies the corresponding confidence of the skeleton from the overlap statistics. This threshold gating logic not only connects the input purity of similarity scoring, but also profoundly reduces the risk of false positive propagation. In the actual deployment of automated harvesting systems, its significance lies in building a reliable correspondence link to support refined decision-making from auxiliary skeletons to main skeletons.

[0082] Step 5: Generate a similarity score by the number of skeletons to be confirmed in each feature map and the similarity between the skeletons to be confirmed and the corresponding skeletons. The feature map with the highest similarity score is the tea bud map, and the skeletons to be confirmed in the tea bud map are the tea bud skeletons.

[0083] Step 5 includes the following:

[0084] The tea bud feature map is selected based on the number of skeletons to be identified and their similarity in each feature map. The selection logic is as follows:

[0085] The proportion of the number of skeletons to be confirmed in this feature map to the total number of skeletons in this feature map is obtained. Simultaneously, the number of duplicate pixel blocks between the skeletons to be confirmed and their corresponding skeletons in this feature map is obtained. A similarity score is then constructed, with the following logic:

[0086] The proportion and the number of repeated pixel blocks are independent variables, and the similarity score is the dependent variable. The similarity score is formed by weighted summation of the proportion and the number of repeated pixel blocks. The weight of the proportion is greater than the weight of the number of repeated pixel blocks. The higher the proportion or the more repeated pixel blocks, the larger the similarity score. Moreover, the proportion has a greater impact on the similarity score than the number of repeated pixel blocks.

[0087] As a preferred embodiment, the formula used for similarity scoring is as follows:

[0088] ,in, Indicates the first Similarity score of each feature map This represents the variable used for feature map retrieval. , , This represents the total number of feature maps. Indicates the first The total number of skeletons in each feature map. Indicates the first The number of skeletons to be identified in each feature map. Indicates the first The number of skeletons to be confirmed and the number of corresponding duplicate pixel blocks. This indicates the skeleton retrieval variable to be confirmed. , , and Weights 1 and 2 respectively, , , ;

[0089] As a preferred embodiment The value can be 10. The value can be 0.5.

[0090] The similarity between the skeleton in the feature map and the skeleton in the depth map is not simply reflected in the number of skeletons in the feature map, but in the "quality". The number of skeletons in the feature map doesn't need to be very large, but they must be extremely similar to certain skeletons in the depth map, and their relative positions in the two maps must be highly detailed to determine that the similar part is a tea bud. This is because, in the process of comparing the corresponding pixel values ​​in the grayscale image based on sampling points, many sampling points are actually located on leaves, and only a small portion are located on tea buds. However, since the color of mature leaves is very similar, while the color of tea buds differs significantly from that of mature leaves, feature maps are mainly divided into two categories: one is the base... Feature maps extracted from leaf colors often result in large areas of leaf images due to the similarity in leaf color. Furthermore, the binarization process erases subtle differences between leaves, leading to a more cluttered skeleton in the leaf-based feature map, which doesn't closely resemble the skeleton derived from depth values ​​in the depth map. Another type of feature map is based on tea buds. This type preserves the buds while removing leaves, eliminating significant interference. Therefore, the resulting skeleton more closely approximates a portion of the skeleton in the depth map. Further comparison between this feature map and the depth map, specifically based on the skeletons of pixels occupying the same positions, allows identification as a tea bud skeleton. This indicates the purity of the feature map. The more prominent the tea buds are in the map, the higher this ratio, and the more likely the feature map is to be composed solely of tea buds. Increase The larger the image, the better the tea bud images can be selected.

[0091] The feature map with the highest similarity score is obtained and labeled as the tea bud map. The skeleton to be confirmed in the tea bud map is the tea bud skeleton.

[0092] The weighted fusion scoring logic of proportion and number of repeating pixel blocks profoundly balances the dual indicators of density and accuracy through nonlinear weight allocation. This composite evaluation mechanism achieves profound global optimization of feature maps at the decision-making level. It not only seals the closed-loop verification from extraction to selection in the overall scheme, but also effectively prioritizes candidates with high semantic consistency in actual tea bud screening. The quantification strategy of weight bias proportion profoundly strengthens the dominant role of scale effect from the perspective of confidence propagation. This scoring gradient logic not only extracts the optimal path from the aforementioned comparison, but also profoundly avoids the amplification of bias in local matching. In the actual scenario of precision agriculture in tea gardens, its significance lies in providing interpretable decision-making basis and promoting the scalable evolution of single-view methods towards high-precision 3D skeleton output.

[0093] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0094] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0095] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0096] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for rapid extraction of 3D skeleton of tea buds from a single perspective based on point cloud analysis, characterized in that, The specific steps include: Step 1: Simultaneously acquire the depth image and grayscale image generated from the RGB image of the tea tree, align the pixel blocks of the two images, obtain the gradient value of each pixel block in the depth image, and divide them into groups according to the similarity and closure of the gradient values ​​of different pixel blocks to form several pixel groups. The pixel blocks in the depth image are grouped using a gradient array, and the grouping logic is as follows: Set a gradient difference threshold. If there is a gradient value in the eight adjacent squares of this pixel block that is less than the gradient value of this pixel block and the difference is greater than the gradient difference threshold, then this pixel block is identified as an edge pixel block. Otherwise, this pixel block is not identified as an edge pixel block. Use the contour search method to determine all continuous edge pixel blocks, identify and obtain the edge pixel groups that form a closed space, connect the pixel blocks in the closed space and merge them into a pixel group, and discard the edge pixel blocks that do not form a closed space. Determine the number of pixel blocks in each pixel group, set an element number threshold, remove pixel groups with a number of pixel blocks lower than the element number threshold, and use only the remaining pixel groups to form several pixel groups; Step 2: Binarize each pixel group, detect the two boundary lines of each pixel group, and use parallel propagation to simultaneously launch a spreading wave inward from the two boundary lines. The line where the two waves meet is marked as the skeleton. The skeleton is extracted for each pixel group separately, as follows: Each pixel block within a pixel group is marked with a pixel value of 255, and each pixel block outside the pixel group is marked with a pixel value of 0. The boundary line of the pixel group is obtained through a contour tracing algorithm. At the same time, the endpoints of the pixel group are obtained through simple endpoint detection. The number of endpoints is at least two. When there are more than two endpoints in a pixel group, two endpoints are randomly saved, and the remaining endpoints are used as points in the boundary line. Two independent boundary lines are formed based on the two endpoints. Starting from the two boundary lines, waves are simultaneously launched from the boundary lines into the pixel group according to the parallel wave propagation algorithm. The line where the two waves meet is the skeleton. Step 3: Take a sampling point on each skeleton in the depth map, record the pixel value of the sampling point in the grayscale image through the coordinate index of the sampling point, and construct the pixel range with the pixel value. Retain the pixel blocks in the grayscale image within the pixel range to form a feature map. A feature map is generated for each sampling point. Step 4: Label each discrete pixel block in each feature map as a pixel group, obtain several pixel groups in the feature map, and obtain the skeleton of each pixel group in the feature map again through Step 2. Compare the skeleton in each feature map with the skeleton in the depth map. According to the number of identical pixel blocks occupied by the skeletons in the two maps, label the skeleton to be confirmed in the feature map and the corresponding skeleton in the depth map. Step 5: Generate a similarity score by the number of skeletons to be confirmed in each feature map and the similarity between the skeletons to be confirmed and the corresponding skeletons. The feature map with the highest similarity score is the tea bud map, and the skeletons to be confirmed in the tea bud map are the tea bud skeletons.

2. The method for rapid extraction of 3D skeleton of tea buds from a single perspective based on point cloud analysis according to claim 1, characterized in that: Tea tree images are acquired using an RGB-D camera. The tea tree images include RGB images and depth images. The RGB images are converted into grayscale images. Each pixel block in the grayscale image records the pixel value at that location, and each pixel block in the depth image records the depth value at that location. The pixel blocks of the grayscale image and the depth image are aligned. The grayscale image and the depth image were processed separately using median filtering.

3. The method for rapid extraction of single-view 3D skeleton of tea buds based on point cloud analysis according to claim 2, characterized in that: The horizontal and vertical neighborhood gradients of each pixel block in the depth map are obtained by using a Sobel convolution kernel and the gradient values ​​are formed. The size of the Sobel convolution kernel is 3x3. A gradient array of a depth image is obtained, wherein the number of rows of the gradient array is equal to the number of rows of pixel blocks inside the depth image, the number of columns of the gradient array is equal to the number of columns of pixel blocks inside the depth image, and the two-dimensional index of each element in the gradient array corresponds one-to-one with the two-dimensional index of each pixel block in the depth image, and each element in the gradient array is the gradient value of the corresponding pixel block.

4. The method for rapid extraction of 3D skeleton of tea buds from a single perspective based on point cloud analysis according to claim 3, characterized in that: Obtain all skeletons in the depth image. Taking the top direction of the depth image as the top, take the point N mm away from the top endpoint of each skeleton as the sampling point. Record the coordinate index of each sampling point in the pixel block of the depth image, and compare this coordinate index with the grayscale image to obtain the pixel value of the sampling point corresponding to each skeleton in the grayscale image.

5. The method for rapid extraction of single-view 3D skeleton of tea buds based on point cloud analysis according to claim 4, characterized in that: A pixel value fluctuation range is set, and with the pixel value obtained for each skeleton as the center, similarity extraction is performed on the grayscale image within the pixel value fluctuation range. The logic of the similarity extraction is as follows: After a skeleton forms a corresponding sampling point pixel value, a pixel range is formed with this pixel value as the center. The upper limit of this range is the sampling point pixel value plus the pixel value fluctuation range, and the lower limit is the sampling point pixel value minus the pixel value fluctuation range. Pixel blocks in the grayscale image whose pixel values ​​are within the pixel range are retained and their pixel values ​​are marked as 255, while the pixel values ​​of other pixel blocks are set to 0, forming a feature map. Obtain feature maps with the same number of skeletons.

6. The method for rapid extraction of 3D skeleton of tea buds from a single perspective based on point cloud analysis according to claim 5, characterized in that: The skeleton in each feature map is compared with the skeleton in the depth map to obtain the skeleton that may be a tea bud for confirmation. The comparison logic is as follows: Each skeleton in the feature map is numbered, and the pixel blocks traversed by each skeleton and the coordinate index of each pixel block are recorded. Similarly, the pixel blocks traversed by each skeleton and the coordinate index of each pixel block are recorded in the depth map. Within the same feature map, the coordinate indices of each skeleton are compared with those of each skeleton in the depth map in numerical order to check for duplicates and the number of duplicate pixel blocks. A duplication threshold is set, representing the number of identical pixel blocks occupied by two skeletons. Skeletons in the feature map that exceed the duplication threshold are marked as skeletons to be confirmed. Each skeleton to be confirmed has a corresponding skeleton in the depth map.

7. The method for rapid extraction of 3D skeleton of tea buds from a single perspective based on point cloud analysis according to claim 6, characterized in that: The tea bud feature map is selected based on the number of skeletons to be identified and their similarity in each feature map. The selection logic is as follows: The proportion of the number of skeletons to be confirmed in this feature map to the total number of skeletons in this feature map is obtained. Simultaneously, the number of duplicate pixel blocks between the skeletons to be confirmed and their corresponding skeletons in this feature map is obtained. A similarity score is then constructed, with the following logic: The proportion and the number of repeated pixel blocks are independent variables, and the similarity score is the dependent variable. The similarity score is formed by weighted summation of the proportion and the number of repeated pixel blocks. The weight of the proportion is greater than the weight of the number of repeated pixel blocks. The higher the proportion or the more repeated pixel blocks, the larger the similarity score. Moreover, the proportion has a greater impact on the similarity score than the number of repeated pixel blocks. The feature map with the highest similarity score is obtained and labeled as the tea bud map. The skeleton to be confirmed in the tea bud map is the tea bud skeleton.

Images