Rhizome medicine posture estimation and cutting point positioning method and system
By using the YOLO model and geometric constraint strategy, combined with the weighted least squares method, high-precision attitude estimation and cutting point localization of rhizomes and medicinal materials were achieved, solving the problems of low efficiency and damage caused by manual operation, and providing an automated solution for rhizome and stem separation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 罗俊杰
- Filing Date
- 2026-03-12
- Publication Date
- 2026-07-03
AI Technical Summary
In the current technology, the separation of rhizomes from root and stem medicinal materials relies on manual operation, which is labor-intensive, inefficient, and easily damages the medicinal materials, making it difficult to meet market demand.
By employing the YOLO model combined with geometric constraint strategies and weighted least squares, geometric relationships are established based on the characteristics of the roots, stems, and root-stem transition zones of rhizomes, enabling rapid and high-precision attitude estimation and cutting point localization.
It achieves high-precision attitude estimation and cutting point positioning for rhizomes and medicinal materials, improves separation efficiency, reduces damage to medicinal materials and loss of effective components, and provides technical support for automation.
Smart Images
Figure CN122335971A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of agricultural automation and computer vision technology, and in particular to a method and system for attitude estimation and cutting point localization of root and rhizome medicinal materials. Background Technology
[0002] The main medicinal components of rhizome-based medicinal materials (such as Bupleurum, Astragalus, and Licorice) are saponins, flavonoids, and other active substances, which are mainly concentrated in the roots, while the stems contain lower levels of these active ingredients. Excessive mixing of these components will affect the quality and efficacy of the medicinal materials. Therefore, root-stem separation is a crucial step in the post-harvest processing of rhizome-based medicinal materials.
[0003] However, the separation of rhizomes from medicinal herbs mainly relies on manual labor, which involves visual observation and tactile judgment of the cutting point and the use of scissors for cutting. This method is not only labor-intensive and inefficient, but also suffers from inconsistent operating standards, which can easily damage the herbs and cause the loss of effective components, resulting in a high loss rate that is difficult to meet market demand. Therefore, there is an urgent need for an automated method that can adapt to the morphological characteristics of rhizomes and achieve rapid and high-precision attitude estimation and cutting point positioning. Summary of the Invention
[0004] The purpose of this invention is to provide a method and system for posture estimation and cutting point localization of rhizome medicinal materials. By utilizing the characteristics of the root, stem, and root-stem transition zone of the rhizome medicinal material, geometric relationships between the various parts can be established, and the posture and cutting point of the rhizome medicinal material can be obtained quickly and stably.
[0005] To achieve the above objectives, the present invention provides the following solution:
[0006] A method for estimating the pose and locating the cutting of rhizome-type medicinal materials includes the following steps:
[0007] Acquire training data, which includes RGB images of root and stem medicinal material samples at each stage and region labels of the whole, stem, root, and root-stem transition zone in the RGB images of root and stem medicinal material samples at each stage.
[0008] A YOLO model is constructed and trained on rhizome medicinal materials. The model includes various parameters and open-source versions of the YOLO model, including but not limited to YOLOv5, YOLOv8, and YOLO11.
[0009] The image of the root and rhizome medicinal material to be processed is input into the trained target detection model to obtain the initial detection boxes of the whole root and rhizome medicinal material, stem, root and root-stem transition area;
[0010] The initial detection frames of the whole root and stem, root and root-stem transition zone of the root and stem medicinal material are optimized by multi-frame processing to obtain the optimized detection frames of each part of the root and stem medicinal material, as well as the corresponding outline and centroid.
[0011] Based on the optimized detection frames and centroids of each part of the root and rhizome medicinal materials, a geometric constraint strategy is used to infer and recover the undetected root and rhizome transition zone;
[0012] Based on the rhizome transition zone and the optimized stem and root, an iterative optimization strategy using rhizome-to-rhizome pruning and weighted least squares method was employed to estimate the attitude angles of rhizome medicinal materials. ;
[0013] Calculated attitude angles The structure and geometric features of the rhizome transition zone are used to locate the cutting points of rhizome medicinal materials.
[0014] Furthermore, the step of acquiring training data specifically involves:
[0015] Data augmentation processing was performed on the RGB images of root and rhizome medicinal material samples at various stages, including: cropping, translation, rotation, mirroring, color change, noise addition, and blurring.
[0016] Furthermore, the step of optimizing the initial detection frames for the whole, stem, root, and root-stem transition zone of the rhizome medicinal material through multi-frame optimization, to obtain the optimized detection frames, specifically involves:
[0017] For the image region corresponding to the initial detection box of each part, filtering, color space conversion, binarization and morphological operations are performed in sequence. The largest connected component is extracted as the contour of the part, and a new bounding rectangle is calculated based on the contour as the optimized detection box, and the centroid of the contour is calculated.
[0018] Furthermore, the step of restoring the root-stem transition zone using a geometric constraint strategy includes:
[0019] When the root-stem transition zone is not detected, calculate the corresponding line segments of the overlapping area of the biaxial regions of the parallel line segments of the detection boxes of the root and stem parts respectively.
[0020] The set of line segments with the largest overlap of the biaxial axes of the two parallel line segments is selected, and the overlapping line segment region with the smallest distance between the two sides is calculated as the initial detection box of the root transition zone.
[0021] The size of the detection box is dynamically adjusted based on the aspect ratio of the initial detection box in the root-stem transition zone, and the adjusted region is optimized by image preprocessing in multi-box optimization to obtain the contour and centroid feature information of the root-stem transition zone.
[0022] Furthermore, the step of iteratively optimizing the rhizome-to-rhizome pruning algorithm to obtain the posture angle of rhizome medicinal materials is divided into diagonal pruning and line pruning, which includes:
[0023] The centroids of the roots, stems, and root-stem transition zones of root and rhizome medicinal materials are obtained from the contours within the current detection frame. Weighted least squares is then used to calculate the initial attitude lines and attitude angles of these three centroids. The weighted least squares expression is as follows: , Where 𝑦 represents the centroidal vector corresponding to the stem, the transition zone between stem and rhizome, and the root of rhizome-type medicinal materials. This is a weight matrix. When the weight of the stem is zero, it is equivalent to the line passing through the centroid of the transition zone between the root and the stem. For the weighted sum of squared residuals, For the weighted least squares estimator, by right Taking the partial derivative and setting it to zero, we get... is the angle of the fitted line, and b is the intercept of the fitted line;
[0024] The target point is the centroid of the rhizome transition zone. The origin point is the centroid of the stem or root. Obtain the direction vector from the source point to the target point. The acute angle corresponding to the direction vector is ,when When the value exceeds a preset threshold, a diagonal cropping algorithm is used to obtain the pose angle of the current rhizome medicinal herb class. ,when When the angle is less than or equal to a preset threshold, use line clipping to obtain the pose angle of the current rhizome medicinal herb class. ;
[0025] The diagonal cropping algorithm, when When the value is greater than a preset threshold, the direction vector is calculated. Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;calculate Find the nearest endpoint of the line segment and obtain the diagonal point of that endpoint in the target box. Obtain the clipping direction vector Based on the clipping direction vector Using the clipping factor λ, calculate the new detection box for the root or stem after diagonal clipping. The expression for calculating the new detection box is: The new detection bounding box for the root or stem is used to recalculate the outline and centroid of the stem or root within that region, and the resulting new centroid serves as the source point for the next iteration. ;
[0026] The line clipping algorithm, when When the value is less than or equal to a preset threshold, calculate the direction vector. Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;get The edges of the target detection box opposite side According to the edge opposite sides Based on the positional relationship and the clipping factor λ, calculate the new detection box for the root or stem after horizontal or vertical clipping. Then, based on the new clipped detection box, recalculate the contour and centroid of the stem or root within that region. The resulting new centroid serves as the source point for the next iteration. ;
[0027] The new centroid of the root or stem after trimming is obtained through the diagonal trimming or line trimming algorithm. Centroid of the target point in the transition zone with the rhizome Calculate the new source point With the target point Angle of direction vector By adjusting the initial (after the previous step's cropping) direction vector angle Angle with the current cropped direction vector The absolute value of the difference is used as the condition for judging the local clipping iteration. The local clipping iteration condition is expressed as follows: Furthermore, to prevent over-pruning of the roots or stems, the maximum number of pruning operations is limited. When the root or stem is partially pruned, the iteration condition is... If the number of cuts is less than a preset threshold, or if the number of cuts at the root or stem exceeds the maximum number of cuts, then the reverse cutting iteration for that part will stop; otherwise... Repeat the local rhizome-to-rhizome pruning iteration for this part;
[0028] The diagonal or linear clipping algorithm is used to obtain the centroid of the new source point in the root or stem after clipping and the centroid of the target point in the root-stem transition zone. The weighted least squares method is then used to fit the posture line of the clipped root and stem medicinal material. By adjusting the initial (after the previous step's cropping) attitude angles Compared to the current cropped pose angle The absolute value of the difference is used as the global condition for the clipping iteration. The expression for the global clipping iteration condition is: When the global pruning iteration condition When the angle is less than the preset threshold, the rhizome-to-rhizome pruning iteration is completed to obtain the final posture angle of the rhizome medicinal material. and the centroid of the transition zone through the rhizome The attitude angle is unit direction vector ,otherwise Repeat the rootstock pruning iteration.
[0029] Furthermore, the posture angle of the rhizome medicinal materials is adopted. The step of determining the cutting point of rhizome-type medicinal materials by analyzing the structure and geometric features of the transition zone with the rhizome is as follows:
[0030] The ratio of the optimized detection box area to the area of its internal contour in the root transition region is calculated as the area scaling factor. The expression for the area scaling factor is: According to the area ratio factor Directional vector of the posture of rhizome medicinal materials The cutting point of rhizome-type medicinal materials is calculated, and the expression for calculating the cutting point of rhizome-type medicinal materials is as follows: ,in Let be the centroid vector of the optimized root-stem transition zone. This is a scaling factor used to adjust the centroid correction step size in the root-stem transition zone. Execute the cut point expression within the preset range; otherwise... Finally, the cutting points of rhizome-type medicinal materials are obtained.
[0031] This invention also provides a system for estimating the posture and locating the cutting point of rhizomes, comprising the following modules:
[0032] The image acquisition and annotation module is used to acquire and annotate images of root and rhizome medicinal materials to build a training dataset.
[0033] The object detection module is used to load the trained object detection model, perform inference on the image of root and stem medicinal materials to be processed, and output the initial detection boxes for the whole, stem, root, and root-stem transition area.
[0034] The multi-frame optimization module is used to perform image preprocessing on the initial detection frames of each part of the root and rhizome medicinal materials, and output the optimized detection frames and their contours and centroids.
[0035] The geometric constraint module is used to restore the root-stem transition area based on the spatial relationship between the root and the stem when the root-stem transition area is not successfully detected and thus missing.
[0036] The attitude estimation module is used to estimate the attitude of rhizome medicinal materials based on the rhizome pruning strategy and weighted least squares iterative optimization.
[0037] The cutting point positioning module is used to accurately locate the cutting point of rhizome medicinal materials based on their posture angle and the structural characteristics of the rhizome transition zone.
[0038] Furthermore, the multi-frame optimization module performs the following specific operations:
[0039] For the image region corresponding to the initial detection box of each part, filtering, color space conversion, binarization and morphological operations are performed in sequence. The largest connected component is extracted as the contour of the part, and a new bounding rectangle is calculated based on the contour as the optimized detection box, and the centroid of the contour is calculated.
[0040] Furthermore, the geometric constraint module includes:
[0041] The region retrieval unit is used to calculate the overlapping area of the corresponding parallel line segments of the detection boxes of the root and stem parts when the root-stem transition zone is not detected.
[0042] The region recovery unit is used to filter out the set of line segments with the largest overlap of the corresponding axes of the two parallel line segments, and calculate the overlapping line segment region with the smallest distance between the two sides, as the initial detection box of the root transition zone.
[0043] The feature extraction unit is used to dynamically adjust the size of the detection box according to the aspect ratio of the initial detection box in the root-stem transition zone, and to perform image preprocessing optimization in multi-box optimization on the adjusted region to obtain the contour and centroid feature information of the root-stem transition zone.
[0044] Furthermore, the attitude estimation module includes:
[0045] The initial pose acquisition unit is used to obtain the centroids from the contours of the roots, stems, and root-stem transition areas in the current detection box of the root and stem medicinal materials. It then uses weighted least squares to calculate the initial pose lines and pose angles of the three centroids. The weighted least squares expression is as follows: , Where 𝑦 represents the centroidal vector corresponding to the stem, the transition zone between stem and rhizome, and the root of rhizome-type medicinal materials. This is a weight matrix. When the weight of the stem is zero, it is equivalent to the line passing through the centroid of the transition zone between the root and the stem. For the weighted sum of squared residuals, For the weighted least squares estimator, by right Taking the partial derivative and setting it to zero, we get... is the angle of the fitted line, and b is the intercept of the fitted line;
[0046] The trimming judgment unit is used to target the centroid of the root-stem transition zone. The origin point is the centroid of the stem or root. Obtain the direction vector from the source point to the target point. The acute angle corresponding to the direction vector is ,when When the value exceeds a preset threshold, a diagonal cropping algorithm is used to obtain the pose angle of the current rhizome medicinal herb class. ,when When the angle is less than or equal to a preset threshold, use line clipping to obtain the pose angle of the current rhizome medicinal herb class. ;
[0047] Diagonal clipping unit, used when When the value is greater than a preset threshold, the direction vector is calculated. Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;calculate Find the nearest endpoint of the line segment and obtain the diagonal point of that endpoint in the target box. Obtain the clipping direction vector Based on the clipping direction vector Using the clipping factor λ, calculate the new detection box for the root or stem after diagonal clipping. The expression for calculating the new detection box is: The new detection bounding box for the root or stem is used to recalculate the outline and centroid of the stem or root within that region, and the resulting new centroid serves as the source point for the next iteration. ;
[0048] Line trimming unit, used when When the value is less than or equal to a preset threshold, calculate the direction vector. Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;get The edges of the target detection box opposite side According to the edge opposite sides Based on the positional relationship and the clipping factor λ, calculate the new detection box for the root or stem after horizontal or vertical clipping. Then, based on the new clipped detection box, recalculate the contour and centroid of the stem or root within that region. The resulting new centroid serves as the source point for the next iteration. ;
[0049] The local iteration judgment unit is used to obtain the new source point centroid of the root or stem after trimming through the diagonal trimming or line trimming unit. Centroid of the target point in the transition zone with the rhizome Calculate the new source point With the target point Angle of direction vector By adjusting the initial (after the previous step's cropping) direction vector angle Angle with the current cropped direction vector The absolute value of the difference is used as the condition for judging the local clipping iteration. The local clipping iteration condition is expressed as follows: Furthermore, to prevent over-cutting of the roots or stems, the maximum number of cuts is limited. When the root or stem is partially pruned, the iteration condition is... If the number of cuts is less than a preset threshold, or if the number of cuts at the root or stem exceeds the maximum number of cuts, then the reverse cutting iteration for that part will stop; otherwise... Repeat the local rhizome-to-rhizome pruning iteration for this part;
[0050] The global iterative judgment unit is used to obtain the centroid of the new source point of the root or stem after trimming and the centroid of the target point in the root-stem transition zone through the diagonal trimming or line trimming unit, and to fit the posture line of the trimmed root and stem medicinal material using the weighted least squares method. By adjusting the initial (after the previous step's cropping) attitude angles Compared to the current cropped pose angle The absolute value of the difference is used as the global condition for the clipping iteration. The expression for the global clipping iteration condition is: When the global pruning iteration condition When the angle is less than the preset threshold, the rhizome-to-rhizome pruning iteration is completed to obtain the final posture angle of the rhizome medicinal material. and the centroid of the transition zone through the rhizome The attitude angle is unit direction vector ,otherwise Repeat the rootstock pruning iteration.
[0051] Furthermore, the cutting point positioning module performs the following specific operations:
[0052] The ratio of the optimized detection box area to the area of its internal contour in the root transition region is calculated as the area scaling factor. The expression for the area scaling factor is: According to the area ratio factor Directional vector of the posture of rhizome medicinal materials The cutting point of rhizome-type medicinal materials is calculated, and the expression for calculating the cutting point of rhizome-type medicinal materials is as follows: ,in Let be the centroid vector of the optimized root-stem transition zone. This is a scaling factor used to adjust the centroid correction step size in the root-stem transition zone. Execute the cut-point expression within the preset range; otherwise, execute... Finally, the cutting points of rhizome-type medicinal materials are obtained.
[0053] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described method for locating the attitude of rhizomes and the cutting point of medicinal materials.
[0054] A computer-readable storage medium storing a computer program that, when executed, implements the above-described method for attitude estimation and cutting point localization of rhizomes and medicinal materials.
[0055] The beneficial effects of the system of this invention are as follows: In the multi-frame optimization module, image noise is effectively filtered out, improving the accuracy of detection frames for various parts of rhizome medicinal materials and providing a high-quality data foundation for subsequent geometric inference; In the geometric constraint module, the spatial positional relationship between the root and the stem is used to recover the occluded or weakly featured transition area of the rhizome, significantly improving the detection rate in this area and solving the performance bottleneck of the single target detection model in this area; In the pose estimation module, the combination of the rhizome-to-root cropping algorithm and the weighted least squares fitting algorithm iteratively optimizes and gradually approximates the main body pose, exhibiting strong robustness to interference factors such as roots, stems and leaves, and achieving high-precision pose estimation; In the cutting point localization module, the cutting point is associated with the pose of the rhizome medicinal material and the stable structural features at the intersection, achieving lightweight and high-precision cutting point localization, providing reliable technical support for the automated rhizome separation of rhizome medicinal materials. Attached Figure Description
[0056] Figure 1 This is a flowchart of the method for estimating the pose of rhizomes and locating cutting points in an embodiment of the present invention;
[0057] Figure 2 This is a diagram illustrating the data enhancement effect of the present invention;
[0058] Figure 3 This is a first schematic diagram showing the output results of the method for estimating the pose of rhizomes and locating cutting points in this invention;
[0059] Figure 4 This is a second schematic diagram showing the output results of the method for estimating the pose of rhizomes and locating cutting points in this invention. Detailed Implementation
[0060] The technology in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings.
[0061] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0062] Example 1
[0063] Reference Figure 1 This invention relates to a method for estimating the pose of rhizome medicinal materials and locating cutting points. The method includes:
[0064] Step 101: Obtain the dataset, the training data including RGB images of root and rhizome medicinal material samples at each stage and region labels of the whole, stem, root, and root-rhizome transition zone in the RGB images of root and rhizome medicinal material samples at each stage;
[0065] Step 102: Construct a YOLO model and train it on root and rhizome medicinal materials. The model includes various parameters and open-source versions of the YOLO model, including but not limited to YOLOv5, YOLOv8, and YOLO11.
[0066] Step 103: Use the training data to train the YOLO model to obtain the identification and localization model of each part of the root and rhizome medicinal material;
[0067] Step 104: Input the RGB image of the root and rhizome medicinal material to be identified into the identification and localization model of each part of the root and rhizome medicinal material;
[0068] Step 105: Using a multi-frame optimization strategy, preprocess the images of each part of the root and stem to remove image noise within the detection area of each part, obtain the contour of the largest connected region within the optimized detection area, and calculate the centroid of the contour of the largest connected region of each part of the root and stem medicinal material.
[0069] Step 106: Using a geometric constraint strategy, the detection box of the root-stem transition region, which was lost during successful detection, is recovered by utilizing the spatial relationship between the root and stem. The contour of the largest connected region of the root-stem transition region is obtained, and the centroid of the contour of the largest connected region of the root-stem transition region is calculated.
[0070] Step 107: Based on the root-rhizome transition zone, the optimized detection block diagram and centroid of the stem and root, the root-rhizome opposing pruning algorithm and the weighted least squares fitting algorithm are used to iteratively fit the attitude line of the root-rhizome medicinal material, and finally obtain the attitude angle α of the root-rhizome medicinal material.
[0071] Step 108: Based on the aforementioned attitude angle and the structural and geometric features of the root-rhizome transition zone, the cutting point of the root-rhizome medicinal material is located, ultimately obtaining the cutting point of the root-rhizome medicinal material. .
[0072] As a further preferred embodiment, step 101 specifically includes: RGB images of root and rhizome medicinal material samples at various stages were collected. The LabelImg tool was used to label the entire root and rhizome of each stage of the medicinal material in the images, as well as the root, stem, and root-rhizome transition zone. Figure 2As shown, data augmentation processing is performed on the RGB images of root and rhizome medicinal material samples at each stage. The data augmentation processing includes: cropping, translation, rotation, mirroring, color change, noise addition, and blurring.
[0073] As a further preferred embodiment, step 105 includes:
[0074] For the image region corresponding to the initial detection box of each part, Gaussian blur and bilateral filtering, HSV color space conversion, binarization and erosion and dilation morphological operations are performed in sequence. The largest connected component is extracted as the contour of the part, and a new bounding rectangle is calculated based on the contour as the optimized detection box, and the centroid of the contour is calculated.
[0075] As a further preferred embodiment, step 106 includes:
[0076] When the root-stem transition zone is not detected, the IOU distance between any two parallel line segments at the corresponding boundaries of the detection boxes for the root and stem parts is calculated, and the expression is as follows: ;
[0077] The region with the smallest IOU distance, the largest overlap, and the smallest spacing between its two corresponding edges is selected as the initial detection box for the root-stem transition zone. The expression is: ;
[0078] The size of the detection box is dynamically adjusted based on the aspect ratio of the initial detection box in the root-stem transition zone, and the adjusted region is optimized by image preprocessing in multi-box optimization to obtain the contour and centroid feature information of the root-stem transition zone.
[0079] As a further preferred embodiment, step 107 includes:
[0080] The centroids of the roots, stems, and root-stem transition zones of root and rhizome medicinal materials are obtained from the contours within the current detection frame. Weighted least squares is then used to calculate the initial attitude lines and attitude angles of these three centroids. The weighted least squares expression is as follows: , Where 𝑦 represents the centroidal vector corresponding to the stem, the transition zone between stem and rhizome, and the root of rhizome-type medicinal materials. This is a weight matrix. When the weight of the stem is zero, it is equivalent to the line passing through the centroid of the transition zone between the root and the stem. For the weighted sum of squared residuals, For the weighted least squares estimator, by right Taking the partial derivative and setting it to zero, we get... is the angle of the fitted line, and b is the intercept of the fitted line;
[0081] The target point is the centroid of the rhizome transition zone. The origin point is the centroid of the stem or root. Obtain the direction vector from the source point to the target point. The acute angle corresponding to the direction vector is ,when At that time, the diagonal clipping algorithm is used to obtain the pose angle of the current rhizome medicinal material class. ,when At that time, use line clipping to obtain the pose angle of the current rhizome medicinal material. ;
[0082] The diagonal cropping algorithm, when When, calculate the direction vector Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;calculate Find the nearest endpoint of the line segment and obtain the diagonal point of that endpoint in the target box. Obtain the clipping direction vector Based on the clipping direction vector Using the clipping factor λ, calculate the new detection box for the root or stem after diagonal clipping. The expression for calculating the new detection box is: The new detection bounding box for the root or stem is used to recalculate the outline and centroid of the stem or root within that region, and the resulting new centroid serves as the source point for the next iteration. ;
[0083] The line clipping algorithm, when When calculating the direction vector Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;get The edges of the target detection box opposite side According to the edge opposite sides Based on the positional relationship and the clipping factor λ, calculate the new detection box for the root or stem after horizontal or vertical clipping. Then, based on the new clipped detection box, recalculate the contour and centroid of the stem or root within that region. The resulting new centroid serves as the source point for the next iteration. ;
[0084] The new centroid of the root or stem after trimming is obtained through the diagonal trimming or line trimming algorithm. Centroid of the target point in the transition zone with the rhizome Calculate the new source point With the target point Angle of direction vector By adjusting the initial (after the previous step's cropping) direction vector angle Angle with the current cropped direction vector The absolute value of the difference is used as the condition for judging the local clipping iteration. The local clipping iteration condition is expressed as follows: Furthermore, to prevent over-cutting of the roots or stems, the maximum number of cuts is limited. When the root or stem is partially pruned, the iteration condition is... Or the number of cuts to the root or stem exceeds the maximum number of cuts. When this happens, the opposite clipping iteration stops; otherwise... Repeat the local rhizome-to-rhizome pruning iteration for this part;
[0085] The method involves using diagonal or linear clipping algorithms to obtain the centroid of the new source point in the root or stem after clipping and the centroid of the target point in the root-stem transition zone. The weighted least squares method is then used to fit the posture line of the clipped root and stem medicinal material. By adjusting the initial (after the previous step's cropping) attitude angles Compared to the current cropped pose angle The absolute value of the difference is used as the global condition for the clipping iteration. The expression for the global clipping iteration condition is: When the global pruning iteration condition The rhizome-to-rhizome pruning iteration was completed to obtain the final posture angle of the rhizome medicinal materials. and the centroid of the transition zone through the rhizome The attitude angle is unit direction vector ,otherwise Repeat the rootstock pruning iteration.
[0086] As a further preferred embodiment, step 108 specifically includes:
[0087] The ratio of the optimized detection box area to the area of its internal contour in the root transition region is calculated as the area scaling factor. The expression for the area scaling factor is: According to the area ratio factor Directional vector of the posture of rhizome medicinal materials The cutting point of rhizome-type medicinal materials is calculated, and the expression for calculating the cutting point of rhizome-type medicinal materials is as follows: ,in Let be the centroid vector of the optimized root-stem transition zone. Scaling factor 0.28, when Execute the above cut-point expression if it is executed; otherwise... This yields the final cutting points for root and rhizome medicinal materials.
[0088] Example 2
[0089] This invention relates to a method for estimating the pose of rhizome medicinal materials and locating cutting points. The method includes the following modules:
[0090] The image acquisition and annotation module is used to acquire and annotate images of root and rhizome medicinal materials to build a training dataset.
[0091] The object detection module is used to load the trained object detection model, perform inference on the image of root and stem medicinal materials to be processed, and output the initial detection boxes for the whole, stem, root, and root-stem transition area.
[0092] The multi-frame optimization module is used to perform image preprocessing on the initial detection frames of each part of the root and rhizome medicinal materials, and output the optimized detection frames and their contours and centroids.
[0093] The geometric constraint module is used to restore the root-stem transition area based on the spatial relationship between the root and the stem when the root-stem transition area is not successfully detected and thus missing.
[0094] The attitude estimation module is used to estimate the attitude of rhizome medicinal materials based on the rhizome pruning strategy and weighted least squares iterative optimization.
[0095] The cutting point positioning module is used to accurately locate the cutting point of rhizome medicinal materials based on their posture angle and the structural characteristics of the rhizome transition zone.
[0096] Furthermore, the multi-frame optimization module performs the following specific operations:
[0097] For the image region corresponding to the initial detection box of each part, filtering, color space conversion, binarization and morphological operations are performed in sequence. The largest connected component is extracted as the contour of the part, and a new bounding rectangle is calculated based on the contour as the optimized detection box, and the centroid of the contour is calculated.
[0098] Furthermore, the geometric constraint module includes:
[0099] The region retrieval unit is used to calculate the overlapping area of the corresponding parallel line segments of the detection boxes of the root and stem parts when the root-stem transition zone is not detected.
[0100] The region recovery unit is used to filter out the set of line segments with the largest overlap of the corresponding axes of the two parallel line segments, and calculate the overlapping line segment region with the smallest distance between the two sides, as the initial detection box of the root transition zone.
[0101] The feature extraction unit is used to dynamically adjust the size of the detection box according to the aspect ratio of the initial detection box in the root-stem transition zone, and to perform image preprocessing optimization in multi-box optimization on the adjusted region to obtain the contour and centroid feature information of the root-stem transition zone.
[0102] Furthermore, the attitude estimation module includes:
[0103] The initial pose acquisition unit is used to obtain the centroids from the contours of the roots, stems, and root-stem transition areas in the current detection box of the root and stem medicinal materials. It then uses weighted least squares to calculate the initial pose lines and pose angles of the three centroids. The weighted least squares expression is as follows: , Where 𝑦 represents the centroidal vector corresponding to the stem, the transition zone between stem and rhizome, and the root of rhizome-type medicinal materials. For example, the weight matrix, When the weight of the stem is zero, it is equivalent to the line passing through the centroid of the transition zone between the root and the rootstock. For the weighted sum of squared residuals, For the weighted least squares estimator, by right Taking the partial derivative and setting it to zero, we get... is the angle of the fitted line, and b is the intercept of the fitted line;
[0104] The trimming judgment unit is used to target the centroid of the root-stem transition zone. The origin point is the centroid of the stem or root. Obtain the direction vector from the source point to the target point. The acute angle corresponding to the direction vector is ,when At that time, the diagonal clipping algorithm is used to obtain the pose angle of the current rhizome medicinal material class. ,when At that time, use line clipping to obtain the pose angle of the current rhizome medicinal material. ;
[0105] Diagonal clipping unit, used when When, calculate the direction vector Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;calculate Find the nearest endpoint of the line segment and obtain the diagonal point of that endpoint in the target box. Obtain the clipping direction vector Based on the clipping direction vector Using the clipping factor λ, calculate the new detection box for the root or stem after diagonal clipping. The expression for calculating the new detection box is: The new detection bounding box for the root or stem is used to recalculate the outline and centroid of the stem or root within that region, and the resulting new centroid serves as the source point for the next iteration. ;
[0106] Line trimming unit, used when When calculating the direction vector Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;get The edges of the target detection box opposite side According to the edge opposite sides Based on the positional relationship and the clipping factor λ, calculate the new detection box for the root or stem after horizontal or vertical clipping. Then, based on the new clipped detection box, recalculate the contour and centroid of the stem or root within that region. The resulting new centroid serves as the source point for the next iteration. ;
[0107] The local iteration judgment unit is used to obtain the new source point centroid of the root or stem after trimming through the diagonal trimming or line trimming unit. Centroid of the target point in the transition zone with the rhizome Calculate the new source point With the target point Angle of direction vector By adjusting the initial (after the previous step's cropping) direction vector angle Angle with the current cropped direction vector The absolute value of the difference is used as the condition for judging the local clipping iteration. The local clipping iteration condition is expressed as follows: Furthermore, to prevent over-cutting of the roots or stems, the maximum number of cuts is limited. When the root or stem is partially pruned, the iteration condition is... Or the number of cuts to the root or stem exceeds the maximum number of cuts. When this happens, the opposite clipping iteration stops; otherwise... Repeat the local rhizome-to-rhizome pruning iteration for this part;
[0108] The global iterative judgment unit is used to obtain the centroid of the new source point of the root or stem after trimming and the centroid of the target point in the root-stem transition zone through the diagonal trimming or line trimming unit, and to fit the posture line of the trimmed root and stem medicinal material using the weighted least squares method. By adjusting the initial (after the previous step's cropping) attitude angles Compared to the current cropped pose angle The absolute value of the difference is used as the global condition for the clipping iteration. The expression for the global clipping iteration condition is: When the global pruning iteration condition At that time, the rhizome reciprocating pruning iteration was completed to obtain the final posture angle of the rhizome medicinal material. and the centroid of the transition zone through the rhizome The attitude angle is unit direction vector ,otherwise Repeat the rootstock pruning iteration.
[0109] Furthermore, the cutting point positioning module performs the following specific operations:
[0110] The ratio of the optimized detection box area to the area of its internal contour in the root transition region is calculated as the area scaling factor. The expression for the area scaling factor is: According to the area ratio factor Directional vector of the posture of rhizome medicinal materials The cutting point of rhizome-type medicinal materials is calculated, and the expression for calculating the cutting point of rhizome-type medicinal materials is as follows: ,in Let be the centroid vector of the optimized root-stem transition zone. This is a scaling factor used to adjust the centroid correction step size in the root-stem transition zone. Execute the cut-point expression within the preset range; otherwise, execute... Finally, the cutting points of rhizome-type medicinal materials were obtained. .
[0111] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described method for locating the attitude of rhizomes and the cutting point of medicinal materials.
[0112] A computer-readable storage medium storing a computer program that, when executed, implements the above-described method for attitude estimation and cutting point localization of rhizomes and medicinal materials.
[0113] The present invention has the following advantages:
[0114] 1) By using multi-frame optimization and constraint strategies, the detection rate of the root-stem transition region of root and stem medicinal materials can be improved, which can avoid the problem of missed detection of root and stem medicinal materials due to the undetected root-stem transition region that is significantly obscured or has weak features.
[0115] 2) Based on the characteristics of rhizome medicinal materials, a rhizome-to-rhizome pruning algorithm and a weighted least squares fitting algorithm are used to estimate the posture of rhizome medicinal materials, achieving fast and high-precision posture estimation of rhizome medicinal materials.
[0116] 3) By utilizing the estimated posture of rhizome medicinal materials and the geometric structural features of the rhizome transition zone, a lightweight mathematical model for locating cutting points of rhizome medicinal materials is established to achieve rapid, stable, and high-precision locating of cutting points of rhizome medicinal materials.
[0117] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. A method for posture estimation and cutting point localization of rhizome medicinal materials, characterized in that, The method for estimating the posture and locating the cutting point of rhizome medicinal materials includes: Acquire training data, which includes RGB images of root and rhizome medicinal material samples and their overall, stem, root, and root-rhizome transition zone region labels; A YOLO object detection model was constructed and trained on rhizome-type medicinal materials. This model was then used to identify different parts of the rhizome-type medicinal materials. Input the image to be processed into the trained model to obtain the initial detection boxes for each part; The initial detection frames of the whole root and stem, root and root-stem transition zone of the root and stem medicinal material are optimized by multi-frame processing to obtain the optimized detection frames of each part of the root and stem medicinal material, as well as the corresponding outline and centroid. Based on the optimized detection frames and centroids of each part of the root and rhizome medicinal materials, a geometric constraint strategy is used to infer and recover the undetected root and rhizome transition zone; Based on the rhizome transition zone and the optimized stem and root, an iterative optimization strategy using rhizome-to-rhizome pruning and weighted least squares method was employed to estimate the attitude angles of rhizome medicinal materials. ; Based on the calculated attitude angle and the structural and geometric features of the rhizome transition zone, the cutting point is located for rhizome medicinal materials.
2. The method for estimating the posture and locating the cutting point of rhizomes according to claim 1 of the patent is characterized in that, The multi-frame optimization process specifically includes: Data augmentation processing was performed on the RGB images of root and rhizome medicinal material samples at various stages, including: cropping, translation, rotation, mirroring, color change, noise addition, and blurring.
3. The method for estimating the posture and locating the cutting point of rhizomes according to claim 1 of the patent is characterized in that, The geometric constraint strategy includes: When the root-stem transition zone is not detected, calculate the corresponding line segments of the overlapping area of the biaxial regions of the parallel line segments of the detection boxes of the root and stem parts respectively. The set of line segments with the largest overlap of the biaxial axes of the two parallel line segments is selected, and the overlapping line segment region with the smallest distance between the two sides is calculated as the initial detection box of the root transition zone.
4. The size of the detection box is dynamically adjusted according to the aspect ratio of the initial detection box in the root-stem transition zone, and the adjusted region is optimized by image preprocessing in multi-box optimization to obtain the contour and centroid feature information of the root-stem transition zone.
5. The method for estimating the posture and locating the cutting point of rhizomes according to claim 1, characterized in that, The steps of the rhizome-to-rhizome pruning strategy and weighted least squares method for estimating attitude angles include: The centroids of the roots, stems, and root-stem transition zones of root and rhizome medicinal materials are obtained from the contours within the current detection frame. Weighted least squares is then used to calculate the initial attitude lines and attitude angles of these three centroids. The weighted least squares expression is as follows: , Where 𝑦 represents the centroidal vector corresponding to the stem, the transition zone between stem and rhizome, and the root of rhizome-type medicinal materials. This is a weight matrix. When the weight of the stem is zero, it is equivalent to the line passing through the centroid of the transition zone between the root and the stem. For the weighted sum of squared residuals, For the weighted least squares estimator, by right Taking the partial derivative and setting it to zero, we get... is the angle of the fitted line, and b is the intercept of the fitted line; The target point is the centroid of the rhizome transition zone. The origin point is the centroid of the stem or root. Obtain the direction vector from the source point to the target point. The acute angle corresponding to the direction vector is ,when When the value exceeds a preset threshold, a diagonal cropping algorithm is used to obtain the pose angle of the current rhizome medicinal herb class. ,when When the angle is less than or equal to a preset threshold, use line clipping to obtain the pose angle of the current rhizome medicinal herb class. ; The diagonal cropping algorithm, when When the value is greater than a preset threshold, the direction vector is calculated. Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;calculate Find the nearest endpoint of the line segment and obtain the diagonal point of that endpoint in the target box. Obtain the clipping direction vector Based on the clipping direction vector Using the clipping factor λ, calculate the new detection box for the root or stem after diagonal clipping. The expression for calculating the new detection box is: The new detection bounding box for the root or stem is used to recalculate the outline and centroid of the stem or root within that region, and the resulting new centroid serves as the source point for the next iteration. ; The line clipping algorithm, when When the value is less than or equal to a preset threshold, calculate the direction vector. Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;calculate The edges of the target detection box opposite side According to the edge opposite sides Based on the positional relationship and the clipping factor λ, calculate the new detection box for the root or stem after horizontal or vertical clipping. Then, based on the new clipped detection box, recalculate the contour and centroid of the stem or root within that region. The resulting new centroid serves as the source point for the next iteration. ; The new centroid of the root or stem after trimming is obtained through the diagonal trimming or line trimming algorithm. Centroid of the target point in the transition zone with the rhizome Calculate the new source point With the target point Angle of direction vector By adjusting the initial (after the previous step's cropping) direction vector angle Angle with the current cropped direction vector The absolute value of the difference is used as the condition for judging the local clipping iteration. The local clipping iteration condition is expressed as follows: Furthermore, to prevent over-pruning of the roots or stems, the maximum number of pruning operations is limited. When the root or stem is partially pruned, the iteration condition is... If the number of cuts is less than a preset threshold, or if the number of cuts at the root or stem exceeds the maximum number of cuts, then the reverse cutting iteration for that part will stop; otherwise... Repeat the local rhizome-to-rhizome pruning iteration for this part; The diagonal or linear clipping algorithm is used to obtain the centroid of the new source point in the root or stem after clipping and the centroid of the target point in the root-stem transition zone. The weighted least squares method is then used to fit the posture line of the clipped root and stem medicinal material. By adjusting the initial (after the previous step's cropping) attitude angles Compared to the current cropped pose angle The absolute value of the difference is used as the global condition for the clipping iteration. The expression for the global clipping iteration condition is: When the global pruning iteration condition When the angle is less than the preset threshold, the rhizome-to-rhizome pruning iteration is completed to obtain the final posture angle of the rhizome medicinal material. and the centroid of the transition zone through the rhizome The attitude angle is unit direction vector ,otherwise Repeat the rootstock pruning iteration.
6. The method for locating the pose and cutting point of rhizomes as described in claim 1, characterized in that, The specific steps for locating the cutting points of rhizome-type medicinal materials are as follows: The ratio of the optimized detection box area to the area of its internal contour in the root transition region is calculated as the area scaling factor. The expression for the area scaling factor is: According to the area ratio factor Directional vector of the posture of rhizome medicinal materials The cutting point of rhizome-type medicinal materials is calculated, and the expression for calculating the cutting point of rhizome-type medicinal materials is as follows: ,in Let be the centroid vector of the optimized root-stem transition zone. This is a scaling factor used to adjust the centroid correction step size in the root-stem transition zone. Execute the cut point expression within the preset range; otherwise... Finally, the cutting points of rhizome-type medicinal materials are obtained.
7. A system for estimating the posture and locating the cutting point of rhizome medicinal materials, characterized in that, Includes the following modules The image acquisition and annotation module is used to construct the training dataset; The object detection module is used to load the trained object detection model, perform inference on the image of root and stem medicinal materials to be processed, and output the initial detection boxes for the whole, stem, root, and root-stem transition area. The multi-frame optimization module is used to perform image preprocessing on the initial detection frames of each part of the root and rhizome medicinal materials, and output the optimized detection frames and their contours and centroids. The geometric constraint module is used to restore the root-stem transition area based on the spatial relationship between the root and the stem when the root-stem transition area is not successfully detected and thus missing. The pose estimation module is used to estimate the pose of rhizome-type medicinal materials based on a rhizome-to-rhizome pruning strategy and weighted least squares iterative optimization. The cutting point positioning module is used to accurately locate the cutting point of rhizome medicinal materials based on their posture angle and the structural characteristics of the rhizome transition zone.
8. The method for locating the pose and cutting point of rhizomes according to claim 6, characterized in that, The multi-frame optimization module performs the following specific operations: For the image region corresponding to the initial detection box of each part, filtering, color space conversion, binarization and morphological operations are performed in sequence. The largest connected component is extracted as the contour of the part, and a new bounding rectangle is calculated based on the contour as the optimized detection box, and the centroid of the contour is calculated.
9. The method for locating the pose and cutting point of rhizomes according to claim 6, characterized in that, The geometric constraint module includes: The region retrieval unit is used to calculate the overlapping area of the corresponding parallel line segments of the detection boxes of the root and stem parts when the root-stem transition zone is not detected. The region recovery unit is used to filter out the set of line segments with the largest overlap of the corresponding axes of the two parallel line segments, and calculate the overlapping line segment region with the smallest distance between the two sides, as the initial detection box of the root transition zone.
10. Feature extraction unit, used to dynamically adjust the size of the detection box according to the aspect ratio of the initial detection box in the root-stem transition zone, and to perform image preprocessing optimization in multi-box optimization on the adjusted region to obtain the contour and centroid feature information of the root-stem transition zone.
11. The method for locating the pose and cutting point of rhizomes according to claim 6, characterized in that, The attitude estimation module includes: The initial pose acquisition unit is used to obtain the centroids from the contours of the roots, stems, and root-stem transition areas in the current detection box of the root and stem medicinal materials. It then uses weighted least squares to calculate the initial pose lines and pose angles of the three centroids. The weighted least squares expression is as follows: , Where 𝑦 represents the centroidal vector corresponding to the stem, the transition zone between stem and rhizome, and the root of rhizome-type medicinal materials. This is a weight matrix. When the weight of the stem is zero, it is equivalent to the line passing through the centroid of the transition zone between the root and the stem. For the weighted sum of squared residuals, For the weighted least squares estimator, by right Taking the partial derivative and setting it to zero, we get... is the angle of the fitted line, and b is the intercept of the fitted line; The trimming judgment unit is used to target the centroid of the root-stem transition zone. The origin point is the centroid of the stem or root. Obtain the direction vector from the source point to the target point. The acute angle corresponding to the direction vector is ,when When the value exceeds a preset threshold, a diagonal cropping algorithm is used to obtain the pose angle of the current rhizome medicinal herb class. ,when When the angle is less than or equal to a preset threshold, use line clipping to obtain the pose angle of the current rhizome medicinal herb class. ; Diagonal clipping unit, used when When the value is greater than a preset threshold, the direction vector is calculated. Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;calculate Find the nearest endpoint of the line segment and obtain the diagonal point of that endpoint in the target box. Obtain the clipping direction vector Based on the clipping direction vector Using the clipping factor λ, calculate the new detection box for the root or stem after diagonal clipping. The expression for calculating the new detection box is: The new detection bounding box for the root or stem is used to recalculate the outline and centroid of the stem or root within that region, and the resulting new centroid serves as the source point for the next iteration. ; Line trimming unit, used when When the value is less than or equal to a preset threshold, calculate the direction vector. Intersection of line segments corresponding to the four sides of the target detection box Obtain the distance to the target point The nearest intersection ;calculate The edges of the target detection box opposite side According to the edge opposite sides Based on the positional relationship and the clipping factor λ, calculate the new detection box for the root or stem after horizontal or vertical clipping. Then, based on the new clipped detection box, recalculate the contour and centroid of the stem or root within that region. The resulting new centroid serves as the source point for the next iteration. ; The local iteration judgment unit is used to obtain the new source point centroid of the root or stem after trimming through the diagonal trimming or line trimming unit. Centroid of the target point in the transition zone with the rhizome Calculate the new source point With the target point Angle of direction vector By adjusting the initial (after the previous step's cropping) direction vector angle Angle with the current cropped direction vector The absolute value of the difference is used as the condition for judging the local clipping iteration. The local clipping iteration condition is expressed as follows: Furthermore, to prevent over-cutting of the roots or stems, the maximum number of cuts is limited. When the root or stem is partially pruned, the iteration condition is... If the number of cuts is less than a preset threshold, or if the number of cuts at the root or stem exceeds the maximum number of cuts, then the reverse cutting iteration for that part will stop; otherwise... Repeat the local rhizome-to-rhizome pruning iteration for this part; The global iterative judgment unit is used to obtain the centroid of the new source point of the root or stem after trimming and the centroid of the target point in the root-stem transition zone through the diagonal trimming or line trimming unit, and to fit the posture line of the trimmed root and stem medicinal material using the weighted least squares method. By adjusting the initial (after the previous step's cropping) attitude angles Compared to the current cropped pose angle The absolute value of the difference is used as the global condition for the clipping iteration. The expression for the global clipping iteration condition is: When the global pruning iteration condition When the angle is less than the preset threshold, the rhizome-to-rhizome pruning iteration is completed to obtain the final posture angle of the rhizome medicinal material. and the centroid of the transition zone through the rhizome The attitude angle is unit direction vector ,otherwise Repeat the rootstock pruning iteration.
12. The method for locating the pose and cutting point of rhizomes according to claim 6, characterized in that, The cutting point positioning module performs the following specific operations: The ratio of the optimized detection box area to the area of its internal contour in the root transition region is calculated as the area scaling factor. The expression for the area scaling factor is: According to the area ratio factor Directional vector of the posture of rhizome medicinal materials The cutting point of rhizome-type medicinal materials is calculated, and the expression for calculating the cutting point of rhizome-type medicinal materials is as follows: ,in Let be the centroid vector of the optimized root-stem transition zone. This is a scaling factor used to adjust the centroid correction step size in the root-stem transition zone. Execute the cut-point expression within the preset range; otherwise, execute... Finally, the cutting points of rhizome-type medicinal materials are obtained.
13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the above-described method for locating the attitude of rhizomes and the cutting point of medicinal materials.
14. A computer-readable storage medium storing a computer program that, when executed, implements the above-described method for attitude estimation and cutting point localization of rhizomes and medicinal materials.