A method for estimating the posture of tea buds and leaves in a tea garden environment.

Abstract

This invention provides a method for estimating the pose of tea buds and leaves in a tea garden environment. The method includes acquiring multiple images of tea buds and leaves in a tea garden environment and dividing all images into a training set and a validation set. A pose estimation model to be trained is constructed, incorporating prior knowledge and establishing a loss function based on this prior knowledge. Based on the training set, the pose estimation model is trained using the loss function to obtain the tea bud and leaf pose estimation model. The validation set is then input into the tea bud and leaf pose estimation model to estimate the pose of the tea buds and leaves. This invention introduces prior knowledge to guide model training, generating a loss function that combines high-dimensional features with prior knowledge. The trained tea bud and leaf pose estimation model can achieve more accurate keypoint estimation. Using the tea bud and leaf pose estimation model to estimate the pose of tea buds and leaves in space can accurately provide the position and orientation of the tea buds and leaves, thereby improving the stability and accuracy of identifying the location of tea bud and leaf stem picking points.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for estimating the pose of tea buds and leaves in a tea garden environment. Background Technology

[0002] The harvesting of premium tea buds and leaves is highly dependent on manual labor, accounting for approximately 70% of the entire production process. It is one of the most labor-intensive and costly stages. Harvesting operations are significantly seasonal and time-sensitive, requiring a large influx of labor for repeated operations within a short period. This makes traditional manual harvesting methods unsustainable and severely restricts the sustainable development of the premium tea industry. Therefore, developing automated mechanized harvesting technology to gradually replace manual labor has become an urgent need and an inevitable trend for the industry to achieve efficient and sustainable development.

[0003] Accurate detection of the stem picking point is a crucial step in automated harvesting within a tea garden environment. Firstly, tea buds and leaves exhibit significant size variations and morphological diversity at different growth stages, making it difficult to consistently and accurately identify the stem picking point location. Secondly, frequent mutual shading between adjacent tea buds and leaves in the tea garden, as well as the overlapping of dense tea buds and leaves from the same viewpoint, greatly increases the difficulty of identifying the stem picking point.

[0004] Therefore, there is a need for a method that can estimate the posture of tea buds and leaves in space, accurately provide the position and orientation of tea buds and leaves, thereby improving the stability and accuracy of identifying the location of stem picking points. Summary of the Invention

[0005] To overcome the problems existing in related technologies, the purpose of this invention is to provide a method for estimating the posture of tea buds and leaves in a tea garden environment. This method can estimate the posture of tea buds and leaves in space, accurately provide the position and orientation of tea buds and leaves, thereby improving the stability and accuracy of identifying the location of stem picking points.

[0006] A method for estimating the posture of tea buds and leaves in a tea garden environment includes:

[0007] Multiple images of tea buds and leaves were collected in a tea garden environment, and all the tea bud and leaf images were divided into a training set and a validation set.

[0008] Construct a pose estimation model to be trained;

[0009] Prior knowledge is introduced, and a loss function is established based on the prior knowledge; wherein, the prior knowledge includes global geometric features and local gradient direction features;

[0010] Based on the training set, the loss function is used to train the pose estimation model to be trained, and the tea bud and leaf pose estimation model is obtained.

[0011] The validation set is input into the tea bud and leaf posture estimation model to estimate the posture of the tea buds and leaves.

[0012] In a preferred embodiment of the present invention, the introduction of prior knowledge includes:

[0013] Draw a priori feature map of key points in tea buds and leaves;

[0014] Geometric feature encoding is performed on the prior feature map of key points of tea buds and leaves to obtain global geometric features;

[0015] Gradient direction feature encoding is performed on the prior feature map of key points of tea buds and leaves to obtain local gradient direction features; the global geometric features and the local gradient direction features constitute prior knowledge.

[0016] In a preferred embodiment of the present invention, establishing the loss function based on the prior knowledge includes:

[0017] Based on the aforementioned prior knowledge, the loss function is established according to the following formula:

[0018]

[0019] Where λ is the scaling factor, K is the prior knowledge, W is the linear projection parameter of the high-dimensional features onto the fully connected layer, β is the regularization intensity ratio, Y is the keypoint pixel position predicted by the pose estimation model to be trained, H is the pixel position of the original label, and Loss is the loss function.

[0020] In a preferred embodiment of the present invention, the step of drawing a priori feature map of key points of tea buds and leaves includes:

[0021] Based on the morphological and structural characteristics of tea buds and leaves, key points of tea buds and leaves are marked; wherein, the key points of tea buds and leaves include the midpoint of the tea bud, the midpoint of the first tea leaf, the intersection of the tea bud and the first tea leaf, and the picking point of the stem with one bud and one leaf.

[0022] Connect the adjacent key points of the tea buds and leaves into a binary tree structure to obtain the prior feature map of the key points of the tea buds and leaves.

[0023] In a preferred embodiment of the present invention, the step of geometrically encoding the prior feature map of the key points of the tea buds and leaves to obtain global geometric features includes:

[0024] Hough line transform detection is performed on the prior feature map of key points of tea buds and leaves to obtain multiple line segments in the binary tree structure;

[0025] Encode all the line segments to obtain global geometric features.

[0026] In a preferred embodiment of the present invention, the step of performing gradient direction feature encoding on the prior feature map of key points of tea buds and leaves to obtain local gradient direction features includes:

[0027] The prior feature map of key points of tea buds and leaves is divided into multiple image blocks;

[0028] The horizontal and vertical gradients of the image patch are calculated using the Sobel operator;

[0029] Calculate the first gradient magnitude and first direction angle of the horizontal gradient, and calculate the second gradient magnitude and second direction angle of the vertical gradient;

[0030] Divide the area into M sub-intervals using the first direction angle and into N sub-intervals using the second direction angle;

[0031] The first gradient intensity magnitude is proportionally assigned to M sub-intervals to obtain a horizontal gradient histogram.

[0032] The second gradient intensity magnitude is proportionally assigned to N sub-intervals to obtain a vertical gradient histogram; wherein, the horizontal gradient histogram and the vertical gradient histogram constitute local gradient direction features.

[0033] In a preferred embodiment of the present invention, after obtaining the vertical gradient histogram, the method further includes:

[0034] The horizontal gradient histogram is normalized according to the following formula:

[0035]

[0036] Among them, f x For local horizontal gradient direction features, v x This is a horizontal gradient histogram. Let be the magnitude of the horizontal gradient histogram, and e be the adjustment parameter;

[0037] The vertical gradient histogram is normalized according to the following formula:

[0038]

[0039] Among them, f y For local vertical gradient direction features, v y This is a vertical gradient histogram. is the modulus of the vertical gradient histogram.

[0040] In a preferred embodiment of the present invention, the acquisition of multiple tea bud and leaf images in a tea garden environment, and the division of all tea bud and leaf images into a training set and a validation set, includes:

[0041] Multiple images of tea buds and leaves were acquired using a visual sensor in the tea garden environment;

[0042] All the tea bud and leaf images were classified and labeled using Labelme software;

[0043] All the tea bud and leaf images were divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0044] In a preferred embodiment of the present invention, after dividing all the tea bud and leaf images into a training set, a validation set, and a test set in an 8:1:1 ratio, the method further includes:

[0045] The training set is augmented with data, which includes any one or more of random horizontal movement, random angle rotation, and random image scaling.

[0046] In a preferred embodiment of the present invention, after drawing the prior feature map of key points of tea buds and leaves, the method further includes:

[0047] Based on the following formula, the coordinate decoding of extreme points in the prior feature map of key points of tea buds and leaves is performed using a second-order Taylor expansion to improve the localization accuracy of peak feature coordinates in the prior feature map of key points:

[0048]

[0049] Where G is the prior feature map of key points in tea buds and leaves, u is the coordinate of the peak feature in the prior feature map of key points in tea buds and leaves, and m is the coordinate of the existing predicted point. H(m) represents the gradient of the existing predicted points in the prior feature map of key points of tea buds and leaves, and H(m) represents the second derivative of the existing predicted points in the prior feature map of key points of tea buds and leaves.

[0050] The beneficial effects of this invention are as follows:

[0051] The method for estimating the pose of tea buds and leaves in a tea garden environment provided by this invention includes acquiring multiple images of tea buds and leaves in a tea garden environment, and dividing all images into a training set and a validation set. A pose estimation model to be trained is constructed, which adopts the Alphapose model. Prior knowledge is introduced, including global geometric features and local gradient direction features. Global geometric features reflect the shape and orientation of the tea buds and leaves, i.e., describe the pose of the tea buds and leaves, while local gradient direction features reflect the changes in local areas of the image. A loss function is established based on the prior knowledge, which combines the keypoint pixel positions predicted by the pose estimation model to be trained with the pixel positions of the original labels. Based on the training set, the pose estimation model to be trained is trained using the loss function to obtain the tea bud and leaf pose estimation model. The validation set is input into the tea bud and leaf pose estimation model to estimate the pose of the tea buds and leaves. This invention introduces prior knowledge, guides model training based on prior knowledge, generates a loss function combining high-dimensional features and prior knowledge, and the trained tea bud and leaf pose estimation model can achieve more accurate keypoint estimation. Using a tea bud and leaf pose estimation model to estimate the pose of tea buds and leaves in space can accurately provide the position and orientation of tea buds and leaves, thereby improving the stability and accuracy of identifying the location of tea bud and leaf stem picking points. Attached Figure Description

[0052] Figure 1 This is a flowchart of the tea bud and leaf posture estimation method in a tea garden environment according to the present invention;

[0053] Figure 2 This is a flowchart illustrating the introduction of prior knowledge in this invention;

[0054] Figure 3 These are images of tea buds and leaves in a tea garden environment, as described in this invention.

[0055] Figure 4 This is a priori feature diagram of key points of tea buds and leaves according to the present invention. Detailed Implementation

[0056] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0057] Example 1

[0058] like Figure 1 As shown in the figure, this embodiment provides a method for estimating the posture of tea buds and leaves in a tea garden environment, including:

[0059] S1: Collect multiple images of tea buds and leaves in the tea garden environment, and divide all the tea bud and leaf images into a training set and a validation set.

[0060] S2: Construct the pose estimation model to be trained.

[0061] S3: Introduce prior knowledge and establish a loss function based on the prior knowledge; wherein, the prior knowledge includes global geometric features and local gradient direction features.

[0062] S4: Based on the training set, the posture estimation model to be trained is trained using the loss function to obtain the tea bud and leaf posture estimation model.

[0063] S5: Input the validation set into the tea bud and leaf posture estimation model to estimate the posture of the tea buds and leaves.

[0064] The process involves collecting multiple images of tea buds and leaves in a tea garden environment, and dividing all these images into a training set and a validation set, including:

[0065] S11: Multiple images of tea buds and leaves are collected in the tea garden environment using a visual sensor.

[0066] S12: Use Labelme software to classify and label all the tea bud and leaf images.

[0067] S13: Divide all the tea bud and leaf images into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0068] A visual sensor was used to acquire multiple images of tea buds and leaves in an unstructured tea garden environment. In this embodiment, the visual sensor was an Intel RealSense D435i depth camera. The visual sensor was used to capture images of tea buds and leaves from different azimuth angles in the unstructured tea garden. Labelme software was used to classify and label all the tea bud and leaf images, and the images were saved as JSON-formatted label files.

[0069] All the tea bud and leaf images are divided into a training set, a validation set, and a test set in an 8:1:1 ratio. The training set is used to train the pose estimation model to be trained, so as to continuously optimize the parameters of the pose estimation model to be trained. The validation set is used to verify the performance of the pose estimation model to be trained, and the test set is used to test the performance of the tea bud and leaf pose estimation model. In this embodiment, the pose estimation model to be trained is the Alphapose model.

[0070] The introduction of prior knowledge includes:

[0071] S31: Draw the prior feature map of key points of tea buds and leaves.

[0072] S32: Geometric feature encoding is performed on the prior feature map of key points of tea buds and leaves to obtain global geometric features.

[0073] S33: Encode the prior feature map of key points of tea buds and leaves using gradient direction features to obtain local gradient direction features; the global geometric features and the local gradient direction features constitute prior knowledge.

[0074] The process of drawing prior feature maps of key points in tea buds and leaves includes:

[0075] S311: Based on the external structural characteristics of tea buds and leaves, mark key points of tea buds and leaves; wherein, the key points of tea buds and leaves include the midpoint of the tea bud, the midpoint of the first tea leaf, the intersection of the tea bud and the first tea leaf, and the picking point of the stem with one bud and one leaf.

[0076] S312: Connect the adjacent key points of the tea buds and leaves into a binary tree structure to obtain the prior feature map of the key points of the tea buds and leaves.

[0077] Based on the morphological and structural characteristics of tea buds and leaves, four key point labels are defined: the midpoint of the tea bud, the midpoint of the first tea leaf, the intersection of the tea bud and the first tea leaf, and the picking point of the stem with one bud and one leaf. Adjacent key points are connected according to the morphological characteristics of tea buds and leaves to obtain multiple line segments. Multiple line segments of the same tea bud and leaf form a binary tree structure.

[0078] Based on the growth pattern of tea leaves, the target of picking a single bud and leaf generally exhibits a binary tree morphology. Multiple line segments are used to connect the four key points of the same tea bud and leaf, forming a binary tree structure. During the recognition process, this morphology is encoded as prior knowledge to train the pose estimation model, learning the distribution patterns of these key points in the tea bud and leaf image.

[0079] Figure 3 Images of tea buds and leaves in a tea garden environment. Figure 4 for Figure 3 The corresponding prior feature map of key points in tea buds and leaves. Figure 4 It contains 3 tea buds and leaves, each of which has a Y-shaped structure, i.e., a binary tree structure. In this figure, the 4 key points and the connecting lines between the key points are all represented in white, and the background area is black.

[0080] The step of encoding the prior feature map of key points of tea buds and leaves using geometric features to obtain global geometric features includes:

[0081] S321: Perform Hough line transform detection on the prior feature map of key points of tea buds and leaves to obtain multiple line segments in the binary tree structure.

[0082] S322: Encode all the line segments to obtain global geometric features.

[0083] Hough line transform detection is used to detect line segments. Prior feature maps of key points on tea buds and leaves are drawn, and then encoded. Hough line transform detection transforms the problem of line detection in image space into peak detection in parameter space. Hough line transform detection includes three steps: polar coordinate representation, parameter space creation, and the use of a voting mechanism to detect lines.

[0084] The step of establishing the loss function based on the prior knowledge includes:

[0085] Based on the aforementioned prior knowledge, the loss function is established according to the following formula:

[0086]

[0087] Where λ is the scaling factor, K is the prior knowledge, W is the linear projection parameter of the high-dimensional features onto the fully connected layer, β is the regularization intensity ratio, Y is the keypoint pixel position predicted by the pose estimation model to be trained, H is the pixel position of the original label, and Loss is the loss function.

[0088] Prior knowledge is derived from global geometric features and local gradient direction features. A loss function is generated by combining the loss term between the keypoint pixel positions predicted by the pose estimation model and the pixel positions of the original labels, as well as the loss term between high-dimensional features and prior knowledge. This loss function is then used to train the pose estimation model, resulting in a tea bud and leaf pose estimation model. This model achieves more accurate keypoint estimation. By using this model to estimate the pose of tea buds and leaves in space, it can accurately provide the position and orientation of the tea buds and leaves, thereby improving the stability and accuracy of identifying the picking points of tea buds and leaves.

[0089] This embodiment provides a method for estimating the pose of tea buds and leaves in a tea garden environment. The method involves acquiring multiple images of tea buds and leaves in the tea garden environment and dividing these images into a training set and a validation set. A pose estimation model to be trained is constructed, using the Alphapose model. Prior knowledge is introduced, including global geometric features and local gradient direction features. Global geometric features reflect the shape and orientation of the tea buds and leaves, describing their pose, while local gradient direction features reflect changes in local areas of the image. A loss function is established based on the prior knowledge, combining the keypoint pixel positions predicted by the pose estimation model and the pixel positions of the original labels. Based on the training set, the pose estimation model is trained using the loss function to obtain the tea bud and leaf pose estimation model. The validation set is then input into the tea bud and leaf pose estimation model to estimate the pose of the tea buds and leaves. This invention introduces prior knowledge, guides model training based on this prior knowledge, and generates a loss function that combines high-dimensional features and prior knowledge. The trained tea bud and leaf pose estimation model can achieve more accurate keypoint estimation. Using a tea bud and leaf pose estimation model to estimate the pose of tea buds and leaves in space can accurately provide the position and orientation of tea buds and leaves, thereby improving the stability and accuracy of identifying the location of tea bud and leaf stem picking points.

[0090] Example 2

[0091] like Figure 1 As shown, this embodiment provides a method for estimating the posture of tea buds and leaves in a tea garden environment. This embodiment describes the differences between it and Embodiment 1. The method includes:

[0092] S1: Collect multiple images of tea buds and leaves in the tea garden environment, and divide all the tea bud and leaf images into a training set and a validation set.

[0093] S2: Construct the pose estimation model to be trained.

[0094] S3: Introduce prior knowledge and establish a loss function based on the prior knowledge; wherein, the prior knowledge includes global geometric features and local gradient direction features.

[0095] S4: Based on the training set, the posture estimation model to be trained is trained using the loss function to obtain the tea bud and leaf posture estimation model.

[0096] S5: Input the validation set into the tea bud and leaf posture estimation model to estimate the posture of the tea buds and leaves.

[0097] like Figure 2 As shown, the introduction of prior knowledge includes:

[0098] S31: Draw the prior feature map of key points of tea buds and leaves.

[0099] S32: Geometric feature encoding is performed on the prior feature map of key points of tea buds and leaves to obtain global geometric features.

[0100] S33: Encode the prior feature map of key points of tea buds and leaves using gradient direction features to obtain local gradient direction features; the global geometric features and the local gradient direction features constitute prior knowledge.

[0101] The step of encoding the prior feature map of key points of tea buds and leaves with gradient direction features to obtain local gradient direction features includes:

[0102] S331: Divide the prior feature map of key points of tea buds and leaves into multiple image blocks.

[0103] S332: The Sobel operator is used to calculate the horizontal and vertical gradients of the image patch.

[0104] S333: Calculate the first gradient magnitude and first direction angle of the horizontal gradient, and calculate the second gradient magnitude and second direction angle of the vertical gradient.

[0105] S334: Divide the first direction angle into M equal sub-intervals, and divide the second direction angle into N equal sub-intervals.

[0106] S335: The magnitude of the first gradient intensity is proportionally assigned to M sub-intervals to obtain a horizontal gradient histogram.

[0107] S336: The second gradient intensity magnitude is proportionally assigned to N sub-intervals to obtain a vertical gradient histogram; wherein, the horizontal gradient histogram and the vertical gradient histogram constitute local gradient direction features.

[0108] The image block size is preset, and the prior feature map of key points of tea buds and leaves is divided according to the size of the image block. For example, if the size of the prior feature map of key points of tea buds and leaves is 1000*1000, and the size of the image block is 100*100, the prior feature map of key points of tea buds and leaves is divided into 10 image blocks.

[0109] The Sobel operator detects edges by calculating the gradient of each pixel in an image. Edges are typically where brightness changes are most significant in an image, and the gradient reflects the intensity and direction of these brightness changes. The Sobel operator uses two 3×3 convolution kernels to calculate the gradients of the image patch in the horizontal and vertical directions, respectively. It calculates the first gradient magnitude G1 and the first orientation angle of the horizontal gradient. Calculate the second gradient magnitude G2 and the second direction angle of the vertical gradient. Subsequently, the magnitude of the first gradient intensity G1 is proportionally assigned to the value according to the first direction angle. Within the M equally divided sub-intervals, a horizontal gradient histogram is obtained. The second gradient intensity magnitude G2 is proportionally assigned to the intervals defined by the second direction angle. The vertical gradient histogram is obtained by dividing the interval into N equal subintervals.

[0110] As an example, the first direction angle Second direction angle All are limited to the range of 0-180°, with the first direction angle... Second direction angle Divide the graph into 9 equal sub-intervals, i.e., M = N = 9. Appropriately distribute the first gradient magnitude G1 into these 9 sub-intervals to obtain the horizontal gradient histogram. Appropriately distribute the second gradient magnitude G2 into these 9 sub-intervals to obtain the vertical gradient histogram.

[0111] After obtaining the vertical gradient histogram, the process also includes:

[0112] The horizontal gradient histogram is normalized according to the following formula:

[0113]

[0114] Among them, f x For local horizontal gradient direction features, v x This is a horizontal gradient histogram. Let be the magnitude of the horizontal gradient histogram, and e be the adjustment parameter;

[0115] The vertical gradient histogram is normalized according to the following formula:

[0116]

[0117] Among them, f y For local vertical gradient direction features, v y This is a vertical gradient histogram. is the modulus of the vertical gradient histogram.

[0118] The intensity values ​​of the M equal sub-intervals divided by the first direction angle are normalized using the L2 norm to obtain local horizontal gradient direction features. These features reflect the horizontal gradient information of each image patch in the prior feature map of tea bud and leaf keypoints. Similarly, the intensity values ​​of the N equal sub-intervals divided by the second direction angle are normalized using the L2 norm to obtain local vertical gradient direction features. These features reflect the vertical gradient information of each image patch in the prior feature map of tea bud and leaf keypoints.

[0119] This embodiment divides the prior feature map of key points of tea buds and leaves into multiple image blocks. The Sobel operator is used to calculate the horizontal and vertical gradients of each image block. The first gradient magnitude and first orientation angle of the horizontal gradient are calculated, as are the second gradient magnitude and second orientation angle of the vertical gradient. The image block is then divided into M equal sub-intervals using the first orientation angle and N equal sub-intervals using the second orientation angle. The first gradient magnitude is proportionally distributed among the M sub-intervals to obtain a horizontal gradient histogram. The second gradient magnitude is proportionally distributed among the N sub-intervals to obtain a vertical gradient histogram. The horizontal gradient histogram reflects the features of horizontal edges within the image block, while the vertical gradient histogram reflects the features of vertical edges. Both the horizontal and vertical gradient histograms are normalized to ensure that the gradient magnitudes within each sub-interval are within a fixed range, allowing for better measurement of gradient differences across different regions of the prior feature map of key points of tea buds and leaves.

[0120] Example 3

[0121] like Figure 1 As shown, this embodiment provides a method for estimating the posture of tea buds and leaves in a tea garden environment. This embodiment describes the differences between it and Embodiment 1. The method includes:

[0122] S1: Collect multiple images of tea buds and leaves in the tea garden environment, and divide all the tea bud and leaf images into a training set and a validation set.

[0123] S2: Construct the pose estimation model to be trained.

[0124] S3: Introduce prior knowledge and establish a loss function based on the prior knowledge; wherein, the prior knowledge includes global geometric features and local gradient direction features.

[0125] S4: Based on the training set, the posture estimation model to be trained is trained using the loss function to obtain the tea bud and leaf posture estimation model.

[0126] S5: Input the validation set into the tea bud and leaf posture estimation model to estimate the posture of the tea buds and leaves.

[0127] The process involves collecting multiple images of tea buds and leaves in a tea garden environment, and dividing all these images into a training set and a validation set, including:

[0128] S11: Multiple images of tea buds and leaves are collected in the tea garden environment using a visual sensor.

[0129] S12: Use Labelme software to classify and label all the tea bud and leaf images.

[0130] S13: Divide all the tea bud and leaf images into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0131] After dividing all the tea bud and leaf images into training, validation, and test sets in an 8:1:1 ratio, the method further includes:

[0132] S14: Perform data augmentation on the training set, wherein the data augmentation includes any one or more of random horizontal movement, random angle rotation, and random image scaling.

[0133] Random horizontal shifting, random angle rotation, and random scaling of images can be applied to the training set. One, two, or three data augmentation methods can be used, and different data augmentation methods can be executed in any order. For example, a portion of the images in the training set can be randomly shifted horizontally, rotated at random angles, and scaled sequentially, and then another portion of the images in the training set can be randomly scaled, shifted horizontally, and rotated at random angles sequentially.

[0134] After drawing the prior feature map of key points of tea buds and leaves, the following is also included:

[0135] Based on the following formula, the coordinate decoding of extreme points in the prior feature map of key points of tea buds and leaves is performed using a second-order Taylor expansion to improve the localization accuracy of peak feature coordinates in the prior feature map of key points:

[0136]

[0137] Where G is the prior feature map of key points in tea buds and leaves, u is the coordinate of the peak feature in the prior feature map of key points in tea buds and leaves, and m is the coordinate of the existing predicted point. H(m) represents the gradient of the existing predicted points in the prior feature map of key points of tea buds and leaves, and H(m) represents the second derivative of the existing predicted points in the prior feature map of key points of tea buds and leaves.

[0138] This embodiment also predicts and modulates the coordinate distribution of key points. First, a Gaussian distribution assumption is introduced into the heatmap distribution output by the tea bud and leaf posture estimation model, and the pixel coordinate distribution function of the heatmap is calculated according to the following formula:

[0139]

[0140] Where x represents the pixel coordinates in the predicted heatmap, u represents the coordinates of the key point to be estimated, i.e., the peak feature coordinates, cov is the covariance matrix, |cov| represents the determinant of the covariance matrix, and (xu) T Let σ denote the transpose of the matrix x minus u, ln denotes the logarithmic function with base e, and σ 2 Let f(x,u,cov) represent the variance, and let f(x,u,cov) represent the distribution function of pixel coordinates in the heatmap.

[0141] In this embodiment, the coordinate distribution of key points is predicted and modulated, and a second-order Taylor expansion is used to achieve coordinate decoding, thereby improving sub-pixel precision coordinate positioning and reducing quantization error.

[0142] Example 4

[0143] This application also provides a computer device, which may be a server. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor in this computer design provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communication with external terminals via a network connection.

[0144] This embodiment also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the tea bud and leaf attitude estimation method in a tea garden environment as described in any one of Embodiments 1-3. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0145] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0146] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for estimating the posture of tea buds and leaves in a tea garden environment, characterized in that, include: Multiple images of tea buds and leaves were collected in a tea garden environment, and all the tea bud and leaf images were divided into a training set and a validation set. Construct a pose estimation model to be trained; Based on the morphological and structural characteristics of tea buds and leaves, key points of tea buds and leaves are marked; wherein, the key points of tea buds and leaves include the midpoint of the tea bud, the midpoint of the first tea leaf, the intersection of the tea bud and the first tea leaf, and the picking point of the stem with one bud and one leaf. Connect the adjacent key points of the tea buds and leaves into a binary tree structure to obtain the prior feature map of the key points of the tea buds and leaves. Hough line transform detection is performed on the prior feature map of key points of tea buds and leaves to obtain multiple line segments in the binary tree structure; Encode all the line segments to obtain global geometric features; The prior feature map of key points of tea buds and leaves is divided into multiple image blocks; The horizontal and vertical gradients of the image patch are calculated using the Sobel operator; Calculate the first gradient magnitude and first direction angle of the horizontal gradient, and calculate the second gradient magnitude and second direction angle of the vertical gradient; Divide the area into M sub-intervals using the first direction angle and into N sub-intervals using the second direction angle; The first gradient intensity magnitude is proportionally assigned to M sub-intervals to obtain a horizontal gradient histogram. The second gradient intensity magnitude is proportionally distributed into N sub-intervals to obtain a vertical gradient histogram; wherein, the horizontal gradient histogram and the vertical gradient histogram constitute local gradient direction features; the global geometric features and the local gradient direction features constitute prior knowledge; Based on the aforementioned prior knowledge, the loss function is established according to the following formula: ； in, Where K is the scaling factor, K is the prior knowledge, and W is the linear projection parameter of the high-dimensional features onto the fully connected layer. Here, Y represents the regularization intensity ratio, H represents the pixel position of the keypoint predicted by the pose estimation model to be trained, and L represents the pixel position of the original label. Loss is the loss function. The prior knowledge includes global geometric features and local gradient direction features. Based on the training set, the loss function is used to train the pose estimation model to be trained, and the tea bud and leaf pose estimation model is obtained. The validation set is input into the tea bud and leaf posture estimation model to estimate the posture of the tea buds and leaves.

2. The method for estimating the posture of tea buds and leaves in a tea garden environment according to claim 1, characterized in that, After obtaining the vertical gradient histogram, the process also includes: The horizontal gradient histogram is normalized according to the following formula: ； Among them, f x For local horizontal gradient direction features, v x This is a horizontal gradient histogram. Let be the magnitude of the horizontal gradient histogram, and e be the adjustment parameter; The vertical gradient histogram is normalized according to the following formula: ； Among them, f y For local vertical gradient direction features, v y This is a vertical gradient histogram. is the modulus of the vertical gradient histogram.

3. The method for estimating the posture of tea buds and leaves in a tea garden environment according to claim 1, characterized in that, The process involves collecting multiple images of tea buds and leaves in a tea garden environment, and dividing all these images into a training set and a validation set, including: Multiple images of tea buds and leaves were acquired using a visual sensor in the tea garden environment; All the tea bud and leaf images were classified and labeled using Labelme software; All the tea bud and leaf images were divided into a training set, a validation set, and a test set in a ratio of 8:1:

1.

4. The method for estimating the posture of tea buds and leaves in a tea garden environment according to claim 3, characterized in that, After dividing all the tea bud and leaf images into training, validation, and test sets in an 8:1:1 ratio, the method further includes: The training set is augmented with data, which includes any one or more of random horizontal movement, random angle rotation, and random image scaling.

5. The method for estimating the posture of tea buds and leaves in a tea garden environment according to claim 1, characterized in that, After obtaining the prior feature map of key points of tea buds and leaves, the method further includes: Based on the following formula, the coordinate decoding of extreme points in the prior feature map of key points of tea buds and leaves is performed using a second-order Taylor expansion to improve the localization accuracy of peak feature coordinates in the prior feature map of key points: ； Where G is the prior feature map of key points in tea buds and leaves, u is the coordinate of the peak feature in the prior feature map of key points in tea buds and leaves, and m is the coordinate of the existing predicted point. H(m) represents the gradient of the existing predicted points in the prior feature map of key points of tea buds and leaves, and H(m) represents the second derivative of the existing predicted points in the prior feature map of key points of tea buds and leaves.

Images