Attitude estimation method for tea buds and leaves in tea garden environment

Abstract

The invention provides a tea bud and leaf attitude estimation method in a tea garden environment, which comprises the following steps: collecting a plurality of tea bud and leaf images in the tea garden environment, and dividing all the tea bud and leaf images into a training set and a verification set; a to-be-trained attitude estimation model is constructed, prior knowledge is introduced, and a loss function is established according to the prior knowledge. And based on the training set, training a to-be-trained attitude estimation model by adopting a loss function to obtain a tea bud leaf attitude estimation model. And inputting the verification set into a tea bud and leaf attitude estimation model, and estimating the attitude of the tea buds and leaves. According to the method, prior knowledge is introduced, model training is guided according to the prior knowledge, a loss function combining high-dimensional features and the prior knowledge is generated, and the tea bud leaf attitude estimation model obtained through training can achieve more accurate key point estimation. The tea bud and leaf posture estimation model is adopted to estimate the poses of the tea buds and leaves in the space, the positions and orientations of the tea buds and leaves can be accurately provided, and therefore the stability and accuracy of recognizing the positions of the tea bud and leaf stalk picking points are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for estimating the posture of tea buds and leaves in a tea garden environment. Background Art

[0002] The plucking of premium tea buds and leaves relies heavily on manual labor, accounting for approximately 70% of the entire production process and being one of the most labor-intensive and costly steps. This work is highly seasonal and time-sensitive, requiring a large amount of labor and repeated operations in a short period of time, making the traditional manual plucking model unsustainable and severely restricting the sustainable development of the premium tea industry. Therefore, the development of automated mechanical plucking technology to gradually replace manual labor has become an urgent need and an inevitable trend for the industry to achieve efficient and sustainable development.

[0003] Accurately detecting picking points on the stem is a critical step in automated tea picking in a tea garden. Firstly, tea buds exhibit significant size variations and morphological diversity at different growth stages, making it difficult to reliably and accurately identify the picking points on the stem. Secondly, the frequent occlusion between adjacent tea buds in a tea garden, as well as the overlapping of densely packed tea buds from the same viewing angle, greatly complicates identifying picking points on the stem.

[0004] Therefore, a method is needed to estimate the posture of tea buds in space and accurately provide the position and orientation of tea buds, thereby improving the stability and accuracy of identifying the position of the stem picking point. Summary of the Invention

[0005] In order to overcome the problems existing in the related art, the purpose of the present invention is to provide a method for estimating the posture of tea buds and leaves in a tea garden environment. This method can estimate the posture of tea buds and leaves in space, accurately provide the position and orientation of the tea buds and leaves, thereby improving the stability and accuracy of identifying the position of the stem picking point.

[0006] A method for estimating the posture of tea buds and leaves in a tea garden environment, comprising:

[0007] Collecting multiple tea bud and leaf images in a tea garden environment, and dividing all the tea bud and leaf images into a training set and a validation set;

[0008] Build a pose estimation model to be trained;

[0009] Introducing prior knowledge and establishing a loss function based on the prior knowledge; wherein the prior knowledge includes global geometric features and local gradient direction features;

[0010] Based on the training set, the posture estimation model to be trained is trained using the loss function to obtain a tea bud leaf posture estimation model;

[0011] The verification set is input into the tea bud leaf posture estimation model to estimate the posture of the tea bud leaf.

[0012] In a preferred technical solution of the present invention, the introduction of prior knowledge includes:

[0013] Draw a priori feature map of key points of tea buds and leaves;

[0014] Performing geometric feature encoding on the prior feature map of the tea bud and leaf key points to obtain global geometric features;

[0015] Gradient direction feature encoding is performed on the prior feature map of the key points of the tea buds and leaves to obtain local gradient direction features; the global geometric features and the local gradient direction features constitute prior knowledge.

[0016] In a preferred technical solution of the present invention, establishing a loss function based on the prior knowledge includes:

[0017] Based on the prior knowledge, the loss function is established according to the following formula:

[0018]

[0019] Wherein, λ is the scaling factor, K is the prior knowledge, W is the linear projection parameter of the high-dimensional feature to the fully connected layer, β is the regularization strength ratio, Y is the pixel position of the key point predicted by the pose estimation model to be trained, H is the pixel position of the original label, and Loss is the loss function.

[0020] In a preferred technical solution of the present invention, the step of drawing a priori characteristic graph of key points of tea buds and leaves comprises:

[0021] Based on the appearance and structural characteristics of the tea buds, the key points of the tea buds are marked; wherein the key points of the tea buds include the middle point of the tea buds, the middle point of the first tea leaf, the intersection of the tea buds and the first tea leaf, and the picking point of the one bud and one leaf stem;

[0022] The adjacent tea bud and leaf key points are connected into a binary tree structure to obtain a priori feature map of the tea bud and leaf key points.

[0023] In a preferred technical solution of the present invention, the geometric feature encoding of the prior feature map of the key points of the tea buds and leaves to obtain the global geometric features includes:

[0024] Performing Hough line transform detection on the prior feature map of the tea bud and leaf key points to obtain multiple line segments in the binary tree structure;

[0025] All the line segments are encoded to obtain global geometric features.

[0026] In a preferred technical solution of the present invention, the step of performing gradient directional feature encoding on the prior feature map of the key points of the tea buds and leaves to obtain local gradient directional features comprises:

[0027] Dividing the tea bud leaf key point prior feature map into multiple image blocks;

[0028] Calculating the horizontal gradient and vertical gradient of the image block using the Sobel operator;

[0029] Calculating a first gradient intensity magnitude and a first direction angle of the horizontal gradient, and calculating a second gradient intensity magnitude and a second direction angle of the vertical gradient;

[0030] The first direction angle is used to equally divide the interval into M subintervals, and the second direction angle is used to equally divide the interval into N subintervals;

[0031] Proportionally classifying the first gradient intensity amplitude into M subintervals to obtain a horizontal gradient histogram;

[0032] The second gradient intensity amplitude is proportionally divided into N subintervals to obtain a vertical gradient histogram; wherein the horizontal gradient histogram and the vertical gradient histogram constitute a local gradient direction feature.

[0033] In a preferred technical solution of the present invention, after obtaining the vertical gradient histogram, the method further includes:

[0034] The horizontal gradient histogram is normalized according to the following formula:

[0035]

[0036] Among them, f x is the local horizontal gradient direction feature, v x is the horizontal gradient histogram, is the modulus of the horizontal gradient histogram, and e is the adjustment parameter;

[0037] The vertical gradient histogram is normalized according to the following formula:

[0038]

[0039] Among them, f y is the local vertical gradient direction feature, v y is the vertical gradient histogram, is the modulus of the vertical gradient histogram.

[0040] In a preferred technical solution of the present invention, the method of collecting multiple tea bud and leaf images in a tea garden environment and dividing all the tea bud and leaf images into a training set and a validation set includes:

[0041] Use visual sensors to collect multiple images of tea buds and leaves in a tea garden environment;

[0042] All tea bud and leaf images were classified and labeled using Labelme software;

[0043] All the tea bud and leaf images were divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0044] In a preferred technical solution of the present invention, after dividing all the tea bud leaf images into a training set, a validation set and a test set in a ratio of 8:1:1, the method further comprises:

[0045] Data enhancement is performed on the training set, where the data enhancement includes any one or more of random horizontal movement, random angle rotation, and random image scaling.

[0046] In a preferred technical solution of the present invention, after drawing the prior feature map of the key points of the tea buds and leaves, the method further includes:

[0047] According to the following formula, the second-order Taylor expansion is used to decode the coordinates of the extreme points of the tea bud leaf key point prior feature map to improve the positioning accuracy of the peak feature coordinates in the key point prior feature map:

[0048]

[0049] Among them, G is the prior feature map of the key points of tea buds and leaves, u is the coordinate of the peak feature in the prior feature map of the key points of tea buds and leaves, and m is the coordinate of the existing prediction point. is the gradient of the existing prediction point in the prior feature map of the key points of tea buds and leaves, and H(m) is the second-order derivative of the existing prediction point in the prior feature map of the key points of tea buds and leaves.

[0050] The beneficial effects of the present invention are:

[0051] The method for estimating the posture of tea buds and leaves in a tea garden environment provided by the present invention includes collecting multiple tea bud and leaf images in a tea garden environment, and dividing all the tea bud and leaf images into a training set and a validation set. A posture estimation model to be trained is constructed, and the posture estimation model to be trained adopts the Alphapose model. Prior knowledge is introduced, and the prior knowledge includes global geometric features and local gradient direction features. The global geometric features can reflect the shape and orientation of the tea bud and leaves, that is, describe the posture of the tea bud and leaves, and the local gradient direction features can reflect the changes in the local area in the image. A loss function is established based on the prior knowledge, and the loss function combines the pixel positions of the key points predicted by the posture estimation model to be trained and the pixel positions of the original labels. Based on the training set, the posture estimation model to be trained is trained using the loss function to obtain a tea bud and leaf posture estimation model. The validation set is input into the tea bud and leaf posture estimation model to estimate the posture of the tea bud and leaves. The present invention introduces prior knowledge, guides model training based on the prior knowledge, generates a loss function that combines high-dimensional features and prior knowledge, and the trained tea bud and leaf posture estimation model can achieve more accurate key point estimation. The tea bud pose estimation model is used to estimate the pose of tea buds in space, which can accurately provide the position and orientation of the tea buds, thereby improving the stability and accuracy of identifying the picking point of the tea bud stems. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of a method for estimating the posture of tea buds and leaves in a tea garden environment according to the present invention;

[0053] Figure 2 It is a flow chart of introducing prior knowledge of the present invention;

[0054] Figure 3 is an image of tea buds and leaves in a tea garden environment of the present invention;

[0055] Figure 4 This is a priori characteristic diagram of key points of tea buds and leaves of the present invention. DETAILED DESCRIPTION

[0056] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present invention more thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0057] Example 1

[0058] like Figure 1 As shown, this embodiment provides a method for estimating the posture of tea buds and leaves in a tea garden environment, including:

[0059] S1: Collect multiple tea bud and leaf images in a tea garden environment, and divide all the tea bud and leaf images into a training set and a validation set.

[0060] S2: Build the pose estimation model to be trained.

[0061] S3: Introduce prior knowledge and establish a loss function based on the prior knowledge; wherein the prior knowledge includes global geometric features and local gradient direction features.

[0062] S4: Based on the training set, the loss function is used to train the posture estimation model to be trained to obtain a tea bud leaf posture estimation model.

[0063] S5: Inputting the verification set into the tea bud leaf posture estimation model to estimate the posture of the tea bud leaf.

[0064] The method comprises collecting multiple tea bud and leaf images in a tea garden environment and dividing all the tea bud and leaf images into a training set and a validation set, including:

[0065] S11: Use visual sensors to collect multiple images of tea buds and leaves in a tea garden environment.

[0066] S12: using Labelme software to classify and label all the tea bud and leaf images.

[0067] S13: Divide all the tea bud and leaf images into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0068] A visual sensor, in this example, an Intel RealSense D435i depth camera, captures multiple images of tea buds and leaves in an unstructured tea garden environment. The visual sensor captures tea buds and leaves from different azimuths in the unstructured tea garden. Labelme software is used to classify and label all tea bud and leaf images, saving them as label files in JSON format.

[0069] All tea bud leaf images are divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The training set is used to train the pose estimation model to be trained to continuously optimize the parameters of the pose estimation model to be trained. The validation set is used to verify the performance of the pose estimation model to be trained, and the test set is used to test the performance of the tea bud leaf pose estimation model. In this embodiment, the pose estimation model to be trained is the Alphapose model.

[0070] The introduction of prior knowledge includes:

[0071] S31: Draw a priori feature map of key points of tea buds and leaves.

[0072] S32: performing geometric feature encoding on the prior feature map of the key points of the tea buds and leaves to obtain global geometric features.

[0073] S33: performing gradient direction feature encoding on the prior feature map of the key points of the tea buds and leaves to obtain local gradient direction features; the global geometric features and the local gradient direction features constitute prior knowledge.

[0074] The method of drawing a priori feature map of key points of tea buds and leaves comprises:

[0075] S311: Based on the external structural characteristics of the tea buds and leaves, the key points of the tea buds and leaves are marked; wherein the key points of the tea buds and leaves include the middle point of the tea buds, the middle point of the first tea leaf, the intersection of the tea buds and the first tea leaf, and the picking point of the one bud and one leaf stem.

[0076] S312: Connecting the adjacent tea bud and leaf key points into a binary tree structure to obtain a priori feature graph of the tea bud and leaf key points.

[0077] Based on the structural characteristics of the tea buds and leaves, we define the middle point of the tea bud, the middle point of the first tea leaf, the intersection of the tea bud and the first tea leaf, and the picking point of the one bud and one leaf stalk, a total of four key point labels. Adjacent key points are connected according to the shape of the tea buds and leaves to obtain multiple line segments. Multiple line segments of the same tea bud and leaf form a binary tree structure.

[0078] Based on the growth patterns of tea leaves, each target, typically a single bud and a single leaf, is represented by a binary tree. Multiple line segments connect the four key points of a single tea bud or leaf to form a binary tree structure. During the recognition process, this structure is encoded as prior knowledge to train the pose estimation model, learning the distribution patterns of these key points within the tea bud or leaf image.

[0079] Figure 3 This is an image of tea buds and leaves in a tea garden environment. Figure 4 for Figure 3 The corresponding tea bud leaf key point prior feature map, Figure 4 There are 3 tea buds in it, and each tea bud presents a Y-shaped structure, that is, a binary tree structure. In this figure, the 4 key points and the connecting line segments between the key points are represented by white, and the background area is black.

[0080] The geometric feature encoding of the prior feature map of the key points of the tea buds and leaves to obtain global geometric features includes:

[0081] S321: Performing Hough line transform detection on the prior feature map of the tea bud and leaf key points to obtain multiple line segments in the binary tree structure.

[0082] S322: Encode all the line segments to obtain global geometric features.

[0083] Hough line transform detection is used to detect line segments, draw a priori feature map of tea bud and leaf key points, and then encode this priori feature map of tea bud and leaf key points. Hough line transform detection transforms the problem of line detection in image space into the problem of peak detection in parameter space. Hough line transform detection involves three steps: polar coordinate representation, creating a parameter space, and using a voting mechanism to detect lines.

[0084] The establishing of a loss function according to the prior knowledge comprises:

[0085] Based on the prior knowledge, the loss function is established according to the following formula:

[0086]

[0087] Wherein, λ is the scaling factor, K is the prior knowledge, W is the linear projection parameter of the high-dimensional feature to the fully connected layer, β is the regularization strength ratio, Y is the pixel position of the key point predicted by the pose estimation model to be trained, H is the pixel position of the original label, and Loss is the loss function.

[0088] Based on prior knowledge composed of global geometric features and local gradient directional features, a loss function is generated by combining the loss term between the pixel positions of key points predicted by the pose estimation model to be trained and the pixel positions of the original labels, as well as the loss term between high-dimensional features and prior knowledge. This loss function is used to train the pose estimation model to obtain a tea bud pose estimation model. This tea bud pose estimation model achieves more accurate key point estimation. Using this model to estimate the pose of tea buds in space accurately provides the position and orientation of the tea buds, thereby improving the stability and accuracy of identifying the picking point on the tea bud stem.

[0089] The method for estimating the posture of tea buds in a tea garden environment provided in this embodiment includes collecting multiple tea bud images in a tea garden environment, and dividing all the tea bud images into a training set and a validation set. A posture estimation model to be trained is constructed, and the posture estimation model to be trained adopts the Alphapose model. Prior knowledge is introduced, and the prior knowledge includes global geometric features and local gradient direction features. The global geometric features can reflect the shape and orientation of the tea buds, that is, describe the posture of the tea buds, and the local gradient direction features can reflect the changes in the local area in the image. A loss function is established based on the prior knowledge, and the loss function combines the pixel positions of the key points predicted by the posture estimation model to be trained and the pixel positions of the original labels. Based on the training set, the posture estimation model to be trained is trained using the loss function to obtain a tea bud posture estimation model. The validation set is input into the tea bud posture estimation model to estimate the posture of the tea bud. The present invention introduces prior knowledge, guides model training based on the prior knowledge, generates a loss function that combines high-dimensional features and prior knowledge, and the trained tea bud posture estimation model can achieve more accurate key point estimation. The tea bud pose estimation model is used to estimate the pose of tea buds in space, which can accurately provide the position and orientation of the tea buds, thereby improving the stability and accuracy of identifying the picking point of the tea bud stems.

[0090] Example 2

[0091] like Figure 1 As shown, this embodiment provides a method for estimating the posture of tea buds in a tea garden environment. This embodiment is based on Example 1 and describes the differences from Example 1. The method includes:

[0092] S1: Collect multiple tea bud and leaf images in a tea garden environment, and divide all the tea bud and leaf images into a training set and a validation set.

[0093] S2: Build the pose estimation model to be trained.

[0094] S3: Introduce prior knowledge and establish a loss function based on the prior knowledge; wherein the prior knowledge includes global geometric features and local gradient direction features.

[0095] S4: Based on the training set, the loss function is used to train the posture estimation model to be trained to obtain a tea bud leaf posture estimation model.

[0096] S5: Inputting the verification set into the tea bud leaf posture estimation model to estimate the posture of the tea bud leaf.

[0097] like Figure 2 As shown, the introduction of prior knowledge includes:

[0098] S31: Draw a priori feature map of key points of tea buds and leaves.

[0099] S32: performing geometric feature encoding on the prior feature map of the key points of the tea buds and leaves to obtain global geometric features.

[0100] S33: performing gradient direction feature encoding on the prior feature map of the key points of the tea buds and leaves to obtain local gradient direction features; the global geometric features and the local gradient direction features constitute prior knowledge.

[0101] The step of performing gradient directional feature encoding on the prior feature map of the key points of the tea buds and leaves to obtain local gradient directional features includes:

[0102] S331: Divide the tea bud leaf key point prior feature map into multiple image blocks.

[0103] S332: Calculate the horizontal gradient and vertical gradient of the image block using the Sobel operator.

[0104] S333: Calculate the first gradient intensity amplitude and the first direction angle of the horizontal gradient, and calculate the second gradient intensity amplitude and the second direction angle of the vertical gradient.

[0105] S334: Use the first direction angle to equally divide the interval into M sub-intervals, and use the second direction angle to equally divide the interval into N sub-intervals.

[0106] S335: Divide the first gradient intensity amplitude into M subintervals in proportion to obtain a horizontal gradient histogram.

[0107] S336: Proportionally classify the second gradient intensity amplitude into N subintervals to obtain a vertical gradient histogram; wherein the horizontal gradient histogram and the vertical gradient histogram constitute a local gradient directional feature.

[0108] The size of the image block is preset, and the prior feature map of the key points of the tea buds and leaves is divided according to the size of the image block. For example, the size of the prior feature map of the key points of the tea buds and leaves is 1000*1000, and the size of the image block is 100*100, and the prior feature map of the key points of the tea buds and leaves is divided into 10 image blocks.

[0109] The Sobel operator detects edges by calculating the gradient of each pixel in the image. Edges are usually the places where the brightness changes most significantly in the image. The gradient can reflect the intensity and direction of the image brightness change. The Sobel operator uses two 3×3 convolution kernels to calculate the gradient of the image block in the horizontal and vertical directions respectively, and calculates the first gradient intensity amplitude G1 and the first direction angle of the horizontal gradient. Calculate the second gradient intensity amplitude G2 and the second direction angle of the vertical gradient Then the first gradient strength amplitude G1 is proportionally attributed to the first direction angle The horizontal gradient histogram is obtained in the M equally divided subintervals. The second gradient intensity amplitude G2 is proportionally assigned to the second direction angle The vertical gradient histogram is obtained in the N equally divided subintervals.

[0110] As an example, the first direction angle and the second direction angle are limited to the range of 0-180°, and the first direction angle and the second direction angle The first gradient intensity value G1 is proportionally assigned to the 9 subintervals to obtain a horizontal gradient histogram. The second gradient intensity value G2 is proportionally assigned to the 9 subintervals to obtain a vertical gradient histogram.

[0111] After obtaining the vertical gradient histogram, the method further includes:

[0112] The horizontal gradient histogram is normalized according to the following formula:

[0113]

[0114] Among them, f x is the local horizontal gradient direction feature, v x is the horizontal gradient histogram, is the modulus of the horizontal gradient histogram, and e is the adjustment parameter;

[0115] The vertical gradient histogram is normalized according to the following formula:

[0116]

[0117] Among them, f y is the local vertical gradient direction feature, v y is the vertical gradient histogram, is the modulus of the vertical gradient histogram.

[0118] The L2 norm is used to normalize the intensity values ​​of the M subintervals divided equally by the first direction angle to obtain the local horizontal gradient directional feature. The local horizontal gradient directional feature reflects the horizontal gradient information of each image block in the prior feature map of the key points of the tea bud and leaf. The L2 norm is used to normalize the intensity values ​​of the N subintervals divided equally by the second direction angle to obtain the local vertical gradient directional feature. The local vertical gradient directional feature reflects the vertical gradient information of each image block in the prior feature map of the key points of the tea bud and leaf.

[0119] In this embodiment, the prior feature map of the key points of tea buds and leaves is divided into multiple image blocks, and the Sobel operator is used to calculate the horizontal gradient and vertical gradient of the image block, the first gradient intensity amplitude and the first direction angle of the horizontal gradient are calculated, and the second gradient intensity amplitude and the second direction angle of the vertical gradient are calculated; the first direction angle is used to equally divide M subintervals, and the second direction angle is used to equally divide N subintervals. The first gradient intensity amplitude is proportionally classified into the M subintervals to obtain a horizontal gradient histogram. The second gradient intensity amplitude is proportionally classified into the N subintervals to obtain a vertical gradient histogram. The horizontal gradient histogram reflects the characteristics of the horizontal edges in the image block, and the vertical gradient histogram reflects the characteristics of the vertical edges in the image block. The horizontal gradient histogram and the vertical gradient histogram are normalized separately so that the gradient intensity amplitude in each subinterval is within a fixed range, which can better measure the gradient differences in different areas of the prior feature map of the key points of tea buds and leaves.

[0120] Example 3

[0121] like Figure 1 As shown, this embodiment provides a method for estimating the posture of tea buds in a tea garden environment. This embodiment is based on Example 1 and describes the differences from Example 1. The method includes:

[0122] S1: Collect multiple tea bud and leaf images in a tea garden environment, and divide all the tea bud and leaf images into a training set and a validation set.

[0123] S2: Build the pose estimation model to be trained.

[0124] S3: Introduce prior knowledge and establish a loss function based on the prior knowledge; wherein the prior knowledge includes global geometric features and local gradient direction features.

[0125] S4: Based on the training set, the loss function is used to train the posture estimation model to be trained to obtain a tea bud leaf posture estimation model.

[0126] S5: Inputting the verification set into the tea bud leaf posture estimation model to estimate the posture of the tea bud leaf.

[0127] The method comprises collecting multiple tea bud and leaf images in a tea garden environment and dividing all the tea bud and leaf images into a training set and a validation set, including:

[0128] S11: Use visual sensors to collect multiple images of tea buds and leaves in a tea garden environment.

[0129] S12: using Labelme software to classify and label all the tea bud and leaf images.

[0130] S13: Divide all the tea bud and leaf images into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0131] After dividing all the tea bud leaf images into a training set, a validation set, and a test set in a ratio of 8:1:1, the method further includes:

[0132] S14: performing data enhancement on the training set, wherein the data enhancement includes any one or any combination of random horizontal movement, random angle rotation, and random image scaling.

[0133] Perform random horizontal shifts, random angular rotations, and random image scaling on the training set. One, two, or three data augmentation methods can be used, and different data augmentation methods can be performed in any order. For example, first perform random horizontal shifts, random angular rotations, and random image scaling on a portion of the training set, and then perform random image scaling, random horizontal shifts, and random angular rotations on another portion of the training set.

[0134] After drawing the prior feature map of the key points of the tea buds and leaves, the method further includes:

[0135] According to the following formula, the second-order Taylor expansion is used to decode the coordinates of the extreme points of the tea bud leaf key point prior feature map to improve the positioning accuracy of the peak feature coordinates in the key point prior feature map:

[0136]

[0137] Among them, G is the prior feature map of the key points of tea buds and leaves, u is the coordinate of the peak feature in the prior feature map of the key points of tea buds and leaves, and m is the coordinate of the existing prediction point. is the gradient of the existing prediction point in the prior feature map of the key points of tea buds and leaves, and H(m) is the second-order derivative of the existing prediction point in the prior feature map of the key points of tea buds and leaves.

[0138] This embodiment also predicts and modulates the coordinate distribution of key points. First, a Gaussian distribution assumption is introduced into the heat map distribution output by the tea bud posture estimation model, and the heat map pixel coordinate distribution function is calculated according to the following formula:

[0139]

[0140] Where x represents the pixel coordinates in the predicted heat map, u represents the coordinates of the key point to be estimated, i.e., the peak feature coordinates, cov is the covariance matrix, |cov| represents the determinant of the covariance matrix, (xu) T represents the transpose of the matrix after subtracting u from x, ln represents the logarithmic function with the natural number e as the base, σ 2 represents the variance, and f(x,u,cov) represents the heat map pixel coordinate distribution function.

[0141] This embodiment predicts and modulates the coordinate distribution of key points, uses a second-order Taylor expansion to implement coordinate decoding, improves sub-pixel precision coordinate positioning, and reduces quantization errors.

[0142] Example 4

[0143] An embodiment of the present application also provides a computer device, which may be a server, wherein the computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor of the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection.

[0144] This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the method for estimating the posture of tea buds and leaves in a tea garden environment described in any one of Examples 1 to 3. It will be understood that the computer-readable storage medium in this embodiment can be either a volatile or non-volatile readable storage medium.

[0145] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0146] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for estimating the posture of tea buds in a tea garden environment, characterized in that: include: Collecting multiple tea bud and leaf images in a tea garden environment, and dividing all the tea bud and leaf images into a training set and a validation set; Build a pose estimation model to be trained; Introducing prior knowledge and establishing a loss function based on the prior knowledge; wherein the prior knowledge includes global geometric features and local gradient direction features; Based on the training set, the posture estimation model to be trained is trained using the loss function to obtain a tea bud leaf posture estimation model; The verification set is input into the tea bud leaf posture estimation model to estimate the posture of the tea bud leaf.

2. The method for estimating the posture of tea buds in a tea garden environment according to claim 1, characterized in that: The introduction of prior knowledge includes: Draw a priori feature map of key points of tea buds and leaves; Performing geometric feature encoding on the prior feature map of the tea bud and leaf key points to obtain global geometric features; Gradient direction feature encoding is performed on the prior feature map of the key points of the tea buds and leaves to obtain local gradient direction features; the global geometric features and the local gradient direction features constitute prior knowledge.

3. The method for estimating the posture of tea buds in a tea garden environment according to claim 1, characterized in that: The establishing of a loss function according to the prior knowledge comprises: Based on the prior knowledge, the loss function is established according to the following formula: Wherein, λ is the scaling factor, K is the prior knowledge, W is the linear projection parameter of the high-dimensional feature to the fully connected layer, β is the regularization strength ratio, Y is the pixel position of the key point predicted by the pose estimation model to be trained, H is the pixel position of the original label, and Loss is the loss function.

4. The method for estimating the posture of tea buds in a tea garden environment according to claim 2, wherein: The method of drawing a priori feature map of key points of tea buds and leaves comprises: Based on the appearance and structural characteristics of the tea buds, the key points of the tea buds are marked; wherein the key points of the tea buds include the middle point of the tea buds, the middle point of the first tea leaf, the intersection of the tea buds and the first tea leaf, and the picking point of the one bud and one leaf stem; The adjacent tea bud and leaf key points are connected into a binary tree structure to obtain a priori feature map of the tea bud and leaf key points.

5. The method for estimating the posture of tea buds and leaves in a tea garden environment according to claim 4, characterized in that: described The geometric feature encoding is performed on the prior feature map of the key points of the tea buds and leaves to obtain global geometric features, including: Performing Hough line transform detection on the prior feature map of the tea bud and leaf key points to obtain multiple line segments in the binary tree structure; All the line segments are encoded to obtain global geometric features.

6. The method for estimating the posture of tea buds in a tea garden environment according to claim 2, characterized in that: The step of performing gradient directional feature encoding on the prior feature map of the key points of the tea buds and leaves to obtain local gradient directional features includes: Dividing the tea bud leaf key point prior feature map into multiple image blocks; Calculating the horizontal gradient and vertical gradient of the image block using the Sobel operator; Calculating a first gradient intensity magnitude and a first direction angle of the horizontal gradient, and calculating a second gradient intensity magnitude and a second direction angle of the vertical gradient; The first direction angle is used to equally divide the interval into M subintervals, and the second direction angle is used to equally divide the interval into N subintervals; Proportionally classifying the first gradient intensity amplitude into M subintervals to obtain a horizontal gradient histogram; The second gradient intensity amplitude is proportionally divided into N subintervals to obtain a vertical gradient histogram; wherein the horizontal gradient histogram and the vertical gradient histogram constitute a local gradient direction feature.

7. The method for estimating the posture of tea buds and leaves in a tea garden environment according to claim 6, characterized in that: After obtaining the vertical gradient histogram, the method further includes: The horizontal gradient histogram is normalized according to the following formula: Among them, f x is the local horizontal gradient direction feature, v x is the horizontal gradient histogram, is the modulus of the horizontal gradient histogram, and e is the adjustment parameter; The vertical gradient histogram is normalized according to the following formula: Among them, f y is the local vertical gradient direction feature, v y is the vertical gradient histogram, is the modulus of the vertical gradient histogram.

8. The method for estimating the posture of tea buds and leaves in a tea garden environment according to claim 1, characterized in that: The method comprises collecting multiple tea bud and leaf images in a tea garden environment and dividing all the tea bud and leaf images into a training set and a validation set, including: A visual sensor is used to collect multiple tea bud and leaf images in a tea garden environment; Labelme software is used to classify and label all the tea bud and leaf images; All the tea bud and leaf images were divided into a training set, a validation set, and a test set in a ratio of 8:1:

1.

9. The method for estimating the posture of tea buds and leaves in a tea garden environment according to claim 8, characterized in that: After dividing all the tea bud leaf images into a training set, a validation set, and a test set in a ratio of 8:1:1, the method further includes: Data enhancement is performed on the training set, where the data enhancement includes any one or more of random horizontal movement, random angle rotation, and random image scaling.

10. The method for estimating the posture of tea buds in a tea garden environment according to claim 2, characterized in that: After drawing the prior feature map of the key points of the tea buds and leaves, the method further includes: According to the following formula, the second-order Taylor expansion is used to decode the coordinates of the extreme points of the tea bud leaf key point prior feature map to improve the positioning accuracy of the peak feature coordinates in the key point prior feature map: Among them, G is the prior feature map of the key points of tea buds and leaves, u is the coordinate of the peak feature in the prior feature map of the key points of tea buds and leaves, and m is the coordinate of the existing prediction point. is the gradient of the existing prediction point in the prior feature map of the key points of tea buds and leaves, and H(m) is the second-order derivative of the existing prediction point in the prior feature map of the key points of tea buds and leaves.

Images