An automatic identification method for tea picking behavior in tea gardens

Abstract

The present invention relates to the field of tea garden tea picking behavior recognition, and specifically to a method for automatically recognizing tea garden tea picking behavior. The scheme includes: obtaining image data of tea picking behavior containing tea baskets; marking human skeletal joints and tea baskets in the image; detecting and marking the marked image, outputting the center coordinates, width, and height of the tea basket, performing mark optimization during the marking process, marking using a bounding box and points with set transparency, and using non-covering marking to display the tea basket detection results only at the edge of the image; using a basic human body detection model to perform human skeletal point sequence detection on the image marked with the tea basket, outputting the coordinates of the human skeletal joints and the confidence of the corresponding human skeletal joints; fusing the tea basket nodes with the human skeletal joints and inputting them into a behavior recognition model, and outputting the probability of the tea picking action category through the behavior recognition model. The accuracy of tea picking behavior recognition is improved. The present invention is suitable for tea picking behavior recognition.

Description

Technical Field

[0001] The present invention relates to the field of tea garden tea picking behavior recognition, and in particular to a method for automatically recognizing tea garden tea picking behavior. Background Art

[0002] In the field of green food production, precision agriculture represented by tea has strict requirements on production process control.

[0003] In the tea production process, the picking stage directly determines the quality of fresh leaves and subsequent processing quality. Traditional tea picking mainly relies on manual experience and has the following technical pain points:

[0004] Inconsistent picking standards: Manual picking is easily affected by subjective factors, such as non-standard picking techniques (pulling, pinching, etc.), mixing of old and young leaves, resulting in poor uniformity of fresh leaves, affecting the quality of finished tea.

[0005] High labor costs: There is a shortage of skilled tea pickers, manual picking is inefficient and the cost is rising year by year, making it difficult to meet production needs, especially during the peak spring tea season.

[0006] Quality traceability is difficult: Manually recorded information such as picking time, location, and personnel is prone to errors, making it difficult to accurately link it to subsequent processing links, which restricts the construction of a tea quality traceability system.

[0007] Existing technologies such as CN111242007A disclose a method for supervising agricultural behavior. This method uses cameras installed at agricultural product planting bases to continuously monitor and collect real-time video streams, and uses a pre-trained behavior detection model to automatically identify the agricultural behavior categories displayed in images or videos, and extract corresponding key frame images and / or key video clips and their corresponding agricultural behavior categories.

[0008] However, these methods cannot effectively address the problem of unstable key points in pose estimation caused by factors such as occlusion, and they also fail to effectively utilize the continuity of motion and the characteristics of video context. Existing machine vision methods based on traditional machine learning have also been applied to the detection of agricultural activities. However, due to their simple image feature extraction methods and the small amount of feature information extracted, they are not very effective in accurately detecting and identifying agricultural activities, resulting in low detection accuracy. This is particularly true for tea picking, where recognition results are poor. Summary of the Invention

[0009] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for automatically identifying tea picking behavior in a tea garden, thereby achieving efficient and accurate identification of tea picking behavior.

[0010] The present invention adopts the following technical solutions to achieve the above-mentioned purpose. The present invention provides a method for automatically identifying tea picking behavior in a tea garden, comprising:

[0011] S1. Acquire image data of tea picking behavior including tea baskets;

[0012] S2, marking the human skeleton joints and tea basket in the image;

[0013] S3. Detect and mark the annotated image using the YOLOv5 object detection model, outputting the center coordinates, width, and height of the tea basket. During the marking process, the marking is optimized, using bounding boxes and points with set transparency. Non-overlapping marking is used, displaying the tea basket detection results only at the edges of the image.

[0014] S4. Use the Alphapose basic human body detection model to perform human skeleton joint sequence detection on the image marked with the tea basket, and output the coordinates of the human skeleton joint points and the confidence scores of the corresponding human skeleton joint points;

[0015] S5. The tea basket nodes are fused with the human skeleton joints and then input into the behavior recognition model, and the behavior recognition model outputs the probability of the tea picking action category.

[0016] Furthermore, the Alphapose basic human detection model includes an improved detection head, a bimodal attention branch module, a spatial transformation network, a single-person pose estimation module, and a parameterized pose non-maximum suppression module;

[0017] Using the Alphapose basic human body detection model to perform human skeleton joint point sequence detection on the tea basket-labeled image includes:

[0018] Replace the original Alphapose detection network with the YOLOv8 detection network, input the labeled tea basket image into the YOLOv8 detection network, and use the detection head of the YOLOv8 detection network to output the human detection box and category confidence;

[0019] The human body detection frame is input into the spatial transformation network for geometric transformation. After the bimodal attention branch module is connected to the output layer of the spatial transformation network, the output of the bimodal attention branch module is used as the input data of the single-person posture estimation module. The data enhanced by the bimodal attention branch module is input into the single-person posture estimation module to generate a heat map of the human skeleton joints. The coordinates of the human skeleton joints are generated through the heat map, and then the coordinates are combined into a posture. The redundant posture detection results are eliminated through the dual-branch distance measurement of the parameterized posture non-maximum suppression module, and the coordinates of the human skeleton joints and the confidence of the corresponding human skeleton joints are output.

[0020] Furthermore, the image with the marked tea basket is input into the YOLOv8 detection network. The detection head of the YOLOv8 detection network outputs the human detection frame and category confidence, specifically including:

[0021] First, the labeled tea basket image is input into the backbone network of the YOLOv8 detection network. It is gradually downsampled through multi-stage feature extraction layers to obtain feature maps of different scales. The feature maps of different scales are input into the neck network. Through upsampling and feature fusion operations, the feature maps of different scales are integrated into a fused feature map. The fused feature map is input into the detection head of the YOLOv8 detection network to generate detection boxes and category confidences. The human body detection box set is then screened out through non-maximum suppression.

[0022] Furthermore, the bimodal attention branch module consists of a bimodal attention branch, a feature fusion mechanism, and a residual optimization strategy. The bimodal attention branch includes an ECA channel branch and a SimAM spatial branch. The ECA channel branch generates a channel-dimensional weight vector through a lightweight gating mechanism and uses 1D global average pooling and cross-channel interaction to calculate channel dependencies. The SimAM spatial branch constructs parameter-free spatial attention based on energy function theory, quantifies the importance of each spatial position through a feature energy map, and preserves the spatial distribution characteristics of the original features.

[0023] The feature fusion mechanism is that the attention weights of the dual-branch output are first normalized by the sigmoid function and then cross-dimensional interaction is achieved through element-by-element multiplication. Subsequently, the features are reorganized through a bottleneck structure consisting of two layers of 1×1 convolution. The first layer of convolution completes channel dimension compression, the batch normalization BN layer and the ReLU activation function enhance the nonlinear expression ability, and the second layer of convolution restores the feature dimension to ensure compatibility with the input features; the residual optimization strategy is that the final output is added to the original input features through the residual connection, and the weight of the last layer of convolution is reset to zero in the initialization stage.

[0024] Furthermore, the tea basket nodes are fused with the human skeleton joints and then input into the behavior recognition model. The behavior recognition model outputs the tea picking action category probabilities, which specifically include:

[0025] The coordinates of the human skeleton joints and the tea basket nodes are spliced together to form a matrix. Each row of the matrix contains (x, y, confidence). Multi-frame node features are stacked along the time axis to form a joint point sequence. x and y represent the coordinates after splicing.

[0026] The input joint point sequence is first standardized by the batch normalization layer of the behavior recognition model, then processed by the improved spatial-temporal convolution unit of the recognition model, and then processed by the average pooling layer and fully connected layer of the recognition model. Finally, the SoftMax classifier of the recognition model outputs the tea picking action category probability.

[0027] Furthermore, the improved space-time convolution unit includes a convolution layer, a first batch of normalization layers, a second batch of normalization layers, a residual connection, a first ADD layer, a second ADD layer, an improved spatial graph convolution module, a spatiotemporal attention mechanism fusion module, a temporal convolution module, a first RELU activation function and a second RELU activation function;

[0028] The input joint point sequence data is processed through the convolution layer and the first batch of normalization layers in turn and then output. The output is residually connected with the original input joint point sequence data. At the same time, the adjacency matrix is input into the improved spatial graph convolution module. The output after the second batch of normalization layers and the data before the residual connection are added in the second ADD layer. The output after addition passes through the second RELU activation function, the spatiotemporal attention mechanism fusion module and the temporal convolution network in turn. The output of the temporal convolution network and the output after the residual connection are added in the first ADD layer, and the result after addition is output after passing through the first RELU activation function.

[0029] Furthermore, the improved spatial graph convolution module is a spatial graph convolution structure that integrates the Transformer module. The input tensor data containing a sequence of human skeletal joint points is input into a two-dimensional convolutional network. After multiplying it with the adjacency matrix, the input is mapped to the Q, K, V space through the first fully connected layer. The attention of Q, K, and V is calculated through the multi-head attention layer, and the weighted value is output. After that, the weighted attention result of a KAN network is nonlinearly transformed and added to the weighted attention again, which is the output result. Q, K, and V are Query, Key, and Value, respectively, representing the three core vectors in the attention mechanism.

[0030] Furthermore, the spatiotemporal attention mechanism fusion module includes a spatial attention mechanism module and a temporal attention mechanism module. The spatial attention mechanism module dynamically adjusts the importance weights of different joints. The specific process is as follows:

[0031] Input features Contains C channels, T time steps and N joint points. First, feature compression is performed along the time dimension through global average pooling to obtain The aggregated features are then used to learn the attention weights of each joint in the spatial dimension. The number of output channels is set to 1 to generate the spatial attention score. The weights are normalized to the [0, 1] interval through the Sigmoid function, and the spatial attention mask is finally output. Multiply the original input feature X point by point to achieve feature enhancement and suppression, and complete feature fusion through residual connection;

[0032] The implementation process of the temporal attention mechanism module is as follows:

[0033] First, global average pooling is performed along the spatial dimension to compress the obtained temporal aggregation features. ,Then the dependency relationship between adjacent time steps is modeled through a one-dimensional convolution operation in the time dimension, and the number of output channels is 1 to generate the temporal attention weight. After normalization by the Sigmoid function, the temporal attention mask is output. , which is weightedly fused with the original input features time-step by time step, and the original feature information is retained through residual connections.

[0034] Furthermore, obtaining image data of tea picking behavior including tea baskets specifically includes:

[0035] Video data of tea picking behavior with tea baskets is obtained and cut into video frames. For blurred video frames, a deblurring model is used to perform deblurring processing.

[0036] Furthermore, the human skeleton joints include nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

[0037] The beneficial effects of the present invention are:

[0038] The present invention obtains image data of tea picking behavior with tea baskets, annotates the human skeletal joints and tea baskets in the image, detects and marks the annotated image using the YOLOv5 object detection model, outputs the center coordinates, width, and height of the tea basket, optimizes the marking process, uses bounding boxes and points with set transparency for marking, and adopts non-covering marking to display the tea basket detection results only at the edge of the image to avoid occluding the human body area and reduce interference with subsequent human skeletal point sequence detection.

[0039] Based on the traditional human skeleton joints (17 joints), a new tea basket node is added. The tea basket node is the 18th node, numbered 17. The two are integrated and unified as the subsequent space-time graph input, which greatly improves the efficiency and accuracy of tea picking behavior recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of a method for automatically identifying tea picking behavior in a tea garden provided by the present invention;

[0041] Figure 2 is a flow chart of the dual-modal attention branch processing data provided by the present invention;

[0042] Figure 3 is a flow chart of processing data by the improved space-time convolution unit provided by the present invention;

[0043] Figure 4This is a flow chart of data processing by the improved spatial graph convolution module provided by the present invention;

[0044] Figure 5 This is a flow chart of data processing by the spatiotemporal attention mechanism fusion module provided by the present invention. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0046] The present invention provides a method for automatically identifying tea picking behavior in a tea garden. Figure 1 As shown, specifically including:

[0047] S1. Acquire image data of tea picking behavior including tea baskets;

[0048] A video clip of tea picking, including tea baskets, was captured and cut into frames. Blurred frames were preprocessed. Deblurring was performed using the DeblurGAN-v2 motion blur model to prevent blur from making key points difficult to identify, which could reduce key point recognition accuracy.

[0049] S2, marking the human skeleton joints and tea basket in the image;

[0050] Perform data labeling on the preprocessed image from step S1. Use the auxiliary tool X-anylabeling to label key points of the human skeleton and the tea basket. This includes the 17 joints of the human skeleton and the tea basket: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle, and tea basket. After the auxiliary labeling is completed, adjust, modify, and relabel.

[0051] S3. Detect and mark the annotated image using the YOLOv5 object detection model, outputting the center coordinates, width, and height of the tea basket. During the marking process, the marking is optimized, using bounding boxes and points with set transparency. Non-overlapping marking is used, displaying the tea basket detection results only at the edges of the image.

[0052] Specifically, you can use bounding boxes and key point markers with a transparency of 30%, 40% or 50%, and use non-covered markers to display the tea basket detection results only at the edge of the image, so as to avoid blocking the human body area and reduce interference with subsequent human skeleton point sequence detection.

[0053] S4. Use the top-down Alphapose basic human body detection model to perform human skeleton joint sequence detection on the image marked with the tea basket, and output the coordinates of the human skeleton joint points and the confidence scores of the corresponding human skeleton joint points;

[0054] Specifically, the Alphapose basic human detection model includes an improved detection head, a bimodal attention branch module, a spatial transformation network, a single-person pose estimation module, and a parameterized pose non-maximum suppression module;

[0055] The original Alphapose detection head, Yolov3, which uses a Darknet-53 backbone network, was replaced with a Yolov8 head using a CSPDarknet backbone network. This design employs a cross-stage partial network design, significantly improving inference speed while maintaining high accuracy, enabling high-precision real-time monitoring. First, the labeled tea basket image is fed into the backbone network. It is then progressively downsampled through multiple feature extraction layers to generate feature maps at different scales. These feature maps are then fed into the neck network, where upsampling and feature fusion are performed to combine the feature maps at different scales into a fused feature map. Finally, the fused feature map is fed into the head network, which generates detection bounding boxes and class confidences. Non-maximum suppression is then used to filter out a set of human detection bounding boxes, coarsely localize human instances, and generate candidate regions that may contain localization errors.

[0056] The detection frame is input into the spatial transformation network for geometric transformation. The input end of the spatial transformation network positioning layer receives the original human body detection frame of the image. The human body detection frame is encoded as a 4-dimensional vector. After inputting it into the input end of the affine transformation parameter generation layer, a 6-dimensional affine matrix is output. The affine transformation parameter generation layer contains 2 fully connected layers.

[0057] The output of the bimodal attention branch module is used as the input data of the single-person pose estimation module. The bimodal attention branch module adopts a dual-branch collaborative design structure and achieves efficient modeling through the dynamic fusion of channel and spatial attention features. The bimodal attention branch module consists of the following core components: bimodal attention branch, feature fusion mechanism, residual optimization strategy; Figure 2 As shown in the figure, the bimodal attention branch includes the ECA (Efficient Channel Attention) channel branch and the SimAM (Simple Attention Module) spatial branch. The ECA channel branch generates a channel dimension weight vector through a lightweight gating mechanism and uses 1D global average pooling and cross-channel interaction to calculate the channel dependency.

[0058] The SimAM spatial branch constructs parameter-free spatial attention based on energy function theory, quantifies the importance of each spatial position through the feature energy map, and retains the spatial distribution characteristics of the original features; the feature fusion mechanism is that the attention weights output by the two branches are first normalized by the sigmoid function and then cross-dimensional interaction is achieved through element-by-element multiplication. Subsequently, the features are reorganized through a bottleneck structure consisting of two layers of 1×1 convolution: the first layer of convolution completes channel dimension compression, the batch normalization layer and the ReLU activation function enhance the nonlinear expression ability, and the second layer of convolution restores the feature dimension to ensure compatibility with the input features.

[0059] The residual optimization strategy is to add the final output to the original input features through a residual connection, and reset the weight of the last layer convolution kernel to zero during the initialization stage to ensure that the contribution of the residual path approaches zero in the early stage of training, effectively avoiding gradient mutations; while reducing computational complexity, the global feature expression ability is improved through a cross-dimensional attention coordination mechanism.

[0060] The data enhanced by the attention mechanism is input into the single-person posture estimation module to generate a key point heat map. The key point coordinates are generated through the heat map, and then the key point coordinates are combined into a posture. Finally, the redundant posture detection results are eliminated through the dual-branch distance metric of the parameterized posture non-maximum suppression module, and the skeleton point coordinates are output. , where n is 16, and Indicates the position coordinates of the nth node, Indicates the confidence level of the point.

[0061] S5. The tea basket nodes are fused with the human skeleton joints and then input into the behavior recognition model, and the behavior recognition model outputs the probability of the tea picking action category.

[0062] Based on the traditional human skeleton diagram (17 joint points), a new tea basket node is added. The tea basket node is the 18th node, numbered 17, and the two are integrated as subsequent input.

[0063] The addition of a tea basket node necessitates updating the original adjacency matrix definition of the spatiotemporal graph convolutional network to construct the spatiotemporal graph. A new connection is added between the tea basket and the right hand. Based on the characteristics of tea plucking, a physical association is established between the tea basket node and the human hand node. Tea plucking primarily involves two hands, so a bidirectional connection is established between the tea basket node and both wrist nodes. The original adjacency matrix A_human is expanded from 17*17 to 18*18 dimensions, and the original matrix A_human is placed in the 17*17 subregion in the upper left corner of the new matrix. A new padding parameter w is added to the matrix to indicate the connection between the wrist and the tea basket, rather than a fixed value of 1.

[0064] The dynamic weight is adjusted and updated according to the distance between the hands and the tea basket. The closer the distance, the higher the weight. The weight calculation formula is: , where d is the Euclidean distance, i is the current node, and j is the neighbor node, so that the spatiotemporal graph convolutional network focuses on the connection between the hands and the tea basket.

[0065] First, the spatiotemporal graph is initialized. The coordinates of 17 human nodes and 1 tea basket node are spliced together to form an 18×3 matrix. Each row of the matrix contains (x, y, confidence), and multi-frame node features are stacked along the time axis to form a joint point sequence of dimension (T, 18, 3). T is the time step, and x and y represent the coordinates after splicing.

[0066] The input joint point sequence is first standardized by the batch normalization layer of the behavior recognition model, then processed by the improved spatial-temporal convolution unit of the behavior recognition model, and then processed by the average pooling layer and fully connected layer of the behavior recognition model. Finally, the SoftMax classifier of the behavior recognition model outputs the tea picking behavior category probability.

[0067] The improved space-time convolution unit mainly includes an improved spatial graph convolution module, a spatiotemporal attention mechanism fusion module, a temporal convolution network, and a residual structure from input to output. The residual structure is used to mitigate network degradation and gradient disappearance problems.

[0068] like Figure 3 As shown, the original joint point sequence is input into the convolution layer, and the output of the first batch of normalization layers is residually connected with the original input, i.e., the original joint point sequence. At the same time, the adjacency matrix is input into the improved spatial graph convolution module. The output after the second batch of normalization layers is added to the data before the residual connection in the second ADD layer. The output result passes through the second RELU activation function, the spatiotemporal attention mechanism fusion module and the temporal convolution network. The output result of the temporal convolution network and the output after the residual connection are added in the first ADD layer, and finally output after the first RELU activation function.

[0069] like Figure 4As shown in the figure, the improved spatial graph convolution module is a spatial graph convolution structure that integrates the Transformer module. The original joint point sequence is input into the two-dimensional convolutional network. After multiplying with the adjacency matrix, the first fully connected layer maps the input to the Q, K, and V space. The multi-head attention layer performs attention calculations on the above Q, K, and V. The number of attention heads in the multi-head attention layer is 4, and the weighted value is output. After that, the weighted attention result is nonlinearly transformed through a KAN (Kolmogorov-Arnold Networks) and added to the weighted attention again to obtain the output result. Q, K, and V are Query, Key, and Value, respectively, representing the three core vectors in the attention mechanism.

[0070] like Figure 5 As shown in Figure 1, the spatiotemporal attention mechanism fusion module includes the SAM (Spatial Attention Mechanism) module and the TAM (Temporal Attention Mechanism) module. The spatial attention mechanism module dynamically adjusts the importance weights of different joints, enabling the model to adaptively enhance the feature response of key areas based on different action characteristics. The specific implementation process is as follows: Among them, the input features Contains C channels, T time steps and N joint points. First, feature compression is performed along the time dimension through global average pooling (AvgPool), and we get Aggregate features. Subsequently, a one-dimensional convolution (Conv1D) operation is performed on the spatial dimension. The attention weights of each joint are learned in the spatial dimension, and the number of output channels is set to 1 to generate the spatial attention score; the weights are normalized to the [0,1] interval through the Sigmoid function, and the spatial attention mask is finally output. Multiply the original input feature X point by point to achieve feature enhancement and suppression, and complete feature fusion through residual connection.

[0071] Similarly, the implementation process of the temporal attention mechanism module is as follows: First, global average pooling is performed along the spatial dimension to compress the obtained temporal aggregation features. . Then, through the one-dimensional convolution (Conv1D) operation in the time dimension Modeling the dependency between adjacent time steps, the number of output channels is 1 to generate temporal attention weights. After normalization, the output temporal attention mask is , which is weightedly fused with the original input features in a time-step manner and retains the original feature information through residual connections;

[0072] After the data is input, it first passes through the spatial attention mechanism module, and the output of the spatial attention mechanism module Perform point multiplication with input X, and then perform residual connection with input data X to obtain , after the time attention mechanism module, the output is ,and After point multiplication, residual connection is performed and the output is finally obtained .

[0073] The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. A method for automatically identifying tea picking behavior in a tea garden, characterized in that: include: S1. Acquire image data of tea picking behavior including tea baskets; S2, marking the human skeleton joints and tea basket in the image; S3. Detect and mark the annotated image using the YOLOv5 object detection model, outputting the center coordinates, width, and height of the tea basket. During the marking process, the marking is optimized, using bounding boxes and points with set transparency. Non-overlapping marking is used, displaying the tea basket detection results only at the edges of the image. S4. Use the Alphapose basic human body detection model to perform human skeleton joint sequence detection on the image marked with the tea basket, and output the coordinates of the human skeleton joint points and the confidence scores of the corresponding human skeleton joint points; S5. The tea basket nodes are fused with the human skeleton joints and then input into the behavior recognition model, and the tea picking behavior category probability is output through the behavior recognition model.

2. The automatic identification method for tea picking behavior in a tea garden according to claim 1, characterized in that: The Alphapose basic human detection model includes an improved detection head, a bimodal attention branch module, a spatial transformation network, a single-person pose estimation module, and a parameterized pose non-maximum suppression module; Using the Alphapose basic human body detection model to perform human skeleton joint point sequence detection on the tea basket-labeled image includes: Replace the original Alphapose detection network with the YOLOv8 detection network, input the labeled tea basket image into the YOLOv8 detection network, and use the detection head of the YOLOv8 detection network to output the human detection box and category confidence; The human body detection frame is input into the spatial transformation network for geometric transformation. After the bimodal attention branch module is connected to the output layer of the spatial transformation network, the output of the bimodal attention branch module is used as the input data of the single-person posture estimation module. The data enhanced by the bimodal attention branch module is input into the single-person posture estimation module to generate a heat map of the human skeleton joints. The coordinates of the human skeleton joints are generated through the heat map, and then the coordinates are combined into a posture. The redundant posture detection results are eliminated through the dual-branch distance measurement of the parameterized posture non-maximum suppression module, and the coordinates of the human skeleton joints and the confidence of the corresponding human skeleton joints are output.

3. The automatic identification method for tea picking behavior in a tea garden according to claim 2, characterized in that: The labeled tea basket image is input into the YOLOv8 detection network. The detection head of the YOLOv8 detection network outputs the human detection frame and category confidence, including: First, the image marked with the tea basket is input into the backbone network of the YOLOv8 detection network. It is gradually downsampled through multi-stage feature extraction layers to obtain feature maps of different scales. The feature maps of different scales are input into the neck network. Through upsampling and feature fusion operations, the feature maps of different scales are integrated into a fused feature map. The fused feature map is input into the detection head of the YOLOv8 detection network to generate detection boxes and category confidences. The human body detection box set is then screened out through non-maximum suppression.

4. The method for automatically identifying tea picking behavior in a tea garden according to claim 2, wherein: The bimodal attention branch module consists of a bimodal attention branch, a feature fusion mechanism, and a residual optimization strategy. The bimodal attention branch includes an ECA channel branch and a SimAM spatial branch. The ECA channel branch generates a channel-dimensional weight vector through a lightweight gating mechanism and uses 1D global average pooling and cross-channel interaction to calculate channel dependencies. The SimAM spatial branch constructs parameter-free spatial attention based on energy function theory, quantifies the importance of each spatial position through a feature energy map, and preserves the spatial distribution characteristics of the original features. The feature fusion mechanism is that the attention weights of the dual-branch output are first normalized by the sigmoid function and then cross-dimensional interaction is achieved through element-by-element multiplication. Subsequently, the features are reorganized through a bottleneck structure consisting of two layers of 1×1 convolution. The first layer of convolution completes channel dimension compression, the batch normalization layer and the ReLU activation function enhance the nonlinear expression ability, and the second layer of convolution restores the feature dimension to ensure compatibility with the input features; the residual optimization strategy is that the final output is added to the original input features through the residual connection, and the weight of the last layer of convolution is reset to zero in the initialization stage.

5. The method for automatically identifying tea picking behavior in a tea garden according to claim 1, characterized in that: The tea basket nodes are fused with the human skeleton joints and then input into the behavior recognition model. The behavior recognition model outputs the tea picking action category probabilities, which include: The coordinates of the human skeleton joints and the tea basket nodes are spliced together to form a matrix. Each row of the matrix contains (x, y, confidence). Multi-frame node features are stacked along the time axis to form a joint point sequence. x and y represent the coordinates after splicing. The input joint point sequence is first standardized by the batch normalization layer of the behavior recognition model, then processed by the improved spatial-temporal convolution unit of the behavior recognition model, and then processed by the average pooling layer and fully connected layer of the behavior recognition model. Finally, the SoftMax classifier of the behavior recognition model outputs the tea picking behavior category probability.

6. The method for automatically identifying tea picking behavior in a tea garden according to claim 5, characterized in that: The improved space-time convolution unit includes a convolution layer, a first batch of normalization layers, a second batch of normalization layers, a residual connection, a first ADD layer, a second ADD layer, an improved spatial graph convolution module, a spatiotemporal attention mechanism fusion module, a temporal convolution module, a first RELU activation function and a second RELU activation function; The input joint point sequence data is processed through the convolution layer and the first batch of normalization layers in turn and then output. The output is residually connected with the original input joint point sequence data. At the same time, the adjacency matrix is input into the improved spatial graph convolution module. The output after the second batch of normalization layers and the data before the residual connection are added in the second ADD layer. The output after addition passes through the second RELU activation function, the spatiotemporal attention mechanism fusion module and the temporal convolution network in turn. The output of the temporal convolution network and the output after the residual connection are added in the first ADD layer, and the result after addition is output after passing through the first RELU activation function.

7. The method for automatically identifying tea picking behavior in a tea garden according to claim 6, characterized in that: The improved spatial graph convolution module is a spatial graph convolution structure that integrates the Transformer module. The input data containing a sequence of human skeletal joint points is input into a two-dimensional convolutional network. After multiplying with the adjacency matrix, the input is mapped to the Q, K, and V spaces through the first fully connected layer. The attention of Q, K, and V is calculated through a multi-head attention layer, and a weighted value is output. The weighted attention result is then nonlinearly transformed through a KAN network and added to the weighted attention again to obtain the output result. Q, K, and V are Query, Key, and Value, respectively, representing the three core vectors in the attention mechanism.

8. The method for automatically identifying tea picking behavior in a tea garden according to claim 6, characterized in that: The spatiotemporal attention mechanism fusion module includes a spatial attention mechanism module and a temporal attention mechanism module. The spatial attention mechanism module dynamically adjusts the importance weights of different joints. The specific process is as follows: Input features Contains C channels, T time steps and N joint points. First, feature compression is performed along the time dimension through global average pooling to obtain The aggregated features are then used to learn the attention weights of each joint in the spatial dimension. The number of output channels is set to 1 to generate the spatial attention score. The weights are normalized to the [0,1] interval through the Sigmoid function, and the spatial attention mask is finally output. Multiply the original input feature X point by point to achieve feature enhancement and suppression, and complete feature fusion through residual connection; The implementation process of the temporal attention mechanism module is as follows: First, global average pooling is performed along the spatial dimension to compress the obtained temporal aggregation features. ,Then the dependency relationship between adjacent time steps is modeled through a one-dimensional convolution operation in the time dimension, and the number of output channels is 1 to generate the temporal attention weight. After normalization by the Sigmoid function, the temporal attention mask is output. , which is weightedly fused with the original input features time-step by time step, and the original feature information is retained through residual connections.

9. The method for automatically identifying tea picking behavior in a tea garden according to claim 1, wherein: The image data of tea picking behavior including tea baskets is obtained by: Video data of tea picking behavior with tea baskets is obtained and cut into video frames. For blurred video frames, a deblurring model is used to perform deblurring processing.

10. The method for automatically identifying tea picking behavior in a tea garden according to claim 1, characterized in that: The human skeleton joints include nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

Images