A tea bud detection method, system and electronic device in a complex background

Abstract

This invention provides a method, system, and electronic device for detecting tea buds under complex backgrounds, belonging to the field of image recognition technology. The method includes: inputting an acquired tea bud image into a feature extraction network, and using a multi-branch parallel attention mechanism to extract multi-scale feature maps from the tea bud image; inputting the multi-scale feature maps into a feature fusion network for cross-scale feature fusion processing to generate fused features; inputting a deep feature map into an encoder for self-attention calculation, generating position codes for each spatial location based on the positional relationships of each spatial location in the deep feature map, and fusing the position codes with the feature sequences of the deep feature map to output encoded enhanced features; inputting the encoded enhanced features and fused features into a decoder, and generating category and position information of the tea buds through a query selection mechanism and cross-attention calculation. This invention effectively improves the detection accuracy and robustness of tea buds under complex backgrounds.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a method, system, and electronic device for detecting tea buds under complex backgrounds. Background Technology

[0002] As a crucial component of China's distinctive agricultural products, the tea industry plays a vital role in promoting rural economic development and rural revitalization. The quality of premium teas is measured by standards such as "tenderness, uniformity, and cleanliness," and their harvesting is highly dependent on the growth state of the tender buds in early spring, requiring significant timeliness and technical expertise. Currently, the harvesting of premium teas still heavily relies on manual labor, with labor costs accounting for over 50% of the total tea production cost. With the increasing structural shortage of rural labor, the situation of "labor shortage" during the tea-harvesting season is becoming increasingly frequent. Achieving automated and intelligent harvesting of tea buds has become an urgent need for the transformation and upgrading of the tea industry. The key to the development of intelligent tea-harvesting robots lies in their visual perception system—that is, how to accurately detect and locate tea buds in the natural tea garden environment.

[0003] Existing methods for detecting tea buds can be broadly categorized into two types: traditional image processing methods and deep learning-based detection methods. Traditional methods rely on manually designed low-level visual features, extracting candidate regions through color space transformation, thresholding, and edge detection. For example, they utilize color differences between buds and older leaves in RGB, HSV, or Lab color spaces for region segmentation, and combine Otsu thresholding and Sobel and Canny edge detection operators for target extraction. With the development of deep learning technology, methods based on convolutional neural networks (CNNs) have gradually become mainstream. Researchers have introduced target detection models such as Faster R-CNN and the YOLO series into tea bud detection tasks. Building on this, some studies have further explored lightweight designs, improving detection efficiency by introducing the Ghost module and coordinate attention mechanism in YOLOv5, and adding a convolutional attention module to the small target detection layer in YOLOv7tiny. In recent years, the Transformer architecture, with its self-attention mechanism's advantages in global feature modeling, has been gradually introduced into agricultural vision tasks. RTDETR, as a detection Transformer that balances global modeling capabilities and real-time performance, has shown good application potential in the field of agricultural detection.

[0004] However, existing methods for detecting tea buds still have the following shortcomings: First, CNN-based detection models are limited by their local receptive fields, making it difficult to effectively model long-distance dependencies and global contextual information. This can easily lead to missed detections and false detections in scenarios where buds and backgrounds are highly similar and targets are densely distributed. Second, existing methods lack the ability to extract fine-grained features from small-scale, dense targets. This can easily cause the loss of edge and texture information during multiple downsampling processes, resulting in limited detection accuracy. Third, although the Transformer model can achieve global modeling, its standard structure has high computational complexity, making it difficult to deploy directly on edge devices with limited computing power, such as tea-picking robots. Fourth, the robustness of existing methods under complex lighting, occlusion, and background interference conditions still needs improvement. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, system and electronic device for detecting tea buds under complex backgrounds, so as to solve the technical problem in the prior art that it is difficult to balance the accuracy and robustness of tea bud detection in complex natural environments.

[0006] To address the aforementioned technical problems, in a first aspect, the present invention provides a method for detecting tea buds under complex backgrounds, comprising: The acquired tea bud images are input into a feature extraction network, and a multi-branch parallel attention mechanism is used to extract multi-scale feature maps from the tea bud images. The multi-scale feature map is input into a feature fusion network for cross-scale feature fusion processing to generate fused features; The deep feature map in the multi-scale feature map is input into the encoder for self-attention calculation. The positional code of each spatial position is generated according to the positional relationship of each spatial position in the deep feature map. The positional code is fused with the feature sequence of the deep feature map to output the encoded enhanced feature. The encoded enhancement features and the fused features are input into the decoder. The target query is filtered from the encoded enhancement features through a query selection mechanism. The target query is then subjected to cross-attention calculation with the encoded enhancement features and the fused features to generate category information and location information of tea buds.

[0007] In one possible implementation, the multi-branch parallel attention mechanism includes local branches, and the convolution operation parameters of the local branches are dynamically adjusted during the feature extraction process.

[0008] In one possible implementation, the dynamic adjustment of the convolution operation parameters of the local branches includes: Global pooling is performed on the multi-scale feature map of the input local branch to extract global descriptive information of the channel dimension; The global description information is mapped to the weight coefficients corresponding to each basic convolutional kernel; The dynamic convolution kernel is obtained by weighting and summing each basic convolution kernel according to the weight coefficients. The dynamic convolution kernel is used to perform convolution operations on the multi-scale feature map of the local branch.

[0009] In one possible implementation, generating the positional encoding of each spatial location based on the positional relationships of each spatial location in the deep feature map includes: Each spatial location in the deep feature map is taken as a graph node, and the local neighborhood of each graph node is determined according to the spatial position relationship. The local neighborhood is a set of other graph nodes that are spatially adjacent to the graph node. A sparse adjacency graph is constructed based on the feature similarity between a graph node and other graph nodes in its local neighborhood. At least some feature information of the Laplacian matrix is ​​determined based on the sparse adjacency graph; the Laplacian matrix is ​​a matrix characterizing the adjacency relationships of graph nodes. Based on the aforementioned feature information, a location code for each spatial position is constructed.

[0010] In one possible implementation, the feature extraction network, the feature fusion network, the encoder, and the decoder constitute an object detection model, which is trained in the following manner: The category information and location information generated by the decoder are optimized using a bounding box regression loss function. The bounding box regression loss function adjusts the loss contribution of different samples according to the degree of overlap between the predicted location information and the real location information. The parameters of the object detection model are updated based on the calculation results of the bounding box regression loss function.

[0011] In one possible implementation, the bounding box regression loss function adjusts the loss contribution of different samples as follows: Determine the intersection-union ratio (IoU) between the predicted location information and the actual location information; The sample weights are determined based on the positional relationship between the intersection-union ratio and the preset interval, so that samples located within the preset interval receive a higher loss contribution than samples located outside the preset interval.

[0012] In one possible implementation, the cross-scale feature aggregation process employs a lightweight convolution operation, which rearranges the channel order after performing grouped convolution and depthwise separable convolution on the multi-scale feature maps.

[0013] In one possible implementation, the cross-scale feature aggregation adopts a cross-stage splitting structure, which divides the multi-scale feature map into two branches along the channel dimension. One branch is transformed by the lightweight convolution operation, and the other branch directly retains the original features. The transformed features and the retained original features are concatenated along the channel dimension and then output.

[0014] Secondly, the present invention also provides a tea bud detection system under complex backgrounds, comprising: The image acquisition module is used to acquire images of tea buds; The feature extraction module is used to extract multi-scale feature maps of the tea bud image using a multi-branch parallel attention mechanism; The feature fusion module is used to perform cross-scale feature fusion processing on the multi-scale feature map to generate fused features; The encoding module is used to input the deep feature map in the multi-scale feature map into the encoder for self-attention calculation, generate position codes for each spatial position according to the positional relationship of each spatial position in the deep feature map, and fuse the position codes with the feature sequence of the deep feature map to output the encoded enhanced features; The decoding module is used to input the encoded enhancement features and the fused features into the decoder, filter the target query from the encoded enhancement features through a query selection mechanism, and perform cross-attention calculation on the target query, the encoded enhancement features and the fused features to generate category information and location information of tea buds.

[0015] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the tea bud detection method under complex backgrounds described in any of the above implementations.

[0016] The beneficial effects of this invention are as follows: Firstly, by employing a multi-branch parallel attention mechanism including local branch dynamic convolution in the feature extraction stage, the convolution kernel response can be adaptively adjusted according to local image features, effectively improving the model's fine-grained feature extraction capability under complex lighting, occlusion, and cluttered background conditions. Furthermore, by introducing learnable positional encoding based on sparse graph Laplacian spectrum during the encoder's self-attention calculation process, the model can adaptively adjust its positional representation according to the actual spatial distribution of targets in the image, enhancing spatial modeling accuracy and the ability to distinguish dense targets. Simultaneously, by employing a feature fusion network including lightweight convolution and cross-stage splitting structures, the computational complexity of the model is significantly reduced while ensuring multi-scale feature representation capabilities, significantly improving the accuracy and robustness of tea bud detection in complex natural environments while maintaining real-time detection efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart of an embodiment of the tea bud detection method under complex background provided by the present invention; Figure 2 A schematic flowchart of another embodiment of the tea bud detection method under complex background provided by the present invention; Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S103; Figure 4 A schematic flowchart of another embodiment of the tea bud detection method under complex background provided by the present invention; Figure 5 A schematic flowchart of another embodiment of the tea bud detection method under complex background provided by the present invention; Figure 6 A schematic diagram of an embodiment of the target detection model provided by the present invention; Figure 7 A schematic diagram of an embodiment of the tea bud detection system under complex background provided by the present invention; Figure 8 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0021] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] This invention provides a method, system, and electronic device for detecting tea buds under complex backgrounds, which will be described below.

[0024] Figure 1 A schematic flowchart of an embodiment of the tea bud detection method under complex backgrounds provided by the present invention is shown below. Figure 1 As shown, the methods for detecting tea buds under complex backgrounds include: S101. Input the acquired tea bud image into the feature extraction network, and use a multi-branch parallel attention mechanism to extract multi-scale feature maps of the tea bud image.

[0025] It should be noted that after acquiring the image of the tea buds to be detected, the image of the tea buds is input into the feature extraction network. The feature extraction network adopts the CSPDarknet structure. This structure extracts multi-scale feature maps of the tea bud image, such as S3, S4, and S5, while preserving low-level detailed features through cross-stage partial feature partitioning and residual gradient propagation mechanism.

[0026] S102. Input the multi-scale feature map into the feature fusion network for cross-scale feature fusion processing to generate fused features.

[0027] It should be noted that the output multi-scale feature maps (S3, S4, S5) are input into the feature fusion network for cross-scale feature fusion processing. The feature fusion network adopts the Crossscale Context Fusion Module (CCFM), which efficiently fuses deep semantic information and shallow spatial information through convolution operations to generate fused features.

[0028] S103. Input the deep feature map in the multi-scale feature map into the encoder for self-attention calculation, generate position codes for each spatial position according to the positional relationship of each spatial position in the deep feature map, and fuse the position codes with the feature sequence of the deep feature map to output the encoded enhanced features.

[0029] It should be noted that: the deep feature map S5 from the multi-scale feature map output in step S101 is obtained, and this deep feature map is input into the Transformer encoder for self-attention calculation. During the self-attention calculation process, the positional encoding of each spatial location is first generated based on the positional relationships of each spatial location in the deep feature map.

[0030] S104. Input the encoded enhancement features and the fusion features into the decoder, filter the target query from the encoded enhancement features through a query selection mechanism, and perform cross-attention calculation on the target query, the encoded enhancement features and the fusion features to generate category information and location information of tea buds.

[0031] It should be noted that the encoded augmented features and fused features are jointly input into the Transformer decoder. First, a query selection mechanism filters the target query from the encoded augmented features. This mechanism not only utilizes classification scores but also incorporates the localization uncertainty of the bounding boxes as a filtering criterion, selecting features with high classification probabilities and strong localization confidence as the initial target query to accelerate model convergence. Then, the filtered target query, encoded augmented features, and fused multi-scale features undergo cross-attention calculation. In the cross-attention calculation, the target query serves as the query vector for the attention mechanism, while the encoded augmented features and fused features serve as the key and value vectors, respectively. A multi-head attention mechanism facilitates information interaction between the target query and image features. Through this cross-attention calculation, the decoder directly outputs the category and location information of the tea buds, without relying on post-processing steps such as non-maximum suppression (NMS), achieving end-to-end target detection.

[0032] In summary, the tea bud detection method under complex backgrounds provided by this invention firstly employs a multi-branch parallel attention mechanism with local branch dynamic convolution in the feature extraction stage. This mechanism adaptively adjusts the convolution kernel response based on local image features, effectively improving the model's fine-grained feature extraction capability under complex lighting, occlusion, and cluttered background conditions. Furthermore, by introducing learnable positional encoding based on sparse graph Laplacian spectrum during the encoder's self-attention calculation process, the model can adaptively adjust its positional representation according to the actual spatial distribution of targets in the image, enhancing spatial modeling accuracy and the ability to distinguish dense targets. Simultaneously, by employing a feature fusion network containing lightweight convolution and a cross-stage splitting structure, the computational complexity of the model is significantly reduced while maintaining multi-scale feature representation capabilities. This significantly improves the accuracy and robustness of tea bud detection under complex natural environments while maintaining real-time detection efficiency.

[0033] In some embodiments of the present invention, the multi-branch parallel attention mechanism includes local branches, and the convolution operation parameters of the local branches are dynamically adjusted during the feature extraction process.

[0034] It should be noted that the feature extraction network incorporates a multi-branch parallel attention mechanism. This mechanism captures spatial features at different receptive field scales through multiple parallel feature extraction branches, specifically including local branches, global branches, and contextual branches. The local branches employ dynamic convolution to adaptively extract local detail features. This dynamic convolution mechanism enables the convolutional filters to adaptively adjust their weights and receptive field size based on the local image statistical characteristics, thereby enhancing the model's ability to perceive the background. Through this process, the feature extraction network outputs multi-scale feature maps of the tea bud image, including shallow feature map S3, mid-level feature map S4, and deep feature map S5. These feature maps correspond to different downsampling factors and semantic abstraction levels, providing a foundation for subsequent multi-scale feature fusion and self-attention modeling.

[0035] In some embodiments of the present invention, such as Figure 2 As shown, the dynamic adjustment of the convolution operation parameters of the local branch includes: S201. Perform global pooling on the multi-scale feature map of the input local branch to extract global descriptive information of the channel dimension.

[0036] It should be noted that: for the multi-scale feature map X∈R of the local input branch... (C×H×W)Global Average Pooling (GAP) is performed, averaging all pixel values ​​in the height and width dimensions of each channel to obtain the global statistics for that channel. This operation compresses the spatial dimension H×W of the feature map to 1×1, outputting C-dimensional global descriptive information that characterizes the global response intensity distribution of each channel in the input feature map.

[0037] S202. Map the global description information to the weight coefficients corresponding to each basic convolution kernel.

[0038] It should be noted that: the global description information is sequentially input into two fully connected layers for non-linear mapping, and a ReLU activation function is applied between the two fully connected layers for activation processing. After two fully connected layers, the output dimension matches the number of basic convolutional kernels n (n=4 in this embodiment). The above output result is then input into the Softmax function, which maps it to the weight coefficients α1, α2, ..., α3 corresponding to each basic convolutional kernel. n And the sum of all weighting coefficients is 1.

[0039] S203. The weighted summation of each basic convolutional kernel is performed according to the weight coefficients to obtain the dynamic convolutional kernel; S204. The dynamic convolution kernel is used to perform convolution operation on the multi-scale feature map of the local branch.

[0040] It should be noted that: let the predefined set of basic convolutional kernels be {W1, W2, ..., W...} n}, where each basic convolutional kernel is a standard convolutional kernel of size k×k. The weight coefficients α1, α2, ..., α3 obtained in step S202 are... n With the corresponding basic convolutional kernels W1, W2, ..., W n Multiply each product separately, then sum the results to obtain the dynamic convolution kernel W. dyn The calculation method is as follows: W dyn =Σ(α i ×W i ), i = 1 to n, where n is a positive integer. The dynamic convolution kernel W dyn The convolutional layer parameters for local branches perform standard convolution operations on the multi-scale feature maps input to those local branches, outputting a local feature map after dynamic convolution processing. This dynamic convolution kernel is dynamically generated based on the content of the current input features and is closely related to them, thus enabling it to adaptively respond to feature extraction needs under different input conditions.

[0041] In this embodiment, through the aforementioned dynamic convolution mechanism, the aggregated weights of the convolution kernels are adaptively generated based on the input features. This allows the convolution operation to dynamically adjust its parameters according to different input content (such as tea bud images under different lighting conditions, different degrees of occlusion, and different background complexities), enhancing the model's adaptability to various complex visual conditions. Furthermore, since the weight generation network consists of only global average pooling and two fully connected layers, the computational overhead is extremely small, and the weighted summation of multiple basic convolution kernels only adds a small number of multiply-accumulate operations. The model maintains high computational efficiency while enhancing feature representation capabilities, effectively improving the model's ability to extract fine-grained features of tea buds and its robustness in detection within complex natural tea garden environments.

[0042] In some embodiments of the present invention, such as Figure 3 As shown, the step of generating the positional code for each spatial location based on the positional relationship of each spatial location in the deep feature map includes: S301. Take each spatial location in the deep feature map as a graph node, and determine the local neighborhood of each graph node according to the spatial position relationship. The local neighborhood is a set of other graph nodes that are spatially adjacent to the graph node.

[0043] It should be noted that: let the deep feature map output by the backbone network be represented as X∈R C×H×W Here, H and W represent the height and width of the feature map, respectively, and C represents the number of channels. Each spatial location in the deep feature map is treated as a graph node, resulting in N = H × W nodes. For each graph node, its local neighborhood is determined based on its spatial position within the feature map. The local neighborhood refers to the set of other graph nodes spatially adjacent to that node, which can be defined using a fixed neighborhood window. For example, using a 3×3 neighborhood window, each graph node's local neighborhood includes itself and its eight adjacent nodes in the four diagonal directions (nodes located at the edges of the feature map only have their actual adjacent nodes in their local neighborhood). In this way, the regular grid topology of the deep feature map is mapped to a graph structure, where each node only establishes a connection with its spatially adjacent nodes, rather than connecting with all nodes.

[0044] S302. Construct a sparse adjacency graph based on the feature similarity between the graph node and other graph nodes in the local neighborhood of the graph node.

[0045] It should be noted that for any graph node and other graph nodes in its local neighborhood, the feature similarity between them is calculated. This feature similarity measures the proximity of two graph nodes in the feature space, specifically using a combination of Euclidean distance and cosine similarity. For two graph nodes i and j, let their feature vectors be respectively... and First, calculate the Euclidean distance between the two. Then calculate the cosine similarity between the two. Then, the two are weighted and combined to form the edge weights between nodes. Since each node only calculates similarity with nodes in its local neighborhood, the constructed adjacency matrix is ​​a sparse matrix, and its non-zero elements only correspond to pairs of nodes with local adjacency relationships, which greatly reduces the number of edges in the graph and reduces the complexity of subsequent calculations.

[0046] S303. Determine at least some feature information of the Laplace matrix based on the sparse adjacency graph; the Laplace matrix is ​​a matrix that characterizes the adjacency relationship of graph nodes.

[0047] It should be noted that the Laplacian matrix is ​​calculated from a sparse adjacency graph. The Laplacian matrix is ​​defined as L=DA, where A is the adjacency matrix (its elements A... ij Let D represent the edge weights between node i and node j, and D be the degree matrix (a diagonal matrix whose diagonal elements are the degrees of each node). Since the adjacency matrix is ​​sparse, the Laplacian matrix is ​​also sparse. In actual computation, there is no need to perform eigenvalue decomposition on the complete Laplacian matrix. Instead, a sparse adjacency matrix and Top-m eigenvalue approximation strategy is used, retaining only the first m low-frequency eigenvalues ​​and their corresponding eigenvectors. Low-frequency eigenvalues ​​refer to the smallest eigenvalues ​​in the Laplacian matrix, and their corresponding eigenvectors reflect the global smooth structure information of the graph. In this embodiment, m is set to 16 to balance spatial representation capability and computational efficiency.

[0048] S304. Construct a location code for each spatial location based on the feature information.

[0049] It should be noted that: for each spatial location i, its corresponding m low-frequency feature values ​​are... And the components of the eigenvectors corresponding to each of these m eigenvalues ​​at the i-th position. By combining them, the original spectral coding vector is obtained. Then, the original spectral encoding vector is input into the linear mapping function. Map it to a learnable positional encoding of the target dimension d. Through this linear mapping, the model can adaptively adjust the representation of the positional encoding according to task requirements during training. Finally, the generated positional encoding is added to and fused with the corresponding feature sequences of the deep feature map, so that the feature sequences, carrying spatial location information, participate in subsequent self-attention calculations.

[0050] In practical implementation, the original input image is not directly constructed. Instead, graph structure modeling is performed on the deep feature map after downsampling by the backbone network. For example, when the scale of the deep feature map is approximately 20×20 or 40×40, the number of nodes is 400 or 1600 respectively, and the computational cost is within an acceptable range. Furthermore, in actual computation, the complete Laplacian matrix is ​​not decomposed. Instead, a sparse adjacency matrix and Top-m eigenvalue approximation strategy is adopted, retaining only the first m low-frequency eigenvalues ​​and their corresponding eigenvectors for location encoding construction to reduce computational complexity. In this embodiment, m is set to 16 to balance spatial expressiveness and computational efficiency. The generated location encoding is then added and fused with the feature sequence of the deep feature map, allowing the feature sequence to carry spatial location information. The fused feature sequence is input into the Transformer encoder for self-attention computation, outputting encoded enhanced features. By introducing the above-mentioned learnable location encoding mechanism, the model can learn spatial embedding representations based on spectral information, significantly enhancing its discriminative ability in the spatial domain, especially suitable for fine-grained detection tasks with significant target morphological changes and strong background interference.

[0051] In this embodiment, through the aforementioned learnable positional encoding mechanism based on the Laplacian spectrum, the model constructs positional codes using low-frequency eigenvalues ​​and eigenvectors of the Laplacian matrix. This enables the model to capture the global topological structure and intrinsic geometric relationships between spatial locations in the feature map from the spectral domain, giving the positional codes clear mathematical meaning and spatial interpretability. Furthermore, by employing a sparse adjacency graph construction and Top-m eigenvalue approximation strategy, graph modeling is performed only on deep feature maps, and only the first 16 low-frequency features are retained. This significantly reduces the computational complexity of Laplacian matrix eigenvalue decomposition, making it feasible for practical deployment. In summary, this invention, while introducing acceptable computational overhead, significantly enhances the model's ability to perceive spatial relationships and distinguish densely distributed, mutually occluded targets.

[0052] In some embodiments of the present invention, such as Figure 4 As shown, the feature extraction network, the feature fusion network, the encoder, and the decoder constitute the target detection model, which is trained in the following manner: S401. The category information and location information generated by the decoder are optimized using a bounding box regression loss function. The bounding box regression loss function adjusts the loss contribution of different samples according to the degree of overlap between the predicted location information and the real location information.

[0053] It should be noted that: such as Figure 6 As shown, Figure 6This is a schematic diagram of the overall architecture of the lightweight DPPADet-RT object detection model. The object detection model adopts an end-to-end Transformer detection architecture, consisting of three parts: a backbone network, a feature fusion network, and a decoder. During model training, images of tea buds are input into the object detection model. After feature extraction, feature fusion, encoder self-attention calculation, and decoder cross-attention calculation, the model outputs predicted category and location information. The location information is represented by a rectangular bounding box, including the center coordinates, width, and height of the bounding box. A bounding box regression loss function is used to optimize the location information output by the decoder. In this embodiment, the bounding box regression loss function is the Focaler-GIoU loss function. First, the intersection-over-union (IoU) ratio between the predicted bounding box and the ground truth bounding box is calculated, which is the ratio of the intersection area to the union area of ​​the predicted bounding box and the ground truth bounding box. Then, the IoU value is remapped according to the following segmentation method to obtain the focused IoU ratio (IoU). ): Through the aforementioned loss function, for difficult samples with low intersection-union ratios (IU) (such as occluded or blurred tea buds), the loss function dynamically adjusts the gradient contribution based on the IU, allowing the model to focus more on these difficult-to-detect samples during training. For easy samples with high IU, the gradient contribution is maintained at a reasonable level, preventing it from dominating the loss function. Simultaneously, by incorporating the GIoU mechanism, even when the predicted bounding box and the ground truth bounding box do not overlap at all, the loss function can still apply an effective penalty signal through the minimum bounding box region, guiding the predicted bounding box to move towards the ground truth bounding box.

[0054] S402. Update the parameters of the target detection model based on the calculation results of the bounding box regression loss function.

[0055] It should be noted that the result of the Focaler-GIoU loss function is used as the total bounding box regression loss value of the model, which, combined with the classification loss value, constitutes the total training loss of the model. The gradient of the total loss relative to each trainable parameter of the model is calculated using the backpropagation algorithm. Then, the optimizer updates the model parameters based on the calculated gradient. In this embodiment, the AdamW optimizer is used for parameter updates, with an initial learning rate set to 1×10, a weight decay coefficient set to 1×10, a batch size set to 4, and 150 training epochs. A linear learning rate warm-up strategy is used in the early stages of training; that is, in the first few epochs of training, the learning rate linearly increases from 0 to the preset initial learning rate to avoid the adverse effects of gradient oscillations on model convergence in the early stages of training. After multiple epochs of iterative optimization, the model parameters gradually converge to the optimal state that minimizes the loss function, thus obtaining the trained object detection model.

[0056] In this embodiment, through the aforementioned Focaler-GIoU loss function, samples located within a preset interval (i.e., samples with a certain level of difficulty but not completely unlearnable) receive a higher loss contribution than samples located outside the preset interval. This effectively avoids the problem of easy samples dominating the gradient update direction during training, guiding the model to pay more attention to tea bud samples that are difficult to detect, such as those with blurred boundaries or local occlusion. Furthermore, by combining the GIoU mechanism with the Focaler mechanism, effective gradient feedback can still be provided when the predicted bounding box and the true bounding box do not overlap, solving the problem of gradient vanishing in the case of non-overlapping traditional loss functions, thereby improving the stability and convergence speed of bounding box regression. In summary, this invention effectively balances the contribution of samples of different quality to model optimization during training, significantly improves the model's ability to detect difficult samples, enhances the accuracy of target bounding box regression, accelerates model convergence, and ultimately makes the trained target detection model more accurate, stable, and robust in locating tea buds in complex natural environments.

[0057] In some embodiments of the present invention, such as Figure 5 As shown, the bounding box regression loss function adjusts the loss contribution of different samples in the following way: S501. Determine the intersection-union ratio of the predicted location information and the actual location information.

[0058] It should be noted that the location information is represented in the form of a rectangular bounding box, including the center coordinates, width, and height of the bounding box. For each predicted bounding box output by the decoder, the Intersection over Union (IoU) ratio between it and the corresponding ground truth bounding box is calculated. The IoU ratio is calculated as the area of ​​the intersection of the predicted bounding box and the ground truth bounding box divided by the area of ​​their union, i.e., IoU = (A∩B) / (A∪B), where A is the predicted bounding box and B is the ground truth bounding box. The IoU value ranges from 0 to 1. The closer the IoU value is to 1, the higher the degree of overlap between the predicted and ground truth bounding boxes, and the more accurate the localization; the closer the IoU value is to 0, the lower the degree of overlap, and the greater the localization deviation.

[0059] S502. Determine the sample weights based on the positional relationship between the intersection-union ratio and the preset interval, so that samples located within the preset interval receive a higher loss contribution than samples located outside the preset interval.

[0060] It should be noted that a linear interval mapping method is used to reconstruct the calculated original IoU values ​​to focus on regression samples in different quality intervals. Let the lower limit of the preset interval be d and the upper limit be d. , where d and All are real numbers between 0 and 1, and d < The weights of each sample are determined according to the following segmentation method: When the original IoU value is below the lower limit d, the predicted bounding box is considered an extremely low-quality sample (such as a predicted box that deviates significantly from the true bounding box), and its reconstructed focus intersection-over-union ratio (IoU) is low. Setting it to 0 means it doesn't contribute any effective gradient in the loss calculation, which is equivalent to ignoring the sample. .

[0061] When the original IoU value is between the lower limit d and the upper limit In the meantime, the predicted bounding box is considered a medium-quality sample, and its reconstructed focus intersection-over-union ratio is calculated using linear interpolation. This allows the gradient contribution of the sample to increase smoothly as the IoU value increases.

[0062] When the original IoU value is higher than the upper limit u, the predicted bounding box is considered a high-quality sample, and its reconstructed focus intersection-over-union ratio (IoU) is high. Setting it to 1 maintains the full gradient contribution in loss calculation, i.e. .

[0063] By adjusting the values ​​of the lower bound d and the upper bound u, the loss function can be made to focus on regression samples in different quality ranges. Based on this, the reconstructed focused intersection-over-union ratio (IoU) is combined with the GIoU loss function to construct the Focaler-GIoU loss function, which is calculated as: L_Focaler-GIoU = L GIoU +IoU - Among them, L GIoU The standard generalized intersection-union (GUID) loss is used, which introduces a minimum bounding box region encompassing both the predicted and ground truth bounding boxes, providing an effective gradient signal even when they do not overlap. Finally, the calculated Focaler-GIoU loss value is used as the loss for the bounding box regression part, combined with the classification loss to form the total training loss of the model, and the model parameters are updated through backpropagation.

[0064] In this embodiment, through the aforementioned linear interval mapping mechanism, samples located within the preset interval [d, u] receive a higher loss contribution than samples located outside the preset interval. This allows the model to flexibly focus on regression samples of varying difficulty levels according to the needs of different detection tasks. For tasks such as tea bud detection, which involve densely distributed small targets and a large number of predicted bounding boxes with low overlap with the true bounding boxes, by reasonably setting the values ​​of d and u, the model can focus on medium-quality samples (i.e., samples that are somewhat difficult but not completely unlearnable). This effectively avoids the problem of extremely low-quality samples generating invalid gradients and high-quality samples dominating the loss during training, thereby guiding the model to pay more attention to tea bud samples that are difficult to detect, such as those with blurred boundaries and local occlusion. Furthermore, by combining the Focaler mechanism with the GIoU mechanism, effective gradient feedback can still be provided through the minimum bounding box region when the predicted bounding box and the true bounding box do not overlap. This solves the problem of gradient vanishing in the case of non-overlapping traditional loss functions, enhances the stability and convergence speed of bounding box regression, and achieves differentiated focusing on samples of different quality during training. This significantly improves the model's ability to detect difficult samples and the accuracy of bounding box regression, accelerates the model's convergence speed, and ultimately makes the trained target detection model more accurate and robust in locating tea buds in complex natural environments.

[0065] In some embodiments of the present invention, a lightweight convolution operation is used in the cross-scale feature aggregation process. The lightweight convolution operation rearranges the channel order by performing grouped convolution and depthwise separable convolution on the multi-scale feature maps.

[0066] It should be noted that lightweight convolution operations are used in the cross-scale feature fusion process to reduce computational complexity. The lightweight convolution operation is the GSConv (Grouped Shuffle Convolution) module. This module rearranges the channel order after performing grouped convolutions and depthwise separable convolutions on multi-scale feature maps. The GSConv lightweight convolution structure, through the collaborative design of grouped convolutions and depthwise separable convolutions, reduces computational complexity while enhancing cross-channel information interaction through channel rearrangement, thus compensating for information loss caused by channel isolation in lightweight convolutions.

[0067] In this embodiment, through the collaborative design of grouped convolution and depthwise separable convolution, GSConv achieves complementary advantages between standard convolution and depthwise separable convolution. Specifically, standard convolution can fully model the hidden relationships between channels, but it requires a large amount of computation; depthwise separable convolution decouples the convolution calculations between channels, significantly reducing the number of parameters and computational complexity, but the information of each channel is isolated from each other during the calculation process, which can easily lead to the loss of semantic features. GSConv splits the feature map into two branches: one part uses standard convolution to retain global correlation features across channels, and the other part uses depthwise separable convolution to extract local spatial features with lower computational cost. Finally, the two parts of features are fully fused through channel rearrangement, so that the output features of depthwise separable convolution are as close as possible to standard convolution in terms of information richness. Furthermore, since the channel rearrangement operation only rearranges the channel order without introducing additional parameters, it maintains extremely low computational overhead while achieving cross-channel information fusion. In the process of cross-scale feature fusion, this invention effectively maintains the information interaction capability and feature expression capability between feature channels by significantly reducing model parameters and computational complexity through lightweight convolution operations, thereby achieving synergistic optimization of detection accuracy and inference efficiency. This makes the target detection model more suitable for resource-constrained edge computing scenarios such as tea-picking robots.

[0068] In some embodiments of the present invention, the cross-scale feature aggregation adopts a cross-stage splitting structure, which divides the multi-scale feature map into two branches along the channel dimension. One branch is transformed by the lightweight convolution operation, and the other branch directly retains the original features. The transformed features and the retained original features are concatenated along the channel dimension and then output.

[0069] It should be noted that the multi-scale feature map is divided into two branches along the channel dimension for processing. The first branch uses standard convolution (such as 1×1 convolution) to perform convolution operations on the input feature map to extract cross-channel global correlation features. The second branch first uses 1×1 convolution to perform channel transformation on the input feature map, and then uses depthwise separable convolution for spatial feature extraction. Depthwise separable convolution decomposes standard convolution into two stages: channel-wise convolution and pointwise convolution. Channel-wise convolution performs convolution operations on each channel of the input feature map independently, without cross-channel information fusion. Pointwise convolution uses a 1×1 convolution kernel to linearly combine the outputs of channel-wise convolution along the channel dimension. Based on this, the outputs of the standard convolution of the first branch and the outputs of the depthwise separable convolution of the second branch are concatenated along the channel dimension, and then the channel order of the concatenated feature map is rearranged through a channel shuffle operation. Channel rearrangement reshapes the stitched feature maps into grouped forms along the channel dimension, redistributes channels across groups through transpose, and then restores them to their original shape. This channel rearrangement operation does not add any additional learnable parameters; it only achieves information interaction between the output features of different branches through data rearrangement.

[0070] In summary, the cross-stage splitting structure is the VoV-GSCSP (VoVNet-GSConv-Cross Stage Partial) feature fusion module. Assume the input multi-scale feature map is X∈R. C1×H×W Where C1 is the number of input channels, and H and W are the height and width of the feature map, respectively. First, the input feature map is compressed and split using a 1×1 convolution, dividing the multi-scale feature map into two branches Xa and Xb along the channel dimension, denoted as follows: and .in, The number of hidden channels is controlled by the expansion ratio parameter e, i.e. (C2 represents the target number of output channels). Through this splitting operation, the features of some channels enter the main transform branch, while the features of other channels are directly transferred across stages, reducing redundant calculations.

[0071] Branches The feature transformation is performed in a bottleneck unit composed of lightweight convolutions from GSConv. GSConv combines the advantages of grouped convolutions and depthwise separable convolutions. Specifically, it first uses 1×1 convolutions to perform channel transformations on the input feature map, and then uses depthwise separable convolutions to extract spatial features. Depthwise separable convolutions decompose standard convolutions into two stages: channel-wise depthwise convolutions and pointwise convolutions. Each channel first undergoes independent spatial convolution, and then they are combined across channels using 1×1 convolutions. Based on this, the outputs of the standard convolutions and the depthwise separable convolutions are concatenated along the channel dimension. Then, a channel shuffle operation is used to rearrange the channel order of the concatenated feature map, enabling information interaction between the output features of different branches. The computational form of this lightweight bottleneck unit can be expressed as: ,in (·) represents the convolutional transform structure formed by combining GSConv. The weights of the shortcut branches are represented by residual connections, which maintain the stability of feature representation and avoid the gradient vanishing problem in deep networks. This design reduces the computational complexity of convolution while maintaining the stability of feature representation through residual connections. Subsequently, the features Y after the GSConv lightweight transformation are combined with the features retained across stages. Concatenate along the channel dimension: Z = Concat( (Y); finally, channel integration is completed through 1×1 convolution to obtain the output features: .in, The overall structure adopts the OSA one-time aggregation method, aggregating features from the preceding paths only at the last layer of the module. This avoids the computational redundancy caused by layer-by-layer stacking in dense connections, while inheriting the advantages of multi-receptive-field feature representation.

[0072] In this embodiment, through the aforementioned cross-stage splitting structure, a portion of the input feature map channels undergoes sufficient transformation via lightweight convolution, while the remaining channels are directly passed across stages. This preserves the original feature information while reducing redundant convolutional computations. Furthermore, by using OSA to aggregate features from each branch only at the end of the module, the advantages of multi-receptive-field feature representation are inherited while effectively avoiding computational redundancy and memory consumption issues caused by dense connections. In summary, this invention, through the collaborative design of cross-stage splitting and lightweight convolution in the cross-scale feature fusion process, significantly reduces model parameters and computational complexity while maintaining sufficient fusion of multi-scale features and stability of gradient propagation. This achieves an effective balance between detection accuracy and inference efficiency, making the target detection model more suitable for resource-constrained edge computing scenarios such as tea-picking robots.

[0073] In all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. Additionally, when embodiments of this application require access to sensitive personal information, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0074] To better implement the tea bud detection method under complex backgrounds in the embodiments of the present invention, based on the tea bud detection method under complex backgrounds, correspondingly, as follows: Figure 7 As shown, this embodiment of the invention also provides a tea bud detection system 700 under complex backgrounds. The tea bud detection system 700 under complex backgrounds includes: Image acquisition module 701 is used to acquire images of tea buds; Feature extraction module 702 is used to extract multi-scale feature maps of the tea bud image using a multi-branch parallel attention mechanism; Feature fusion module 703 is used to perform cross-scale feature fusion processing on the multi-scale feature map to generate fused features; The encoding module 704 is used to input the deep feature map in the multi-scale feature map into the encoder for self-attention calculation, generate position codes for each spatial position according to the positional relationship of each spatial position in the deep feature map, and fuse the position codes with the feature sequence of the deep feature map to output the encoded enhanced features; The decoding module 705 is used to input the encoded enhancement features and the fused features into the decoder, filter the target query from the encoded enhancement features through a query selection mechanism, and perform cross-attention calculation on the target query, the encoded enhancement features and the fused features to generate category information and location information of tea buds.

[0075] The tea bud detection system 700 under complex backgrounds provided in the above embodiments can realize the technical solutions described in the embodiments of the tea bud detection method under complex backgrounds. The specific implementation principles of each module or unit can be found in the corresponding content of the embodiments of the tea bud detection method under complex backgrounds, which will not be repeated here.

[0076] like Figure 8 As shown, the present invention also provides an electronic device 800. The electronic device 800 includes a processor 801, a memory 802, and a display 803. Figure 8Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0077] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 802 or process data, such as the method for detecting tea buds in a complex background in this invention.

[0078] In some embodiments, processor 801 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 801 may be local or remote. In some embodiments, processor 801 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-cloud, etc., or any combination thereof.

[0079] In some embodiments, memory 802 may be an internal storage unit of electronic device 800, such as a hard disk or memory of electronic device 800. In other embodiments, memory 802 may also be an external storage device of electronic device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 800.

[0080] Furthermore, the memory 802 may include both internal storage units of the electronic device 800 and external storage devices. The memory 802 is used to store application software and various types of data installed on the electronic device 800.

[0081] In some embodiments, display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light Emitting Diode) touchscreen. Display 803 is used to display information from electronic device 800 and to display a visual user interface. Components 801 and 803 of electronic device 800 communicate with each other via a system bus.

[0082] In one embodiment, when the processor 801 executes the tea bud detection program in the memory 802 under a complex background, the following steps can be implemented: The acquired tea bud images are input into a feature extraction network, and a multi-branch parallel attention mechanism is used to extract multi-scale feature maps from the tea bud images. The multi-scale feature map is input into a feature fusion network for cross-scale feature fusion processing to generate fused features; The deep feature map in the multi-scale feature map is input into the encoder for self-attention calculation. The positional code of each spatial position is generated according to the positional relationship of each spatial position in the deep feature map. The positional code is fused with the feature sequence of the deep feature map to output the encoded enhanced feature. The encoded enhancement features and the fused features are input into the decoder. The target query is filtered from the encoded enhancement features through a query selection mechanism. The target query is then subjected to cross-attention calculation with the encoded enhancement features and the fused features to generate category information and location information of tea buds.

[0083] It should be understood that when the processor 801 executes the tea bud detection program in the complex background in the memory 802, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0084] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 800 mentioned. Electronic device 800 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 800 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0085] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0086] The above provides a detailed description of the tea bud detection method, system, and electronic equipment under complex backgrounds provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting tea buds under complex backgrounds, characterized in that, include: The acquired tea bud images are input into a feature extraction network, and a multi-branch parallel attention mechanism is used to extract multi-scale feature maps from the tea bud images. The multi-scale feature map is input into a feature fusion network for cross-scale feature fusion processing to generate fused features; The deep feature map in the multi-scale feature map is input into the encoder for self-attention calculation. The positional code of each spatial position is generated according to the positional relationship of each spatial position in the deep feature map. The positional code is fused with the feature sequence of the deep feature map to output the encoded enhanced feature. The encoded enhancement features and the fused features are input into the decoder. The target query is filtered from the encoded enhancement features through a query selection mechanism. The target query is then subjected to cross-attention calculation with the encoded enhancement features and the fused features to generate category information and location information of tea buds.

2. The method according to claim 1, characterized in that, The multi-branch parallel attention mechanism includes local branches, and the convolution operation parameters of the local branches are dynamically adjusted during the feature extraction process.

3. The method according to claim 2, characterized in that, The dynamic adjustment of the convolution operation parameters of the local branches includes: Global pooling is performed on the multi-scale feature map of the input local branch to extract global descriptive information of the channel dimension; The global description information is mapped to the weight coefficients corresponding to each basic convolutional kernel; The dynamic convolution kernel is obtained by weighting and summing each basic convolution kernel according to the weight coefficients. The dynamic convolution kernel is used to perform convolution operations on the multi-scale feature map of the local branch.

4. The method according to claim 1, characterized in that, The step of generating position codes for each spatial location based on the positional relationships of each spatial location in the deep feature map includes: Each spatial location in the deep feature map is taken as a graph node, and the local neighborhood of each graph node is determined according to the spatial position relationship. The local neighborhood is a set of other graph nodes that are spatially adjacent to the graph node. A sparse adjacency graph is constructed based on the feature similarity between a graph node and other graph nodes in its local neighborhood. At least some feature information of the Laplacian matrix is ​​determined based on the sparse adjacency graph; the Laplacian matrix is ​​a matrix characterizing the adjacency relationships of graph nodes. Based on the aforementioned feature information, a location code for each spatial position is constructed.

5. The method according to claim 1, characterized in that, The feature extraction network, the feature fusion network, the encoder, and the decoder constitute the target detection model, which is trained in the following manner: The category information and location information generated by the decoder are optimized using a bounding box regression loss function. The bounding box regression loss function adjusts the loss contribution of different samples according to the degree of overlap between the predicted location information and the real location information. The parameters of the object detection model are updated based on the calculation results of the bounding box regression loss function.

6. The method according to claim 5, characterized in that, The bounding box regression loss function adjusts the loss contribution of different samples in the following way: Determine the intersection-union ratio (IoU) between the predicted location information and the actual location information; The sample weights are determined based on the positional relationship between the intersection-union ratio and the preset interval, so that samples located within the preset interval receive a higher loss contribution than samples located outside the preset interval.

7. The method according to claim 1, characterized in that, The cross-scale feature aggregation process employs lightweight convolution operations, which rearrange the channel order by performing grouped convolutions and depthwise separable convolutions on the multi-scale feature maps.

8. The method according to claim 7, characterized in that, The cross-scale feature aggregation adopts a cross-stage splitting structure, which divides the multi-scale feature map into two branches along the channel dimension. One branch is transformed by the lightweight convolution operation, and the other branch directly retains the original features. The transformed features and the retained original features are concatenated along the channel dimension and then output.

9. A tea bud detection system under complex backgrounds, characterized in that, include: The image acquisition module is used to acquire images of tea buds; The feature extraction module is used to extract multi-scale feature maps of the tea bud image using a multi-branch parallel attention mechanism; The feature fusion module is used to perform cross-scale feature fusion processing on the multi-scale feature map to generate fused features; The encoding module is used to input the deep feature map in the multi-scale feature map into the encoder for self-attention calculation, generate position codes for each spatial position according to the positional relationship of each spatial position in the deep feature map, and fuse the position codes with the feature sequence of the deep feature map to output the encoded enhanced features; The decoding module is used to input the encoded enhancement features and the fused features into the decoder, filter the target query from the encoded enhancement features through a query selection mechanism, and perform cross-attention calculation on the target query, the encoded enhancement features and the fused features to generate category information and location information of tea buds.

10. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the method for detecting tea buds in a complex background as described in any one of claims 1 to 8.

Images