Weed target identification method under complex condition based on deep learning

Abstract

The invention discloses a weed target identification method under a complex condition based on deep learning, and the method comprises the steps: firstly completing the semi-automatic labeling of a data set through LabelMe software, and then carrying out the manual labeling through CVAT, namely ComputerVisionAnnotation Tool; then, a Transform architecture of a CoatNet detection network is improved, a Gaussian noise detection layer is introduced into the DCMHA, and a Dropout regularization technology is introduced, so that the calculation efficiency is improved; further, a Coatnet network structure is improved, and network layers are deepened, so that the feature extraction capability of the network on a complex target is improved; on the basis, designing an asymmetric smooth focus loss function, namely an Asymetric Smoothed Focus Loss function, converting the cross entropy function into a focus loss function, namely a Focus Loss function, and introducing a label smoothing strategy and asymmetry; and finally, inputting an original video image into the deep network obtained by training to obtain a final weed recognition result. According to the method, the expression ability of detail information is enhanced, and the method has higher robustness for a weed target identification task under complex conditions.

Description

Technical Field

[0001] The present invention belongs to the field of target detection and recognition, and in particular, relates to a weed target recognition method under complex conditions based on deep learning. Background Art

[0002] Target detection and recognition is an important part of agricultural weed control. Currently, there are two main methods of weed control: chemical weed control and mechanical weed control. The use of pesticides and herbicides will lead to soil and water pollution, affect the health of the global ecosystem, destroy biodiversity, and may cause the spread of resistant weeds. Long-term use will also have an indirect impact on global climate change, affecting ecological balance and human health. Mechanical weed control can effectively distinguish between crops and weeds through target recognition detection methods, thereby improving weed control efficiency, reducing resource waste, and reducing negative impacts on the environment. In this context, mechanical weed control methods are becoming increasingly important. Therefore, in order to improve the efficiency, robustness and accuracy of weed target recognition tasks, agricultural weed control is now moving towards full automation and full intelligence, thereby reducing resource waste and protecting the ecological environment.

[0003] As global warming, biodiversity loss, pollution, overexploitation of resources and ecosystem degradation become increasingly serious, chemical weed control methods have become increasingly unfeasible, and the efficiency of weed detection and recognition tasks will directly affect the efficiency of mechanical weed control, thereby indirectly affecting the growth of crops in the entire area. In recent years, as deep learning technology has been widely used in the field of target detection and recognition, the efficiency and accuracy of weed recognition and detection tasks have been greatly improved. Using these technologies, the system can accurately detect, identify and extract weed information from images or videos of different soil conditions such as drought, salinity, and fertility. This automated process can significantly improve the efficiency of mechanical weed control and reduce damage to the environment. However, in actual applications, due to high exposure, blurred shooting and unclear subjects caused by lighting, weather, shooting angle, shooting distance, bumpy roads and other problems in natural scenes, the traditional weed detection and recognition methods based on deep learning still have many unsatisfactory aspects.

[0004] As for the attention mechanism, the multi-head attention mechanism proposed by Vaswani et al. (Vaswani A, Shardlow T, et al. Attention is All You Need [C] / / Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 5998-6008.) performs well in capturing the relationship between different feature subspaces. This mechanism can more effectively extract the subtle differences between weeds and crops by calculating multiple attention heads in parallel, and enhance the model's ability to identify small targets in complex backgrounds. However, the computational overhead is large, especially when processing high-resolution images, and its efficiency optimization still needs to be paid attention to. As for the network structure, the dilated convolutional network proposed by Yu et al. (Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions [J]. arXiv preprint arXiv: 1511.07122, 2015.) provides richer context information for weed detection by increasing the receptive field without increasing the number of parameters. This structure can better cope with the challenges brought by complex background and miniaturization of targets, and improve the detection accuracy of weeds. However, this method still needs to be combined with other technologies to improve the overall performance when dealing with extremely small targets. Regarding the design of the loss function, the Focal Loss proposed by Lin et al. (LinT, Goyal P, Girshick R, et al. Focal Loss for Dense Object Detection[C] / / Proceedings of the IEEE International Conference on Computer Vision.2017:2980-2988.) effectively solves the problem of category imbalance, especially for small target detection. By adjusting the focus of the loss function, the model pays more attention to samples that are difficult to classify, significantly improving the ability to identify weeds in complex scenes. However, debugging and selecting appropriate hyperparameters remains a challenge and needs to be optimized according to the specific data set.

[0005] In summary, although weed detection and recognition technology based on deep learning has made some progress, it still needs to be improved to better meet the needs of high efficiency and high accuracy in agricultural weeding due to changes in actual scene lighting, complex weed backgrounds, weed obstruction by foreign objects, and differences in camera shooting angles of different models. This will bring more efficient and intelligent operation and management to the field of agricultural weeding, and promote the development and improvement of the entire agricultural machinery weeding. Summary of the invention

[0006] The present invention aims to overcome the deficiencies in the above-mentioned prior art and proposes a weed target recognition method under complex conditions based on deep learning, so as to improve the efficiency and accuracy of weed recognition under high-demand mechanical weeding.

[0007] In order to achieve the above invention, the present invention provides a method for identifying weed targets under complex conditions based on deep learning, comprising the following steps:

[0008] (1) Collect image samples of weeds;

[0009] (2) Preprocess and label the image;

[0010] (3) Improve the Transformer architecture of the fifth stage in the CoatNet detection network;

[0011] (4) Improve the CoatNet network structure;

[0012] (5) Design an asymmetric smoothed focal loss function, namely Asymmetric Smoothed Focal Loss: Replace the cross entropy function with the focal loss, namely the Focal Loss function, and introduce label smoothing strategy and asymmetry;

[0013] (6) Input the original video image into the trained deep network to obtain the final weed recognition result.

[0014] The collecting of weed image samples refers to collecting image samples in different environments, such as drought, saline-alkali, fertile and other different soil conditions, as well as different inclination angles and distances, so that the image samples contain pictures of different growth conditions, light intensity, complex background, foreign body occlusion, multi-angle rotation and inclination and different scales; the preprocessing and labeling of the image refers to using a Gaussian filter to reduce the noise in the image to ensure that the weed features are more obvious, using histogram equalization technology to improve the contrast between the weeds and the background, making the weeds easier to identify, and then using LabelMe software for semi-automatic labeling and then using CVAT, i.e. Computer Vision Annotation Tool, for manual labeling.

[0015] Furthermore, the improved Transformer architecture of the fifth stage in the CoatNet detection network refers to replacing the MHA module in the Transformer architecture of the fifth stage of the CoatNet detection network with an improved DCMHA, and introducing two activation functions in DCFormer; the improved DCMHA refers to introducing a Gaussian noise detection layer in DCMHA, and introducing Dropout regularization technology in Softmax; the introduction of the Gaussian noise detection layer refers to adding Gaussian noise with a standard deviation σ=0.5 to the Query, Key and Value input vectors of each attention mechanism, performing noise preprocessing on the data, so that the model can better adapt to random disturbances and noise in the data, thereby enhancing its robustness in different environments, and And the model is forced to learn more general features, helping to prevent the model from overfitting the training data; the introduction of the Dropout regularization technology refers to the introduction of Dropout in the Softmax function to form a weight matrix of the attention mechanism applied to DropMax. During the training process, Dropout randomly sets some weights in the weight matrix of the attention mechanism to zero to reduce the model's dependence on specific features, thereby improving the generalization ability of the model, and also makes the model pay more attention to important features. It can dynamically adjust the focus of attention according to different input data, reduce the model's calculation burden, and improve calculation efficiency; the introduction of two activation functions in DCFormer refers to the introduction of two nonlinear activation functions Noisy ReLU and GELU, i.e. Gaussian Error Linear Unit, in DCFormer's FFN, i.e. Feedforward Neural Network. Noisy ReLU introduces noise on the basis of ReLU. By using Noisy ReLU at the low level, the randomness and diversity of the model are increased, and the sensitivity and expression ability of the model to the input data are improved. ReLU is used at the high level to keep the model fast and simple, and provide sufficient nonlinear capabilities to process complex data features.

[0016] Furthermore, the improved CoatNet network structure refers to adding a third convolution stage after the second convolution stage in CoatNet, and the convolution used in the newly added third convolution stage is the improved MBConv-fused convolution, by adding a DEA module, namely Detail Enhancement Attention, and a BSA module, namely Background SuppressionAttention, background suppression attention module, to increase the receptive field and improve the capture of local details; the DEA module refers to first performing maximum pooling and average pooling operations on the input feature map, splicing the pooling results and generating spatial attention weights through convolution, thereby strengthening the edge and detail information in the feature map; the BSA module refers to performing global average pooling on the input feature map, generating channel attention weights through a two-layer fully connected network, suppressing background information and adjusting the importance of each channel; the deepening of the network level refers to setting the number of convolution blocks to 6 in the newly added third convolution stage, increasing the number of Transformer blocks in the fourth stage combining convolution and Transformer to 56, and adjusting the number of channels of each attention head in this stage to 54, so as to achieve a balance between size and number of channels and improve the overall computational efficiency of the model; in addition, residual connections are added between the first convolution stage and the third convolution stage, and between the second convolution stage and the fifth stage of the network, so as to enhance information transmission and gradient flow, avoid gradient vanishing and reduce computational cost, and ensure that while increasing the complexity of the model, the computational efficiency and gradient stability are maintained.

[0017] Furthermore, the design of an asymmetric smooth focal loss function refers to replacing the traditional cross entropy loss function with an improved focal loss function, and introducing a label smoothing strategy and asymmetry; the improved focal loss function refers to introducing weight factors and focusing parameters in the focal loss function, and by adjusting the values ​​of the weight factors and focusing parameters, the model can better cope with the problems of imbalanced positive and negative samples and difficult-to-classify samples; the introduction of a label smoothing strategy refers to introducing soft one-hot to add noise, replacing the traditional one-hot encoded label vector with an updated label vector, and reducing the weight of the true label category, thereby effectively suppressing overfitting; the introduction of asymmetry refers to removing the focal parameters of the loss part related to rare classes, retaining the suppression of background elements, assigning different losses to each category, and selectively using focal parameters for enhancement or suppression, thereby overcoming the harmful suppression of rare classes and the enhancement of background classes, thereby improving the signal-to-noise ratio, and further optimizing the performance of the model in class imbalance scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1is a flow chart of an embodiment of the present invention;

[0019] Figure 2 This is a diagram of the improved DCFormer architecture according to an embodiment of the present invention;

[0020] Figure 3 This is a structural diagram of the improved DCMHA according to an embodiment of the present invention;

[0021] Figure 4 A network structure diagram of an embodiment of the present invention;

[0022] Figure 5 Schematic diagram of the background suppression attention module (BSA) of an embodiment of the present invention;

[0023] Figure 6 A structural diagram of a detail enhanced attention module (DEA) according to an embodiment of the present invention;

[0024] Figure 7 This is a structural diagram of an asymmetric smoothed focal loss function according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The technical route of the present invention is further described in detail below through specific embodiments and drawings.

[0026] An embodiment of the method for identifying weed targets under complex conditions based on deep learning of the present invention comprises the following steps:

[0027] 1. Collect image samples of weeds

[0028] 10,000 images were collected in different environments, such as drought, saline-alkali, fertile and other soil conditions, as well as image samples from multiple angles (such as overhead, side, etc.) and different distances (close up, far away), so that the image samples contain pictures of different growth conditions, light intensities, complex backgrounds, foreign object occlusions, and multi-angle rotation and tilt. Then they were divided into training set, test set and validation set in an 8:1:1 manner. The training set is used for model training, the validation set is used to adjust parameters and verify the model effect, and the test set is used to finally evaluate the model performance.

[0029] 2. Preprocess and label the image

[0030] By applying a Gaussian filter to reduce the noise in the image, we ensure that the weed features are more obvious. Using histogram equalization technology, we increase the contrast between the weeds and the background, making the weeds easier to identify. Then, we use LabelMe software for semi-automatic annotation and CVAT, the Computer Vision Annotation Tool, for manual annotation.

[0031] 3. Improve the Transformer architecture of the fifth stage in the CoatNet detection network.

[0032] In order to overcome the problems that each head in the original MHA works independently, which may lead to the low-rank bottleneck of the attention score matrix, and multiple heads learn similar information, resulting in head redundancy and waste of parameters and computing resources, this patent proposes to add an improved DCMHA to replace the MHA in the Transformer architecture to form a corresponding improved DCFormer to improve feature extraction. Specifically, the original MHA in the Transformer architecture is replaced with DCMHA that introduces a dynamic combination mechanism including two Composes. The core of DCMHA is a Compose function that can dynamically transform the attention score according to the input, thereby improving the adaptability and expressiveness of the model, and by dynamically combining attention heads, it can also greatly reduce the head redundancy problem and improve the utilization efficiency of parameters. Based on this, a Gaussian noise detection layer with a standard deviation of σ = 0.5 is introduced on the input vector of each attention mechanism to preprocess the data, which can make the model better adapt to random disturbances and noise in the data, thereby enhancing its robustness in different environments, and forcing the model to learn more general features, helping to prevent the model from overfitting the training data; Dropout, a regularization technique, is introduced in the Softmax function to form the weight matrix of DropMax applied to the attention mechanism. During the training process, before calculating Softmax, a Dropout layer is defined. This layer randomly resets some weights to zero during the training process, obtains the attention score through matrix batch multiplication, and sends it to the first Compose operation to dynamically combine the attention scores of different heads. The Dropout layer is applied to the calculated attention scores, and a part of the attention scores are randomly discarded. The result after Dropout is normalized by the Softmax function and sent to the Compose operation again to dynamically combine the attention weights of different heads. Finally, the attention scores and values ​​of different heads, i.e., Value, sequences are merged and projected to reduce the model's dependence on specific features, thereby improving the generalization ability of the model. In the DCFormer framework, two nonlinear activation functions, Noisy ReLU and GELU, are introduced in FFN. Noisy ReLU is used in the low layer to increase the randomness and diversity of the model, improve the sensitivity and expressiveness of the model to the input data, and ReLU is used in the high layer to keep the model fast and simple, providing sufficient nonlinear capabilities to process complex data features.

[0033] The core idea of ​​DropMax is to introduce Dropout binary mask variables in the Softmax function. These variables are used to retain the probability ρ kBased on, adaptive learning is performed through variational inference. DropMax is calculated according to formula (1):

[0034]

[0035] Among them, Ber represents Bernoulli distribution, ρ k is the retention probability of category k, and ε is a sufficiently small number (e.g. 10 -20 ), to prevent the denominator from disappearing. z is the dropout mask, x is the input feature, y is the target category, ψ and θ are the model parameters.

[0036] The core of DCMHA, namely Dynamically Composable Multi-HeadAttention, is the Compose function, which enables DCMHA to flexibly adjust the contribution of different heads according to the different characteristics of the input data, thereby improving the adaptability and expressiveness of the model. In DCMHA, the Compose function is calculated according to formula (2):

[0037]

[0038] Among them: A :ij Denotes the query vector Q i and the key vector K j The original attention vector between b is an independent i or K j The weight matrix can be seen as a basic component on which the dynamic component is superimposed. From Q i The calculated dynamic weights are used to project to lower dimensions and then back to the original dimensions. It is a gating mechanism, and it is also from Q i Computed for element-wise multiplication. and It is symmetrical from K j Computed dynamic weights and gating.

[0039] Adding Gaussian noise to the input vector of the attention mechanism allows the model to better adapt to random perturbations and noise in the data, thereby enhancing its robustness in different environments. In DCMHA, the Gaussian noise function is calculated according to formula (3):

[0040]

[0041] Where: x is a random variable, μ is the mean, σ 2 is the variance, and e is the base of the natural logarithm.

[0042] 4. Improve CoatNet network structure

[0043] In order to overcome the problems of overfitting and gradient disappearance caused by the large number of hyperparameters involved in CoatNet due to its hybrid architecture, this patent proposes to improve the network structure of CoatNet and deepen the network level. Specifically, the improved network structure refers to adding a third convolution stage after the second convolution stage in CoatNet. The convolution used in the newly added third convolution stage is the improved MBConv-fused convolution. By adding DEA modules and BSA modules to the traditional MBConv-fused, the receptive field is increased and the capture of local details is improved. The DEA module refers to first performing maximum pooling and average pooling operations on the input feature map, splicing the pooling results, and generating spatial attention weights through convolution, thereby strengthening the edge and detail information in the feature map; the BSA module refers to performing global average pooling on the input feature map, generating channel attention weights through a two-layer fully connected network, suppressing background information and adjusting the importance of each channel. At the same time, deepening the network layer means setting the number of convolution blocks to 6 in the newly added third convolution stage, increasing the number of Transformer blocks in the fourth stage combining convolution and Transformer to 56, and adjusting the number of channels of each attention head in this stage to 54, so as to achieve a balance between size and number of channels and improve the overall computational efficiency of the model; in addition, residual connections are added between the first convolution stage and the third convolution stage, and between the second convolution stage and the fifth stage of the network (attention stage) to enhance information transfer and gradient flow, avoid gradient vanishing and reduce computational costs, and ensure that while increasing the complexity of the model, the computational efficiency and gradient stability are maintained.

[0044] A third convolution stage is added after the second convolution stage in CoatNet. The newly added third convolution stage uses the improved MBConv-fused convolution to increase the receptive field and improve the capture of local details. Specifically, Figure 4As shown in the figure, in the newly added third convolution stage, the number of convolution kernels is 8 and the number of channels is 576. The improved MBConv-fused convolution combines standard convolution and depth-separable convolution to make feature extraction more efficient and capture multi-scale feature information. In the improved MBConv-fused module, the DEA module and BSA module are also added to further enhance the expressiveness of features. The BSA module processes the input features before the convolution operation to reduce the interference of irrelevant background features, so that the model focuses more on the foreground information, thereby improving the accuracy of feature extraction. The feature map suppressed by the BSA module is then subjected to 3x3 convolution to extract features, and the number of channels is adjusted through 1x1 convolution to ensure the consistency of the feature map in subsequent processing. Next, the convolutional feature map is divided into two branches: one branch directly retains the convolutional features, and the other branch enters the DEA module to enhance edge and detail features; the DEA module further improves the model's ability to capture subtle features by strengthening the detail information in the features; the feature map output by DEA adjusts the number of channels through 1x1 convolution so that it has the same number of channels as the directly retained feature map to ensure the consistency of the fusion process; finally, the features of the two branches are fused together through element-by-element addition to obtain an enhanced feature map, which not only retains the original feature information but also enhances the expression of edges and details. This improved design effectively suppresses background noise through the BSA module, and enhances the foreground detail features through the DEA module, which significantly improves the model's feature extraction ability under complex backgrounds. Combined with the channel adjustment of 1x1 convolution, it ensures the effective fusion of features between branches, thereby improving the overall expressiveness and robustness of the model, especially when dealing with complex weed recognition tasks, it can achieve higher accuracy and stability.

[0045] Among them, the newly added DEA modules, such as Figure 6 As shown in the figure, the DEA module processes the feature map after the convolution operation. The core goal of the DEA module is to enhance the detail information in the features, especially the edge and texture features. Specifically, the DEA module extracts spatial information through average pooling and maximum pooling operations respectively, and splices this information together to generate attention weights in the spatial dimension. Then, the weights generate a refined attention distribution through convolution operations and Sigmoid activation functions. Finally, the generated attention weights are multiplied element-by-element with the input feature map to form an enhanced feature map, and the details and edge information are further enhanced. The DEA module significantly improves the model's ability to capture features of small targets and complex scenes by enhancing the details of important areas, enabling the model to more accurately identify subtle differences between different targets.

[0046] X out=σ(Conv(Concat(MaxPool(X in ),AvgPool(X in ))))⊙X in (4)

[0047] X in Represents the input feature map, the size is B×C×H×W, MaxPool(X in ) and AvgPool(X in ) are the maximum pooling and average pooling operations on the input feature map, respectively, with an output size of B×1×H×W. Concat means concatenating the two pooled feature maps in the channel dimension to generate a feature map of size B×2×H×W. Conv is a 3×3 convolution operation, which is used to further process the concatenated feature map, with an output size of B×1×H×W, and extract more detailed spatial information. σ is the Sigmoid activation function, which is used to map the convolved feature map to between 0 and 1 and generate spatial attention weights. ⊙ represents the element-by-element multiplication operation, which multiplies the generated attention weights with the input feature map X in Multiply them to get the output feature map X out . The BSA module is applied before the convolution operation to suppress irrelevant background information in the feature map and ensure that the model can focus on the foreground features. The BSA module globally aggregates the spatial dimensions of the input feature map through a global average pooling operation to generate global context features. Then, background suppression weights are generated through a fully connected layer and activated through a Sigmoid function so that the weight values ​​fluctuate between 0 and 1. The generated background suppression weights are applied to the input feature map through element-by-element multiplication, so that irrelevant background features are effectively suppressed and foreground features are enhanced. After being processed by the BSA module, the feature map enters the convolution layer for further feature extraction, thereby reducing the interference of background noise on feature extraction and enhancing the model's attention to the foreground area.

[0048] X out =σ(W2ReLU(W1GlobalAvgPool(X in )))⊙X in (5)

[0049] X in Represents the input feature map, size is B×C×H×W, GlobalAvgPool(X in) is a global average pooling operation with an output size of B×C×1×1, which represents modeling of the global background information of the input feature map. W1 is the weight of the first fully connected layer, which compresses the global background information from C channels to C / r, where r is the reduction factor, usually 4 or 8. W2 is the weight of the second fully connected layer, which restores the global background information from C / r to C, ensuring that the number of channels of the output feature is consistent with the input. σ is the Sigmoid activation function, which is used to generate background suppression weights and control the weights between 0 and 1. ⊙ represents an element-by-element multiplication operation, which multiplies the generated background suppression weights with the input feature map X in Multiply them to get the output feature map X after background noise suppression out .

[0050] Deepening the network layer includes adjusting the configuration of the original convolution blocks and attention heads to include more convolution blocks and more attention heads, while reducing the number of channels of each attention head to achieve a balance between size and number of channels. Specifically, in the third convolution stage, the number of convolution blocks is set to 6. Compared with the number of convolution blocks in the first two stages, the small increase in the third convolution stage enables the convolution layer to better adapt to the introduced BSA module and DEA module, while ensuring the balance between feature extraction and computational complexity of the network. By setting the number of convolution blocks to 6, the model will not increase the excessive computational burden due to too many convolution blocks, and can ensure that the network has enough capacity at this stage to cope with complex feature extraction requirements and enhance the feature representation capabilities of the middle and deep layers. In the fourth stage, which is the stage where convolution and Transformer are combined, the number of Transformer blocks is increased to 56, and the number of channels of each attention head is adjusted to 54. Through this design, the ability to capture global features is improved. Compared with the previous stage, more attention blocks and a smaller number of channels per attention head achieve a balance between computational efficiency and feature extraction, enabling the model to handle complex dependencies at a deep level.

[0051] In order to further improve the network's feature transfer capability and avoid the gradient vanishing problem, the CoatNet architecture adds residual connections between the first and third convolution stages, and between the second convolution stage and the fifth stage (attention stage) of the network. Specifically, Figure 4As shown in Figure 3. By introducing residual connections between these different stages, the model can achieve more efficient information transfer and ensure that features flow more smoothly in the deep network. The residual connection between the first convolution stage and the third convolution stage allows the features extracted from the shallow layer to be directly passed to the deeper convolution layer. This cross-layer connection method allows the model to retain more shallow detail information and enhance the model's ability to capture local features. At the same time, the introduction of residual connections also effectively alleviates the gradient vanishing problem that may occur in the model in a multi-layer network, ensuring that the gradient can be smoothly passed to the shallower layer in the back propagation. In this way, the features of the first convolution stage can be reused in the third convolution stage, allowing the model to retain more detail information when extracting features layer by layer. Similarly, the residual connection between the second convolution stage and the fifth stage of the network ensures that the features extracted from the middle layer can be directly passed to the deep network. This long-distance connection method allows the attention module in the fifth stage to refer to the feature representation of the middle layer when processing global features, enhancing the model's perception of features of different scales and complexity. In addition, such residual connections can further prevent the gradient from disappearing in deep networks, ensuring the stability and effectiveness of deep models in feature learning.

[0052] 5. Design an asymmetric smoothed focal loss function, i.e. Asymmetric Smoothed Focal Loss

[0053] Considering that the original cross entropy function cannot handle the problem of sample imbalance well, this patent proposes to use an asymmetric smooth focal loss function. Specifically, the traditional cross entropy loss function is replaced by a focal loss function, namely the FocalLoss function, and weight factors and focal parameters are introduced into the focal loss function. By adjusting the values ​​of the weight factors and focal parameters, the model can better cope with the problems of positive and negative sample imbalance and difficult-to-classify samples. On this basis, soft one-hot is introduced to add noise, and the traditional one-hot encoded label vector is replaced with the updated label vector, thereby reducing the weight of the true label category and effectively suppressing overfitting. On this basis, the focal parameters of the loss part related to rare classes are removed, and the suppression of background elements is retained. Different losses are assigned to each category, and the focal parameters are selectively used for enhancement or suppression, thereby overcoming the harmful suppression of rare classes and the enhancement of background classes, thereby improving the signal-to-noise ratio and further optimizing the performance of the model in category imbalance scenarios.

[0054] The traditional cross entropy loss function is replaced by the focal loss function, namely the Focal Loss function, and the weight factor and focus parameter are introduced into the focal loss function, such as Figure 7As shown in the figure, the weight factor and focus parameter are introduced, and the value of the weight factor is set to 0.5 and the value of the focus parameter is set to 3, so that the model pays more attention to the problem of sample imbalance and difficult-to-classify samples, which has a certain effect on solving the problem of positive and negative sample imbalance and difficult-to-classify samples. The specific improved focus function is calculated according to formula (6):

[0055] Loss i =-α i (1-p i ) γ log(p i ) (6)

[0056] Among them, α i is a weight factor, which represents the weight of the i-th class and is used to solve the problem of sample imbalance. When the number of positive samples is small, we can reduce α i The value of p is used to reduce the weight of negative samples; i Represents the probability of being predicted as the i-th category; γ is a focusing parameter used to make the model focus on samples that are difficult to classify. The larger γ is, the more attention the model pays to low-confidence samples.

[0057] Introduce soft one-hot to add noise, replace the traditional one-hot encoded label vector with the updated label vector, and reduce the weight of the true label category, such as Figure 7 As shown in the figure, in multi-classification tasks, the neural network will output the confidence score of each category corresponding to the current data, normalize it through softmax, and finally get the probability that the current data belongs to each category. The neural network will force itself to learn in the direction with the largest difference between the correct label and the wrong label. When the training data is not enough to represent all sample features, it will cause the network to overfit. Therefore, by adding noise through soft one-hot and reducing the weight of the real sample label category when calculating the loss function, the label error problem in the model can be alleviated, the relationship between the categories in the training data can be enhanced, the network prediction is more accurate, and the impact of overfitting is finally suppressed. The principle is to use the updated label vector Replace the traditional one-hot encoding label vector and calculate it according to formula (7):

[0058]

[0059] Where K is the number of label categories and α is a hyperparameter that determines the amount of smoothing.

[0060] The use of label smoothing methods alleviates the label error problem in the model, enhances the relationship between categories in the training data, and makes the network prediction more accurate.

[0061] The label smoothing strategy is added to the focal loss function, which is calculated according to formula (8):

[0062]

[0063] Although the focal loss achieves the suppression of background classes, the focal parameters apply to all classes, so the loss of rare classes will also be suppressed. Therefore, asymmetry is introduced. On this basis, the focal parameters of the loss part related to rare classes are removed, and the suppression of background elements is retained. Different losses are assigned to each class, and the focal parameters are selectively used for enhancement or suppression, such as Figure 7 As shown in Figure 2, for the rare class r, the focus parameter of the loss part is removed. This means that when calculating the loss of the rare class, (1-p t,r ) γ This focus parameter. The purpose of this is to avoid excessive suppression of the loss of rare classes. For non-target categories c≠r, the focus parameter is retained for loss calculation. t,c ) γ This focus parameter plays a role in reducing the loss weight of the model on samples that are easy to classify. The parameter δ = 0.6 is set to adjust the weights of the target category (rare class) and the non-target category in the loss function. When the δ value is large, the loss of the target category (rare class) accounts for a relatively large proportion of the total loss, that is, the focus on the rare class is relatively enhanced. The obtained focus loss function is modified asymmetrically. The modified asymmetric smooth focus loss removes the focus parameter of the loss part related to the rare class r, while retaining the suppression of background elements. It is calculated specifically according to formula (9):

[0064]

[0065] Among them, δ is a parameter that controls the weight of positive and negative samples. δ is used to adjust the weight of target and non-target categories in the loss function. N is the total number of samples. Here, N is used to normalize the loss function so that the loss contribution of each sample is average. It means to sum all samples belonging to the rare category (target category) r. is the label value of the target category r, which has been adjusted by the label smoothing strategy. For label smoothing, It will usually be slightly less than 1 (unlike 1 in the original one-hot encoding). Specifically, The values ​​are: p t,r It indicates the probability that the model predicts that the sample belongs to category r. log(p t,c ) is the logarithm of this probability, which represents the logarithmic loss of category r. It means to sum all non-target categories (i.e. c≠r). is the label value of the non-target category c, which has also been adjusted by the label smoothing strategy. For label smoothing, It is usually slightly larger than 0 (unlike 0 in the original one-hot encoding). Specifically, The values ​​are: (1-p t,c ) γ is the modulation factor in the focal loss function, which is used to reduce the loss weight of the model on easy-to-classify samples. γ is the modulation factor. If p t,c The closer it is to 1, the greater the value of 1-p t,c The smaller it is, the smaller (1-p t,c ) γ The smaller it is, the less loss this part will suffer. t,c It indicates the probability that the model predicts that the sample belongs to category c. log(p t,c ) is the logarithm of this probability, which is used to calculate the logarithmic loss of category c.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] (1) The present invention replaces the MHA module in the fifth-stage Transformer architecture of the CoatNet detection network to improve DCMHA, which better makes up for the low-rank bottleneck and head redundancy of the MHA in the original Transformer architecture. It can dynamically transform the attention score and weight matrix according to the input, thereby allowing the model to flexibly adjust the allocation of attention according to the different characteristics of the data, and through dynamic combination, improves the utilization efficiency and performance of parameters.

[0068] (2) Based on the DCFormer structure, the present invention introduces a Gaussian noise detection layer on the input vector of each attention mechanism to preprocess the data so that the model can better adapt to random disturbances and noise in the data, thereby enhancing its robustness in different environments and forcing the model to learn more general features to help prevent the model from overfitting the training data; Dropout, a regularization technique, is introduced in the softmax function and applied to the weight matrix of the attention mechanism. During the training process, Dropout randomly sets certain weights in the weight matrix of the attention mechanism to zero to reduce the model's dependence on specific features, thereby improving the model's generalization ability and making the model pay more attention to important features. The focus of attention can be dynamically adjusted according to different input data, thereby reducing the model's computational burden and improving computational efficiency.

[0069] (3) The present invention introduces two nonlinear activation functions, Noisy ReLU and GELU, into the FFN of DCFormer. Noisy ReLU is used at the low level to increase the randomness and diversity of the model, thereby improving the sensitivity and expressiveness of the model to the input data. ReLU is used at the high level to maintain the speed and simplicity of the model, thereby providing sufficient nonlinear capabilities to process complex data features.

[0070] (4) The present invention adds a third convolution stage after the second convolution stage in CoatNet. The convolution used in the newly added third convolution stage is the improved MBConv-fused convolution. By adding DEA module and BSA module to the traditional MBConv-fused, the receptive field is enlarged and the capture of local details is improved.

[0071] (5) The present invention adjusts the configuration of the original convolution blocks and attention heads to more convolution blocks and more attention heads, while reducing the number of channels of each attention head and improving the overall computational efficiency of the model; residual connections are added between the first convolution stage and the third convolution stage, and between the second convolution stage and the fifth convolution stage, so as to enhance information transmission and gradient flow, avoid gradient vanishing and reduce computational cost, and ensure that while increasing the complexity of the model, the computational efficiency and gradient stability are maintained.

[0072] (6) The present invention replaces the traditional cross entropy loss function with the focal loss function, introduces weight factors and focusing parameters into the focal loss function, and by adjusting the values ​​of the weight factors and focusing parameters, the model can better cope with the problems of imbalanced positive and negative samples and difficult-to-classify samples; on this basis, by introducing soft one-hot to add noise, the traditional one-hot encoded label vector is replaced with the updated label vector, the weight of the true label category is reduced, the focus parameters of the loss part related to the rare class are removed, different losses are assigned to each category, and the focus parameters are selectively used for enhancement or suppression, thereby overcoming the harmful suppression of rare classes and the enhancement of background classes, thereby improving the signal-to-noise ratio and further optimizing the performance of the model in the category imbalance scenario.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the specific implementation methods of the present invention can still be modified or replaced without departing from the spirit and scope of the present invention. Any modification or replacement should be included in the scope of protection of the claims of the present invention.

Claims

1. A weed target recognition method under complex conditions based on deep learning, characterized in that The CoatNet network structure is improved and the network layer is deepened to improve the network performance. The class imbalance problem is solved by designing an asymmetric smoothed focal loss function, namely, an Asymmetric Smoothed Focal Loss function. The Transformer architecture in the CoatNet detection network is also improved to improve the generalization ability of the model. The method includes the following steps: (1) collecting weed image samples; (2) preprocessing and labeling the images; (3) improving the Transformer architecture of the fifth stage in the CoatNet detection network; (4) Improve the CoatNet network structure; (5) Design an asymmetric smooth focal loss function: replace the cross entropy function with the focal loss function, i.e., the Focal Loss function, and introduce label smoothing strategy and asymmetry; (6) inputting the original video image into the trained deep network to obtain the final weed recognition result; the collecting of weed image samples refers to collecting image samples under different environments, such as drought, saline-alkali, fertile and other different soil conditions, as well as different tilt angles and distances, so that the image samples contain pictures under different growth conditions, light intensity, complex background, foreign body occlusion, multi-angle rotation and tilt and different scales; the preprocessing and labeling of the image refers to reducing the noise in the image by applying a Gaussian filter to ensure that the weed features are more obvious, using histogram equalization technology to improve the contrast between the weeds and the background, making the weeds easier to identify, and then using LabelMe software for semi-automatic labeling and then using CVAT, i.e. Computer Vision Annotation Tool, for manual labeling.

2. According to claim 1, a method for identifying weed targets under complex conditions based on deep learning is characterized in that: The improved Transformer architecture of the fifth stage in the CoatNet detection network refers to replacing the MHA module in the Transformer architecture of the fifth stage of the CoatNet detection network with an improved DCMHA, and introducing two activation functions in DCFormer; the improved DCMHA refers to introducing a Gaussian noise detection layer in DCMHA, and introducing Dropout regularization technology in Softmax; the introduction of the Gaussian noise detection layer refers to adding Gaussian noise with a standard deviation σ=0.5 to the Query, Key and Value input vectors of each attention mechanism, performing noise preprocessing on the data, so that the model can better adapt to random disturbances and noise in the data, thereby enhancing its robustness in different environments, and forcing the model to learn more general features. Help prevent the model from overfitting the training data; the introduction of Dropout regularization technology refers to the introduction of Dropout regularization technology in the Softmax function to form the weight matrix of DropMax applied to the attention mechanism. During the training process, Dropout randomly sets some weights in the weight matrix of the attention mechanism to zero to reduce the model's dependence on specific features, thereby improving the generalization ability of the model, and also making the model pay more attention to important features. It can dynamically adjust the focus of attention according to different input data, reduce the model's calculation burden, and improve calculation efficiency; the introduction of two activation functions in DCFormer refers to the introduction of two nonlinear activation functions Noisy ReLU and GELU, namely Gaussian Error Linear Unit, in DCFormer's FFN, namely FeedforwardNeuralNetwork. By using Noisy ReLU at the low level to increase the randomness and diversity of the model, improve the model's sensitivity and expression ability to the input data, and use ReLU at the high level to keep the model fast and simple, and provide sufficient nonlinear capabilities to process complex data features.

3. The method for identifying weed targets under complex conditions based on deep learning according to claim 1, characterized in that: The improved CoatNet network structure refers to adding a third convolution stage after the second convolution stage in CoatNet, and adding residual connections between the first convolution stage and the third convolution stage, and between the second convolution stage and the fifth stage of the network; the newly added third convolution stage refers to the convolution used in the newly added third convolution stage is the improved MBConv-fused convolution, and the improved MBConv-fused convolution refers to adding a DEA module, namely Detail Enhancement Attention, and a BSA module, namely Background Suppression Attention, to the traditional MBConv-fused to increase the receptive field and improve the capture of local details; the DEA module refers to first performing maximum pooling and average pooling operations on the input feature map, splicing the pooling results, and generating spatial attention weights through convolution, thereby strengthening the edge and detail information in the feature map; the BSA module refers to performing global average pooling on the input feature map, generating channel attention weights through a two-layer fully connected network, suppressing background information and adjusting the importance of each channel.

4. The method for identifying weed targets under complex conditions based on deep learning according to claim 1, characterized in that: The design of an asymmetric smooth focal loss function refers to replacing the traditional cross entropy loss function with an improved focal loss function, and introducing a label smoothing strategy and asymmetry; the improved focal loss function refers to introducing a weight factor and a focusing parameter into the focal loss function, and by adjusting the values ​​of the weight factor and the focusing parameter, the model can better cope with the problems of imbalanced positive and negative samples and difficult-to-classify samples; the introduction of a label smoothing strategy refers to introducing softone-hot to add noise; the introduction of asymmetry refers to selectively using the focal parameters for enhancement or suppression, and the selective use of the focal parameters for enhancement or suppression refers to first removing the focal parameters of the loss part related to the rare class, and then retaining the focal parameters of the non-target class, and finally, setting the parameter δ for controlling the weights of positive and negative samples to 0.6, thereby overcoming the harmful suppression of the rare class and the enhancement of the background class, thereby improving the signal-to-noise ratio, and further optimizing the performance of the model in the category imbalance scenario.

Images