Wheat and associated weed identification method

Abstract

The invention discloses a wheat and associated weed identification method, and relates to the technical field of agricultural information. Comprising the following steps: acquiring field image data, taking wheat and associated weeds as target objects, and performing data annotation on all the target objects in the field image data to obtain a multi-source heterogeneous fusion data set; clustering the multi-source heterogeneous fusion data set through an improved K-means algorithm to obtain an initial anchor frame; inputting the field image data into a YOLOv10 model to obtain an identification result; determining a joint loss function according to the difference degree between the initial anchor frame and the identification result; and training the YOLOv10 model according to the joint loss function, and inputting the real-time field image data into the trained YOLOv10 model to obtain a real-time identification result. According to the invention, background area interference can be reduced, and data quality stability is improved; deformation characteristics of weeds and wheat caused by shielding and angle change are effectively captured, and the recognition accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of agricultural information technology, and in particular to a wheat and companion weed identification method. BACKGROUND

[0002] Agriculture not only provides food and clothing for human society, but also guarantees the development and progress of the entire economy. Field weeds have become an important threat to agricultural food security. Because they have both wild plant characteristics and cultivated crop habits, they usually have strong adaptability and stress resistance to the environment, and compete fiercely with crops for resources such as light, water, nutrients, and growth space. At present, there are 1454 species of farmland weeds, of which 130 are difficult to control and malignant, usually causing crop yield reduction of 10%-20%, and in severe cases, more than 50% or even no harvest. With an average of more than 1.4 billion mu of weed occurrence and damage area per year, and an annual cost of 23.5 billion yuan for weed control, weeds still cause a reduction of 50 million tons of grain, with a direct economic loss of nearly 100 billion yuan per year.

[0003] Traditional identification methods mainly rely on manual observation and image processing technology. After collecting pictures, preprocessing work such as noise reduction, enhancement, and color space conversion is needed; then threshold segmentation is performed, and the maximum inter-class variance method (Otsu) or clustering algorithm (K-means, fuzzy C-means) is used to separate the target and background, thereby identifying weeds.

[0004] The image acquisition of the traditional identification method relies on the traditional method under fixed conditions, and farmland images need to be obtained under specific illumination, angle, and height, which is easily affected by shadows, obstructions, or weather changes, resulting in unstable data quality and reducing the accuracy of the identification result. SUMMARY

[0005] Therefore, it is necessary to provide a wheat and companion weed identification method in view of the above technical problems.

[0006] The embodiment of the present application provides a wheat and companion weed identification method, which comprises the following steps: Obtaining field image data comprising wheat and companion weeds, wherein the companion weeds comprise grass weeds and broadleaf weeds; taking the wheat, grass weeds, and broadleaf weeds as target objects, performing data labeling on all target objects in the field image data to obtain a multi-source heterogeneous fusion data set; Taking 1-IoU as a distance measurement function of the K-means algorithm to obtain an improved K-means algorithm; performing clustering on the labeled boxes of all target objects in the multi-source heterogeneous fusion data set by using the improved K-means algorithm to obtain initial anchor boxes of the wheat and companion weeds; performing position transformation on the initial anchor boxes of the wheat and companion weeds by using a mosaic mixed data enhancement strategy to obtain processed initial anchor boxes; Input the field image data into the YOLOv10 model to obtain the identification result of the wheat and the companion weed; determine the joint loss function of the YOLOv10 model according to the difference between the processed initial anchor frame and the identification result; According to the joint loss function, implement the back propagation algorithm to train the YOLOv10 model, and update the weight matrix and the bias term in each back propagation process through the gradient descent method; determine the weight matrix and the bias term corresponding to the minimum joint loss function within the preset iteration number, and obtain the trained YOLOv10 model; Input the real-time field image data into the trained YOLOv10 model to obtain the real-time identification result of the wheat and the companion weed.

[0007] Optionally, the initial anchor frame of the wheat and the companion weed is obtained by improving the K-means algorithm to cluster the labeled frames of all target objects in the multi-source heterogeneous fusion data set, and the initial anchor frame of the wheat and the companion weed is obtained, specifically including: The width and height of the labeled frame of all target objects are extracted from the multi-source heterogeneous fusion data set, and a plurality of labeled frames are randomly selected as initial clustering centers; The distance between each labeled frame and the initial clustering center is calculated through 1-IoU iteration to assign each labeled frame to the nearest clustering center, and a plurality of clustering clusters are obtained; The initial clustering center is updated through the width mean and the height mean of all labeled frames in each clustering cluster until the clustering center converges, a plurality of final clustering centers are obtained, and the plurality of final clustering centers are taken as the initial anchor frame of the wheat and the companion weed.

[0008] Optionally, the position of the initial anchor frame of the wheat and the companion weed is transformed through the mosaic mixed data augmentation strategy to obtain the processed initial anchor frame, specifically including: Four pairs of subgraphs are randomly selected from the multi-source heterogeneous fusion data set based on the following formula to obtain a fusion graph: , ; Wherein, I mosaic is the fusion graph, I i is the four pairs of randomly selected subgraphs, X i is the horizontal coordinate of each subgraph, Y i is the vertical coordinate of each subgraph, and i=1, 2, 3, 4; The initial anchor frame of the wheat and the companion weed is coordinate-transformed according to the position of the subgraph in the fusion graph to obtain the processed initial anchor frame.

[0009] Optionally, wheat, grass weeds and broadleaf weeds are taken as target objects, data labeling is performed on all target objects in the field image data, and a multi-source heterogeneous fusion data set including the labeled boxes of all target objects is obtained, specifically including: Through the image labeling tool and the visual model inference tool, the wheat is labeled as wheat, the grass weed is labeled as weed-1, and the broadleaf weed is labeled as weed-2, and the multi-source heterogeneous fusion data set is obtained.

[0010] Optionally, when training the YOLOv10 model, if the number of acquired field image data is less than a set threshold, the field image data is augmented by the image data augmentation library to simulate different angles, different shooting distances and different light conditions of the farmland environment, and the augmented field image data is obtained.

[0011] Optionally, the field image data is input into the YOLOv10 model to obtain the recognition result of the wheat and the companion weed, specifically including: The YOLOv10 model includes a slice operation structure, a convolution layer, a backbone network, a down-sampling convolution layer and a head network connected in sequence, and the backbone network includes a basic convolution layer and a deformable convolution structure in parallel; The field image data is divided into a plurality of subgraphs by the slice operation structure according to a set step, the dimension order of the plurality of subgraphs is adjusted and the plurality of subgraphs are spliced according to the channel dimension, and an initial feature map is obtained; The initial feature map is subjected to a convolution operation by the convolution layer to adjust the channel number of the initial feature map, and an intermediate feature map is obtained; The intermediate feature map is split into two parts by the basic convolution layer and the deformable convolution structure using the basic convolution operation and the deformable convolution operation, and the two split intermediate feature maps are spliced in the channel dimension, and a splicing result is obtained; The splicing result is subjected to a down-sampling convolution operation by the down-sampling convolution layer, and a multi-scale feature map is obtained; The multi-scale feature map is subjected to classification detection by the head network, and wheat recognition results and companion weed recognition results reflecting the position, size and classification information of the wheat and the plurality of companion weeds are obtained.

[0012] Optionally, the field image data is divided into a plurality of subgraphs by the slice operation structure according to a set step based on the following formula: ; ; Wherein, X is the field image data, X split is a subgraph; The dimension order of the plurality of subgraphs is adjusted and spliced according to the channel dimension based on the following formula to obtain the initial feature map: ; ; wherein, X focus is the initial feature map.

[0013] Optionally, the intermediate feature map is split into two parts by the basic convolution layer and the deformable convolution structure through the basic convolution operation and the deformable convolution operation, and the two split intermediate feature maps are spliced in the channel dimension to obtain a splicing result, specifically including: The intermediate feature map is preprocessed based on the following formula: ; ; wherein, F base is the preprocessed intermediate feature map, and W2 is the second convolution kernel; The preprocessed intermediate feature map is split into two parts based on the following formula through the basic convolution layer and the deformable convolution structure to obtain a first part and a second part: ; wherein, F a is the first part, and F b is the second part; The first part is subjected to 3x3 convolution based on the following formula to obtain a first intermediate feature map: ; The second part is subjected to convolution compression based on the following formula to obtain a second intermediate feature map: ; The first intermediate feature map and the second intermediate feature map are spliced in the channel dimension based on the following formula to obtain a splicing result: ; wherein, W3 is the third convolution kernel, W4 is the fourth convolution kernel, W5 is the fifth convolution kernel, F bottlenec is the first intermediate feature map, F shprtcut is the second intermediate feature map, and F concat is the splicing result.

[0014] Optionally, the joint loss function of the YOLOv10 model is determined according to the difference degree between the initial anchor frame and the recognition result, specifically including: The positioning loss function is determined based on the following formula: ; The classification loss function is determined based on the following formula: ; The positioning loss function and the classification loss function are weighted and summed based on the following formula to obtain a joint loss function: ; Wherein, CIoU is the positioning loss function, Focal Loss is the classification loss function, Total loss is the joint loss function, λ CIoU is the positioning loss coefficient, λ Focal is the classification loss coefficient, and IoU is the distance measurement function.

[0015] Compared with the prior art, the wheat and companion weed identification method provided by the embodiment of the application has the following beneficial effects: The application uses 1-IoU as the distance measurement function of the K-means algorithm, so that the clustering process is directly related to the core indicators of the detection task, and the generated initial anchor frame is more consistent with the width-height distribution of the actual target; because of this anchor frame optimization based on the size of the real target, the coverage ability of multi-morphology target objects is significantly improved, so that the interference of the background area can be reduced, and the stability of the data quality can be improved.

[0016] In addition, the application uses a joint loss function composed of a positioning loss function and a classification loss function to train the YOLOv10 model; the positioning loss function accurately optimizes the boundary box regression, and the classification loss function relieves the class imbalance of weeds and wheat; the combination of the two ensures that the field image data can be effectively learned, enhances the transfer learning ability of the YOLOv10 model, and takes into account the positioning and classification accuracy.

[0017] Finally, the application adopts a subgraph division and channel splicing strategy to split the field image data into multiple subgraphs and integrate the spatial information, thereby reducing the influence of the quality fluctuation of a single image; the combination of the basic convolution operation and the deformable convolution operation can adaptively adjust the receptive field shape, effectively capture the deformation characteristics of weeds and wheat caused by occlusion and angle changes, solve the problem that the traditional identification method is easily affected by environmental interference and causes the accuracy to decrease, and improve the identification accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 FIG. 1 is a model construction schematic diagram of a wheat and companion weed identification method provided in an embodiment; Figure 2 FIG. 2 is a model training schematic diagram of a wheat and companion weed identification method provided in an embodiment; Figure 3 FIG. 3 is a model verification schematic diagram of a wheat and companion weed identification method provided in an embodiment; Figure 4 FIG. 4 is a flowchart of a wheat and companion weed identification method provided in an embodiment. DETAILED DESCRIPTION

[0019] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0020] In one embodiment, a wheat and companion weed identification method is provided, as shown in Figure 4 The method comprises: acquiring field image data comprising wheat and companion weeds, the companion weeds comprising: grass weeds and broadleaf weeds. Wheat, grass weeds and broadleaf weeds are taken as target objects, and all target objects in the field image data are data-labeled to obtain a multi-source heterogeneous fusion dataset. 1-IoU is taken as a distance measurement function of a K-means algorithm to obtain an improved K-means algorithm. The bounding boxes of all target objects in the multi-source heterogeneous fusion dataset are clustered by the improved K-means algorithm to obtain initial anchor boxes of wheat and companion weeds. The initial anchor boxes of wheat and companion weeds are positionally transformed by a mosaic mixed data augmentation strategy to obtain processed initial anchor boxes. The field image data is input into a YOLOv10 model to obtain identification results of wheat and companion weeds. A joint loss function of the YOLOv10 model is determined according to the difference between the processed initial anchor boxes and the identification results. The YOLOv10 model is trained by implementing a back propagation algorithm according to the joint loss function, and the weight matrix and the bias term in each back propagation process are updated by a gradient descent method. The weight matrix and the bias term corresponding to the minimum joint loss function within a preset number of iterations are determined to obtain a trained YOLOv10 model. Real-time field image data is input into the trained YOLOv10 model to obtain real-time identification results of wheat and companion weeds.

[0021] The companion weeds comprise grass weeds and broadleaf weeds. Wheat, grass weeds and broadleaf weeds in the field image data are labeled by an image labeling tool and a visual model inference tool; wheat is labeled as wheat, grass weeds are labeled as weed-1, and broadleaf weeds are labeled as weed-2.

[0022] During training of the YOLOv10 model, if the number of acquired field image data is less than a set threshold, the field image data is augmented by an image data augmentation library to simulate farmland environments under different angles, different shooting distances and different lighting conditions to obtain augmented field image data.

[0023] The wheat recognition result and the companion weed recognition result are obtained by a YOLOv10 model. The YOLOv10 model comprises a slice operation structure, a convolution layer, a backbone network, a down-sampling convolution layer and a head network connected in sequence, and the backbone network comprises a basic convolution layer and a deformable convolution structure in parallel.

[0024] The field image data is divided into a plurality of subgraphs by the slice operation structure according to a set step, the dimension order of the plurality of subgraphs is adjusted, and the plurality of subgraphs are spliced according to the channel dimension to obtain an initial feature graph. The initial feature graph is subjected to a convolution operation by the convolution layer to adjust the channel number of the initial feature graph to obtain an intermediate feature graph. The intermediate feature graph is split into two parts by the basic convolution layer and the deformable convolution structure through basic convolution operation and deformable convolution operation, and the two split intermediate feature graphs are spliced in the channel dimension to obtain a splicing result. The splicing result is subjected to a down-sampling convolution operation by the down-sampling convolution layer to obtain a multi-scale feature graph. The multi-scale feature graph is subjected to classification detection by the head network to obtain the wheat recognition result and the companion weed recognition result reflecting the position, size and classification information of the wheat and the plurality of companion weeds.

[0025] The initial anchor boxes of the wheat and the companion weeds are obtained by clustering the labeled boxes of all target objects in the multi-source heterogeneous fusion dataset by improving the K-means algorithm, and specifically comprising: The width and height of the labeled boxes of all target objects are extracted from the multi-source heterogeneous fusion dataset, and a plurality of labeled boxes are randomly selected as initial clustering centers. The distance between each labeled box and the initial clustering center is calculated by 1-IoU iteration to assign each labeled box to the nearest clustering center to obtain a plurality of clustering clusters. The initial clustering centers are updated by the width mean and height mean of all labeled boxes in each clustering cluster until the clustering centers converge to obtain a plurality of final clustering centers, and the plurality of final clustering centers are used as the initial anchor boxes of the wheat and the companion weeds.

[0026] The specific implementation is as follows: 1. Establishment of a multi-source heterogeneous fusion dataset.

[0027] Field wheat and weed images are collected for model training, which are derived from a shooting and open-source multi-source heterogeneous fusion dataset. The weeds are common grass weeds and broadleaf weeds in wheat fields, and the wheat, weeds and soil are separated by image segmentation technology.

[0028] 2. Data labeling.

[0029] LabelImg tool is used to label the wheat, grass and broadleaf respectively.

[0030] 3. Enhancement of the multi-source heterogeneous fusion dataset.

[0031] The image is augmented using the Augmentor image data augmentation library, and the resulting multi-source heterogeneous fusion dataset is randomly divided into a training set, a validation set, and a test set in an 8:1:1 ratio. 4. Set the network model parameters.

[0032] In the yaml configuration file of YOLOv10, the network parameters are set according to the computer resources and requirements, including the input image size, the number of classes, and the number of iterations, and ensure that the CUDA supported graphics card is used.

[0033] 5. The K-means clustering algorithm is used to solve the initial anchor box for the multi-source heterogeneous fusion dataset.

[0034] (1) The wheat, grass weeds and broadleaf weeds in the field image data are taken as target objects, and the width and height of all target objects are extracted from the training set; (2) The width and height of the target object are normalized to the range [0, 1] for easy clustering; (3) The K-means algorithm is used to cluster the width and height of the target object, and the number of clusters is set to 9 (corresponding to 3 scales and 3 anchor boxes for each scale of YOLOv10), and the distance between each target object and the cluster center is calculated using 1-IoU as the distance metric, and it is assigned to the nearest cluster center; (4) Then update the cluster center to the mean of the width and height of all target objects in this cluster, until the cluster center converges, and after the clustering is completed, the center point of each cluster is taken as the real box.

[0035] 6. The nn.Conv2d function provided by PyTorch is used to implement convolution operation.

[0036] (1) The input image (3x640x640) is sliced by the slice operation structure (Focus structure), and the image is divided into 4 subgraphs with a 2x2 stride, each subgraph size is 3x320x320; (2) These subgraphs are spliced in the channel dimension to form a 12x320x320 feature map; (3) The number of channels is reduced from 12 to 64 through the first convolution, and a 64x320x320 intermediate feature map is obtained; (4) The intermediate feature map enters the backbone network, passes through the basic convolution layer, and then passes through the deformable convolution operation structure (C2f-DCNv4), and the intermediate feature map is divided into two parts; (5) The two parts of the intermediate feature map are spliced in the channel dimension, and the spliced result is down-sampled by the second convolution operation with a stride of 2, gradually reducing the resolution of the feature map and increasing the number of channels, and finally obtaining a multi-scale feature map; (6) Detecting the multi-scale feature map to obtain wheat recognition results and companion weed recognition results reflecting the position, size and classification information of wheat and multiple types of companion weeds.

[0037] 7. Training using a hybrid optimization enhancement strategy and a CIoU-Focal joint loss function.

[0038] (1) Resizing the multi-source heterogeneous fusion dataset containing multiple labeled boxes of wheat and weed images through an adaptive Mosaic enhancement technique to meet the preset size requirements, and then inputting the data in the form of a set batch (minibatch) to the backbone part; (2) The backbone part is composed of a feature extraction module (Backbone), a feature fusion module (Neck) and a detection module (Head), which respectively undertake the functions of extraction, fusion and detection. These three modules together form a forward propagation path, through which the position, size and classification information of wheat and weeds can be output.

[0039] (3) Using a specific loss function to calculate the difference between the predicted box and the real box, obtaining the loss value Loss, and then implementing the back propagation algorithm according to the Loss value; (4) Updating the weight matrix and bias in the forward propagation process by gradient descent method, which aims to optimize the cumulative batch rather than a single batch; (5) In each training cycle, the gradient is updated once, and the weight matrix and bias term when the minimum loss value is reached under the preset number of iterations are calculated; (6) Using the optimal parameters obtained within the specified number of iterations to derive the prediction information of the wheat and weeds to be detected.

[0040] As shown in Figure 1 , a specific embodiment of the present application is provided: 1. Database establishment: Collect field image data for model training, and obtain at least 900 pictures, and the field image data contains wheat and multiple types of companion weeds, and the companion weeds are common gramineous weeds and broadleaf weeds in wheat fields.

[0041] 2. Data labeling: When labeling the field image data, the open-source image labeling tool labelImg is mainly used, and the visual model inference tool Roboflow is used as an auxiliary tool for labeling; The overall samples are divided into gramineous weeds, broadleaf weeds and wheat, wherein the gramineous weeds in the sample are labeled as "weed-1", the broadleaf weeds are labeled as "weed-2", and the wheat is labeled as "wheat"; When labeling, the bounding box is close to the target border. For highly overlapping and large-area grass weeds, the bare ground surface is used as the boundary line to ensure full coverage of the target object.

[0042] 3. Enhancement of multi-source heterogeneous fusion data set: If the number of pictures is not enough for model training, use Augmentor image data augmentation library to expand the field image data: Install Augmentor, give the pip install command, and then import the library; Create a Pipeline object to get the data enhancement pipeline, specify the directory where the pictures are located, and use the ". / images" path; Add various enhancement operations, such as rotation, flip, scaling, cropping, brightness adjustment, etc. Augmentor will automatically create an output directory under the specified path, and the enhanced images are saved in the your_input_path / output directory by default; The following is a Python code example of this process: import Augmentor p = Augmentor.Pipeline(". / images") p.rotate(probability=0.5,max_left_rotation=10,max_right_rotation=10) p.flip_left_right(probability=0.5) p.flip_top_bottom(probability=0.3)p.zoom_random(probability=0.5,percentage_area=0.8) p.random_brightness(probability=0.5,min_factor=0.7,max_factor=1.2) p.sample(600) After expanding the original image library, the pictures are uniformly resized to 640x640 pixels; After expansion, 500 wheat, grass and broadleaf weed pictures are obtained, a total of 1500 pictures as PASCAL VOC (target detection data set), according to the 8:1:1 ratio, divided into training set, validation set and test set; This data enhancement method simulates different angles, different shooting distances and different lighting conditions in natural environment to alleviate overfitting and improve the robustness of the model.

[0043] 4. Set the network model parameters: In the configuration file yaml of the YOLOv10 network model, according to the size of the computer memory, the size of the video memory, the recognition effect and the training speed required by the user, the size of the convolutional neural network input image, the number of recognition categories, the iteration number are set, and the type of video card supporting CUDA acceleration is required, for example, the configuration of the basic model: According to the video memory of the computer used, select the pre-defined model configuration file provided by YOLOv10 (located in the models / directory): select yolov10n / s if the video memory ≤4GB; select yolov10m if the video memory is 6-8GB; select yolov10l / x if the video memory ≥11GB.

[0044] 5. Use K-means clustering algorithm to solve the initial anchor box for the multi-source heterogeneous fusion data set: Extract the width and height of all labeled boxes in the multi-source heterogeneous fusion data set, and construct the multi-source heterogeneous fusion data set , where w i and h i is the normalized width and height (relative to the image size) of the i th labeled box; These sizes will be normalized to the range [0, 1] for ease of clustering; Randomly select K =9 samples as the initial cluster centers ; Use 1-IoU as the distance metric to iteratively calculate the distance between each labeled box and the cluster center, and assign it to the nearest cluster center, as follows: a. For each labeled box ( w i , h i ), calculate its distance d ij from all cluster centers ; Then , and assign ( w i , h i ) to the cluster j =argmin j d ij ; b. For each cluster j ​, recalculate the center as the mean value of all bounding boxes in the cluster: ; ; wherein, S j is the set of bounding boxes belonging to the cluster j , |C| represents the size of the set; S j ; c. Repeat the assignment and update steps until the amount of change in cluster centers is less than a threshold (e.g. ) or the maximum number of iterations is reached; The final K cluster centers are the initial anchor boxes, whose width and height distribution matches the target size in the multi-source heterogeneous fusion data set. The center point of each cluster will be used as the size of the initial anchor box, which can improve the boundary box regression efficiency and detection accuracy of YOLOv10 during training; 6. Use the nn.Conv2d function provided by PyTorch to implement convolution operation: The input image (3x640x640) is sliced by the slice operation structure, and the field image data is divided into multiple subgraphs according to the specified step size: The image is divided into 4 subgraphs with a step size of 2x2, and each subgraph has a size of 3x320x320.

[0045] , then ; wherein, X is the field image data, X split is the subgraph.

[0046] Adjust the dimension order of multiple subgraphs and splice the channels to obtain the initial feature map: ; then ; wherein, X focus is the initial feature map (12x320x320).

[0047] The first convolution operation is used to convolve the initial feature map to reduce the number of channels of the initial feature map (from 12 to 64), and an intermediate feature map of 64x320x320 is obtained, the specific process is as follows: Use the first convolution kernel , i.e. 3x3 convolution, step size is 1, padding is 1, which can get ; then ; wherein F out is the intermediate feature map, and W1 is the first convolution kernel.

[0048] The intermediate feature map enters the backbone network, passes through the basic convolution layer, and the second convolution kernel , i.e. 3x3 convolution, stride 1, padding 1, to obtain ; then ; wherein F base is the pre-processed intermediate feature map, and W2 is the second convolution kernel.

[0049] Through the deformable convolution structure (C2f-DCNv4), the pre-processed feature map F base is divided into two parts: ; wherein F a is the first part, and F b is the second part.

[0050] Through the Bottleneck residual block containing two 3x3 convolutions, we obtain: ; wherein W3 is the third convolution kernel, W4 is the fourth convolution kernel, ; Through 1x1 convolution compression, we can obtain: ; wherein W5 is the fifth convolution kernel, ; Then the two part feature maps are spliced in the channel dimension using the concat operation: ; wherein F bottlenec is the first intermediate feature map, F shprtcut is the second intermediate feature map, and F concat is the splicing result.

[0051] And through the convolution kernel with a stride of 2, down-sampling is performed to gradually reduce the resolution of the feature map and increase the number of channels, and finally a multi-scale feature map is generated: ; wherein F down is the multi-scale feature map, W6 is the sixth convolution kernel, and W6∈R 16×16×1×1 .

[0052] 7. The method according to any one of claims 1-6, whereinFigure 2 As shown, the training is performed using a hybrid optimization enhancement strategy and a CIoU-Focal joint loss function: Through an adaptive mosaic mixed data enhancement strategy (Mosaic enhancement technology), four I1, I2, I3, and I4 are spliced into a large image I from a wheat and weed image multi-source heterogeneous fusion dataset containing multiple labeled boxes mosaic , with a size of 2Sx2S (S is the target input size, which is 640 pixels in this project); The position of each sub-image is randomly distributed, and its size and aspect ratio are adjusted, , ; wherein, wherein, I mosaic is a fusion image, I i is four randomly selected sub-images, X i is the horizontal coordinate of each sub-image, Y i is the vertical coordinate of each sub-image, i=1, 2, 3, 4.

[0053] The initial anchor box of wheat and companion weeds is transformed according to the position of the sub-image in the fusion image to obtain the processed initial anchor box.

[0054] Further mixed enhancement operations such as random cropping, color jittering, and random rotation are applied to the Mosaic enhanced image to meet the preset size requirement. Then the data is input to the backbone in the form of a set batch (minibatch); The backbone is composed of a feature extraction module (Backbone), a feature fusion module (Neck), and a detection module (Head), which are responsible for extraction, fusion, and detection, respectively. These three modules together form a forward propagation path, through which the position, size, and classification information of wheat and weeds can be output;

[0055] The Backbone module is optimized by replacing the C2f structure with a deformable convolution C2f-DCNv4. The specific operation is as follows: Add the C2f_DCNv4 module to ultralytics / nn / modules / block.py, and implement it with the following code: class C2f_DCNv4(nn.Module): def __init__(self, c1, c2,...): super().__init__() self.conv = Conv(c1, c2 / / 2, 1) self.dcnv4 = DCNv4(c2 / / 2, kernel_size=3,...) # integrate DCNv4 operator def forward(self, x): x = self.conv(x) x = self.dcnv4(x) return x Add 'C2f_DCNv4' to the __all__ list at the top of block.py‌ Import the module in ultralytics / nn / modules / __init__.py; Optimize the Backbone module by adding a deformable convolution C2f-DCNv4 layer between the SCDown structure and the C2fCIB structure. The specific operation is as follows: Adjust the module order in the YOLO yaml configuration file. The code is as follows: backbone: - [-1, 1, SCDown, [channels]] # down-sampling layer - [-1, 1, C2f_DCNv4, [channels, True]] # add DCNv4 module - [-1, 1, C2fCIB, [channels]] # subsequent CIB module‌ Optimize the Head module by adding a small target detection head. The specific operation is as follows: Open the yolov10.yaml file and add a line under anchors: -[4, 6, 8, 14, 15, 11] # P2; Use a specific CIoU-Focal combined loss function, which is Positioning loss function CIoU Loss: ; Classification loss function Focal Loss: ; The combined loss function is the weighted sum of the positioning loss and the classification loss: ; Where CIoU is the positioning loss function, Focal Loss is the classification loss function, Total loss is the combined loss function, λ CIoU is the positioning loss coefficient, λ FocalFor the classification loss coefficient, IoU is the distance metric function, and λ CIoU = 0.05, λ Focal = 0.5. The difference between the predicted frame and the real frame is calculated, the loss value Loss is calculated through the loss function, and then the back propagation algorithm is implemented according to the Loss value. The real frame is a manually labeled bounding box used to accurately label the position and size of the target object (such as weeds). As shown in Figure 3 The predicted frame is a bounding box detected by the algorithm, which represents the area where the algorithm thinks there may be a target object. The higher the overlap between the predicted frame and the real frame, the lower the loss value, and the higher the detection accuracy.

[0056] The weight matrix and bias in the forward propagation process are updated by gradient descent method, which aims to optimize the cumulative batch rather than a single batch; In each training period, the gradient is updated once, and the weight matrix and bias term when the minimum loss value is reached under the preset number of iterations are calculated; Model verification: using the optimal parameters obtained within the specified number of iterations, the predicted information of the wheat and weeds to be detected is derived; Prepare the image to be detected on the computer, change the configuration file yaml and the path of the image to be detected, and finally the computer saves the picture of the target detection result by executing the model program.

[0057] The beneficial effects that can be achieved by the present application include but are not limited to: The use of hybrid optimization enhancement strategy improves the recognition accuracy of the model in variable farmland environment, and increases the practicability of the model under the challenges of complex background, different light conditions and crop growth cycle changes.

[0058] The adaptive Mosaic enhancement generates complex scenes and multi-scale targets, which complement the multi-scale feature fusion capability of the Neck module, and significantly improve the small target detection effect.

[0059] YOLOv10 embeds the function of calculating the initial anchor frame into the model, and adaptively calculates the best anchor frame value in different training sets each time the model is trained, which better matches targets of different sizes.

[0060] The multi-layer deformable convolution C2f-DCNv4 structure can ensure that the model greatly improves the feature extraction capability without increasing the calculation amount, and improves the classification accuracy.

[0061] Adding a small target detection head can improve the detection capability of the model for small targets and reduce the missed detection rate.

[0062] The combined loss function CIoU-Focal loss ensures that the enhanced complex data can be effectively learned, and the enhanced model has the ability of transfer learning. The positioning loss function CIoU loss precisely optimizes the boundary box regression. The classification loss function FocalLoss alleviates the class imbalance between the weeds and the wheat. The combination of the two makes the model balance the positioning and classification accuracy.

[0063] The above-described embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application.

Claims

1. A method for identifying wheat and its associated weeds, characterized in that, include: Acquire field image data including wheat and associated weeds, wherein the associated weeds include: grass weeds and broadleaf weeds; take wheat, grass weeds and broadleaf weeds as target objects, and perform data annotation on all target objects in the field image data to obtain a multi-source heterogeneous fusion dataset; Using 1-IoU as the distance metric function of the K-means algorithm, an improved K-means algorithm is obtained. The improved K-means algorithm is used to cluster the bounding boxes of all target objects in the multi-source heterogeneous fusion dataset to obtain the initial anchor boxes of wheat and associated weeds. The initial anchor boxes of wheat and associated weeds are transformed by a mosaic data augmentation strategy to obtain the processed initial anchor boxes. Field image data is input into the YOLOv10 model to obtain the identification results of wheat and associated weeds; the joint loss function of the YOLOv10 model is determined based on the degree of difference between the processed initial anchor boxes and the identification results. The YOLOv10 model is trained by implementing the backpropagation algorithm based on the joint loss function, and the weight matrix and bias terms are updated in each backpropagation process by gradient descent. The weight matrix and bias terms corresponding to the minimum joint loss function within a preset number of iterations are determined to obtain the trained YOLOv10 model. Real-time field image data is input into the trained YOLOv10 model to obtain real-time identification results of wheat and associated weeds.

2. The method for identifying wheat and associated weeds as described in claim 1, characterized in that, The improved K-means algorithm is used to cluster the bounding boxes of all target objects in the multi-source heterogeneous fusion dataset to obtain the initial anchor boxes for wheat and associated weeds. Specifically, this includes: The width and height of the bounding boxes of all target objects are extracted from the multi-source heterogeneous fusion dataset, and multiple bounding boxes are randomly selected as the initial cluster centers. The distance between each bounding box and the initial cluster center is calculated through 1-IoU iteration to assign each bounding box to the nearest cluster center, resulting in multiple clusters. The initial cluster centers are updated by taking the average width and height of all bounding boxes in each cluster until the cluster centers converge, resulting in multiple final cluster centers. These final cluster centers are then used as the initial anchor boxes for wheat and associated weeds.

3. The method for identifying wheat and associated weeds as described in claim 1, characterized in that, The process of transforming the initial anchor frames of wheat and associated weeds using a mosaic data augmentation strategy to obtain processed initial anchor frames specifically includes: Based on the following formula, four sub-graphs are randomly selected from the multi-source heterogeneous fusion dataset and stitched together to obtain the fused graph: ， ； Among them, I mosaic For the fusion graph, I i For four randomly selected subgraphs, X i Y is the x-coordinate of each subplot. i Let i be the ordinate of each subgraph, i = 1, 2, 3, 4; The initial anchor frames of wheat and associated weeds are transformed according to their positions in the fused graph to obtain the processed initial anchor frames.

4. The method for identifying wheat and associated weeds as described in claim 1, characterized in that, The process involves using wheat, grassy weeds, and broadleaf weeds as target objects, and labeling all target objects in the field image data to obtain a multi-source heterogeneous fusion dataset, specifically including: Wheat was labeled as "wheat", grass weeds as "weed-1", and broadleaf weeds as "weed-2" using image annotation tools and visual model inference tools, resulting in a multi-source heterogeneous fusion dataset.

5. The method for identifying wheat and associated weeds as described in claim 1, characterized in that, It also includes that, when training the YOLOv10 model, if the number of field image data obtained is less than a set threshold, the field image data is augmented using an image data augmentation library to simulate farmland environments under different angles, shooting distances, and lighting conditions, thereby obtaining augmented field image data.

6. The method for identifying wheat and associated weeds as described in claim 1, characterized in that, The process of inputting field image data into the YOLOv10 model to obtain identification results for wheat and associated weeds specifically includes: The YOLOv10 model includes: a slicing operation structure, a convolutional layer, a backbone network, a downsampling convolutional layer, and a head network connected in sequence. The backbone network includes parallel basic convolutional layers and deformable convolutional structures. The field image data is divided into multiple sub-images according to a set stride by a slicing operation structure. The dimensional order of the multiple sub-images is adjusted and they are stitched together according to the channel dimension to obtain the initial feature map. The initial feature map is convolved by a convolutional layer to adjust the number of channels in the initial feature map, thus obtaining an intermediate feature map. The intermediate feature map is split into two parts by using basic convolutional layers and deformable convolutional structures, and the two split intermediate feature maps are then concatenated along the channel dimension to obtain the concatenated result. The stitched result is subjected to downsampling convolution operation by downsampling convolution layer to obtain multi-scale feature map; By classifying and detecting multi-scale feature maps using a head network, we obtain wheat identification results and associated weed identification results that reflect the location, size, and classification information of wheat and various types of associated weeds.

7. The method for identifying wheat and associated weeds as described in claim 6, characterized in that, Based on the following formula, field image data is divided into multiple sub-maps according to a set stride using a slicing operation structure: ； ； in, X For field image data, X split For subgraphs; The initial feature map is obtained by adjusting the dimensional order of multiple sub-images and concatenating them according to the channel dimensions based on the following formula: ； ； Among them, X focus This is the initial feature map.

8. The method for identifying wheat and associated weeds as described in claim 6, characterized in that, The process involves splitting the intermediate feature map into two parts using basic convolutional layers and deformable convolutional structures, and then concatenating the two split intermediate feature maps along the channel dimension to obtain the concatenated result. Specifically, this includes: The intermediate feature maps are preprocessed based on the following formula: ； ； Among them, F out For intermediate feature maps, F base This is the preprocessed intermediate feature map, and W2 is the second convolution kernel; Based on the following formula, the preprocessed intermediate feature map is divided into two parts using a basic convolutional layer and a deformable convolutional structure, resulting in the first part and the second part: ； Among them, F a For the first part, F b This is the second part; Based on the following formula, a 3x3 convolution is performed on the first part to obtain the first intermediate feature map: ； Based on the following formula, the second part is convolutionally compressed to obtain the second intermediate feature map: ； The first and second intermediate feature maps are concatenated along the channel dimension based on the following formula to obtain the concatenated result: ； Where W3 is the third convolution kernel, W4 is the fourth convolution kernel, W5 is the fifth convolution kernel, and F... bottlenec For the first intermediate feature map, F shprtcut For the second intermediate feature map, F concat This is the result of splicing.

9. The method for identifying wheat and associated weeds as described in claim 1, characterized in that, The determination of the joint loss function of the YOLOv10 model based on the degree of difference between the initial anchor box and the recognition result specifically includes: The localization loss function is determined based on the following formula: ； The classification loss function is determined based on the following formula: ； The joint loss function is obtained by weighted summing the localization loss function and the classification loss function based on the following formula: ； Where CIoU is the localization loss function, Focal Loss is the classification loss function, Total loss is the joint loss function, and λ is the inequality. CIoU λ is the positioning loss coefficient. Focal is the classification loss coefficient, and IoU is the distance metric function.

Images