Training method, application method, device and equipment for weed seed detection model

Abstract

The present application provides a training method, application method, device, and apparatus for a weed seed detection model. The method includes: obtaining an image dataset of weed seeds; adding a module that upsamples a first feature map to a second feature map in a feature pyramid network of a preset target detection model and deleting the first sampling module to obtain a modified feature pyramid network; adding a module that downsamples the second feature map to the first feature map in a path aggregation network of the target detection model and deleting the second sampling module to obtain a modified path aggregation network; replacing the coarse-fine module in the backbone network of the target detection model with a reparameterized visual geometry group model and discarding the third sampling module to obtain a modified backbone network; obtaining a weed seed detection model based on the modified feature pyramid network, the modified path aggregation network, and the modified backbone network, and training the weed seed detection model using a training set in the image dataset.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a training method, application method, device and equipment for a weed seed detection model. Background Art

[0002] In recent years, agricultural technology has kept pace with the times. The deep integration of scientific and technological innovation with the agricultural industry has given rise to smart agriculture. Leveraging advanced technology, smart agriculture manages and controls agricultural production through informatization, intelligence, and automation. This approach aims to improve agricultural production efficiency, reduce resource consumption, and protect the ecological environment. It has become a key development direction for modern agriculture.

[0003] Existing weed seed detection model training methods rely on manual inspection or traditional computer vision techniques. However, these methods often lack accuracy for small objects like weed seeds, resulting in high missed detection rates. Furthermore, they suffer from excessive model redundancy, making them difficult to meet the demands of real-time detection. Summary of the Invention

[0004] This application aims to provide a training method, application method, device and equipment for a weed seed detection model, which can solve the problems of insufficient small target detection accuracy and high model redundancy.

[0005] In a first aspect, embodiments of the present application disclose a method for training a weed seed detection model, the method comprising:

[0006] Get a dataset of weed seed images;

[0007] A first path is added to a feature pyramid network of a preset object detection model, and a first sampling module is deleted to obtain a modified feature pyramid network; the first path is used to upsample the first feature map to the size of the second feature map and then fuse it, and the size of the first feature map is smaller than the size of the second feature map; the first sampling module is used to upsample the third feature map to the size of the fourth feature map and then fuse it, and the size of the third feature map is smaller than the size of the fourth feature map;

[0008] Adding a second path to the path aggregation network of the target detection model and deleting the second sampling module to obtain a modified path aggregation network; the second path is used to downsample the second feature map to the size of the first feature map and then fuse it; the second sampling module is used to downsample the fourth feature map to the size of the third feature map and fuse it;

[0009] Replacing the coarse-fine module in the backbone network of the object detection model with a reparameterized visual geometry group model and discarding the third sampling module to obtain a modified backbone network;

[0010] A weed seed detection model is obtained according to the modified feature pyramid network, the modified path aggregation network and the modified backbone network, and the weed seed detection model is trained using a training set in the image dataset.

[0011] In a second aspect, an embodiment of the present application discloses an application method of a weed seed detection model, the method comprising:

[0012] Obtaining a seed image to be identified;

[0013] The seed image to be identified is input into a trained weed seed detection model to obtain an output weed seed identification result, wherein the weed seed detection model is trained by the weed seed detection model training method described in the first aspect.

[0014] In a third aspect, an embodiment of the present application discloses a training device for a weed seed detection model, the device comprising:

[0015] A first acquisition module is used to acquire an image dataset of weed seeds;

[0016] A first network improvement module is configured to add a first path to a feature pyramid network of a preset object detection model and delete a first sampling module to obtain a modified feature pyramid network; the first path is configured to upsample a first feature map to the size of a second feature map and then fuse the first feature map, where the size of the first feature map is smaller than the size of the second feature map; the first sampling module is configured to upsample a third feature map to the size of a fourth feature map and then fuse the third feature map, where the size of the third feature map is smaller than the size of the fourth feature map;

[0017] A second network improvement module is configured to add a module for downsampling the second feature map to the first feature map in the path aggregation network of the target detection model, and delete the second sampling module to obtain a modified path aggregation network; the second sampling module is configured to downsample the fourth feature map to the third feature map;

[0018] a third network improvement module, configured to replace the coarse-fine module in the backbone network of the object detection model with a reparameterized visual geometry group model and discard the third sampling module to obtain a modified backbone network;

[0019] A model training module is used to obtain a weed seed detection model based on the modified feature pyramid network, the modified path aggregation network and the modified backbone network, and train the weed seed detection model using a training set in the image dataset.

[0020] In a fourth aspect, an embodiment of the present application discloses an application device of a weed seed detection model, the device comprising:

[0021] A second acquisition module is used to acquire a seed image to be identified;

[0022] The model application module is used to input the seed image to be identified into a trained weed seed detection model to obtain an output weed seed identification result, wherein the weed seed detection model is trained by the weed seed detection model training method described in the first aspect.

[0023] In a fifth aspect, an embodiment of the present application discloses an electronic device, comprising a processor and a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0024] In summary, in this embodiment, on the one hand, by adding a new path that is more suitable for small targets to the target detection model, the detection accuracy of small targets such as weed seeds is improved. On the other hand, by discarding some paths in the target detection model, the model calculation amount is reduced without significantly affecting the model performance, thus achieving a lightweight weed seed detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flowchart of the steps of a training method for a weed seed detection model provided in an embodiment of the present application.

[0026] Figure 2 This is a flowchart of the specific steps of a training method for a weed seed detection model provided in an embodiment of the present application.

[0027] Figure 3 This is a network framework diagram of a feature pyramid network provided in an embodiment of the present application.

[0028] Figure 4 This is a network framework diagram of a path aggregation network provided in an embodiment of the present application.

[0029] Figure 5 This is a network framework diagram of a re-parameterized visual geometry group model provided in an embodiment of the present application.

[0030] Figure 6 This is a network framework diagram of a weed seed detection model provided in an embodiment of the present application.

[0031] Figure 7 This is a flowchart of the steps of an application method of a weed seed detection model provided in an embodiment of the present application.

[0032] Figure 8This is a block diagram of a training device for a weed seed detection model provided in an embodiment of the present application.

[0033] Figure 9 This is a block diagram of an application device of a weed seed detection model provided in an embodiment of the present application.

[0034] Figure 10 This is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0036] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0037] In the description of this disclosure, unless otherwise specified, "plurality" refers to two or more than two, and other quantifiers are similar; "at least one item", "one or more items" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one item a can represent any number of a; for another example, one or more items among a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural; "and / or" is a type of relationship that describes the association of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " indicates that the related objects are in an "or" relationship.

[0038] Although operations or steps are described in a particular order in the drawings in the embodiments of the present disclosure, this should not be understood as requiring that these operations or steps be performed in the particular order shown or in a serial order, or that all of the operations or steps shown be performed to obtain a desired result. In the embodiments of the present disclosure, these operations or steps may be performed serially; these operations or steps may also be performed in parallel; or some of these operations or steps may be performed.

[0039] First, the application scenarios of the present disclosure are described.

[0040] Take Palmer amaranth, for example. Its invasion of farmland can inhibit crop growth, leading to severe yield and quality losses. For example, in its native North American agricultural region, Palmer amaranth has caused a 65% decrease in cotton yield, a 79% decrease in soybean yield, and a 91% loss in corn yield. Current methods for detecting quarantine weed seeds primarily rely on manual inspection or traditional computer vision techniques, but these methods suffer from significant limitations: 1. Inadequate detection accuracy for small objects: The P3-P5 detection heads in the traditional YOLOv8n model (corresponding to a downsampling ratio of 1 / 8 to 1 / 32) lack feature resolution for tiny seeds (such as Palmer amaranth seeds, which measure only 1×1 mm), resulting in a high rate of missed detections. 2. Excessive model redundancy: The large number of parameters in standard convolutional layers limits inference speed when deployed on edge devices, making it difficult to meet real-time detection requirements.

[0041] To solve the above-mentioned problems, the present application provides a training method, application method, device and equipment for a weed seed detection model. The specific implementation methods of the present application are described in detail below with reference to the accompanying drawings.

[0042] Figure 1 This embodiment provides a training method for a weed seed detection model, referring to Figure 1 , the method may include the following steps.

[0043] Step 101: Obtain a weed seed image dataset.

[0044] In some embodiments, step 101 may include: capturing a preset number of weed seed images with a camera, counting valid samples in the weed seed images, and performing preprocessing to obtain a weed seed image dataset.

[0045] For example, for the weed seeds Palmer amaranth seeds, the preset number may be 1,000 images. Step 101 may include: using an industrial camera to separately photograph Palmer amaranth seed groups under different lighting conditions, and collecting 1,000 images to construct a Palmer amaranth seed dataset.

[0046] For another example, step 101 may include: simulating an actual application scenario, mixing seeds of other sizes into the image collection, and collecting a total of 1,000 images to construct a weed seed dataset.

[0047] In some embodiments, counting valid samples in a weed seed image and performing preprocessing to obtain a weed seed image dataset may include: adding annotation information to the weed seed image using an automatic annotation tool to count and obtain valid samples; and converting a label file in an original format corresponding to the image dataset into a label file in a standard format.

[0048] Optionally, the original format can be JavaScript Object Notation (JSON), or the standard format can be the text (txt) format commonly used by YOLO (You Only Look Once). For example, use the X-Anylabeling tool to add label information to images in an image dataset, count valid samples, and generate a JSON-formatted label file corresponding to the image dataset; then convert the JSON-formatted label file to the txt-formatted label file commonly used by YOLO.

[0049] In some embodiments, step 101 may further include dividing the image dataset into a training set and a validation set according to a preset ratio.

[0050] Optionally, the preset ratio can be 70% for the training set and 30% for the validation set. For example, if the image dataset consists of 1,000 images, and the Amaranth seed dataset is divided into a training set and a validation set in a 7:3 ratio, the training set will include 700 images and the validation set will include 300 images.

[0051] Step 102: Add a first path to the feature pyramid network of the preset object detection model and delete the first sampling module to obtain a modified feature pyramid network.

[0052] The first path is used to upsample the first feature map to the size of the second feature map and then fuse it. The size of the first feature map is smaller than that of the second feature map. The first sampling module is used to upsample the third feature map to the size of the fourth feature map and then fuse it. The size of the third feature map is smaller than that of the fourth feature map.

[0053] In some embodiments, the preset target detection model can be a YOLOv8n model.

[0054] In the embodiments of this application, Feature Pyramid Networks (FPN) is a network structure used for feature fusion. Its main function is to fuse feature maps of different scales, allowing the model to capture target information at different scales. For example, in the YOLOv8n model, FPN upsamples and fuses the multiple scale feature maps (P2, P3, P4, P5, etc.) output by the backbone network.

[0055] It should be noted that upsampling is also known as image enlargement or image interpolation. The main purpose of upsampling is to enlarge the original image so that it can be displayed on a higher resolution display device. The principle of upsampling is that image enlargement almost always uses the interpolation method, that is, new elements are inserted between pixels based on the original image pixels using a suitable interpolation algorithm. Interpolation algorithms also include traditional interpolation, edge image-based interpolation, and area-based image interpolation.

[0056] In some embodiments, the upsampling method in step 102 may include any one of the following: bilinear interpolation, transposed convolution, and unpooling.

[0057] In the embodiment of the present application, the size of the first feature map is smaller than the size of the second feature map. For example, the first feature map may be a P3 feature map, and the second feature map may be a P2 feature map.

[0058] It should be noted that P2 and P3 represent feature maps of different scales output by the backbone network. The feature map of P2 has the largest size, the least semantic information, but the richest spatial information; the size of the feature map of P3 is smaller than that of P2, but has more semantic information. By upsampling, the size of the feature map of P3 will be adjusted to the same size as that of the feature map of P2. Figure 1 Similarly, fusing the two can make the model perform better in small target detection, because small targets are easier to detect in large-size feature maps (P2), and fusing the semantic information of P3 helps improve the accuracy of detection.

[0059] In the embodiment of the present application, the first sampling module is used to upsample the third feature map to a fourth feature map.

[0060] In the embodiment of the present application, the size of the third characteristic map is smaller than that of the fourth characteristic map. For example, the third characteristic map may be a P5 characteristic map, and the fourth characteristic map may be a P4 characteristic map.

[0061] It's important to note that the P5 feature map has the smallest size and the richest semantic information, but less spatial information. The P4 feature map is larger than the P5, with intermediate semantic and spatial information. In the original FPN architecture, the P5 feature map is upsampled to the P4 size and then fused. By discarding the first upsampling module, the model no longer performs upsampling and fusion of the P5 and P4 feature maps, reducing the model's computational complexity and parameter count.

[0062] Step 103: Add a second path to the path aggregation network of the target detection model and delete the second sampling module to obtain a modified path aggregation network.

[0063] The second path is used to downsample the second feature map to the size of the first feature map and then fuse it. The second sampling module is used to downsample the fourth feature map to the size of the third feature map and then fuse it.

[0064] In an embodiment of the present application, a Path Aggregation Network (PAN) is a network structure used to enhance the multi-scale feature fusion capability of a target detection model. It is improved on the basis of FPN and can more effectively fuse feature information of different scales, thereby improving the detection performance of the target detection model on targets of different sizes.

[0065] It should be noted that down sampling is a process in deep learning and signal processing that reduces the spatial resolution and amount of data, thereby reducing the computational complexity, capturing a wider range of contextual information, and preventing overfitting by reducing the size of feature maps.

[0066] In some embodiments, the upsampling method in step 102 may include any one of the following: pooling, convolution with a step size greater than 1, etc.

[0067] It can be understood that P2 and P3 represent feature maps of different scales output by the backbone network. P2 has rich spatial information, and the P3 feature map has relatively stronger semantic information. By downsampling the P2 feature map to the size of the P3 feature map and then performing feature fusion, the P3 feature map can incorporate more spatial information while retaining the semantic information, thereby improving the model's ability to locate and detect small targets.

[0068] In the embodiment of the present application, the second sampling module is used to downsample the fourth feature map to the third feature map.

[0069] It's important to note that in the original PAN architecture, the P4-to-P5 downsampling module is designed to transfer P4 information to P5, further enhancing P5's feature expression capabilities. Because P4 and P5 already contain sufficiently rich semantic information, downsampling P4 to P5 doesn't significantly improve model performance. By removing the second downsampling module, we can reduce computational complexity and improve model inference speed without significantly impacting model performance.

[0070] Step 104: Replace the coarse-fine module in the backbone network of the target detection model with the reparameterized visual geometry group model, and discard the third sampling module to obtain a modified backbone network.

[0071] In this embodiment of the present application, the backbone network is used to extract features from the input image, providing information for subsequent object detection tasks. The backbone network adopts a layered architecture, gradually downsampling and extracting features from the input image, generating feature maps of different scales. These feature maps are then passed to the subsequent FPN and PAN for further processing.

[0072] It should be noted that the backbone network includes a convolutional (Conv) module, a coarse-to-fine (C2f) module, and a spatial pyramid pooling fast (SPPF) module. The C2f module is an improved version of the cross-stage partial network (CSP) module in the YOLOv8n model. The C2f module splits the input features, extracting some features through multiple convolutional layers, while directly connecting the other features across layers, and finally fusing the two features. Optionally, for ease of description, the C2f layer in this application may also be referred to as the "localization classification layer."

[0073] In an embodiment of the present application, the Re-parameterization Visual Geometry Group (RepVGG) model is a convolutional neural network architecture that has performed well in the field of computer vision. Because it uses a single-path structure during inference, it avoids the computational overhead caused by branching and addition operations in a multi-branch structure. Compared with some complex network structures, RepVGG has obvious advantages in memory usage and computing resource requirements, and is more suitable for running on resource-constrained devices. Moreover, although it has a simple structure during inference, through the multi-branch structure during training, RepVGG can learn rich feature information, and its performance is not affected. Replacing the coarse-fine module in the backbone network of the target detection model with the Re-parameterization Visual Geometry Group model can achieve fast and accurate inference without being restricted by the hardware platform.

[0074] In the embodiment of the present application, the third sampling module may be a fifth downsampling module.

[0075] It should be noted that by discarding the third downsampling module, the feature map, which originally required five downsampling passes, now only undergoes four downsampling passes. This reduces the number of downsampling passes and improves the feature map resolution, preserving more image details and facilitating the detection of small objects. Furthermore, by discarding the fifth downsampling module, one layer of computation is eliminated from the network, reducing the computational complexity and parameter count of the model, making it more lightweight and faster.

[0076] Step 105: Obtain a weed seed detection model based on the modified feature pyramid network, the modified path aggregation network, and the modified backbone network, and train the weed seed detection model using a training set in the image dataset.

[0077] In the embodiment of the present application, the structures of the feature pyramid network, path aggregation network, and backbone network are modified. The FPN is enhanced and optimized based on the original structure, deleting some paths and adding new paths that are more suitable for small targets. The PAN is modified on the original structure, reconstructing the information interaction direction and focusing on the fusion of mid- and low-level features. The C2f module in the backbone is replaced with the RepVGG model, and the number of sampling times is reduced. In this way, the YOLOv8n model is improved and a weed seed detection model is obtained.

[0078] In some embodiments, the weed seed detection model is trained by building a virtual environment for training the model on a Windows Server 2022 Datacenter server.

[0079] In summary, in this embodiment, on the one hand, by adding a new path that is more suitable for small targets to the target detection model, the detection accuracy of small targets such as weed seeds is improved. On the other hand, by discarding some paths in the target detection model, the model calculation amount is reduced without significantly affecting the model performance, thus achieving a lightweight weed seed detection model.

[0080] Figure 2 This is a flowchart of the specific steps of a training method for a weed seed detection model provided in an embodiment of the present application. Figure 2 The training method of the above-mentioned weed seed detection model may include the following steps.

[0081] Step 201: Obtain a weed seed image dataset.

[0082] The method of this step has been described in the aforementioned step 101 and will not be repeated here.

[0083] Step 202: Upsampling is performed between the first preset layer and the second preset layer in the feature pyramid network through a first path to expand the size of the first feature map to the size of the second feature map.

[0084] In the embodiment of the present application, the first preset layer is the 15th layer, and the second preset layer is the 16th layer.

[0085] For example, see Figure 3 , the network framework diagram of the feature pyramid network is as follows Figure 3 As shown. Between the 15th and 16th layers in the feature pyramid network, upsampling is performed through the first path to expand the P3 feature map to the size of the P2 feature map module. Figure 3 The network consists of one upsampling layer (Upsample), one feature fusion layer (Concat), and one localization and classification layer (C2f), with 128 channels. The P2 feature map has a resolution of 160×160, which is one-quarter the input image resolution (640×640). The first input for upsampling is the fused and spliced ​​P3 feature map from the FPN network, specifically the 15th layer of the feature pyramid network. This doubles the size of the original P3 feature map from the 15th layer. The second input for concat splicing is the second downsampled feature map from the backbone network, specifically the second layer of the feature pyramid network. This doubled-up P3 feature map is concatenated with the second downsampled feature map. C2f fuses these spliced ​​feature maps to enhance the expressive power of the P2 feature map.

[0086] Step 203: Delete the first upsampling layer, the feature fusion layer, and the positioning classification layer between the third preset layer and the fourth preset layer in the feature pyramid network to obtain a modified feature pyramid network.

[0087] In the embodiment of the present application, the third preset layer is the 10th layer, and the fourth preset layer is the 12th layer.

[0088] In the embodiment of the present application, the positioning classification layer is the C2f layer.

[0089] Step 204: Downsampling is performed through the second path in the path aggregation network to reduce the size of the second feature map to the size of the first feature map.

[0090] For example, see Figure 4 , the network framework diagram of the path aggregation network is as follows Figure 4 As shown. The network structure of the module that downsamples the P2 feature map to the P3 feature map in the path aggregation network, Figure 4The network consists of one convolutional layer (Conv), one feature fusion layer (Concat), and one localization and classification layer (C2f layer), with 256 channels. The first input source for Conv downsampling is the P2 feature map enhanced by C2f fusion in step 203, specifically the 13th layer in the path aggregation network. This layer reduces the size of the P2 feature map in the 13th layer by half. The second input source for the feature fusion layer (Concat) is the P3 feature map after fusion and splicing in the FPN network, specifically the 15th layer in the feature pyramid network. This layer is used to splice the P3 feature map after fusion and splicing in the FPN network with the P2 feature map reduced in size by half. C2f fuses these spliced ​​feature maps to enhance the expressive power of the P3 feature map.

[0091] Step 205: Delete the second upsampling layer, the feature fusion layer, and the positioning classification layer between the third preset layer and the fourth preset layer in the path aggregation network to obtain a modified path aggregation network.

[0092] In the embodiment of the present application, the third preset layer is layer 10, and the fourth preset layer is layer 12. For example, the positioning classification layer may be a C2f module.

[0093] Step 206: Replace the coarse-fine module in the backbone network with a reparameterized visual geometry group model.

[0094] In some embodiments, the re-parameterized visual geometry group model includes: an initial module (Stem Block), three re-parameterized modules (Re-parameterized Block), two residual modules (Residual Block), an average pooling module (Avg Pool), a connection layer (Fully Connected, FC) and an output layer (Soft max); wherein the re-parameterized module includes: a group convolution (Group Conv), a composite activation function module (BatchNorm+SiLU), a shape adjustment module (Adjust Shape) and a fused convolution module (Concat+Conv).

[0095] It should be noted that the residual module is an important structure in deep learning. It solves the problems of vanishing and exploding gradients in deep neural network training by introducing skip connections or identity mapping. The core idea of ​​the residual module is to allow the network to learn the residual between the input and output, that is, the difference, rather than directly learning the mapping relationship. The residual module is typically composed of two or three convolutional layers, including a batch normalization layer (BN) and an activation function. In the residual module, the input is not only processed by these convolutional layers but also passed directly to the output via skip connections.

[0096] For example, see Figure 5 , the network structure of RepVGG is as follows Figure 5 As shown, the training and inference phases are divided into different structures. The training architecture consists of a main branch with a 3×3 convolution kernel, a 1×1 convolution branch, and an identity branch containing only batch normalization. During inference, the 3×3 convolution operations of the main branch, the 1×1 convolution branches, and the identity branch are merged into a unified convolution operation to improve inference speed. During training, RepVGG first performs group convolution on an input feature map of size C×H×W. The 3×3 convolutions in the main branch extract spatial features, generating a C′×H′×W′ feature map. The 1×1 convolution branch adjusts the number of channels, outputting a C′′×H′′×W′ feature map. The identity branch performs a residual connection, directly passing the C×H×W feature map to the input. Batch normalization is performed after the output of each branch. The Adjust Shape module uses the concatenation operation to combine the output feature maps of the three branches into a new feature map. Finally, it performs a nonlinear transformation using an activation function (SiLU) to produce the final feature map. During inference, the 3×3 convolution main branch, 1×1 convolution branch, and identity branch used during training are combined into a unified convolution operation. Multiple Batch Normalization layers are also integrated, eliminating the need for multiple branch calculations and improving inference speed.

[0097] Step 207: Delete the convolutional layer located at the fifth preset layer and the coarse-fine module located at the sixth preset layer in the backbone network to obtain a modified backbone network.

[0098] In the embodiment of the present application, the fifth preset layer is the 5th layer, and the sixth preset layer is the 6th layer.

[0099] In some embodiments, step 207 may include: deleting the convolution layer located at the fifth preset layer and the C2f module located at the sixth preset layer in the backbone network, and deleting the corresponding Conv convolution layer.

[0100] It should be noted that due to the removal of the convolutional layer at the fifth preset layer and the coarse-fine module at the sixth preset layer in the backbone network, the feature extraction and processing tasks originally performed by these two layers will no longer be performed, resulting in changes to the feature maps output by the backbone network. Because higher-layer feature maps contain richer semantic information, they reduce computational effort and model complexity, increase model inference speed, and significantly enhance the model's real-time processing capabilities. Furthermore, the simplified model structure is simpler, easier to understand, and deploy, but this may reduce the model's expressiveness and generalization capabilities.

[0101] Step 208: Obtain a weed seed detection model based on the modified feature pyramid network, the modified path aggregation network, and the modified backbone network, and train the weed seed detection model using a training set in the image dataset.

[0102] The method of this step has been described in the aforementioned step 105 and will not be repeated here.

[0103] For example, see Figure 6 , the network structure of the weed seed detection model is as follows Figure 6 See Figure 6 ,The weed seed detection model includes a modified feature pyramid network, a ,modified path aggregation network, a modified backbone network and a head network, ,wherein a P2 feature layer enhancement module is added to the ,feature pyramid network, e.g. Figure 3 In the 15th and 16th layers, the P3 feature map is expanded to the size of the P2 feature map module through upsampling through the first path; the multi-scale feature interaction method is optimized in the path aggregation network, such as Figure 4 In the path aggregation network, the P2 feature map is downsampled to the P3 feature map module; the redundant downsampling layers of the backbone network are trimmed and the backbone network is replaced with the RepVGG reparameterized structure, for example Figure 5 This improves detection accuracy while reducing the model size. The detection frame rate on edge devices reaches 36.18 fps. Compared with the traditional YOLOv8n model, the recall rate is improved by 16.99% and the model size is reduced by 68.7%.

[0104] In some embodiments, after step 208, the training method of the weed seed detection model may further include:

[0105] Sub-step 209: using the validation set in the image dataset, evaluating the mean average precision, detection speed, number of parameters, and model size of the weed seed detection model;

[0106] Sub-step 210: When the average precision, detection speed, number of parameters, and model size all meet preset conditions, the result of this iteration is used as the trained weed seed detection model.

[0107] In the embodiment of the present application, mean average precision (mAP) is an indicator that comprehensively evaluates the accuracy and recall rate of the model, and is closely related to precision and recall rate.

[0108] Optionally, precision represents the ratio of the number of samples correctly predicted as positive (True Positive, TP) to the number of samples predicted as positive (TP + False Positive, FP). Recall represents the ratio of the number of samples correctly predicted as positive (TP) to the number of samples actually positive (TP + False Negative, FN). For example, the formula for calculating precision is as follows:

[0109]

[0110] The recall calculation formula is as follows:

[0111]

[0112] In some embodiments, mean average precision (mAP) is one of the key metrics for measuring the performance of object detection models. When calculating mAP, the precision of all categories is averaged. AP@0.5 represents the average precision for each category when the intersection-over-union (IoU) threshold is 0.5. mAP@0.5 averages the AP values ​​of all categories to reflect the model's ability to maintain high precision at different recall rates. A higher mAP@0.5 value indicates that the model is able to maintain good precision at high recall rates. The mAP calculation formula is as follows:

[0113]

[0114] In the above formula, N represents the number of categories, and APi@0.5 represents the average precision value of the i-th category.

[0115] In the embodiments of the present application, the parameter quantity can be expressed in Giga Floating-point Operations Per Second (GFLOPs). GFLOPs can measure the ability of a central processing unit (CPU) or graphics processing unit (GPU) to process graphics and deep learning models, and conversely, can be used to represent the CPU or GPU computing power resources required for a neural network model.

[0116] In the embodiment of the present application, the unit of model size is megabytes (MB).

[0117] In the embodiments of this application, the detection frame rate (FPS) is used to measure the speed at which the model processes images (or video frames), that is, how many frames of images can be processed per second. It is a measure of inference efficiency and reflects the performance of the model on a given hardware. FPS is short for frame rate, and a high FPS means smoother movement and a better visual experience. The unit of FPS is Hertz (Hz). The formula for calculating FPS is as follows:

[0118]

[0119] In the above formula, the unit of average inference time is milliseconds.

[0120] In some embodiments, in order to understand the gain effect of each module on the target detection model, multiple groups of ablation experiments are set up. The ablation experiments are shown in Table 1 and are specifically shown below.

[0121] Table 1

[0122]

[0123] First, the ablation experiment data shown in Table 1 shows that the introduction of the Feature Pyramid Network-Path Aggregation Network (FAN-PAN) optimization significantly improves the precision and recall of the object detection model, reaching 96.2% and 92.2%, respectively. This optimization significantly improves the accuracy of the weed seed detection model, especially when dealing with complex backgrounds and small objects.

[0124] Secondly, from the ablation experiment data shown in Table 1, it can be seen that the average precision is improved by 17.44%, indicating that after optimization, the accuracy of the weed seed model in the embodiment of the present application is much improved compared with the traditional target detection model.

[0125] Thirdly, backbone network optimization and RepVGG optimization successfully improved inference speed by reducing computational effort and optimizing the backbone network, respectively. This resulted in the final weed seed model maintaining a high FPS of 36.18, which was not significantly lower than the initial model's FPS (38.33).

[0126] Finally, the size of the weed seed model was further reduced to 1.86MB, but the number of parameters did not increase significantly, only increasing from the initial 8.1GFLOPs to 9.3GFLOPs. In other words, while the weed seed model significantly reduced its size, the demand for hardware computing power did not increase significantly. Therefore, it provides a more efficient deployment solution for memory-constrained edge devices, while maintaining a high level of accuracy.

[0127] In summary, the weed seed model provided by the embodiment of this application achieved a 16.99% increase in recall and a 19.10% increase in average precision compared to the initial model. The model size was reduced by 68.69%, while the detection frame rate only decreased by 5.61%. This change demonstrates that the optimization method employed significantly improved detection performance with only a small loss in detection speed, particularly for small target detection. Furthermore, the memory usage of the weed seed model was significantly reduced, making it more suitable for efficient real-time detection on memory-constrained edge devices.

[0128] Figure 7 This is an application method of a weed seed detection model provided in this embodiment, referring to Figure 7 , the method may include the following steps.

[0129] Step 301: Obtain a seed image to be identified.

[0130] Step 302: Input the seed image to be identified into the trained weed seed detection model to obtain an output weed seed identification result.

[0131] In the embodiment of the present application, the weed seed detection model is trained by the weed seed detection model training method described in the above embodiment.

[0132] In summary, in the embodiments of the present application, the trained weed seed detection model improves the detection accuracy of small targets such as weed seeds, which is conducive to real-time recognition of seed images.

[0133] Figure 8 4 is a structural diagram of a weed seed detection model training device provided in an embodiment of the present application. The weed seed detection model training device 400 may include the following modules.

[0134] A first acquisition module 401 is used to acquire an image dataset of weed seeds;

[0135] A first network improvement module 402 is configured to add a first path to a feature pyramid network of a preset object detection model and delete a first sampling module to obtain a modified feature pyramid network; the first path is configured to upsample the first feature map to the size of the second feature map and then fuse the first feature map, where the size of the first feature map is smaller than the size of the second feature map; the first sampling module is configured to upsample the third feature map to the size of the fourth feature map and then fuse the third feature map, where the size of the third feature map is smaller than the size of the fourth feature map;

[0136] The second network improvement module 403 is used to add a module for downsampling the second feature map to the first feature map in the path aggregation network of the target detection model and delete the second sampling module to obtain a modified path aggregation network; the second sampling module is used to downsample the fourth feature map to the third feature map;

[0137] A third network improvement module 404 is configured to replace the coarse-fine module in the backbone network of the object detection model with a reparameterized visual geometry group model and discard the third sampling module to obtain a modified backbone network;

[0138] The model training module 405 is used to obtain a weed seed detection model based on the modified feature pyramid network, the modified path aggregation network and the modified backbone network, and train the weed seed detection model using a training set in the image dataset.

[0139] Optionally, the first network improvement module 402 includes:

[0140] An upsampling module, configured to expand the size of a first feature map to the size of a second feature map by upsampling between a first preset layer and a second preset layer in a feature pyramid network;

[0141] The first deletion module is used to delete the first upsampling layer, the feature fusion layer, and the positioning classification layer between the third preset layer and the fourth preset layer in the feature pyramid network to obtain a modified feature pyramid network.

[0142] Optionally, the second network improvement module 403 includes:

[0143] a downsampling module, configured to reduce the size of the second feature map to the size of the first feature map by downsampling in the path aggregation network;

[0144] The second deletion module is used to delete the second upsampling layer, the feature fusion layer and the positioning classification layer between the third preset layer and the fourth preset layer in the path aggregation network to obtain a modified path aggregation network.

[0145] Optionally, the third network improvement module 404 includes:

[0146] A network replacement module is used to replace the coarse-fine module in the backbone network with a reparameterized visual geometry group model;

[0147] The third deletion module is used to delete the convolutional layer located at the fifth preset layer and the coarse-fine module located at the sixth preset layer in the backbone network to obtain a modified backbone network.

[0148] Optionally, the reparameterized visual geometry group model includes: an initial module, three reparameterized modules, two residual blocks, an average pooling module, a connection layer and an output layer; wherein the reparameterized module includes: a grouped convolution, a composite activation function module, a shape adjustment module and a fused convolution module.

[0149] Optionally, the training of the above grass seed detection model also includes the following modules.

[0150] The model evaluation module is used to evaluate the mean average precision, detection speed, number of parameters, and model size of the weed seed detection model using the validation set in the image dataset;

[0151] The model iteration module is used to use the result of this iteration as the trained weed seed detection model when the average accuracy, detection speed, parameter quantity and model size all meet the preset conditions.

[0152] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0153] Figure 9 1 is a structural diagram of an application device of a weed seed detection model provided in an embodiment of the present application. The application device 500 of the weed seed detection model may include the following modules.

[0154] The second acquisition module 501 is used to acquire a seed image to be identified.

[0155] The model application module 502 is used to input the seed image to be identified into the trained weed seed detection model to obtain an output weed seed identification result. The weed seed detection model is trained by the above-mentioned weed seed detection model training method.

[0156] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0157] Figure 10 FIG. 6 is a block diagram of an electronic device 600 according to another embodiment of the present invention. For example, the electronic device 600 may be provided as a server. Figure 6 The electronic device 600 includes a processing component 622, which further includes one or more processors, and a memory resource represented by a memory 632 for storing instructions executable by the processing component 622, such as an application. The application stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 622 is configured to execute the instructions to perform a method for generating a text archive provided in an embodiment of the present application.

[0158] The electronic device 600 may further include a power supply component 626 configured to perform power management of the electronic device 600, a wired or wireless network interface 650 configured to connect the electronic device 600 to a network, and an input / output (I / O) interface 656. The electronic device 600 may operate based on an operating system stored in the memory 632, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or the like.

[0159] In an embodiment of the present application, memory 632 can be used to store software programs and various data. Memory 632 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function or an image playback function), and the like. Furthermore, memory 632 may include volatile memory or non-volatile memory, or both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 632 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0160] The present application also provides a readable storage medium having a program or instruction stored thereon. When executed by a processor, the program or instruction implements each process of the above-mentioned embodiment of the method for training a weed seed detection model and can achieve the same technical effect. To avoid repetition, the details are not described here.

[0161] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0162] The present embodiment provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-mentioned training method for a weed seed detection model, and can achieve the same technical effects. To avoid repetition, they are not described here.

[0163] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0164] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of this application.

[0165] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A training method for a weed seed detection model, characterized in that: The method comprises: Get a dataset of weed seed images; A first path is added to a feature pyramid network of a preset object detection model, and a first sampling module is deleted to obtain a modified feature pyramid network; the first path is used to upsample the first feature map to the size of the second feature map and then fuse it, and the size of the first feature map is smaller than the size of the second feature map; the first sampling module is used to upsample the third feature map to the size of the fourth feature map and then fuse it, and the size of the third feature map is smaller than the size of the fourth feature map; Adding a second path to the path aggregation network of the target detection model and deleting the second sampling module to obtain a modified path aggregation network; the second path is used to downsample the second feature map to the size of the first feature map and then fuse it; the second sampling module is used to downsample the fourth feature map to the size of the third feature map and fuse it; Replacing the coarse-fine module in the backbone network of the object detection model with a reparameterized visual geometry group model; deleting the convolutional layer at the fifth preset layer and the coarse-fine module at the sixth preset layer in the backbone network to obtain a modified backbone network; the reparameterized visual geometry group model includes: an initial module, three reparameterized modules, two residual blocks, an average pooling module, a connection layer, and an output layer; A weed seed detection model is obtained according to the modified feature pyramid network, the modified path aggregation network and the modified backbone network, and the weed seed detection model is trained using a training set in the image dataset.

2. The method according to claim 1, characterized in that The step of adding a first path to a feature pyramid network of a preset object detection model and deleting a first sampling module to obtain a modified feature pyramid network includes: Upsampling is performed between a first preset layer and a second preset layer in the feature pyramid network through the first path to expand the size of the first feature map to the size of the second feature map; The first upsampling layer, the feature fusion layer, and the positioning classification layer between the third preset layer and the fourth preset layer in the feature pyramid network are deleted to obtain a modified feature pyramid network.

3. The method according to claim 1, characterized in that The step of adding a second path to the path aggregation network of the target detection model and deleting the second sampling module to obtain a modified path aggregation network includes: Downsampling the second path in the path aggregation network to reduce the size of the second feature map to the size of the first feature map; The second upsampling layer, the feature fusion layer, and the positioning classification layer between the third preset layer and the fourth preset layer in the path aggregation network are deleted to obtain a modified path aggregation network.

4. The method according to claim 3, characterized in that The reparameterization module includes: a grouped convolution, a composite activation function module, a shape adjustment module and a fused convolution module.

5. The method according to claim 1, wherein The method further comprises: Using the validation set in the image dataset, the mean average precision, detection speed, number of parameters, and model size of the weed seed detection model were evaluated; When the average precision mean, the detection speed, the number of parameters, and the model size all meet preset conditions, the result of this iteration is used as the trained weed seed detection model.

6. An application method of a weed seed detection model, characterized in that: The method comprises: Obtaining a seed image to be identified; The seed image to be identified is input into a trained weed seed detection model to obtain an output weed seed identification result, wherein the weed seed detection model is trained by the weed seed detection model training method according to any one of claims 1 to 5.

7. A training device for a weed seed detection model, characterized in that: The device comprises: A first acquisition module is used to acquire an image dataset of weed seeds; A first network improvement module is configured to add a first path to a feature pyramid network of a preset object detection model and delete a first sampling module to obtain a modified feature pyramid network; the first path is configured to upsample a first feature map to the size of a second feature map and then fuse the first feature map, where the size of the first feature map is smaller than the size of the second feature map; the first sampling module is configured to upsample a third feature map to the size of a fourth feature map and then fuse the third feature map, where the size of the third feature map is smaller than the size of the fourth feature map; a second network improvement module, configured to add a second path to the path aggregation network of the target detection model and delete the second sampling module to obtain a modified path aggregation network; the second path is configured to downsample the second feature map to the size of the first feature map and then fuse it; the second sampling module is configured to downsample the fourth feature map to the size of the third feature map and then fuse it; A third network improvement module is configured to replace the coarse-fine module in the backbone network of the object detection model with a reparameterized visual geometry group model; delete the convolutional layer located at the fifth preset layer and the coarse-fine module located at the sixth preset layer in the backbone network to obtain a modified backbone network; the reparameterized visual geometry group model includes: an initial module, three reparameterized modules, two residual blocks, an average pooling module, a connection layer, and an output layer; A model training module is used to obtain a weed seed detection model based on the modified feature pyramid network, the modified path aggregation network and the modified backbone network, and train the weed seed detection model using a training set in the image dataset.

8. An application device of a weed seed detection model, characterized in that: The device comprises: A second acquisition module is used to acquire a seed image to be identified; A model application module is used to input the seed image to be identified into a trained weed seed detection model to obtain an output weed seed identification result, wherein the weed seed detection model is trained by the weed seed detection model training method according to any one of claims 1 to 5.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 5 or claim 6 is implemented.

Images