Disease and pest detection method based on improved yolov9 model and electronic equipment

Abstract

The invention relates to a disease and insect pest detection method based on an improved yolov9 model and electronic equipment. The method comprises the following steps: acquiring an image data set of a to-be-detected disease and insect pest infected crop; constructing a target detection network based on an improved yolov9 model, wherein the improved yolov9 model comprises a dual-auxiliary reversible branch; and for the trained target detection network, only retaining the backbone network to obtain a model for inferring disease and pest positions, and outputting a target detection result of the to-be-detected image data set. Compared with the prior art, the method has the advantages of high detection accuracy, no need of additional reasoning cost and the like.

Description

Technical Field

[0001] The invention relates to the technical field of target detection, and in particular to a pest detection method and electronic equipment based on an improved YOLOv9 model. Background Art

[0002] Models based on deep learning have demonstrated performance far exceeding that of previous artificial intelligence systems in multiple fields such as computer vision, language processing, and speech recognition. In the agricultural field, by analyzing farmland images taken by drones, target detection technology is used to identify and locate crops infected with pests and diseases, thereby guiding precise pesticide application, reducing pesticide use, and improving crop yields and quality.

[0003] In recent years, researchers in the field of deep learning have mainly focused on how to develop more powerful system architectures and learning methods, such as convolutional neural networks (CNNs), Transformers, Perceivers, and Mambas. In addition, some researchers have tried to develop more general objective functions, such as loss functions, label assignments, and auxiliary supervision. The above studies are all trying to accurately find the mapping between input and target tasks. However, most methods ignore the fact that there may be non-negligible information loss in the input data during the feedforward process. This information loss may lead to deviations in the gradient flow, which is then used to update the model. The above problems may cause the deep network to establish incorrect associations between the target and the input, causing the trained model to produce incorrect predictions.

[0004] In deep networks, the phenomenon of input data losing information during the feedforward process is often referred to as an information bottleneck. Using a reversible architecture can directly avoid this phenomenon, and is also the main mainstream method to solve the information bottleneck today: this method mainly uses repeated input data to maintain the information of the input data in an explicit way; however, the reversible architecture requires additional layers to combine repeated input data, which will significantly increase the cost of inference. The introduction of the concept of deep supervision can also alleviate this phenomenon: adding supervisory signals at different depth levels of the network rather than just at the last layer to improve the network's ability to learn features and enhance the generalization of the model.

[0005] Yolov9 proposed PGI (Programmable Gradient Information), which generates reliable gradients through auxiliary reversible branches, so that deep features can still maintain the key characteristics of performing target tasks. The design of auxiliary reversible branches can avoid the semantic loss that may be caused by integrating multi-path features in the traditional deep supervision process. Although the use of reversible architecture on auxiliary branches has achieved excellent results, PGI has not fully utilized the advantage of auxiliary branches that do not need to participate in the reasoning stage, making the reasoning cost of target detection relatively high.

[0006] After searching, Chinese invention patent publication number CN117115640A discloses a method, device and equipment for detecting pests and diseases based on improved YOLOv8. The method includes: obtaining an agricultural pest and disease image dataset; performing detection based on the agricultural pest and disease image dataset based on the trained Sim-YOLOv8 model to obtain target detection results. Among them, the Sim-YOLOv8 model is trained based on the agricultural pest and disease image training set, and the agricultural pest and disease image training set is a dataset after data screening, data expansion, data enhancement and data annotation. . This existing patent has problems with detection accuracy and low cost.

[0007] How to improve the accuracy and cost of target detection of crops infected with pests and diseases has become a technical problem that needs to be solved. Summary of the invention

[0008] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a pest detection method and electronic equipment based on an improved YOLOv9 model.

[0009] The purpose of the present invention can be achieved by the following technical solutions:

[0010] According to one aspect of the present invention, a method for detecting pests and diseases based on an improved yolov9 model is provided, the method comprising the following steps:

[0011] Step S1, obtaining an image dataset of crops infected with pests and diseases to be detected;

[0012] Step S2, constructing a target detection network based on an improved yolov9 model, wherein the improved yolov9 model includes a double auxiliary reversible branch;

[0013] Step S3, for the trained target detection network, only retain its backbone network to obtain a model for inferring the location of pests and diseases, and output the target detection result of the image data set in step S1.

[0014] Preferably, the dual auxiliary reversible branches are fused with the backbone network at the shallow and deep parts respectively.

[0015] Preferably, the dual auxiliary reversible branches include a GELAN-symmetry module, that is, a symmetrical GELAN module; the GELAN-symmetry module is an improvement on GELAN, and the two parts have the same structure.

[0016] More preferably, any block of the GELAN-symmetry module is a CSP block.

[0017] More preferably, the depth of the GELAN-symmetry is 2, and the depth of the CSP block is 1.

[0018] Preferably, the auxiliary reversible branch is integrated with the main network in stages, that is, in the initial stage of training, the main network and the two auxiliary branches are trained separately.

[0019] Preferably, the target detection network uses a Leaky-ReLU activation function.

[0020] Preferably, the image data set to be detected is input into the target detection network and trained through a main trunk and two auxiliary branches respectively;

[0021] The training process includes:

[0022] Step S3-1, initializing network weights;

[0023] Step S3-2, forward propagation and calculation of loss;

[0024] Step S3-3, back propagate and update weights according to gradients;

[0025] Step S3-4, repeat steps S3-2 to S3-3 according to the epoch number, train the network, and delete the auxiliary branch after the training is completed.

[0026] Preferably, the depth of the main network is the same as that of the auxiliary branches, and the depth of the modules at the splicing and fusion point is also the same.

[0027] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1) The present invention introduces dual auxiliary reversible branches. Compared with a single auxiliary branch, the improved dual auxiliary reversible branches have less information loss of input data in the feedforward process, so that the trained backbone network layer has better effect, further improving the accuracy of target detection, and is suitable for detecting subtle changes in pest and disease infection. The reversible architecture is used to maintain complete information and the reversible architecture is applied to the auxiliary branch (non-backbone network), so that the training model can obtain higher quality without increasing additional reasoning cost.

[0030] 2) The present invention proposes GELAN-symmetry, which makes further improvements on the basis of GELAN and is only applied to auxiliary branches, so that the auxiliary branches can better extract features, thereby improving the performance of the backbone network.

[0031] 3) The present invention integrates the auxiliary reversible branches with the main network in stages. In the initial stage of training, the main network and the two auxiliary branches are trained separately without performing the splicing and fusion stage. This staged fusion strategy not only improves the training efficiency and saves computing costs, but also enhances the accuracy and robustness of the final model to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the structure of the improved yolov9 model in the present invention;

[0033] Figure 2 It is a schematic diagram of the structure of the GELAN-symmetry module in the present invention;

[0034] Figure 3 It is a schematic diagram of the structure of the CSP block in the present invention;

[0035] Figure 4 Schematic diagram of the process of the present invention;

[0036] Figure 5 Schematic diagram of the process of training the target detection network in the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] This embodiment relates to a method for detecting pests and diseases based on an improved yolov9 model.

[0039] In the yolov9 model, there is an auxiliary branch that improves the training effect of the backbone network by applying a reversible architecture to provide better gradient information for individual modules in the backbone. However, a single auxiliary reversible branch directly affects only a limited number of modules in the network, which may lead to poor training effects in other layers, especially in deeper layers. Figure 1 In yolov9, the modules numbered 1-31 are on the trunk. In yolov9, the auxiliary branch is only directly associated with modules 5, 7, and 9, so that the modules after 10 in the trunk branch cannot quickly receive the guidance of the auxiliary reversible branch; and in addition to applying the reversible architecture, the auxiliary branch of yolov9 has not made much innovation in the auxiliary branch, so the advantage of the auxiliary branch not needing to participate in the calculation of the reasoning stage is not fully utilized.

[0040] In order to further improve the performance of the target detection model and enable the auxiliary reversible branch to provide more gradient flow information for model training, the present invention proposes a method to replace the auxiliary reversible branch of PGI of yolov9 with a double auxiliary reversible branch and apply the new GELAN-symmetry module in the double auxiliary reversible branch. Therefore, it is considered to add a double auxiliary reversible branch on the basis of the original architecture and improve the module on the auxiliary reversible branch.

[0041] like Figure 1 As shown, the present invention is based on the improvement of yolov9 structure as follows:

[0042] 1) Construct a double auxiliary reversible branch, such as Figure 1 The modules marked with numbers 1 to 31 are on the trunk. The input data are respectively from 1: convolution layer through the trunk branch; from 38: convolution layer through the first auxiliary reversible branch, and from 59: convolution layer through the second auxiliary reversible branch.

[0043] 2) The dual auxiliary reversible branches are fused with the backbone network in the shallow and deep parts respectively: the first auxiliary reversible auxiliary branch is directly connected and fused with the modules numbered 5, 7, and 9 on the main branch; the second auxiliary reversible branch is directly connected and fused with the modules numbered 16, 19, and 22 on the main branch.

[0044] 3) Applying the GELAN-symmetry (symmetric generalized efficient layer aggregation network) proposed in the present invention to the dual auxiliary reversible branches. The structure of the GELAN-symmetry module is as follows: Figure 2 shown. Figure 2 In any block, CSP blocks are used. Through ablation experiments, it is found that when the depth of GELAN-symmetry is 2 (the number of blocks is 2) and the depth of CSP blocks is set to 1, the trained model has the highest accuracy.

[0045] If the CSP block is replaced or the depth of GELAN-symmetry is adjusted, the layers or modules at the same depth of the main branch should also be adjusted accordingly, so that the depth (total number of layers) of the auxiliary branch is the same as that of the main network, and the depth of the modules at the splicing and fusion point is the same, to avoid errors in the transferred gradient flow due to the different depths of the auxiliary branch and the main branch.

[0046] 4) The auxiliary reversible branches are integrated with the main network in stages. In the initial stage of training (the first 10 epochs), the main network and the two auxiliary branches are trained separately without splicing and fusion, that is, the fusion phase is ignored or skipped. Figure 1 CBLinear and CBFuse in (see Figure 132, 33, 34, 42, 35, etc.) to ensure that in the early stage of training, each can effectively learn and adapt to specific features while saving computing costs.

[0047] In order to further improve the performance of the target detection model and enable the auxiliary reversible branch to provide more gradient flow information for model training, the present invention proposes a method to replace the auxiliary reversible branch of PGI of yolov9 with a double auxiliary reversible branch and apply a new GELAN-symmetry module in the double auxiliary reversible branch.

[0048] like Figure 4 , the method comprises the following steps:

[0049] Step S1, obtaining an image dataset of crops infected by pests and diseases;

[0050] Step S2, constructing a yolov9 object detection network with dual auxiliary reversible branches using GELAN-symmetry;

[0051] Step S3, using the acquired data set to train and test the target detection network;

[0052] Step S4, remove the dual auxiliary reversible branch part in the network, retain only the trunk part for target detection of the image to be detected, input the image data set of step S1 into the target detection network, and output the detection result.

[0053] Further:

[0054] Step S1 includes: obtaining a public image dataset with a required type from the Internet, and dividing the dataset into a training set and a test set.

[0055] Step S2 includes: constructing a target detection network based on the improved yolov9, replacing the auxiliary reversible branch part of the programmable gradient information (PGI) with a double auxiliary reversible branch, and replacing the GELAN module with a GELAN-symmetry module in the double auxiliary reversible branch.

[0056] Step S3 includes: using the acquired data set to train the target detection network constructed in step 2, and in the test phase (inference), only using the backbone part of the target detection network to infer the input.

[0057] Step S4 includes: deleting the network trained in step 3, retaining only the trained trunk part and completely deleting the dual auxiliary reversible branches, so as to obtain a network that can be put into use for target detection of pests and diseases.

[0058] like Figure 5 , the target detection network training process includes:

[0059] S3-1, initialize network weights;

[0060] S3-2, forward propagation and calculation of loss;

[0061] S3-3, back propagate and update weights according to the gradient;

[0062] S3-4, repeat S3-2 to S3-3 according to the number of epochs, train the network, and delete the auxiliary branch after the training is completed.

[0063] In S3-1, the bias items of each layer are initialized to 0; Xavier initialization is used for the convolutional layers in the network, and the weights are initialized from the Gaussian distribution N(0,σ 2 ), where the standard deviation σ is determined by the following formula:

[0064]

[0065] Here in is the number of input features.

[0066] For the BN (batch normalization) layer, the scaling factors are all initialized to 1. Leaky-ReLU is used in the network to replace the conventional ReLU activation function, and the negative slope of Leaky-ReLU is set to a=0.01.

[0067] In S3-2, as shown in the attached Figure 1 As shown in the figure, the input data passes through a main trunk and two auxiliary branches. For any branch, it first passes through two convolutional layers with a convolution kernel size of 3×3 and a step size of 2. Each convolutional layer used in the network is followed by a batch normalization layer and a Leaky-ReLU activation function (not marked in the figure).

[0068] After the GELAN-symmetry module (RepNCSPELAN module in the main trunk), as shown in the attached Figure 2 As shown in the figure, the corresponding anyblock defaults to a CSP block with a depth of 1. The process of CSP block is shown in the attached Figure 3 As shown in the figure, the input is evenly divided into two parts according to the channel dimension, and the second part is converted by the convolution layer of the dense connection (Dense Block) and then concatenated with the first part in the channel dimension. The calculation formula corresponding to the CSP block is as follows:

[0069] F out =Conv(concat(P 1 ,P′ 2 ))

[0070] P 1 Characteristic map of the first part of the crop directly infected by pests and diseases, P 2 ′ This is the feature map of the second part of the crops infected by pests and diseases after the convolution operation. The two are concatenated and fused through a standard convolution.

[0071] Then we introduce a deep supervision mechanism into the layers in the backbone (corresponding to the attached Figure 1 5, 7, 9, etc.), feature fusion must be performed first. The method is to first perform a 1×1 convolution operation on the feature map to make the dimension of the feature map of the middle layer of the trunk consistent with the dimension of the feature map of the layer of the auxiliary branch, and then use the element-by-element addition method to perform fusion. The fusion formula is as follows:

[0072] F fuse =Conv 1×1 (M 1 )+M 2

[0073] M 1 is the output feature map of the layer that introduces deep supervision in the backbone, M 2 is the output of the layer of the auxiliary branch to be fused.

[0074] Continue forward propagation and finally reach the prediction head to calculate the loss function. The loss function of each prediction head is as follows:

[0075]

[0076] L CIoU is the bounding box regression loss, IoU(B,B * ) is the intersection-over-union ratio of the bounding box, ρ 2 (b,b * ) is the Euclidean distance between the center points of the predicted bounding box and the true bounding box, a is the diagonal length of the minimum closure area that can enclose the predicted bounding box and the true bounding box, α is a weight coefficient, and v is a measure of the consistency of the aspect ratio of the bounding box.

[0077]

[0078] Among them, L C is the classification loss, assuming there are I categories, the predicted category probability is The true label is p i .

[0079]

[0080] Among them, L cf is the confidence loss, assuming the predicted confidence is The true confidence is c (1 if the target exists, 0 if it does not exist).

[0081] In summary, the total loss for each prediction head is:

[0082] L=λL CIoU +βL C +γL cf

[0083] λ, β, γ are the weights of the loss of each part, and the final loss of the entire network is L total It is the weighted sum of each prediction head.

[0084] In S3-3, starting from the output layer (each prediction head), the gradient of each layer is calculated. The parameters of each layer of the network are adjusted using the Adam optimizer based on the calculated gradient. Among them, for the layers that apply the deep supervision mechanism, gradient merging is required. The specific formula is as follows:

[0085]

[0086] in, is the gradient of the backbone loss function with respect to the parameters of this layer, is the gradient of the loss function on the reversible auxiliary branch with respect to the parameters of this layer, is the gradient after merging, and a and b are weight hyperparameters.

[0087] In S3-2, steps S2 and S3 are repeated according to the set epoch number. Due to the characteristics of auxiliary reversible branches, the model is generally trained according to the training strategy from scratch, so a larger number of epochs is required. For example, for the MS COCO2017 dataset, the default total number of training rounds is 500 epochs. After the training is completed, the backbone structure and the parameter information of each layer above it are saved, corresponding to the attached Figure 1 1-31 of the , and delete the double auxiliary reversible branch (including structural information and parameter values), corresponding to the attached Figure 1 38-58 and 59-92 of , thus obtaining the final model for reasoning.

[0088] Step S4 includes: deleting the network trained in step S3, retaining the trained backbone part and completely deleting the dual auxiliary reversible branches, obtaining a network that can be put into use, inputting the image to be detected into the trained backbone network, and outputting the target detection result.

[0089] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0090] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0091] The processing unit performs the various methods and processes described above, such as methods S1 to S4. For example, in some embodiments, methods S1 to S4 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S4 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S4 in any other appropriate manner (e.g., by means of firmware).

[0092] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0093] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0094] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0095] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting pests and diseases based on an improved yolov9 model, characterized in that: The method comprises the following steps: Step S1, obtaining an image dataset of crops infected with pests and diseases to be detected; Step S2, constructing a target detection network based on an improved yolov9 model, wherein the improved yolov9 model includes a double auxiliary reversible branch; Step S3, for the trained target detection network, only retain its backbone network to obtain a model for inferring the location of pests and diseases, and output the target detection result of the image data set in step S1.

2. A method for detecting pests and diseases based on an improved yolov9 model according to claim 1, characterized in that: The dual auxiliary reversible branches are fused with the backbone network at the shallow and deep parts respectively.

3. A method for detecting pests and diseases based on an improved yolov9 model according to claim 1, characterized in that: The dual auxiliary reversible branch comprises a GELAN-symmetry module, that is, a symmetrical GELAN module; the GELAN-symmetry module is an improvement on GELAN, and the two parts have the same structure.

4. A method for detecting pests and diseases based on an improved yolov9 model according to claim 3, characterized in that: Any block of the GELAN-symmetry module is a CSP block.

5. A method for detecting pests and diseases based on an improved yolov9 model according to claim 4, characterized in that: The depth of the GELAN-symmetry is 2, and the depth of the CSP block is 1.

6. A method for detecting pests and diseases based on an improved yolov9 model according to claim 1, characterized in that: The auxiliary reversible branch is integrated with the main network in stages, that is, in the initial stage of training, the main network and the two auxiliary branches are trained separately.

7. A method for detecting pests and diseases based on an improved yolov9 model according to claim 1, characterized in that: The object detection network described uses the Leaky-ReLU activation function.

8. A method for detecting pests and diseases based on an improved yolov9 model according to claim 1, characterized in that: The image data set to be detected is input into the target detection network, and is trained through a main trunk and two auxiliary branches respectively; The training process includes: Step S3-1, initializing network weights; Step S3-2, forward propagation and calculation of loss; Step S3-3, back propagate and update weights according to gradients; Step S3-4, repeat steps S3-2 to S3-3 according to the epoch number, train the network, and delete the auxiliary branch after the training is completed.

9. A method for detecting pests and diseases based on an improved yolov9 model according to claim 1, characterized in that: The depth of the main network is the same as that of the auxiliary branches, and the depth of the modules at the splicing and fusion point is also the same.

10. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 9 is implemented.

Images