Pest identification and counting method, system, device and readable storage medium

Abstract

The present invention discloses a pest identification and counting method, including obtaining pictures of various pest types to form an original data set, calibrating the original data set; performing a first process on the calibrated original data set, training and verifying a target detection model based on the first process result; selecting pest pictures of similar types for a second process to train and verify a classification model; using a target detection model to perform object detection on the pest pictures to be detected to obtain pest detection results; performing a segmentation process on the pest pictures to be detected based on location information of similar categories, and using a classification model to classify the pest types on the segmentation results to obtain pest classification results; based on the location information, preliminary classification results and pest classification results, the types and quantity of pests are obtained. The present invention uses AI image recognition technology to complete the image acquisition and counting functions of trapping insects remotely, monitors target insects in real time, and reduces the inconvenience and inaccuracy of manual segmentation and re-verification of data.

Description

Technical Field

[0001] The present invention relates to the field of identification technology, and in particular to a pest identification and counting method, system, device and readable storage medium. Background Art

[0002] Insect pheromones (sex attractants) have the advantages of strong specificity, no drug resistance, environmental friendliness, 100% compatibility with other control technologies, and significant improvement in the quality of agricultural products. Sex attractants have become one of the green control technologies advocated by the country. "Intelligent Insect Sex Attractant Detection and Reporting" uses sex attractants to combine traditional pest sex traps with Internet of Things information technology to monitor and biologically control insects that may cause harm in agricultural and forestry production. It can guide scientific chemical control in the field, reduce the use of pesticides, thereby reducing production costs and greatly improving the quality of agricultural products, increasing the value of agricultural products, and responding to the plant protection policy of prevention first and comprehensive control in pest control.

[0003] In the prior art, traditional traps are simple and crude, they only trap but do not collect data, and cannot be monitored, and cannot achieve the effect of guiding prevention and control. Existing sexual lure detection and reporting system products on the market are bulky, difficult to move, and expensive. There is a widespread phenomenon of false counts. Ordinary users have demand but no purchasing power. At present, big data research is relatively extensive. Can big data and traditional traps be combined to develop low-cost intelligent sexual lure detection and reporting system products? This application solves this problem. Summary of the invention

[0004] The present invention aims at the shortcomings of the prior art and provides a pest identification and counting method, system, device and readable storage medium.

[0005] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0006] A pest identification and counting method comprises the following steps:

[0007] Obtain pictures of various pest types to form an original data set, and perform calibration on the original data set;

[0008] Performing a first processing on the calibrated original data set, and training and verifying the target detection model based on the first processing result; performing a second processing on insect pest pictures of similar types selected from the calibrated original data set, and training and verifying the classification model based on the second processing result;

[0009] The target detection model is used to perform object detection on the pest image to obtain the pest detection result, wherein the detection result includes the location information and score result of each pest in the image to be detected, and the preliminary category and similar category are obtained based on the location information and score result;

[0010] The images of the pests to be detected are segmented based on the position information of similar categories, and the classification model is used to classify the pests according to the segmentation results to obtain the pest classification results;

[0011] Based on the location information, the preliminary classification result and the pest classification result, the types and quantities of the pests are obtained.

[0012] As an implementable method, the first processing is performed on the original data set, and the target detection model is trained and verified based on the first processing result, specifically:

[0013] Perform sample enhancement processing on the calibrated original data set and divide it into a training data set, a test data set and a verification data set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing;

[0014] The training data set is input into the target detection model for training. After the training is completed, the test data set and the validation data set are used as input to verify the training results, thereby obtaining the target detection model.

[0015] As an implementable method, the insect pest pictures of similar types are selected from the calibrated original data set for second processing, and the classification model is trained and verified based on the second processing result, specifically:

[0016] Based on the similarity of pest categories, the calibrated original data set is selected, and similar pest category data sets are selected and cut;

[0017] Based on the cutting processing result, sample enhancement processing is performed and divided into a classification model training data set, a classification model test data set and a classification model verification set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing;

[0018] The classification model training data set is input into the classification model for training. After the training is completed, the classification model test data set and the classification model verification data set are used as input to verify the training results and then obtain the classification model.

[0019] As an implementation method, before inputting the training data set into the target detection model for training, the following steps are also included:

[0020] The anchor box parameter values ​​of the target detection model are modified based on the k-means clustering algorithm.

[0021] As an implementation method, before inputting the classification model training data set into the classification model for training, the step further includes:

[0022] The loss function in the classification model is calculated based on the forward network propagation algorithm, and the parameters of the classification model are updated through the back propagation algorithm to obtain the revised classification model.

[0023] As an implementation method, the following steps are also included: perfecting the image of the pest to be detected, specifically:

[0024] The images in the original dataset are classified according to the body structure of the pests, and each one is labeled based on the structural features of each type of pest;

[0025] If the pest image in the to-be-detected pest image is an incomplete image, then the structural features of the pest image in the image are extracted, and the extracted features are compared with each type of structural features marked in the original data set to obtain a comparison result, wherein at least three groups of comparison results are recommended;

[0026] Based on the comparison result, the remaining part of the incomplete pest image is supplemented to obtain a complete pest image to be detected.

[0027] A pest identification and counting system includes a data acquisition module, a processing training module, a result detection module, a processing classification module and a comprehensive result module;

[0028] The data acquisition module is used to acquire pictures of various pest types to form an original data set, and perform calibration processing on the original data set;

[0029] The processing and training module is used to perform a first processing on the calibrated original data set, and train and verify the target detection model based on the first processing result; perform a second processing on insect pest pictures of similar types selected from the calibrated original data set, and train and verify the classification model based on the second processing result;

[0030] The result detection module is configured to: use the target detection model to perform object detection on the pest image to be detected to obtain the pest detection result, wherein the detection result includes the position information and score result of each pest in the image to be detected, and obtain the preliminary category and similar category based on the position information and the score result;

[0031] The processing and classification module is used to perform segmentation processing on the image of the pest to be detected based on the position information of similar categories, and classify the pest types based on the segmentation processing results using a classification model to obtain a pest classification result;

[0032] The comprehensive result module is used to obtain the types and quantities of pests based on the location information, the preliminary classification results and the pest classification results.

[0033] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0034] Obtain pictures of various pest types to form an original data set, and perform calibration on the original data set;

[0035] Performing a first processing on the calibrated original data set, and training and verifying the target detection model based on the first processing result; performing a second processing on insect pest pictures of similar types selected from the calibrated original data set, and training and verifying the classification model based on the second processing result;

[0036] The target detection model is used to perform object detection on the pest image to obtain the pest detection result, wherein the detection result includes the location information and score result of each pest in the image to be detected, and the preliminary category and similar category are obtained based on the location information and score result;

[0037] The images of the pests to be detected are segmented based on the position information of similar categories, and the classification model is used to classify the pests according to the segmentation results to obtain the pest classification results;

[0038] Based on the location information, the preliminary classification result and the pest classification result, the types and quantities of the pests are obtained.

[0039] A device for identifying and counting pests, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following method steps when executing the computer program:

[0040] Obtain pictures of various pest types to form an original data set, and perform calibration on the original data set;

[0041] Performing a first processing on the calibrated original data set, and training and verifying the target detection model based on the first processing result; performing a second processing on insect pest pictures of similar types selected from the calibrated original data set, and training and verifying the classification model based on the second processing result;

[0042] The target detection model is used to perform object detection on the pest image to obtain the pest detection result, wherein the detection result includes the location information and score result of each pest in the image to be detected, and the preliminary category and similar category are obtained based on the location information and score result;

[0043] The images of the pests to be detected are segmented based on the position information of similar categories, and the classification model is used to classify the pests according to the segmentation results to obtain the pest classification results;

[0044] Based on the location information, the preliminary classification result and the pest classification result, the types and quantities of the pests are obtained.

[0045] An insect sex attractant device comprises a main body and a device for identifying and counting pests. The device for identifying and counting pests is arranged on or inside the main body with the main body as a carrier.

[0046] The present invention has significant technical effects due to the adoption of the above technical solution:

[0047] The intelligent sex lure detection and reporting algorithm developed based on the method, device and system of the present invention uses AI image recognition technology to complete the remote image acquisition and counting functions of trapped insects, conducts real-time monitoring of target insects, and visualizes the dynamic growth and decline of target insects in the field to users through clear images and accurate data. It not only solves the pain points of data misreporting and false alarming caused by infrared counting in existing sex lure detection and reporting products on the market, but also reduces the inconvenience and inaccuracy of manual insect segmentation and re-verification of data. Applying the method of the present invention to sex lure devices can increase the recognition and technical functions of existing sex lure devices, and make them light, compact, inexpensive and of good quality, so that the concept of integrated ecological pest control can be better implemented in agricultural and forestry pest control in production, and actively responds to the green prevention and control and environmentally friendly policies advocated by the country. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 It is a schematic diagram of the overall process of the method of the present invention;

[0050] Figure 2 It is a schematic diagram of the overall structure of the system of the present invention;

[0051] Figure 3 is a flow chart of a specific embodiment of the present invention;

[0052] Figure 4 It is a schematic diagram of the structure of densenet in the present invention;

[0053] Figure 5-Figure 6 A schematic diagram of test results of an embodiment;

[0054] Figure 7-Figure 8 It is a specific identification schematic diagram in another embodiment. DETAILED DESCRIPTION

[0055] The present invention is further described in detail below in conjunction with embodiments. The following embodiments are for explanation of the present invention but the present invention is not limited to the following embodiments.

[0056] At present, insect pheromones (sex attractants) have the advantages of strong specificity, no drug resistance, environmental friendliness, 100% compatibility with other control technologies, and significant improvement in the quality of agricultural products. Sex attractants have become one of the green control technologies advocated by the country. "Intelligent Insect Sex Attractant Detection and Reporting" uses sex attractants to combine traditional pest sex traps with Internet of Things information technology to monitor and biologically control insects that may cause harm in agricultural and forestry production. It can guide scientific chemical control in the field, reduce the use of pesticides, thereby reducing production costs and greatly improving the quality of agricultural products, increasing the value of agricultural products, and responding to the plant protection policy of prevention first and comprehensive control in pest control.

[0057] In the prior art, traditional traps are simple and crude, they only trap but do not collect data, and cannot be monitored, and cannot achieve the effect of guiding prevention and control. Existing sexual lure detection and reporting system products on the market are bulky, difficult to move, and expensive. There is a widespread phenomenon of false counts. Ordinary users have demand but no purchasing power. At present, big data research is relatively extensive. Can big data and traditional traps be combined to develop low-cost intelligent sexual lure detection and reporting system products? This application solves this problem.

[0058] Embodiment 1:

[0059] A pest identification and counting method, such as Figure 1 As shown, the following steps are included:

[0060] S100, obtaining pictures of various insect pest types to form an original data set, and performing calibration processing on the original data set;

[0061] S200, performing a first processing on the calibrated original data set, and training and verifying the target detection model based on the first processing result; performing a second processing on insect pest pictures of similar types selected from the calibrated original data set, and training and verifying the classification model based on the second processing result;

[0062] S300, using a target detection model to perform object detection on the image of the pest to be detected to obtain a pest detection result, wherein the detection result includes the position information and score result of each pest in the image to be detected, and obtaining a preliminary category and a similar category based on the position information and the score result;

[0063] S400, cutting the image of the pest to be detected based on the position information of similar categories, and classifying the pest types based on the cutting results using a classification model to obtain a pest classification result;

[0064] S500: Obtain the types and quantities of pests based on the location information, the preliminary classification result and the pest classification result.

[0065] In the present invention, the CSPDarkNet53 target detection model in the darknet framework is used in combination with the densenet classification model to train the image model of the specific pests captured by the sex attractant device, and the results of the two models are fused to finally obtain the type and quantity of the pests. The general process includes the following steps: (1) deploying the self-developed sex attractant device in the wild, placing the attractant core in the field to trap specific pests, and taking photos with the built-in camera to build a sex attractant pest database; (2) constructing the yolov4 target detection algorithm, using the CSPDarkNet53 network as the feature extractor, and using three yolo layers to perform three tests, with the feature sampling times of 32, 16, and 8 respectively. The detection is performed on the map, and the SPP module is added at the same time, and the original FPN method is replaced by the aggregation method of different backbone-level parameters in PANet; (3) In order to distinguish similar samples and eliminate the problem of misdetection of irrelevant targets in the detection task, the present invention constructs a densenet classification model, cuts out the difficult samples in the training data set as the classification training data set, and trains the second model; (4) Based on the location information of similar categories, the images of the pests to be detected are cut and processed, and the classification model is used to classify the pest species of the results of the cutting process to obtain the pest classification results; (5) Based on the location information, the preliminary classification results and the pest classification results, the types and quantities of pests are obtained, that is, the pictures collected in the real scene can be analyzed to obtain the final detection results. The method of the present invention can visualize the dynamic growth and decline of target insects in the field in front of users through clear images and accurate data, which not only solves the pain points of the existing sex-induced detection products on the market that use infrared counting and have data misreporting and false alarms, but also reduces the inconvenience and inaccuracy of manual insect segmentation and re-verification of data.

[0066] In addition, another application of the present invention is to apply the method, system or device of the present invention to an insect sex attractant device, so that the type and number of captured pests can be directly obtained, and the existing sex attractant device can be equipped with identification and technical functions. The device is also light, compact, inexpensive and of good quality, so that the concept of integrated ecological pest management can be better implemented in agricultural and forestry pest management in production, actively responding to the green prevention and control and environmentally friendly policies advocated by the country.

[0067] The company's self-developed sex attractant equipment is deployed in the wild to collect pest images, and these images are used as the original data set. The more images, the better. After the original data set is collected, the labelimg calibration tool is used to calibrate the images in the original data set. Generally, one image corresponds to one xml calibration file. The calibration process is specifically as follows: the pest category information and the pest location information are marked on the original data set, and the sample enhancement process is performed after calibration.

[0068] The training data set of the present invention has a total of 5,000 pictures, including about 40,000 pest targets, totaling 8 categories; there are a total of 8 categories in the original data set, of which 3 features are relatively unclear, so the 3 difficult categories in the original data set can be used as pest category data sets to construct a training classification data set, totaling 3 categories and a total of 9,000 small pictures.

[0069] In step S200, the original data set is first processed, and the target detection model is trained and verified based on the first processing result, specifically:

[0070] Perform sample enhancement processing on the calibrated original data set and divide it into a training data set, a test data set and a verification data set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing;

[0071] The training data set is input into the target detection model for training. After the training is completed, the test data set and the validation data set are used as input to verify the training results, thereby obtaining the target detection model.

[0072] In step S200, the insect pest pictures of similar types are selected from the calibrated original data set for second processing, and the classification model is trained and verified based on the second processing result, specifically:

[0073] Based on the similarity of pest categories, the calibrated original data set is selected, and similar pest category data sets are selected and cut;

[0074] Based on the cutting processing result, sample enhancement processing is performed and divided into a classification model training data set, a classification model test data set and a classification model verification set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing;

[0075] The classification model training data set is input into the classification model for training. After the training is completed, the classification model test data set and the classification model verification data set are used as input to verify the training results and then obtain the classification model.

[0076] Since some categories are very similar, a simple detection network cannot accurately distinguish similar categories. Therefore, based on the results of the detection model, the unrecognized images need to be passed through a category classification tool. Therefore, when training the classification model, it is necessary to select a database of similar pest categories, cut the targets of each type of pest, and save them as new small pictures to form a new pest category data set; these data sets are sample enhanced before training.

[0077] In one embodiment, the input image size of the constructed classification network is 256x256, and the pest category dataset is used to train the deep classification model using the DenseNet network in the Darknet framework. The specific network structure of the DenseNet classification network is as shown in the attached figure. Figure 4 As shown in the figure, the DenseNet network uses the DenseBlock+Transition structure to directly connect feature maps from different layers, achieve feature reuse, and improve efficiency.

[0078] In one embodiment, before inputting the training data set into the target detection model for training, the step further includes:

[0079] The anchor box parameter values ​​of the target detection model are modified based on the k-means clustering algorithm.

[0080] In one embodiment, before inputting the classification model training data set into the classification model for training, the step further includes:

[0081] The loss function in the classification model is calculated based on the forward network propagation algorithm, and the parameters of the classification model are updated through the back propagation algorithm to obtain the revised classification model.

[0082] After the steps described in the above embodiments, the training process of the target detection model and the classification model is as follows:

[0083] The input size of the training image needs to be increased to 1024x1024;

[0084] CSPDarkNet53 is selected as the model backbone network because it includes 29 3x3 convolutional layers, a 725x725 receptive field and 27.6M parameters;

[0085] The spp module is added to the CSPDarkNet53 network to increase the receptive field and separate the most important contextual features without reducing the network's running speed;

[0086] Using PANet as a parameter aggregation method from different backbone levels for different detector levels, replacing the original FPN method;

[0087] Further add data augmentation methods such as Mosaic, Mixup, CutMix and Self-Adversarial Training (SAT) in the training configuration file;

[0088] The k-means clustering algorithm is used to perform clustering statistics on the sexual seduction dataset at a size of 1024x1024 to obtain the optimal anchor point parameters;

[0089] In the Darknet framework, the DenseNet model is used to train a small image classification model. Compared with ResNet, DenseNet has a more radical dense connection mechanism. Each layer accepts all previous layers as its additional input. In ResNet, each layer is short-circuited to the previous layer by element-wise addition. In DenseNet, each layer is connected to all previous layers in the channel dimension and serves as the input of the next layer. For a network with L layers, DenseNet contains Compared with ResNet, this is a dense connection. Moreover, DenseNet directly connects feature maps from different layers, which can achieve feature reuse and improve efficiency.

[0090] Expressed in formula, the output of the traditional network at layer l is:

[0091] x l =H l (x l-1 )

[0092] ResNet, on the other hand, adds an identity function from the previous layer input:

[0093] x l =H l (x l-1 )+x l-1

[0094] In the DenseNet used in this invention, all the previous layers’ so-called inputs are connected:

[0095] x l =H l ([x0, x1, ..., x l-1 ])

[0096] CNN networks generally need to reduce the parameter size of feature maps through pooling layers or convolution layers with stride greater than 1, while the dense connection method of DenseNet requires the size of the features before and after to remain the same. In order to keep the same dimension, the DenseNet network uses the DenseBlock+Transition structure, where DenseBlock is a module containing many layers, the feature map of each layer is the same, and the layers are densely connected. The Transition module is two adjacent DenseBlocks, and the feature map dimension is reduced through the pooling layer. In DenseBlock, the feature map size of each layer is the same, and the front and back can be connected in the channel dimension. The nonlinear combination function in DenseBlock uses the structure of BN+ReLU+3x3 convolution layer. At the same time, there is a hyperparameter k called growth rate in DenseNet. All layers in all DenseBlocks output k feature maps after convolution, and the number of channels is k, which can also be called using k convolution kernels. Generally speaking, a smaller k (such as 12) can achieve better performance. Assuming that the number of feature map channels of the input layer is k0, the number of channels of the L layer input is k0+k(L-1). Therefore, as the number of layers increases, even if the k value is small, the input of DenseBlock will be a lot. This is due to feature reuse. Each layer has only k unique features. In order to solve the problem of too large input of the later layers, DenseBlock uses a bottleneck layer to reduce the amount of calculation and adds a 1x1 convolution layer to the original structure. This structure is called DenseNet-B structure, that is, BN+ReLU+1x1_Conv+BN+ReLU+3x3_Conv, in which the 1x1 convolution layer obtains 4k feature maps, which is used to reduce the number of features and improve calculation efficiency.

[0097] For the Transition layer, its function is to connect two adjacent DenseBlocks and reduce the size of the feature map. Transition includes a 1x1 convolution layer and a 2x2 average pooling layer. Its structure is BN+ReLU+1x1_Conv+2x2_AvgPooling. At the same time, the Transition layer can compress the model. DenseNet performs better than ResNet on public data sets, so the present invention uses DenseNet as the basic skeleton network of the classification model. After the above steps, the target detection model and classification model are finally trained.

[0098] When using convolutional neural networks to train the detection model, the detection task training set is pre-enhanced, CSPDarkNet53 is used as the model backbone network, and the input size is 1024x1024x3; the network is initialized, the loss function is calculated through forward network propagation, and the model parameters are updated through back propagation, and a total of 100,000 iterations are completed; the stochastic gradient descent method of Momentum is used, and the weight is 0.949; the training batch size is set to 64, and the subdivision is set to 16. Too large a batch size or too small a subdivision will lead to insufficient video memory; the initial learning rate is set to 0.001, and the learning rate is reduced to 0.0001 and 0.00001 at 80,000 and 90,000 times; the k-means algorithm is used to cluster the data set to the most appropriate anchors value; during the training of the model, if the intersection-over-union ratio of the candidate box and the groundtruth is greater than 0.5, it is regarded as a positive sample, and if it is less than 0.5, it is regarded as a negative sample; it takes about 30 hours to complete the training.

[0099] When using convolutional neural networks to train classification models, DenseNet is used as the model backbone network, the input size is 256x256x3, the network is initialized, the loss function is calculated through forward network propagation, and the model parameters are updated through back propagation. A total of 50,000 iterations are completed; the stochastic gradient descent method of Momentum is used, and the weight is 0.9; the training batch size is set to 128, the subdivision is set to 4; the initial learning rate is set to 0.1; the complete training takes about 15 hours.

[0100] Based on the location information of similar categories, the images of the pests to be detected are cut and processed, and the classification model is used to classify the pest species of the results of the cutting process to obtain the pest classification results, and the results of the detection model and the classification model are fused. Based on the location information, the preliminary classification results and the pest classification results, the types and quantities of the pests are obtained. Among the 8 target pests of the present invention, 5 have obvious characteristics, and excellent results can be obtained through the target detection model. The other 3 (spodoptera frugiperda, armyworm, armyworm) relatively similar pests need to be classified by the classification model again, and the final result is obtained by fusing the two. If the detection result is one of the five relatively simple pests, the result will be returned directly; if the detection result is one of the three more difficult species, the image where the detected box is located will be input into the DenseNet classification network, and the classification result will be compared with the detection result, and the species with a higher score will be obtained as the final output result. The final detection results will be saved in dictionary format, transmitted to the terminal through redis, and displayed; According to the test log of Top Cloud Agriculture's outdoor intelligent sex attractant prototype, a total of 8 test sex attractants were deployed in the field during the time period from June 14 to September 3, and a total of 576 sets of valid data were captured. The accuracy rate was about 95% when the algorithm recognition results were compared with the manual proofreading results; the test results of the three main pests are shown in the following table:

[0101] Pest types Number of pictures Number of algorithm recognition Manual calibration number Recognition rate Borer borer 292 1751 1798 97.38％ Fall Armyworm 159 254 258 98.45％ Asian corn borer 26 165 184 89.67％

[0102] In another embodiment, rice planthoppers are used as the detection object, and the types of rice planthoppers are detected, such as Figure 7-8As shown. The first layer model uses the yolov4 target detection model. The yolov4 target detection algorithm has been improved a lot compared to yolov3, mainly in the following aspects: Taking into account the amount of calculation, yolov4 improves the input end during training, so that training can also achieve good results on a single GPU; the network data enhancement has the greatest performance improvement in Mosaic data enhancement. The Mosaic used in yolov4 refers to the CutMix data enhancement method proposed at the end of 2019, but CutMix only uses two pictures for splicing, while Mosaic data enhancement uses 4 pictures, randomly scaled, randomly cropped, and randomly arranged for splicing, making the target distribution more uniform, which helps to improve performance; the backbone network CSPDarknet53 used by yolov4 is based on the Yolov3 backbone network Darknet53. The Backbone structure generated contains 5 CSP modules, and the size of the convolution kernel in front of each CSP module is 3*3, stride=2, so it can play a role in downsampling. Because Backbone has 5 CSP modules and the input image is 608*608, the law of feature map change is: 608->304->152->76->38->19. After 5 CSP modules, a feature map of size 19*19 is obtained. And the Mish activation function is only used in Backbone, and the Leaky_relu activation function is still used in the back of the network. The advantage of this is to enhance the learning ability of CNN, so that it can maintain accuracy while being lightweight, and at the same time reduce the computing cost and memory cost; Neck innovation of yolov4. In the field of target detection, in order to better extract fusion features, some layers are usually inserted in the Backbone and output layers. This part is called Neck. It is equivalent to the neck of the target detection network, which is also very critical. The Neck structure of yolov4 mainly adopts the SPP module and FPN+PAN.

[0103] In this embodiment, the second layer model of the rice planthopper detection method for large-resolution images uses a bilinear network model based on Resnet 50. The bilinear network includes two branch CNN structures. The two branches can be the same network or different networks. This embodiment uses Resnet 50 as the same branch network. After the image is sent to the two branch Resnet 50 under this network, the two acquired feature branches are subjected to corresponding fusion operations. The loss function selects the cross entropy loss function, and the optimization method selects SGD optimization. The initial learning rate is set to 0.01, the batch size is set to 8, the decay rate is set to 0.00001, the iteration cycle is 20, and the top-5 evaluation index is used.

[0104] The first-layer classifier dataset is two rice planthoppers (white-backed planthopper and brown planthopper) pest images, all of which are collected from intelligent insect monitoring lamps. The images are stored in JPG format, with a resolution of 5472×3648 (W×H) and a total of 2000 images, which are evenly divided into 16 small-resolution images with a resolution of 1368×912 (W×H) and a total of 32000 images. The Labelimg dataset labeling tool was used to label all target pests in the data sample, and all the data were reviewed and corrected by experts to form the original dataset, which was evenly and randomly divided into training dataset, validation dataset and test dataset according to the ratio of 7.5:1.5:1. The original image was segmented to obtain a single planthopper individual, and then the single insect individual was randomly rotated at multiple angles to produce the second-layer classifier database, including 38000 white-backed planthoppers, 36000 brown planthoppers, and 41000 non-target planthoppers. The first-layer model was carried out under the Darknet framework. In order to improve the detection accuracy of the model, the input network parameters width and height are set to 960. Considering the memory problem, in the network configuration file, the batch size is set to 64, indicating that the parameters will be updated once 64 images are input to the network. Set subdivisions to 8, indicating that the number of samples input into the training network at one time is batch / subdivisions=8. Momentum affects the speed at which the gradient descends to the optimal value, which is usually set to 0.9. The present invention sets momentum to 0.9. In order to better extract features, the maximum number of iterations is set to 200,000, and the iteration step is set to 20,000 before 150,000. 150,000 to 200,000 is set to 10,000, and the initial learning rate is set to 0.01. The second-layer classification model is trained based on Pytorch, and the input network parameters width and height are set to 224, the batch_size size is set to 128, the initial learning rate is 0.001, and the learning rate drops to 0.1 times that of the previous stage every 100 epochs. The total number of iterations is 200,000. For the first-layer target detection model, the K-means clustering algorithm is used to calculate the anchor box value of each sub-model separately, so that three sets of anchor box values ​​of different scales are obtained to replace the original network anchor value.

[0105] In addition, in the process of implementing the present application method, the following steps are also included: the image of the pest to be detected is processed, specifically:

[0106] The images in the original dataset are classified according to the body structure of the pests, and each one is labeled based on the structural features of each type of pest;

[0107] If the pest image in the to-be-detected pest image is an incomplete image, then the structural features of the pest image in the image are extracted, and the extracted features are compared with each type of structural features marked in the original data set to obtain a comparison result, wherein at least three groups of comparison results are recommended;

[0108] Based on the comparison result, the remaining part of the incomplete pest image is supplemented to obtain a complete pest image to be detected.

[0109] This embodiment is actually for solving the problem of the types of incomplete pests. The body structure of pests is actually classified according to whether the pests have wings, spots on the wings, antennae, feet, tails or shells. It is not a rigorous classification in textbooks, because the types of pests need to be judged more carefully and accurately later. In order to obtain the best matching results for incomplete images, the more marked features the better. If the pest image in the to-be-detected pest image is an incomplete image, then the pest is compared based on the marked features. The more the structural features overlap, the better the comparison result. Finally, at least three results will be selected from the comparison results. Based on this result, the incomplete part of the pest is completed to obtain a relatively complete pest image. It should be noted that when completing, the completed part must be larger than the incomplete part, so that the connected part overlaps. Then, these overlapping and complete pest images are input into the target detection model. The target detection model performs the detection operation by detecting the pest images with overlapping parts. If at least one of the three groups of results shows that only one pest is detected, it means that the comparison result recommended in the early stage is correct. Based on the result of only detecting one pest, it is determined what kind of pest this is, and a specific pest recognition result is obtained. If the pest image with overlapping parts spliced ​​out based on the three results is input into the target detection model, multiple recognition results will be obtained. The one with the most repetitions among the multiple recognition results is found as the final pest detection result.

[0110] Embodiment 2:

[0111] A pest identification and counting system, such as Figure 2 As shown, it includes a data acquisition module 100, a processing training module 200, a result detection module 300, a processing classification module 400 and a comprehensive result module 500;

[0112] The data acquisition module 100 is used to acquire pictures of various pest types to form an original data set, and perform calibration processing on the original data set;

[0113] The processing and training module 200 is used to perform a first processing on the calibrated original data set, and train and verify the target detection model based on the first processing result; perform a second processing on insect pest pictures of similar types selected from the calibrated original data set, and train and verify the classification model based on the second processing result;

[0114] The result detection module 300 is configured to: use the target detection model to perform object detection on the pest image to be detected to obtain the pest detection result, wherein the detection result includes the position information and score result of each pest in the image to be detected, and obtain the preliminary category and similar category based on the position information and the score result;

[0115] The processing and classification module 400 is used to perform segmentation processing on the image of the pest to be detected based on the position information of similar categories, and classify the pest types based on the segmentation processing results using a classification model to obtain a pest classification result;

[0116] The comprehensive result module 500 is used to obtain the types and quantities of pests based on the location information, the preliminary classification results and the pest classification results.

[0117] In one embodiment, a calibration module 600 is further included, wherein the calibration module 600 is configured to: mark the pest category information and the pest location information on the original data set.

[0118] The processing training module 200 is configured to:

[0119] Perform sample enhancement processing on the calibrated original data set and divide it into a training data set, a test data set and a verification data set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing;

[0120] The training data set is input into the target detection model for training. After the training is completed, the test data set and the validation data set are used as input to verify the training results, thereby obtaining the target detection model.

[0121] The processing training module 200 is configured to: select the calibrated original data set based on the similarity of the pest categories, select the similar pest category data set, and perform cutting processing;

[0122] Based on the cutting processing result, sample enhancement processing is performed and divided into a classification model training data set, a classification model test data set and a classification model verification set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing;

[0123] The classification model training data set is input into the classification model for training. After the training is completed, the classification model test data set and the classification model verification data set are used as input to verify the training results and then obtain the classification model.

[0124] The processing training module 200 is configured to: before inputting the training data set into the target detection model for training, it also includes:

[0125] The anchor box parameter values ​​of the target detection model are modified based on the k-means clustering algorithm.

[0126] The processing training module 200 is configured to: before inputting the classification model training data set into the classification model for training, the step further includes:

[0127] The loss function in the classification model is calculated based on the forward network propagation algorithm, and the parameters of the classification model are updated through the back propagation algorithm to obtain the revised classification model.

[0128] Embodiment 3:

[0129] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0130] Obtain pictures of various pest types to form an original data set, and perform calibration on the original data set;

[0131] Performing a first processing on the calibrated original data set, and training and verifying the target detection model based on the first processing result; performing a second processing on insect pest pictures of similar types selected from the calibrated original data set, and training and verifying the classification model based on the second processing result;

[0132] The target detection model is used to perform object detection on the pest image to obtain the pest detection result, wherein the detection result includes the location information and score result of each pest in the image to be detected, and the preliminary category and similar category are obtained based on the location information and score result;

[0133] The images of the pests to be detected are segmented based on the position information of similar categories, and the classification model is used to classify the pests according to the segmentation results to obtain the pest classification results;

[0134] Based on the location information, the preliminary classification result and the pest classification result, the types and quantities of the pests are obtained.

[0135] In one embodiment, when the processor executes the computer program, the first processing of the original data set is implemented, and the target detection model is trained and verified based on the first processing result, specifically:

[0136] Perform sample enhancement processing on the calibrated original data set and divide it into a training data set, a test data set and a verification data set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing;

[0137] The training data set is input into the target detection model for training. After the training is completed, the test data set and the validation data set are used as input to verify the training results, thereby obtaining the target detection model.

[0138] In one embodiment, when the processor executes the computer program, the second processing is performed by selecting pest images of similar types from the calibrated original data set, and training and verifying the classification model based on the second processing result, specifically:

[0139] Based on the similarity of pest categories, the calibrated original data set is selected, and similar pest category data sets are selected and cut;

[0140] Based on the cutting processing result, sample enhancement processing is performed and divided into a classification model training data set, a classification model test data set and a classification model verification set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing;

[0141] The classification model training data set is input into the classification model for training. After the training is completed, the classification model test data set and the classification model verification data set are used as input to verify the training results and then obtain the classification model.

[0142] In one embodiment, when the processor executes the computer program, before implementing the step of inputting the training data set into the target detection model for training, the step further includes:

[0143] The anchor box parameter values ​​of the target detection model are modified based on the k-means clustering algorithm.

[0144] In one embodiment, when the processor executes the computer program, before implementing the step of inputting the classification model training data set into the classification model for training, the step further includes:

[0145] The loss function in the classification model is calculated based on the forward network propagation algorithm, and the parameters of the classification model are updated through the back propagation algorithm to obtain the revised classification model.

[0146] In one embodiment, when the processor executes the computer program, the calibration process is implemented specifically by marking the pest category information and the pest location information on the original data set.

[0147] Embodiment 4:

[0148] In one embodiment, a device for identifying and counting pests is provided, which can be a server or a mobile terminal. The device for identifying and counting pests includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the device for identifying and counting pests is used to provide computing and control capabilities. The memory of the device for identifying and counting pests includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database is used to store all data of the device for identifying and counting pests. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the method for identifying and counting pests is implemented.

[0149] Embodiment 5:

[0150] An insect sex attractant device comprises a main body and a device for identifying and counting pests. The device for identifying and counting pests is arranged on or inside the main body with the main body as a carrier.

[0151] The device of the pest identification technology of the present invention is arranged in or on the body of the insect sex lure device. The intelligent sex lure detection and reporting algorithm of the present invention completes the remote image acquisition and counting functions of trapped insects through AI image recognition technology, monitors the target insects in real time, and visualizes the dynamic growth and decline of target insects in the field to the user through clear images and accurate data. It not only solves the pain point of data misreporting and false alarming caused by infrared counting of existing sex lure detection and reporting products on the market, but also reduces the inconvenience and inaccuracy of manual insect segmentation and re-verification of data. The application in the sex lure device can not only increase the recognition and technical functions of the existing sex lure device, but also will not increase the size of the insect sex lure device. It is light, compact, inexpensive, and enables the concept of integrated ecological pest control to be better implemented in agricultural and forestry pest control in production, and actively responds to the green prevention and control and environmentally friendly policies advocated by the country.

[0152] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0153] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] It should be noted that:

[0158] The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0159] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. In addition, it should be noted that the shapes, names, etc. of the parts and components of the specific embodiments described in this specification may be different. All equivalent or simple changes made according to the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. Those skilled in the art of the technical field to which the present invention belongs may make various modifications or supplements to the described specific embodiments or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A pest identification and counting method, characterized in that: The following steps are involved: Obtain pictures of various pest types to form an original data set, and perform calibration on the original data set; Performing a first processing on the calibrated original data set, and training and verifying the target detection model based on the first processing result; Selecting pest images of similar types from the calibrated original data set for second processing, and training and verifying the classification model based on the second processing results; The target detection model is used to perform object detection on the pest image to obtain the pest detection result, wherein the detection result includes the location information and score result of each pest in the image to be detected, and the preliminary category and similar category are obtained based on the location information and score result; The images of the pests to be detected are segmented based on the position information of similar categories, and the classification model is used to classify the pests according to the segmentation results to obtain the pest classification results; Based on the location information, the preliminary classification result and the pest classification result, the types and quantities of the pests are obtained; The first processing of the original data set and training and verifying the target detection model based on the first processing result are specifically as follows: Perform sample enhancement processing on the calibrated original data set and divide it into a training data set, a test data set and a verification data set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing; The training data set is input into the target detection model for training. After the training is completed, the test data set and the validation data set are used as input to verify the training results, thereby obtaining the target detection model. Among them, the second processing is performed by selecting pest pictures of similar types from the original data set based on the calibration, and training and verifying the classification model based on the second processing result, specifically: Based on the similarity of pest categories, the calibrated original data set is selected, and similar pest category data sets are selected and cut; Based on the cutting processing results, sample enhancement processing is performed and divided into a classification model training data set, a classification model test data set and a classification model verification set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing; The classification model training data set is input into the classification model for training. After the training is completed, the classification model test data set and the classification model verification data set are used as input to verify the training results and then obtain the classification model.

2. The pest identification and counting method according to claim 1, characterized in that: Before the training data set is input into the target detection model for training steps, it also includes: The anchor box parameter values ​​of the target detection model are modified based on the k-means clustering algorithm.

3. The pest identification and counting method according to claim 1, characterized in that: Before inputting the classification model training data set into the classification model for training, the following steps are also included: The loss function in the classification model is calculated based on the forward network propagation algorithm, and the parameters of the classification model are updated through the back propagation algorithm to obtain the revised classification model.

4. The pest identification and counting method according to claim 1, characterized in that: The following steps are also included: The images of the pests to be detected are processed as follows: The images in the original dataset are classified according to the body structure of the pests, and each one is labeled based on the structural characteristics of each type of pest; If the pest image in the to-be-detected pest image is an incomplete image, then the structural features of the pest image in the image are extracted, and the extracted features are compared with each type of structural features marked in the original data set to obtain a comparison result, wherein at least three groups of comparison results are recommended; Based on the comparison result, the remaining part of the incomplete pest image is supplemented to obtain a complete pest image to be detected.

5. A pest identification and counting system, characterized in that: It includes data acquisition module, processing training module, result detection module, processing classification module and comprehensive result module; The data acquisition module is used to acquire pictures of various pest types to form an original data set, and perform calibration processing on the original data set; The processing and training module is used to perform a first processing on the calibrated original data set, and train and verify the target detection model based on the first processing result; Selecting pest images of similar types from the calibrated original data set for second processing, and training and verifying the classification model based on the second processing results; The result detection module is configured to: use the target detection model to perform object detection on the pest image to be detected to obtain the pest detection result, wherein the detection result includes the position information and score result of each pest in the image to be detected, and obtain the preliminary category and similar category based on the position information and the score result; The processing and classification module is used to perform segmentation processing on the image of the pest to be detected based on the position information of similar categories, and classify the pest types based on the segmentation processing results using a classification model to obtain a pest classification result; The comprehensive result module is used to obtain the types and quantities of pests based on the location information, the preliminary classification results and the pest classification results; The first processing of the original data set and training and verifying the target detection model based on the first processing result are specifically as follows: Perform sample enhancement processing on the calibrated original data set and divide it into a training data set, a test data set and a verification data set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing; The training data set is input into the target detection model for training. After the training is completed, the test data set and the validation data set are used as input to verify the training results, thereby obtaining the target detection model. Among them, the second processing is performed by selecting pest pictures of similar types from the original data set based on the calibration, and training and verifying the classification model based on the second processing result, specifically: Based on the similarity of pest categories, the calibrated original data set is selected, and similar pest category data sets are selected and cut; Based on the cutting processing results, sample enhancement processing is performed and divided into a classification model training data set, a classification model test data set and a classification model verification set, wherein the enhancement processing includes one or more of rotation processing, brightness and hue adjustment processing, noise removal processing and image mirror processing; The classification model training data set is input into the classification model for training. After the training is completed, the classification model test data set and the classification model verification data set are used as input to verify the training results and then obtain the classification model.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

7. A device for identifying and counting pests, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

Images