Insect pest detection method and device

Abstract

The invention provides an insect pest detection method and device, and the method comprises the steps: obtaining at least one initial region image corresponding to a target detection region, the target detection region comprises to-be-detected vegetation, and the initial region image comprises multispectral information; image preprocessing is carried out on each initial area image, at least one area image to be processed is obtained, and the area image to be processed comprises vegetation index information corresponding to the vegetation to be detected; the to-be-processed area images are input into an insect pest detection model, a predicted insect pest image area output by the insect pest detection model is obtained, and the predicted insect pest image area is generated by the insect pest detection model according to the vegetation index information; and determining a target insect pest area corresponding to the to-be-detected vegetation in the target detection area according to each predicted insect pest image area.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a pest detection method, a pest detection device, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Agricultural pests are an important factor affecting crop yield and quality. Traditional pest detection methods mainly rely on manual inspections and fixed detection equipment to observe and record crop pests. Although these methods can detect pests to a certain extent, with the expansion of agricultural production scale and the complexity of pest types, the disadvantages of traditional methods are becoming increasingly prominent, specifically manifested in low efficiency, poor real-time performance, insufficient accuracy, high labor intensity, inconvenient data management, weak early warning capabilities, and low economic benefits. Traditional pest detection methods have been difficult to meet the needs of efficient, accurate, and real-time pest detection in modern agricultural production. These drawbacks make it difficult to detect and control pests in a timely manner, affecting crop yield and quality, and increasing the cost and risk of agricultural production. Therefore, a new pest detection method is urgently needed to solve the above problems. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a pest detection method. The present application also relates to a pest detection device, a computing device, a computer-readable storage medium and a computer program product to solve the above problems existing in the prior art.

[0004] According to a first aspect of an embodiment of the present application, a pest detection method is provided, comprising: Acquire at least one initial area image corresponding to a target detection area, wherein the target detection area includes vegetation to be detected, and the initial area image includes multispectral information; Performing image preprocessing on each initial region image to obtain at least one region image to be processed, wherein the region image to be processed includes vegetation index information corresponding to the vegetation to be detected; Inputting each image of the area to be processed into the pest detection model to obtain a predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information; The target pest area corresponding to the vegetation to be detected is determined in the target detection area according to each predicted pest image area.

[0005] According to a second aspect of an embodiment of the present application, there is provided an insect pest detection device, comprising: An image acquisition module is configured to acquire at least one initial area image corresponding to a target detection area, wherein the target detection area includes vegetation to be detected, and the initial area image includes multispectral information; An image preprocessing module is configured to perform image preprocessing on each initial region image to obtain at least one region image to be processed, wherein the region image to be processed includes vegetation index information corresponding to the vegetation to be detected; A model prediction module is configured to input each image of the area to be processed into the pest detection model to obtain a predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information; The pest positioning module is configured to determine the target pest area corresponding to the vegetation to be detected in the target detection area according to each predicted pest image area.

[0006] According to a third aspect of an embodiment of the present application, a computing device is provided, including: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above method are implemented.

[0007] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program / instruction, and the steps of the above method are implemented when the computer program / instruction is executed by a processor.

[0008] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0009] The pest detection method provided by the present application obtains at least one initial area image corresponding to a target detection area, wherein the target detection area includes vegetation to be detected and the initial area image includes multispectral information; performs image preprocessing on each initial area image to obtain at least one to-be-processed area image, wherein the to-be-processed area image includes vegetation index information corresponding to the vegetation to be detected; inputs each to-be-processed area image into a pest detection model to obtain a predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information; and determines a target pest area corresponding to the vegetation to be detected in the target detection area according to each predicted pest image area.

[0010] An embodiment of the present application provides a pest detection method based on visual technology, which uses a multispectral sensor to collect rich spectral information. Combining image preprocessing and pest detection models, the accuracy of pest detection is improved. Real-time detection and early warning of pests are achieved. The fully automated pest detection method deployed on the server reduces dependence on manpower and reduces the cost of pest detection and prevention. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a flow chart of a pest detection method provided by an embodiment of the present application; Figure 2 This is a processing flow chart of a pest detection method applied to a crop pest detection scenario provided by an embodiment of the present application; Figure 3 is a structural schematic diagram of an insect pest detection device provided in one embodiment of the present application; Figure 4 It is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0012] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0013] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0014] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0015] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0016] In the present application, a pest detection method is provided. The present application also relates to a pest detection device, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0017] Figure 1 A flowchart of an insect pest detection method provided according to an embodiment of the present application is shown, which specifically includes the following steps: Step 102: Acquire at least one initial area image corresponding to the target detection area, wherein the target detection area includes vegetation to be detected, and the initial area image includes multispectral information.

[0018] The target detection area is the area where crop disease detection is required in the method provided in the present application, and the crops to be detected are the crops planted in the target detection area. In practical applications, the target detection area may be a farmland, and the vegetation to be detected is the crops planted in the farmland. In another specific embodiment provided in the present application, the target detection area may be a flower garden, and the vegetation to be detected is the flowers planted in the flower garden, etc.

[0019] The initial region image is at least one image collected for the target detection region. In practical applications, the target detection region may be relatively large, and one image may not be able to capture the entire target detection region. Therefore, a method of shooting in different regions can be adopted to obtain multiple initial region images corresponding to the target detection region. Multiple initial region images can be combined and spliced ​​into the target detection region.

[0020] The crop disease detection method provided in the embodiment of the present application can be applied to a cloud server or a client, which is not limited in this embodiment. The initial area images of the razor in the embodiment of the present application are all images carrying multispectral information collected by a multispectral sensor.

[0021] In a specific implementation provided in the present application, obtaining at least one initial area image corresponding to the target detection area includes: At least one initial area image corresponding to the target detection area is acquired based on an image acquisition device carrying a multi-spectral sensor.

[0022] In this embodiment, the initial area image for the target detection area is obtained by an image acquisition device. In practical applications, at least one initial area image for the target detection area can be acquired by moving the image acquisition device. In the specific method provided in the embodiment of the present application, a multispectral sensor is provided in the image acquisition device, and the acquired initial area image is multispectral image data.

[0023] Multispectral sensors have high spectral resolution and can capture key bands related to pests (such as visible light band, near infrared band, infrared band, etc.) to obtain subtle differences in vegetation health status. They have high spatial resolution and help identify small-scale pest characteristics. They have a high signal-to-noise ratio and can improve image quality. In order to adapt to the use of image acquisition equipment, multispectral sensors should be lightweight and miniaturized. Multispectral sensors also support standard data interfaces, which facilitate integration with image acquisition equipment and servers and support real-time data transmission.

[0024] In another specific implementation provided by the present application, the target detection area is usually a large area such as farmland, flowers and other areas that are inconvenient to enter. If the camera is taken manually with a handheld camera, there are problems such as inconvenient shooting and inappropriate shooting angles. With the development of drone technology, drones have also been widely used in agricultural production. Drones can cover a large area of ​​farmland in a short time, and collect image data of the vegetation to be detected in real time through high-definition cameras, sensors, etc., so as to facilitate subsequent image processing for the target detection area. Based on this, in a specific implementation provided by the present application, the image acquisition device includes a drone. Accordingly, at least one initial area image corresponding to the target detection area is obtained based on an image acquisition device carrying a multispectral sensor, including: Obtaining an electronic map of the target detection area; Generating flight information and image acquisition information of the UAV according to the electronic map; Sending the flight information and the image acquisition information to the drone, so that the drone carrying the multispectral sensor flies based on the flight information, and captures at least one initial area image corresponding to the target detection area based on the image acquisition information; Receive at least one initial area image taken by the drone.

[0025] In this embodiment, in order to ensure that the drone can fully cover the target detection area, an electronic map corresponding to the target detection area can be obtained first. In actual applications, the electronic map can be obtained from a third-party map agency.

[0026] After obtaining the electronic map of the target detection area, the flight information and image acquisition information of the drone can be generated based on the electronic map. Flight information can be understood as the information of the drone when performing a flight mission, such as flight navigation path, flight altitude, flight speed, flight angle, etc. Image acquisition information can be understood as the shooting parameters of the camera installed on the drone, such as shutter shooting interval, shooting clarity, shooting mode, etc.

[0027] After the flight information and the image acquisition information are determined, the flight information and the image acquisition information can be sent to the drone so that the drone can move according to the flight information. The target detection area is photographed according to the image acquisition information to obtain at least one initial area image.

[0028] Specifically, the drone is equipped with a GPS navigation and positioning module and a multispectral sensor. After obtaining flight information, the flight trajectory of the drone can be controlled according to the GPS navigation and positioning module, and a parallel line flight trajectory can be used to ensure that the entire target detection area can be covered. At the same time, the flight interval of the drone must also meet the subsequent image stitching requirements to avoid information loss caused by missing images.

[0029] In a specific embodiment provided in the present application, a four-rotor drone is selected, the drone has a maximum load of 2 kg, a flight time of 30 minutes, and is equipped with a multispectral camera. The multispectral camera contains 5 bands, and the spatial resolution is 5 cm / pixel at an altitude of 50 meters. The flight altitude is set to 50 meters to ensure that the spatial resolution meets the requirements for pest identification. The flight information is that the route spacing is 25 meters and the flight speed is 5 meters / second. The drone flies according to the flight information, and the multispectral camera collects initial area images at a frequency of 1 frame per second, and a total of more than 3,000 initial area images are collected.

[0030] In the method provided in the embodiment of the present application, in order to improve the efficiency of data processing, the WiFi module on the drone can be used to establish a communication connection with the server, and the collected initial area image can be sent to the server through the WiFi module for subsequent image processing. The server side receives at least one initial area image taken by the drone based on the communication connection. The transmission rate of WiFi transmission is relatively high, which is suitable for short-distance real-time image transmission and is adapted to the operation scenario of the drone and the server within a certain range.

[0031] In actual applications, WiFi communication is only one of the implementation methods. In actual applications, other communication methods such as Bluetooth and infrared can also be used for data transmission. This is not limited in the method provided in this application, and the actual application shall prevail.

[0032] Step 104: performing image preprocessing on each initial region image to obtain at least one region image to be processed, wherein the region image to be processed includes vegetation index information corresponding to the vegetation to be detected.

[0033] After obtaining each initial area image, subsequent processing can be performed based on each initial area image. However, since each initial area image captured is an original image, it may have some impact on subsequent image processing due to shooting angle, lighting, etc. In addition, there may be a large number of repeated areas in multiple initial area images. Therefore, in order to make subsequent data processing more efficient, in the method provided in the embodiment of the present application, it is necessary to first calibrate and splice each initial area image and then segment it to obtain a plurality of area images to be processed, wherein the area images to be processed are images that are subsequently input into the pest detection model for recognition processing.

[0034] In a specific implementation provided in the present application, image preprocessing is performed on each initial region image to obtain at least one region image to be processed, including: Perform image correction and splicing on each initial region image to obtain an initial region image to be processed; The vegetation index information corresponding to the initial image of the area to be processed is calculated, and the initial image of the area to be processed is updated according to the vegetation index information to generate at least one image of the area to be processed.

[0035] In practical applications, each initial region image is first corrected and stitched, specifically including image correction, image stitching and registration, so as to obtain an initial region image to be processed, wherein the initial region image to be processed refers to an image stitched together after each initial region image is processed. After obtaining the initial region image to be processed, the image can also be augmented and denoised.

[0036] In a specific embodiment provided in the present application, image correction includes radiation correction, geometric correction, spectral correction, etc. In practical applications, after the image collected by the drone is transmitted to the server, the calibration board supporting the camera can be used to perform radiation correction on the collected image, and then professional software can be used to perform geometric correction to eliminate the distortion information in the image. Spectral correction is performed according to the spectral response curve of the multi-spectral sensor. After the correction is completed, the SIFT feature point matching algorithm is used to splice the images to generate an initial image of the area to be processed covering the entire target detection area. The multi-band registration algorithm is used to ensure that the images of each band can achieve pixel-level alignment.

[0037] After obtaining the initial image of the area to be processed, the initial image of the area to be processed may be further subjected to feature extraction and index calculation, wherein the index calculation specifically refers to calculating vegetation index information based on multispectral information of the image of the area to be processed. Specifically, calculating vegetation index information corresponding to the initial image of the area to be processed includes: Acquire at least one vegetation index parameter corresponding to the initial image of the area to be processed; At least three vegetation index information are generated according to each vegetation index parameter.

[0038] In the method provided in the present application, at least one vegetation index parameter is extracted from the initial image of the area to be processed. Specifically, the vegetation index parameter refers to information collected by a multispectral sensor. In this embodiment, the vegetation index parameter includes at least near infrared band (NIR), red light band (RED), red edge band (RE), short wave infrared band (SWIR), green band (GREEN), etc.

[0039] According to each vegetation index parameter, multiple vegetation index information can be calculated respectively. In the method provided in this application, the vegetation index information specifically includes the normalized difference vegetation index (NDVI), the difference vegetation index (DVI), the red edge index (REI), the leaf spot index (LSI), the moisture sensitivity index (VSI), the green vegetation index (GVI), etc.

[0040] The Normalized Difference Vegetation Index (NDVI) is the most commonly used vegetation index. It is used to reflect the health of vegetation and is widely used to detect crop growth, pests and diseases, and moisture conditions. The Normalized Difference Vegetation Index is calculated using the reflectivity difference between the near infrared band (NIR) and the red band (RED). The specific calculation formula is shown in the following formula 1: Formula 1 The Difference Vegetation Index (DVI) uses the difference between the reflectance of the near infrared band (NIR) and the red band (RED) to calculate the health of crops. It is a simplified form of the Normalized Difference Vegetation Index. The calculation method is shown in the following formula 2: Formula 2 The red edge index (REI) is detected based on the reflectance change between the red edge band (RE) and the near infrared band (NIR). The red edge band (RE) is an important characteristic area in the vegetation spectrum, and its changes are usually closely related to the growth status, nutritional status, and degree of pests and diseases of crops. Its calculation method is shown in the following formula 3: Formula 3 The Leaf Spot Index (LSI) detects changes in leaf spots through the difference in reflectivity between the shortwave infrared band (SWIR) and the red light band (RED), and is particularly suitable for detecting leaves that have been damaged by spots. The calculation method is shown in the following formula 4: Formula 4 The moisture sensitivity index (VSI) combines the difference in reflectance between the short-wave infrared band (SWIR) and the near-infrared band (NIR), and is particularly used to detect crop emergency responses caused by water stress, pests and diseases. Its calculation method is shown in the following formula 5: Formula 5 The Green Vegetation Index (GVI) is an index specifically used to reflect crop health. It evaluates the growth status of plants by analyzing the reflectance of the green band (GREEN). Healthy crops usually have stronger reflectance in the green band. Its calculation method is shown in the following formula 6: Formula 6 Through the above calculations, multiple vegetation index information corresponding to the initial area image to be processed can be calculated respectively. The multiple vegetation index information is used to provide a data basis in subsequent image processing.

[0041] In another specific implementation provided by the present application, the initial image of the area to be processed is updated according to the vegetation index information to generate at least one image of the area to be processed, including: Based on a preset segmentation size, the initial to-be-processed region image is segmented into a plurality of initial to-be-processed region sub-images; Determine a target initial sub-image of the region to be processed, wherein the target initial sub-image of the region to be processed is any one of a plurality of initial sub-images of the region to be processed; Three target vegetation index information are randomly determined from at least three vegetation index information, and three channels of the target initial to-be-processed area sub-image are updated according to the three target vegetation index information to obtain an to-be-processed area image corresponding to the target initial to-be-processed area sub-image.

[0042] In the method provided in the present application, after obtaining the vegetation index information corresponding to the initial image of the area to be processed, the vegetation index information can be used to update the initial image of the area to be processed, thereby generating at least one image of the area to be processed.

[0043] In practical applications, three values ​​of the RGB three channels of the initial image of the area to be processed are randomly selected from multiple vegetation index information to update the values, so as to perform subsequent data analysis. However, in order to refer to more vegetation index information in subsequent processing, the initial image of the area to be processed can be divided into multiple sub-images, and then updated with vegetation index information respectively. This allows more reference to multiple vegetation index information in multiple sub-images, making subsequent image analysis more comprehensive.

[0044] In the specific implementation provided in the present application, the initial region to be processed image is first divided into a plurality of initial region to be processed sub-images according to a preset segmentation size. For example, the preset segmentation size may be 640*640, and the initial region to be processed image is divided into a plurality of initial region to be processed sub-images according to the segmentation size.

[0045] After obtaining multiple initial sub-images of the area to be processed, the multiple initial sub-images of the area to be processed can be processed separately, and the processing between the multiple initial sub-images of the area to be processed can be implemented in parallel. In this embodiment, one of the target initial sub-images of the area to be processed is taken as an example for explanation, and the same method is used to process other initial sub-images of the area to be processed.

[0046] After determining the target initial sub-image of the area to be processed, three target vegetation index information are randomly selected from multiple vegetation index information, and the three target vegetation index information are used to update the values ​​of the RGB three channels in the target initial sub-image of the area to be processed. For example, the multiple vegetation index information includes NDVI, DVI, REI, LSI, VSI, and GVI. Randomly select three of them (such as [RED, NIR, NDVI], or [RED, VSI, NDVI]), replace the values ​​of the RGB three channels in the target initial sub-image of the area to be processed, and generate an image of the area to be processed composed of vegetation index information. The values ​​of the RGB three channels of the image of the area to be processed are composed of vegetation index information.

[0047] Step 106: input each image of the area to be processed into the pest detection model to obtain the predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information.

[0048] After obtaining multiple images of the area to be processed, the images of the area to be processed can be input into the pest detection model for processing. The pest detection model is trained to identify pest areas in the multispectral image based on the multispectral image. The pest detection model can extract and process the features of each image of the area to be processed, and identify the pest image area based on the vegetation index information in each image of the area to be processed.

[0049] In a specific embodiment provided in the present application, the pest detection model includes an embedding module, a feature extraction module, a feature fusion module and an output module; Inputting each area image to be processed into the pest detection model to obtain the predicted pest image area output by the pest detection model, including: Input each image of the region to be processed into the embedding module to obtain embedded image feature information corresponding to each image of the region to be processed; Input each embedded image feature information into the feature extraction module to obtain the initial image feature information output by the feature extraction module; Inputting each initial image feature information into the feature fusion module for multi-scale feature fusion to obtain fused image feature information output by the feature fusion module; The feature information of each fused image is input into the output module to obtain the predicted pest image area output by the output module.

[0050] In practical applications, the pest detection model includes an embedding module, a feature extraction module, a feature fusion module and an output module.

[0051] The pest detection model is used to process images. First, the image of the area to be processed must be embedded. The embedding module is a network layer used to convert the image into a feature vector that can be processed by the model. The image of the area to be processed is an image based on human vision and is an objective reflection of natural scenery. For computers, it needs to be converted into a language that can be processed by the computer, that is, the image of the area to be processed is input into the embedding module for processing to obtain the embedded image features output by the embedding module.

[0052] The main task of the feature extraction module is to extract the feature information of the input image of the area to be processed. In the method provided in the embodiment of the present application, CSPDarknet is used as the backbone network. CSPDarknet is a network structure based on the CSPNet concept, which can efficiently extract feature information of different scales from the image and improve the detection accuracy and speed.

[0053] The main task of the feature fusion module is to perform multi-scale feature fusion on the extracted feature information. In the method provided in the embodiment of the present application, HS-FPN is used as the network of the feature fusion module. In practical applications, HS-FPN includes two key parts: a feature selection unit and a feature fusion unit.

[0054] The feature selection unit uses channel attention and dimension matching mechanisms to screen feature maps of different scales. Through pooling operations (such as global average pooling, global maximum pooling, etc.) and weight calculation, the module effectively extracts important information in each channel.

[0055] The feature fusion unit combines the selected low-scale features with high-scale features through a selective feature fusion mechanism. After the high-scale features are expanded, they are resized through bilinear interpolation or transposed convolution and then fused with the low-scale features, thereby enhancing the model's ability to express pest characteristics.

[0056] For example, for a high-scale feature and a low-scale feature For high-scale features, we first use a transposed convolution with a size of 2 and a kernel size of 3*3 to expand them and obtain new features In order to unify the dimensions of high-scale features and low-scale features, bilinear interpolation is used to up- or down-sample high-scale features to obtain sampled scale features. Next, the channel attention mechanism is used to convert high-scale features into corresponding attention weights to filter low-scale features. After obtaining features with the same dimension, the filtered low-scale features are fused with the high-scale features to enhance the feature representation of the model. After feature fusion, the fused image feature information is obtained. , obtain the fused image feature information as shown in the following formula 7: Formula 7 After obtaining the fused image feature information, the fused feature information is input into the output module. After being processed by the output module, the pest detection model outputs the predicted pest image area.

[0057] In a specific embodiment provided in the present application, a method for training a pest detection model is also provided. Specifically, the pest detection model is trained and generated by the following steps: Acquire at least one sample area image corresponding to the sample detection area, wherein the sample area image includes a sample pest area corresponding to the sample vegetation; Performing image preprocessing on each sample area image to obtain at least one sample area image to be processed, wherein the sample area image to be processed includes vegetation index information corresponding to the sample vegetation; Inputting each sample area image to be processed into the pest detection model to obtain a predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information; Calculating a model loss value according to the predicted pest image area and the sample pest area; The model parameters of the pest detection model are adjusted according to the model loss value, and the pest detection model is continuously trained until a model training stop condition is reached.

[0058] In the training method provided in the embodiment of the present application, a multispectral image annotation tool is used to annotate the multispectral image to generate a sample area image. The sample area image refers to a multispectral image acquired by a multispectral camera. After the multispectral image is obtained, the pest target in the image is annotated using an annotation tool that supports multispectral images to obtain a sample area image.

[0059] After obtaining the sample region image, the sample region image is subjected to image preprocessing to obtain at least one sample region image to be processed. It should be noted that the process of performing image preprocessing on the sample region image is similar to the process of performing image preprocessing on each initial region image to obtain at least one region image to be processed. For details about the image preprocessing, please refer to the above-mentioned related content and will not be repeated here.

[0060] After obtaining at least one image of the sample area to be processed, the image of the sample area to be processed is input into the pest detection model for processing. The pest detection model makes a prediction based on the vegetation index information in the image of the sample area to be processed to obtain the predicted pest image area output by the pest detection model.

[0061] At this time, the pest detection model is not a trained model, so the predicted pest image area output by it is also inaccurate. It is necessary to further compare the sample pest area with the predicted pest image area to calculate the model loss value. The model parameters of the pest detection model are adjusted according to the model loss value, and the pest detection model is continuously trained until the model training stop condition is reached.

[0062] In the method provided in the embodiment of the present application, the HIoU loss function used to calculate the model loss value is specifically, the formula for calculating the model loss value is shown in the following formula 8: Formula 8 in, is the sample marking box of the pest area. x, y are the coordinates of the marking point of the sample marking box, and w, h are the width and height of the sample marking box. is the predicted marking box for the pest area, To predict the coordinates of the marking points of the marking box, The width and height of the predicted marker box. is the intersection width of the sample label box and the predicted label box, The intersection height of the sample label box and the predicted label box. It is the union area of ​​the sample label box and the predicted label box. , is the size of the minimum bounding box. for aspect ratio consistency. is the weight coefficient. is the weighted aspect ratio consistency.

[0063] The model loss value can be calculated by the above formula 8, the model parameters of the pest detection model can be adjusted according to the model loss value, and the pest detection model can be continuously trained until the model training stop condition is reached.

[0064] In the method provided in the embodiment of the present application, the stopping condition of the model training is that the model evaluation index reaches a preset threshold. In this embodiment, the method further includes: Calculating the prediction accuracy, prediction recall and precision mean according to the predicted pest image area and the sample pest area; The prediction accuracy, the prediction recall rate and the precision mean are used as model evaluation indicators of the pest detection model; When the model evaluation index reaches a preset threshold, it is determined that the pest detection model reaches a training stop condition.

[0065] The prediction accuracy rate indicates the ratio of samples with true positive values ​​to all samples predicted as positive examples. The calculation formula of the prediction accuracy rate is shown in the following formula 9: Formula 9 The prediction recall rate indicates the proportion of positive samples that are correctly predicted. The prediction recall rate calculation formula is shown in the following formula 10: Formula 10 Among them, TP (True Positive) represents the number of samples correctly predicted as positive, TN (True Negative) represents the number of samples correctly predicted as negative, FP (False Positive) represents the number of samples incorrectly predicted as positive (actually negative), and FN (False Negative) represents the number of samples incorrectly predicted as negative (actually positive).

[0066] Mean Average Precision (mAP) is a better indicator for measuring classification and positioning effects. mAP@0.5 means that when the IoU (Intersection over Union) is set to 0.5, the AP (Average Precision) of all images in each category is calculated, and then the average of all categories is calculated. The calculation formula of mAP is shown in the following formula 11: Formula 11 in, k Representative pest types, P i represents the prediction accuracy of each type of pest, and Ri represents the prediction recall of each type of pest.

[0067] In a specific embodiment provided in the present application, the learning rate of the pest detection model is set to 0.001, the batch size is 16, and the training rounds are 200 rounds. Transfer learning is performed using pre-trained weights. On the validation set, the mean accuracy of the model reaches 88%, the prediction accuracy reaches 85%, and the prediction recall reaches 86%. The detection speed of the trained pest detection model reaches 5 images per second, and a marking box of the predicted pest image area can be generated on the image, and the pest type and prediction confidence are marked at the same time.

[0068] Step 108: determining the target pest area corresponding to the vegetation to be detected in the target detection area according to each predicted pest image area.

[0069] The predicted pest image area output by the pest detection model is the pixel coordinates marked on the image. In order to better analyze the pests, the pixel coordinates can also be converted into geographic coordinates in the actual target detection area. Thus, the corresponding pest distribution map is generated. The pest distribution map is drawn in the geographic information system to visualize the pest information in the target detection area.

[0070] In another specific embodiment provided by the present application, the method further comprises: Acquire at least one historical pest area corresponding to the vegetation to be detected; According to each historical pest area and the target pest area, pest change trend information corresponding to the vegetation to be detected is determined.

[0071] In practical applications, pest detection can be performed periodically on the target detection area. For example, detection can be performed once a day, once a week, etc. Based on this, at least one historical pest area corresponding to the vegetation to be detected in the target detection area can also be obtained. Through the historical pest area and the target pest area detected at the current time, a comparison can be made to predict the pest change trend of the vegetation to be detected in the future. This helps technicians determine whether the vegetation to be detected will get better or worse in the future.

[0072] In practical applications, the image data of actual pest detection can also be annotated and used as training data to further continuously train the pest detection model so that the pest detection model can obtain better prediction results and improve the recognition ability of the model.

[0073] The method provided in the embodiment of the present application provides a pest detection method based on visual technology, which uses a multispectral sensor to collect rich spectral information. Combining image preprocessing and pest detection models, the accuracy of pest detection is improved. Real-time detection and early warning of pests are achieved. The fully automated pest detection method deployed on the server reduces dependence on manpower and reduces the cost of pest detection and prevention.

[0074] The following combination Figure 2 Taking the application of the pest detection method provided by the present application in the detection of pests of crops as an example, the pest detection method is further described. Figure 2 A processing flow chart of a pest detection method applied to a pest detection scenario of crops provided by an embodiment of the present application is shown, which specifically includes the following steps: Step 202: Obtain an electronic map of the target detection area, and generate flight information and image acquisition information of the UAV according to the electronic map.

[0075] Step 204: Send the flight information and image acquisition information to the drone, so that the drone flies based on the flight information and captures at least one initial area image corresponding to the target detection area based on the image acquisition information. The drone is equipped with a multispectral sensor, and the initial area image includes multispectral information.

[0076] Step 206: Receive at least one initial area image captured by the drone, and perform image correction and stitching on each initial area image to obtain an initial image to be processed area image.

[0077] Step 208: Obtain at least one vegetation index parameter corresponding to the initial image to be processed area image, and generate at least three vegetation index information based on each vegetation index parameter.

[0078] Step 210: Based on a preset segmentation size, segment the initial image to be processed area image into multiple initial image to be processed area sub-images.

[0079] Step 212: Randomly determine three target vegetation index information for each initial image to be processed area sub-image from at least three vegetation index information, and update the three channels of each initial image to be processed area sub-image with the three target vegetation index information to obtain at least one image to be processed area image.

[0080] Step 214: Input each image to be processed area image into the pest detection model to obtain the predicted pest image area output by the pest detection model, where the predicted pest image area is generated by the pest detection model according to each vegetation index information.

[0081] Step 216: Determine the target pest area corresponding to the vegetation to be detected in the target detection area according to each predicted pest image area.

[0082] Step 218: Obtain at least one historical pest area corresponding to the vegetation to be detected, and determine the pest change trend information corresponding to the vegetation to be detected according to each historical pest area and the target pest area.

[0083] The method provided in the embodiments of the present application provides a pest detection method based on vision technology, which uses a multispectral sensor to collect rich spectral information. Combining image preprocessing and a pest detection model improves the accuracy of pest detection. Realizes real-time detection and early warning of pests. A fully automated pest detection method deployed on a server reduces the dependence on manpower and lowers the cost of pest detection and control.

[0084] Corresponding to the above method embodiments, the present application also provides embodiments of a pest detection device. Figure 3 The structural schematic diagram of a pest detection device provided by an embodiment of the present application is shown. As Figure 3As shown, the device comprises: The image acquisition module 302 is configured to acquire at least one initial area image corresponding to the target detection area, wherein the target detection area includes vegetation to be detected, and the initial area image includes multispectral information; The image preprocessing module 304 is configured to perform image preprocessing on each initial region image to obtain at least one region image to be processed, wherein the region image to be processed includes vegetation index information corresponding to the vegetation to be detected; The model prediction module 306 is configured to input each to-be-processed area image into the pest detection model to obtain a predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information; The pest positioning module 308 is configured to determine the target pest area corresponding to the vegetation to be detected in the target detection area according to each predicted pest image area.

[0085] Optionally, the device further comprises an insect pest trend module configured to: Acquire at least one historical pest area corresponding to the vegetation to be detected; According to each historical pest area and the target pest area, pest change trend information corresponding to the vegetation to be detected is determined.

[0086] Optionally, the image preprocessing module 304 is further configured to: Perform image correction and splicing on each initial region image to obtain an initial region image to be processed; The vegetation index information corresponding to the initial image of the area to be processed is calculated, and the initial image of the area to be processed is updated according to the vegetation index information to generate at least one image of the area to be processed.

[0087] Optionally, the image preprocessing module 304 is further configured to: Acquire at least one vegetation index parameter corresponding to the initial image of the area to be processed; At least three vegetation index information are generated according to each vegetation index parameter.

[0088] Optionally, the image preprocessing module 304 is further configured to: Based on a preset segmentation size, the initial to-be-processed region image is segmented into a plurality of initial to-be-processed region sub-images; Determine a target initial sub-image of the region to be processed, wherein the target initial sub-image of the region to be processed is any one of a plurality of initial sub-images of the region to be processed; Three target vegetation index information are randomly determined from at least three vegetation index information, and three channels of the target initial to-be-processed area sub-image are updated according to the three target vegetation index information to obtain an to-be-processed area image corresponding to the target initial to-be-processed area sub-image.

[0089] Optionally, the pest detection model includes an embedding module, a feature extraction module, a feature fusion module and an output module; The model prediction module 306 is further configured to: Input each image of the region to be processed into the embedding module to obtain embedded image feature information corresponding to each image of the region to be processed; Input each embedded image feature information into the feature extraction module to obtain the initial image feature information output by the feature extraction module; Inputting each initial image feature information into the feature fusion module for multi-scale feature fusion to obtain fused image feature information output by the feature fusion module; The feature information of each fused image is input into the output module to obtain the predicted pest image area output by the output module.

[0090] Optionally, the device further includes a model training module configured to: Acquire at least one sample area image corresponding to the sample detection area, wherein the sample area image includes a sample pest area corresponding to the sample vegetation; Performing image preprocessing on each sample area image to obtain at least one sample area image to be processed, wherein the sample area image to be processed includes vegetation index information corresponding to the sample vegetation; Inputting each sample area image to be processed into the pest detection model to obtain a predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information; Calculating a model loss value according to the predicted pest image area and the sample pest area; The model parameters of the pest detection model are adjusted according to the model loss value, and the pest detection model is continuously trained until a model training stop condition is reached.

[0091] Optionally, the model training module is further configured to: Calculating the prediction accuracy, prediction recall and precision mean according to the predicted pest image area and the sample pest area; The prediction accuracy, the prediction recall rate and the precision mean are used as model evaluation indicators of the pest detection model; When the model evaluation index reaches a preset threshold, it is determined that the pest detection model reaches a training stop condition.

[0092] Optionally, the image acquisition module 302 is further configured to: At least one initial area image corresponding to the target detection area is acquired based on an image acquisition device carrying a multi-spectral sensor.

[0093] Optionally, the image acquisition device includes a drone; The image acquisition module 302 is further configured to: Obtaining an electronic map of the target detection area; Generating flight information and image acquisition information of the UAV according to the electronic map; Sending the flight information and the image acquisition information to the drone, so that the drone carrying the multispectral sensor flies based on the flight information, and captures at least one initial area image corresponding to the target detection area based on the image acquisition information; Receive at least one initial area image taken by the drone.

[0094] The device provided in the embodiment of the present application provides a pest detection method based on visual technology, which uses a multispectral sensor to collect rich spectral information. Combining image preprocessing and pest detection models, the accuracy of pest detection is improved. Real-time detection and early warning of pests are achieved. The fully automated pest detection method deployed on the server reduces dependence on manpower and reduces the cost of pest detection and prevention.

[0095] The above is a schematic scheme of an insect pest detection device of this embodiment. It should be noted that the technical scheme of the insect pest detection device and the technical scheme of the above insect pest detection method belong to the same concept, and the details not described in detail in the technical scheme of the insect pest detection device can be referred to the description of the technical scheme of the above insect pest detection method.

[0096] Figure 4 The block diagram of a computing device 400 according to an embodiment of the present application is shown. The components of the computing device 400 include but are not limited to a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0097] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0098] In one embodiment of the present application, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0099] The computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 400 may also be a mobile or stationary server.

[0100] The processor 420 is used to execute the following computer program / instructions, which implement the steps of the above-mentioned pest detection method when executed by the processor.

[0101] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above pest detection method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above pest detection method.

[0102] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the above-mentioned pest detection method when executed by a processor.

[0103] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned pest detection method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned pest detection method.

[0104] An embodiment of the present specification further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned pest detection method when executed by a processor.

[0105] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above-mentioned pest detection method belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical scheme of the above-mentioned pest detection method.

[0106] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0108] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0109] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0110] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A method for detecting insect pests, characterized in that: include: Acquire at least one initial area image corresponding to a target detection area, wherein the target detection area includes vegetation to be detected, and the initial area image includes multispectral information; Performing image preprocessing on each initial region image to obtain at least one region image to be processed, wherein the region image to be processed includes vegetation index information corresponding to the vegetation to be detected; Inputting the images of each area to be processed into the pest detection model to obtain the predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information; The target pest area corresponding to the vegetation to be detected is determined in the target detection area according to each predicted pest image area.

2. The method according to claim 1, characterized in that Also includes: Acquire at least one historical pest area corresponding to the vegetation to be detected; According to each historical pest area and the target pest area, pest change trend information corresponding to the vegetation to be detected is determined.

3. The method according to claim 1, characterized in that Performing image preprocessing on each initial region image to obtain at least one region image to be processed includes: Perform image correction and splicing on each initial area image to obtain an initial area image to be processed; The vegetation index information corresponding to the initial image of the area to be processed is calculated, and the initial image of the area to be processed is updated according to the vegetation index information to generate at least one image of the area to be processed.

4. The method according to claim 3, characterized in that Calculating vegetation index information corresponding to the initial area image to be processed, including: Acquire at least one vegetation index parameter corresponding to the initial image of the area to be processed; At least three vegetation index information are generated according to each vegetation index parameter.

5. The method according to claim 4, characterized in that The method of updating the initial image of the area to be processed according to the vegetation index information to generate at least one image of the area to be processed includes: Based on a preset segmentation size, the initial to-be-processed region image is segmented into a plurality of initial to-be-processed region sub-images; Determine a target initial sub-image of the region to be processed, wherein the target initial sub-image of the region to be processed is any one of a plurality of initial sub-images of the region to be processed; Three target vegetation index information are randomly determined from at least three vegetation index information, and three channels of the target initial to-be-processed area sub-image are updated according to the three target vegetation index information to obtain an to-be-processed area image corresponding to the target initial to-be-processed area sub-image.

6. The method according to claim 1, characterized in that The pest detection model includes an embedding module, a feature extraction module, a feature fusion module and an output module; Inputting each area image to be processed into the pest detection model to obtain the predicted pest image area output by the pest detection model, including: Input each image of the region to be processed into the embedding module to obtain embedded image feature information corresponding to each image of the region to be processed; Input each embedded image feature information into the feature extraction module to obtain the initial image feature information output by the feature extraction module; Inputting each initial image feature information into the feature fusion module for multi-scale feature fusion to obtain fused image feature information output by the feature fusion module; The feature information of each fused image is input into the output module to obtain the predicted pest image area output by the output module.

7. The method according to claim 1, characterized in that The pest detection model is trained and generated through the following steps: Acquire at least one sample area image corresponding to the sample detection area, wherein the sample area image includes a sample pest area corresponding to the sample vegetation; Performing image preprocessing on each sample area image to obtain at least one sample area image to be processed, wherein the sample area image to be processed includes vegetation index information corresponding to the sample vegetation; Inputting each sample area image to be processed into the pest detection model to obtain a predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information; Calculating a model loss value according to the predicted pest image area and the sample pest area; The model parameters of the pest detection model are adjusted according to the model loss value, and the pest detection model is continuously trained until a model training stop condition is reached.

8. The method according to claim 7, characterized in that Also includes: Calculating the prediction accuracy, the prediction recall and the precision mean according to the predicted pest image area and the sample pest area; The prediction accuracy, the prediction recall rate and the precision mean are used as model evaluation indicators of the pest detection model; When the model evaluation index reaches a preset threshold, it is determined that the pest detection model reaches a training stop condition.

9. The method according to claim 1, characterized in that Acquiring at least one initial area image corresponding to the target detection area includes: At least one initial area image corresponding to the target detection area is acquired based on an image acquisition device carrying a multi-spectral sensor.

10. The method according to claim 9, characterized in that The image acquisition device includes a drone; Acquiring at least one initial area image corresponding to the target detection area based on an image acquisition device carrying a multispectral sensor includes: Obtaining an electronic map of the target detection area; Generating flight information and image acquisition information of the UAV according to the electronic map; Sending the flight information and the image acquisition information to the drone, so that the drone carrying the multispectral sensor flies based on the flight information, and captures at least one initial area image corresponding to the target detection area based on the image acquisition information; Receive at least one initial area image taken by the drone.

11. A pest detection device, characterized in that: include: An image acquisition module is configured to acquire at least one initial area image corresponding to a target detection area, wherein the target detection area includes vegetation to be detected, and the initial area image includes multispectral information; An image preprocessing module is configured to perform image preprocessing on each initial region image to obtain at least one region image to be processed, wherein the region image to be processed includes vegetation index information corresponding to the vegetation to be detected; A model prediction module is configured to input each image of the area to be processed into the pest detection model to obtain a predicted pest image area output by the pest detection model, wherein the predicted pest image area is generated by the pest detection model according to each vegetation index information; The pest positioning module is configured to determine the target pest area corresponding to the vegetation to be detected in the target detection area according to each predicted pest image area.

12. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium storing a computer program / instruction, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images