An insect density estimation system

Abstract

A system for estimating insect density belongs to the field of artificial intelligence. The present invention solves the problem that existing insect density estimation methods require a lot of manpower and material resources, and the accuracy of estimation results is poor. The system of the present invention includes a monitoring device and a server, and an insect density estimation model is deployed on the server, and the insect density estimation model includes a target detection module and a density estimation module; the monitoring device is used to capture an environmental image and send the captured environmental image to the server; the target detection module is used to perform target detection on the captured environmental image, and the density estimation module is used to perform insect density estimation according to the target detection result. The present invention can be applied to insect density estimation.

Description

Technical Field

[0001] The invention belongs to the field of artificial intelligence, and in particular relates to an insect density estimation system. Background Art

[0002] At present, agriculture and forestry around the world suffer huge economic losses every year due to pest disasters. Pest disasters are a huge threat to agriculture and forestry. Severe pest disasters may cause significant losses to agriculture and forestry and even destroy the local ecological balance. Some pest disasters occur due to the failure to detect the signs of pests early. When the pests have already formed a scale, it is too late. The existing insect density estimation method requires professionals to conduct field surveys with professional equipment. This estimation method requires a lot of manpower and material resources, and the estimation results of this estimation method will be affected by subjective factors of personnel, resulting in poor accuracy of the estimation results and inability to effectively warn of pests. Summary of the invention

[0003] The purpose of the present invention is to propose an insect density estimation system to solve the problem that the existing insect density estimation method needs to consume a lot of manpower and material resources and the accuracy of the estimation result is poor.

[0004] The technical solution adopted by the present invention to solve the above technical problems is:

[0005] An insect density estimation system, the system comprising a monitoring device and a server, wherein an insect density estimation model is deployed on the server, and the insect density estimation model comprises a target detection module and a density estimation module;

[0006] The monitoring device is used to capture an environmental image and send the captured environmental image to a server;

[0007] The target detection module is used to perform target detection on the captured environment image, and the density estimation module is used to estimate the insect density according to the target detection result.

[0008] Furthermore, before performing target detection on the captured environment image, the captured environment image needs to be preprocessed;

[0009] Preprocessing refers to segmenting the captured environment image, normalizing each sub-image obtained by segmentation, and inputting each normalized sub-image into the target detection module.

[0010] Furthermore, the working process of the target detection module includes two stages: feature extraction and target detection;

[0011] For any sub-image after normalization, a feature map of the sub-image is extracted through the feature extraction stage, insects in the sub-image are identified through the target detection stage, and a target probability map is generated for each type of insect in the sub-image according to the insect identification result;

[0012] The target probability map is generated as follows:

[0013] For any insect category i for which insect density estimation is to be performed, a target probability map with the same shape as the sub-image feature map is initialized for the category. In the target probability map, the values ​​of the pixels in the bounding box area where the i-th insect is located are set to the probability that the insect in the bounding box area is predicted to be the i-th category, and the values ​​of the pixels in the non-bounding box area are set to α.

[0014] Furthermore, before using the target detection module, the target detection module needs to be trained first, and the loss function L used in the training is:

[0015] L=λ reg L reg +λ cls L cls

[0016] Among them, L reg is the bounding box coordinate regression loss, L cls is the insect category cross entropy loss, λ reg and λ cls is the weight factor.

[0017] Furthermore, the working process of the density estimation module is:

[0018] Step 1: For any sub-image after normalization, the feature map corresponding to the sub-image and the target probability map corresponding to the i-th insect are used as inputs of the density estimation module;

[0019] Step 2: Perform a linear transformation on the feature map to obtain a linear transformation result; then perform bit-by-bit multiplication on the linear transformation result and the target probability map corresponding to the i-th insect, and pass the result obtained by the bit-by-bit multiplication through the softmax activation function to obtain the attention score;

[0020] Perform matrix multiplication of the attention score and the linear transformation result to obtain an attention feature map; then perform residual connection between the feature map corresponding to the sub-image and the attention feature map, and input the result of the residual connection into the fully connected density predictor corresponding to the i-th insect to obtain the density of the i-th insect in the sub-image;

[0021] Step 3: for a plurality of sub-images from the same original image, average the densities of the i-th insects in the plurality of sub-images, and use the average value as the density of the i-th insects in the original image;

[0022] For all original images captured in a certain period of time, the density of the i-th insect in each original image is averaged to obtain the density of the i-th insect in the period of time;

[0023] Then, the maximum value is selected from the density of the i-th insect in each period of the day, and the selected maximum value is used as the density of the i-th insect for the whole day;

[0024] Step 4: Similarly, use the methods of steps 1 to 3 to obtain the density of each type of insect throughout the day.

[0025] Furthermore, when training the density estimation module, it is necessary to freeze all parameters of the object detection module and only update the parameters of the density estimation module;

[0026] The loss function used in the training of the density estimation module is:

[0027]

[0028] Among them, D′ i is the predicted density of the i-th insect, D i is the actual density of the i-th insect category, and N is the total number of insect categories.

[0029] Furthermore, the fully connected density predictor consists of a fully connected layer, and the output dimension of the fully connected density predictor is 1.

[0030] The beneficial effects of the present invention are:

[0031] The density estimation module of the method of the present invention takes the actual insect density as the target during training, learns the mapping relationship between the number of visible insects in the image and the actual insect density in the model, and then automatically estimates the actual insect density according to the number of visible insects in the image, thereby improving the accuracy of insect density estimation, reducing the dependence on human resources, improving the degree of automation of monitoring, and reducing the consumption of manpower and material resources. The method of the present invention can be widely used in insect density estimation in agriculture and forestry. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a structural schematic diagram of the insect density estimation system of the present invention;

[0033] Figure 2 It is a schematic diagram of converting a target detection result into a target probability map in the target detection module of the present invention;

[0034] Figure 3It is a structural schematic diagram of a density estimation module of the present invention;

[0035] Figure 4 It is a schematic diagram of the execution flow of the insect density estimation system of the present invention. DETAILED DESCRIPTION

[0036] The present application is further described in detail below through specific implementations in combination with the accompanying drawings. Obviously, the described implementations are only part of the implementations of the present invention, not all of the implementations. Based on the implementations in the present invention, other implementations obtained by ordinary technicians in the field without making creative work are all within the scope of protection of the present invention.

[0037] Specific implementation method 1: Combination Figure 1 The present embodiment is described as follows. The present embodiment describes an insect density estimation system, the system comprising a monitoring device and a server, and an insect density estimation model is deployed on the server, the insect density estimation model comprising a target detection module and a density estimation module;

[0038] The monitoring device is used to capture environmental images and send the captured environmental images to the server through manual copying or wireless / wired communication;

[0039] The target detection module is used to perform target detection on the captured environment image, and the density estimation module is used to estimate the insect density according to the target detection result.

[0040] Specific implementation method 2: This implementation method is a further limitation of specific implementation method 1. Before performing target detection on the captured environment image, the captured environment image needs to be preprocessed;

[0041] Preprocessing refers to segmenting the captured environment image, normalizing each sub-image obtained by segmentation, and inputting each normalized sub-image into the target detection module.

[0042] The other steps and parameters are the same as those in the first embodiment.

[0043] This embodiment divides the environment image into multiple sub-images to match the input size of the density estimation module, avoiding scaling or cropping the image, thereby improving the accuracy of the model. In addition, normalizing the sub-images can eliminate the scale and distribution differences of the data.

[0044] Specific implementation method three: This implementation method is a further limitation of specific implementation method two, and the working process of the target detection module includes two stages: feature extraction and target detection;

[0045] For any sub-image after normalization, a feature map of the sub-image is extracted through the feature extraction stage, insects in the sub-image are identified through the target detection stage, and a target probability map is generated for each type of insect in the sub-image according to the insect identification result;

[0046] The target probability map is generated as follows:

[0047] For any insect category i for which insect density estimation is to be performed, a target probability map with the same shape as the sub-image feature map is initialized for the category. In the target probability map, the values ​​of the pixels in the bounding box area where the i-th insect is located are set to the probability that the insect in the bounding box area is predicted to be the i-th category, and the values ​​of the pixels in the non-bounding box area are set to α.

[0048] The other steps and parameters are the same as those in the second specific implementation manner.

[0049] α is a positive number close to zero, such as 0.0001. For the overlapping area of ​​multiple bounding boxes, the pixel value is the accumulation of the insect category probability outputs corresponding to the multiple bounding boxes.

[0050] Specific implementation method 4: This implementation method is a further limitation of specific implementation method 3. Before using the target detection module, the target detection module needs to be trained first. The loss function L used in the training is:

[0051] L=λ reg L reg +λ cls L cls

[0052] Among them, L reg is the bounding box coordinate regression loss, L cls is the insect category cross entropy loss, λ reg and λ cls is the weight factor.

[0053] The target detection module of the present invention can use Faster-RCNN and YOLO models, but is not limited to the above models. Weight factor λ reg and λ cls It is used to balance the contributions of the two and update the model parameters through the back propagation algorithm.

[0054] The other steps and parameters are the same as those in the third specific implementation manner.

[0055] Specific implementation mode 5: This implementation mode is a further limitation of specific implementation mode 4. The working process of the density estimation module is as follows:

[0056] Step 1: For any sub-image after normalization, the feature map corresponding to the sub-image and the target probability map corresponding to the i-th insect are used as inputs of the density estimation module;

[0057] Step 2: Perform a linear transformation on the feature map to obtain a linear transformation result; then perform bit-by-bit multiplication on the linear transformation result and the target probability map corresponding to the i-th insect, and pass the result obtained by the bit-by-bit multiplication through the softmax activation function to obtain the attention score;

[0058] Perform matrix multiplication of the attention score and the linear transformation result to obtain the attention feature map; in order to prevent the model from forgetting the information contained in the original feature map, the feature map corresponding to the sub-image is residually connected with the attention feature map, and the result of the residual connection is input into the fully connected density predictor corresponding to the i-th insect to obtain the density of the i-th insect in the sub-image;

[0059] Step 3: for a plurality of sub-images from the same original image, average the densities of the i-th insects in the plurality of sub-images, and use the average value as the density of the i-th insects in the original image;

[0060] For all original images captured in a certain period of time, the density of the i-th insect in each original image is averaged to obtain the density of the i-th insect in the period of time;

[0061] Then, the maximum value is selected from the density of the i-th insect in each period of the day, and the selected maximum value is used as the density of the i-th insect for the whole day;

[0062] Step 4: Similarly, use the methods of steps 1 to 3 to obtain the density of each type of insect throughout the day.

[0063] This implementation sets a density predictor for each category of insects instead of using one density predictor to predict the density of all categories of insects. The advantage is that the density estimation module can be decoupled from the output category, which is convenient for adding or reducing the insect categories that need to estimate the density, and is more flexible.

[0064] This embodiment takes into account the influence of two factors: the error of a single shot and the large difference in the number of insect targets in images at different times due to insect habits, thereby avoiding estimation errors caused by different levels of activity of different insects at different time periods.

[0065] The other steps and parameters are the same as those in the fourth embodiment.

[0066] Specific implementation method 6: This implementation method is a further limitation of specific implementation method 5. When training the density estimation module, it is necessary to freeze all parameters of the target detection module and only update the parameters of the density estimation module.

[0067] The loss function used in the training of the density estimation module is:

[0068]

[0069] Among them, D′ i is the predicted density of the i-th insect, D i is the actual density of the i-th insect category, and N is the total number of insect categories.

[0070] The other steps and parameters are the same as those in the fifth embodiment.

[0071] The form of each data in the insect density estimation dataset is: an image taken by a monitoring device and the density of each category of insects at the time of shooting (the data is collected manually on the spot).

[0072] Specific implementation method seven: This implementation method is a further limitation of specific implementation method six. The fully connected density predictor is composed of a fully connected layer, and the output dimension of the fully connected density predictor is 1.

[0073] In this embodiment, the output of the fully connected density predictor is insect density.

[0074] Example 1

[0075] The present embodiment provides an insect density estimation system, which can automatically control the monitoring device to shoot high-definition insect images, extract features and classify insect images, and estimate insect density. The system includes a monitoring device and a server, and the monitoring device can transmit the captured images to the server. The server deploys an insect density estimation model based on target detection, and the insect density estimation model uses the feature map extracted by the target detection module and the target detection output to predict the insect density. To train the insect density estimation model, a target detection model must be trained first. In the density estimation model, the feature map, bounding box and target category of the image are first obtained through the target detection model, and the bounding box and the corresponding target category are converted into a probability map. Then, the feature map and the probability map are used as input to predict the insect density through the attention network. The insect density estimation module also includes a spatiotemporal density estimation strategy, which uses the average or maximum value method according to the insect density in different time periods to obtain a comprehensive insect density, while taking into account the influence of the error of a single shot and the large difference in the number of targets in different time images caused by the habits of insects. The embodiment of the present invention can automatically estimate the density of insects and provide effective data support for the monitoring and prevention of insects. It can reduce the manpower demand for pest control.

[0076] The schematic diagram of the insect density estimation system is shown in the figure. Figure 1 As shown in the figure, the execution process of the system is as follows Figure 4 As shown, the system specifically includes:

[0077] Monitoring equipment: Monitoring equipment can capture images at fixed time intervals and transmit the images to the server through manual copy or wireless / wired communication.

[0078] Server: The server deploys an insect density estimation model that can estimate the insect density in the environment based on the received images.

[0079] Insect Density Estimation Model:

[0080] 1) Implementation of image preprocessing module

[0081] The task of the image preprocessing module is to process the original image according to the data requirements of the target detection module.

[0082] Specifically, for example, if the input of the target detection module is 448×448, and the image size of the surveillance device is 1024×1024, it is cropped to 896×896 to match the input of the target detection module. It can be cropped by the center and the rest is discarded. The cropped 896×896 image is then divided into four sub-images, which are respectively sent to the subsequent modules for density estimation. The density estimation values ​​of the four sub-images are averaged as the density of the original image.

[0083] 2) Implementation of the target detection module

[0084] The target detection module uses a neural network model based on Faster-RCNN. The training data of the model includes agricultural and forestry environment images, and the data set contains target detection annotations of target insect categories. The training loss consists of bounding box coordinate regression loss and insect category cross entropy loss, and the calculation formula is as follows:

[0085] L=λ reg L reg +λ cls L cls

[0086] Among them, L reg is the bounding box coordinate regression loss, L cls is the insect category cross entropy loss. reg and λ cls is a weight factor used to balance the contributions of the two.

[0087] The target detection module also includes converting the target detection results into a target probability map. The specific conversion method is as follows: Figure 2As shown. First, a tensor with the same shape as the feature map is initialized for each insect category, and the initialized tensor is used as the target probability map of the corresponding category. For the bounding box selected area in the target probability map, the probability map value is the probability output of the predicted category. For the overlapping area of ​​multiple bounding boxes, the value is the accumulation of the insect category probability output corresponding to multiple bounding boxes. The value of the remaining area not in the bounding box is a positive number close to zero, such as 0.0001.

[0088] 3) Implementation of density estimation module

[0089] The specific implementation of the density estimation module is as follows Figure 3 As shown in Figure 1. The density estimation module receives the feature map output by the target detection module and the target probability map of a certain category as input. The attention mechanism is introduced through operations such as linear transformation, bitwise multiplication and residual connection. In the implementation of the attention mechanism, the feature map is first linearly transformed. The shape and size of the feature map remain unchanged before and after the linear transformation. Then, the linearly transformed feature map is bitwise multiplied with the target probability map. Then, the softmax activation function is used to calculate the attention score of the feature map. The attention score is matrix multiplied with the linearly transformed feature map to obtain the attention feature map. In order to prevent the model from forgetting the information contained in the original feature map, the initial feature map and the attention feature map are added bit by bit for residual connection. Finally, the result of the residual connection is input into the fully connected density predictor of the corresponding category to obtain the insect density of the corresponding category. Each target insect category corresponds to a fully connected density predictor.

[0090] When training the density estimation module, all parameters of the target detection module are frozen. The density estimation module updates parameters through the back propagation algorithm to minimize the mean square error between the predicted density and the true density. The specific training loss calculation formula is as follows:

[0091]

[0092] in is the density of the i-th insect predicted by the model, D i is the actual density, and N is the training batch size.

[0093] 4) Implementation of spatiotemporal density estimation module

[0094] The input of the spatiotemporal density estimation module is the estimated results of all insect density in a certain day, and the insect density estimation results are divided into fixed time periods (such as half an hour). In each divided time period, the insect density results are averaged as the insect density corresponding to the time period. The maximum value of the average insect density estimation value of all time periods is taken as the density estimation result of the day. Splitting the insect density estimation results and taking the average value helps smooth noise and improve the stability of density estimation. At the same time, the final result selects the maximum value of the average density of each time period of the day, which helps to capture the peak period of insect activity.

[0095] Figure 4 The following is a schematic diagram of the execution flow of the intelligent system for estimating insect density according to the present embodiment, including:

[0096] S101: The monitoring device regularly captures images of the surrounding environment.

[0097] S102: The monitoring device sends the taken photos to the server.

[0098] S103: The server calls the image preprocessing module to preprocess the image.

[0099] S104: The server calls the target detection module to obtain the feature map and target probability map.

[0100] S105: The server calls the density estimation module to estimate the insect density based on the feature map and the target probability map.

[0101] S106: The server will call the spatiotemporal density estimation module to calculate the average density of each time period and the density of each type of insect on that day based on all density estimation results of the day.

[0102] Because the actual number of insects is generally greater than the number of visible insects in the image, this embodiment does not calculate the insect density only through the recognition results of target detection. In the training of the density estimation module, this embodiment takes the actual insect density as the target and learns the mapping relationship between the number of visible insects and the actual insect density in the model. The model learns to estimate the actual insect density based on the number of visible insects and environmental information. In addition, compared to estimating the insect density based on the results of only one shot, this embodiment calculates the average insect density of all shooting results for a period of time, reducing the estimation error. For multiple time periods within a day, the maximum value is taken, which can avoid the underestimation of insect density in most of the time due to the insect activity habits, thereby improving the accuracy of insect density estimation. This embodiment can be widely used in agriculture, forestry and other fields, reducing the dependence on human resources and improving the degree of automation of monitoring.

[0103] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. An insect density estimation system, characterized in that: The system includes a monitoring device and a server, and an insect density estimation model is deployed on the server, and the insect density estimation model includes a target detection module and a density estimation module; The monitoring device is used to capture an environmental image and send the captured environmental image to a server; The target detection module is used to perform target detection on the captured environment image, and the density estimation module is used to estimate the insect density according to the target detection result; Before performing target detection on the captured environment image, the captured environment image needs to be preprocessed; Preprocessing refers to segmenting the captured environment image, normalizing each sub-image obtained by segmentation, and inputting each normalized sub-image into the target detection module; The working process of the target detection module includes two stages: feature extraction and target detection; For any sub-image after normalization, a feature map of the sub-image is extracted through the feature extraction stage, insects in the sub-image are identified through the target detection stage, and a target probability map is generated for each type of insect in the sub-image according to the insect identification result; The target probability map is generated as follows: For any insect category i for which insect density estimation is to be performed, a target probability map with the same shape as the sub-image feature map is initialized for the category. In the target probability map, the values ​​of the pixels in the bounding box area where the i-th insect is located are set to the probability that the insect in the bounding box area is predicted to be the i-th category, and the values ​​of the pixels in the non-bounding box area are set to α; The working process of the density estimation module is as follows: Step 1: For any sub-image after normalization, the feature map corresponding to the sub-image and the target probability map corresponding to the i-th insect are used as inputs of the density estimation module; Step 2: Perform a linear transformation on the feature map to obtain a linear transformation result; Then, the linear transformation result is multiplied bit by bit with the target probability map corresponding to the i-th insect, and the result of the bit-by-bit multiplication is passed through the softmax activation function to obtain the attention score; Perform matrix multiplication of the attention score and the linear transformation result to obtain the attention feature map; Then, the feature map corresponding to the sub-image is residually connected with the attention feature map, and the result of the residual connection is input into the fully connected density predictor corresponding to the i-th insect to obtain the density of the i-th insect in the sub-image; Step 3: for a plurality of sub-images from the same original image, average the densities of the i-th insects in the plurality of sub-images, and use the average value as the density of the i-th insects in the original image; For all original images captured in a certain period of time, the density of the i-th insect in each original image is averaged to obtain the density of the i-th insect in the period of time; Then, the maximum value is selected from the density of the i-th insect in each period of the day, and the selected maximum value is used as the density of the i-th insect for the whole day; Step 4: Similarly, use the methods of steps 1 to 3 to obtain the density of each type of insect throughout the day.

2. An insect density estimation system according to claim 1, characterized in that: Before using the target detection module, the target detection module needs to be trained first. The loss function L used in the training is: L=λ reg L reg +λ cls L cls Among them, L reg is the bounding box coordinate regression loss, L cls is the insect category cross entropy loss, λ reg and λ cls is the weight factor.

3. An insect density estimation system according to claim 2, characterized in that: When training the density estimation module, you need to freeze all the parameters of the object detection module and only update the parameters of the density estimation module. The loss function used in the training of the density estimation module is: Among them, D′ i is the predicted density of the i-th insect, D i is the actual density of the i-th insect category, and N is the total number of insect categories.

4. An insect density estimation system according to claim 3, characterized in that: The fully connected density predictor consists of a fully connected layer, and the output dimension of the fully connected density predictor is 1.

Images