A Vector Detection Method and System under Multiple View Distances

Abstract

The present invention relates to a method for detecting disease vectors at multiple viewing distances, including: obtaining an image data set of multiple target regions at different viewing distances; constructing a convolutional neural network with multi-feature activation, and training to obtain a self-calibration model for disease vector images; obtaining the target regions corresponding to the calibrated disease vector images, and correcting and dividing them into a training set and a validation set; training with improved YOLOv5 to obtain the best disease vector detection model during the training process; obtaining the quantized best disease vector detection model; using the quantized best disease vector detection model to detect disease vector images, identifying the categories and accuracy rates of disease vectors, and realizing the identification of disease vectors. The present invention also discloses a disease vector detection system at multiple viewing distances. The present invention is applicable to the identification of disease vectors at multiple viewing distances, can accurately identify the categories of disease vectors without scene constraints, has high robustness; reduces the cost and time of the identification process, and the accuracy rate of the identification results is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of vector biological image processing, and in particular to a method and system for detecting vector organisms at multiple viewing distances. Background Art

[0002] Currently, in accordance with the unified deployment of the General Administration, the work of monitoring and identifying vector organisms at national border ports is being carried out within each directly affiliated customs area, mainly realizing the identification of vector organisms such as rats, mosquitoes, flies, cockroaches, fleas, ticks, mites, midges, etc. The traditional process of identifying vector organisms is divided into links such as on-site sample capture, sample transportation, and laboratory manual identification. This method has a long identification cycle, high cost, and low accuracy. Therefore, it is urgent to break through the traditional method of identifying vector organisms and use a mobile application platform to achieve intelligent, real-time, and accurate identification of vector organisms.

[0003] For the image processing of collected vector organisms, advanced deep learning can be combined to achieve the detection and recognition of vector organism targets. Currently, the more popular object detectors are divided into two categories: single-stage and two-stage object detectors. Among them, the two-stage object detector involves generating region proposals and classification. This type of model can improve the detection performance of small targets, but the complexity of generating region proposals is high, and the inference time in practical applications is ignored; the single-stage object detector does not need to generate region proposals, directly predicts the target category and locates the target position when extracting target features, runs faster, and has slightly lower accuracy than the two-stage object detector.

[0004] YOLOv5 is a single-stage object detector with potentially good performance. However, for the images of vector organisms with small targets, it is still difficult to achieve better accuracy at different viewing distances in an unconstrained scenario. In addition, so far, for the identification of vector organisms, there has not been a method of transplanting a multi-viewing-distance vector organism target detection model into a vector organism detection system to achieve intelligent identification of vector organisms. Summary of the Invention

[0005] The primary object of the present invention is to provide a method for detecting vector organisms at multiple viewing distances, which can detect vector organisms at different viewing distances, especially has good robustness to the images of vector organisms with small targets or under unconstrained scenarios, and has a higher accuracy of the identification result.

[0006] To achieve the above object, the present invention adopts the following technical solutions: A method for detecting vector organisms at multiple viewing distances, the method comprising the following steps in sequence:

[0007] (1) Collect images of different types of vector organism samples, perform preprocessing, and obtain an image data set with multi-target regions at different viewing distances;

[0008] (2)Determine the labels for all categories of vector-borne organisms, label a small portion of the vector-borne organism images in the image dataset of each category of vector-borne organisms to obtain the labeled image dataset, construct a convolutional neural network with multi-layer feature activation, input the labeled image dataset into the convolutional neural network with multi-layer feature activation, fuse the feature maps activated in each layer, and train to obtain a self-calibration model for vector-borne organism images; use the self-calibration model for vector-borne organism images to self-calibrate the remaining images in the image dataset of multi-target regions at different viewing distances, obtain the target regions corresponding to the calibrated vector-borne organism images, and correct the target regions. Divide the corrected calibrated image dataset into a training set and a validation set according to a ratio of 8:2;

[0009] (3)Replace the backbone network module of YOLOv5 with a densely connected network module with multi-layer feature activation mapping to obtain an improved YOLOv5 network model, train the training set and the validation set to obtain the best vector-borne organism detection model during the training process, and store the data nodes and weights of the best vector-borne organism detection model as a ".pt" format file;

[0010] (4)Use the anti-quantization formula for the nodes and weights of the best vector-borne organism detection model, calculate matrix convolution, export the quantized best vector-borne organism detection model, and store the data nodes and weights of the quantized best vector-borne organism detection model as a ".tflite" format file;

[0011] (5)Call the data nodes and weights of the quantized best vector-borne organism detection model in the ".tflite" format file, perform inference analysis on the vector-borne organism image to be detected, and output the identified vector-borne organism category and accuracy rate to achieve vector-borne organism identification.

[0012] The specific steps of step (1) are as follows:

[0013] (1a)For the target region in the image of the vector-borne organism sample , use edge feature extraction to obtain all coordinate points in the target region ;

[0014] (1b)Perform preprocessing, where the preprocessing refers to geometric transformations such as enlargement, reduction, lighting, rotation, cropping, translation, and compounding and bilinear interpolation processing, and map all coordinate points in the target region to the coordinates in the new target region in the image , to obtain an image with multi-target regions;

[0015] (1c) Repeat step (1b) until all the images to be processed are traversed, obtaining a dataset of vector-borne disease images with multi-object regions at different viewing distances.

[0016] The specific steps of step (2) are as follows:

[0017] (2a) For the image dataset of multi-object regions at different viewing distances, determine the labels of all categories of vector-borne diseases, and calibrate a small part of the vector-borne disease images in the image dataset of each category of vector-borne diseases, obtaining a calibrated image dataset;

[0018] (2b) Construct a convolutional neural network with multi-layer feature activation, and perform upsampling on the output of the pooling layer of the th layer to obtain an image with the same resolution as the calibrated image dataset;

[0019] (2c) Perform an accumulation operation on the output after upsampling of the th pooling layer to obtain a heat map with multi-layer feature activation:

[0020]

[0021] Among them, represents the feature map of the th upsampled output of the pooling layer with a target feature region; represents the upsampling factor of the th pooling layer; represents the number of feature maps of the th pooling layer; represents the number of the

[0022] (2d) Combine the heat map and the collected vector-borne disease images to locate the edge features of the target region of the vector-borne disease, and calculate the best bounding box of the target region according to the edge features;

[0023] (2e) Calibrate a small part of the preprocessed image dataset, input it into the convolutional neural network with multi-layer feature activation, and train to obtain a self-calibration model for vector-borne diseases;

[0024] (2f) Use the self-calibration model for vector-borne diseases to perform self-calibration on the remaining images in the image dataset of multi-object regions at different viewing distances, obtain the target regions of the remaining vector-borne disease images, and perform correction;

[0025] (2g) Divide all the calibrated vector-borne disease image datasets after correction into a training set and a validation set according to a ratio of 8:2.

[0026] The specific steps of step (3) are as follows:

[0027] (3a) Construct a densely connected network module, and cascade the features of the layer. Then the output feature map of the layer is:

[0028]

[0029] Wherein, is a non-linear transformation function, is the feature map after cascading the

[0030] (3b) Replace the backbone network in the YOLOv5 network with the densely connected network module to obtain an improved YOLOv5 network model;

[0031] (3c) Input the calibrated training set and validation set of vector-borne disease images, and use the improved YOLOv5 network model for training to obtain the best vector-borne disease detection model.

[0032] In the step (4), the data nodes and weights of the best vector-borne disease detection model are adopted by the dequantization formula, and the specific calculation of the matrix convolution means: according to the data nodes and weights of the best vector-borne disease detection model obtained by training, for the input quantized tensor and the convolution kernel , use the dequantization formula to calculate the matrix convolution, and infer the quantized output as:

[0033]

[0034] Wherein, , , are zero values used to align floating-point and quantized values, is the scaling factor, is the number of data nodes included in the best vector-borne disease detection model, , and are all weight indices.

[0035] Another object of the present invention is to provide a system for a vector-borne disease detection method under multiple viewing distances, including:

[0036] An image and video acquisition module, which is used to collect the data information of vector-borne disease images, including selecting from the album, taking pictures with the camera, and video recording data of the camera;

[0037] An image preprocessing module, which performs cropping, rotation, label conversion, and video image frame conversion on vector-borne disease images;

[0038] The vector-borne disease detection module performs inference and analysis on the preprocessed image input in the quantized optimal vector-borne disease detection model to obtain the target category and accuracy rate.

[0039] The identification result visualization module, for the vector-borne disease detection under the camera function, draws the target area, Chinese label of the category, and category accuracy rate of the vector-borne disease. For the vector-borne disease detection under the album and photo-taking functions, it outputs the Chinese label, Latin label, and category accuracy rate of the vector-borne disease target.

[0040] The exit module is used to exit the camera video and the vector-borne disease detection system.

[0041] As can be seen from the above technical solutions, the beneficial effects of the present invention are as follows: First, the present invention combines mobile replication to enhance vector-borne disease image data, multi-layer feature activation mapping self-calibration, and an improved YOLOv5 network model, and can detect vector-borne diseases at different viewing distances, especially having good robustness for vector-borne disease images with small targets or without scene constraints. Second, for the identification of vector-borne diseases, compared with the traditional process of vector-borne disease identification, the present invention can identify vector-borne diseases in real time, reduce the cost and time in the identification process, and has a higher accuracy rate of the identification result compared with manual identification. Third, the actual mobile terminal detection results of vector-borne diseases of the present invention show that at multiple viewing distances, the present invention can accurately identify the categories of vector-borne diseases. Description of the Drawings

[0042] Figure 1 is the method flow chart of the present invention;

[0043] Figure 2 is the system structure diagram of the present invention;

[0044] Figure 3 is the heat map of multi-layer feature activation of vector-borne diseases in the present invention;

[0045] Figure 4 is the P-R curve graph of the training result of the vector-borne disease detection model in the present invention;

[0046] Figure 5 is the confusion matrix graph of the test set of the optimal vector-borne disease detection model in the present invention;

[0047] Figure 6 is the result graph of the quantized optimal vector-borne disease detection model in the present invention for detecting vector-borne disease images at different viewing distances;

[0048] Figure 7 is the result graph of the model in the present invention for identifying vector-borne diseases under the album and photo-taking functions of the mobile Android platform;

[0049] Figure 8This is the result diagram of identifying vector organisms using the model under the camera function of the Android platform on the mobile side in the present invention. Detailed implementation manners

[0050] As Figure 1 shown, a method for detecting vector organisms at multiple viewing distances, the method includes the following steps in sequence:

[0051] (1) Collect images of different types of vector organism samples, perform preprocessing, and obtain an image data set with multi-target regions at different viewing distances;

[0052] (2) Determine the labels of all categories of vector organisms, calibrate a small part of the vector organism images in the image data set of each category of vector organisms, obtain the calibrated image data set, construct a convolutional neural network with multi-layer feature activation, input the calibrated image data set into the convolutional neural network with multi-layer feature activation, fuse the feature maps activated by each layer, and train to obtain a self-calibration model for vector organism images; use the self-calibration model for vector organism images to self-calibrate the remaining images in the image data set of multi-target regions at different viewing distances, obtain the target regions corresponding to the calibrated vector organism images, and correct the target regions. Divide the corrected calibrated image data set into a training set and a validation set according to a ratio of 8:2;

[0053] (3) Replace the backbone network module of YOLOv5 with a densely connected network module with multi-layer feature activation mapping to obtain an improved YOLOv5 network model, train the training set and the validation set to obtain the best vector organism detection model during the training process, and store the data nodes and weights of the best vector organism detection model as a ".pt" format file;

[0054] (4) Use the anti-quantization formula for the nodes and weights of the best vector organism detection model, calculate matrix convolution, export the quantized best vector organism detection model, and store the data nodes and weights of the quantized best vector organism detection model as a ".tflite" format file;

[0055] (5) Call the data nodes and weights of the quantized best vector organism detection model in the ".tflite" format file, perform inference analysis on the vector organism image to be detected, and output the identified vector organism category and accuracy rate to achieve vector organism identification.

[0056] The step (1) specifically includes the following steps:

[0057] (1a) For the target region in the image of the vector organism sample , use edge feature extraction to obtain all coordinate points in the target region ;

[0058] (1b) Perform preprocessing, where the preprocessing refers to geometric transformations such as magnification, reduction, illumination, rotation, cropping, translation, and compounding and bilinear interpolation processing to map all coordinate points in the target area to coordinates in the new target area in the new image, obtaining an image with multiple target areas ; ;

[0059] (1c) Repeat step (1b) until all the images to be processed are traversed, obtaining a dataset of vector-borne disease images with multiple target areas at different viewing distances.

[0060] The specific steps of step (2) include the following steps:

[0061] (2a) For the image dataset with multiple target areas at different viewing distances, determine the labels of all categories of vector-borne diseases, and calibrate a small part of the vector-borne disease images in the image dataset of each category, obtaining a calibrated image dataset;

[0062] (2b) Construct a convolutional neural network with multi-layer feature activation, and perform upsampling on the output of the pooling layer of the layer to obtain an image with the same resolution as the calibrated image dataset;

[0063] (2c) Accumulate and calculate the outputs after upsampling of the pooling layers to obtain a heat map with multi-layer feature activation:

[0064]

[0065] where represents the feature map of the th pooling layer output after upsampling with a target feature area; represents the upsampling factor of the th pooling layer; represents the number of feature maps of the th pooling layer;

[0066] (2d) Combine the heat map and the collected vector-borne disease images to locate the edge features of the vector-borne disease target area, and calculate the best bounding box of the target area according to the edge features;

[0067] (2e) Calibrate a small part of the preprocessed image dataset, input it into the convolutional neural network with multi-layer feature activation, and train to obtain a self-calibration model for vector-borne diseases;

[0068] (2f) Use the vector-borne organism self-calibration model to perform self-calibration on the remaining images in the image dataset of multi-target regions at different viewing distances, obtain the target regions of the remaining vector-borne organism images, and make corrections;

[0069] (2g) Divide all the calibrated vector-borne organism image datasets after correction into a training set and a validation set according to a ratio of 8:2.

[0070] The specific steps of step (3) include the following steps:

[0071] (3a) Construct a densely connected network module, and cascade the features at the layer. Then the output feature map of the layer is:

[0072]

[0073] where is a non-linear transformation function, is the feature map after cascading at the

[0074] (3b) Replace the backbone network in the YOLOv5 network with a densely connected network module to obtain an improved YOLOv5 network model;

[0075] (3c) Input the calibrated vector-borne organism image training set and validation set, and use the improved YOLOv5 network model for training to obtain the best vector-borne organism detection model.

[0076] In step (4), calculating the matrix convolution by using the anti-quantization formula for the data nodes and weights of the best vector-borne organism detection model specifically means: According to the data nodes and weights of the best vector-borne organism detection model obtained by training, for the input quantized tensors and the convolution kernel , calculate the matrix convolution using the anti-quantization formula, and infer the quantized output as:

[0077]

[0078] where , , are zero values used to align floating-point and quantized values, is the scaling factor, is the number of data nodes included in the best vector-borne organism detection model, , and are all weight indices.

[0079] As shown Figure 2 in the figure, the present system includes:

[0080] An image and video acquisition module, which is used to acquire the data information of the vector-borne organism images, including selecting from the album, taking pictures with the camera, and recording video data with the camera;

[0081] An image preprocessing module, which performs cropping, rotation, label conversion, and video image frame conversion on the vector-borne organism images;

[0082] A vector-borne organism detection module, which inputs the preprocessed images into the quantized optimal vector-borne organism detection model for inference and analysis to obtain the target category and accuracy rate;

[0083] An identification result visualization module, for the vector-borne organism detection under the camera function, draws the target area of the vector-borne organism, the Chinese label of the category, and the category accuracy rate. For the vector-borne organism detection under the album and photo-taking functions, outputs the Chinese label, Latin label, and category accuracy rate of the vector-borne organism target;

[0084] An exit module, which is used to exit the camera video and the vector-borne organism detection system.

[0085] As shown Figure 3 in the figure, the present invention can locate the vector-borne organism area. The vector-borne organism detection model trained by the present invention has an average precision of 0.963 for all categories when the IoU (the ratio of the intersection to the union of the predicted bounding box and the true bounding box) threshold is greater than 0.5. As shown Figure 4 in the figure, for 17 categories of vector-borne organisms, namely Aedes stimulans, Parasarcophaga albiceps, Periplaneta americana, Periplaneta australasiae, Lucilia bazini, Armigeres subalbatus, Lucilia cuprina, Chrysomya megacephala, Periplaneta fuliginosa, Blattella germanica, Helicophagella melanura, Musca domestica, Musca pattoni, Blattella rhombifolia, Lucilia sericata, Musca sorbens, and Chrysomya wiedemann, they are sequentially labeled as aedes, albiceps, americana, australasiae, bazini, coquillett, cuprina, fabricius, fuliginosa, gblattella, melanura, musca, pattoni, rhombifolia, sericata, sorbens, and wiedemann. There are a total of 1622 data sets, which are divided into a training set and a test set according to 8:2, the epoch is 300, and the PR result curve in the process of training the vector-borne organism detection model. As shown Figure 5 in the figure, for the detection of Periplaneta americana, Periplaneta australasiae, Periplaneta fuliginosa, and Blattella germanica within the same genus, it has a relatively high accuracy rate, which can avoid the cross-interference of the target characteristics of vector-borne organisms within the same genus. As shown Figure 6 in the figure, by comparing the true category labels of the vector-borne organisms, the optimal vector-borne organism detection model can accurately detect the categories of vector-borne organisms in all images.

[0086] The present invention combines multi-layer feature activation self-calibration and an improved YOLOv5 network model to obtain effective activation feature maps of vector pests at different viewing distances. Under the constraints of small targets or unconstrained scenarios, a vector pest detection model with better robustness is obtained, effectively suppressing the cross-interference of the characteristics of vector pests of the same species.

[0087] Figure 7 Load the optimized vector pest detection model onto the Android platform and test the vector pest detection result images in two ways: album and taking pictures, to obtain the Chinese and Latin names and accuracy rates of the detected vector pest categories. Figure 8 To test the vector pest detection result images without scene constraints under the camera function on the Android mobile platform, the vector pests can be identified in real time without manual intervention, and the vector pest category name and detection accuracy rate are given.

[0088] In summary, the present invention is applicable to the identification of vector pests at multiple viewing distances, can accurately identify the vector pest categories without scene constraints, and has high robustness; the present invention can identify vector pests in real time, reduce the cost and time of the identification process, and has a higher accuracy rate of the identification results compared with manual identification; the actual vector pest mobile detection results of the present invention show that at multiple viewing distances, the present invention can accurately identify the vector pest categories.

Claims

1. A vector-borne disease detection method under multiple viewing distances, characterized in that: The method includes the following steps in sequence: (1) Collect images of different types of vector organisms, perform preprocessing, and obtain an image dataset with multi-object regions at different viewing distances; (2) Determine the labels of all categories of vector organisms, calibrate a small part of the vector organism images in the image dataset of each category of vector organisms to obtain a calibrated image dataset, construct a convolutional neural network with multi-layer feature activation, input the calibrated image dataset into the convolutional neural network with multi-layer feature activation, fuse the feature maps activated in each layer, and train to obtain a self-calibration model for vector organism images; use the self-calibration model for vector organism images to self-calibrate the remaining images in the image dataset of multi-object regions at different viewing distances, obtain the target regions corresponding to the calibrated vector organism images, and correct the target regions. Divide the corrected calibrated image dataset into a training set and a validation set according to a ratio of 8:2; (3) Replace the backbone network module of YOLOv5 with a densely connected network module with multi-layer feature activation mapping to obtain an improved YOLOv5 network model, train the training set and the validation set to obtain the best vector organism detection model during the training process, and store the data nodes and weights of the best vector organism detection model as a ".pt" format file; (4) Use the anti-quantization formula for the nodes and weights of the best vector organism detection model, calculate matrix convolution, export the quantized best vector organism detection model, and store the data nodes and weights of the quantized best vector organism detection model as a ".tflite" format file; (5) Call the data nodes and weights of the quantized best vector organism detection model in the ".tflite" format file, perform inference analysis on the vector organism image to be detected, and output the identified vector organism category and accuracy rate to achieve vector organism identification; In step (4), the data nodes and weights of the optimal vector-borne disease detection model are used in the dequantization formula to calculate the matrix convolution, specifically referring to: based on the data nodes and weights of the optimal vector-borne disease detection model obtained through training, for the input quantized tensor and the convolution kernel , the matrix convolution is calculated using the dequantization formula, and the quantized output is: ; Among them, , , are zero values used to align floating-point and quantization values, is the scaling factor, is the number of data nodes included in the optimal vector-borne disease detection model, , and are all weight indices.

2. The method for detecting vector organisms under multiple viewing distances according to claim 1, wherein: The specific steps of step (1) include the following steps: (1a) For the image of the vector biological sample in the target area , use edge feature extraction to obtain all coordinate points in the target area ; (1b) Perform preprocessing, where the preprocessing refers to geometric transformations of magnification, reduction, illumination, rotation, cropping, translation, and composition and bilinear interpolation processing to map all coordinate points in the target region to coordinates in a new target region in the image, obtaining an image with multiple target regions ; ; (1c) Repeat step (1b) until all images to be processed are traversed to obtain a vector organism image dataset with multi-object regions at different viewing distances.

3. The vector biological detection method under multiple viewing distances according to claim 1, wherein: The specific steps of step (2) include the following steps: (2a) For the image dataset of multi-object regions at different viewing distances, determine the labels of all categories of vector organisms, calibrate a small part of the vector organism images in the image dataset of each category of vector organisms to obtain a calibrated image dataset; (2b) Construct a convolutional neural network that activates multi-layer features, and perform upsampling on the output of the pooling layer of the layer to obtain an image with the same resolution as the calibrated image dataset; ​ (2c) Accumulate the outputs after upsampling of pooling layers to obtain a heat map of multi-layer feature activation: ; Among them, represents the th feature map with a target feature region after upsampling the output of the th pooling layer; represents the upsampling factor of the th pooling layer; represents the number of feature maps of the th pooling layer; (2d) Combine the heat map and the collected vector organism image to locate the edge features of the vector organism target region, and calculate the best bounding box of the target region according to the edge features; (2e) Calibrate the preprocessed small part of the image dataset and input it into the convolutional neural network with multi-layer feature activation to train and obtain a self-calibration model for vector organisms; (2f) Use the self-calibration model for vector organisms to self-calibrate the remaining images in the image dataset of multi-object regions at different viewing distances to obtain the target regions of the remaining vector organism images and correct them; (2g) Divide the corrected calibrated vector organism image dataset into a training set and a validation set according to a ratio of 8:

2.

4. The vector biological detection method under multiple viewing distances according to claim 1, wherein: The specific steps of step (3) include the following steps: (3a) Construct a densely connected network module, and at the hierarchical cascade features of the layer, then the output feature map of the layer is: ; Among them, is a non-linear conversion function, is the feature map after hierarchical concatenation; (3b) Replace the backbone network in the YOLOv5 network with a densely connected network module to obtain an improved YOLOv5 network model; (3c) Input the calibrated training set and validation set of vector-borne disease images, and use the improved YOLOv5 network model for training to obtain the best vector-borne disease detection model.

5. A system for implementing the method for detecting vectors at multiple viewing distances according to any one of claims 1 to 4, characterized in that, It includes: An image and video acquisition module, which is used to collect the data information of vector-borne disease images, including selecting from the album, taking pictures with the camera, and recording video data with the camera; An image preprocessing module, which crops, rotates, converts labels, and converts video image frames of vector-borne disease images; A vector-borne disease detection module, which inputs the preprocessed image into the quantized best vector-borne disease detection model for inference and analysis to obtain the target category and accuracy; An identification result visualization module, which for the vector-borne disease detection under the camera function, draws the target area of the vector-borne disease, the Chinese label of the category, and the category accuracy. For the vector-borne disease detection under the album and camera functions, it outputs the Chinese label, Latin label, and category accuracy of the vector-borne disease target; An exit module, which is used to exit the camera video and the vector-borne disease detection system.

Images