Small target identification method and oncomelania identification method based on image processing

Abstract

The invention discloses a small target identification method and oncomelania identification method based on image processing, and the method comprises the following steps: S1, obtaining a to-be-identified small target image under visible light, and carrying out the marking through a man-machine combination mode of machine marking and manual marking; s2, performing random data enhancement on the marked small target image; performing sliding window image cutting processing on the small target image after data enhancement; cutting the small target image into a small-size image with a fixed size; dividing the small-size images into a training set and a test set; s3, inputting the training set into a target detection model for training; and S4, inputting the test set of the small target image into the trained target detection model to obtain a position result of the small target image. According to the method, the small target image after data enhancement is subjected to manual annotation and sliding window image cutting processing, and the ability of the target detection model to detect and identify the oncomelania small target is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition, and in particular to a small target image recognition method. Background Art

[0002] Target recognition is now widely used in various fields, such as the detection and identification of schistosomiasis in complex outdoor environments. Schistosomiasis is a globally distributed zoonotic parasitic disease. Monitoring and controlling snails is one of the important measures to prevent and control schistosomiasis. In order to accurately grasp the current status of snail distribution, 12 endemic provinces in the country conduct large-scale snail surveys every year, with an average annual snail inspection area of ​​about 6 billion square meters. Snail monitoring is still an arduous task for schistosomiasis monitoring and early warning. However, during biological surveys, some organisms are very small and the geographical environment is relatively complex. The collected images have a relatively complex background, resulting in low target recognition accuracy.

[0003] Snails often breed on the soil surface of reeds, weeds, woods, ditches, etc. Since the height and posture of mobile phone shooting recognition are often limited by the on-site environment, the ratio of the total pixel size of the snail target to the total pixel size of the entire image in the collected image fluctuates greatly, and most of them tend to account for a small proportion, that is, the snail target is small, and the complex background such as branches and weeds accounts for a large proportion; the existing snail classification and recognition model is difficult to recognize such small target images. Therefore, solving the detection and recognition of snail images of different scales is the key to the application of on-site snail intelligent monitoring.

[0004] The Chinese patent application (application number: 201910103672.8) provides a snail detection, identification and statistics system, which uses CT imaging equipment to collect images of snails in the soil and identifies the snails in the collected images. However, the system detects snails based on the light source collected by the CT device and cannot detect snails under visible light.

[0005] The Chinese patent application (application number: 202110953707.4) discloses a snail collection and detailed classification and identification system. Based on the fine-grained image recognition technology of deep convolutional neural networks, it proposes an accurate identification system for Oncomelania hupensis and its similar snails in a natural scene. On this basis, combined with the snail image database and encyclopedia knowledge base, a mobile phone APP oncomelania hupensis intelligent identification system was developed to solve the problem that the existing technology cannot automatically classify and identify Oncomelania hupensis and its similar snails; however, the system has difficulty in identifying Oncomelania hupensis images with smaller targets in complex backgrounds. Summary of the invention

[0006] The present invention provides a small target recognition method and a snail recognition method based on image processing, which are used for accurately recognizing small target images and small target snail images under complex backgrounds.

[0007] To achieve the above effects, the technical solution of the present invention is as follows:

[0008] The small target recognition method based on image processing includes the following steps:

[0009] S1: Obtain a small target image to be identified under visible light, and annotate it using a combination of machine annotation and manual annotation;

[0010] The marking process is as follows:

[0011] Machine labeling: Use the image rectangle labeling tool LabelImg to label the small target image, obtain the coordinates, width, height and name of the target, and save it in the form of an XML file to create the label corresponding to the image in the small target image;

[0012] Manual labeling: Use a labeling box to show the location of the target on the small target image, and classify and label to distinguish between target and non-target organisms. The labeling box follows the labeling principle and is close to the outer contour of the target.

[0013] S2: Perform random data augmentation on the annotated small target images; perform sliding window cropping on the data augmented small target images: crop them into small-sized images of fixed size; divide the small-sized images into training sets and test sets;

[0014] The sliding window image cutting process is as follows: the small target images are screened according to the image quality evaluation standard, and the small target images with focus errors and blur are deleted; the input small target images are subjected to sliding window image cutting, and the image cutting area is obtained according to a certain step length by using a fixed-size sliding window, and the coordinates of the targets in the area are mapped; for large-size small target images, the image cutting process is carried out from left to right, first up and then down, with a fixed small size and a preset overlap rate, and the small target images are cut into small-size images of fixed size in sequence;

[0015] S3: input the training set into the target detection model for training, and use the total loss function to calculate the loss value of the target detection model, and determine whether the loss value no longer changes. If so, stop training, otherwise continue training to obtain a target detection model that can be used to detect the target position;

[0016] S4: Input the test set of small target images into the trained target detection model to obtain the location results of the small target images.

[0017] In the above scheme, the present invention uses a sliding window cutting method to optimize the problem that the existing snail image algorithm has limited ability to detect and identify small snail targets.

[0018] Furthermore, the manual labeling in step S1 is specifically as follows: using a labeling box to display the position of the target on the small target image, classifying and labeling to distinguish between target and non-target organisms, the labeling box follows the labeling principle, and the labeling box is close to the outer contour of the target; modifying and supplementing the results of the machine labeling, discovering the target and non-target data that are missed in the machine labeling and making supplementary labels, correcting the wrongly labeled targets and non-targets in the machine labeling, adjusting the size and position of the target box according to the detection requirements, and the machine corrects the targets marked as non-targets to negative samples.

[0019] Furthermore, the annotation principle of manual annotation in step S1 is specifically as follows:

[0020] Edge rule: The annotation frame must be close to the edge of the small target image target for frame annotation. The frame cannot be too small or too large.

[0021] Overlapping rule: When two targets overlap in the small target image, if the unobstructed part of the small target image after being occluded is greater than 50% of the target size, the small target image after being occluded can be framed, and the two target frames are allowed to have overlapping parts; if one of the small targets blocks part of the other small target, the shape of the other small target needs to be supplemented during frame annotation, and then frame annotation is performed;

[0022] Independent rule: Each small target image object is framed separately, and similar small target image objects with similar positions are framed separately;

[0023] No frame rule: If the small target image is blurry, too dark, or overexposed, no frame marking will be performed;

[0024] Boundary check: Ensure that the frame coordinates are not on the boundary of the small target image to prevent out-of-bounds errors during data loading or data expansion.

[0025] Furthermore, in step S2, data enhancement is performed on the small target image, specifically:

[0026] First, use Gaussian blur and median filtering methods to remove noise in small target images, and then use unsharp masking and Laplace filtering for sharpening;

[0027] In view of the color changes of small targets under different lighting and weather conditions, adaptive histogram equalization is used to adjust the contrast, and color correction technology such as white balance adjustment, color constancy algorithm, and color correction is used to restore the true color of small target images in their natural state;

[0028] Mosaic enhancement: Take 1 batch of data from the small target image, randomly select 4 images from it, randomly crop and piece them into a new image, repeat the operation batch size times, and generate batch size mosaic enhanced small target images;

[0029] Mixup enhancement: Randomly select two small target images from the mosaic enhanced small target image, mix the small target image itself and the label in proportion to form a mixup enhanced image;

[0030] HSV enhancement: In HSV space, perform HSV transformation on the three channels of the mixup enhanced image to obtain an HSV enhanced image; the three channels are H color, S saturation, and B brightness;

[0031] Albumentation enhancement: At least one of random blur, upside down flip, horizontal flip, and random brightness contrast is selected with a certain probability to process the HSV enhanced image to obtain the Albumentation enhanced image;

[0032] CLAHE image enhancement: Adjust the local contrast of the Albumentation enhanced image to enhance the effective signal of the small target image and suppress the noise signal to obtain the small target image after data enhancement.

[0033] Furthermore, the hyperparameters of the model training in step S3 are: the training cycle is set to 300, the Adam optimizer is used for optimization, the batch size is set to 128, and the initial learning rate is set to 0.001.

[0034] Furthermore, in step S3, the training set is input into the target detection model for training, and the training steps are specifically as follows:

[0035] S31: Use the target detection model to extract features from the training set and obtain a feature map;

[0036] S32: using the feature map to predict the location of the target box in the small target image and the score classification information;

[0037] S33: Remove duplicate target boxes by using the non-maximum suppression method;

[0038] S34: Calculate the loss of the target box and update the weight matrix of the target detection model through the back propagation algorithm;

[0039] S35: Repeat steps S31 to S34 until the target detection model converges to obtain a trained target detection model; after the training is completed, use the test set to verify the accuracy of the trained target detection model.

[0040] Furthermore, the total loss function in step 3 includes bounding box regression loss, classification loss and object probability loss; the bounding box regression loss is calculated using the GIoU loss function, which is calculated as follows:

[0041]

[0042] loss box =1-GIoU

[0043] Among them, A is the predicted box, B is the real box, and C represents the minimum convex closed box containing A and B, that is, the minimum rectangle parallel to the coordinate axis that surrounds the real box and the predicted box of the small target image; loss box represents the bounding box regression loss;

[0044] The classification loss uses the binary cross entropy loss function, namely BCEloss, and its formula is as follows:

[0045]

[0046] Among them, n represents the total number of samples, y n represents the actual value, x n Indicates the output value; loss class represents the classification loss, S represents the length of the square grid into which the small target image is divided, and i represents the current grid index; If grid i contains the target, the parameter is 1, otherwise it is 0, which is used to filter out the grids containing the target; p i (c) represents the predicted value of the c-type target in the current grid i; represents the true value of the c-type target in the current grid i, where c represents the target category;

[0047] loss object is the object probability loss, which consists of two parts: the confidence prediction of the detection box containing the small target And the confidence prediction of the detection box without small objects

[0048]

[0049] Where S represents the length of the square grid into which the small target image is divided, B represents the number of boxes in the current grid, i represents the current grid index, and j represents the box index in the current grid; C i Indicates the confidence of the true value of target i, which is 1 if target i exists and 0 if it does not exist; Indicates the confidence of the predicted value of target i; It is 1 if the jth box of the i-th grid is responsible for the current object, otherwise it is 0; If the jth box of the i-th grid is not responsible for predicting the target, it is 1, otherwise it is 0; noobj It is 1 if there is no object in the grid, otherwise it is 0;

[0050] Get the total loss function loss:

[0051] loss=loss box+loss class +loss objct .

[0052] Furthermore, when evaluating the target detection model in step S4, the target detection model is evaluated based on the training result obtained by inputting the small target image into the target detection model for training and the recognition result. The evaluation indicators of the test result include precision and recall:

[0053]

[0054] Where TP represents the number of small target image targets that are positive samples that are correctly predicted as positive samples; FP represents the number of small target image targets that are negative samples that are incorrectly predicted as positive samples; FN represents the number of small target image targets that are positive samples that are incorrectly predicted as negative samples;

[0055] Evaluation criteria: The average precision rate Precision is calculated to obtain the average precision rate AP; the average precision rate AP of each category is calculated to obtain the average precision rate mAP; the recall rate Recall is taken as the main factor, and the precision rate Precision, recall rate Recall, and average precision mAP are comprehensively considered. When the overall indicators of the trained target detection model are significantly better than the overall indicators of the previous model, the current target detection model is considered to be effective, otherwise it is considered to be ineffective.

[0056] Furthermore, when step S4 uses the evaluated target detection model for recognition, the detection effect of the target detection model is verified, and the process is as follows:

[0057] S41: Acquire a small target image to be identified by taking a photo or uploading a picture;

[0058] S42: Perform image compression and automatic flip detection processing on the small target image:

[0059] Image compression: Considering the speed of model detection, small target images are appropriately compressed before uploading them to improve the detection speed without affecting the performance.

[0060] Automatic flip detection: Some mobile phones automatically flip when shooting, so the application detects the flip state to prevent coordinate misalignment;

[0061] S43: Image cutting: according to the predefined size and step size, the small target image is cut into cut images of the same size to obtain a cut image dataset;

[0062] S44: Use the evaluated target detection model to detect each cut image in the cut image dataset, obtain the cut image result after filtering out duplicate cut images, and map the cut image result back to the coordinate position of the original small target image to obtain the position result of the small target image;

[0063] S45: After the image cut detection is completed, duplicate image cuts are screened out, and duplicate targets in overlapping areas of adjacent image cuts that may appear are screened out; the final image cut detection results are counted and output for display.

[0064] The snail recognition method based on image processing comprises the following steps: obtaining a small target image of a snail to be recognized under visible light, and annotating it by a combination of machine annotation and manual annotation; performing random data enhancement on the annotated small target image of the snail; performing sliding window image cutting processing on the small target image of the snail after data enhancement: cropping it into a small-size image of a fixed size; dividing the small-size image into a training set and a test set; inputting the training set into a target detection model for training, and using a total loss function to calculate a loss value for the target detection model, judging whether the loss value no longer changes, if so, stopping the training, otherwise continuing the training, and obtaining a target detection model that can be used to detect the position of the snail; inputting the test set of the small target image of the snail into the trained target detection model to obtain the position result of the snail.

[0065] In the above scheme, the present invention optimizes the size design of the anchor frame to be close to the size range of the snail target, reduces the difference in regression quantity, reduces the difficulty of learning the target detection model, and improves the recall rate.

[0066] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0067] The present invention performs sliding window cutting processing on the small target image after data enhancement, improves the ability of the target detection model to detect and identify small target images and small snail targets, and improves the recognition accuracy. The fixed size cutting method is used to increase the pixel ratio of the small snail target and improve the recognition efficiency; the human-machine combined data annotation method is adopted, and manual review is performed on the basis of machine prediction, which improves the annotation efficiency and reduces the missed labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The drawings are only used for illustrative purposes and are not to be construed as limitations of the present invention. To better illustrate the present embodiment, some parts of the drawings may be omitted, enlarged, or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the drawings.

[0069] Figure 1 A schematic diagram of the steps of a method for identifying snails in a small target image under visible light provided by an embodiment of the present invention;

[0070] Figure 2 It is a schematic diagram of small target image recognition under visible light provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Target detection is to determine the target category in a natural image and find its location in the image. As a core research issue in the field of artificial intelligence, target detection and recognition requires detection algorithms to separate the target of interest from the background and determine the category and coordinate position of the target compared to target recognition tasks. Traditional target detection methods are applied to snail image recognition, but this traditional target detection algorithm has two major defects: first, in terms of region selection, it is not possible to accurately estimate the location of the target; second, in the feature extraction stage, it is not possible to extract effective target features with high robustness, and target positioning, feature extraction and classification are all different steps. At the same time, the parameter adjustments of some algorithms are also independent of each other and cannot be adjusted at the same time. It is difficult for traditional target detection and recognition methods to obtain satisfactory accuracy and speed, and it will be difficult to apply them to large-scale on-site snail surveys in the future.

[0073] Example 1

[0074] For easier understanding, see Figure 1 An embodiment of the small target recognition method based on image processing provided by the present invention comprises the following steps:

[0075] S1: Obtain a small target image to be identified under visible light, and annotate it using a combination of machine annotation and manual annotation;

[0076] The marking process is as follows:

[0077] Machine labeling: Use the image rectangle labeling tool LabelImg to label the small target image, obtain the coordinates, width, height and name of the target, and save it in the form of an XML file to create the label corresponding to the image in the small target image;

[0078] Manual labeling: Use a labeling box to show the location of the target on the small target image, and classify and label to distinguish between target and non-target organisms. The labeling box follows the labeling principle and is close to the outer contour of the target.

[0079] In the specific implementation process, the manual labeling in step S1 is specifically to use a labeling box to display the position of the target on the small target image, classify and label to distinguish between target and non-target organisms, the labeling box follows the labeling principle, and the labeling box is close to the outer contour of the target; the results of the machine labeling are modified and supplemented, the target and non-target data missed in the machine labeling are found and supplemented, the wrongly marked targets and non-targets in the machine labeling are corrected, the size and position of the target box are adjusted according to the detection requirements, and the machine corrects the targets marked as non-targets to negative samples.

[0080] In the specific implementation process, when annotating small target images, the characteristics of the target and the common environment should be understood and familiarized, and the annotation principles should be the same; trial annotation of small target images should be carried out, that is, each annotator should annotate 20 pictures first; quality inspection should be carried out on small target images, that is, quality inspection should be carried out on the results of the trial annotation, and the quality inspection should be fed back to the annotator, who will modify the annotation results; the trial annotation and quality inspection steps need to be repeated until each annotator can meet the annotation requirements; when annotating, when the annotation task lasts for a long time, the entire annotation process is iterated many times, and a table is set up to record the annotation task allocation information in order to monitor the progress of each annotator.

[0081] In the process of manual labeling, some small target images that are difficult to judge will be encountered, and timely feedback needs to be given to the labeling standard setter. The labeling standard setter determines the labeling status of the small target images and synchronously updates the labeling standards for all labelers. For the quality of manual labeling, acceptance inspection is required, and the labeling results of each labeler are quickly browsed to check the following: wrong labeling, missing labeling, whether the size and position of the box are qualified, and whether the labeling quality is consistent before and after; unqualified labeling results need to be fed back to the labeler in a timely manner so that they can be improved in time in subsequent labeling work; the labeling results of each labeler are merged and organized into fixed-format data for model training.

[0082] In the specific implementation process, the annotation principle of manual annotation in step S1 is specifically as follows:

[0083] Edge rule: The annotation frame must be close to the edge of the small target image target for frame annotation. The frame cannot be too small or too large.

[0084] Overlapping rule: When two targets overlap in the small target image, if the unobstructed part of the small target image after being occluded is larger than 50% of the target size, the small target image after being occluded can be framed, and the two target frames are allowed to have overlapping parts; if one of the small targets blocks part of the other small target, the shape of the other small target needs to be supplemented during frame annotation, and then frame annotation is performed.

[0085] Independent rule: Each small target image object is framed separately, and similar small target image objects with similar positions are framed separately;

[0086] No frame rule: If the small target image is blurry, too dark, or overexposed, no frame marking will be performed;

[0087] Boundary check: Ensure that the frame coordinates are not on the boundary of the small target image to prevent out-of-bounds errors during data loading or data expansion.

[0088] S2: Perform random data augmentation on the annotated small target images; perform sliding window cropping on the data augmented small target images: crop them into small-sized images of fixed size; divide the small-sized images into training sets and test sets;

[0089] In the specific implementation process, in step S2, data enhancement is performed on the small target image, specifically:

[0090] First, use Gaussian blur and median filtering methods to remove noise in small target images, and then use unsharp masking and Laplace filtering for sharpening;

[0091] In a specific embodiment, when Gaussian blur processing is performed on an image, the standard deviation σ is σ G =1.5, the convolution kernel size is K G =5×5;

[0092] When median filtering is performed, the convolution kernel size is K M =3×3;

[0093] When unsharp masking is performed, the quantization factor is Q U =1.0, the threshold is T U =10;

[0094] When Laplace filtering is used, the convolution kernel size is K L =3×3, the scaling factor is S L =1.0, offset is O L =0;

[0095] In view of the color changes of small targets under different lighting and weather conditions, adaptive histogram equalization is used to adjust the contrast, and color correction technology such as white balance adjustment, color constancy algorithm, and color correction is used to restore the true color of small target images in their natural state;

[0096] In a specific embodiment, during the adaptive histogram equalization process, C CLAHE =2.0, the window size is W CLAHE =8×8;

[0097] When adjusting white balance, the gain parameter is G WB =1.2;

[0098] When the color constancy algorithm is used, the gamma correction parameter is γ C =2.2;

[0099] When color correction is performed, the color matrix used is [C R ,C G ,C B ],in:

[0100]

[0101] Mosaic enhancement: Take 1 batch of data from the small target image, randomly select 4 images from it, randomly crop and piece them into a new image, repeat the operation batch size times, and generate batch size mosaic enhanced small target images;

[0102] Mixup enhancement: Randomly select two small target images from the mosaic enhanced small target image, mix the small target image itself and the label in proportion to form a mixup enhanced image;

[0103] HSV enhancement: In HSV space, perform HSV transformation on the three channels of the mixup enhanced image to obtain an HSV enhanced image; the three channels are H color, S saturation, and B brightness;

[0104] Albumentation enhancement: At least one of random blur, upside down flip, horizontal flip, and random brightness contrast is selected with a certain probability to process the HSV enhanced image to obtain the Albumentation enhanced image;

[0105] CLAHE image enhancement: Adjust the local contrast of the Albumentation enhanced image to enhance the effective signal of the small target image and suppress the noise signal to obtain the small target image after data enhancement.

[0106] Data augmentation can also select random affine transformations: translation, scaling, and rotation according to actual usage requirements.

[0107] In the specific implementation process, the sliding window image cutting process in step S2 is as follows: the small target images are screened according to the image quality evaluation standard, and the small target images with focus errors and blur are deleted; the small target images are renamed with date and location as keywords to facilitate management and prevent duplication of names;

[0108] The input small target image is subjected to sliding window cutting processing. A fixed-size sliding window is used to obtain the cutting area according to a certain step size, and the coordinates of the target in the area are mapped. For large-size small target images, the image is cut from left to right, top to bottom, with a fixed small size and a 50% overlap rate, and the small target images are cropped into small-size images of a fixed size of 640X640 in sequence.

[0109] In a specific embodiment, when the sliding window is used to cut the image, the fixed size of the window is W S = 640 × 640 pixels, step size is S W =320 pixels.

[0110] In a further embodiment, data enhancement techniques such as Mosaic, MixUp, HSV and Albumentation are also used to add rotation, scaling, flipping, random blurring, random brightness, contrast and other operations to the cropped small-size images, so as to expand the number and diversity of snail images and simulate various changes in real scenes to enhance the robustness and generalization ability of the model.

[0111] In a specific embodiment, when the Mosaic technology is used for data enhancement, the image random splicing ratio is P M =0.5;

[0112] When using MixUp technology for data enhancement, the mixing ratio is R M =0.4;

[0113] When HSV data is enhanced, the hue range is H HSV =[-10°,+10°], the saturation range is S HSV =[-20%,+20%], the brightness range is V HSV =[-15%,+15%];

[0114] When using Albumentation for data enhancement, the rotation angle is A R = ±15°, scaling ratio R Alb is [0.8,1.2], and the random blur kernel size is K B =5×5.

[0115] S3: input the training set into the target detection model for training, and use the total loss function to calculate the loss value of the target detection model, and determine whether the loss value no longer changes. If so, stop training, otherwise continue training to obtain a target detection model that can be used to detect the target position;

[0116] In the specific implementation process, the hyperparameters of the model training in step S3 are: the training cycle is set to 300, the Adam optimizer is used for optimization, the batch size is set to 128; and the initial learning rate is set to 0.001.

[0117] The initial learning rate of the present invention is 10 -6 ~10 -2 Within the range, take several learning rate values, iterate the network once for each learning rate, and record the loss value; form a curve between the loss value and the learning rate value, and select the learning rate corresponding to the lowest point of the non-fluctuating segment in the curve as the initial learning rate for model training.

[0118] In the specific implementation process, the total loss function in step 3 includes the bounding box regression loss, classification loss and object probability loss; the bounding box regression loss is calculated using the GIoU loss function, and the calculation method is as follows:

[0119]

[0120] loss box =1-GIoU

[0121] Among them, A is the predicted box, B is the real box, and C represents the minimum convex closed box containing A and B, that is, the minimum rectangle parallel to the coordinate axis that surrounds the real box and the predicted box of the small target image; loss box represents the bounding box regression loss;

[0122] The classification loss uses the binary cross entropy loss function, namely BCEloss, and its formula is as follows:

[0123]

[0124] Among them, n represents the total number of samples, y n represents the actual value, x n Indicates the output value; loss class represents the classification loss, S represents the length of the square grid into which the small target image is divided, and i represents the current grid index; If grid i contains the target, the parameter is 1, otherwise it is 0, which is used to filter out the grids containing the target; p i (c) represents the predicted value of the c-type target in the current grid i; represents the true value of the c-type target in the current grid i, where c represents the target category;

[0125] loss object is the object probability loss, which consists of two parts: the confidence prediction of the detection box containing the small target And the confidence prediction of the detection box without small objects

[0126]

[0127] Where S represents the length of the square grid into which the small target image is divided, B represents the number of boxes in the current grid, i represents the current grid index, and j represents the box index in the current grid; C i Indicates the confidence of the true value of target i, which is 1 if target i exists and 0 if it does not exist; Indicates the confidence of the predicted value of target i; It is 1 if the jth box of the i-th grid is responsible for the current object, otherwise it is 0; If the jth box of the i-th grid is not responsible for predicting the target, it is 1, otherwise it is 0; noobj It is 1 if there is no object in the grid, otherwise it is 0;

[0128] Get the total loss function loss:

[0129] loss=loss box +loss class +loss objct .

[0130] In step S3, the training set is input into the target detection model for training, and the specific training steps are as follows:

[0131] S31: Use the target detection model to extract features from the training set and obtain a feature map;

[0132] S32: using the feature map to predict the location of the target box in the small target image and the score classification information;

[0133] S33: Remove duplicate target boxes by using the non-maximum suppression method;

[0134] S34: Calculate the loss of the target box and update the weight matrix of the target detection model through the back propagation algorithm;

[0135] S35: Repeat steps S31 to S34 until the target detection model converges to obtain a trained target detection model; after the training is completed, use the test set to verify the accuracy of the trained target detection model.

[0136] It should be noted that the weight matrix is ​​located in the feature extraction layer of the target detection model. Through training, it is known that the target detection model converges after 300 rounds of training, and the loss value no longer changes. Therefore, the preset maximum number of training rounds is 300 rounds.

[0137] S4: Figure 2 As shown in FIG, the test set of small target images is input into the trained target detection model to obtain the position result of the small target image.

[0138] In the specific implementation process, when evaluating the target detection model in step S4, the target detection model is evaluated according to the training result obtained by inputting the small target image into the target detection model for training and the recognition result. The evaluation indicators of the test result include precision and recall:

[0139]

[0140] Where TP represents the number of small target image targets that are positive samples that are correctly predicted as positive samples; FP represents the number of small target image targets that are negative samples that are incorrectly predicted as positive samples; FN represents the number of small target image targets that are positive samples that are incorrectly predicted as negative samples;

[0141] Evaluation criteria: The average precision rate Precision is calculated to obtain the average precision rate AP; the average precision rate AP of each category is calculated to obtain the average precision rate mAP; the recall rate Recall is taken as the main factor, and the precision rate Precision, recall rate Recall, and average precision mAP are comprehensively considered. When the overall indicators of the trained target detection model are significantly better than the overall indicators of the previous model, the current target detection model is considered to be effective, otherwise it is considered to be ineffective.

[0142] If the current model works well, it can be used for online app testing; if all indicators are not less than 75%, the model can be used to assist in manual data labeling;

[0143] If the current model does not work well, retrain the model by adding and checking small target images, adjusting model hyperparameters, etc. until the ideal effect is achieved.

[0144] It should be noted that when step S4 uses the evaluated target detection model for target recognition, the detection effect of the target detection model is verified using the app end, and the process is as follows:

[0145] S41: Acquire a small target image to be identified by taking a photo or uploading a picture;

[0146] S42: Perform image compression and automatic flip detection processing on the small target image:

[0147] Image compression: Considering the speed of model detection, small target images are appropriately compressed before uploading them to improve the detection speed without affecting the performance.

[0148] Automatic flip detection: Some mobile phones automatically flip when shooting, so the application detects the flip state to prevent coordinate misalignment;

[0149] S43: Image cutting: according to the predefined size and step size, the small target image is cut into cut images of the same size to obtain a cut image dataset;

[0150] S44: Use the evaluated target detection model to detect each cut image in the cut image dataset, obtain the cut image result after filtering out duplicate cut images, and map the cut image result back to the coordinate position of the original small target image to obtain the position result of the small target image;

[0151] S45: After the image cut detection is completed, duplicate image cuts are screened out, and duplicate targets in overlapping areas of adjacent image cuts that may appear are screened out; the final image cut detection results are counted and output for display.

[0152] The present invention manually labels and performs sliding window cutting processing on the small target images after data enhancement, so that the target detection model can detect small targets more accurately; a human-machine combined labeling method is used to improve the labeling efficiency; on the basis of the original data preprocessing, mosaic and mixup data enhancement methods suitable for small target detection are added to improve the recognition accuracy.

[0153] It should be noted that the embodiments of the present invention do not limit the specific feature extraction network to be used in the feature extraction process. Any existing backbone network (such as ResNeSt, ResNeXt, EfficientNet, etc.) can meet the application requirements and can be selected according to actual needs.

[0154] Example 2

[0155] Specifically, based on Example 1, the scheme is described in combination with specific implementation examples to further reflect the technical effects of the scheme. Specifically:

[0156] The method for identifying snails based on image processing is characterized by comprising the following steps:

[0157] S1: Acquire a small target image of a snail to be identified under visible light, and annotate it using a combination of machine annotation and manual annotation;

[0158] The width and height of the target in the small target image are both less than one tenth of the width and height of the small target image, so that the snail is identified as a small target; the labeling process is:

[0159] Machine labeling: Use the image rectangle labeling tool LabelImg to label the snail small target image, obtain the coordinates, width, height and name of the target object, and save the label corresponding to the image in the snail small target image in the form of an xml file;

[0160] Manual labeling: Use a labeling box to display the position of the snail target on the small snail target image, classify and label to distinguish snails and non-snail organisms, and follow the labeling principles. The labeling box is close to the outer contour of the snail;

[0161] S2: Perform random data enhancement on the labeled snail small target images; perform sliding window cutting on the data enhanced snail small target images: crop them into small-sized images of fixed size; divide the small-sized images into training sets and test sets;

[0162] The sliding window image cutting process is as follows: the snail small target images are screened according to the image quality evaluation standard, and the snail small target images with wrong focus and blur are deleted; the input snail small target images are subjected to sliding window image cutting, and the image cutting area is obtained according to a certain step length by using a fixed-size sliding window, and the coordinates of the snails in the area are mapped; for the large-size snail small target images, the image cutting process is carried out from left to right, first up and then down, with a fixed small size and a 50% overlap rate, and the snail small target images are cut into small-size images with a fixed size of 640x640 in sequence;

[0163] S3: input the training set into the target detection model for training, and use the total loss function to calculate the loss value of the target detection model, and determine whether the loss value no longer changes. If so, stop training, otherwise continue training to obtain a target detection model that can be used to detect the position of Oncomelania snails;

[0164] S4: Input the test set of the snail small target image into the trained target detection model to obtain the position result of the snail small target image.

[0165] The acquisition equipment standard for obtaining the small target image to be identified under visible light in step S1 is: using an Android phone with a relatively new system version, which can take pictures with a resolution of 3024x4032; using a mobile phone device that meets the standard to obtain the small target image to be identified under visible light in the area where Oncomelania snails are rampant, and obtain a batch of original picture data. Image shooting standard: the image is clear and correctly focused on the ground in the snail frame area; the whole snail frame and the quarter frame are shot separately, and the shooting area should just contain the snail frame; the image resolution is 3024x4032; image quality evaluation standard: correct focus, the area of ​​interest is not blurred; the area inside the snail frame is completely shot.

[0166] In the specific implementation process, the size distribution statistics of the target data set were made for the fixed 3024*4032 small snail target image size and the specified target photography method. Among them, the size of the snails mostly falls in the range of 40-300. According to the statistical results, a sliding window with a size of 640 and a step size of 320 is used for sliding window cutting processing, which can improve the accuracy of the snail detection model and avoid the situation where the target at the edge of the cutting is incomplete.

[0167] Example 3

[0168] It should be noted that the training of the target detection model of the present invention is carried out under the Linux Ubuntu system, and the EfficientDET training environment with PyTorch as the framework is preferably configured.

[0169] The efficientdet target detection model belongs to the deep learning target detection algorithm, which has a relatively fast detection speed and good recognition accuracy. The image samples in the image data set are divided into 80% training set and 20% test set, and the target detection model is trained respectively. After the training is completed, the Efficientdet snail detection model for detecting the target position can be obtained.

[0170] The target detection model was trained using the training set in the snail image. The size of the snail image was set to 640X640, the maximum number of training rounds was 300, and a test was performed after each round of training. The learning rate was adjusted by multiplying the existing learning rate by 0.1 as the new learning rate after reaching the preset number of iterations. The learning rate was adjusted after 50 and 150 rounds of training respectively. The training was stopped when the loss value of the total loss function tended to be flat, and the target detection model was finally obtained.

[0171] The trained model (i.e., target detection model) is tested using the test set in the snail image, and evaluation indicators such as model recall rate and precision are analyzed. Parameter adjustment and other processing methods are performed for missed detections and false alarms in the test set. The relevant training is repeated until the evaluation indicators reach a relatively good effect.

[0172] This example also provides a comparison between the Efficientdet snail detection model and other model algorithms, as follows:

[0173] The target detection algorithms EfficientNet, SSD and Faster RCNN are used. The parameters of these algorithms are 12M, 104M and 40.1M respectively.

[0174] The above three models detect "Hubei snail" or "non-Hubei snail" targets in 1,375 outdoor images. The annotations of these datasets were performed by experts in schistosomiasis control.

[0175] The hardware configuration is 4 NVIDIA GeForce RTX 3090 graphics cards and an Intel(R) Xeon(R) Gold5320 CPU processor. All experiments are performed using a PyTorch-based framework. All models use the same dataset and transfer learning training method to iteratively update model weights. The same hardware configuration is used throughout the training process to ensure performance consistency.

[0176] The indicators of model evaluation include precision, sensitivity, specificity, accuracy, mean average precision (mAP), and area under the ROC curve (AUC). The true value includes the true label and the target box, and the predicted value includes the predicted label, the detection box, and the confidence score. When the IoU between the detection box of the predicted value and the target box of the true value is greater than 0.5, the predicted value is considered to have accurately located the position of the target. On this basis, when the confidence score threshold is set to 0, the predicted value with the highest confidence score is regarded as the true value, and its corresponding predicted label is represented as a category of the true value.

[0177] Statistical analysis was performed using SPSS version 26.0. Differences in sensitivity, specificity, and accuracy between models were assessed using the χ2 test and the Bonferroni method. P < 0.05 indicated statistical significance.

[0178] The test results are shown in Table 1. Different letters after the numbers in each column in Table 1 indicate that the values ​​are significantly different from each other at a significance level of 5%.

[0179] Table 1 Model test results

[0180]

[0181]

[0182] As shown in Table 1, EfficientNet and SSD have good consistency with the gold standard, with kappa values ​​of 0.71 and 0.61, respectively (Table 2). EfficientNet has the highest accuracy, precision, recall, specificity, and mAP for detecting target objects. The AUC of EfficientNet reaches the highest value (0.85). The three models have significant differences in accuracy, recall, and specificity for detecting target objects (P<0.05). Pairwise comparisons show that the accuracy and recall of EfficientNet are significantly higher than those of other models (P<0.05). These results indicate that EfficientNet is the best detection model.

[0183] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A small target recognition method based on image processing, characterized in that: The following steps are involved: S1: Obtain a small target image to be identified under visible light, and annotate it using a combination of machine annotation and manual annotation; The marking process is as follows: Machine labeling: Use the image rectangle labeling tool LabelImg to label the small target image, obtain the coordinates, width, height and name of the target, and save it in the form of an XML file to create the label corresponding to the image in the small target image; Manual labeling: Use a labeling box to show the location of the target on the small target image, and classify and label to distinguish between target and non-target organisms. The labeling box follows the labeling principle and is close to the outer contour of the target. S2: Perform random data augmentation on the annotated small target images; perform sliding window cropping on the data augmented small target images: crop them into small-sized images of fixed size; divide the small-sized images into training sets and test sets; The sliding window image cutting process is as follows: the small target images are screened according to the image quality evaluation standard, and the small target images with focus errors and blur are deleted; the input small target images are subjected to sliding window image cutting, and the image cutting area is obtained according to a certain step length by using a fixed-size sliding window, and the coordinates of the targets in the area are mapped; for large-size small target images, the image cutting process is carried out from left to right, first up and then down, with a fixed small size and a preset overlap rate, and the small target images are cut into small-size images of fixed size in sequence; S3: input the training set into the target detection model for training, and use the total loss function to calculate the loss value of the target detection model, and determine whether the loss value no longer changes. If so, stop training, otherwise continue training to obtain a target detection model that can be used to detect the target position; S4: Input the test set of small target images into the trained target detection model to obtain the location results of the small target images.

2. The small target recognition method based on image processing according to claim 1, characterized in that: The manual labeling in step S1 is specifically to use a labeling box to display the position of the target on the small target image, classify and label to distinguish between target and non-target organisms, the labeling box follows the labeling principle, and the labeling box is close to the outer contour of the target; modify and supplement the results of the machine labeling, find the target and non-target data missed in the machine labeling and make supplementary labels, correct the wrongly labeled targets and non-targets in the machine labeling, adjust the size and position of the target box according to the detection requirements, and the machine corrects the targets marked as non-targets to negative samples.

3. The small target recognition method based on image processing according to claim 2 is characterized in that: The specific annotation principles of manual annotation in step S1 are: Edge rule: The annotation frame must be close to the edge of the small target image target for frame annotation. The frame cannot be too small or too large. Overlapping rule: When two targets overlap in the small target image, if the unobstructed part of the small target image after being occluded is greater than 50% of the target size, the small target image after being occluded can be framed, and the two target frames are allowed to have overlapping parts; if one of the small targets blocks part of the other small target, the shape of the other small target needs to be supplemented during frame annotation, and then frame annotation is performed; Independent rule: Each small target image object is framed separately, and similar small target image objects with similar positions are framed separately; No frame rule: If the small target image is blurry, too dark, or overexposed, no frame marking will be performed; Boundary check: Ensure that the frame coordinates are not on the boundary of the small target image to prevent out-of-bounds errors during data loading or data expansion.

4. The small target recognition method based on image processing according to claim 3 is characterized in that: In step S2, data enhancement is performed on the small target image, specifically: First, use Gaussian blur and median filtering methods to remove noise in small target images, and then use unsharp masking and Laplace filtering for sharpening; In view of the color changes of small targets under different lighting and weather conditions, adaptive histogram equalization is used to adjust the contrast, and color correction technology such as white balance adjustment, color constancy algorithm, and color correction is used to restore the true color of small target images in their natural state; Mosaic enhancement: Take 1 batch of data from the small target image, randomly select 4 images from it, randomly crop and piece them into a new image, repeat the operation batch size times, and generate batch size mosaic enhanced small target images; Mixup enhancement: Randomly select two small target images from the mosaic enhanced small target image, mix the small target image itself and the label in proportion to form a mixup enhanced image; HSV enhancement: In HSV space, perform HSV transformation on the three channels of the mixup enhanced image to obtain an HSV enhanced image; the three channels are H color, S saturation, and B brightness; Albumentation enhancement: At least one of random blur, upside down flip, horizontal flip, and random brightness contrast is selected with a certain probability to process the HSV enhanced image to obtain the Albumentation enhanced image; CLAHE image enhancement: Adjust the local contrast of the Albumentation enhanced image to enhance the effective signal of the small target image and suppress the noise signal to obtain the small target image after data enhancement.

5. The small target recognition method based on image processing according to claim 1, characterized in that: The hyperparameters of model training in step S3 are: the training cycle is set to 300, the Adam optimizer is used for optimization, the batch size is set to 128, and the initial learning rate is set to 0.

001.

6. The small target recognition method based on image processing according to claim 5, characterized in that: In step S3, the training set is input into the target detection model for training, and the specific training steps are as follows: S31: Use the target detection model to extract features from the training set and obtain a feature map; S32: using the feature map to predict the location of the target box in the small target image and the score classification information; S33: Remove duplicate target boxes by using the non-maximum suppression method; S34: Calculate the loss of the target box and update the weight matrix of the target detection model through the back propagation algorithm; S35: Repeat steps S31 to S34 until the target detection model converges to obtain a trained target detection model; after the training is completed, use the test set to verify the accuracy of the trained target detection model.

7. The small target recognition method based on image processing according to claim 6 is characterized in that: The total loss function in step S3 includes bounding box regression loss, classification loss and object probability loss; the bounding box regression loss is calculated using the GIoU loss function, which is calculated as follows: loss box =1-GIoU Among them, A is the predicted box, B is the real box, and C represents the minimum convex closed box containing A and B, that is, the minimum rectangle parallel to the coordinate axis that surrounds the real box and the predicted box of the small target image; loss box represents the bounding box regression loss; The classification loss uses the binary cross entropy loss function, namely BCEloss, and its formula is as follows: Among them, n represents the total number of samples, y n represents the actual value, x n Indicates the output value; loss class represents the classification loss, S represents the length of the square grid into which the small target image is divided, and i represents the current grid index; If grid i contains the target, the parameter is 1, otherwise it is 0, which is used to filter out the grids containing the target; p i (c) represents the predicted value of the c-type target in the current grid i; represents the true value of the c-type target in the current grid i, where c represents the target category; loss object is the object probability loss, which consists of two parts: the confidence prediction of the detection box containing the small target And the confidence prediction of the detection box without small objects Where S represents the length of the square grid into which the small target image is divided, B represents the number of boxes in the current grid, i represents the current grid index, and j represents the box index in the current grid; C i Indicates the confidence of the true value of target i, which is 1 if target i exists and 0 if it does not exist; Indicates the confidence of the predicted value of target i; It is 1 if the jth box of the i-th grid is responsible for the current object, otherwise it is 0; If the jth box of the i-th grid is not responsible for predicting the target, it is 1, otherwise it is 0; noobj It is 1 if there is no object in the grid, otherwise it is 0; Get the total loss function loss: loss=loss box +loss class +loss objct 。 8. The small target recognition method based on image processing according to claim 7, characterized in that: When evaluating the target detection model in step S4, the target detection model is evaluated based on the training result obtained by inputting the small target image into the target detection model for training and the recognition result. The evaluation indicators of the test result include precision and recall: Where TP represents the number of small target image targets that are positive samples that are correctly predicted as positive samples; FP represents the number of small target image targets that are negative samples that are incorrectly predicted as positive samples; FN represents the number of small target image targets that are positive samples that are incorrectly predicted as negative samples; Evaluation criteria: average the precision rate Precision to get the average precision rate AP; average the average precision rate AP of each category to get the average precision mAP; The recall rate Recall is the main factor, and the precision Precision, recall rate Recall, and average precision mAP are comprehensively considered. When the overall indicators of the trained target detection model are significantly better than the overall indicators of the previous model, the current target detection model is considered to be effective, otherwise it is considered to be ineffective.

9. The small target recognition method based on image processing according to claim 8, characterized in that: When the evaluated target detection model is used for target recognition in step S4, the detection effect of the target detection model is verified, and the process is as follows: S41: Acquire a small target image to be identified by taking a photo or uploading a picture; S42: Perform image compression and automatic flip detection processing on the small target image: Image compression: Considering the speed of model detection, small target images are appropriately compressed before uploading them to improve the detection speed without affecting the performance. Automatic flip detection: Some mobile phones automatically flip when shooting, so the application detects the flip state to prevent coordinate misalignment; S43: Image cutting: according to the predefined size and step size, the small target image is cut into cut images of the same size to obtain a cut image dataset; S44: using the evaluated target detection model to detect each cut image in the cut image dataset, obtaining a cut image result after filtering out duplicate cut images, and mapping the cut image result back to the coordinate position of the original small target image to obtain a position result of the small target image; S45: After the image cut detection is completed, duplicate image cuts are screened out, and duplicate targets in overlapping areas of adjacent image cuts that may appear are screened out; the final image cut detection results are counted and output for display.

10. A method for identifying snails based on image processing, implemented according to any one of the methods for identifying small targets based on image processing according to claims 1 to 9, characterized in that: The snail recognition steps are as follows: obtaining a small target image of the snail to be identified under visible light, and annotating it by a combination of machine annotation and manual annotation; The labeled snail small target image is subjected to random data enhancement; the data-enhanced snail small target image is subjected to sliding window cutting processing: the image is cropped into a small-size image of a fixed size; The small-size images are divided into a training set and a test set; the training set is input into the target detection model for training, and the loss value of the target detection model is calculated using the total loss function to determine whether the loss value no longer changes. If so, the training is stopped, otherwise the training is continued to obtain a target detection model that can be used to detect the position of snails; the test set of small target images of snails is input into the trained target detection model to obtain the position result of the snails.

Images