Small target recognition method based on image processing and snail recognition method

Abstract

The application discloses a small target recognition method based on image processing and a snail recognition method, and the snail recognition method comprises the following steps: S1, acquiring a small target image to be recognized under visible light, and marking the small target image by using a man-machine combined mode of machine marking and manual marking; S2, performing random data enhancement on the marked small target image; performing sliding window cutting processing on the small target image after data enhancement; and cutting the small target image into small-size images with a fixed size; and dividing the small-size images into a training set and a test set; S3, inputting the training set into a target detection model for training; S4, inputting the test set of the small target image into the trained target detection model, and obtaining a position result of the small target image. The small target image after data enhancement is manually marked and subjected to sliding window cutting processing, so that the ability of the target detection model to detect and recognize small snails is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target recognition, in particular to a small target image recognition method. BACKGROUND

[0002] Target recognition is now widely used in various fields, such as detection and recognition of schistosoma in complex outdoor environments. Schistosomiasis is a global human-animal parasitic disease, and monitoring and controlling oncomelania is one of the important measures to prevent and control schistosomiasis. In order to accurately grasp the distribution of oncomelania, large-scale oncomelania survey work is carried out in 12 endemic provinces every year, with an average of about 6 billion square meters of oncomelania survey area per year. Oncomelania monitoring is still a difficult task for schistosomiasis monitoring and early warning. However, due to biological survey, some organisms are small in size, and the geographical environment is complex, and the collected pictures have a complex background, resulting in low target recognition accuracy.

[0003] Oncomelania breeding environment is often distributed on the surface of mud in reeds, weeds, forests, ditches, etc. Due to the limitation of recognition height and posture in the field application of mobile phone shooting, the ratio of total pixels of oncomelania target to total pixels of the whole picture in the collected image fluctuates greatly, and most of the ratio is relatively small, that is, the oncomelania target is small, and the complex background such as branches and weeds accounts for a large proportion; the existing oncomelania classification and recognition model is difficult to recognize such small target image. Therefore, solving the detection and recognition of oncomelania images of different scales is the key to the application of oncomelania intelligent monitoring in the field.

[0004] Chinese patent application (application number: 201910103672.8) provides an oncomelania detection and recognition and statistics system, which collects oncomelania images in soil by using CT imaging equipment, and recognizes oncomelania in the collected images. However, this system is based on CT equipment to detect oncomelania under light source, and cannot detect oncomelania under visible light.

[0005] Chinese patent application (application number: 202110953707.4) discloses a snail collection and fine classification and recognition system, which is based on deep convolutional neural network fine-grained image recognition technology, proposes an accurate recognition system for oncomelania and similar snails in natural scenes, and on this basis, combines snail image database and encyclopedia knowledge base to develop a mobile phone APP oncomelania intelligent recognition system, solving the problem that existing technology cannot automatically classify and recognize oncomelania and similar snails; however, this system is difficult to recognize small oncomelania images in complex background. SUMMARY

[0006] The present application provides a small target recognition method based on image processing and an oncomelania recognition method, which is used for accurate recognition of small target images and small target oncomelania images in complex background.

[0007] To achieve the above effects, the technical solutions of the present application are as follows:

[0008] The small target recognition method based on image processing comprises the following steps:

[0009] S1: Obtain a small target image to be recognized under visible light, and use a man-machine combined labeling method of machine labeling and manual labeling;

[0010] The labeling process is as follows:

[0011] Machine labeling: use the image rectangular frame labeling tool LabelImg to label the small target image, obtain the coordinates, width and height of the target object, and the name information of the target object, and save the label corresponding to the image in the small target image in the form of an xml file;

[0012] Manual labeling: use the labeling frame to display the position of the target on the small target image, classify the target and non-target organisms, and the labeling frame follows the labeling principle and closely follows the outer contour of the target;

[0013] S2: Randomly data augment the labeled small target image; perform sliding window cutting processing on the data augmented small target image: crop into small size images of a fixed size; divide the small size images into a training set and a test set;

[0014] The sliding window cutting processing process is as follows: according to the image quality evaluation standard, the small target image is screened, and the small target images with focus errors and blurring are deleted; the input small target image is subjected to sliding window cutting processing, a fixed size sliding window is used, a cutting area is obtained according to a certain step size, and the coordinates of the target in the area are mapped; for large size small target images, follow the method from left to right, top to bottom, cut the small target images into small size images of a fixed size with a preset overlap rate, and sequentially crop the small target images into small size images of a fixed size;

[0015] S3: input the training set into the target detection model for training, and use the total loss function to calculate the loss value of the target detection model, judge whether the loss value is no longer changed, if yes, stop training, otherwise continue training, and obtain the target detection model which can be used to detect the position of the target;

[0016] S4: input the test set of the small target image into the trained target detection model, and obtain the position result of the small target image.

[0017] In the above scheme, the present application uses the sliding window cutting method to optimize the problem that the existing snail image algorithm has limited ability to detect and recognize small snail targets.

[0018] Furthermore, the manual labeling in step S1 is specifically as follows: using a labeling box to display the position of the target on the small target image, classifying and labeling to distinguish target and non-target organisms, the labeling box follows the labeling principle, and the labeling box is close to the outer contour of the target; modifying and supplementing the results of the machine labeling, discovering the target and non-target data that are missed in the machine labeling and making supplementary labels, correcting the wrongly marked targets and non-targets in the machine labeling, adjusting the size and position of the target box according to the detection requirements, and the machine corrects the targets marked as non-targets to negative samples.

[0019] Furthermore, the annotation principles of manual annotation in step S1 are specifically as follows:

[0020] Edge rule: The annotation frame must be close to the edge of the small target image object for frame annotation. The frame cannot be too small or too large.

[0021] Overlap rule: When two targets overlap in a small target image, if the unobstructed portion of the occluded small target image is greater than 50% of the target size, the occluded small target image can be framed, and the two target frames are allowed to overlap. If one small target partially blocks another small target, the shape of the other small target needs to be supplemented during frame annotation before frame annotation.

[0022] Independent rule: Each small target image object is framed separately, and similar small target image objects with similar locations are framed separately;

[0023] No frame rule: If the small target image is blurry, too dark, or overexposed, no frame marking will be performed;

[0024] Boundary check: Ensure that the frame coordinates are not on the boundary of the small target image to prevent out-of-bounds errors during data loading or data expansion.

[0025] Furthermore, in step S2, data enhancement is performed on the small target image, specifically:

[0026] First, use Gaussian blur and median filtering to remove noise from small target images, and then use unsharp masking and Laplacian filtering to sharpen them;

[0027] To address the color changes of small targets under different lighting and weather conditions, adaptive histogram equalization is used to adjust the contrast. White balance adjustment, color constancy algorithm, and color correction technology are used to restore the true color of small target images in their natural state.

[0028] Mosaic enhancement: Take 1 batch of data from the small target image, randomly select 4 images from it, randomly crop and piece them together to form a new image, repeat the operation batch size times, and generate batch size mosaic enhanced small target images;

[0029] mixup augmentation: randomly select two small target images from the small target images after mosaic augmentation, mix the small target images and the labels in proportion to form a mixup augmented image;

[0030] HSV augmentation: in HSV space, HSV transform is performed on the three channels of the mixup augmented image to obtain an HSV augmented image; the three channels are H color, S saturation, and B brightness;

[0031] Albumentation augmentation: select at least one of random blur, upside-down flip, horizontal flip, and random brightness-contrast with a certain probability to process the HSV augmented image to obtain an Albumentation augmented image;

[0032] CLAHE image augmentation: adjust the local contrast of the Albumentation augmented image to enhance the effective signal of the small target image and suppress the noise signal to obtain the small target image after data augmentation.

[0033] Further, the hyperparameters of the model training in step S3 are: the training period is set to 300, the Adam optimizer is used for optimization, the batch size is set to 128, and the initial learning rate is set to 0.001.

[0034] Further, in step S3, the training set is input into the target detection model for training, and the training steps are specifically:

[0035] S31: using the target detection model to extract features from the training set to obtain a feature map;

[0036] S32: using the feature map to predict the position of the target frame in the small target image and the score classification information;

[0037] S33: filtering out duplicate target frames by the non-maximum suppression method;

[0038] S34: calculating the loss of the target frame and updating the weight matrix of the target detection model by the back propagation algorithm;

[0039] S35: repeating steps S31-S34 until the target detection model converges to obtain a trained target detection model; after the training is completed, the accuracy of the trained target detection model is verified using the test set.

[0040] Further, the total loss function in step 3 includes a bounding box regression loss, a classification loss, and an object probability loss; the bounding box regression loss is calculated using a GIoU loss function, and the calculation method is as follows:

[0041]

[0042] loss box =1-GIoU

[0043] Wherein, A is the prediction frame, B is the real frame, C represents the minimum convex closed frame containing A and B, that is, the smallest rectangle parallel to the coordinate axis enclosing the small target image real frame and the prediction frame; loss box represents the bounding box regression loss;

[0044] The classification loss uses the binary cross-entropy loss function, that is, BCEloss, and its formula is as follows:

[0045]

[0046] Wherein, n represents the total number of samples, y n represents the actual value, x n represents the output value; loss class represents the classification loss, S represents the length of the square grid divided by the small target image, i represents the current grid index; represents the grid i contains the target, and the parameter is 1, otherwise 0, which is used to filter out the grid containing the target; p i (c) represents the prediction value of the c class target in the current grid i; represents the real value of the c class target in the current grid i, and c represents the target class;

[0047] loss object is the object probability loss, which is composed of two parts: the confidence prediction of the detection frame containing the small target and the confidence prediction of the detection frame not containing the small target

[0048]

[0049] Wherein, S represents the length of the square grid divided by the small target image, B represents the number of bounding boxes in the current grid, i represents the current grid index, and j represents the bounding box index in the current grid; C i represents the real value confidence of the target i, which is 1 if the target i exists, and 0 if it does not exist; represents the confidence of the prediction value of the target i; represents the jth bounding box of the ith grid, which is 1 if it is responsible for the current object, and 0 otherwise; represents the jth bounding box of the ith grid, which is 1 if it is not responsible for predicting the target, and 0 otherwise; λ noobj represents 1 if there is no object in the grid, otherwise 0;

[0050] The total loss function loss is obtained:

[0051] loss=lossbox +loss class +loss object .

[0052] Furthermore, when evaluating the target detection model in step S4, the target detection model is evaluated based on the training results obtained by inputting the small target image into the target detection model for training and the recognition results. The evaluation indicators of the test results include precision and recall:

[0053]

[0054] Where TP represents the number of small target image samples that are correctly predicted as positive samples; FP represents the number of small target image samples that are incorrectly predicted as positive samples; FN represents the number of small target image samples that are incorrectly predicted as negative samples.

[0055] Evaluation criteria: The average precision rate is calculated as AP; the average precision rate AP of each class is calculated as mAP; the recall rate is mainly used, and the precision rate, recall rate, and mAP are comprehensively considered. When the overall indicators of the trained target detection model are significantly better than the overall indicators of the previous model, the current target detection model is considered to be effective; otherwise, it is considered to be ineffective.

[0056] Furthermore, in step S4, when the evaluated target detection model is used for recognition, the detection effect of the target detection model is verified, and the process is as follows:

[0057] S41: Acquire an image of a small target to be identified by taking a photo or uploading a picture;

[0058] S42: Perform image compression and automatic flip detection on the small target image:

[0059] Image compression: Considering the speed of model detection, small target images are appropriately compressed before uploading, which improves the detection speed without affecting performance.

[0060] Automatic flip detection: Some mobile phones automatically flip when shooting, so the application detects the flip state to prevent coordinate misalignment;

[0061] S43: Image Slicing: Slice the small target image into slices of the same size according to the predefined size and step size to obtain a slice dataset of an image;

[0062] S44: using the evaluated target detection model to detect each cutout in the cutout dataset, screening out repeated cutouts to obtain a cutout result, and mapping the cutout result back to the coordinate position of the original small target image to obtain a position result of the small target image;

[0063] S45: after the cutout detection is completed, the repeated cutouts are screened out, and the repeated targets in the overlapping area of adjacent cutouts are screened out; the final cutout detection result is counted and output.

[0064] The snail identification method based on image processing includes the following steps: obtaining a small target image of a snail to be identified under visible light, and labeling the small target image of the snail in a man-machine combined manner using machine labeling and manual labeling; performing random data enhancement on the labeled small target image of the snail; performing sliding window cutout processing on the data-enhanced small target image of the snail: cutting into a small size image with a fixed size; dividing the small size image into a training set and a test set; inputting the training set into a target detection model for training, and using a total loss function to calculate a loss value of the target detection model, determining whether the loss value is no longer changed, if yes, stopping training, otherwise, continuing training, to obtain a target detection model that can be used to detect the position of the snail; inputting the test set of the small target image of the snail into the trained target detection model to obtain a position result of the snail.

[0065] In the above scheme, the size of the anchor frame is optimized to be close to the size range of the snail target, the difference of the regression amount is reduced, the learning difficulty of the target detection model is reduced, and the recall rate is improved.

[0066] Compared with the prior art, the technical scheme of the present application has the following advantages:

[0067] The small target image after data enhancement is subjected to sliding window cutout processing, the ability of the target detection model for small target image and snail small target detection and identification is improved, and the identification accuracy is improved. The fixed size cutout method is used to improve the pixel ratio of the snail small target and improve the identification efficiency; the man-machine combined data labeling method is used to improve the labeling efficiency and reduce the missing labeling based on machine prediction. BRIEF DESCRIPTION OF DRAWINGS

[0068] The drawings are only used for illustrative description and cannot be understood as a limitation on the present application; in order to better illustrate the embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0069] Figure 1 A visible light small target image snail identification method step schematic diagram is provided for the embodiments of the present application;

[0070] Figure 2 Schematic diagram of small target image recognition under visible light provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] Object detection involves identifying the target class and location within a natural image. As a core research issue in artificial intelligence, object detection and recognition, compared to object recognition, requires a detection algorithm to separate the target of interest from the background and determine its class and coordinate location. Traditional object detection methods have been applied to snail image recognition, but these algorithms suffer from two major drawbacks: first, they cannot accurately estimate the target's location during region selection; second, they cannot extract robust and effective target features during the feature extraction phase. Furthermore, target positioning, feature extraction, and classification are all separate steps, and some algorithm parameter adjustments are independent and cannot be adjusted simultaneously. Consequently, traditional object detection and recognition methods struggle to achieve satisfactory accuracy and speed, making them difficult to apply to large-scale on-site snail surveys in the future.

[0073] Example 1

[0074] For easier understanding, see Figure 1 An embodiment of the small target recognition method based on image processing provided by the present invention includes the following steps:

[0075] S1: Acquire an image of a small target to be identified under visible light and annotate it using a combination of machine and manual annotation.

[0076] The marking process is as follows:

[0077] Machine labeling: Use the image rectangle labeling tool LabelImg to label the small target image, obtain the coordinates, width, height and name of the target object, and save it in the form of an XML file to create the label corresponding to the image in the small target image;

[0078] Manual labeling: Use a labeling box to show the location of the target on the small target image, classify and label to distinguish target and non-target organisms, and follow the labeling principles to ensure that the labeling box closely fits the outer contour of the target.

[0079] In the implementation process, the manual labeling in step S1 is specifically as follows: a labeling box is used to show the position of the target on the small target image, classification labeling distinguishes between target and non-target organisms, the labeling box follows the labeling principle, and the labeling box is close to the outer contour of the target; the results of machine labeling are modified and supplemented, the missed target and non-target data in machine labeling are found and supplemented, the wrongly labeled target and non-target in machine labeling are corrected, the size and position of the target box are adjusted according to the detection requirements, and the target labeled as non-target by the machine is corrected as a negative sample.

[0080] In the implementation process, when labeling the small target image, the characteristics of the target and the common environment are understood and familiar, and the labeling principle is the same; the small target image is trial labeled, that is, each labeling personnel first labels 20 pictures; the small target image is quality inspected, that is, the trial labeling results are quality inspected, the quality inspection feedback is given to the labeling personnel, and the labeling personnel modify the labeling results; the trial labeling and quality inspection need to be circulated until each labeling personnel can meet the labeling requirements; when labeling, the whole labeling process is iterated for many times when the labeling task lasts for a long time, and a table is set to record the labeling task allocation information, so as to supervise the progress of each labeling personnel.

[0081] In the process of manual labeling, some small target images that are difficult to judge will be encountered, which need to be fed back to the labeling standard setter in time, the labeling standard setter determines the labeling situation of the small target image, and synchronously updates the labeling standard to all labeling personnel. For the quality of manual labeling, quality inspection needs to be carried out, the labeling results of each labeling personnel are quickly browsed, and the contents are checked: whether the wrong labeling, the missed labeling, the size and position of the box are qualified, and whether the labeling quality is consistent before and after; the unqualified labeling result needs to be fed back to the labeling personnel in time, so as to improve in the subsequent labeling work; the labeling results of each labeling personnel are merged, and the data in a fixed format are arranged for model training.

[0082] In the implementation process, the labeling principle of manual labeling in step S1 is specifically as follows:

[0083] Edge sticking rule: the labeling box needs to be close to the edge of the small target image target for frame labeling, and cannot be too small or too large.

[0084] Overlap rule: when there are two targets overlapping in the small target image, if the unblocked part of the blocked small target image is greater than 50% of the size of the target, the blocked small target image can be selected, and the two target boxes are allowed to have an overlapping part; if one small target blocks part of another small target, the shape of the other small target needs to be supplemented when frame labeling is performed, and then frame labeling is performed.

[0085] Independent rule: each small target image target is separately framed, and the same type of small target image targets with close positions are separately frame labeled;

[0086] No frame rule: If the small target image is blurry, too dark, or overexposed, no frame marking will be performed;

[0087] Boundary check: Ensure that the frame coordinates are not on the boundary of the small target image to prevent out-of-bounds errors during data loading or data expansion.

[0088] S2: Perform random data augmentation on the annotated small target images; perform sliding window cropping on the data augmented small target images: crop them into small-sized images of a fixed size; and divide the small-sized images into training and test sets.

[0089] In the specific implementation process, the data enhancement of the small target image is performed in step S2, specifically:

[0090] First, use Gaussian blur and median filtering to remove noise from small target images, and then use unsharp masking and Laplacian filtering to sharpen them;

[0091] In a specific embodiment, when Gaussian blur processing is performed on an image, the standard deviation σ is σ G =1.5, the convolution kernel size is K G =5×5;

[0092] When median filtering is performed, the convolution kernel size is K M =3×3;

[0093] When unsharp masking is performed, the quantization factor is Q U =1.0, the threshold is T U =10;

[0094] When Laplace filtering is performed, the convolution kernel size is K L =3×3, the scaling factor is S L =1.0, offset is 0 L =0;

[0095] To address the color changes of small targets under different lighting and weather conditions, adaptive histogram equalization is used to adjust the contrast. White balance adjustment, color constancy algorithm, and color correction technology are used to restore the true color of small target images in their natural state.

[0096] In a specific embodiment, during the adaptive histogram equalization process, C CLAHE =2.0, the window size is W CLAHE =8×8;

[0097] When adjusting white balance, the gain parameter is G wB =1.2;

[0098] Gamma correction parameter is γ when color constancy algorithm is processed C = 2.2;

[0099] Color matrix used when color correction is processed is [C R , C G , C B ], wherein:

[0100]

[0101] Mosaic enhancement: 1 batch of data is taken out from the small target image, 4 images are randomly taken from it, and a new image is spliced by random cropping. The operation is repeated batch size times to generate batch size small target images after mosaic enhancement;

[0102] Mixup enhancement: two small target images are randomly selected from the small target image after mosaic enhancement, and the small target image itself and the label are mixed in proportion to form a mixup enhanced image;

[0103] HSV enhancement: in HSV space, HSV transformation is performed on the three channels of the mixup enhanced image to obtain an HSV enhanced image; the three channels are H color, S saturation, and B brightness;

[0104] Albumentation enhancement: at least one of random blur, up-down flip, horizontal flip, and random brightness contrast is selected with a certain probability to process the HSV enhanced image to obtain an Albumentation enhanced image;

[0105] CLAHE image enhancement: the local contrast of the Albumentation enhanced image is adjusted to enhance the small target image effective signal and suppress noise signal to obtain a small target image after data enhancement.

[0106] Data augmentation can also select random affine transformation: translation, scaling, and rotation according to actual use requirements.

[0107] In the specific implementation process, the sliding window cutting process in step S2 is: according to the image quality evaluation standard, the small target image is screened, and the small target image with focus error and blur is deleted; the small target image is renamed with date and place as the key word, which is convenient for management and prevents the naming from being repeated;

[0108] The small target image input is subjected to sliding window cutting processing, a fixed size sliding window is used, and the cutting area is obtained according to a certain step size, and the coordinates of the target in the area are mapped; for a large size small target image, the cutting processing is performed in a fixed small size of 50% overlap ratio from left to right and from top to bottom, and the small target image is cut into a fixed size of 640*640 small size image.

[0109] In a specific embodiment, when cutting the image using a sliding window, the fixed size of the window is W S = 640*640 pixels, and the step size is S W = 320 pixels.

[0110] In a further embodiment, the small size image after cutting is also subjected to rotation, scaling, flipping, random blur, and random brightness and contrast operations by using data enhancement techniques such as Mosaic, MixUp, HSV, and Albumentation, to expand the number and diversity of the snail images, simulate various changes in real scenes, and enhance the robustness and generalization ability of the model.

[0111] In a specific embodiment, when using the Mosaic technique for data enhancement, the random image splicing ratio is P M = 0.5.

[0112] When using the MixUp technique for data enhancement, the mixing ratio is R M = 0.4.

[0113] When using the HSV data enhancement processing, the hue change range is H HSV = [-10°, +10°], the saturation change range is S HSV = [-20%, +20%], and the brightness change range is V HSV = [-15%, +15%]; when using Albumentation for data enhancement, the rotation angle is A R = ±15°, the scaling ratio R Alb is [0.8, 1.2], and the random blur kernel size is K B = 5*5.

[0114] S3: input the training set into the target detection model for training, and use the total loss function to calculate the loss value of the target detection model, judge whether the loss value is no longer changed, if yes, stop training, otherwise continue training, and obtain a target detection model that can be used to detect the position of the target;

[0115] In the implementation process, the hyperparameters of the model training in step S3 are as follows: the training period is set to 300, the Adam optimizer is used for optimization, the batch size is set to 128, and the initial learning rate is set to 0.001.

[0116] The initial learning rate of the present application is in the range of 10 -6 ~ 10 -2 , a plurality of learning rate values are taken, the network is iterated once for each learning rate, and the loss value is recorded; a curve between the loss value and the learning rate value is formed, and the learning rate corresponding to the lowest point in the undulating section of the curve is selected as the initial learning rate of the model training.

[0117] In the implementation process, the total loss function in step 3 includes a bounding box regression loss, a classification loss, and an object probability loss; the bounding box regression loss is calculated using a GIoU loss function, and the calculation method is as follows:

[0118]

[0119] loss box = 1 - GIoU

[0120] Wherein, A is a predicted box, B is a real box, and C represents a minimum convex closed box containing A and B, that is, the smallest rectangle parallel to the coordinate axis that encloses the real box and the predicted box of the small target image; loss box represents the bounding box regression loss;

[0121] The classification loss uses a binary cross-entropy loss function, that is, a BCEloss, and the formula is as follows:

[0122]

[0123] Wherein, n represents the total number of samples, y n represents the actual value, x n represents the output value; loss class represents the classification loss, S represents the length of the square grid into which the small target image is divided, and i represents the current grid index; represents 1 if the grid i contains a target, and 0 otherwise, and is used to filter out the grid containing the target; p i (c) represents the predicted value of the c-th target in the current grid i; represents the real value of the c-th target in the current grid i, and c represents the target class;

[0124] loss object is the object probability loss, which is composed of two parts: the confidence prediction of the detection box containing the small target and the confidence prediction of the detection box not containing the small target.

[0125]

[0126] wherein S represents the length of the square grid into which the small target image is divided, B represents the number of bounding boxes of the current grid, i represents the index of the current grid, and j represents the index of the bounding box in the current grid; C i represents the confidence of the true value of the target i, and is 1 when the target i exists and 0 when the target i does not exist; represents the confidence of the predicted value of the target i; represents 1 when the jth bounding box of the ith grid is responsible for the current object, and 0 otherwise; represents 1 when the jth bounding box of the ith grid is not responsible for the predicted target, and 0 otherwise; λ noobj represents 1 when there is no object in the grid, and 0 otherwise;

[0127] The total loss function loss is obtained as follows:

[0128] loss = loss box + loss class + loss object .

[0129] In step S3, the training set is input into the target detection model for training, and the training steps are as follows:

[0130] S31: using the target detection model to extract features of the training set to obtain a feature map;

[0131] S32: using the feature map to predict the position of the target box in the small target image and the score classification information;

[0132] S33: using the method of non-maximum suppression to screen out repeated target boxes;

[0133] S34: calculating the loss of the target box and updating the weight matrix of the target detection model through the back propagation algorithm;

[0134] S35: repeating steps S31-S34 until the target detection model converges to obtain the trained target detection model; after the training is completed, the accuracy of the trained target detection model is verified using the test set.

[0135] It should be noted that the weight matrix is located in the feature extraction layer of the target detection model; it is known through training that the target detection model converges when trained for 300 rounds, and the loss value no longer changes, so the preset maximum training rounds is 300 rounds.

[0136] S4: as shown in Figure 2 , the test set of the small target image is input into the trained target detection model to obtain the position result of the small target image.

[0137] In the implementation process, when the target detection model is evaluated in step S4, the target detection model is evaluated according to the training result obtained by inputting the small target image into the target detection model for training and the recognition result, and the evaluation indexes of the test result include precision and recall:

[0138]

[0139] In the formula, TP represents the number of small target images that are correctly predicted as positive samples; FP represents the number of small target images that are incorrectly predicted as positive samples; and FN represents the number of small target images that are incorrectly predicted as negative samples.

[0140] Evaluation criteria: the average precision AP is obtained by averaging the precision; the average precision mAP is obtained by averaging the average precision AP of each class; the recall is mainly considered, and the precision, the recall, and the average precision mAP are comprehensively considered; when the overall index of the training target detection model is obviously better than the overall index of the last model, it is considered that the current target detection model has good effect, otherwise it is considered that the effect is not good.

[0141] If the current model has good effect, it can be used for online app testing; when each index is not less than 75%, the model can be used for assisting manual annotation data.

[0142] If the current model has poor effect, the model is retrained by increasing, checking small target images, adjusting model hyperparameters, and the like until the ideal effect is achieved.

[0143] It should be noted that when the target detection model is used for target recognition in step S4, the target detection model is verified by using the app end, and the process is as follows:

[0144] S41: obtaining a small target image to be recognized by photographing or uploading a picture;

[0145] S42: performing image compression and automatic flip detection processing on the small target image:

[0146] Image compression: considering the model detection speed, the small target image is appropriately compressed before uploading the picture, so as to improve the detection speed without affecting the performance;

[0147] Automatic flip detection: some mobile phones have automatic flip when photographing, so the application end detects the reverse state to prevent the occurrence of coordinate misplacement;

[0148] S43: Picture cutting: according to the predefined size and step, the small target image is cut into cut pictures of the same size, and a cut picture dataset of a picture is obtained;

[0149] S44: using the evaluated target detection model to detect each cut picture in the cut picture dataset, screening out repeated cut pictures to obtain a cut picture result, and mapping the cut picture result back to the coordinate position of the original small target image to obtain a position result of the small target image;

[0150] S45: after the cut picture detection is completed, the repeated cut pictures are screened out, and the repeated targets in the overlapping area of adjacent cut pictures are screened out; the final cut picture detection result is counted and output.

[0151] The small target image after data enhancement is manually annotated and windowed and cut, so that the target detection model detects the small target more accurately; the annotation efficiency is improved by using the annotation method combining man and machine; on the basis of the original data preprocessing, the mosaic and mixup data enhancement methods suitable for small target detection are added, and the recognition accuracy is improved.

[0152] It should be noted that the embodiment of the present application does not limit the feature extraction network actually used in the feature extraction process, and any existing backbone network (such as ResNeSt, ResNeXt, EfficientNet, etc.) can meet the application requirements, which can be selected according to actual needs.

[0153] Embodiment 2

[0154] Specifically, on the basis of embodiment 1, the scheme is described in combination with specific examples, and the technical effect of the present scheme is further embodied. Specifically:

[0155] The image processing oncomelania identification method is characterized in that it comprises the following steps:

[0156] S1: obtaining a small target image of oncomelania to be identified under visible light, and using a man-machine combined labeling method of machine labeling and manual labeling;

[0157] The width and height of the target in the small target image are less than one tenth of the width and height of the small target image, so that the oncomelania is identified as a small target; the labeling process is:

[0158] Machine labeling: using an image rectangular frame labeling tool LabelImg to label the small target image of oncomelania, obtaining coordinate, width and height and target name information of the target object, and saving the label corresponding to the image in the small target image of oncomelania in the form of an xml file;

[0159] Manual annotation: The position of the snail target in the snail small target image is displayed using the annotation box, and the classification annotation distinguishes between snails and non-snail organisms. The annotation box follows the annotation principle and is close to the outer contour of the snail.

[0160] S2: The labeled snail small target image is randomly data-augmented; the data-augmented snail small target image is processed by sliding window cutting: cut into small size images of fixed size; the small size images are divided into training set and test set;

[0161] The sliding window cutting process is as follows: the snail small target image is screened according to the image quality evaluation standard, and the snail small target image with incorrect focus and blur is deleted; the input snail small target image is processed by sliding window cutting, a fixed size sliding window is used, and the cutting area is obtained according to a certain step size, and the coordinates of the snail in the area are mapped; for large size snail small target images, follow the method from left to right, top to bottom, cut the images into small size images of fixed size with 50% overlap rate, and cut the snail small target images into small size images of fixed size of 640x640 in turn;

[0162] S3: The training set is input into the target detection model for training, and the total loss function is used to calculate the loss value of the target detection model. It is judged whether the loss value is no longer changed. If yes, the training is stopped, otherwise the training is continued, and the target detection model which can be used to detect the position of the snail is obtained.

[0163] S4: The test set of the snail small target image is input into the trained target detection model to obtain the position result of the snail small target image.

[0164] The acquisition device standard for obtaining the small target image to be identified under visible light in step S1 is as follows: a new Android phone with a system version is used, which can shoot pictures with a resolution of 3024x4032; a standard-compliant mobile device is used to obtain the small target image to be identified under visible light in the snail-infested area, obtaining a batch of original picture data. Image shooting standards: clear image, correct focus to the ground in the snail frame area; shoot the whole snail frame and four-part frame respectively, and the shooting area should exactly contain the snail frame; image resolution is 3024x4032; image quality evaluation standard: correct focus, non-blurry focus area; complete shooting of the snail frame area.

[0165] In the implementation process, the size distribution statistics are made on the target data set obtained according to the fixed 3024*4032 snail small target image size and the specified target shooting mode; wherein, the size of the snail mostly falls within the range of 40-300, according to the statistical result, the sliding window cutting processing is performed by adopting the sliding window with the size of 640 and the step length of 320, which can improve the precision of the snail detection model, and also avoids the incomplete situation of the edge target of the cutting image.

[0166] Example 3

[0167] It should be noted that the training of the target detection model of the present application is in a linux ubuntu system, and the efficientdet training environment with pytorch as the framework is preferably configured.

[0168] The efficientdet target detection model belongs to the deep learning target detection algorithm, has a relatively fast detection speed, and has a good recognition accuracy. The image samples in the image data set are respectively divided into 80% training set and 20% test set, the target detection model is trained respectively, and the efficientdet snail detection model for detecting the target position is obtained after the training is completed.

[0169] The training set in the snail image is used to train the target detection model, the snail image size is set to 640X640, the maximum training round is 300 rounds, the test is performed once per round of training, the learning rate adjustment mode is that the existing learning rate is multiplied by 0.1 after reaching the preset iteration number to serve as the new learning rate, and the adjustment is performed after 50, 150 rounds of training. When the loss value of the total loss function tends to be flat, the training is stopped, and finally the target detection model is obtained.

[0170] The test set in the snail image is used to test the trained model (i.e. the target detection model), the evaluation indexes such as the recall rate and the precision rate of the model are analyzed, the parameter adjustment and other processing modes are performed according to the missed detection and false alarm in the test set, and the repeated related training is performed until the evaluation indexes reach a relatively good effect.

[0171] The present embodiment also provides a comparison between the efficientdet snail detection model and other model algorithms, which is as follows:

[0172] The target detection algorithms EfficientNet, SSD and Faster RCNN are used. The parameters of these algorithms are 12M, 104M and 40.1M respectively.

[0173] The above three models respectively detect the "Hubei snail" or "non-Hubei snail" target in 1375 outdoor shot images. The data set annotations are performed by experts in schistosomiasis control.

[0174] Hardware configuration is 4 NVIDIA GeForce RTX 3090 graphics cards and an Intel(R) Xeon(R) Gold 5320 CPU processor. All experiments are conducted using a PyTorch-based framework. All models use the same dataset and transfer learning training method, iteratively updating model weights. The entire training process uses the same hardware configuration to ensure performance consistency.

[0175] The evaluation metrics of the models include precision, sensitivity, specificity, accuracy, mean average precision (mAP), and area under the ROC curve (AUC). The true values include true labels and target boxes, while the predicted values include predicted labels, detection boxes, and confidence scores. When the IoU of the detection box of the predicted value and the target box of the true value is greater than 0.5, it is considered that the predicted value accurately locates the position of the target. On this basis, when the confidence score threshold is set to 0, the predicted value with the highest confidence score is considered as the true value, and its corresponding predicted label is represented as a class of the true value.

[0176] Statistical analysis was performed using SPSS version 26.0. The evaluation of the differences in sensitivity, specificity, and accuracy between models used χ2test and Bonferroni method. P<0.05 indicates statistical significance.

[0177] The test results are shown in Table 1, where different letters after each column of numbers in Table 1 indicate that the values are significantly different from each other at the 5% significance level.

[0178] Table 1 Model test results

[0179]

[0180] As shown in Table 1, EfficientNet and SSD have good consistency with the gold standard, with kappa values of 0.71 and 0.61, respectively (Table 2). EfficientNet has the highest accuracy, precision, recall, specificity, and mAP in detecting target objects. The AUC of EfficientNet reaches the highest value (0.85). The differences in detection accuracy, recall, and specificity of target objects among the three models are significant (P<0.05). Pairwise comparison shows that the accuracy and recall of EfficientNet are significantly higher than those of other models (P<0.05). These results indicate that EfficientNet is the best detection model.

[0181] Obviously, the above embodiments of the present application are merely exemplary but not intended to limit the embodiments of the present application. Based on the above description, any other variations or changes can be made by those skilled in the art without departing from the spirit and principles of the present application. It is not necessary to list all the embodiments here. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall fall within the scope of the claims of the present application.

Claims

1. A small target recognition method based on image processing, characterized in that: The following steps are involved: S1: Acquire an image of a small target to be identified under visible light and annotate it using a combination of machine and manual annotation. The marking process is as follows: Machine labeling: Use the image rectangle labeling tool LabelImg to label the small target image, obtain the coordinates, width, height and name of the target object, and save it in the form of an XML file to create the label corresponding to the image in the small target image; Manual labeling: Use a labeling box to show the location of the target on the small target image, classify and label to distinguish target and non-target organisms, and follow the labeling principles to ensure that the labeling box closely fits the outer contour of the target. S2: Perform random data augmentation on the annotated small target images; perform sliding window cropping on the data augmented small target images: crop them into small-sized images of a fixed size; and divide the small-sized images into training and test sets. The sliding window image cropping process is as follows: small target images are screened according to image quality evaluation criteria, and small target images with focus errors or blur are deleted; the input small target images are subjected to sliding window image cropping, using a fixed-size sliding window to obtain the image cropping area according to a certain step size, and coordinate mapping of the targets within the area; for large-sized small target images, the image cropping process is carried out from left to right, first up and then down, with a fixed small size and a preset overlap ratio, and the small target images are sequentially cropped into small-size images of fixed sizes; S3: Input the training set into the target detection model for training, and use the total loss function to calculate the loss value of the target detection model, and determine whether the loss value no longer changes. If so, stop training; otherwise, continue training to obtain a target detection model that can be used to detect target positions; S4: Input the test set of small target images into the trained target detection model to obtain the position results of the small target images; The manual labeling in step S1 is specifically to use a labeling box to display the position of the target on the small target image, classify and label to distinguish target and non-target organisms, follow the labeling principles, and keep the labeling box close to the outer contour of the target; modify and supplement the results of the machine labeling, find the target and non-target data missed in the machine labeling and make supplementary labels, correct the wrongly marked targets and non-targets in the machine labeling, adjust the size and position of the target box according to the detection requirements, and the machine corrects the targets marked as non-targets to negative samples.

2. The small target recognition method based on image processing according to claim 1, characterized in that: The specific annotation principles for manual annotation in step S1 are: Edge rule: The annotation frame must be close to the edge of the small target image object for frame annotation. The frame cannot be too small or too large. Overlap rule: When two targets overlap in a small target image, if the unobstructed portion of the occluded small target image is greater than 50% of the target size, the occluded small target image can be framed, and the two target frames are allowed to overlap. If one small target partially blocks another small target, the shape of the other small target needs to be supplemented during frame annotation before frame annotation. Independent rule: Each small target image object is framed separately, and similar small target image objects with similar locations are framed separately; No frame rule: If the small target image is blurry, too dark, or overexposed, no frame marking will be performed; Boundary check: Ensure that the frame coordinates are not on the boundary of the small target image to prevent out-of-bounds errors during data loading or data expansion.

3. The small target recognition method based on image processing according to claim 2, characterized in that: In step S2, data enhancement is performed on the small target image, specifically: First, use Gaussian blur and median filtering to remove noise from small target images, and then use unsharp masking and Laplacian filtering to sharpen them; To address the color changes of small targets under different lighting and weather conditions, adaptive histogram equalization is used to adjust the contrast. White balance adjustment, color constancy algorithm, and color correction technology are used to restore the true color of small target images in their natural state. Mosaic enhancement: Take 1 batch of data from the small target image, randomly select 4 images from it, randomly crop and piece them together to form a new image, repeat the operation batch size times, and generate batch size mosaic enhanced small target images; Mixup enhancement: Randomly select two small target images from the mosaic-enhanced small target image, mix the small target images themselves and the labels in proportion to form a mixup enhanced image; HSV enhancement: In HSV space, perform HSV transformation on the three channels of the mixup enhanced image to obtain an HSV enhanced image; the three channels are H color, S saturation, and B brightness; Albumentation enhancement: At least one of random blur, upside down flip, horizontal flip, and random brightness contrast is selected with a certain probability to process the HSV enhanced image to obtain the Albumentation enhanced image; CLAHE image enhancement: Adjust the local contrast of the Albumentation enhanced image to enhance the effective signal of the small target image and suppress the noise signal, obtaining a data-enhanced small target image.

4. The small target recognition method based on image processing according to claim 1, characterized in that: The hyperparameters for model training in step S3 are: the training cycle is set to 300, the Adam optimizer is used for optimization, the batch size is set to 128, and the initial learning rate is set to 0.

001.

5. The small target recognition method based on image processing according to claim 4, characterized in that: In step S3, the training set is input into the target detection model for training. The specific training steps are as follows: S31: Use the target detection model to extract features from the training set and obtain feature maps; S32: Use the feature map to predict the location of the target box in the small target image and the score classification information; S33: Remove duplicate target boxes through non-maximum suppression method; S34: Calculate the loss for the target box and update the weight matrix of the target detection model through the backpropagation algorithm; S35: Repeat steps S31 to S34 until the target detection model converges to obtain a trained target detection model; after the training is completed, use the test set to verify the accuracy of the trained target detection model.

6. The small target recognition method based on image processing according to claim 5, characterized in that: The total loss function in step S3 includes bounding box regression loss, classification loss, and object probability loss. The bounding box regression loss is calculated using the GIoU loss function as follows: loss box =1-GIoU Among them, A is the predicted box, B is the real box, and C represents the minimum convex closed box containing A and B, that is, the minimum rectangle parallel to the coordinate axis that surrounds the real box and the predicted box of the small target image; loss box represents the bounding box regression loss; The classification loss uses the binary cross entropy loss function, namely BCEloss, whose formula is as follows: Among them, n represents the total number of samples, y n Indicates the actual value, x n Indicates the output value; loss class represents the classification loss, S represents the length of the square grid into which the small target image is divided, and i represents the current grid index; If grid i contains the target, the parameter is 1, otherwise it is 0, which is used to filter out the grids containing the target; p i (c) represents the predicted value of the target of type c in the current grid i; represents the true value of the c-type target in the current grid i, where c represents the target category; loss object The object probability loss consists of two parts: the confidence prediction of the detection box containing the small target And the confidence prediction of the detection box without small targets Where S represents the length of the square grid into which the small target image is divided, B represents the number of boxes in the current grid, i represents the current grid index, and j represents the box index in the current grid; C i Indicates the confidence of the true value of target i, which is 1 if target i exists and 0 if it does not exist; Indicates the confidence level of the predicted value of target i; It is 1 if the jth box of the i-th grid is responsible for the current object, otherwise it is 0; If the jth box of the i-th grid is not responsible for predicting the target, it is 1, otherwise it is 0; noobj 1 if there is no object in the grid, otherwise 0; Get the total loss function loss: loss=loss box +loss class +loss object 。 7. The small target recognition method based on image processing according to claim 6, characterized in that: When evaluating the target detection model in step S4, the target detection model is evaluated based on the training results obtained by inputting the small target image into the target detection model for training and the recognition results. The evaluation indicators of the test results include precision and recall: Where TP represents the number of small target image samples that are correctly predicted as positive samples; FP represents the number of small target image samples that are incorrectly predicted as positive samples; FN represents the number of small target image samples that are incorrectly predicted as negative samples. Evaluation criteria: average the precision rate to get the average precision rate AP; average the average precision rate AP of each class to get the average precision mAP; The recall rate Recall is the main factor, and the precision rate Precision, recall rate Recall, and average precision mAP are comprehensively considered. When the overall indicators of the trained target detection model are significantly better than the overall indicators of the previous model, the current target detection model is considered to be effective; otherwise, it is considered to be ineffective.

8. The small target recognition method based on image processing according to claim 7, characterized in that: When the evaluated target detection model is used for target recognition in step S4, the detection effect of the target detection model is verified. The process is as follows: S41: Acquire an image of a small target to be identified by taking a photo or uploading a picture; S42: Perform image compression and automatic flip detection on the small target image: Image compression: Considering the speed of model detection, small target images are appropriately compressed before uploading, which improves the detection speed without affecting performance. Automatic flip detection: Some mobile phones automatically flip when shooting, so the application detects the flip state to prevent coordinate misalignment; S43: Image Slicing: Slice the small target image into slices of the same size according to the predefined size and step size to obtain a slice dataset of an image; S44: Use the evaluated target detection model to detect each cut image in the cut image dataset, obtain a cut image result after filtering out duplicate cut images, and map the cut image result back to the coordinate position of the original small target image to obtain a position result of the small target image; S45: After the image slice detection is completed, duplicate image slices are screened out, and duplicate targets in overlapping areas of adjacent image slices that may appear are screened out; the final image slice detection results are counted and output for display.

9. A method for identifying snails based on image processing, implemented according to any one of claims 1 to 8, characterized in that: The snail recognition steps are as follows: obtaining a small target image of the snail to be identified under visible light, and annotating it using a combination of machine and manual annotation; Perform random data augmentation on the labeled snail small target image; perform sliding window cropping on the data augmented snail small target image: crop it into a small-size image of a fixed size; The small-size images are divided into a training set and a test set; the training set is input into the target detection model for training, and the loss value of the target detection model is calculated using the total loss function to determine whether the loss value no longer changes. If so, the training is stopped; otherwise, the training is continued to obtain a target detection model that can be used to detect the position of snails; the test set of small target snail images is input into the trained target detection model to obtain the position result of the snail.

Images