A method for segmenting field insect images

Abstract

The present invention relates to a method for field insect image segmentation, comprising the following steps: locating target pests using a target detection method, removing most background interference before image segmentation. The target detection method removes segmentation-related network structures from the MASK R-CNN algorithm model and converts the algorithm output into coordinate information for the region where each pest target is located in the sample image; constructing an optimized lightweight semantic segmentation algorithm UNet and an adjusted MASKR-CNN algorithm, including replacing the original feature extraction network of UNet with a lightweight MobileNet to segment captured image fragments primarily containing pests; and finally, integrating the trained target detection model and semantic segmentation model into a new two-stage insect segmentation algorithm. The present invention has the advantages of high segmentation accuracy and strong resistance to background interference.

Description

Technical Field

[0001] The invention relates to the technical field of digital image analysis and segmentation, and in particular to a field insect image segmentation method. Background Art

[0002] Crop pest infestations are a common agricultural disaster. Due to their diverse variety, high frequency, wide impact area, and significant detrimental effects, the scope and severity of these infestations have become significant factors impacting and restricting grain production. Different pest species often require different control methods, making timely and accurate identification of pest species crucial for crop pest control.

[0003] Due to the complex and ever-changing backgrounds of field environments and the difficulty in obtaining close-up images of insects, using image recognition methods to directly identify small insect targets against complex backgrounds is challenging. Before identifying insects in an image, a reliable image processing method is to segment the target as completely as possible. Currently, there is limited research on image segmentation methods for small agricultural insect targets, and most existing studies suffer from low segmentation accuracy and poor algorithm deployment and application capabilities. Therefore, to improve insect recognition accuracy and promote timely and effective pest control, it is urgent to develop an image segmentation algorithm suitable for small insect targets against complex field backgrounds. Summary of the Invention

[0004] The present invention solves the background interference and image compression problems existing in the prior art when segmenting small target insects, and provides a field insect image segmentation method with high segmentation accuracy and strong anti-background interference capability.

[0005] To solve the above technical problems, the present invention provides a technical solution: a field insect image segmentation method, comprising the following steps:

[0006] S1. Collect RGB images of different insects under various backgrounds to obtain dataset A, and perform data augmentation on the collected insect images to obtain dataset B. Data augmentation methods can include image rotation and image flipping.

[0007] S2. Label the image data in dataset B, accurately mark the area where the target insect is located in the entire sample image, and obtain labels for target detection and semantic segmentation after labeling;

[0008] S3. Divide the labeled dataset B into a training set, a validation set, and a test set in a ratio of 3:1:1 to obtain dataset C. During the division process, it is necessary to ensure that the samples of different types in different datasets are relatively balanced.

[0009] S4. Use target detection to locate the insects in the sample image and remove most of the background interference before image segmentation. The target detection method removes the segmentation-related network structure from the MASK R-CNN algorithm and converts the algorithm output into the coordinate information of the area where each insect target is located in the sample image;

[0010] S5. Build an optimized lightweight semantic segmentation algorithm, UNet, and an adjusted Mask R-CNN algorithm. This includes replacing UNet's original feature extraction algorithm with a lightweight MobileNet to segment image segments primarily containing insects captured using coordinate information.

[0011] S6. Train the adjusted Mask R-CNN algorithm using the training and validation sets of dataset C. Test the trained model using the test set of dataset C to obtain an object detection model that can return the location of the target insect. Before training begins, compress the input image to 256×256 or 512×512 size. The larger the image size, the higher the equipment requirements.

[0012] S7. Use the training set and validation set of dataset C to train the optimized lightweight semantic segmentation algorithm UNet, and use the test set of dataset C to test the trained model to obtain a semantic segmentation model that can accurately segment the target insects;

[0013] S8. Integrate the trained target detection model and semantic segmentation model into a two-stage insect segmentation algorithm that first captures the target region and then performs segmentation. This includes taking the minimum bounding rectangle of the region containing all insects as the insect position of the current sample, and expanding the target region by 1.2 times without exceeding the sample boundary before image capture. This region expansion must be completed while maintaining the centroid position unchanged. Because the image is compressed to a size of 256×256 during the sample input stage of the target detection algorithm, when capturing the insect region, the obtained position information must be restored proportionally to the corresponding area in the original image using the reverse method.

[0014] Furthermore, the MobileNet network structure in step S5 is an optimized MobileNet network structure, and the optimization contents include: the input size of the network, the step size of the first 3×3 convolution and the last depth-wise separable convolution, the targeted removal of the ReLU activation operation after the 1×1 convolution in each depth-wise separable convolution module, and the removal of some network layers related to classification.

[0015] Furthermore, the transfer learning method is used in the model training process to accelerate the model convergence speed and improve the model segmentation effect. The entire model training process is divided into two training stages, with 50 training rounds in each stage; in the first training stage, the batch size is set to 4, the initial learning rate is 0.0001, and the parameters of the network feature extraction part are frozen; in the second training stage, the batch size is set to 2, the initial learning rate is 0.00001, and the entire network is trained; in both training stages, a variable learning rate method is used for training.

[0016] The present invention has the following advantages:

[0017] 1. A new segmentation algorithm is designed by combining the semantic segmentation method with the target detection method; this algorithm can reduce the interference of background on the segmentation results and improve the segmentation accuracy.

[0018] 2. Using the target detection algorithm, the target area is extracted from the original image and used for semantic segmentation; this improves the segmentation effect of small targets.

[0019] 3. Improve the Unet segmentation algorithm. Use the adjusted Mobilenet model as the feature extraction network to make the overall model lightweight. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a two-stage segmentation method for field insect images of the present invention.

[0021] Figure 2 It is a schematic diagram of the network structure of a two-stage model of field insect images of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be described in further detail below with reference to the accompanying drawings.

[0023] In conjunction with all the accompanying drawings, a field insect image segmentation method includes the following steps:

[0024] S1. Collect RGB images of different insects under various backgrounds to obtain dataset A, and perform data augmentation on the collected insect images to obtain dataset B. Data augmentation methods can include image rotation and image flipping.

[0025] S2. Label the image data in dataset B, accurately mark the area where the target insect is located in the entire sample image, and obtain the label file for target detection and semantic segmentation after labeling;

[0026] S3. Divide the labeled dataset B into a training set, a validation set, and a test set in a ratio of 3:1:1 to obtain dataset C. During the division process, it is necessary to ensure that the samples of different types in different datasets are relatively balanced.

[0027] S4. Use target detection to locate the insects in the sample image and remove most of the background interference before image segmentation. The target detection method removes the segmentation-related network structure from the MASK R-CNN algorithm and converts the algorithm output into the coordinate information of the area where each insect target is located in the sample image;

[0028] S5. Build an optimized lightweight semantic segmentation algorithm, UNet, and a modified Mask R-CNN algorithm. This involves replacing UNet's original feature extraction algorithm with a lightweight MobileNet to segment image segments primarily containing insects captured using coordinate information, and optimizing the MobileNet network structure. Optimizations include: network input size, stride lengths of the first 3×3 convolution and the last depthwise separable convolution, targeted removal of ReLU activations after the 1×1 convolution in each depthwise separable convolution module, and removal of some classification-related network layers. The optimized MobileNet network structure is shown in Table 1.

[0029] Table 1 Structural parameters of MobileNet after optimization

[0030]

[0031]

[0032] Note: dw stands for depthwise convolution.

[0033] S6. Train the adjusted Mask R-CNN algorithm using the training and validation sets of dataset C. Test the trained model using the test set of dataset C to obtain an object detection model that can return the location of the target insect. Before training begins, compress the input image to 256×256 or 512×512 size. The larger the image size, the higher the equipment requirements.

[0034] S7. Use the training set and validation set of dataset C to train the optimized lightweight semantic segmentation algorithm UNet, and use the test set of dataset C to test the trained model to obtain a semantic segmentation model that can accurately segment the target insects;

[0035] S8. Integrate the trained target detection model and semantic segmentation model into a two-stage insect segmentation algorithm that first captures the target region and then performs segmentation. This includes taking the minimum bounding rectangle of the region containing all insects as the insect position of the current sample, and expanding the target region by 1.2 times without exceeding the sample boundary before image capture. This region expansion must be completed while maintaining the centroid position unchanged. Because the image is compressed to a size of 256×256 during the sample input stage of the target detection algorithm, when capturing the insect region, the obtained position information must be restored proportionally to the corresponding area in the original image using the reverse method.

[0036] Furthermore, the transfer learning method is used in the model training process to accelerate the model convergence speed and improve the model segmentation effect. The entire model training process is divided into two training stages, with 50 training rounds in each stage; in the first training stage, the batch size is set to 4, the initial learning rate is 0.0001, and the parameters of the network feature extraction part are frozen; in the second training stage, the batch size is set to 2, the initial learning rate is 0.00001, and the entire network is trained; in both training stages, a variable learning rate method is used for training.

[0037] When the present invention is specifically implemented:

[0038] Example 1

[0039] Figure 1 The figure shows a flow chart of the two-stage segmentation method of field insect images of the present invention, and takes the common lepidopteran insects in the field as an example to introduce the specific embodiment of the present invention.

[0040] Insect image collection: Collect image data of target insects of various ages in different environments, and pay attention to maintaining a balance of samples during the collection process.

[0041] Data augmentation: For insect categories with fewer samples, data augmentation is performed using methods such as rotation and flipping.

[0042] Dataset creation. Label the processed sample data using LabelMe software. Specifically, the labeling method involves accurately marking the target insect region within the entire sample image. This produces a label file for semantic segmentation and object detection. The training, validation, and test sets are divided in a 3:1:1 ratio. This division ensures a relatively balanced distribution of samples across the datasets.

[0043] Two-stage segmentation model construction. Figure 2 Shown is a schematic diagram of the network structure of the two-stage segmentation method.

[0044] Phase 1:

[0045] Target detection model construction. Figure 2 The network structure of the target detection module in

[15] is used to construct the target detection network. The specific internal structure of the network can refer to the MASK R-CNN algorithm.

[0046] Target detection model training and testing. Conventional target detection algorithms require a new target detection dataset to be created. However, the target detection algorithm used in this invention is improved based on the MASK R-CNN algorithm. Therefore, the target detection model can be trained and tested directly using the semantic segmentation dataset created above.

[0047] Since the number of parameters of MASK R-CNN is large, the input sample image needs to be compressed to a size of 256×256 before training begins.

[0048] During object detection model training, we used transfer learning to accelerate model convergence and improve detection performance. The model was trained for 100 epochs, with a batch size of 2 and an initial learning rate of 0.0001. We used a variable learning rate approach for training.

[0049] Determine whether the insects in most samples can be accurately detected. If the detection effect is poor, it is necessary to adjust the structure of the target detection network or adjust the hyperparameters in the model training process (such as batch size, learning rate, etc.) to retrain the model.

[0050] Background removal: Use the target detection model to remove background from the original sample. The specific removal method is described in claim 3.

[0051] Phase 2:

[0052] Semantic segmentation model construction. Figure 2 The semantic segmentation network is constructed based on the network structure of the semantic segmentation module in

[15] . The specific internal structure of the network can refer to the UNet segmentation algorithm.

[0053] Semantic segmentation model training and testing. Before training begins, the input image needs to be compressed to a size of 512×512. During the model training process, a transfer learning method is used to accelerate the model convergence and improve the model segmentation effect. The entire model training process is divided into two training phases, each with 50 training rounds. In the first training phase, the batch size is set to 4, the initial learning rate is set to 0.0001, and the parameters of the network feature extraction part are frozen (not involved in training). In the second training phase, the batch size is set to 2, the initial learning rate is set to 0.00001, and the entire network is trained. A variable learning rate method is used for training in both training phases.

[0054] Model integration and insect segmentation. After the semantic segmentation model's test results meet the segmentation requirements, the trained object detection model and semantic segmentation model are integrated into a new two-stage insect segmentation algorithm. This integration involves appropriately adjusting the object detection results to obtain image segments corresponding to the original sample area, which serve as input to the semantic segmentation algorithm.

[0055] Model application. The image sample to be segmented is fed into the two-stage insect image segmentation model. If the model detects an insect target in the image, it will perform fine segmentation on the insect in the image, generating high-quality image data. If the model does not detect an insect in the image, the image will be identified as a non-insect sample and no further segmentation will be performed.

[0056] The above description of the present invention and its embodiments is non-limiting, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by the above description and, without departing from the purpose of the present invention, designs a structure and embodiment similar to the technical solution without creatively designing, they shall fall within the scope of protection of the present invention.

Claims

1. A method for segmenting field insect images, characterized by: The following steps are involved: S1. Collect RGB images of different insects under various backgrounds to obtain dataset A, and perform data enhancement on the collected insect images to obtain dataset B. S2. Label the image data in dataset B, accurately mark the area where the target insect is located in the entire sample image, and obtain labels for target detection and semantic segmentation after labeling; S3. Divide the labeled dataset B into a training set, a validation set, and a test set to obtain dataset C. During the division process, it is necessary to ensure that the samples of different types in different datasets are relatively balanced. S4. Use target detection to locate the insects in the sample image and remove most of the background interference before image segmentation. The target detection method removes the segmentation-related network structure from the MASK R-CNN algorithm and converts the algorithm output into the coordinate information of the area where each insect target is located in the sample image; S5. Build an optimized lightweight semantic segmentation algorithm, UNet, and an adjusted Mask R-CNN algorithm. This includes replacing UNet's original feature extraction algorithm with a lightweight MobileNet to segment image segments primarily containing insects captured using coordinate information. S6. Use the training set and validation set of dataset C to train the adjusted MASK R-CNN algorithm, and use the test set of dataset C to test the trained model to obtain a target detection model that can return the location of the target insect; S7. Use the training set and validation set of dataset C to train the optimized lightweight semantic segmentation algorithm UNet, and use the test set of dataset C to test the trained model to obtain a semantic segmentation model that can accurately segment the target insects; S8. Integrate the trained target detection model and semantic segmentation model into a two-stage insect segmentation algorithm that first captures the target region and then performs segmentation. This includes taking the minimum bounding rectangle of the region containing all insects as the insect position of the current sample, and expanding the target region by 1.2 times without exceeding the sample boundary before image capture. This region expansion must be completed while maintaining the centroid position unchanged. Because the image is compressed to a specified size during the sample input stage of the target detection algorithm, when capturing the insect region, the obtained position information must be proportionally restored to the corresponding region in the original image using the reverse method. S9. Use the above two-stage insect segmentation model to segment the sample image to obtain samples with accurate background removal, providing high-quality data for insect identification.

2. The method for segmenting field insect images according to claim 1, wherein: The MobileNet network structure in step S5 is an optimized MobileNet network structure. The optimization contents include: the input size of the network, the step size of the first 3×3 convolution and the last depthwise separable convolution, the targeted removal of the ReLU activation operation after the 1×1 convolution in each depthwise separable convolution module, and the removal of some network layers related to classification.

3. The field insect image segmentation method according to claim 1, wherein: In step S8, the minimum circumscribed rectangle of the area where all insects are located is taken as the insect position of the current sample. Before image capture, the final target area needs to be expanded by 1.2 times while keeping the centroid position unchanged and not exceeding the sample boundary. The obtained position information is restored proportionally to the corresponding area in the original image using the opposite method, and image fragments of the corresponding area in the original image are captured according to the position information for image segmentation.

4. The method for segmenting field insect images according to claim 3, wherein: During the model training process, the transfer learning method is used to accelerate the model convergence speed and improve the model segmentation effect.

Images