A data set forming method for rice field armyworm board pest detection

Abstract

This invention relates to a dataset synthesis method for detecting pests using sticky insect traps in rice fields, comprising: acquiring a set of target pest images; collecting images of field debris and accidentally trapped insects; acquiring a white sticky insect trap background; automatically labeling the sticky insect trap images using a trained insect point detection network to obtain the coordinates of the insect center point in each sticky insect trap image; dividing the images into multiple density levels based on the number of insect center point coordinates, statistically generating a spatial probability heatmap for each density level, forming a set of pasting location coordinates; randomly selecting target pest instances and pasting them onto the white sticky insect trap background according to the coordinates in the pasting location coordinate set; randomly pasting the selected accidentally trapped insect instances and field debris instances onto the white sticky insect trap background to generate a synthesized sticky insect trap image; requiring only a small amount of manual point labeling to train the point detection network for automatic relabeling, significantly reducing labeling costs, and the insect distribution in the synthesized image more closely matches the real spatial patterns.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring technology for agricultural pests, and in particular to a method for synthesizing datasets for detecting sticky insect pests in rice fields. Background Technology

[0002] Rice is an important food crop, and timely monitoring and identification of rice field pests are crucial for pest and disease control, pesticide reduction, and agricultural production management. Traditional rice pest monitoring typically relies on manual field patrols, photography, and identification, which suffers from problems such as high workload, low efficiency, poor real-time performance, and strong subjectivity. With the development of deep learning object detection technology, automatically identifying rice pests using object detection models has become a feasible solution.

[0003] Existing pest detection models typically rely on training with a large number of labeled images. Among publicly available pest datasets, IP102 is a commonly used large-scale pest image dataset, containing 102 categories of insect pests and over 75,000 images. However, the images in the IP102 dataset are primarily natural scene images, with pests usually attached to crops, leaves, branches, the ground, or other natural backgrounds. These images differ significantly from actual sticky insect trapping scenarios in rice fields.

[0004] In actual rice paddy pest monitoring, the ideal approach is to use sticky traps to capture pests, then capture images of the traps with a camera, and automatically detect and identify the pests on the traps. This method reduces the workload of manual field patrols and manual photography of individual insects. However, if detection models are trained directly using natural scene pest data such as IP102, the models will still primarily learn the appearance characteristics of pests in natural backgrounds, making it difficult to adapt to the detection tasks of small targets, dense targets, and complex interference objects against the background of sticky traps.

[0005] Furthermore, the IP102 dataset contains some non-realistic insect images, vector graphics, duplicate images, and class inclusion relationships. For example, IP102 contains the broad category "Aphid" and subcategories such as "English grain aphid" and "bird cherry-oat aphid," with potential inclusion, overlap, or duplicate images between different categories. Directly using this dataset for training can easily introduce erroneous and duplicate samples, impacting the model's detection performance in real-world rice paddy sticky insect trap scenarios.

[0006] On the other hand, the Pest24 dataset is closer to the intended application of sticky trap pest detection, as its images contain a large number of trapped insects, making it suitable for studying the problem. However, in practical use, it has been found that the Pest24 dataset has a serious problem of missing labels, especially in areas where insects are densely packed, small, incomplete, or have low contrast with the background. The original annotations cannot fully reflect the actual distribution of insects on the sticky trap. Directly using the original Pest24 annotations to statistically determine the spatial distribution probability of insects on the sticky trap will lead to inaccurate statistical results, thus affecting the realism of subsequent synthesized images.

[0007] Meanwhile, publicly available datasets of insect pests on sticky insect boards, like Pest24, are relatively few. Real-world sticky insect board images often contain a large number of target pests, non-target insects, natural enemies of the pests, dead branches and leaves, grass clippings, and broken limbs and wings. Manually selecting pests one by one and identifying their categories is not only time-consuming and laborious, but also requires a high level of entomological expertise from the annotators. For non-experts, determining whether an insect exists in a certain area of ​​an image is relatively easy, but accurately identifying the specific type of insect is quite difficult.

[0008] Existing dataset synthesis methods typically employ a simple random pasting approach, extracting target instances from existing images, randomly scaling and rotating them, and then pasting them onto the background. However, the spatial distribution of insects on real insect sticky boards is not entirely random; it is influenced by factors such as the placement of the sticky board, insect behavior, and the area of ​​the board surface, exhibiting certain spatial distribution patterns. If only a uniform random pasting method is used, the synthesized result is prone to an unrealistic "scattered sesame seeds" distribution, failing to reflect the true scene of the insect sticky board.

[0009] In addition, directly pasting the cut-out insect subjects onto the background of the sticky insect board can easily produce problems such as jagged edges, black borders, abrupt colors, inconsistent lighting, and a paper-cut look, reducing the quality of the synthesized image.

[0010] Therefore, a dataset synthesis method is needed for pest detection tasks using sticky insect traps in rice fields. This method should be able to generate pest detection training data that better reflects the actual sticky insect trap scenario in rice fields, based on publicly available pest data, real sticky insect trap data, and field noise samples. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the prior art and provide a dataset synthesis method for detecting pests on sticky insect boards in rice fields that is more in line with real-world sticky insect board scenarios and reduces the cost of manual annotation.

[0012] The technical solution adopted in this invention is a data set synthesis method for detecting sticky insect pests in rice fields, which includes the following steps: S1. Obtain a collection of images of different categories of target pests; S2. Collect images of field debris and accidentally caught insects; S3. Obtain the white sticky insect board background; S4. Remove the background from the target pest images, field debris images, and accidentally caught insect images in the target pest image set to obtain target pest instances, accidentally caught insect instances, and field debris instances with transparent backgrounds. S5. Select a portion of sticky insect board images from the publicly available sticky insect board dataset and manually annotate them to obtain annotated images; train an insect point detection network using the annotated images; use the trained insect point detection network to automatically re-annotate the remaining sticky insect board images in the publicly available sticky insect board dataset to obtain the coordinates of the insect center point in each sticky insect board image. S6. Based on the number of insect center point coordinates in each obtained sticky insect board image, divide the density into multiple density levels. Based on all sticky insect board images in each density level, generate a spatial probability heat map corresponding to each density level. S7. Based on the total number of target pest instances to be placed in the image to be synthesized, select a spatial probability heatmap of the corresponding density level, and based on its probability density, sample a number of pasting position coordinates equal to the total number to form a set of pasting position coordinates. S8. Randomly select a number of target pest instances equal to the total number from the target pest instances obtained in step S4, and randomly select several instances from the instances of accidentally caught insects and instances of field debris respectively. S9. Paste the target pest instances selected in step S8 onto the white sticky insect board background in step S3 according to the coordinates in the pasting position coordinate set obtained in step S7. Randomly paste the accidentally caught insect instances and field debris instances selected in step S8 onto the white sticky insect board background to generate a composite sticky insect board image, and automatically generate detection labels for each target pest instance after pasting.

[0013] The beneficial effects of this invention are as follows: By employing the aforementioned dataset synthesis method for detecting pests on sticky insect traps in rice fields, and introducing accidentally trapped insects and field debris as interference instances, the synthesized images more closely resemble real sticky insect trap scenarios, enhancing the robustness of the insect detection model against non-target objects. Addressing the issue of missed labeling in the traditional Pest24 dataset, the above method requires only a small amount of manual point annotation to train the point detection network for automatic re-annotation, significantly reducing annotation costs. Furthermore, based on the re-annotation results, a spatial probability heatmap is generated to guide the placement of pest instances, making the insect distribution in the synthesized images more consistent with real spatial patterns, thereby improving the detection accuracy of subsequent detection models.

[0014] Preferably, step S1 includes the following steps: S11. Obtain a publicly available pest dataset, filter out target pest images related to rice fields from the publicly available pest dataset, and clean and merge the filtered target pest images to obtain a set of target pest images of different categories. S12. Using Internet images and a diffusion generation model, the image samples of the different categories of target pests are expanded to obtain the final image sets of different categories of target pests.

[0015] By filtering target pest categories related to rice fields from public datasets and then manually cleaning, merging, and classifying them, irrelevant categories, non-realistic images, vector graphics, and duplicate samples were effectively removed, significantly improving the image quality of the instance library. On this basis, the samples were further expanded using Internet images and diffusion generation models, enriching the diversity of pests and providing more abundant, higher-quality, and more balanced pest image data for training the target detection model.

[0016] Preferably, step S3 includes: acquiring an image of a white sticky insect board, enhancing the image of the white sticky insect board with illumination perturbation, the illumination perturbation including brightness perturbation and color temperature perturbation, to obtain a white sticky insect board background.

[0017] By perturbing the brightness and color temperature of the white sticky insect board background, the real shooting environment under different lighting conditions was simulated, making the synthesized image more diverse in terms of brightness and color temperature. This avoids the trained detection model from over-adapting to a single lighting condition and enhances the model's generalization ability and robustness under different lighting environments.

[0018] Preferably, step S4 includes the following steps: using a pre-trained background segmentation model to extract the foreground from the images in the target pest image set, the images of accidentally caught insects, and the images of field debris, and then converting their original backgrounds to transparent backgrounds to obtain transparent background images of target pest instances, images of accidentally caught insect instances, and images of field debris instances; cropping each instance image to remove the transparent edges around the instance in each instance image, retaining only the smallest bounding rectangle region that just encloses the instance, to obtain instances of target pests, instances of accidentally caught insects, and instances of field debris.

[0019] By extracting the foreground and removing the original background using a pre-trained background segmentation model, high-quality transparent background instances were obtained, eliminating the interference of complex backgrounds in the original image on the synthesized image. Furthermore, by compactly cropping the transparent edges around the instances, the positional deviation and redundancy of the bounding boxes during subsequent pasting were reduced, improving the localization accuracy of target instances and the accuracy of detection annotations in the synthesized image, thereby improving the overall quality of the synthesized dataset.

[0020] Preferably, step S5 includes the following steps: S51. Obtain a publicly available sticky insect board dataset, and select a portion of sticky insect board images from the publicly available sticky insect board dataset; S52. For each selected sticky insect board image, mark the center position of each insect in the image with a point to obtain a point-marked image; S53. Construct an insect point detection network; S53. Input the labeled image into the insect point detection network for training to obtain the trained insect point detection network; wherein, during training, each labeled point in the labeled image is converted into a Gaussian heat map label, so that each labeled point corresponds to a Gaussian peak in the heat map, and the Focal Loss loss function is used to train the insect point detection network to obtain the trained insect point detection network. S54. Input the remaining sticky insect board images from the publicly available sticky insect board dataset into the trained insect point detection network for prediction, and output the insect center point probability heatmap for each image; then extract the local maxima of each insect center point probability heatmap to obtain the center point coordinates of each insect.

[0021] The insect point detection network is trained by manually annotating a small number of points, and then the trained insect point detection network is used to automatically re-annotate a large-scale public sticky insect board dataset, which greatly reduces the cost of manual annotation. At the same time, the annotation points are converted into Gaussian heat map labels and trained with Focal Loss, which effectively solves the problem of imbalance between positive and negative samples, improves the network's localization accuracy for dense small targets, and thus obtains more complete and accurate information on the spatial distribution of insects.

[0022] Preferably, in step S53, the construction of the insect point detection network specifically includes: the insect point detection network includes an encoder, a decoder, an attention module, and a heatmap output head; the encoder uses ResNet18 as the backbone feature extraction network to extract image features at different scales; the decoder gradually restores spatial resolution through upsampling; the encoder and decoder fuse shallow detail features and deep semantic features through skip connections; the attention module includes a channel attention module and a spatial attention module, which are used to enhance channels sensitive to insect features and suppress background texture interference, respectively; the heatmap output head is used to output a probability heatmap of the insect center point.

[0023] By fusing shallow details and deep semantic features, the network's ability to locate small insect targets on sticky insect boards is improved; the channel attention and spatial attention modules enhance the insect body feature response and suppress background texture interference, respectively, thereby improving the network's detection accuracy in complex scenes.

[0024] Preferably, step S6 includes the following steps: S61. Normalize the coordinates of the insect center point of each sticky insect board image obtained in step S5 to a unified sticky insect board coordinate system to obtain the normalized center point coordinates of each sticky insect board image; the unified sticky insect board coordinate system takes the upper left corner of the sticky insect board image as the origin and the width and height of the sticky insect board image as the reference, and normalizes the coordinates of each center point to the range of [0,1]. S62. Count the number of normalized center point coordinates for each sticky insect board image, and divide the density into multiple levels based on the total number of coordinates. S63. For each density level, establish a standard grid with a preset resolution, and map the normalized center point coordinates of pests extracted from all images within that level to the standard grid; count the number of center point coordinates falling within each grid cell of the standard grid to generate a two-dimensional frequency distribution matrix; perform Gaussian smoothing and probability normalization on the two-dimensional frequency distribution matrix to obtain the spatial probability heatmap corresponding to that density level.

[0025] By normalizing the coordinates of the insect center points in different sticky insect board images to a unified coordinate system, the influence of image size differences on statistics is eliminated; by generating a probability heatmap that reflects the true spatial distribution pattern, an accurate and reliable distribution basis is provided for the layout of pest instances in subsequent synthetic images.

[0026] Preferably, in step S7, the specific process of sampling a set of pasting position coordinates based on their probability density, in an amount equal to the total number, includes the following steps: S71. Repeatedly sample randomly according to the probability density distribution of the selected spatial probability heatmap until the number of pasting position coordinates is equal to the total number. S72. According to the set uniform sampling ratio, randomly select a number of pasting position coordinates corresponding to the uniform sampling ratio from the pasting position coordinates and remove them, and uniformly and randomly sample an equal number of new position coordinates within the background range of the sticky insect board; the pasting position coordinates that were not removed and the new position coordinates obtained by uniform random sampling together constitute the pasting position coordinate set.

[0027] Sampling is performed according to the probability density indicated by the spatial probability heatmap, so that the distribution of insects in the synthesized image is more consistent with the spatial distribution pattern of the real sticky insect board scene. At the same time, uniform random sampling is introduced for mixing according to a set ratio, which avoids the synthesized image from relying entirely on a fixed heatmap pattern, enhances data diversity, and can prevent the detection model from overfitting to specific spatial locations, thereby improving the model's generalization ability. Attached Figure Description

[0028] Figure 1 This is a schematic diagram illustrating the process principle of a data set synthesis method for detecting pests using sticky insect traps in rice fields, as described in this invention. Figure 2 This is a schematic diagram of the visualization results of images in the Pest24 dataset after automatic point annotation in this invention; Figure 3 This is a spatial probability heatmap of insects generated by automatic relabeling based on the Pest24 dataset in this invention. Detailed Implementation

[0029] The invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can implement it based on the description. The scope of protection of the invention is not limited to these specific embodiments.

[0030] This invention provides a data set synthesis method for detecting sticky insect pests in rice fields, such as... Figure 1 As shown, the method includes the following steps: Step 1: Filter relevant pest categories in rice fields from the publicly available pest dataset IP102.

[0031] Specifically, pest categories related to rice field scenes were screened from the publicly available pest dataset IP102, resulting in 14 categories of target pests in rice fields.

[0032] Since not all insects listed in IP102 will appear in rice fields, this invention does not directly use all 102 types of pests in IP102. Instead, it selects 14 target pests based on the needs of pest detection in rice fields, as shown in Table 1, which provides a detailed description of the 14 target pests.

[0033] Table 1.14 Description of Target Pests Step 2: Clean, merge, and organize the selected pest images.

[0034] Specifically, the selected 14 categories of pest images were manually screened to remove non-realistic pest images, vector images, images with incorrect categories, images without obvious insect bodies, low-quality images, and duplicate images.

[0035] To address the issues of category inclusion and cross-category problems in dataset IP102, this invention performs merging and classification processing. For example, dataset IP102 contains the broad category of Aphids and multiple subcategories of aphids, and there may be duplicate images or inclusion relationships between different categories. Based on the requirements of rice field detection tasks, this invention reorganizes and merges the relevant categories to form a set of 14 target pest image categories more suitable for sticky trap pest detection tasks.

[0036] In a specific embodiment, after screening, cleaning and merging, more than 1,200 images of 14 types of rice field pests were obtained.

[0037] Step 3: Expand the pest sample using internet images and diffusion generation models.

[0038] To increase the quantity and appearance diversity of images of the 14 target pests, this invention further supplements the screening results of dataset IP102 by manually screening relevant pest images from the Internet.

[0039] Building upon this, a diffusion generation model is used to further expand the pest image sample. In a specific embodiment, the Kohya_ss tool is employed, with StableDiffusion 1.5 as the underlying model, and the pest image generation model is trained using LoRA fine-tuning. The training data sample includes pest images filtered from IP102 and pest images manually selected from the internet.

[0040] The trained diffusion generation model generates images of pests, and the generated results are manually screened to remove images with abnormal morphology, incorrect category, incomplete structure, or poor quality. In a specific embodiment, more than 500 diffusion-generated pest samples are obtained after screening.

[0041] Using the above method, a set of 14 target pest images was constructed, consisting of images selected from dataset IP102, Internet-enhanced images, and images generated by diffusion models.

[0042] Step 4: Collect images of field debris and accidentally caught insects.

[0043] To make the images of the synthesized sticky insect traps more closely resemble real rice paddy scenes, this invention not only synthesizes the target pests but also introduces accidentally trapped insects and field debris as interference factors.

[0044] In a specific embodiment, environmental debris such as dead leaves, weeds, grass clippings, and broken leaves in rice fields were actually collected, and 102 debris instances were obtained from them.

[0045] Simultaneously, samples of other insects that might be accidentally trapped by sticky traps were collected, such as natural enemies of pests, non-pest flying insects, or other small field insects. In a specific embodiment, 96 examples of other easily trapped insect samples were compiled.

[0046] The aforementioned debris and accidentally trapped insects appear as noise or interference targets in the synthetic image, which is used to simulate non-target content present in real sticky insect boards, thereby improving the adaptability of subsequent detection models to complex scenes.

[0047] Step 5: Construct and obtain the background for the sticky insect board.

[0048] This invention uses an image of a square white sticky insect trap as the background for image synthesis. This background is used to simulate the image acquisition scene after the sticky insect trap in a rice field.

[0049] During compositing, a slight illumination perturbation is applied to the background image to simulate brightness and color temperature variations under different shooting conditions. In a specific embodiment, the background brightness fluctuation range is set to 0.9 to 1.1, and the color temperature perturbation range is set to 0.97 to 1.03. This method ensures that the synthesized image has some variation in overall illumination, preventing the model from adapting only to single brightness and color temperature conditions.

[0050] Step 6: Extract examples of target pests, accidentally caught insects, and field debris.

[0051] Subject extraction was performed on images of 14 types of target pests, other insects that are easily caught by accident, and field debris such as dead leaves and weeds. The original background was removed to obtain foreground instances with transparent backgrounds.

[0052] In a specific embodiment, the Rembg library is used to call the U2Net model to extract the main instance, and the original background is converted into a transparent background to obtain an instance image with an alpha channel.

[0053] After the main subject is extracted, the instance image is compactly cropped, meaning all transparent edges around the instance are removed, leaving only the smallest bounding rectangle that just covers the insect or debris. This process reduces positional shifts and inaccurate annotations caused by excessively large transparent blank areas during subsequent pasting.

[0054] After the above steps, the following types of transparent foreground instances are obtained: 14 types of target rice pest instances, other easily accidentally caught insect instances, and field environment debris instances such as dead leaves, weeds, and broken leaves.

[0055] Step 7: Perform point-label re-annotation on the Pest24 dataset to obtain point-labeled images.

[0056] To obtain the spatial distribution patterns of insects in real sticky insect boards, this invention utilizes the Pest24 dataset for spatial probability statistics. However, due to the omission issue in the original Pest24 annotations, its original bounding boxes cannot be directly used to annotate and statistically analyze the spatial distribution.

[0057] Therefore, this invention randomly selects a portion of images from the Pest24 dataset for manual re-annotation. In a specific embodiment, 1000 Pest24 images are randomly selected for manual point annotation.

[0058] Unlike traditional bounding box annotation, this invention uses point annotation. Annotators only need to determine whether an insect exists in a certain area of ​​the image and mark a point at the center of the insect; they do not need to draw a complete bounding box or identify the specific insect category.

[0059] The reason for using point annotation is that the purpose of this invention is not to reconstruct the Pest24 classification and detection dataset, but to statistically analyze the spatial distribution probability of insect locations on real sticky insect boards. For non-professional annotators, determining whether an insect is present at a certain location is easier than determining the insect category. Therefore, point annotation can significantly reduce the difficulty and cost of manual annotation while ensuring the statistical requirements of spatial distribution.

[0060] Step 8: Train an insect point detection network using the labeled images. Use the trained insect point detection network to automatically relabel the remaining sticky insect board images in the Pest24 dataset to obtain the coordinates of the insect center point in each sticky insect board image.

[0061] Using the 1000 labeled images obtained in step seven, an insect point detection network is trained for predicting the insect center point on sticky insect traps. This network takes an image of the sticky insect trap as input and outputs a heatmap of the insect center point probability.

[0062] In a specific embodiment, the insect point detection network adopts a customized PestPointNet structure. This network uses ResNet18 as the backbone feature extraction network and combines multi-scale feature fusion, skip connections, channel attention mechanisms, and spatial attention mechanisms to improve the ability to locate small target insects in sticky insect boards.

[0063] Specifically, the insect point detection network includes an encoder, a decoder, an attention module, and a heatmap output head. The encoder extracts image features at different scales; the decoder gradually restores spatial resolution through upsampling; skip connections are used to fuse shallow detail features and deep semantic features; the channel attention module is used to enhance channels sensitive to insect features; and the spatial attention module is used to suppress interfering areas such as background textures and grid lines on the sticky insect board.

[0064] During training, manually labeled points are converted into Gaussian heatmap labels, generating a two-dimensional Gaussian response near each insect's center point. The network learns to predict the heatmap of the insect's center point. Since the number of positive sample points in the heatmap is much smaller than that in the background area, this invention uses FocalLoss, which addresses the imbalance between positive and negative samples in the heatmap, for training.

[0065] After training, the trained insect point detection network is used to automatically predict the center point of more than 20,000 images in Pest24 that have not been manually relabeled, and outputs the probability heatmap of the insect center point for each image; then, local maxima extraction is performed on the probability heatmap of the insect center point to obtain the coordinates of the center point of each insect.

[0066] like Figure 2 The image shown is a visualization of the Pest24 dataset after automatic point labeling. Figure 2 In the image, green dots represent the insect center points calculated from the original Pest24 bounding boxes, i.e., the intersection of the two diagonals of the bounding boxes; red dots represent the insect center points automatically predicted by the trained insect point detection network. Due to omissions in the original Pest24 annotations, green dots only indicate the locations of existing targets in the original annotations and cannot fully represent the distribution of all insects in the image; red dots are used to supplement the missing areas in the original annotations, thereby obtaining a more complete distribution of insect center points on the sticky traps, and further used to construct a spatial probability heatmap of real sticky trap insects.

[0067] Step 9: Based on the number of insect center point coordinates in each obtained sticky insect board image, divide the area into multiple density levels. Based on all sticky insect board images within each density level, generate a spatial probability heatmap corresponding to each density level.

[0068] The coordinates of the insect center point of each obtained sticky insect board image are normalized to a unified sticky insect board coordinate system.

[0069] In the unified insect sticky board coordinate system, the upper left corner of the insect sticky board is taken as the origin of the coordinate system. The width and height of the insect sticky board are normalized to the range of 0 to 1, and the center point of each insect is represented as the normalized coordinate.

[0070] To simulate the distribution of real sticky insect boards under different insect population densities, this invention divides images into multiple density levels based on the number of insects in each image. In a specific embodiment, it is divided into four levels, corresponding to four spatial heatmaps: tier01, tier02, tier03, and tier04, respectively. Figure 3 As shown.

[0071] For each density setting, the frequency of insect center point coordinates appearing in different areas of the sticky insect board was statistically analyzed, and a corresponding spatial probability heatmap was generated after smoothing. Composite images with fewer insects used lower-density heatmaps, while composite images with more insects used higher-density heatmaps.

[0072] In this way, the present invention obtains the spatial distribution patterns of insects learned from real sticky insect board images.

[0073] like Figure 3 The image shows a spatial probability heatmap of insects on a sticky insect board, obtained based on the Pest24 re-annotation results. This invention first maps the obtained insect center point coordinates to a standardized sticky insect board coordinate system. Then, it categorizes the samples according to the number of insects or insect population density in the image, generating spatial probability heatmaps for different density levels. In the image, areas with stronger color responses indicate a higher probability of insect presence under the corresponding population density, while areas with weaker color responses indicate a lower probability of insect presence. When synthesizing the dataset, the heatmap for the corresponding density level is selected based on the number of insects in the image to be generated, and the sticking location of pest instances is sampled according to the probability distribution of the heatmap, thereby improving the realism of the spatial distribution of insects in the synthesized image.

[0074] Step 10: Sampling the location of pest pasting based on spatial probability heatmap.

[0075] When generating a composite sticky insect board image, first determine the number of targets and difficulty level that need to be synthesized in the current image, and then select the corresponding spatial probability heatmap based on the number of targets.

[0076] Subsequently, the placement of pest instances was sampled based on the probability density of the spatial probability heatmap. Areas where insects are more likely to appear on real sticky insect boards were sampled more frequently during the synthesis process; areas where insects are less likely to appear on real sticky insect boards were sampled less frequently.

[0077] To avoid the model memorizing fixed spatial positions, this invention also enhances the sampling position processing, including horizontal flipping, vertical flipping, 90-degree rotation, and random coordinate jitter.

[0078] Meanwhile, to enhance the model's generalization ability to low-probability and edge regions, this invention mixes heatmap sampling with uniform random sampling. In a specific embodiment, the uniform sampling mixing probability is set to 0.30, meaning there is a 30% probability that heatmap sampling is not used, and instead, uniform random sampling is performed within the sticky insect board area. In the remaining cases, location sampling is performed based on the true spatial probability heatmap.

[0079] In this way, the synthesized images not only preserve the distribution patterns of insects on real sticky traps, but also avoid relying entirely on fixed heatmap patterns.

[0080] Step 11: Perform size, rotation, and scaling transformations on the foreground instance.

[0081] Before pasting, the size, rotation, and scaling of the target pest instances, accidentally trapped insect instances, and field debris instances are controlled. In a specific embodiment, the instance size is dynamically set relative to the short side of the sticky insect board background: the size of the pest instance occupies the background... The size of environmental noise instances, such as dead leaves, weeds, and broken leaves, occupies 3% to 10% of the short side of the background; the size of these instances occupies 2% to 10% of the short side of the background. Instances are randomly rotated and scaled to increase their pose diversity. Bilinear interpolation is used during scaling and rotation to reduce black borders, ringing, or edge artifacts in the alpha channel image during scaling.

[0082] Step 12: Perform edge feathering and local ambient light blending.

[0083] To make the insect instances appear more natural when pasted onto the background of the sticky insect board, this invention performs edge feathering and local ambient light adaptive fusion processing on the foreground instances.

[0084] First, Gaussian blur is applied to the alpha transparency channel of the example image to slightly soften the foreground edges. This process reduces jagged edges and a paper-cut look after subject extraction.

[0085] Secondly, before pasting an instance, a local background area corresponding to the location where it will be pasted is captured, and the RGB mean value of this local area is calculated. Then, the non-transparent pixels of the foreground instance are linearly blended with the local background mean color, so that the instance color slightly absorbs the background ambient color.

[0086] This process can be represented as: The blended foreground color = original foreground color × 0.85 + local background uniform color × 0.15.

[0087] In this way, the insect body can maintain a certain consistency in color with the background of the sticky insect board, simulating the reflection on the insect body surface, the semi-transparent wings, and the color overflow of the camera in real shooting, thereby reducing the abruptness caused by direct pasting.

[0088] Step 13: Synthesize the sticky insect board image and generate detection labels.

[0089] Following the above sampling, transformation, and fusion strategies, instances of 14 types of target pests, instances of accidentally trapped insects, and noise instances such as dead leaves and weeds were sequentially pasted onto a real white sticky insect board background to obtain a synthetic sticky insect board image.

[0090] For 14 types of target pest instances, the system automatically generates target detection labels based on their actual location, size, and bounding rectangle after pasting, including the category number and detection box coordinates.

[0091] Other insects that are easily mistaken for pests, as well as environmental noise such as dead leaves, weeds, and broken leaves, can be used as background interference targets in the synthesis, but they will not be labeled as the 14 target pest categories. This will help the subsequent detection model learn to distinguish between target pests and non-target interference, reducing the false detection rate.

[0092] Finally, a dataset consisting of synthetic sticky insect board images and automatically detected labeled files is generated. This dataset can be used to train a rice field sticky insect board pest detection model.

[0093] Using the above-mentioned method for synthesizing datasets for detecting sticky insect pests in rice fields, we can obtain: First, for the rice field sticky insect detection scenario, 14 types of rice field-related pests were screened out from IP102 instead of using all 102 types of pests directly, reducing interference from categories irrelevant to the application scenario.

[0094] Second, the IP102 screening results were manually cleaned, categorized, and duplicated samples were removed, thus eliminating non-realistic pest images, vector images, and duplicate images, thereby improving the quality of the target pest instance library.

[0095] Third, by using internet images and diffusion generation models to expand the pest sample, the problem of insufficient sample quantity and limited variety of postures for some rice pest categories was alleviated.

[0096] Fourth, by introducing other noise examples such as insects that are easily caught by mistake and dead leaves and weeds in the field, the synthesized image is made closer to the real image of sticky insect traps in rice fields, which helps to improve the robustness of the detection model to non-target interference.

[0097] Fifth, this invention addresses the issue of missing annotations in the original Pest24 annotations by using a small number of annotations to train the point detection network, and then automatically re-annotating large-scale Pest24 images, thereby reducing the cost of manual annotation and improving the completeness of insect spatial distribution statistics.

[0098] Sixth, this paper generates a spatial probability heatmap based on the re-annotated insect center points of Pest24, and uses this heatmap to guide the pasting position of pest instances, so that the distribution of insects in the synthetic image is more consistent with the real insect sticky board scene, rather than a completely uniform random distribution.

[0099] Seventh, heatmap sampling is mixed with uniform random sampling, and the sampling coordinates are flipped, rotated and jittered to enhance the results. This preserves the true distribution pattern and avoids the model memorizing fixed positions, thereby improving the generalization ability of the synthesized data.

[0100] Eighth, by perturbing background brightness, color temperature, alpha edge feathering, and adaptive fusion of local ambient light, the problems of black edges, jagged edges, paper-cut look, and inconsistent lighting caused by image cutout and pasting are reduced, thus improving the visual realism of the synthesized image.

Claims

1. A method for synthesizing datasets for detecting pests using sticky traps in rice fields, characterized in that, The method includes the following steps: S1. Obtain a collection of images of different categories of target pests; S2. Collect images of field debris and accidentally caught insects; S3. Obtain the white sticky insect board background; S4. Remove the background from the target pest images, field debris images, and accidentally caught insect images in the target pest image set to obtain target pest instances, accidentally caught insect instances, and field debris instances with transparent backgrounds. S5. Select a portion of sticky insect board images from the publicly available sticky insect board dataset and manually annotate them to obtain annotated images; Insect point detection network is trained using labeled images; the trained insect point detection network is then used to automatically relabel the remaining sticky insect board images in the public sticky insect board dataset to obtain the coordinates of the insect center point in each sticky insect board image. S6. Based on the number of insect center point coordinates in each obtained sticky insect board image, divide the density into multiple density levels. Based on all sticky insect board images in each density level, generate a spatial probability heat map corresponding to each density level. S7. Based on the total number of target pest instances to be placed in the image to be synthesized, select a spatial probability heatmap of the corresponding density level, and based on its probability density, sample a number of pasting position coordinates equal to the total number to form a set of pasting position coordinates. S8. Randomly select a number of target pest instances equal to the total number from the target pest instances obtained in step S4, and randomly select several instances from the instances of accidentally caught insects and instances of field debris respectively. S9. Paste the target pest instances selected in step S8 onto the white sticky insect board background in step S3 according to the coordinates in the pasting position coordinate set obtained in step S7. Randomly paste the accidentally caught insect instances and field debris instances selected in step S8 onto the white sticky insect board background to generate a composite sticky insect board image, and automatically generate detection labels for each target pest instance after pasting.

2. The data set synthesis method for detecting sticky insect pests in rice fields according to claim 1, characterized in that, The specific process of step S1 includes the following steps: S11. Obtain a publicly available pest dataset, filter out target pest images related to rice fields from the publicly available pest dataset, and clean and merge the filtered target pest images to obtain a set of target pest images of different categories. S12. Using Internet images and a diffusion generation model, the image samples of the different categories of target pests are expanded to obtain the final image sets of different categories of target pests.

3. The data set synthesis method for detecting sticky insect pests in rice fields according to claim 1, characterized in that, The specific process of step S3 includes: acquiring an image of a white sticky insect board, enhancing the image of the white sticky insect board with illumination perturbation, the illumination perturbation including brightness perturbation and color temperature perturbation, and obtaining a white sticky insect board background.

4. The method for synthesizing datasets for detecting sticky insect pests in rice fields according to claim 1, 2, or 3, characterized in that, Step S4 includes the following steps: using a pre-trained background segmentation model to extract the foreground from the images in the target pest image set, the images of accidentally caught insects, and the images of field debris, and then converting their original backgrounds to transparent backgrounds to obtain transparent background images of target pest instances, images of accidentally caught insect instances, and images of field debris instances; cropping each instance image to remove the transparent edges around the instance in each instance image, retaining only the smallest bounding rectangle region that just encloses the instance, to obtain instances of target pests, instances of accidentally caught insects, and instances of field debris.

5. The data set synthesis method for detecting sticky insect pests in rice fields according to claim 4, characterized in that, Step S5 includes the following steps: S51. Obtain a publicly available sticky insect board dataset, and select a portion of sticky insect board images from the publicly available sticky insect board dataset; S52. For each selected sticky insect board image, mark the center position of each insect in the image with a point to obtain a point-marked image; S53. Construct an insect point detection network; S53. Input the labeled image into the insect point detection network for training to obtain the trained insect point detection network; wherein, during training, each labeled point in the labeled image is converted into a Gaussian heat map label, so that each labeled point corresponds to a Gaussian peak in the heat map, and the Focal Loss loss function is used to train the insect point detection network to obtain the trained insect point detection network. S54. Input the remaining sticky insect board images from the publicly available sticky insect board dataset into the trained insect point detection network for prediction, and output the insect center point probability heatmap for each image; then extract the local maxima of each insect center point probability heatmap to obtain the center point coordinates of each insect.

6. The data set synthesis method for detecting sticky insect pests in rice fields according to claim 5, characterized in that, In step S53, the construction of the insect point detection network specifically includes: the insect point detection network includes an encoder, a decoder, an attention module, and a heatmap output head; the encoder uses ResNet18 as the backbone feature extraction network to extract image features at different scales; the decoder gradually restores spatial resolution through upsampling; the encoder and decoder fuse shallow detail features and deep semantic features through skip connections; the attention module includes a channel attention module and a spatial attention module, which are used to enhance channels sensitive to insect features and suppress background texture interference, respectively; the heatmap output head is used to output a probability heatmap of the insect center point.

7. The data set synthesis method for detecting sticky insect pests in rice fields according to claim 5, characterized in that, Step S6 includes the following steps: S61. Normalize the coordinates of the insect center point of each sticky insect board image obtained in step S5 to a unified sticky insect board coordinate system to obtain the normalized center point coordinates of each sticky insect board image; the unified sticky insect board coordinate system takes the upper left corner of the sticky insect board image as the origin and the width and height of the sticky insect board image as the reference, and normalizes the coordinates of each center point to the range of [0,1]. S62. Count the number of normalized center point coordinates for each sticky insect board image, and divide the density into multiple levels based on the total number of coordinates. S63. For each density level, establish a standard grid with a preset resolution, and map the normalized center point coordinates of pests extracted from all images within that level to the standard grid; count the number of center point coordinates falling within each grid cell of the standard grid to generate a two-dimensional frequency distribution matrix; perform Gaussian smoothing and probability normalization on the two-dimensional frequency distribution matrix to obtain the spatial probability heatmap corresponding to that density level.

8. The data set synthesis method for detecting sticky insect pests in rice fields according to claim 7, characterized in that, In step S7, the specific process of sampling a set of pasting position coordinates based on their probability density, in an amount equal to the total number of coordinates, includes the following steps: S71. Repeatedly sample randomly according to the probability density distribution of the selected spatial probability heatmap until the number of pasting position coordinates is equal to the total number. S72. According to the set uniform sampling ratio, randomly select a number of pasting position coordinates corresponding to the uniform sampling ratio from the pasting position coordinates and remove them, and uniformly and randomly sample an equal number of new position coordinates within the background range of the sticky insect board; the pasting position coordinates that were not removed and the new position coordinates obtained by uniform random sampling together constitute the pasting position coordinate set.

Images