Method and device for generating solid waste dataset based on sam segmentation model
By segmenting and automatically labeling color solid waste images using the SAM segmentation model, and combining copying and random stacking to generate a dataset under stacking conditions, the problem of generating low-cost and efficient labeled solid waste datasets is solved, achieving both dataset diversity and efficiency.
Patent Information
- Application Number
- CN202310786726.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-29
- Publication Date
- 2025-11-11
- Estimated Expiration
- 2043-06-29
AI Technical Summary
Existing technologies struggle to generate labeled solid waste datasets under stacked conditions in a low-cost and efficient manner, and traditional methods require extensive manual annotation and additional hardware.
The SAM segmentation model is used to segment color solid waste images, automatically annotating the contour information of object instances, and generating various stacking distributions through copying and random stacking to automatically generate dataset annotation files under sparse conditions.
It enables low-cost and efficient generation of labeled solid waste datasets, significantly reducing manual annotation costs, increasing dataset diversity, and adapting to complex stacked distribution conditions.
Smart Images
Figure CN117011639B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data augmentation, specifically relating to a method and apparatus for generating solid waste datasets based on the SAM segmentation model. Background Technology
[0002] With increasing urbanization and population growth, the large population and dense buildings generate a significant amount of household waste and construction debris, making the recycling of this solid waste a serious problem. Traditional recycling methods require substantial human resources, while deep learning-based computer vision technology can achieve automatic object recognition and classification, improving recycling efficiency. Instance segmentation can be well applied to contour recognition and classification of solid waste, but model training requires large datasets. Manually labeling the contours of objects in images is time-consuming and labor-intensive. While using depth cameras can obtain accurate object contours, it increases the hardware cost. Therefore, generating labeled datasets for stacked environments at low cost and high efficiency is a major challenge. Summary of the Invention
[0003] This invention proposes a method and apparatus for generating solid waste datasets based on the SAM segmentation model, which can generate labeled datasets under stacking conditions at low cost and high efficiency.
[0004] This invention is achieved through the following technical solution:
[0005] The method for generating solid waste datasets based on the SAM segmentation model includes the following steps:
[0006] Step S1: Collect color images of solid waste. All object instances in each solid waste image belong to the same solid waste category.
[0007] Step S2: Segment each solid waste image using the SAM segmentation model to obtain the contour information of each object instance in each solid waste image;
[0008] Step S3: Automatically classify and label the object instances in each solid waste image, and combine the contour information of each object instance in each solid waste image to obtain the dataset labeling file;
[0009] Step S4: Based on the object contour information in the dataset annotation file, copy each discrete object instance any number of times. All copied discrete object instances constitute a solid waste pool.
[0010] Step S5: Consider the combination of two stacks of different types of object instances, obtain the stacking combinations of all categories, copy the object instances corresponding to each stacking combination, and randomly stack them at different positions and with different degrees of stacking. Add the stacked instances to the solid waste pool.
[0011] This involves copying the object instance corresponding to each stacking combination and randomly stacking it at different positions and with varying degrees of stacking. Specifically, this includes:
[0012] For each stacking combination, a first instance and a second instance are randomly selected. The first instance and the second instance belong to the two categories corresponding to the stacking combination, respectively. A point within the outline range of the first instance is randomly selected as the occlusion point. A point in the set of outline points of the second instance is taken as the boundary point of the second instance. The coordinates of the boundary point of the second instance are equal to the coordinates of the occlusion point. At the same time, the image pixels within the range of the second instance are covered with the image pixels of the first instance to obtain the stacked instance.
[0013] Step S6: Select the background image to be pasted and generate distribution points. Randomly select stacked or discrete instances in the solid waste pool and paste them to the location of the distribution points.
[0014] Furthermore, the acquisition of color solid waste images in step S1 specifically includes:
[0015] Step S11: Divide the solid waste into several batches, each batch being of the same category, and place each batch of solid waste sparsely on the conveyor belt;
[0016] Step S12: Use an area array color camera positioned above the conveyor belt to take pictures of the solid waste on the conveyor belt to obtain a color image of the solid waste.
[0017] Furthermore, the solid waste categories include paper, PP lunch boxes, and Tetra Pak cartons.
[0018] Furthermore, in step S2, after obtaining the contour information of each object instance in the solid waste image, the area information is used to remove the interfering contours in the solid waste image.
[0019] Furthermore, in step S4, copying each discrete object instance specifically involves: generating a rectangular box surrounding each object instance based on the outline information of the object instance in the dataset annotation file; and copying and cropping the object instance based on the object instance's x-coordinate, y-coordinate, the width of the rectangular box, and the height of the rectangular box.
[0020] Furthermore, in step S6, a background image is selected, distribution points are generated in the background image, and random jitter is added.
[0021] This invention is also achieved through the following technical solutions:
[0022] A solid waste dataset generation device based on the SAM segmentation model includes:
[0023] Acquisition module: used to acquire color solid waste images. The object instances in each solid waste image are sparse, and all object instances in each solid waste image belong to the same solid waste category.
[0024] The dataset annotation file acquisition module is used to segment each solid waste image using the SAM segmentation model, obtain the contour information of each object instance in each solid waste image, and automatically label the object instances in each solid waste image. Combining the contour information of each object instance in each solid waste image, the dataset annotation file under sparse working conditions is obtained.
[0025] Copy module: Used to copy each discrete object instance any number of times based on the object outline information in the dataset annotation file. All copied discrete object instances constitute the solid waste pool.
[0026] Stacking module: It is used to consider the combination of two stacks of different types of object instances, obtain the stacking combinations of all categories, copy the object instances corresponding to each stacking combination, and randomly stack them at different positions and with different stacking degrees. The stacked instances are then added to the solid waste pool.
[0027] This involves copying the object instance corresponding to each stacking combination and randomly stacking it at different positions and with varying degrees of stacking. Specifically, this includes:
[0028] For each stacking combination, a first instance and a second instance are randomly selected. The first instance and the second instance belong to the two categories corresponding to the stacking combination, respectively. A point within the outline range of the first instance is randomly selected as the occlusion point. A point in the set of outline points of the second instance is taken as the boundary point of the second instance. The coordinates of the boundary point of the second instance are equal to the coordinates of the occlusion point. At the same time, the image pixels within the range of the second instance are covered with the image pixels of the first instance to obtain the stacked instance.
[0029] Paste module: Used to select the background image to paste and generate distribution points. Randomly select stacked or discrete instances in the solid waste pool and paste them to the location of the distribution points.
[0030] The present invention has the following beneficial effects:
[0031] 1. This invention first ensures that all object instances in each solid waste image belong to the same solid waste category when acquiring solid waste images. It then uses the SAM segmentation model to segment each solid waste image and automatically labels the object instances in each solid waste image according to their categories. This automatically generates a JSON annotation file for sparse conditions without the need for additional model training and hardware equipment, which can significantly reduce the cost of manual annotation and achieve good instance segmentation results. For the complex stacking distribution conditions during the detection process, it considers the pairwise stacking combinations of object instances of different categories to obtain stacking combinations of all categories. It then copies the object instances corresponding to each stacking combination and randomly stacks them at different positions and with different degrees of stacking, thereby expanding the stacking distribution to a variety of different situations. This effectively increases the diversity of the dataset and ultimately achieves low-cost and efficient generation of labeled datasets for stacking conditions. Attached Figure Description
[0032] The present invention will now be described in further detail with reference to the accompanying drawings.
[0033] Figure 1 This is a flowchart of the present invention.
[0034] Figure 2 The present invention relates to an image acquisition system.
[0035] Figure 3 An example image of solid waste paper according to the present invention.
[0036] Figure 4 An example image of the solid waste PP lunch box of the present invention.
[0037] Figure 5 An example image of a Tetra Pak bag for solid waste according to the present invention.
[0038] Figure 6 This is an example image of the outline label for the target object of the present invention.
[0039] Figure 7 This is an example of a dataset annotation file for this invention.
[0040] Figure 8 This is an example of a generated image under the stacking condition of the present invention. Detailed Implementation
[0041] like Figure 1 As shown, the method for generating solid waste datasets based on the SAM segmentation model includes the following steps:
[0042] Step S1: Collect color solid waste images. The object instances in each solid waste image are sparse, and all object instances in each solid waste image belong to the same solid waste category.
[0043] Specifically, using, such as Figure 2 The image acquisition system shown performs image acquisition. The image acquisition system includes a conveyor belt, an area array color camera set above the conveyor belt, and two LED light sources respectively set above the conveyor belt and located on both sides of the area array color camera. The specific installation structure of the area array color camera and the LED light sources is existing technology.
[0044] During collection, solid waste is divided into several batches, with each batch containing the same type of solid waste. The objects in each batch are placed sparsely on the conveyor belt. When each batch of solid waste is transported to the area array color camera, the area array color camera is controlled to take pictures, resulting in color images of the solid waste taken from top to bottom. This makes subsequent category labeling more convenient and faster.
[0045] In this embodiment, solid waste categories include paper, PP lunch boxes, and Tetra Pak cartons, as shown in the image example. Figures 3 to 5 As shown;
[0046] Step S2: Segment each solid waste image using the SAM segmentation model to obtain the contour information of each object instance in each solid waste image. Then, use area information to remove interfering contours in the solid waste images to obtain the contour label of the target object, such as... Figure 6 As shown; the specific process for removing interfering contours is existing technology;
[0047] Step S3: Automatically classify and label the object instances in each solid waste image. Combine the contour information of each object instance in each solid waste image to obtain the dataset annotation file (JSON file) under sparse conditions, such as... Figure 7 As shown; under the aforementioned photo settings, the object instances in each solid waste image belong to the same category, which is more conducive to category labeling;
[0048] Step S4: Based on the object contour information in the dataset annotation file, copy each discrete object instance any number of times. All copied discrete object instances constitute a solid waste pool.
[0049] The copying process is as follows:
[0050] Based on the outline information of the object instances in the JSON file, generate a rectangle that surrounds each object instance. Based on the x-coordinate, y-coordinate, width, and height of the rectangle, copy and crop the object instance; the number of copies can be freely selected.
[0051] Step S5: Consider the combination of two stacks of different types of object instances, obtain the stacking combinations of all categories, copy the object instances corresponding to each stacking combination, and randomly stack them at different positions and with different degrees of stacking. Add the stacked instances to the solid waste pool.
[0052] This involves copying the object instance corresponding to each stacking combination and randomly stacking it at different positions and with varying degrees of stacking. Specifically, this includes:
[0053] Considering the pairwise stacking combinations of PP lunch boxes, paper, and Tetra Pak, obtain all stacking combinations. For each stacking combination, randomly select a first instance and a second instance. The first instance and the second instance belong to the two categories corresponding to the stacking combination, respectively. Randomly select a point within the outline range of the first instance as the masking point. Take a point in the set of outline points of the second instance as the boundary point of the second instance, so that the coordinates of the boundary point of the second instance are equal to the coordinates of the masking point. At the same time, cover the image pixels of the first instance with the image pixels within the range of the second instance to obtain the stacked instance.
[0054] Step S6: Select the pasted background image and generate distribution points, and add random jitter. Randomly select stacked or discrete instances from the solid waste pool and paste them to the distribution point positions to obtain the generated image under stacking conditions, thereby achieving the purpose of augmenting the dataset. Figure 8 As shown.
[0055] Correspondingly, the solid waste dataset generation device based on the SAM segmentation model is characterized by comprising:
[0056] Acquisition module: used to acquire color solid waste images. The object instances in each solid waste image are sparse, and all object instances in each solid waste image belong to the same solid waste category.
[0057] The dataset annotation file acquisition module is used to segment each solid waste image using the SAM segmentation model, obtain the contour information of each object instance in each solid waste image, and automatically label the object instances in each solid waste image. Combining the contour information of each object instance in each solid waste image, the dataset annotation file under sparse working conditions is obtained.
[0058] Copy module: Used to copy each discrete object instance any number of times based on the object outline information in the dataset annotation file. All copied discrete object instances constitute the solid waste pool.
[0059] Stacking module: It is used to consider the combination of two stacks of different types of object instances, obtain the stacking combinations of all categories, copy the object instances corresponding to each stacking combination, and randomly stack them at different positions and with different stacking degrees. The stacked instances are then added to the solid waste pool.
[0060] This involves copying the object instance corresponding to each stacking combination and randomly stacking it at different positions and with varying degrees of stacking. Specifically, this includes:
[0061] For each stacking combination, a first instance and a second instance are randomly selected. The first instance and the second instance belong to the two categories corresponding to the stacking combination, respectively. A point within the outline range of the first instance is randomly selected as the occlusion point. A point in the set of outline points of the second instance is taken as the boundary point of the second instance. The coordinates of the boundary point of the second instance are equal to the coordinates of the occlusion point. At the same time, the image pixels within the range of the second instance are covered with the image pixels of the first instance to obtain the stacked instance.
[0062] Paste module: Used to select the background image to paste and generate distribution points. Randomly select stacked or discrete instances in the solid waste pool and paste them to the location of the distribution points to obtain the generated image under stacked conditions.
[0063] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the present invention. All equivalent changes and modifications made in accordance with the scope of the patent application and the contents of the specification of the present invention should still fall within the scope of the patent of the present invention.
Claims
1. A method for generating solid waste datasets based on the SAM segmentation model, characterized by: Includes the following steps: Step S1: Collect color images of solid waste. All object instances in each solid waste image belong to the same solid waste category. Step S2: Segment each solid waste image using the SAM segmentation model to obtain the contour information of each object instance in each solid waste image; Step S3: Automatically classify and label the object instances in each solid waste image, and combine the contour information of each object instance in each solid waste image to obtain the dataset labeling file; Step S4: Based on the object contour information in the dataset annotation file, copy each discrete object instance any number of times. All copied discrete object instances constitute a solid waste pool. Step S5: Consider the combination of two stacks of different types of object instances, obtain the stacking combinations of all categories, copy the object instances corresponding to each stacking combination, and randomly stack them at different positions and with different degrees of stacking. Add the stacked instances to the solid waste pool. This involves copying the object instance corresponding to each stacking combination and randomly stacking it at different positions and with varying degrees of stacking. Specifically, this includes: For each stacking combination, a first instance and a second instance are randomly selected. The first instance and the second instance belong to the two categories corresponding to the stacking combination, respectively. A point within the outline range of the first instance is randomly selected as the occlusion point. A point in the set of outline points of the second instance is taken as the boundary point of the second instance. The coordinates of the boundary point of the second instance are equal to the coordinates of the occlusion point. At the same time, the image pixels within the range of the second instance are covered with the image pixels of the first instance to obtain the stacked instance. Step S6: Select the background image to be pasted and generate distribution points. Randomly select stacked or discrete instances in the solid waste pool and paste them to the location of the distribution points.
2. The method for generating solid waste datasets based on the SAM segmentation model according to claim 1, characterized in that: The acquisition of color solid waste images in step S1 specifically includes: Step S11: Divide the solid waste into several batches, each batch being of the same category, and place each batch of solid waste sparsely on the conveyor belt; Step S12: Use an area array color camera positioned above the conveyor belt to take pictures of the solid waste on the conveyor belt to obtain a color image of the solid waste.
3. The method for generating solid waste datasets based on the SAM segmentation model according to claim 1, characterized in that: The solid waste categories include paper, PP lunch boxes, and Tetra Pak cartons.
4. The method for generating solid waste datasets based on the SAM segmentation model according to claim 1, 2, or 3, characterized in that: In step S2, after obtaining the contour information of each object instance in the solid waste image, the area information is used to remove the interfering contours in the solid waste image.
5. The method for generating solid waste datasets based on the SAM segmentation model according to claim 1, 2, or 3, characterized in that: In step S4, copying each discrete object instance specifically involves: generating a rectangular box surrounding each object instance based on the outline information of the object instances in the dataset annotation file; and copying and cropping the object instances based on the x-coordinate, y-coordinate, width, and height of the rectangular box.
6. The method for generating solid waste datasets based on the SAM segmentation model according to claim 1, 2, or 3, characterized in that: In step S6, a background image is selected, distribution points are generated in the background image, and random jitter is added.
7. A solid waste dataset generation device based on the SAM segmentation model, characterized in that: include: Acquisition module: used to acquire color solid waste images. The object instances in each solid waste image are sparse, and all object instances in each solid waste image belong to the same solid waste category. The dataset annotation file acquisition module is used to segment each solid waste image using the SAM segmentation model, obtain the contour information of each object instance in each solid waste image, and automatically label the object instances in each solid waste image. Combining the contour information of each object instance in each solid waste image, the dataset annotation file under sparse working conditions is obtained. Copy module: Used to copy each discrete object instance any number of times based on the object outline information in the dataset annotation file. All copied discrete object instances constitute the solid waste pool. Stacking module: It is used to consider the combination of two stacks of different types of object instances, obtain the stacking combinations of all categories, copy the object instances corresponding to each stacking combination, and randomly stack them at different positions and with different stacking degrees. The stacked instances are then added to the solid waste pool. This involves copying the object instance corresponding to each stacking combination and randomly stacking it at different positions and with varying degrees of stacking. Specifically, this includes: For each stacking combination, a first instance and a second instance are randomly selected. The first instance and the second instance belong to the two categories corresponding to the stacking combination, respectively. A point within the outline range of the first instance is randomly selected as the occlusion point. A point in the set of outline points of the second instance is taken as the boundary point of the second instance. The coordinates of the boundary point of the second instance are equal to the coordinates of the occlusion point. At the same time, the image pixels within the range of the second instance are covered with the image pixels of the first instance to obtain the stacked instance. Paste module: Used to select the background image to paste and generate distribution points. Randomly select stacked or discrete instances in the solid waste pool and paste them to the location of the distribution points.