Method and device for efficient classification and component proportion estimation of building solid waste

Abstract

The invention discloses a building solid waste efficient classification and component proportion estimation method and device. The method comprises the steps that a camera is used for collecting a building solid waste image with a black belt conveyor as the background; using a pre-trained target detection model YOLOv10 to identify, classify and position the building solid waste image; the method comprises the following steps of: segmenting a target by using an instance segmentation large model SegmentAnything Model (SAM), and estimating the area of the target to obtain the proportion of each component; and the mechanical arm is combined with the position information to carry out separation operation on various building solid wastes. Compared with a traditional building solid waste classification recycling method, the method can reduce manpower and material resource investment through deep learning, machine learning and other methods, and the industry benefits are improved. In addition, the method can accurately identify the proportion and position information of each component in the building solid waste, efficiently sorts and recycles the building solid waste, and is beneficial to promoting cyclic regeneration of effective materials in the building solid waste.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for efficiently classifying construction solid waste and estimating component ratios. Background Art

[0002] With the acceleration of urbanization, the output of construction solid waste is also increasing, bringing tremendous pressure to society and the environment. However, promoting the resource utilization of construction solid waste can alleviate the environmental and social problems caused by construction solid waste. Therefore, it is of great practical significance to study the efficient classification and component estimation of construction solid waste. At present, most construction solid waste is sorted manually, which is inefficient and the processing scale cannot meet the needs of the industry. Using machine vision technology to intelligently classify construction solid waste and reduce manpower and material input can effectively improve sorting efficiency and improve industry benefits. On the other hand, it can effectively improve the classification accuracy and recycling rate of construction solid waste, and reduce solid waste emissions and land occupancy. Therefore, it is necessary to establish a method and device for efficient classification and component ratio estimation of construction solid waste based on machine vision. Summary of the invention

[0003] In order to solve the defects in the prior art, the present invention discloses a method for efficient classification of construction solid waste and estimation of component ratios, and the technical solution is as follows:

[0004] S1, using a camera to collect images of construction solid waste with a black belt conveyor as the background;

[0005] S2, uses the pre-trained object detection model YOLOv10 to identify, classify and locate the construction solid waste images;

[0006] S3, use the instance segmentation model SegmentAnything Model (SAM) to segment the target, estimate its area and obtain the proportion of each component;

[0007] S4, after obtaining the location information, performs separation operations on various types of construction solid waste.

[0008] Beneficial Effects

[0009] (1) Based on the target detection algorithm, the location information and category information of various types of materials in construction solid waste can be effectively obtained, which is conducive to the separation of various types of materials;

[0010] (2) Based on the instance segmentation model, the material contours can be accurately segmented, the area proportion of each material can be obtained, and the component proportions of construction solid waste can be estimated. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of the image acquisition system of the present invention;

[0012] Figure 2 It is a schematic diagram of an embodiment of the method for classifying and estimating components of construction solid waste of the present invention;

[0013] Figure 3 This is a process flow chart of the construction solid waste sorting process of the present invention;

[0014] Figure 4 It is a structural diagram of the target detection model YOLOv10 in the present invention;

[0015] Figure 5 It is a structural diagram of the instance segmentation large model SAM in the present invention. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. These embodiments are provided to make the present invention more thoroughly and completely conveyed to those skilled in the art.

[0017] The first aspect of the present invention provides a method for efficiently sorting construction solid waste and estimating component ratios. The schematic diagram of the embodiment of the method is shown in FIG. Figure 2 As shown, the following steps are included:

[0018] S1, using a camera to collect images of construction solid waste with a black belt conveyor as the background;

[0019] Specifically, the image acquisition system is Figure 1 As shown, it includes: an industrial camera, a lighting lamp, a computer and a construction solid waste conveying system. The industrial camera is connected to the lighting lamp and the computer as a whole and is set above the heavy-running belt of the construction solid waste conveying system. The industrial camera can be used to obtain the image of construction solid waste with the black belt conveyor as the background, which is convenient for the sorting of various materials of construction solid waste.

[0020] S2, uses the pre-trained object detection model YOLOv10 to identify, classify and locate the construction solid waste images;

[0021] Specifically, the structure of the YOLOv10 model is as follows Figure 4 As shown. Compared with other models in the YOLO series, this model creatively combines the one-to-one and one-to-many modes. It not only uses the rich supervision signals of the one-to-many branch to train and optimize the model parameters to ensure the accuracy of the model, but also uses the one-to-one branch for inference to avoid NMS post-processing to improve the inference speed, that is, double label assignment. The components in YOLOv10 include the backbone, downsampling layers, stages and heads with basic building blocks. The main execution efficiency-driven model designs are: lightweight classification head, spatial channel decoupled downsampling, and sort-guided module design.

[0022] Specifically, the image processing module first uses the YOLOv10 target detection model to train the construction solid waste image dataset using transfer learning and data enhancement algorithms to learn the different image features corresponding to each material. After the training is completed, the required target detection model is obtained, which can be used to identify each material in the construction solid waste image and obtain its category information and location information.

[0023] Furthermore, the dataset is a collection of construction solid waste images taken in advance with black belts as the background. During the dataset production process, the labeling tool labelme software is used to annotate the targets in the image. The labels of the training set are the categories and coordinate positions of each target in the image. Then the YOLOv10 algorithm is used to train the training set. After the training is completed, a weight file is obtained. With the target detection model, the recognition of various materials can be realized, that is, the detected targets are divided into the set categories and their coordinate positions are obtained.

[0024] YOLOv10 introduces dual label assignment, retaining the same structure of the original one-to-many branch and using one-to-one matching to obtain label assignment. Both branches use a unified matching metric;

[0025]

[0026] Among them, m represents the matching metric, p is the classification score, and b denote the bounding boxes of the prediction and instance, respectively. s denotes the spatial prior indicating whether the predicted anchor point is within the instance. α and β are two important hyperparameters that balance the influence of the semantic prediction task and the location regression task. The one-to-many and one-to-one metrics are denoted as m o2m =m(α o2m ,β o2m ) and m o2o =m(α o2o ,β o2o ), where the subscripts o2m and o2o represent one-to-many branches and one-to-one branches, respectively. o2m , α o2m , β o2m represents the one-to-many branch metric and two hyperparameters, m o2o , α o2o , β o2o Represents the metrics and two hyperparameters of the one-to-one branch; these metrics affect the label assignment and supervision information of the two heads.

[0027] Assume that the one-to-many branch generates positive samples as Ω, and the one-to-one branch generates a metric as m o2o,i =m * o2o The ith prediction of , and then the classification target can be derived:

[0028]

[0029] Among them, t o2m,j is the task alignment loss value for the jth prediction in the one-to-many branch, j∈Ω; u * Represents the maximum predicted IoU, m * o2o is the maximum one-to-one matching score; IoU is the intersection over union ratio, which refers to the overlap rate of the “predicted bounding box” and the “real bounding box”, that is, the ratio of their intersection and union.

[0030] The supervision gap between the two branches is proposed by the 1-Wasserstein distance of different classification targets:

[0031]

[0032] Among them, A represents the supervision gap between the two branches, t o2o,i is the task alignment loss value of the i-th prediction in the one-to-one branch, t o2m,k Align the task loss value for the k-th prediction in the one-to-many branch.

[0033] S3, use the instance segmentation model SegmentAnythingModel (SAM) to segment the material, estimate its area and obtain the proportion of each component;

[0034] Specifically, SegmentAnythingModel (SAM) is a new AI model released by MetaAI that can segment any object in an image. The model is based on the new architecture of DetectionTransformer (DETR), combining the most advanced object detection and segmentation techniques with the attention mechanism used in natural language. The SAM model mainly consists of three modules: image encoding module, hint encoding module and mask decoding module. The model architecture is as follows: Figure 5 As shown in the figure. The structure of the SAM general model is not complicated, but with a very large amount of training data, it is believed to have learned the general features of an object containing pixel areas, so it has good zero-shot segmentation performance even for completely unfamiliar environments, showing results comparable to or even slightly exceeding those of well-trained supervised learning models. As the first suggestible image segmentation base model, SAM was trained on the large-scale SA-1B dataset, which has an unprecedented number of images and annotations, which gives the model excellent zero-shot generalization capabilities.

[0035] The image encoding module in the deep learning instance segmentation model SegmentAnythingModel (SAM) uses the vision transformer (VIT) architecture. Compared with the convolutional network, it is believed that it can improve the upper limit of the model training effect when the amount of data is large enough; the prompt encoding module uses convolution to encode dense mask prompts, and points and boxes use position encoding; the mask decoding module efficiently combines the image embedding matrix output by the image encoding module and the encoded prompt information, and derives the mask corresponding to this prompt information. First, the geometric contour features of the particle contour are extracted. The pixel value of the segmented material contour is between 0-1. The image is first thresholded, and then the area parameters are extracted separately.

[0036] The thresholding method converts the derived single mask into a digital matrix and performs thresholding according to formula (1);

[0037]

[0038] Where pix(x,y) is the pixel value at the (x,y) coordinate position on the original image, and pix'(x,y) is the pixel at the (x,y) coordinate position on the processed image;

[0039] The calculation formula of particle contour area is shown in formula (2):

[0040] A=α 2 ·∑n x,y ,x=1,2,3,···,w; y=1,2,3,···,h (2)

[0041] Where A is the calculated area of ​​the target contour, w is the image width, h is the image height, n x,y Indicates the mapping value of the pixel. When the pixel value is 0, n x,y is 1, α is the scale conversion factor, representing the real length represented by a single pixel (unit: mm per pixel);

[0042] By inputting the construction solid waste image and a 32×32 prompt point grid (the point grid size can be prompted according to the image size and the number of particles) into the instance segmentation large model, a material mask with precise contour boundaries can be obtained.

[0043] Furthermore, the OpenCV built-in functions findContours and contourArea are used to calculate the area of ​​the segmented materials, and then the output of the target detection algorithm YOLOv10 is integrated to obtain the category information and area size of the material. Therefore, the area size of each category of material can be calculated and the proportion of each component of construction solid waste can be estimated.

[0044] S4, after obtaining the location information, performs separation operations on various types of construction solid waste.

[0045] In this step, the mechanical control module is used to receive the category information and location information of the material, generate the motion trajectory through the internal program and code writing, and control the robot arm to grab the material and place it in different positions. Then it is necessary to send the material coordinate position identified by the target detection to the receiving module of the robot arm, and then the robot arm converts the pixel coordinates into the corresponding coordinate position in the robot arm's own coordinate system, and then combines the speed of the belt movement and the time for the robot arm to move to the specified position to give the robot arm an advanced coordinate offset. For example: a red brick is detected, and the pixel coordinate position of the red brick is (x, y). The mechanical control module converts the coordinates into the coordinates of the robot arm's own coordinate system (X, Y). After comprehensively considering the running speed of the conveyor belt and the moving time of the robot arm, it is determined that the robot arm needs to move from the initial position to the coordinates (X+X1, Y) to grab the red brick. The horizontal coordinate here represents the running direction of the conveyor belt, and the vertical coordinate represents the vertical direction of the running direction of the conveyor belt. The robot arm grabs the red brick to the red brick placement area to complete the efficient sorting of construction solid waste.

[0046] The second aspect of the present invention provides a device for efficiently sorting construction solid waste and estimating component ratios. The entire process is as follows: Figure 3 As shown, including:

[0047] The image acquisition module is used to use a camera to shoot the area within the conveyor belt. It includes: an industrial camera, a lighting lamp, a computer, and a construction solid waste conveying system. The industrial camera is connected to the lighting lamp and the computer as a whole and is set above the heavy running belt of the construction solid waste conveying system to obtain a dispersed and flattened image of the construction solid waste with the black conveyor belt as the background;

[0048] The image processing module is used to use the pre-trained target detection model YOLOv10 to identify the construction solid waste image, obtain the location information and category information of each material, and then use the instance segmentation model to fuse the target detection results to segment the materials, extract area parameters and category information to infer the component ratio.

[0049] The mechanical control module sends the material category information and location information to the receiving module of the robotic arm, and then generates a motion trajectory through internal programs and code writing to control the robotic arm to grab the material and realize the separation of construction solid waste.

[0050] In summary, the present invention provides a method and device for efficient classification and component ratio estimation of construction solid waste, including: using a camera to collect construction solid waste images with a black belt conveyor as the background; using a pre-trained target detection model YOLOv10 to identify, classify and locate the construction solid waste images; using the instance segmentation large model SegmentAnythingModel (SAM) to segment the target, estimate its area to obtain the proportion of each component; transmit the location information to the receiving module and use a mechanical arm to perform separation operations on various types of construction solid waste. The present invention can perform large-scale and efficient sorting of construction solid waste, reduce the manpower and material resources consumed by traditional construction solid waste sorting, improve the social and environmental problems caused by the stacking of construction solid waste, and improve industry benefits.

[0051] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for efficient classification and component ratio estimation of construction solid waste, characterized by: S1, using a camera to collect images of construction solid waste with a black conveyor belt conveyor as the background; S2, uses the pre-trained object detection model YOLOv10 to identify, classify and locate the construction solid waste images; S3, use the instance segmentation model SegmentAnything Model (SAM) to segment the target, estimate its area and obtain the proportion of each component; S4, after obtaining the location information, performs separation operations on various types of construction solid waste.

2. The method for efficient classification and component ratio estimation of construction solid waste according to claim 1 is characterized by: The construction solid waste is scattered and spread flat on a black conveyor belt, and the conveyor belt range is photographed with a camera. The captured images or videos contain various construction solid waste materials. The material categories are marked in the captured images or videos, and the target outlines are segmented to obtain the proportion of each category of materials.

3. The method for efficient classification and component ratio estimation of construction solid waste according to claim 1 is characterized by: The training set is enhanced by using a computer vision algorithm, and the training set is trained by transfer learning to obtain a target detection YOLOv10 model. The training set is a large number of construction solid waste images taken in advance with a black conveyor belt as the background.

4. The method for efficient classification and component ratio estimation of construction solid waste according to claim 3 is characterized by: Use software annotation tools to annotate the training set.

5. The method for efficient classification and component ratio estimation of construction solid waste according to claim 3 is characterized by: YOLOv10 introduces dual label assignment, retaining the same structure of the original one-to-many branch and using one-to-one matching to obtain label assignment; both branches use a unified matching metric; Among them, m represents the matching metric, p is the classification score, and b denote the bounding boxes of the prediction and instance, respectively. s denotes the spatial prior indicating whether the predicted anchor point is within the instance. α and β are two important hyperparameters that balance the influence of the semantic prediction task and the location regression task. The one-to-many and one-to-one metrics are denoted as m o2m =m(α o2m ,β o2m ) and m o2o =m(α o2o ,β o2o ), where the subscripts o2m and o2o represent one-to-many branches and one-to-one branches, respectively. o2m , α o2m , β o2m represents the one-to-many branch metric and two hyperparameters, m o2o , α o2o , β o2o Represents the metrics and two hyperparameters of the one-to-one branch; these metrics affect the label assignment and supervision information of the two heads.

6. The method for efficient classification and component ratio estimation of construction solid waste according to claim 5 is characterized by: Assume that the one-to-many branch generates positive samples as Ω, and the one-to-one branch generates a metric as m o2o,i =m * o2o The ith prediction of , and then the classification target can be derived: Among them, t o2m,j is the task alignment loss value for the jth prediction in the one-to-many branch, j∈Ω; u * Represents the maximum predicted IoU. IoU is the intersection-over-union ratio, which refers to the overlap ratio of the "predicted bounding box" and the "real bounding box", that is, the ratio of their intersection and union; m * o2o is the maximum one-to-one matching score. The supervision gap between the two branches is proposed by the 1-Wasserstein distance of different classification targets: A=t o2o,i -||(i∈Ω)t o2m,i +∑ k∈Ω\{i} t o2m,k Among them, A represents the supervision gap between the two branches, t o2o,i is the task alignment loss value of the i-th prediction in the one-to-one branch, t o2m,k Align the task loss value for the k-th prediction in the one-to-many branch.

7. The method for efficient classification and component ratio estimation of construction solid waste according to claim 3 is characterized by: The components in YOLOv10 include the backbone, downsampling layers, stages and heads with basic building blocks; the execution efficiency-driven model designs include: lightweight classification head, spatial channel decoupled downsampling, and sorting-guided module design.

8. The method for efficient classification and component ratio estimation of construction solid waste according to claim 1 is characterized by: The output results of image target detection are input into the instance segmentation model to segment the materials, and the component proportions of these construction solid wastes are estimated using the obtained categories and areas.

9. The method for efficient classification and component ratio estimation of construction solid waste according to claim 8, characterized by: Use the deep learning instance segmentation model SegmentAnything Model (SAM) to encode dense mask prompts using convolution, and position encoding for points and boxes; The mask decoding module efficiently combines the image embedding matrix output by the image encoding module and the encoded hint information, and derives the mask corresponding to the hint information.

10. The method for efficient classification and component ratio estimation of construction solid waste according to claim 9, characterized by: The SAM model is used to extract the geometric contour features of the particle contour. The pixel value of the segmented material contour is between 0 and 1. The image is thresholded first, and then the area parameters are extracted respectively.

11. The method for efficient classification and component ratio estimation of construction solid waste according to claim 10, characterized by: The thresholding method converts the derived single mask into a digital matrix and performs thresholding according to formula (1); Where pix(x,y) is the pixel value at the (x,y) coordinate position on the original image, and pix'(x,y) is the pixel at the (x,y) coordinate position on the processed image; The calculation formula of particle contour area is shown in formula (2): A=α 2 ·∑n x,y ,x=1,2,3,···,w;y=1,2,3,···,h (2) Where A is the calculated area of ​​the target contour, w is the image width, h is the image height, n x,y Indicates the mapping value of the pixel. When the pixel value is 0, n x,y is 1, α is the scale conversion factor, representing the actual length represented by a single pixel, unit: millimeter per pixel.

12. The method for efficient classification of construction solid waste and estimation of component ratio according to claim 10, characterized by: Use OpenCV built-in functions findContours and contourArea to calculate the area of ​​the segmented material.

13. The method for efficient classification of construction solid waste and estimation of component ratio according to any one of claims 1 to 12, characterized in that: The mechanical control device uses the material category and location information obtained through target detection to separate and recycle them.

14. A classification device, characterized in that: The device comprises the method for efficient classification of construction solid waste and estimation of component ratios according to any one of claims 1 to 13, wherein the classification device is used for classification of construction solid waste and comprises: An image acquisition module is used to use a camera to shoot the area within the conveyor belt range to obtain an image of construction solid waste that is dispersed and laid flat with the black conveyor belt as the background; An image processing module is used to identify the construction solid waste image using a pre-trained target detection model to obtain the location information and category information of each material, and then use the instance segmentation model to segment the material according to the target detection result, extract the area parameter to infer the component ratio; Mechanical control module: It is used to receive the category and location information of the material, generate the motion trajectory through internal program and code writing, control the mechanical arm to grab the material, and realize the separation of construction solid waste.

15. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is run, the device where the non-volatile storage medium is located is controlled to execute the method for efficient classification of construction solid waste and component ratio estimation as described in any one of claims 1 to 13.

16. An electronic device for efficient classification and component ratio estimation of construction solid waste, characterized in that: It comprises a processor and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein the computer-readable instructions, when executed, execute the method for efficient classification of construction solid waste and component ratio estimation as described in any one of claims 1 to 13.

Images