Machine vision-based automatic sorting method for impurities in recycled aggregate

Abstract

The application discloses a kind of based on machine vision's recycled aggregate impurity automatic sorting method, comprising the following steps: step 1, the aggregate impurity to be handled is carried out flotation treatment to remove the impurities floating on water surface, and then dewatering;Step 2, the aggregate impurity after being treated in step 1 is sorted visually;Step 3, the aggregate after being identified in step 2 is grabbed and classified by mechanical claw and is crushed.The sorting method has high degree of automation, can reduce manual intervention, reduce labor cost, and maintain consistent performance level in long time operation, improve the stability of overall system;YOLOv7 network is improved in visual sorting, so that the trained algorithm can accurately identify and detect recycled aggregate impurities with low parameter quantity.

Description

Technical Field

[0001] This invention relates to the field of recycled aggregate impurity sorting technology, specifically to an automatic sorting method for recycled aggregate impurities based on machine vision. Background Technology

[0002] Recycled aggregates are reusable building materials obtained by extracting and processing waste buildings, concrete or other materials.

[0003] Recycled aggregates play several important roles in building and infrastructure projects: They help reduce the demand for virgin resources, promote sustainable building and design, and reduce the construction industry's dependence on limited natural resources; they also help reduce the environmental impact of construction and demolition activities; and by reusing waste materials, waste generation can be reduced, alleviating pressure on landfills. The use of recycled aggregates can improve the performance of building materials, including strength and durability, which helps ensure the stability and long-term reliability of building structures. However, efficient and advanced recycled aggregate recycling and sorting technologies may require significant investment, as waste building materials are often complex mixtures, including concrete, bricks, wood, and other materials. Effectively separating and classifying these materials is a challenging task. Currently, most recycled aggregate recycling still relies on manual sorting, where operators often need to work for hours in workshops wearing breathing masks. While manual sorting of recycled aggregates has many advantages, it can also cause irreversible harm to the human body. Furthermore, manual sorting of recycled aggregates typically requires a large workforce, leading to high labor costs. Improving sorting speed and accuracy is a direction that those skilled in the art are dedicated to researching. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an automatic sorting method for impurities in recycled aggregates based on machine vision, which has high sorting efficiency, high precision and low cost.

[0005] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: An automatic sorting method for impurities in recycled aggregates based on machine vision includes the following steps: Step 1: The aggregate impurities to be processed are subjected to flotation to remove impurities floating on the water surface, and then dewatered; Step 2: Visually sort the aggregate impurities processed in Step 1, as follows: Step 21: Take photos of waste aggregate mixed with impurities, manually remove invalid image data that does not contain the target to be detected, build a dataset, and label the dataset; Step 22: Improve the YOLOv7 network to build the YOLOv7-FB model: a. Feature fusion layer improvement: In the YOLOv7 network head part, four FB-Concat modules are used to replace the Concat module to connect different feature layers. The FB-Concat module adds random weights to the original Concat module. b. Improved convolutional block: The Conv in the CBS layer of the ELAN-H part of the YOLOv7 network head is modified to Conv-2. Conv-2 splits the standard convolutional operation into two, and parallelizes the standard convolutional operation and DW convolution. c. Loss function improvement: The CIOU loss function in the YOLOv7 network is replaced with the EIOU loss function. The expression for the EIOU loss function is as follows: , in C w , C h The width and height of the smallest bounding box that covers both the predicted and ground truth boxes; d. Improve the model hyperparameter settings. During the warm-up training phase, the learning rate is increased from 0 to a preset value of 0.01. After the warm-up training, cosine annealing is used to update the learning rate. The parameters in the experimental training are set as follows: image-size is set to 640×640, batch-size is set to 32, learning rate is set to 0.01, momentum parameter is set to 0.937, weight-decay is set to 0.0005, and epochs are set to 200. Step 23: Train the YOLOv7-FB model established in Step 22 using the dataset created in Step 21; Step 24: Input the images of recycled aggregate impurities obtained by the camera into the YOLOv7-FB model trained in step 23 for recycled aggregate impurity identification. Step 3: The aggregate identified in Step 2 is grasped, sorted, and crushed by a mechanical gripper.

[0006] As a specific implementation method, in step 21, the dataset is manually annotated using the online annotation tool makesense.ai.

[0007] As a specific implementation method, in step 1, a water tank screw conveyor is used in the flotation process, and a dewatering screw conveyor is used in the dewatering process.

[0008] Compared with the prior art, the advantages of the present invention are: 1) Most of the impurities in recycled aggregates are lightweight objects such as plastics, wood blocks, and branches. These impurities can be quickly and effectively removed through flotation. At the same time, most of the recycled aggregates are demolition and construction waste, which contains a lot of dust. These particles can cause irreversible damage to the lungs when inhaled. Flotation can effectively remove floating dust particles, thus preventing harm when people come into contact with them later. 2) This invention uses visual inspection to sort aggregates and impurities, which is highly automated, reduces manual intervention and labor costs, and maintains a consistent performance level during long-term operation, thus improving the overall system stability. 3) Improvements to the YOLOv7 network enable the trained algorithm to identify and detect impurities in recycled aggregates with high accuracy and low parameter count. The experiment uses preheating training to maintain the depth stability of the network model and prevent the network model from oscillating due to an excessively high initial learning rate during training. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the sorting device in the embodiment; Figure 2 This is a flow chart of the sorting process of the sorting device in the embodiment; Figure 3 This is a logic diagram of the vision detection module grasping aggregate in the embodiment; Figure 4 This is an algorithm block diagram of the YOLOv7-FB model in the embodiment; Figure 5 This is a block diagram of the improved convolutional blocks in the YOLOv7-FB model in the embodiment; Among them, 100 is the feeding conveyor belt; 200 is the water trough screw conveyor; 300 is the dewatering screw conveyor; 400 is the vision inspection module; 500 is the robotic gripper; 600 is the impurity collection box; and 700 is the crusher. Detailed Implementation

[0010] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. This invention provides an automatic sorting method for impurities in recycled aggregates based on machine vision, comprising the following steps: Step 1: The aggregate impurities to be processed are subjected to flotation to remove impurities floating on the water surface, and then dewatered; Step 2: Visually sort the aggregate impurities processed in Step 1, as follows: Step 21: Take photos of waste aggregate mixed with impurities, manually remove invalid image data that does not contain the target to be detected, build a dataset, and label the dataset; Specifically, a dataset of photographs of waste aggregate mixed with impurities was collected for algorithm training. The final dataset was expected to contain four categories of targets: red bricks, wooden boards, plastics, and branches / wires. The dataset was collected automatically from above the production line. The collected images contained a large number of invalid images that did not contain the targets to be detected. The entire dataset needed to be manually screened to remove images that did not contain any targets to be detected. All algorithms used were supervised deep learning network algorithms, so the dataset needed to be labeled. In this example, the Makesense.ai online tool was used to label the image dataset. Makesense.ai software supports multiple label types, such as rectangles, lines, points, and polygons, and can output various file formats that meet the requirements of different models. By drawing the outer rectangles of the targets to be detected, the demolition and construction waste dataset was manually labeled, thus establishing the dataset. Step 22: Improve the YOLOv7 network to build the YOLOv7-FB model. See [link / reference]. Figure 4 As shown: a. Feature fusion layer improvement: In the YOLOv7 network head part, four FB-Concat modules are used to replace the Concat module to connect different feature layers. The FB-Concat module adds learnable and updatable random weights to the original Concat module. In the original YOLOv7 network, the feature fusion layer is similar to PAFPN, using four concats to connect different feature layers to achieve the purpose of fusing feature maps at different levels. However, different features have different effects on accuracy; some features will inevitably have a greater impact on accuracy, while others will have a smaller impact. Simply fusing different features is obviously very inefficient and may even prevent the network from fully learning the target features. Therefore, this example improves the feature fusion module by adding learnable and updatable weights to the feature fusion and naming the modified concat FB-Concat, replacing the four concats in the original network head part. By changing the weights, the influence of different features on the final fused feature map can be changed, making it easier for the network to learn the decisive features to improve accuracy. b. Improved convolutional blocks, see [link / reference] Figure 5 As shown, the Conv in the CBS layer of the ELAN-H part of the YOLOv7 network head is modified to Conv-2. Conv-2 splits the standard convolution operation into two, and performs the standard convolution operation and DW convolution in parallel. Specifically, real-time identification and handling of demolition and construction waste requires not only accurate target identification within complex concrete mixtures but also rapid detection to meet the demands of picking up and sorting impurities on a moving conveyor belt. While directly replacing standard convolution with deepwise convolution reduces the number of parameters, it also loses significant information, leading to decreased accuracy. To achieve the highest possible accuracy while reducing computational costs, please refer to... Figure 5 In this example, the Conv function in several CBS layers of the ELAN-H part of the network head was modified. The modified convolutional block was named Conv-2. The standard convolution operation was split into two, and the standard convolution operation and DW convolution were performed in parallel. The number of channels after the two operations was half that of the original standard convolution alone. Finally, the results of the two operations were shuffled and added together for output. This method ensures the amount of information by directly connecting the input with a standard convolution and a DW convolution, and reduces the number of parameters by using parallel DW convolution. c. Loss function improvement: The CIOU loss function in the YOLOv7 network is replaced with the EIOU loss function. The expression for the EIOU loss function is as follows: , in C w , C h The width and height of the smallest bounding box that covers both the predicted and ground truth boxes; d. Improved model hyperparameter settings: During the warm-up training phase, the learning rate was increased from 0 to a preset value of 0.01. After the warm-up training, cosine annealing was used to update the learning rate. To update and improve the weights of the network model, stochastic gradient descent was used during model training. The parameters in the experimental training were set as follows: image-size was set to 640×640, batch-size was set to 32, learning rate was set to 0.01, momentum parameter was set to 0.937, weight-decay was set to 0.0005, and epochs were set to 200. Step 23: Train the YOLOv7-FB model established in Step 22 using the dataset created in Step 21; Step 24: Input the images of recycled aggregate impurities obtained by the camera into the YOLOv7-FB model trained in step 23 for recycled aggregate impurity identification. Step 3: The aggregate identified in Step 2 is grasped, sorted, and crushed by a mechanical gripper.

[0011] The present invention also provides a sorting device employing the above-described sorting method, the sorting device comprising a feeding conveyor belt 100, a flotation conveying section, a vision inspection module 400, and a crusher 700.

[0012] In this example, the feeding conveyor belt uses an upward-sloping Z-shaped baffle conveyor belt, which is responsible for transporting the piles of recycled aggregate to the entire sorting system. The baffles divide the large amount of recycled aggregate into several parts. The large amount of recycled aggregate stacked on the conveyor belt slides down in the upward-sloping part and is blocked by the baffles located below the aggregate on the conveyor belt, and then is sent into the flotation conveying section.

[0013] The flotation conveying section consists of a trough-type screw conveyor 200 and a dewatering screw conveyor 300. The trough-type screw conveyor comprises a trough and a screw conveyor belt located at the bottom of the trough. Recycled aggregate, conveyed by the Z-shaped baffle conveyor belt, is transported into the trough via the screw conveyor belt at the bottom. Some lighter impurities are initially screened out, such as lighter branches, wood blocks, wires, and plastics, which float on the surface and are removed by dredging. Heavier recycled aggregate and red brick impurities sink to the bottom and come into contact with the screw conveyor belt, thus being fed forward. During the feeding process, due to the continuous movement and vibration of the screw conveyor belt, most of the squeezed or entangled light impurities are separated from the aggregate, thus achieving better flotation.

[0014] At the tail of the trough-type screw conveyor, there is an upwardly inclined dewatering screw conveyor with an inclination of 30°-40°. The highest point of this screw conveyor is higher than the upper surface of the trough, thus achieving dewatering. The material in this part is mainly some concrete blocks and red bricks from construction, as well as some impurities that were not removed by flotation due to excessive weight. The product after a large number of impurities have been removed by the trough-type screw conveyor is sent to the vision inspection module 400 through the dewatering screw conveyor.

[0015] Here, the visual inspection module mainly consists of an integrated box comprising a computer, a camera, and a light source. It is equipped with a trained high-precision visual inspection algorithm capable of identifying impurities in recycled aggregate. The camera faces the horizontal conveyor belt below, and the recycled aggregate at the front of the conveyor belt is identified by the camera. The identified impurities are located, and their position information is sent to the robotic gripper 500 at the rear. The robotic gripper is located at the rear of the integrated box, with one on each side of the conveyor belt's width. Several impurity collection boxes 600 are placed on both sides of the conveyor belt, each numbered and used to hold the impurities picked up by the robotic gripper. After dewatering, the impurities are fed into the horizontal conveyor belt and sorted by the visual inspection module. The visual inspection algorithm on this module can accurately identify impurities such as red bricks, plastics, branches, and wires. Impurities not filtered out during flotation are identified and picked up into designated impurity collection boxes at this stage. Since red bricks are not removed during flotation, this stage mainly focuses on detecting and identifying them. Detected red bricks are quickly and accurately placed into the corresponding impurity collection boxes by the robotic gripper. After impurity sorting, the remaining reusable recycled aggregate will be fed into crusher 700 for crushing. This crusher will break the large-volume, irregularly shaped recycled aggregate into smaller volumes for subsequent secondary use.

[0016] It should be emphasized that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A machine vision-based automatic sorting method for impurities in recycled aggregate, characterized in that, Includes the following steps: Step 1: The aggregate impurities to be processed are subjected to flotation to remove impurities floating on the water surface, and then dewatered; Step 2: Visually sort the aggregate impurities processed in Step 1, as follows: Step 21: Take photos of waste aggregate mixed with impurities, manually remove invalid image data that does not contain the target to be detected, build a dataset, and label the dataset; Step 22: Improve the YOLOv7 network to build the YOLOv7-FB model: a. Feature fusion layer improvement: In the YOLOv7 network head part, four FB-Concat modules are used to replace the Concat module to connect different feature layers. The FB-Concat module adds random weights to the original Concat module. b. Improved convolutional block: The Conv in the CBS layer of the ELAN-H part of the YOLOv7 network head is modified to Conv-2. Conv-2 splits the standard convolutional operation into two, and parallelizes the standard convolutional operation and DW convolution. c. Loss function improvement: The CIOU loss function in the YOLOv7 network is replaced with the EIOU loss function. The expression for the EIOU loss function is as follows: ， in C w , C h The width and height of the smallest bounding box that covers both the predicted and ground truth boxes; d. Improve the model hyperparameter settings. During the warm-up training phase, the learning rate is increased from 0 to a preset value of 0.

01. After the warm-up training, cosine annealing is used to update the learning rate. The parameters in the experimental training are set as follows: image-size is set to 640×640, batch-size is set to 32, learning rate is set to 0.01, momentum parameter is set to 0.937, weight-decay is set to 0.0005, and epochs are set to 200. Step 23: Train the YOLOv7-FB model established in Step 22 using the dataset created in Step 21; Step 24: Input the images of recycled aggregate impurities obtained by the camera into the YOLOv7-FB model trained in step 23 for recycled aggregate impurity identification. Step 3: The aggregate identified in Step 2 is grasped, sorted, and crushed by a mechanical gripper.

2. The automatic sorting method for impurities in recycled aggregate based on machine vision according to claim 1, characterized in that, In step 21, the dataset is manually annotated using the online annotation tool makesense.ai.

3. The automatic sorting method for impurities in recycled aggregate based on machine vision according to claim 1, characterized in that, In step 1, a water tank screw conveyor was used in the flotation process, and a dewatering screw conveyor was used in the dewatering process.

Images