A method and system for sorting debris on a vibrating screen

Abstract

This invention relates to the field of machine vision technology, specifically to a method and system for sorting debris on a vibrating screen. The method first uses several first cameras to simultaneously acquire images of different areas of the detection zone at the inlet of the vibrating screen. After deblurring and image stitching, debris is identified to determine its planar position and the first waiting position of the gripper. The gripper is then moved from its initial position to the first waiting position. Next, a second camera on the gripper acquires the three-dimensional coordinates of the debris, reflecting its height, and a second waiting position is planned. Simultaneously, as the gripper moves from the first to the second waiting position, its posture is adjusted according to the shape of the debris. Finally, after the gripper reaches the second waiting position, it maintains its gripping posture and moves directly downwards to the three-dimensional coordinates of the debris to grasp it. This method effectively improves the timeliness and accuracy of the gripper's grasping action, while shortening the overall operation time and increasing grasping efficiency.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a method and system for sorting debris on a vibrating screen. Background Technology

[0002] In the coal washing plant production process, sorting impurities from vibrating screens is primarily aimed at improving coal quality, ensuring production safety, and enhancing economic efficiency. As a key piece of equipment in coal sorting, the vibrating screen classifies coal by particle size. Impurities mixed in (such as mesh, rubber, wood blocks, and metal) not only reduce coal purity, affecting combustion efficiency or subsequent processing, but may also damage the screen or downstream equipment, increasing maintenance costs. Furthermore, flammable materials in the impurities can pose safety hazards, and metal parts can even lead to mechanical failures or spark risks. Therefore, impurity sorting on vibrating screens is essential.

[0003] Currently, the sorting of debris on vibrating screens in coal washing plants mainly relies on manual sorting. However, this method suffers from low sorting efficiency and a high rate of missed detection, making it difficult to meet the continuous and efficient production demands of modern coal washing plants. Furthermore, workers are exposed to vibration, noise, and dust for extended periods, facing occupational health risks such as pneumoconiosis and hearing damage. Intelligent sorting robots, through AI visual recognition and precise robotic grasping, can operate 24 / 7, significantly improving sorting accuracy and efficiency while reducing the risk of production safety accidents. However, during the operation of the vibrating screen, the continuous vibration makes it difficult for intelligent sorting robots to accurately identify and grasp target debris, resulting in a high rate of mis-sorting and missed sorting. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for sorting debris on a vibrating screen that improves the path planning for grabbing multiple debris on the vibrating screen, reduces the amount of calculation, and improves the sorting accuracy and efficiency.

[0005] The technical solution of this invention is as follows: A method for sorting debris on a vibrating screen includes the following operations: S1. Several first cameras simultaneously acquire images of different areas of the detection area on the inclined screen, resulting in several detection area images; after deblurring, the images are stitched together to obtain a stitched detection area image; the detection area is located at the entrance of the vibrating screen; the stitched detection area image is processed by target detection to obtain a debris recognition result; based on the debris position in the debris recognition result, a first waiting position is obtained, and the gripper is moved from the initial position to the first waiting position; S2. Based on the second camera on the gripper, acquire point cloud data of the debris to obtain the three-dimensional coordinates of the debris; based on the three-dimensional coordinates of the debris, acquire the second waiting position of the gripper; move the gripper from the first waiting position to the second waiting position, and during the movement, adjust the gripper to the corresponding grasping posture according to the shape of the debris in the debris recognition result. S3. After the gripper moves to the second waiting position, it maintains the gripping posture and moves downward to the three-dimensional coordinates of the debris to grip the debris.

[0006] With the initial position of the gripper as the origin, the projection of the coal flow direction on the ground on the vibrating screen is the positive Y-axis, the direction perpendicular to the ground is the positive Z-axis, and the X-axis is established perpendicular to the Z-axis and the Y-axis; in S1, the first waiting position is (x1, y1+deltY, z0+deltZ1), (x1, y1) is the position of the debris, z0 is the vertical coordinate of the initial position, deltY and deltZ1 are the first longitudinal distance and the first vertical distance, respectively, and deltY and deltZ1 are both greater than 0.

[0007] In S2, the second waiting position is (x2, y2, z2 + beltZ2), (x2, y2, z2) are the three-dimensional coordinates of the debris, beltZ2 is the second vertical distance, and beltZ2 is less than 0.

[0008] In S2, the gripping posture of the gripper includes the gripper opening degree and the gripper rotation angle; the gripper rotation angle is the angle between the coal flow direction on the vibrating screen in the splicing diagram of the detection area; the gripper opening degree is the sum of the maximum width of the debris and the reserved safety gap, or the sum of the minimum width of the debris and the reserved safety gap.

[0009] When the sum of the maximum width of the debris and the reserved safety gap is not greater than the gripper opening threshold, the gripper opening is the sum of the maximum width of the debris and the reserved safety gap, and the gripper gripping position is at the maximum width of the debris; when the sum of the maximum width of the debris and the reserved safety gap is greater than the gripper opening threshold, the gripper opening is the sum of the minimum width of the debris and the reserved safety gap, and the gripper gripping position is at the minimum width of the debris.

[0010] In S1, the deblurring process for the current detection area image is as follows: Gaussian filtering and edge enhancement are performed on the current detection area image to obtain the current denoised image; a blur kernel is obtained based on the vibration amplitude and direction of the vibrating screen and the resolution of the first camera; based on the blur kernel, deconvolution processing based on blur iteration is performed on the current denoised image to obtain the initial deblurred image of the current detection area; the initial deblurred image of the current detection area is sharpened to enhance edge details and improve image detail contrast to obtain the current deblurred image of the current detection area.

[0011] The image stitching process is as follows: After converting all deblurred images of the detection regions into grayscale images of the same size, feature point pairs between adjacent grayscale images of the detection regions are obtained; based on each feature point pair, the corresponding homography matrix is ​​obtained as the transformation matrix of the corresponding adjacent deblurred images of the detection regions; based on the transformation matrix, one deblurred image of the detection regions is mapped onto another deblurred image of the detection regions, and the superimposed pixels of the overlapping regions in the adjacent deblurred images of the detection regions are weighted and averaged to obtain the stitched image of the detection regions; the deblurred images of the detection regions are obtained by deblurring the detection region images.

[0012] A debris sorting system for a vibrating screen, used to implement the above-mentioned debris sorting method for a vibrating screen, includes: A vibrating screen, with an inclined screen surface, is used to convey coal lumps and debris; A gantry frame spans the screen surface and is positioned in the forward direction of the vibrating screen. Several first vision units are installed on the upper part of the gantry frame, with the shooting direction of the first vision units facing the screen surface of the vibrating screen. A robot is installed on the upper part of the gantry frame. The first visual unit is used to acquire images of the detection area and transmit them to the computing unit; The second vision unit is used to acquire point cloud data of clutter and transmit it to the computing unit; The computing unit is used to perform deblurring, image stitching, and target detection processing on the image transmitted from the first vision unit to obtain the clutter recognition result. Based on the clutter position in the clutter recognition result, it obtains the first waiting position, converts it into a control signal, and transmits it to the control unit. It is also used to obtain the three-dimensional coordinates of the clutter based on the point cloud data of the clutter transmitted from the second vision unit. Based on the three-dimensional coordinates of the clutter, it obtains the second waiting position of the gripper, converts it into a control signal, and transmits it to the control unit. The control unit is used to control the movement of the gripper and the gripping of debris based on the control signals transmitted from the computing unit; The robot has grippers attached to its end for grasping debris; A miscellaneous waste basket is placed on one side of the vibrating screen outlet to hold miscellaneous items; The lighting unit is located on the upper part of the gantry frame, with the illumination direction facing the vibrating screen surface.

[0013] The beneficial effects of this invention are as follows: This invention provides a method for sorting debris on a vibrating screen. First, several first cameras simultaneously acquire images of different areas at the screen inlet detection zone. After deblurring and image stitching, debris is identified to determine its planar position and the first waiting position of the gripper. The gripper is then moved from its initial position to the first waiting position, allowing it to adjust its position in advance and reducing the time spent on subsequent posture and movement adjustments. Next, a second camera on the gripper acquires the three-dimensional coordinates of the debris, reflecting its height, and a second waiting position is planned. Simultaneously, during the movement of the gripper from the first to the second waiting position, its posture is adjusted according to the debris's shape, allowing it to reach the optimal gripping position more quickly. Finally, once the gripper reaches the second waiting position, it maintains its gripping posture and moves directly downwards to the debris's three-dimensional coordinates for grasping. This effectively improves the timeliness and accuracy of gripping, while shortening the overall operation time and increasing gripping efficiency. Attached Figure Description

[0014] The solutions and advantages of this application will become clear to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0015] In the attached diagram: Figure 1 This is a schematic diagram of the system structure in this embodiment. Figure 2 This is a schematic diagram of the gripper structure in the embodiment; 1. Vibrating screen; 2. Gantry frame; 3. First vision unit; 4. Gripper; 5. Robot; 6. Lighting unit; 7. Control unit; 8. Miscellaneous basket. Detailed Implementation

[0016] Exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings.

[0017] This embodiment provides a debris sorting system for a vibrating screen, used to implement a method for sorting debris on a vibrating screen. (See also...) Figure 1 ,include: Vibrating screen 1 has an inclined screen surface with the vibration direction perpendicular to the screen surface. The vibrating screen is used to transport coal blocks and impurities, and to classify the coal according to particle size through screening. The inclination of the screen surface near the inlet of the vibrating screen is greater than that near the outlet of the vibrating screen. This facilitates the rapid identification of impurities near the inlet of the vibrating screen and their slow movement near the outlet of the vibrating screen, ensuring that the robot gripper can grasp them in a timely manner. The gantry frame 2 spans the screen surface and is positioned in the forward direction of the vibrating screen. Several first vision units 3 are installed on the upper part of the gantry frame, with the first vision units 3 shooting directly at the screen surface of the vibrating screen. A robot 5 is suspended in an inverted installation method at the middle of the upper part of the gantry frame. This installation method breaks through the limitation of insufficient arm span of traditional ground-installed robots, and at the same time, it utilizes the advantages of high-altitude operation to avoid interference with the screen surface equipment, providing a basic guarantee for subsequent accurate sorting operations. The first visual unit 3 is used to acquire images of the detection area and transmit them to the computing unit; The second vision unit is used to acquire point cloud data of the debris and transmit it to the computing unit; the second camera is set on the robot gripper 4 to facilitate the real-time acquisition of the accurate three-dimensional coordinate position of the debris. The computing unit is used to perform deblurring, image stitching, and target detection processing on the image transmitted from the first vision unit to obtain the clutter recognition result. Based on the clutter position in the clutter recognition result, it obtains the first waiting position, converts it into a control signal, and transmits it to the control unit. It is also used to obtain the three-dimensional coordinates of the clutter based on the point cloud data of the clutter transmitted from the second vision unit. Based on the three-dimensional coordinates of the clutter, it obtains the second waiting position of the gripper, converts it into a control signal, and transmits it to the control unit. The control unit 7 is used to control the movement of the gripper 4 and to grasp the debris according to the control signals transmitted from the computing unit; Robot 5, with gripper 4 attached to the end of its robotic arm, see [link / reference]. Figure 2 Used to grab debris; The lighting unit 6 (preferably a spotlight) is set on the upper part of the gantry frame, with the irradiation direction facing the screen surface of the vibrating screen 1, to ensure that the light uniformly covers the entire visual inspection area and reduce false detections caused by shadows or reflections. Miscellaneous item basket 8 is placed on one side of the exit in the forward direction of the vibrating screen and is used to hold miscellaneous items.

[0018] Specifically, the screen surface of the vibrating screen 1 is inclined, and the vibration direction is perpendicular to the screen surface. At the end point of the vibrating screen's forward direction (the direction of coal flow on the vibrating screen) (the outlet of the vibrating screen), a gantry frame 2 is set across the screen surface of the vibrating screen to ensure its structural stability and to avoid occupying additional space. Several first vision units 3 (first cameras) are installed on the upper crossbeam of the gantry frame, with the shooting direction facing the screen surface of the vibrating screen. The first vision unit 3 (first camera) is used to capture images of the detection area on the vibrating screen to identify the debris on the detection area at the entrance of the vibrating screen, facilitating the acquisition of the initial planar position information of the debris. The second vision unit (second camera) is set on the gripper 4 to acquire the point cloud data of the debris, determine the height of the debris, and facilitate subsequent gripping planning.

[0019] In addition, a robot 5 is also installed in the middle of the upper crossbeam of the gantry frame 2 (in the inverted installation method, there is a hoisting robot), so that the robot arm can cover the entire screen surface and avoid obstructing the camera's field of view, while also being able to detect debris in advance; the robot's end is equipped with a rotatable gripper 4, which is used to grab debris on the screen surface.

[0020] Meanwhile, a lighting unit 6 (preferably a spotlight) is installed on the gantry frame 2 to ensure uniform illumination covering the entire screen surface, improving illumination consistency and helping to improve the accuracy of debris detection, reducing false detections caused by shadows or reflections. A debris basket 8 is provided on one side of the vibrating screen outlet, where the grippers pick up debris and place it in the debris basket 8.

[0021] The aforementioned first camera and robot are calibrated. The calibration method is as follows: select a corner point on the vibrating screen, determine its position in the camera coordinate system and robot coordinate system respectively using a calibration tool (such as a checkerboard), calculate the rotation matrix from the camera coordinate system to the robot coordinate system, and based on the rotation matrix, convert the three-dimensional coordinates of the camera coordinate system into coordinates in the robot coordinate system, thereby establishing an accurate coordinate mapping relationship to ensure that the detected debris position can be accurately converted into robot motion commands.

[0022] The second camera and the robot were also calibrated. The calibration method was as follows: the robot arm end effector was controlled to drive the second camera to take pictures of the same calibration plate in different poses. The three-dimensional coordinates of the corner points of the calibration plate in the camera coordinate system were determined by the intrinsic parameters and depth information of the second camera. At the same time, the end effector coordinate system parameters of the robot arm in each shooting pose (based on the robot base coordinate system) were recorded. The transformation matrix from the camera coordinate system to the robot arm end effector coordinate system was calculated. Based on this transformation matrix, the three-dimensional coordinates of the object detected by the second camera can be transformed to the robot arm end effector coordinate system first, and then combined with the robot kinematic model to transform to the robot base coordinate system. This establishes an accurate coordinate mapping relationship and ensures that the position of the object detected by the second camera can be accurately converted into the motion command of the robot end effector.

[0023] This embodiment provides a method for sorting debris on a vibrating screen, including the following operations: S1. Several first cameras simultaneously acquire images of different areas of the detection area on the inclined screen, resulting in several detection area images; after deblurring, the images are stitched together to obtain a stitched detection area image; the detection area is located at the entrance of the vibrating screen; the stitched detection area image is processed by target detection to obtain a debris recognition result; based on the debris position in the debris recognition result, a first waiting position is obtained, and the gripper is moved from the initial position to the first waiting position; S2. Based on the second camera on the gripper, acquire point cloud data of the debris to obtain the three-dimensional coordinates of the debris; based on the three-dimensional coordinates of the debris, acquire the second waiting position of the gripper; move the gripper from the first waiting position to the second waiting position, and during the movement, adjust the gripper to the corresponding grasping posture according to the shape of the debris in the debris recognition result. S3. After the gripper moves to the second waiting position, it maintains the gripping posture and moves downward to the three-dimensional coordinates of the debris to grip the debris.

[0024] The specific operating steps are detailed below.

[0025] S1. Several first cameras simultaneously acquire images of different areas of the detection area on the inclined screen, resulting in several detection area images; after deblurring, the images of the detection areas are stitched together to obtain a stitched image of the detection area; the detection area is located at the entrance of the vibrating screen; the stitched image of the detection area is processed by target detection to obtain the debris recognition result; based on the location of the debris in the debris recognition result, the first waiting position is obtained, and the gripper is moved from the initial position to the first waiting position.

[0026] By simultaneously acquiring images of different areas of the detection zone at the inlet of the vibrating screen using several primary cameras, and then deblurring the images to eliminate vibration interference before stitching them together to form a complete detection zone image, the stitched image of the detection zone can be made clear and comprehensive, providing a high-quality image foundation for subsequent target detection. This allows for accurate identification of debris and determination of its planar position. Based on this position, the first waiting position of the gripper is determined, and then the gripper is moved from its initial position to the first waiting position. This allows the gripper to adjust its position in advance, reducing the time spent on subsequent adjustment of the gripper's posture and movement, and improving the accuracy and efficiency of subsequent debris grasping.

[0027] First, several primary cameras simultaneously acquire images of different areas of the vibrating screen on the inclined screen surface, resulting in several images of the detection area. (See also...) Figure 1 The detection area is located at the entrance of the vibrating screen.

[0028] Then, after deblurring the images of several detection areas, the image details blurred due to vibration are removed, resulting in several deblurred images of the detection areas; the deblurred images of the detection areas are then stitched together to obtain a stitched image of the detection areas.

[0029] The steps for deblurring the image of the current detection area are as follows.

[0030] Step 1: Perform Gaussian filtering and edge enhancement on the current detection area image to eliminate noise and suppress boundary artifacts, thus obtaining the current denoised image.

[0031] Step 2: Based on the vibration amplitude and direction of the vibrating screen and the resolution of the camera (first camera), obtain the fuzzy kernel.

[0032] The formula for calculating the fuzzy kernel is as follows: , , The location in the current detection region image is ( x , y The fuzzy kernel value at ) The blur length (in pixels). , , These are vibration amplitude, vibration direction, and camera (first camera) resolution, respectively.

[0033] Step 3: Based on the blur kernel, perform deconvolution processing on the current denoised image based on blur iteration to perform image deviator blur, restore the details that have been blurred and diffused, and obtain the initial deblurred image of the current detection area.

[0034] The deconvolution process based on fuzzy iteration is achieved through the following formula: , For the first k+1 The position of the current detection region in the initial deblurred image after the next iteration is ( x , y The estimated pixel value at position ) For the first k The position of the current detection region in the initial deblurred image after the next iteration is ( x , y The estimated pixel value at position ) The position in the current denoised image is ( x , y The pixel value at ) This is for convolution processing.

[0035] Step 4: The initial deblurred image of the current detection area is sharpened to enhance edge details and improve image detail contrast, thus obtaining the deblurred image of the current detection area.

[0036] The above sharpening process can be achieved using the following formula: , Deblur the image of the current detection area. This is the initial deblurred image for the current detection region. This is the sharpening factor. To increase the sharpening intensity.

[0037] The image stitching operation is as follows: after converting all deblurred images of the detection regions into grayscale images of the same size, feature point pairs between adjacent grayscale images of the detection regions are obtained; based on each feature point pair, the corresponding homography matrix is ​​obtained as the transformation matrix of the corresponding adjacent deblurred images of the detection regions; based on the transformation matrix, one deblurred image of the adjacent deblurred images of the detection regions is mapped to another deblurred image of the detection regions, and the superimposed pixels of the overlapping regions in the adjacent deblurred images of the detection regions are weighted and averaged to obtain the stitched image of the detection regions.

[0038] Next, the mosaic image of the detection area is processed by object detection (including but not limited to the Faster R-CNN model) to identify clutter in the detection area, obtain the shape of the clutter and the location information of the clutter in the detection area (clutter position), and obtain the clutter identification result.

[0039] The method for obtaining the morphology of the clutter is as follows: based on the clutter recognition and detection results, the clutter in the spliced ​​image of the detection area is processed by image segmentation (including but not limited to using the YOLOv8 model or GAN network), the clutter image is extracted, and edge contour fitting is performed to obtain a clutter segmentation image that reflects the morphology of the clutter.

[0040] The location of the debris is the center position of the smallest outer matrix in the debris segmentation map corresponding to the spliced ​​image of the detection area.

[0041] Finally, based on the location of the debris in the debris recognition result, the first waiting position of the gripper is obtained, and the gripper is moved from the initial position to the first waiting position in order to adjust its posture in advance and wait for the debris to arrive.

[0042] Taking the initial position of the gripper as the origin, the projection of the coal flow direction on the ground on the vibrating screen is the positive Y-axis, the direction perpendicular to the ground is the positive Z-axis, and the X-axis is established perpendicular to the Z-axis and the Y-axis. Then the first waiting position is (x1, y1+deltY, z0+deltZ1), where (x1, y1) is the position of the debris, z0 is the vertical coordinate of the initial position, and deltY and deltZ1 are the first longitudinal distance and the first vertical distance, respectively.

[0043] S2. Based on the second camera on the gripper, acquire point cloud data of the debris to obtain the three-dimensional coordinates of the debris; based on the three-dimensional coordinates of the debris, acquire the second waiting position of the gripper; move the gripper from the first waiting position to the second waiting position, and during the movement, adjust the gripper to the corresponding grasping posture according to the shape of the debris in the debris recognition result.

[0044] The second camera on the gripper acquires the three-dimensional coordinates of the object and plans the second waiting position. At the same time, the gripper's posture is adjusted according to the shape of the object during movement, which allows the gripper to reach the optimal gripping preparation position more quickly, reducing invalid movement time to ensure timeliness. Furthermore, through precise three-dimensional positioning and posture adaptation, the gripping accuracy can be greatly improved, reducing the risk of gripping deviation or falling off, and ensuring gripping accuracy.

[0045] First, since the vibration intensity of the vibrating screen is small, the horizontal displacement of the debris is very small and can be ignored. Therefore, in this embodiment, when the gripper is in the second waiting position, the point cloud data of the debris is obtained based on the second camera on the gripper (preferably a 3D camera, which can realize the fusion of the RGB and depth information of the debris, making it easier to locate and identify the debris and obtain point cloud data). The three-dimensional coordinates of the debris are then obtained.

[0046] Then, based on the three-dimensional coordinates of the debris, the second waiting position of the gripper is obtained. The second waiting position is (x2, y2, z2 + beltZ2), where (x2, y2, z2) are the three-dimensional coordinates of the debris (the center point cloud coordinates of the debris point cloud data), and beltZ2 is the second vertical distance, which is less than 0.

[0047] Finally, the gripper is moved from the first waiting position to the second waiting position. During this movement, the gripper is adjusted to the corresponding gripping posture based on the shape of the debris in the debris identification result. This not only avoids touching the debris and coal blocks when adjusting the posture during subsequent debris grabbing, thus preventing the debris position from changing and affecting the gripping effect, but also saves time and reduces changes in the debris position by adjusting the posture during the movement to the second waiting position, rather than adjusting the posture after reaching it, thus improving the success rate of gripping.

[0048] The gripper's grasping posture includes the gripper opening degree and the gripper rotation angle. The gripper opening degree (opening width) is the sum of the maximum width of the debris and the reserved safety gap, or the sum of the minimum width of the debris and the reserved safety gap. The gripper posture is flexibly adjusted according to the shape of the debris to ensure that the debris can be gripped. Specifically, when the sum of the maximum width of the debris and the reserved safety gap is not greater than the gripper opening degree threshold, the gripper opening degree is the sum of the maximum width of the debris and the reserved safety gap, and the gripper gripping position is at the maximum width of the debris; when the sum of the maximum width of the debris and the reserved safety gap is greater than the gripper opening degree threshold, the gripper opening degree is the sum of the minimum width of the debris and the reserved safety gap, and the gripper gripping position is at the minimum width of the debris. The directions of the maximum and minimum widths of the debris are perpendicular to the direction of the maximum length of the debris.

[0049] The gripper rotation angle is the angle between the length direction (maximum length direction of the debris) corresponding to the minimum outer matrix of the debris in the spliced ​​image of the detection area and the forward direction of the vibrating screen.

[0050] S3. After the gripper moves to the second waiting position, it maintains its gripping posture and moves directly downwards to the three-dimensional coordinates of the debris to perform the debris grabbing. This effectively improves the timeliness and accuracy of the gripper's gripping, while shortening the overall operation time and increasing gripping efficiency. After the gripper grabs the debris, it places the debris in the debris basket, then the gripper returns to its initial position, restoring the initial state, and performs the next round of debris detection and grabbing.

[0051] This embodiment provides a method for sorting debris on a vibrating screen. First, several first cameras simultaneously acquire images of different areas at the inlet of the vibrating screen. After deblurring and image stitching, debris is identified to determine its planar position and the first waiting position of the gripper. The gripper is then moved from its initial position to the first waiting position, allowing it to adjust its position in advance and reducing the time spent on subsequent posture and movement adjustments. Next, a second camera on the gripper acquires the three-dimensional coordinates of the debris, reflecting its height, and a second waiting position is planned. Simultaneously, during the movement of the gripper from the first to the second waiting position, its posture is adjusted according to the shape of the debris, allowing it to reach the optimal gripping position more quickly. Finally, after the gripper reaches the second waiting position, it maintains its gripping posture and moves directly downwards to the three-dimensional coordinates of the debris to grasp it. This effectively improves the timeliness and accuracy of gripping, while shortening the overall operation time and increasing gripping efficiency.

Claims

1. A method for sorting debris on a vibrating screen, characterized in that, This includes the following operations: S1. Several first cameras simultaneously acquire images of different areas of the detection area on the inclined screen, resulting in several detection area images; after deblurring, the images of the detection areas are stitched together to obtain a stitched image of the detection area; the detection area is located at the entrance of the vibrating screen. The deblurring process for the current detection region image is as follows: Gaussian filtering and edge enhancement are applied to the current detection region image to obtain the current denoised image; Based on the vibration amplitude and direction of the vibrating screen and the resolution of the first camera, a blur kernel is obtained; based on the blur kernel, the current denoised image is subjected to deconvolution processing based on blur iteration to obtain the initial deblurred image of the current detection area. The initial deblurred image of the current detection area is sharpened to enhance edge details and improve image detail contrast, resulting in the current deblurred image of the current detection area. The formula for calculating the fuzzy kernel is as follows: ， ， The location in the current detection region image is ( x , y The fuzzy kernel value at ) For fuzzy length, , , These are vibration amplitude, vibration direction, and camera resolution, respectively. The image stitching process is as follows: after converting all the deblurred images of the detection regions into grayscale images of the same size, feature point pairs between adjacent grayscale images of the detection regions are obtained; based on each feature point pair, the corresponding homography matrix is ​​obtained as the transformation matrix of the corresponding adjacent deblurred images of the detection regions. Based on the transformation matrix, one of the deblurred images of adjacent detection regions is mapped to another deblurred image of the detection region, and the superimposed pixels of the overlapping regions in the adjacent deblurred images of the detection regions are weighted and averaged to obtain the detection region stitched image. The spliced ​​image of the detection area is processed by target detection to obtain the debris identification result. Based on the debris position in the debris identification result, the first waiting position is obtained. The gripper is moved from the initial position to the first waiting position (x1, y1+deltY, z0+deltZ1), where (x1, y1) is the debris position, z0 is the vertical coordinate of the initial position, deltY and deltZ1 are the first longitudinal distance and the first vertical distance, respectively, and both deltY and deltZ1 are greater than 0. With the initial position of the gripper as the origin, the projection of the coal flow direction on the ground on the vibrating screen is the positive Y-axis, the direction perpendicular to the ground is the positive Z-axis, and the X-axis is established perpendicular to the Z-axis and the Y-axis. S2. Based on the second camera on the gripper, acquire the point cloud data of the debris to obtain the three-dimensional coordinates of the debris; based on the three-dimensional coordinates of the debris, acquire the second waiting position (x2, y2, z2 + beltZ2) of the gripper, (x2, y2, z2) is the three-dimensional coordinates of the debris, beltZ2 is the second vertical distance, and beltZ2 is less than 0. The gripper is moved from the first waiting position to the second waiting position, and during the movement, the gripper is adjusted to the corresponding grasping posture according to the shape of the debris in the debris recognition result. S3. After the gripper moves to the second waiting position, it maintains the gripping posture and moves downward to the three-dimensional coordinates of the debris to grip the debris.

2. The method for sorting debris on a vibrating screen according to claim 1, characterized in that, In S2, the gripping posture of the gripper includes the gripper opening degree and the gripper rotation angle; the gripper rotation angle is the angle between the coal flow direction on the vibrating screen in the splicing diagram of the detection area; the gripper opening degree is the sum of the maximum width of the debris and the reserved safety gap, or the sum of the minimum width of the debris and the reserved safety gap.

3. The method for sorting debris on a vibrating screen according to claim 2, characterized in that, When the sum of the maximum width of the debris and the reserved safety gap is not greater than the gripper opening threshold, the gripper opening is the sum of the maximum width of the debris and the reserved safety gap, and the gripper gripping position is at the maximum width of the debris. When the sum of the maximum width of the debris and the reserved safety gap is greater than the gripper opening threshold, the gripper opening is the sum of the minimum width of the debris and the reserved safety gap, and the gripper gripping position is at the minimum width of the debris.

4. A debris sorting system for a vibrating screen, used to implement the debris sorting method for a vibrating screen as described in claim 1, characterized in that, include: A vibrating screen, with an inclined screen surface, is used to convey coal lumps and debris; A gantry frame spans the screen surface and is positioned in the forward direction of the vibrating screen. Several first vision units are installed on the upper part of the gantry frame, with the shooting direction of the first vision units facing the screen surface of the vibrating screen. A robot is installed on the upper part of the gantry frame. The first visual unit is used to acquire images of the detection area and transmit them to the computing unit; The second vision unit is used to acquire point cloud data of clutter and transmit it to the computing unit; The computing unit is used to perform deblurring, image stitching, and target detection on the image transmitted from the first vision unit to obtain the clutter recognition result. Based on the clutter position in the clutter recognition result, the first waiting position is obtained, and the result is converted into a control signal and transmitted to the control unit. And to obtain the three-dimensional coordinates of the clutter based on the point cloud data of the clutter transmitted from the second vision unit; based on the three-dimensional coordinates of the clutter, to obtain the second waiting position of the gripper, and to transmit it to the control unit after converting it into a control signal; The control unit is used to control the movement of the gripper and the gripping of debris based on the control signals transmitted from the computing unit; The robot has grippers attached to its end for picking up objects.

5. A debris sorting system on a vibrating screen according to claim 4, characterized in that, It also includes a miscellaneous basket, placed on one side of the vibrating screen outlet, for holding miscellaneous items.

6. A debris sorting system on a vibrating screen according to claim 4, characterized in that, It also includes a lighting unit, which is installed on the upper part of the gantry frame and the direction of illumination is directly facing the screen surface of the vibrating screen.

Images