Sorting method and system for negative pressure suction cups of mechanical arm

Abstract

The invention discloses a sorting method and system for a negative pressure suction cup of a manipulator, belongs to the technical field of garbage recycling, and aims at solving the technical problems that the success rate of existing plastic garbage sorting is low, effective recognition is difficult to carry out, the efficiency is low, the labor cost is high, and intelligent plastic garbage sorting is not facilitated. The method comprises the following steps: collecting a garbage image with continuous frames on a transmission belt; performing target calibration processing on all garbage in the garbage image, and determining a calibration coordinate of each garbage; carrying out coordinate nonlinear correction on the selected calibration garbage in the calibration coordinates to obtain an actual coordinate corresponding to each garbage; the garbage under each actual coordinate is subjected to characteristic recognition and calibration treatment of recyclable plastic garbage, and sortable coordinates are determined; and through a mechanical arm negative pressure suction cup in the mechanical arm negative pressure suction cup equipment, grabbing control of an air pump is conducted on the recyclable plastic garbage under the sortable coordinates, and a garbage sorting result is generated.

Description

Technical Field

[0001] The present application relates to the field of waste recycling, and in particular to a sorting method and system for a robot negative pressure suction cup. Background Art

[0002] At present, industrial robots have been widely used in product production lines and other fields [1]. For example, there are a large number of industrial robots in the fields of automobile manufacturing, precision machining, and logistics. At present, the working scope of industrial robots has been continuously expanded with the development of artificial intelligence technology. Highly intelligent industrial robots to complete industrial production have gradually become the mainstream. Among them, the development of computer vision technology has played a very obvious role in promoting the development of industrial robots.

[0003] With the development of society, the utilization rate of plastic products is getting higher and higher. Therefore, achieving high added value recycling of waste plastic waste has become an important task for plastic products at present. For the recycling of waste plastic products, the key lies in efficient separation. Traditional recycling methods, such as manual sorting, wind sorting and near-infrared spectroscopy sorting, often cannot take into account low cost, high efficiency, high recognition rate and low pollution. Therefore, it is urgent to build a garbage sorting system based on artificial intelligence technology, which can not only help enterprises to achieve rapid separation of waste plastic waste, but also promote its effective recycling and reuse. However, the existing plastic waste sorting has a low success rate, slow efficiency, and is prone to high costs. At the same time, plastic materials are of various types, colors and shapes, which are difficult to be effectively identified by traditional sorting methods. Second, plastic waste is often mixed with a large number of other impurities, such as metal, paper, glass, etc. The presence of these impurities seriously affects the sorting effect of plastic waste. Third, the traditional sorting method is inefficient and costly when dealing with large amounts of plastic waste, and it is easy to pollute the environment. Summary of the invention

[0004] The embodiments of the present application provide a sorting method and system of a robot negative pressure suction cup, which are used to solve the following technical problems: the existing plastic waste sorting has a low success rate, is difficult to effectively identify, is inefficient, and has high labor costs, which is not conducive to intelligent plastic waste sorting.

[0005] The present application embodiment adopts the following technical solutions:

[0006] On the one hand, an embodiment of the present application provides a sorting method using a manipulator negative pressure suction cup, comprising: based on an industrial camera pre-installed in a manipulator negative pressure suction cup device, collecting garbage images with continuous frames located on a transmission belt; performing target calibration processing on all garbage in the garbage image to determine the calibration coordinates of each piece of garbage; performing nonlinear correction on the coordinates of the calibrated garbage selected in the calibration coordinates to obtain the actual coordinates corresponding to each piece of garbage; through a preset plastic garbage recognition model, performing feature recognition and calibration processing on the recyclable plastic garbage at each of the actual coordinates to determine the sortable coordinates; through the manipulator negative pressure suction cup in the manipulator negative pressure suction cup device, the air pump is controlled to grab the recyclable plastic garbage at the sortable coordinates to generate a garbage sorting result.

[0007] The embodiment of the present application can realize fast and continuous sorting of garbage by collecting images through continuous frames of industrial cameras, thereby improving sorting efficiency. The target calibration process can accurately determine the coordinate position of each garbage, thereby improving the accuracy of sorting. In addition, the nonlinear correction of the coordinates can more accurately reflect the actual position of the garbage, thereby reducing sorting errors caused by image processing errors. Using the preset plastic garbage recognition model, recyclable plastic garbage can be automatically identified and calibrated, reducing manual intervention and improving the degree of automation. The air pump grasping control of the manipulator negative pressure suction cup can also be used to accurately control the grasping of recyclable plastic garbage, thereby reducing damage and misoperation. At the same time, the automation and precise control of the entire process help to improve the accuracy and quality of garbage sorting, thereby optimizing the sorting results. Operators can reduce physical labor and work intensity. It can also reduce misclassification and omission, thereby improving the overall sorting quality. It can be applied to garbage sorting of different types and sizes, and has strong adaptability and scalability.

[0008] In a feasible implementation, based on an industrial camera pre-installed in a manipulator negative pressure suction cup device, garbage images with continuous frames on a transmission belt are collected, specifically comprising: installing the industrial camera on a retaining frame of the manipulator negative pressure suction cup device; wherein the framing area of ​​the industrial camera includes a plane area where the transmission belt is located; based on the transmission speed of the transmission belt, periodically adjusting the shooting interval of the industrial camera to determine a shooting interval time; wherein the shooting interval time is used to collect all continuous frame images on the transmission belt; according to the shooting interval time and through the industrial camera, performing image acquisition processing on a number of garbage on the transmission belt to obtain an initial image; performing stitching processing on adjacent images of incomplete garbage pixels in the edge area of ​​the continuous initial image to generate the garbage image; wherein the garbage images all contain complete single garbage pixel areas.

[0009] In a feasible implementation manner, before subjecting all garbage in the garbage image to target calibration processing and determining the calibration coordinates of each garbage, the method further includes: performing multi-angle image acquisition processing on sample garbage in a historical garbage type set under preset reference conditions to obtain a historical garbage morphological image; wherein the reference conditions include at least: lighting conditions, angle conditions, distance conditions, occlusion conditions, and garbage deformation conditions; extracting corner point features, edge features, and grayscale change features in the historical garbage morphological image; and generating target anchor point information for garbage calibration based on the corner point features, the edge features, and the grayscale change features.

[0010] In a feasible implementation manner, all garbage in the garbage image is subjected to target calibration processing to determine the calibration coordinates of each garbage, specifically including: taking the plane where the transmission belt in the garbage image is located as the horizontal plane, and taking the vertical line where the manipulator negative pressure suction cup is located as the normal vector axis of the horizontal plane; constructing a sorting three-dimensional space coordinate system based on the horizontal plane and the normal vector axis; performing linear regression fitting on the internal parameters in the industrial camera through a preset least squares method to obtain the camera internal parameters based on the industrial camera; and calculating the external parameters of the shooting performance of the industrial camera based on the translation vector of the transmission belt to obtain the camera external parameters; according to the camera internal parameters and the camera external parameters of the industrial camera, and through the sorting three-dimensional space coordinate system, all garbage pixel features in the garbage image are identified and processed under the target anchor point information, and the target contour of each garbage and the calibration coordinates between the industrial camera are determined; wherein, the calibration coordinates are relative coordinates based on the principal point coordinates; the principal point coordinates are determined based on the camera internal parameters and the camera external parameters.

[0011] In a feasible implementation manner, nonlinear correction of coordinates is performed on the calibrated garbage selected in the calibrated coordinates to obtain actual coordinates corresponding to each piece of garbage, specifically including: performing an expansion correction process on the pixel area of ​​the calibrated garbage regarding radial distortion through a preset Taylor series to obtain radial correction parameters; performing correction process on the pixel area of ​​the calibrated garbage regarding tangential distortion coefficients according to the nonlinear transformation of the industrial camera during imaging to determine the tangential correction coefficients; performing nonlinear correction process on the calibrated coordinates of each piece of garbage according to a sorting three-dimensional space coordinate system and through the radial correction parameters and the tangential correction coefficients to obtain actual coordinates corresponding to each piece of garbage; wherein the actual coordinates are the pixel coordinates of the garbage image.

[0012] In a feasible implementation manner, after performing nonlinear correction on the coordinates of the calibrated garbage selected in the calibrated coordinates to obtain the actual coordinates corresponding to each piece of garbage, the method further includes: performing coordinate mapping processing on the conveyor belt plane for the actual coordinates of each piece of garbage in the garbage image according to the principle of similarity and the mapping relationship between the image and the real world to obtain static mapping coordinates; wherein the static mapping coordinates are the static coordinates of each piece of garbage at the current moment when the garbage image is collected; based on the transportation speed of the conveyor belt, dynamically calculating the translation vector of the static mapping coordinates to obtain dynamic mapping coordinates; wherein the dynamic mapping coordinates are dynamic coordinates transformed over time; based on the static mapping coordinates and the dynamic mapping coordinates of each piece of garbage on the conveyor belt, a set of actual coordinates of each piece of garbage is obtained.

[0013] In a feasible implementation, before a preset plastic waste identification model is used to perform feature identification and calibration processing of recyclable plastic waste on the garbage at each actual coordinate to determine the sortable coordinates, the method also includes: encoding the classification features of historical recyclable plastic waste images into a data set to obtain a classification feature data set; wherein the classification features include at least: color features, material features, texture features and morphological features of recyclable plastic waste; through a preset target positioning algorithm, the classification feature data set is trained on target recognition of recyclable plastic features and recyclable plastic sorting requirements, and the accuracy, recall rate, average precision and average precision mean of the network topology structure in the target positioning algorithm are batch stochastic gradient iteratively calculated to construct the plastic waste identification model; wherein the input end of the plastic waste identification model is all garbage images in the garbage image, and the output end is the recyclable plastic garbage image in the garbage image.

[0014] In a feasible implementation, a preset plastic waste identification model is used to perform feature identification and calibration processing of recyclable plastic waste on the garbage under each of the actual coordinates to determine the sortable coordinates, specifically including: inputting the current garbage image into the plastic waste identification model to identify and mark the recyclable plastic waste image; performing image matching processing on the garbage image corresponding to the actual coordinate set of each garbage through the recyclable plastic waste image to determine the sortable coordinates of the recyclable plastic waste image; obtaining the actual coordinate set corresponding to the sortable coordinates, and defining the actual coordinate set to the sortable coordinates.

[0015] In a feasible implementation, the recyclable plastic waste under the sortable coordinates is controlled by the manipulator negative pressure suction cup in the manipulator negative pressure suction cup device to grab and control the air pump to generate a garbage sorting result, specifically including: based on the sortable coordinates, generating a grabbing movement trajectory and an air pump start time for controlling the manipulator negative pressure suction cup; grabbing and controlling the recyclable plastic waste under the sortable coordinates through the grabbing movement trajectory and the air pump start time; and based on the recycling movement trajectory, recycling the grabbed recyclable plastic waste to generate the garbage sorting result; wherein, the garbage sorting result includes: a successful recycling result and a failed recycling result.

[0016] On the other hand, the embodiment of the present application also provides a sorting system for a manipulator negative pressure suction cup, comprising: an image acquisition module, which is used to collect garbage images with continuous frames located on a transmission belt based on an industrial camera pre-installed in the manipulator negative pressure suction cup device; perform target calibration processing on all garbage in the garbage image to determine the calibration coordinates of each garbage; perform nonlinear correction on the coordinates of the calibrated garbage selected in the calibration coordinates to obtain the actual coordinates corresponding to each garbage; a target calibration module, which is used to perform feature recognition and calibration processing of recyclable plastic garbage under each actual coordinate through a preset plastic garbage recognition model to determine the sortable coordinates; a manipulator sorting module, which is used to control the air pump to grab the recyclable plastic garbage under the sortable coordinates through the manipulator negative pressure suction cup in the manipulator negative pressure suction cup device to generate a garbage sorting result; wherein the manipulator negative pressure suction cup device comprises: a manipulator, a negative pressure suction cup, a retaining frame, a baffle and a central control system.

[0017] The present application provides a method and system for sorting a robot negative pressure suction cup. Compared with the prior art, the embodiments of the present application have the following beneficial technical effects:

[0018] 1. Improve sorting efficiency: By collecting images through continuous frames of industrial cameras, garbage can be sorted quickly and continuously, improving sorting efficiency.

[0019] 2. Accurate target calibration: Through target calibration processing, the coordinate position of each garbage can be accurately determined, which improves the accuracy of sorting.

[0020] 3. Non-linear correction: Non-linear correction of coordinates can more accurately reflect the actual location of the garbage and reduce sorting errors caused by image processing errors.

[0021] 4. Intelligent identification and calibration: Using the preset plastic waste identification model, recyclable plastic waste can be automatically identified and calibrated, reducing manual intervention and improving the degree of automation.

[0022] 5. Precisely control the grabbing: Through the air pump grabbing control of the manipulator's negative pressure suction cup, the grabbing of recyclable plastic waste can be precisely controlled, reducing damage and misoperation.

[0023] 6. Optimize garbage sorting results: Automation and precise control of the entire process help improve the accuracy and quality of garbage sorting, thereby optimizing the sorting results.

[0024] 7. Reduce labor intensity: Due to the improvement of automation, operators can reduce physical labor and reduce work intensity.

[0025] 8. Improve sorting quality: Through accurate identification and grasping, it can reduce misclassification and omissions and improve the overall sorting quality.

[0026] 9. Strong adaptability: This method can be applied to garbage sorting of different types and sizes, and has strong adaptability and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0028] Figure 1 A flow chart of a method for sorting a manipulator negative pressure suction cup provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the front view of a manipulator negative pressure suction cup device provided in an embodiment of the present application;

[0030] Figure 3 A schematic side view of the structure of a manipulator negative pressure suction cup device provided in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of a top view of a manipulator negative pressure suction cup device provided in an embodiment of the present application;

[0032] Figure 5 A schematic structural diagram of a robotic negative pressure suction cup sorting system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0034] The present application embodiment provides a method for sorting a manipulator negative pressure suction cup, such as Figure 1 As shown, the method for sorting the manipulator negative pressure suction cup specifically includes steps S101-S105:

[0035] S101, based on the industrial camera pre-installed in the robot negative pressure suction cup device, collect garbage images with continuous frames on the transmission belt.

[0036] Specifically, the industrial camera needs to be installed on the holder of the manipulator negative pressure suction cup device first, wherein the viewing area of ​​the industrial camera includes the plane area where the transmission belt is located.

[0037] Furthermore, based on the transmission speed of the transmission belt, the shooting interval of the industrial camera is periodically adjusted to determine the shooting interval time, wherein the shooting interval time is used to collect all continuous frame images on the transmission belt.

[0038] Furthermore, according to the shooting interval, an industrial camera is used to collect and process images of some garbage on the transmission belt to obtain an initial image.

[0039] Furthermore, the incomplete junk pixels in the edge areas of the continuous initial images are processed by splicing adjacent images to generate junk images, wherein the junk images all contain complete single junk pixel areas.

[0040] In one embodiment, first, an industrial camera is mounted on a holder of a manipulator negative pressure suction cup device to ensure that the camera's framing area covers the plane area where the transmission belt is located. According to the transmission speed of the transmission belt, the shooting interval of the industrial camera is adjusted to determine a suitable shooting interval time. This time is used to collect all continuous frame images on the transmission belt. Using the adjusted shooting interval, the garbage on the transmission belt is imaged by the industrial camera to obtain a series of initial images. The continuous initial images are processed, and the incomplete garbage pixels in the edge area are spliced ​​with the adjacent images to generate a garbage image containing a complete single garbage pixel area. A more complete and higher quality garbage image can be obtained, which is conducive to subsequent image processing and analysis. By ensuring the integrity of the garbage image, the erroneous recognition and processing caused by incomplete images can be reduced. Since the image is clearer and more complete, the sorting system can more accurately identify and locate the garbage, thereby improving the sorting accuracy. The method can adjust the shooting interval according to the actual transmission speed of the transmission belt, improve the adaptability of the system, and can be applied to transmission belts of different speeds. By predetermining the shooting interval, the amount of calculation in subsequent image processing can be reduced and the processing speed can be improved.

[0041] S102: Perform target calibration processing on all garbage in the garbage image to determine the calibration coordinates of each garbage.

[0042] Specifically, by using preset reference conditions, the sample garbage in the historical garbage type set is imaged from multiple angles to obtain a historical garbage morphological image, wherein the reference conditions at least include: illumination conditions, angle conditions, distance conditions, occlusion conditions, and garbage deformation conditions.

[0043] Furthermore, the corner features, edge features and grayscale change features in the historical garbage morphology images are extracted, and target anchor point information for garbage calibration is generated based on the corner features, edge features and grayscale change features.

[0044] Furthermore, the plane where the conveyor belt in the garbage image is located is taken as the horizontal plane, and the vertical line where the manipulator negative pressure suction cup is located is taken as the normal vector axis of the horizontal plane. Based on the horizontal plane and the normal vector axis, a three-dimensional space coordinate system for sorting is constructed.

[0045] Furthermore, the internal parameters of the industrial camera are linearly regressed and fitted by the preset least square method to obtain the internal parameters of the camera based on the industrial camera. And based on the translation vector of the transmission belt, the external parameters of the shooting performance of the industrial camera are calculated to obtain the external parameters of the camera.

[0046] Furthermore, it is also necessary to identify and process all garbage pixel features in the garbage image under the target anchor information according to the internal and external parameters of the industrial camera and by sorting the three-dimensional space coordinate system, and determine the target outline of each garbage and the calibration coordinates between it and the industrial camera. Among them, the calibration coordinates are relative coordinates based on the principal point coordinates; the principal point coordinates are determined based on the internal and external parameters of the camera.

[0047] In one embodiment, using preset reference conditions such as illumination, angle, distance, occlusion and garbage deformation, the sample garbage in the historical garbage type set is subjected to multi-angle image acquisition and processing to obtain a comprehensive historical garbage morphological image. Corner features, edge features and grayscale change features are extracted from the historical garbage morphological image, which are helpful for subsequent garbage identification and calibration. Based on the extracted features, target anchor point information for garbage calibration is generated, which is essential for identifying and locating garbage. The plane where the transmission belt is located is set as a horizontal plane, and the vertical line of the manipulator negative pressure suction cup is used as the normal vector axis of the horizontal plane to construct a three-dimensional space coordinate system for sorting. The internal parameters of the industrial camera are linearly regressed and fitted using the least squares method to obtain the internal parameters of the camera. At the same time, the external parameters of the camera's shooting performance are calculated based on the translation vector of the transmission belt. Using the camera's internal parameters, external parameters and the sorting three-dimensional space coordinate system, all garbage pixel features in the garbage image are identified and processed to determine the target contour of each garbage and its calibration coordinates between the industrial camera and the industrial camera.

[0048] S103: Perform nonlinear correction on the coordinates of the calibrated garbage selected in the calibrated coordinates to obtain the actual coordinates corresponding to each garbage.

[0049] Specifically, the radial distortion of the pixel area of ​​the calibrated garbage is firstly expanded and corrected using a preset Taylor series to obtain radial correction parameters.

[0050] Furthermore, according to the nonlinear transformation of the industrial camera during imaging, the pixel area of ​​the calibration garbage is corrected for the tangential distortion coefficient to determine the tangential correction coefficient.

[0051] Furthermore, according to the sorting three-dimensional space coordinate system, the calibrated coordinates of each garbage are subjected to nonlinear correction processing through radial correction parameters and tangential correction coefficients to obtain the actual coordinates corresponding to each garbage, wherein the actual coordinates are the pixel coordinates of the garbage image.

[0052] As a feasible implementation method, according to the principle of similarity and the mapping relationship between the image and the real world, the actual coordinates of each piece of garbage in the garbage image are processed by coordinate mapping on the plane of the conveyor belt to obtain static mapping coordinates. The static mapping coordinates are the static coordinates of each piece of garbage at the current moment when the garbage image is collected. Based on the transportation speed of the conveyor belt, the translation vector of the static mapping coordinates is dynamically calculated to obtain dynamic mapping coordinates. The dynamic mapping coordinates are dynamic coordinates that change over time. Based on the static mapping coordinates and dynamic mapping coordinates of each piece of garbage on the conveyor belt, the actual coordinate set of each piece of garbage is obtained.

[0053] In one embodiment, the radial distortion correction processing is performed on the pixel area of ​​the calibrated garbage using Taylor series expansion to obtain radial correction parameters. Radial distortion is usually caused by geometric defects of the lens, such as spherical distortion. According to the nonlinear characteristics of industrial camera imaging, the pixel area of ​​the calibrated garbage is corrected for tangential distortion and the tangential correction coefficient is determined. Tangential distortion is usually manifested as the distortion of a straight line at the edge. Using the sorting three-dimensional space coordinate system, combined with the radial correction parameters and the tangential correction coefficient, the calibrated coordinates of each garbage are subjected to nonlinear correction processing. This step aims to obtain the actual coordinates of each garbage, that is, the pixel coordinates of the garbage image. Based on the principle of similarity relationship and the mapping relationship between the image and the real world, the actual coordinates of each garbage in the garbage image are mapped to the conveyor belt plane. First, the static mapping coordinates are obtained, which are the coordinates of each garbage at the current moment when the garbage image is collected. Considering the transportation speed of the conveyor belt, the translation vector is calculated, and the static mapping coordinates are converted into dynamic mapping coordinates, which are coordinates that change over time. By combining the static mapping coordinates and the dynamic mapping coordinates, the actual coordinate set of each garbage is obtained. These coordinates reflect the actual position and dynamic changes of the garbage during the sorting process.

[0054] S104: Using a preset plastic waste identification model, the features of recyclable plastic waste are identified and calibrated for the waste at each actual coordinate, and the sortable coordinates are determined.

[0055] Specifically, it is necessary to first encode the classification features of the historical recyclable plastic waste images into a data set to obtain a classification feature data set, wherein the classification features at least include: color features, material features, texture features, and morphological features of the recyclable plastic waste.

[0056] Furthermore, through the preset target positioning algorithm, the classification feature data set is trained for target recognition of recyclable plastic features and recyclable plastic sorting requirements, and the accuracy, recall rate, average precision and average precision mean of the network topology structure in the target positioning algorithm are calculated by batch random gradient iteration to construct a plastic waste recognition model. Among them, the input end of the plastic waste recognition model is all the garbage images in the garbage image, and the output end is the recyclable plastic garbage images in the garbage image.

[0057] Furthermore, the current garbage image is input into the plastic garbage recognition model to identify and mark the recyclable plastic garbage image. Using the marked recyclable plastic garbage image, the garbage image corresponding to the actual coordinate set of each garbage is image matched to determine the sortable coordinates of the recyclable plastic garbage image.

[0058] Furthermore, it is also necessary to obtain the actual coordinate set corresponding to the sortable coordinates and define the actual coordinate set into the sortable coordinates.

[0059] In one embodiment, the classification features of historical recyclable plastic waste images are extracted and encoded to obtain a classification feature data set containing color, material, texture and morphological features. The classification feature data set is trained using a preset target positioning algorithm to identify recyclable plastic waste and its sorting requirements. During the training process, the accuracy, recall, average precision and average precision mean of the network topology are calculated through batch random gradient iteration to optimize the model performance. A plastic waste recognition model is constructed, the input of which is a garbage image and the output is an identified recyclable plastic waste image. The current garbage image is input into the recognition model, and the model identifies and marks the recyclable plastic waste image therein. Using the identified recyclable plastic waste image, the garbage image corresponding to the actual coordinate set is image matching processed to determine the sortable coordinates of the recyclable plastic waste image.

[0060] S105, using the manipulator negative pressure suction cup in the manipulator negative pressure suction cup device, the air pump is controlled to grab the recyclable plastic garbage under the sortable coordinates to generate the garbage sorting result.

[0061] Specifically, firstly, based on the sortable coordinates, a grabbing movement trajectory and an air pump start time for controlling the negative pressure suction cup of the robot are generated.

[0062] Furthermore, the recyclable plastic waste under the sortable coordinates is grabbed and controlled by the grabbing movement trajectory and the air pump start time. Based on the recycling movement trajectory, the grabbed recyclable plastic waste is recycled and processed to generate the waste sorting results. The waste sorting results include: recycling success results and recycling failure results.

[0063] in, Figure 2 This is a schematic diagram of the front view of a manipulator negative pressure suction cup device provided in an embodiment of the present application. Figure 3 A schematic diagram of the side structure of a manipulator negative pressure suction cup device provided in an embodiment of the present application, Figure 4 A schematic diagram of a top view of a manipulator negative pressure suction cup device provided in an embodiment of the present application is shown in FIG. Figure 2 , 3 As shown in , 4, the manipulator negative pressure suction cup device includes: a manipulator 201, a negative pressure suction cup 202, a holding frame 203, a baffle 204 and a central control system 205. The recyclable plastic waste can be recycled by utilizing the mechanical movement of the manipulator negative pressure suction cup device.

[0064] In addition, the present application also provides a sorting system for a manipulator negative pressure suction cup, such as Figure 5 As shown, the system 500 specifically includes:

[0065] The image acquisition module 510 is used to collect garbage images with continuous frames on the transmission belt based on the industrial camera pre-installed in the robot negative pressure suction cup device; perform target calibration processing on all garbage in the garbage image to determine the calibration coordinates of each garbage; perform nonlinear correction on the coordinates of the calibrated garbage selected in the calibration coordinates to obtain the actual coordinates corresponding to each garbage.

[0066] The target calibration module 520 is used to perform feature recognition and calibration processing of recyclable plastic waste at each actual coordinate through a preset plastic waste recognition model to determine the sortable coordinates.

[0067] The robot sorting module 530 is used to control the air pump to grab the recyclable plastic garbage under the sortable coordinates through the robot negative pressure suction cup in the robot negative pressure suction cup device to generate garbage sorting results.

[0068] This application can realize fast and continuous sorting of garbage by collecting images through continuous frames of industrial cameras, thereby improving sorting efficiency. Target calibration processing can accurately determine the coordinate position of each garbage, thereby improving the accuracy of sorting. And nonlinear correction of coordinates can more accurately reflect the actual position of garbage, reducing sorting errors caused by image processing errors. Using the preset plastic garbage recognition model, recyclable plastic garbage can be automatically identified and calibrated, reducing manual intervention and improving the degree of automation. The air pump grasping control of the manipulator negative pressure suction cup can also be used to accurately control the grasping of recyclable plastic garbage, reducing damage and misoperation. At the same time, the automation and precise control of the entire process help to improve the accuracy and quality of garbage sorting, thereby optimizing the sorting results. Operators can reduce physical labor and work intensity. It can also reduce misclassification and omissions, and improve the overall sorting quality. It can be applied to garbage sorting of different types and sizes, with strong adaptability and scalability.

[0069] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0070] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0071] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for sorting a robot negative pressure suction cup, characterized in that: The method comprises: Based on the industrial camera pre-installed in the manipulator negative pressure suction cup device, the garbage image with continuous frames on the transmission belt is collected; Performing target calibration processing on all garbage in the garbage image to determine the calibration coordinates of each garbage; Performing nonlinear correction on the coordinates of the calibrated garbage selected in the calibrated coordinates to obtain the actual coordinates corresponding to each garbage; Through the preset plastic waste identification model, the features of recyclable plastic waste are identified and calibrated for each of the waste at the actual coordinates to determine the sortable coordinates; The recyclable plastic waste under the sortable coordinates is grasped and controlled by an air pump through the manipulator negative pressure suction cup in the manipulator negative pressure suction cup device to generate a waste sorting result.

2. A method for sorting a robot negative pressure suction cup according to claim 1, characterized in that: Based on the industrial camera pre-installed in the robot negative pressure suction cup device, the garbage images with continuous frames on the conveyor belt are collected, including: The industrial camera is mounted on the holder of the manipulator negative pressure suction cup device; wherein the viewing area of ​​the industrial camera includes the plane area where the transmission belt is located; Based on the transmission speed of the transmission belt, the shooting interval of the industrial camera is periodically adjusted to determine the shooting interval time; wherein the shooting interval time is used to completely collect all continuous frame images on the transmission belt; According to the shooting interval, the industrial camera is used to collect and process the images of the garbage on the transmission belt to obtain an initial image; The incomplete junk pixels in the edge area of ​​the continuous initial image are processed by splicing adjacent images to generate the junk image; wherein the junk image contains a complete single junk pixel area.

3. A method for sorting a robot negative pressure suction cup according to claim 1, characterized in that: Before performing target calibration processing on all garbage in the garbage image to determine the calibration coordinates of each garbage, the method further includes: Through preset reference conditions, sample garbage in the historical garbage type set is subjected to multi-angle image acquisition and processing to obtain a historical garbage morphological image; wherein the reference conditions at least include: illumination conditions, angle conditions, distance conditions, occlusion conditions, and garbage deformation conditions; Extracting corner features, edge features and grayscale change features from the historical garbage morphology image; Based on the corner point features, the edge features and the grayscale change features, target anchor point information for garbage calibration is generated.

4. A method for sorting a robot negative pressure suction cup according to claim 3, characterized in that: All garbage in the garbage image is subjected to target calibration processing to determine the calibration coordinates of each garbage, specifically including: The plane where the transmission belt in the garbage image is located is taken as the horizontal plane, and the vertical line where the negative pressure suction cup of the robot is located is taken as the normal vector axis of the horizontal plane; Based on the horizontal plane and the normal vector axis, construct a sorting three-dimensional space coordinate system; By using a preset least squares method, linear regression fitting is performed on the internal parameters of the industrial camera to obtain the internal parameters of the camera based on the industrial camera; and based on the translation vector of the transmission belt, external parameters of the shooting performance of the industrial camera are calculated to obtain the external parameters of the camera; According to the camera internal parameters and the camera external parameters of the industrial camera and through the sorting three-dimensional space coordinate system, all garbage pixel features in the garbage image are identified and processed under the target anchor point information to determine the target outline of each garbage and the calibration coordinates between the industrial camera; wherein the calibration coordinates are relative coordinates based on the principal point coordinates; the principal point coordinates are determined based on the camera internal parameters and the camera external parameters.

5. A method for sorting a robot negative pressure suction cup according to claim 1, characterized in that: The nonlinear correction of the coordinates of the calibrated garbage selected in the calibrated coordinates is performed to obtain the actual coordinates corresponding to each garbage, specifically including: By using a preset Taylor series, a radial distortion expansion correction process is performed on the pixel area of ​​the calibrated garbage to obtain a radial correction parameter; According to the nonlinear transformation of the industrial camera during imaging, the pixel area of ​​the calibration garbage is corrected for the tangential distortion coefficient to determine the tangential correction coefficient; According to the sorting three-dimensional space coordinate system, and through the radial correction parameter and the tangential correction coefficient, the calibrated coordinates of each garbage are subjected to nonlinear correction processing to obtain the actual coordinates corresponding to each garbage; wherein the actual coordinates are the pixel coordinates of the garbage image.

6. A method for sorting a robot negative pressure suction cup according to claim 1, characterized in that: After performing nonlinear correction on the coordinates of the calibrated garbage selected in the calibrated coordinates to obtain the actual coordinates corresponding to each garbage, the method further includes: According to the principle of similarity and the mapping relationship between the image and the real world, the actual coordinates of each piece of garbage in the garbage image are processed by coordinate mapping on the plane of the conveyor belt to obtain static mapping coordinates; wherein the static mapping coordinates are the static coordinates of each piece of garbage at the current moment when the garbage image is collected; Based on the transport speed of the transmission belt, a translation vector of the static mapping coordinate is dynamically calculated to obtain a dynamic mapping coordinate; wherein the dynamic mapping coordinate is a dynamic coordinate transformed over time; Based on the static mapping coordinates of each garbage on the conveyor belt and the dynamic mapping coordinates, an actual coordinate set of each garbage is obtained.

7. A method for sorting a robot negative pressure suction cup according to claim 1, characterized in that: Before the garbage at each actual coordinate is subjected to feature recognition and calibration processing of recyclable plastic garbage by using a preset plastic garbage recognition model and determining the sortable coordinates, the method further includes: Performing data set encoding processing on the classification features of historical recyclable plastic waste images to obtain a classification feature data set; wherein the classification features at least include: color features, material features, texture features, and morphological features of the recyclable plastic waste; Through a preset target positioning algorithm, target recognition training is performed on the classification feature data set regarding the features of recyclable plastics and the requirements for sorting recyclable plastics, and batch random gradient iterative calculations are performed on the accuracy, recall rate, average precision and average precision mean of the network topology structure in the target positioning algorithm to construct the plastic waste recognition model; Among them, the input end of the plastic waste recognition model is all garbage images in the garbage image, and the output end is the recyclable plastic garbage images in the garbage image.

8. A method for sorting a robot negative pressure suction cup according to claim 1, characterized in that: Through the preset plastic waste identification model, the features of recyclable plastic waste are identified and calibrated for each of the wastes at the actual coordinates to determine the sortable coordinates, specifically including: Inputting the current garbage image into the plastic garbage recognition model to identify and mark the recyclable plastic garbage image; By using the recyclable plastic waste image, image matching processing is performed on the garbage image corresponding to the actual coordinate set of each garbage to determine the sortable coordinates of the recyclable plastic waste image; A set of actual coordinates corresponding to the sortable coordinates is acquired, and the set of actual coordinates is defined into the sortable coordinates.

9. A method for sorting a robot negative pressure suction cup according to claim 1, characterized in that: The recyclable plastic waste under the sortable coordinates is grasped and controlled by the manipulator negative pressure suction cup in the manipulator negative pressure suction cup device to generate the waste sorting results, which specifically include: Based on the sortable coordinates, generating a grabbing movement trajectory and an air pump start time for controlling the negative pressure suction cup of the manipulator; The recyclable plastic waste under the sortable coordinates is grasped and controlled by the grasping movement trajectory and the air pump start-up time; and based on the recycling movement trajectory, the grasped recyclable plastic waste is recycled to generate the waste sorting result; wherein, the waste sorting result includes: a successful recycling result and a failed recycling result.

10. A sorting system for a robot negative pressure suction cup, characterized in that: The system comprises: The image acquisition module is used to collect garbage images with continuous frames on the transmission belt based on the industrial camera pre-installed in the manipulator negative pressure suction cup device; perform target calibration processing on all garbage in the garbage image to determine the calibration coordinates of each garbage; perform nonlinear correction on the coordinates of the calibrated garbage selected in the calibration coordinates to obtain the actual coordinates corresponding to each garbage; The target calibration module is used to identify and calibrate the characteristics of recyclable plastic waste at each actual coordinate through a preset plastic waste identification model to determine the sortable coordinates; A manipulator sorting module is used to control the air pump to grab the recyclable plastic waste under the sortable coordinates through the manipulator negative pressure suction cup in the manipulator negative pressure suction cup device to generate a waste sorting result; Among them, the manipulator negative pressure suction cup equipment includes: a manipulator, a negative pressure suction cup, a retaining frame, a baffle and a central control system.

Images