A method and system for sorting recyclable garbage resources

By performing feature map processing and attention calculation on waste images on the garbage conveying device, recyclable resources are identified and manipulated to capture them, the problems of low efficiency and poor accuracy of traditional manual sorting are solved, and efficient and accurate capabilities are achieved Recycling resource identification and sorting.

CN118681825BActive Publication Date: 2024-11-15CHINA ROC FUTURE CO
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411163500.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-11-15
Estimated Expiration
2044-08-23

AI Technical Summary

Technical Problem

Traditional waste sorting methods rely on manual selection, which is inefficient and inaccurate, especially the difficulty in handling high-value recyclable resources such as iron, copper, glass, etc., and machine vision technology is in light changes and stain coverage. It is difficult to identify when interfering with impurities.

Method used

By taking the waste image on the garbage conveying device, downsampling and feature map normalization, obtaining reference points based on the outline diagram, calculating the offset vector and weight of the attention head, deleting abnormal offsets, and performing deformable multi-head attention Force calculation, input the decoder to identify recyclable resources, and drive the robot to grab the resources.

Benefits of technology

It improves the identification accuracy and sorting efficiency of recyclable resources, reduces manual intervention, and can accurately identify metal materials in waste of different sizes and shapes, improving the sorting accuracy and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118681825B_ABST
    Figure CN118681825B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of resource recycling, and specifically to a method and system for sorting recyclable garbage resources. The method comprises the following steps: downsampling a preprocessed waste image to obtain feature maps of multiple sizes, obtaining a reference point based on a contour map of the feature map; calculating an offset vector and a weight vector of each attention head of the reference point in the same feature map, forming a set with the offset of each sampling point and the weight corresponding to the offset, searching for an abnormal set of the set, deleting the attention head corresponding to the offset vector containing the offset in the abnormal set when calculating the deformable multi-head attention of each feature map of the sampling point, and obtaining a deformable attention of each reference point based on the deformable multi-head attention of each feature map; inputting the deformable attention into a decoder to obtain a recyclable resource recognition result, and driving a manipulator to grab the recyclable resource according to the position of the recyclable resource recognition result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of resource recovery, and in particular to a method and system for sorting recyclable garbage resources. Background Art

[0002] The main components of stale garbage in landfills are humus, plastic, masonry, rubber, textiles, glass, metal, etc. The calorific value is low and the composition is complex. Due to the low calorific value, incineration cannot be carried out with stale garbage alone as raw material. It needs to be mixed with high calorific value domestic garbage to maintain the thermal balance of the system. The use of anaerobic pyrolysis technology can improve the quality of stale garbage without the need for other supplementary fuels, greatly reducing the disposal cost. However, iron, copper, aluminum, glass, sand and gravel in the garbage cannot be processed by pyrolysis, and these resources are high-value recyclable resources. Traditional waste sorting methods mainly rely on manual selection, which not only consumes a lot of manpower, but also has low sorting efficiency and cannot be guaranteed in accuracy. In particular, metal waste such as iron, copper, aluminum, etc. has high recycling value, but due to their differences in shape, size and color, manual sorting is difficult and prone to errors.

[0003] The rapid development of machine vision technology has provided a new solution for the automatic sorting of waste. By using high-resolution cameras and image processing algorithms, it is possible to automatically identify and classify metal materials such as iron, copper, and aluminum in waste. This sorting method based on machine vision can not only improve sorting efficiency, but also significantly improve sorting accuracy and reduce the need for manual intervention. However, the shapes, colors, and sizes of waste are diverse and often mixed together, which poses a great challenge to image recognition algorithms. In particular, the difficulty of identifying metal materials is further increased when light changes, stains are covered, and other impurities interfere. Summary of the invention

[0004] In order to reduce the workload of recyclable resource sorting and improve the recognition rate of recyclable resources, the present invention provides a method for sorting recyclable garbage resources, the method comprising the following steps:

[0005] Taking images of waste on a garbage conveying device, downsampling the preprocessed images to obtain feature maps of multiple sizes, normalizing the sizes of all feature maps, and obtaining reference points based on contour maps of the feature maps;

[0006] Calculate the offset vector and weight vector of each attention head of the reference point in the same feature map, form a set of the offset of each sampling point and the weight corresponding to the offset, search for the abnormal set of the set, and when calculating the deformable multi-head attention of each feature map of the sampling point, delete the attention head corresponding to the offset vector containing the offset in the abnormal set, and obtain the deformable attention of each reference point based on the deformable multi-head attention of each feature map;

[0007] The deformable attention is input into a decoder to obtain a recyclable resource identification result, and the manipulator is driven to grab the recyclable resource according to the location of the recyclable resource identification result.

[0008] Preferably, the reference point is obtained based on the contour map of the feature map, specifically:

[0009] Obtain the contour point set in the contour map of each feature map, and obtain the position of each contour point in the feature map, take the contour point set of the first contour map as the reference set, and the contour points in the reference set as the reference contour points;

[0010] For each reference contour point, obtain a contour point closest to the reference contour point from each of the remaining contour point sets, calculate the average distance between the reference contour point and all the nearest contour points, and sort the reference contour points in the reference set in ascending order of the average distance;

[0011] The preset number of reference points is obtained, and reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set.

[0012] Preferably, the reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set, specifically:

[0013] The preset number of reference points in the sorted reference set is used as reference points; or,

[0014] Obtain the preset number of reference contour points in the sorted reference set and the contour point closest to the reference contour point in the contour point set of each feature map, and take the average value of the positions of the reference contour point and the nearest contour point as the position of the reference point.

[0015] Preferably, the searching for the abnormal set of the set is specifically:

[0016] For each reference point, the number of sampling points is obtained, the set consisting of the offset and the weight is clustered into clusters of the number of sampling points, the distance from each set to the cluster center is calculated, and the set with a distance greater than a threshold is taken as an abnormal set.

[0017] Preferably, the method of driving the manipulator to grab the recyclable resources according to the location of the recyclable resource identification result is specifically:

[0018] The moving trajectory of the recyclable resources on the conveyor belt is predicted based on the conveyor belt speed and the location of the recyclable resource identification results, the position and time that the robot needs to reach is calculated, and a control instruction is sent to the robot. The robot moves to the specified position according to the instruction, grabs the recyclable resources, and places them in the corresponding recycling container.

[0019] In another aspect, the present invention provides a system for sorting recyclable waste resources, the system comprising the following modules:

[0020] An image capturing module is used to capture images of waste on the garbage conveying device, downsample the preprocessed images to obtain feature maps of multiple sizes, normalize the sizes of all feature maps, and obtain reference points based on contour maps of the feature maps;

[0021] The encoding module is used to calculate the offset vector and weight vector of each attention head of the reference point in the same feature map, form a set of the offset of each sampling point and the weight corresponding to the offset, search for an abnormal set of the set, and when calculating the deformable multi-head attention of each feature map of the sampling point, delete the attention head corresponding to the offset vector containing the offset in the abnormal set, and obtain the deformable attention of each reference point based on the deformable multi-head attention of each feature map;

[0022] The decoding and grasping module is used to input the deformable attention into the decoder and obtain the recyclable resource identification result, and drive the manipulator to grasp the recyclable resource according to the location of the recyclable resource identification result.

[0023] Preferably, the reference point is obtained based on the contour map of the feature map, specifically:

[0024] Obtain the contour point set in the contour map of each feature map, and obtain the position of each contour point in the feature map, take the contour point set of the first contour map as the reference set, and the contour points in the reference set as the reference contour points;

[0025] For each reference contour point, obtain a contour point closest to the reference contour point from each of the remaining contour point sets, calculate the average distance between the reference contour point and all the nearest contour points, and sort the reference contour points in the reference set in ascending order of the average distance;

[0026] The preset number of reference points is obtained, and reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set.

[0027] Preferably, the reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set, specifically:

[0028] The preset number of reference points in the sorted reference set is used as reference points; or,

[0029] Obtain the preset number of reference contour points in the sorted reference set and the contour point closest to the reference contour point in the contour point set of each feature map, and take the average value of the positions of the reference contour point and the nearest contour point as the position of the reference point.

[0030] Preferably, the searching for the abnormal set of the set is specifically:

[0031] For each reference point, the number of sampling points is obtained, the set consisting of the offset and the weight is clustered into clusters of the number of sampling points, the distance from each set to the cluster center is calculated, and the set with a distance greater than a threshold is taken as an abnormal set.

[0032] Preferably, the method of driving the manipulator to grab the recyclable resources according to the location of the recyclable resource identification result is specifically:

[0033] The moving trajectory of the recyclable resources on the conveyor belt is predicted based on the conveyor belt speed and the location of the recyclable resource identification results, the position and time that the robot needs to reach is calculated, and a control instruction is sent to the robot. The robot moves to the specified position according to the instruction, grabs the recyclable resources, and places them in the corresponding recycling container.

[0034] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0035] To address the difficulty in identifying recyclable resources in garbage, the present invention obtains contour information of the captured image, focuses on the contour of the image, extracts reference points from the contour, and removes attention heads where abnormal sampling points are located, thereby improving the accuracy of identifying recyclable resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of Embodiment 1;

[0037] Figure 2 It is a schematic diagram of the relationship between reference point, offset and weight;

[0038] Figure 3 Schematic diagram of the relationship between sampling points and reference points;

[0039] Figure 4 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION

[0040] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In a first embodiment of the present invention, the present invention provides a method for sorting recyclable garbage resources, such as Figure 1 As shown, the method comprises the following steps:

[0043] S1, photographing an image of waste on a garbage conveying device, downsampling the preprocessed image to obtain feature maps of multiple sizes, normalizing the sizes of all feature maps, and obtaining reference points based on a contour map of the feature map;

[0044] When recycling and sorting garbage, images of waste on a garbage conveying device are captured, wherein the conveying device is preferably a conveyor belt. The conveyor belt runs at a certain speed, and when garbage is conveyed on the conveyor belt, in order to avoid mutual occlusion between garbage objects, the distribution of garbage should be sparse, that is, try not to overlap each other, or not completely occlude each other.

[0045] After capturing the waste image on the conveyor belt, the image is downsampled multiple times. Different convolution kernels or pooling can also be used for the waste to obtain feature maps of multiple sizes. Preferably, multiple downsampling, such as convolution, is directly used to obtain feature maps of different sizes. Due to the different sizes of feature maps, the sizes of all feature maps are normalized, specifically, normalized to between 0 and 1, that is, the coordinates of each pixel in each feature map are normalized to between [0,1]. Reference points are a concept in Deformable DETR, which is used to calculate deformable attention and can reduce the amount of calculation in DETR. The selection of reference points is important for recognition speed and accuracy. In target recognition, contours are the main reference content for recognition. For example, most areas of an iron block are the same. Although this part of the area can help identify the iron block, the contour represents the area of ​​the iron block, which is very helpful for improving the recognition and positioning of recyclable resources. However, the position of the contour will change during the downsampling process, that is, the position of the contour may change in feature maps of different sizes. The reference point of the present invention is obtained based on the contour map of all feature maps. In one embodiment, the reference point is obtained based on the contour map of the feature map, specifically:

[0046] Obtain the contour point set in the contour map of each feature map, and obtain the position of each contour point in the feature map, take the contour point set of the first contour map as the reference set, and the contour points in the reference set as the reference contour points;

[0047] For example, there are 4 feature maps. The contour map of each feature map is calculated. Each contour map has contour points. In this way, the contour points of each contour map constitute the contour point set of this contour map. The contour point has a specific position in the contour map. The position of this position in the feature map corresponding to the contour map is the position of the contour point in the feature map. For example, the feature Figure 2 The contour map is the contour Figure 2 ,contour Figure 2 The contour point set is recorded as contour point set 2. Each contour point in contour point set 2 is in the feature Figure 2 There is a position in . The contour point set of the first contour map is taken as the benchmark set, where the first feature map has the largest size among all feature maps, and its corresponding contour map has the largest size as well. The contour points in the benchmark set are the benchmark contour points.

[0048] For each reference contour point, obtain a contour point closest to the reference contour point from each of the remaining contour point sets, calculate the average distance between the reference contour point and all the nearest contour points, and sort the reference contour points in the reference set in ascending order of the average distance;

[0049] Assume that there are 4 feature maps after downsampling. The contour point set of the first feature map is the benchmark set. The coordinates of a benchmark contour point are (0.2, 0.6). For the remaining 3 contour point sets, find the contour point closest to (0.2, 0.6) for each contour point set. Since there are 3 contour point sets left, three contour points closest to point (0.2, 0.6) will be obtained, and these three contour points are located in different contour point sets. The distance is calculated by calculating the distance between the coordinates of two contour points in the corresponding feature map. Taking the second contour point set as an example, assuming that point (0.2, 0.5) is closest to (0.2, 0.6), then point (0.2, 0.5) in the second contour point set is the contour point closest to the benchmark contour point. Each benchmark contour point corresponds to multiple of the closest contour points. The average distance between the benchmark contour point and the contour point closest to the benchmark contour point is calculated, and the benchmark contour points in the benchmark set are sorted in order from the smallest to the smallest average distance. For example, there are 3 reference contour points, the average distance of the first reference contour point is 1, the average distance of the second reference contour point is 0.2, and the average distance of the third reference contour point is 0.6. Then the order of sorting the reference contour points in the reference set is the second reference contour point, the third reference contour point, and the first reference contour point.

[0050] The preset number of reference points is obtained, and reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set.

[0051] Assuming that the number of preset reference points is 1024, 1024 reference contour points are selected from the sorted reference set as reference points. In one embodiment, the reference contour points of the preset number of reference points in the sorted reference set are used as reference points, that is, the first 1024 reference contour points in the sorted reference set are selected as reference points, and the positions of the reference points are the positions of the selected 1024 reference points.

[0052] If each layer of the encoder uses a feature map, reference points are obtained in the above manner, and a feature map of the first layer is obtained. For other layers, obtaining reference points includes but is not limited to obtaining reference points in the manner of obtaining reference points in Deformable DETR. The layers described here refer to the layers of the encoder in Deformable DETR, and each layer may have one feature map or multiple feature maps. Obtaining reference points from a contour map based on a feature map described in the present invention refers to a process of obtaining feature maps of multiple sizes by downsampling the preprocessed image, and obtaining a feature map of the first layer based on the feature maps of multiple sizes.

[0053] If each layer of the encoder uses feature maps of multiple sizes, different preset numbers of reference points are set for other feature maps. For example, the Deformable DETR encoder has three feature maps of different scales per layer. Three preset numbers of reference points are set for these three feature maps. The reference points are obtained in the above method for the feature map with the largest size in the first layer, and the other feature maps in the first layer use the default reference point acquisition method in Deformable DETR.

[0054] In another embodiment, a contour map-based method is also used to obtain reference points of the second feature map of the first layer. Specifically, the contour point set in the contour map of the feature map of the second size after downsampling is used as a reference set, and the reference points of the second feature map of the first layer are obtained in the same manner as above. For the reference points of the third feature map of the first layer, the reference points are obtained in the same manner as above. At this time, the contour point set in the contour map of the feature map of the second size after downsampling is used as the reference set, and so on. It will not be repeated here.

[0055] For example, if the preset reference point of the second feature map of the first layer is 512, then the element in the contour point set of the second size after downsampling is obtained as the reference contour point. In another embodiment, the number of reference points of all feature maps of each layer is the same, that is, the size of the feature map of each layer is the same. In order to prevent the reference points from being concentrated in a small range, in another embodiment, the distribution of the reference points is determined. If multiple reference points are distributed in a preset range, one of the reference points is deleted, and the reference points are obtained from the sorted reference set again, and so on. Generally speaking, the number of reference points is relatively large and will not be concentrated in a small range.

[0056] In one embodiment, the reference point obtained based on the contour map of the feature map is used to obtain the reference point of the first feature map of the first layer of the encoder. If the encoder has multiple feature maps per layer, the other feature maps of each layer are obtained by using the method in the DeformableDETR model. For the other feature maps of the first layer, the backbone network is still used to extract features, and then the other feature maps of the first layer are obtained by using the method in the Deformable DETR model.

[0057] In another embodiment, the preset number of reference points in the sorted reference set and the contour point closest to the reference contour point in the contour point set of each feature map are obtained, and the average value of the positions of the reference contour point and the nearest contour point is used as the position of the reference point.

[0058] Each of the remaining contour point sets has a contour point that is closest to the benchmark contour point, and the average position of the benchmark contour point and the contour point closest to the benchmark contour point is used as the position of the reference point. For example, 512 benchmark contour points are selected from the sorted benchmark set, and each of the remaining 3 contour point sets has a contour point that is closest to the selected benchmark contour point, and the average value of the coordinates of the 3 closest contour points and the selected benchmark contour point is used as the position of the reference point.

[0059] S2, calculate the offset vector and weight vector of each attention head of the reference point in the same feature map, form a set of the offset of each sampling point and the weight corresponding to the offset, search for the abnormal set of the set, and when calculating the deformable multi-head attention of each feature map of the sampling point, delete the attention head corresponding to the offset vector containing the offset in the abnormal set, and obtain the deformable attention of each reference point based on the deformable multi-head attention of each feature map;

[0060] After obtaining the reference point, the deformable attention of each reference point on each feature map is calculated. Specifically, the offset vector and attention head weight vector of the reference point query feature vector are obtained for each reference point, and each attention head obtains the output of the attention head according to the offset vector and the weight vector. Among them, each offset vector has K elements, and each weight vector also has K vectors, K is the preset number of sampling points, preferably, K=3 or 4. That is, each sampling point corresponds to an offset and weight.

[0061] Assuming the number of sampling points is 4, the first feature map has 3 attention heads. For a given reference point, this reference point will have a set of 12 offsets and weights, such as Figure 2 shown.

[0062] The offset is a measure of where the reference point focuses, and the weight is a measure of the attention of the reference point. On the same feature map, a better situation is that a reference point pays attention to the same or similar area on the feature map. If an abnormal set appears, this set is likely to be noise, such as Figure 3 As shown, reference point A has 4 sampling points. In the first and second deformable attentions, the offsets and weights of the 4 sampling points are similar, but in the third deformable attention, they deviate significantly from the first two, which will affect the final recognition and the output of this attention head needs to be eliminated.

[0063] Specifically, for a given reference point, after obtaining a set consisting of the offsets and weights of all sampling points in the same feature map, search for abnormal sets in the set. In a preferred embodiment, for each reference point, obtain the number of sampling points, cluster the set consisting of the offsets and weights into clusters of the number of sampling points, calculate the distance from each set to the cluster center, and take the set with a distance greater than the threshold as an abnormal set. For example, a reference point is R1, and the number of sampling points is 4. The set of sampling points of this reference point on each feature map is clustered. The clustering preferably uses the K-means clustering method, where K=4, that is, clustered into 4 categories, each sampling point is a category, and then calculate the distance from each set to the cluster center, and take the set with a distance greater than the threshold as an abnormal set. Other methods can also be used to search for abnormal sets of the set, and the abnormal set search method is not specifically limited here.

[0064] When calculating the deformable attention of a given reference point on a certain feature map, the attention heads where the abnormal set is located are deleted, that is, the attention heads where the abnormal set is located do not participate in the calculation of multi-head attention, which can avoid the influence of abnormal sampling points in a certain attention head on the result.

[0065] After obtaining the deformable multi-head attention on each feature map, when each layer has multiple feature maps, a multi-scale deformable multi-head attention can be further obtained. Preferably, the multi-scale deformable multi-head attention module in the Deformable DETR model is used for calculation. It should be noted that S2 will be executed when calculating the deformable multi-head attention at each layer of the encoder, or it can be executed only in the first layer or several layers, and the other layers are still calculated using the deformable multi-head attention method in the Deformable DETR model.

[0066] S3, inputting the deformable attention into a decoder and obtaining a recyclable resource identification result, and driving a manipulator to grab the recyclable resource according to the location of the recyclable resource identification result.

[0067] The result of the multi-scale deformable multi-head attention is input into a decoder with a deformable attention module. Preferably, the decoder adopts a decoder of the Deformable DETR model. The decoder obtains the recyclable resource identification result, which includes the type of recyclable resources and the position in the waste image. The control module can drive the manipulator to grab the recyclable resources according to the conveyor belt speed and the position, and put them into the corresponding container such as a basket. Specifically, the moving trajectory of the recyclable resources on the conveyor belt is predicted according to the conveyor belt speed and the position of the recyclable resource identification result, the position and time that the manipulator needs to reach is calculated, and a control instruction is sent to the manipulator. The manipulator moves to the specified position according to the instruction, grabs the recyclable resources, and places them in the corresponding recycling container.

[0068] In a second embodiment of the present invention, the present invention provides a system for sorting recyclable garbage resources, such as Figure 4 As shown, the system includes the following modules:

[0069] An image capturing module is used to capture images of waste on the garbage conveying device, downsample the preprocessed images to obtain feature maps of multiple sizes, normalize the sizes of all feature maps, and obtain reference points based on contour maps of the feature maps;

[0070] The encoding module is used to calculate the offset vector and weight vector of each attention head of the reference point in the same feature map, form a set of the offset of each sampling point and the weight corresponding to the offset, search for an abnormal set of the set, and when calculating the deformable multi-head attention of each feature map of the sampling point, delete the attention head corresponding to the offset vector containing the offset in the abnormal set, and obtain the deformable attention of each reference point based on the deformable multi-head attention of each feature map;

[0071] The decoding and grasping module is used to input the deformable attention into the decoder and obtain the recyclable resource identification result, and drive the manipulator to grasp the recyclable resource according to the location of the recyclable resource identification result.

[0072] Preferably, the reference point is obtained based on the contour map of the feature map, specifically:

[0073] Obtain the contour point set in the contour map of each feature map, and obtain the position of each contour point in the feature map, take the contour point set of the first contour map as the reference set, and the contour points in the reference set as the reference contour points;

[0074] For each reference contour point, obtain a contour point closest to the reference contour point from each of the remaining contour point sets, calculate the average distance between the reference contour point and all the nearest contour points, and sort the reference contour points in the reference set in ascending order of the average distance;

[0075] The preset number of reference points is obtained, and reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set.

[0076] Preferably, the reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set, specifically:

[0077] The preset number of reference points in the sorted reference set is used as reference points; or,

[0078] Obtain the preset number of reference contour points in the sorted reference set and the contour point closest to the reference contour point in the contour point set of each feature map, and take the average value of the positions of the reference contour point and the nearest contour point as the position of the reference point.

[0079] Preferably, the searching for the abnormal set of the set is specifically:

[0080] For each reference point, the number of sampling points is obtained, the set consisting of the offset and the weight is clustered into clusters of the number of sampling points, the distance from each set to the cluster center is calculated, and the set with a distance greater than a threshold is taken as an abnormal set.

[0081] Preferably, the method of driving the manipulator to grab the recyclable resources according to the location of the recyclable resource identification result is specifically:

[0082] The moving trajectory of the recyclable resources on the conveyor belt is predicted based on the conveyor belt speed and the location of the recyclable resource identification results, the position and time that the robot needs to reach is calculated, and a control instruction is sent to the robot. The robot moves to the specified position according to the instruction, grabs the recyclable resources, and places them in the corresponding recycling container.

[0083] In a third embodiment of the present invention, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first embodiment is implemented.

[0084] In a fourth embodiment of the present invention, the present invention further provides a computer device, which includes at least a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described in the first embodiment is implemented.

[0085] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for sorting recyclable garbage resources, characterized in that: The method comprises the following steps: Taking images of waste on a garbage conveying device, downsampling the preprocessed images to obtain feature maps of multiple sizes, normalizing the sizes of all feature maps, and obtaining reference points based on contour maps of the feature maps; Calculate the offset vector and weight vector of each attention head of the reference point in the same feature map, form a set of the offset of each sampling point and the weight corresponding to the offset, search for the abnormal set of the set, and when calculating the deformable multi-head attention of each feature map of the sampling point, delete the attention head corresponding to the offset vector containing the offset in the abnormal set, and obtain the deformable attention of each reference point based on the deformable multi-head attention of each feature map; Inputting the deformable attention into a decoder and obtaining a recyclable resource identification result, and driving a manipulator to grab the recyclable resource according to the location of the recyclable resource identification result; The reference point is obtained based on the contour map of the feature map, specifically: Obtain the contour point set in the contour map of each feature map, and obtain the position of each contour point in the feature map, take the contour point set of the first contour map as the reference set, and the contour points in the reference set as the reference contour points; For each reference contour point, obtain a contour point closest to the reference contour point from each of the remaining contour point sets, calculate the average distance between the reference contour point and all the nearest contour points, and sort the reference contour points in the reference set in ascending order of the average distance; The preset number of reference points is obtained, and reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set.

2. The method according to claim 1, characterized in that The reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set, specifically: The preset number of reference points in the sorted reference set is used as reference points; or, Obtain the preset number of reference contour points in the sorted reference set and the contour point closest to the reference contour point in the contour point set of each feature map, and take the average value of the positions of the reference contour point and the nearest contour point as the position of the reference point.

3. The method according to claim 1, characterized in that The search for the abnormal set of the set is specifically: For each reference point, the number of sampling points is obtained, the set consisting of the offset and the weight is clustered into clusters of the number of sampling points, the distance from each set to the cluster center is calculated, and the set with a distance greater than a threshold is taken as an abnormal set.

4. The method according to claim 1, characterized in that The method of driving the manipulator to grab the recyclable resources according to the location of the recyclable resource identification result is specifically: The moving trajectory of the recyclable resources on the conveyor belt is predicted based on the conveyor belt speed and the location of the recyclable resource identification results, the position and time that the robot needs to reach is calculated, and a control instruction is sent to the robot. The robot moves to the specified position according to the instruction, grabs the recyclable resources, and places them in the corresponding recycling container.

5. A garbage recyclable resource sorting system, characterized in that: The system includes the following modules: An image capturing module is used to capture images of waste on the garbage conveying device, downsample the preprocessed images to obtain feature maps of multiple sizes, normalize the sizes of all feature maps, and obtain reference points based on contour maps of the feature maps; The encoding module is used to calculate the offset vector and weight vector of each attention head of the reference point in the same feature map, form a set of the offset of each sampling point and the weight corresponding to the offset, search for an abnormal set of the set, and when calculating the deformable multi-head attention of each feature map of the sampling point, delete the attention head corresponding to the offset vector containing the offset in the abnormal set, and obtain the deformable attention of each reference point based on the deformable multi-head attention of each feature map; A decoding and grasping module, used for inputting the deformable attention into a decoder and obtaining a recyclable resource identification result, and driving a manipulator to grasp the recyclable resource according to the location of the recyclable resource identification result; The reference point is obtained based on the contour map of the feature map, specifically: Obtain the contour point set in the contour map of each feature map, and obtain the position of each contour point in the feature map, take the contour point set of the first contour map as the reference set, and the contour points in the reference set as the reference contour points; For each reference contour point, obtain a contour point closest to the reference contour point from each of the remaining contour point sets, calculate the average distance between the reference contour point and all the nearest contour points, and sort the reference contour points in the reference set in ascending order of the average distance; The preset number of reference points is obtained, and reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set.

6. The system according to claim 5, characterized in that The reference points are obtained according to the reference contour points of the preset number of reference points in the sorted reference set, specifically: The preset number of reference points in the sorted reference set is used as reference points; or, Obtain the preset number of reference contour points in the sorted reference set and the contour point closest to the reference contour point in the contour point set of each feature map, and take the average value of the positions of the reference contour point and the nearest contour point as the position of the reference point.

7. The system according to claim 5, characterized in that The search for the abnormal set of the set is specifically: For each reference point, the number of sampling points is obtained, the set consisting of the offset and the weight is clustered into clusters of the number of sampling points, the distance from each set to the cluster center is calculated, and the set with a distance greater than a threshold is taken as an abnormal set.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 4.