Recoverable material sorting treatment method and device based on AI

Abstract

The invention discloses an AI-based recoverable material sorting treatment method and device, and relates to the technical field of recoverable material sorting. In the method, a first image corresponding to a first sample is obtained, and the first sample is any renewable resource plastic bottle in the recoverable material; processing the first image to obtain a second image; inputting the second image into a preset classification recognition model for processing to obtain a classification result; according to the classification result, the material category is determined, a stacking area corresponding to the material category is determined, and the stacking area is the area where the renewable resource plastic bottles corresponding to the material category are stacked; and sending a grabbing instruction to the mechanical arm so as to control the mechanical arm to grab the first sample to the stacking area. By implementing the technical scheme provided by the invention, the whole process automation from identification to sorting can be realized, so that the need of manual assistance is avoided, and the overall sorting efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of recyclable sorting, and particularly to an AI-based recyclable sorting and processing method and device. Background Art

[0002] In the field of environmental protection industry, the sorting link of renewable resources plays a crucial role. Through a fine sorting process, various renewable resources such as metals, plastics, and waste electrical and electronic products can be effectively distinguished, and then the classified utilization and optimal allocation of resources can be realized. This not only significantly reduces the phenomenon of resource waste, but also greatly improves the utilization efficiency of resources, which has far-reaching significance for reducing the dependence on primary resources, reducing resource consumption and environmental pollution.

[0003] Currently, the commonly used technical solutions in the field of renewable resource sorting include the application of optical sorters or the combination of optical sorters and sorting robots. In the original input of renewable resource plastic bottles, there are often various materials mixed. Due to its advantages of high speed and high throughput, the optical sorter is usually the first choice for preliminary screening. However, in the intelligent sorting process, the optical sorter can only perform blowing selection for a single category, and the remaining mixed materials need to enter the sorting robot process for further sorting. When the sorting robot performs the sorting task, it needs to perform multi-class intelligent recognition and sorting on all input renewable resources. Different from the one-to-many classification sorting method of the optical sorter, the sorting robot needs to identify all input materials one by one and sort multiple categories at the same time. However, in the current sorting process, when the robot realizes high-efficiency recognition and sorting of renewable resource samples, the sorting process still requires manual assistance, which will lead to a relatively low overall sorting efficiency.

[0004] Therefore, there is an urgent need for an AI-based recyclable sorting and processing method and device that can solve the above technical problems. Summary of the Invention

[0005] This application provides an AI-based recyclable sorting and processing method and device. This method can realize the full automation from recognition to sorting, thus avoiding the need for manual assistance and improving the overall sorting efficiency.

[0006] In the first aspect, this application provides an AI-based recyclable sorting and processing method, which is applied to a sorting and processing platform. The method includes: obtaining a first image corresponding to a first sample, where the first sample is any renewable resource plastic bottle in recyclables; processing the first image to obtain a second image; inputting the second image into a preset classification and recognition model for processing to obtain a classification result; determining the material category according to the classification result, and determining the stacking area corresponding to the material category, where the stacking area is the area for stacking renewable resource plastic bottles corresponding to the material category; sending a grasping instruction to the robotic arm to control the robotic arm to grasp the first sample to the stacking area.

[0007] By adopting the above technical solution, the first image of the first sample is obtained, avoiding the need for manual selection or marking of samples. The first image is processed to obtain a second image, and the second image is input into a preset classification and recognition model for recognition to obtain a classification result. The preset classification and recognition model can accurately identify the material category of the input second image, reduce manual intervention, determine the material category according to the classification result, and determine the stacking area corresponding to the material category, and send a grasping instruction to the robotic arm to control the robotic arm to grasp the first sample to the stacking area without manual operation, thus realizing the full automation from recognition to sorting, avoiding the need for manual assistance, and improving the overall sorting efficiency.

[0008] Optionally, before obtaining the first image corresponding to the first sample, the method further includes: receiving a sorting request sent by a user device, scanning a conveyor belt according to the sorting request to obtain a third image, where the conveyor belt is used to convey renewable resource plastic bottles to be detected; performing polygon annotation on the target contour of a second sample in the third image to obtain a fourth image, where the second sample is any renewable resource plastic bottle appearing in the third image, and the target contour is the outer contour corresponding to the second sample; if it is determined that the fourth image is in a completely annotated state, then output the fourth image as the first image.

[0009] By adopting the above technical solution, receiving the sorting request sent by the user device, immediately responding and scanning the conveyor belt, thereby obtaining the third image of the renewable resource plastic bottles to be detected, performing polygon annotation on the plastic bottles in the third image, and once the annotation is completed, using the annotated third image as the input for subsequent classification recognition and processing, greatly shortening the sorting cycle, and it is also necessary to judge the quality of the image to avoid problems caused by incomplete images or inaccurate annotations.

[0010] Optionally, before sending the grasping instruction to the robotic arm, the method further includes: inputting the first image into a preset rotated rectangle detection model for processing to obtain vertex coordinates; calculating the vertex coordinates to obtain target center coordinates; determining the target outer shape corresponding to the first sample according to the first image; judging whether the target outer shape is a centrosymmetric outer shape; when the target outer shape is a centrosymmetric outer shape, confirming to integrate the target center coordinates and the stacking area to obtain a grasping instruction; sending the grasping instruction to the robotic arm to control the robotic arm to grasp the first sample according to the target center coordinates.

[0011] By adopting the above technical solution, inputting the first image into a preset rotated rectangular box detection model can accurately identify the vertex coordinates of the first sample in the first image. By calculating the vertex coordinates, the target center coordinates, i.e., the center position of the plastic bottle, can be further determined, which helps improve the accuracy of the robotic arm's grasping. It is necessary to determine whether the target shape is a centrally symmetric shape. For a plastic bottle with a centrally symmetric shape, grasping its center position can ensure the stability and balance of the grasping process, reducing the risk of grasping failure or plastic bottle damage caused by improper grasping positions.

[0012] Optionally, after determining whether the target shape is a centrally symmetric shape, the method further includes: when the target shape is not a centrally symmetric shape, confirming to calculate the center point compensation algorithm for the target center coordinates to obtain the center of gravity coordinates; integrating the center of gravity coordinates and the stacking area to obtain a grasping instruction, and sending the grasping instruction to the robotic arm so that the robotic arm can grasp the first sample according to the center of gravity coordinates.

[0013] By adopting the above technical solution, for a plastic bottle with a non-centrally symmetric shape, its center of gravity often does not lie at the geometric center. Therefore, directly grasping the geometric center may lead to unstable grasping, and the plastic bottle is likely to slip or tilt during the grasping process. The center of gravity coordinates calculated by the center point compensation algorithm can more accurately reflect the actual weight distribution of the plastic bottle, thereby ensuring stability during grasping, reducing friction and loss during the grasping process, and improving the intact rate and sorting efficiency of the plastic bottle.

[0014] Optionally, processing the first image to obtain a second image specifically includes: scaling the first image to obtain the scaled first image; filling the pixel values of the scaled first image to obtain the second image.

[0015] By adopting the above technical solution, adjusting the size of the first image as needed, scaling helps ensure that the image matches the size requirements of subsequent processing steps. During the scaling process, pixel distortion may occur. By filling the pixel values, the image can be made smoother and more coherent, improving the processing efficiency of sorting.

[0016] Optionally, inputting the second image into a preset classification and recognition model for processing to obtain a classification result specifically includes: extracting features from the second image to obtain target features; inputting the target features into a fully connected layer for classification processing to obtain multiple class probabilities, where one class probability corresponds to one material; sorting the multiple class probabilities from largest to smallest to obtain a sorting result; obtaining the target class probability, where the target class probability is the class probability corresponding to the first place in the sorting result; and outputting the target label corresponding to the target class probability as the classification result.

[0017] By adopting the above technical solution, key features related to material classification can be extracted from the second image. The extracted target features are input into a fully connected layer for classification processing, and multiple class probabilities can be obtained. Each class probability corresponds to a material. By sorting and selecting the target class probability, the most likely material classification result can be ensured, achieving accurate classification of renewable resource plastic bottles and improving the classification accuracy.

[0018] Optionally, determining the material category according to the classification result specifically includes: determining the target label according to the classification result; determining the material category based on the target label according to the preset mapping relationship, where one target label corresponds to one material category.

[0019] By adopting the above technical solution, the classification result gives the target label corresponding to the material category. The target label represents the material of the plastic bottle in the image. According to the mapping relationship, the target label can be converted into a specific material category. The mapping relationship ensures the accuracy and readability of the classification result and improves the efficiency of data processing.

[0020] In the second aspect of the present application, an AI-based recyclable sorting and processing device is provided. The device is a sorting and processing platform, which includes an acquisition unit, a processing unit, and a sending unit. The acquisition unit acquires the first image corresponding to the first sample, and the first sample is any renewable resource plastic bottle in the recyclables; the processing unit processes the first image to obtain a second image; inputs the second image into a preset classification and recognition model for processing to obtain a classification result; determines the material category according to the classification result, and determines the stacking area corresponding to the material category. The stacking area is the area for stacking renewable resource plastic bottles corresponding to the material category; the sending unit sends a grasping instruction to the robotic arm to control the robotic arm to grasp the first sample to the stacking area.

[0021] Optionally, the acquisition unit is used to receive the sorting request sent by the user device, scan the conveyor belt according to the sorting request to obtain a third image. The conveyor belt is used to convey the renewable resource plastic bottles to be detected; the processing unit is used to perform polygon annotation on the target contour of the second sample in the third image to obtain a fourth image. The second sample is any renewable resource plastic bottle that appears in the third image, and the target contour is the external contour corresponding to the second sample; if it is determined that the fourth image is in a complete annotation state, the fourth image is output as the first image.

[0022] Optionally, the processing unit is configured to process the first image by inputting it into a preset rotated rectangular box detection model to obtain vertex coordinates; calculate the vertex coordinates to obtain target center coordinates; determine the target shape corresponding to the first sample according to the first image; determine whether the target shape is a centrally symmetric shape; when the target shape is a centrally symmetric shape, confirm to integrate the target center coordinates and the stacking area to obtain a grasping instruction; the sending unit is configured to send the grasping instruction to the robotic arm so as to control the robotic arm to grasp the first sample according to the target center coordinates.

[0023] Optionally, when the target shape is not a centrally symmetric shape, the processing unit is configured to confirm to perform a center point compensation algorithm calculation on the target center coordinates to obtain the center of gravity coordinates; the sending unit is configured to integrate the center of gravity coordinates and the stacking area to obtain a grasping instruction, and send the grasping instruction to the robotic arm so that the robotic arm grasps the first sample according to the center of gravity coordinates.

[0024] Optionally, the processing unit is configured to scale the first image to obtain a scaled first image; perform pixel value filling on the scaled first image to obtain a second image.

[0025] Optionally, the processing unit is configured to extract features from the second image to obtain target features; input the target features into a fully connected layer for classification processing to obtain multiple class probabilities, where one class probability corresponds to one material; sort the multiple class probabilities from largest to smallest to obtain a sorting result; the obtaining unit is configured to obtain a target class probability, and the target class probability is the class probability corresponding to the first position in the sorting result; the processing unit is configured to output the target label corresponding to the target class probability as the classification result.

[0026] Optionally, the processing unit is configured to determine a target label according to the classification result; determine a material category based on the target label according to a preset mapping relationship, where one target label corresponds to one material category.

[0027] In a third aspect of the present application, an electronic device is provided. The electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is configured to execute the instructions stored in the memory, so that an electronic device executes the method according to any one of the above in the present application.

[0028] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of the above in the present application is executed.

[0029] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Obtain the first image of the first sample, which avoids the need for manual selection or marking of samples. Process the first image to obtain a second image, and input the second image into a preset classification and recognition model for recognition to obtain a classification result. The preset classification and recognition model can accurately identify the material category of the input second image, reduce manual intervention, determine the material category according to the classification result, and determine the stacking area corresponding to the material category. Send a grasping instruction to the robotic arm to control the robotic arm to grasp the first sample to the stacking area without manual operation, thereby realizing the full automation from recognition to sorting, avoiding the need for manual assistance, and improving the overall sorting efficiency.

[0030] 2. Receive the sorting request sent by the user device, immediately respond and scan the conveyor belt to obtain the third image of the recyclable plastic bottles to be detected. Perform polygon annotation on the plastic bottles in the third image. Once the annotation is completed, use the annotated third image as the input for subsequent classification recognition and processing, which greatly shortens the sorting cycle. It is also necessary to judge the quality of the image to avoid problems caused by incomplete images or inaccurate annotations. Description of the Drawings

[0031] Figure 1 is a schematic flowchart of a method for sorting and processing recyclable materials based on AI provided by an embodiment of the present application; Figure 2 is a schematic structural diagram of a device for sorting and processing recyclable materials based on AI provided by an embodiment of the present application; Figure 3 is a schematic structural diagram of an electronic device disclosed by an embodiment of the present application.

[0032] Description of the reference numerals: 201, acquisition unit; 202, processing unit; 203, sending unit; 300, electronic device; 301, processor; 302, memory; 303, user interface; 304, network interface; 305, communication bus. Detailed Embodiments

[0033] In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0034] In the description of the embodiments of the present application, words such as "for example" or "for illustration" are used to give examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for illustration" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "for example" or "for illustration" is intended to present the relevant concepts in a specific manner.

[0035] In the description of the embodiments of the present application, the term "a plurality of" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0036] In the field of the environmental protection industry, the sorting link of renewable resources plays a crucial role. Through a fine sorting process, various types of renewable resources such as metals, plastics, and waste electrical and electronic products can be effectively distinguished, and then the classified utilization and optimal allocation of resources can be realized. This not only significantly reduces the phenomenon of resource waste, but also greatly improves the utilization efficiency of resources, which is of far-reaching significance for reducing the dependence on primary resources, reducing resource consumption and environmental pollution.

[0037] Currently, the commonly adopted technical solutions in the field of renewable resource sorting include the application of optical sorters or the combination of optical sorters and sorting robots. In the original input of renewable resource plastic bottles, there are often various materials of renewable resources mixed. Due to its advantages of high speed and high throughput, the optical sorter is usually the first choice for preliminary screening. However, in the intelligent sorting process, the optical sorter can only achieve the blowing selection of a single category, and the remaining mixed materials need to enter the sorting robot process for further sorting. When the sorting robot performs the sorting task, it needs to perform multi-category intelligent recognition and sorting on all the input renewable resources. Different from the one-to-many classification sorting method of the optical sorter, the sorting robot needs to identify all the input materials one by one and sort multiple categories at the same time. However, in the current sorting process, when the robot realizes the high-efficiency recognition and sorting of renewable resource samples, the sorting process still requires manual assistance, which will lead to a relatively low overall sorting efficiency.

[0038] Therefore, how to solve the problem that robots still need manual assistance in the current sorting process when achieving high-efficiency identification and sorting of renewable resource samples. A sorting and processing method for recyclables based on AI provided by an embodiment of the present application is applied to a sorting and processing platform. The sorting and processing platform of the present application can be a platform that provides material sorting services for renewable resource sorting institutions. Figure 1 It is a schematic flowchart of a sorting and processing method for recyclables based on AI provided by an embodiment of the present application. Refer to Figure 1 , this method includes the following steps S101 - step S105.

[0039] S101: Obtain a first image corresponding to a first sample, where the first sample is any renewable resource plastic bottle among recyclables.

[0040] In the above S101, before the sorting and processing platform obtains the first image corresponding to the first sample, it needs to scan all the samples to be detected on the conveyor belt according to the received sorting request to obtain a complete image, then label according to the shape of each sample to be detected to obtain the image corresponding to each sample to be detected, and then sequentially identify the images corresponding to each sample to be detected to determine the material corresponding to the sample to be detected in the image.

[0041] In addition, receive the sorting request sent by the user device, scan the conveyor belt according to the sorting request to obtain a third image. The conveyor belt is used to convey renewable resource plastic bottles to be detected; perform polygon annotation on the target contour of the second sample in the third image to obtain a fourth image. The second sample is any renewable resource plastic bottle that appears in the third image, and the target contour is the outer contour corresponding to the second sample; if it is determined that the fourth image is in a completely annotated state, then output the fourth image as the first image. Specifically, receive the signal from the user device. The user device (such as a touch screen, computer terminal, etc.) sends a sorting request by wired or wireless means. After receiving the request, parse the information therein, including the material corresponding to the type of item to be sorted (in this case, renewable resource plastic bottles). Confirm the received request to ensure the integrity and accuracy of the information. According to the information in the sorting request, start the conveyor belt and the scanning device thereon (such as a camera, sensor, etc.). The scanning device continuously scans the items on the conveyor belt to capture their dynamic images. During the scanning process, the scanning frequency and exposure time will be adjusted according to the moving speed of the items and the performance of the scanning device to ensure that the captured images are clear and complete. At this time, the items refer to renewable resource plastic bottles. The scanning device transmits the captured images to the sorting and processing platform, and the sorting and processing platform preprocesses the images, such as denoising, enhancing contrast, etc., to improve the image quality. The processed images are saved as the third image. The sorting and processing platform uses image processing algorithms (such as edge detection, contour extraction, etc.) to detect the items in the third image and identify the target contour of the renewable resource plastic bottles (the second sample). Use a polygon annotation tool to draw a polygon frame on the image according to the detected target contour. The polygon frame should closely fit the target contour to accurately label the position and shape of the item. During the annotation process, the sorting and processing platform will automatically adjust the number and position of the vertices of the polygon frame according to the shape and size of the item to ensure the accuracy and integrity of the annotation. Check the annotation result to see if the polygon frame completely covers the target contour and does not exceed the target range. If the check fails, the sorting and processing platform will re-annotate or prompt the user to make manual corrections. According to the annotation result and the check information, determine whether the fourth image is in a completely annotated state. The completely annotated state means that the polygon frame accurately and completely annotates the target contour without omission or error. Methods such as overlap comparison or area calculation can be used to determine whether the fourth image is in a completely annotated state, that is, compare the annotated fourth image with the area corresponding to the second sample in the third image, and then determine whether the annotated fourth image is in a completely annotated state. The completely annotated state can also be understood as the plastic bottle being complete and the image being clear and unobstructed. If there are multiple samples in the third image, label the other samples in the third image in turn according to the method of labeling the second sample above to obtain multiple labeled images.After confirming that the fourth image is in a completely labeled state, the fourth image is output as the first image. After outputting the first image, it is necessary to identify and sort the material of the plastic bottle in the first image.

[0042] S102: Process the first image to obtain a second image.

[0043] In the above S102, after obtaining the first image, the first image is processed to obtain a second image, which specifically includes: scaling the first image to obtain a scaled first image; filling the pixel values of the scaled first image to obtain a second image. Specifically, after obtaining the first image, the original size corresponding to the first image is obtained, and then the size of the scaled image is determined according to actual needs. The proportional relationship between the size of the scaled image and the original size is determined, and then the scaling function provided by the image processing library is used to perform a scaling operation on the first image. The first image, the target size, and the scaling algorithm are input to obtain a scaled first image. After determining the size scaling of the first image, it is also necessary to fill the pixel values of the scaled first image, that is, obtain the background pixel value corresponding to the scaled first image and fill the background pixel value. A pixel value threshold can be set, and the scaled first image is filled according to the pixel value threshold to obtain a filled image, that is, a second image. After obtaining the second image, the second image is verified to ensure that the filling effect and the scaling effect meet actual needs.

[0044] For example, if the scaling threshold is set to a resolution of 224*224, that is, the first image needs to be scaled into a picture with a resolution of 224*224, and then the background filling pixel value RGB=(114, 114, 114) is used, and the background pixel value of the scaled first image is filled to the gray color of RGB=(114, 114, 114) to obtain a second image.

[0045] S103: Input the second image into a preset classification and recognition model for processing to obtain a classification result.

[0046] In the above S103, before inputting the second image into the preset classification recognition model for processing to obtain the classification result, it is necessary to construct the preset classification recognition model. The construction process is as follows: Obtain the second images corresponding to multiple samples to be detected. It is necessary to ensure that the sizes and pixel values of the multiple second images have been adjusted to the required format, and then perform feature extraction on the second images to obtain target features. The target features contain the key information in the second images and are used for subsequent classification decisions. Processing can be performed using a Vision Transformer (ViT) model and a multi-layer perceptron output head. Then, perform classification processing on the extracted target features. In this application, the materials of renewable resource plastic bottles are divided into 5 categories according to the business, with one category corresponding to one material. The 5 categories respectively include semi-transparent HDPE material, opaque white HDPE material, opaque non-white HDPE material, variegated PET material, and other materials. Calculate the probabilities of the target features with the 5 categories in sequence to determine the probabilities of the target features being similar to each category. Then, sort the probability vectors, select the category label with the highest probability, and output this label as the classification result. Two connection layers can be used to perform classification processing on the extracted features. The output of the last fully connected layer is usually a vector containing category probabilities. Each category of this vector corresponds to the probability of a category. By selecting the category with the highest probability, the final classification result can be obtained. After constructing the preset classification recognition model, subsequently, after inputting a new image, the preset classification recognition model performs AI processing and calculation on the new image to obtain the material information of the sample in the image.

[0047] After obtaining the preset classification recognition model according to the above construction process, the second image is input into the preset classification recognition model for processing to obtain a classification result, which specifically includes: extracting features of the second image to obtain target features; inputting the target features into a fully connected layer for classification processing to obtain multiple category probabilities, where one category probability corresponds to one material; sorting the multiple category probabilities from largest to smallest to obtain a sorting result; obtaining the target category probability, which is the category probability corresponding to the first place in the sorting result; and outputting the target label corresponding to the target category probability as the classification result. Specifically, a pre-trained model can be used to segment the second image. The pre-trained model is based on Vision Transformer (ViT), which is a model based on the Transformer architecture. In ViT-Small, the second image is first segmented into a series of small patches, and each small patch is regarded as a "token". These tokens are input into the encoder of the Transformer for processing. The encoder consists of multiple self-attention layers and feed-forward neural network layers. Through this process, the model can extract high-level features in the image. After the encoder of ViT-Small, there is usually a multi-layer perceptron (MLP) as the output head. This MLP consists of several fully connected layers (FC) and is used to further process and transform the features output by the encoder for subsequent classification tasks. The output of the MLP Head can be regarded as the feature representation extracted from ViT-Small. These features contain the key information in the image and are used for subsequent classification decisions. After the MLP Head, there are usually two or more layers of fully connected layers connected. These fully connected layers are used to further classify the extracted target features. The output of the last fully connected layer is usually a vector containing category probabilities. The extracted target features are converted into vector form and used as the input of the fully connected layer. The input feature vector is linearly transformed through the weight matrix and bias vector of the fully connected layer, and a non-linear process is applied by using an activation function (such as ReLU, Sigmoid, etc.). The output of the fully connected layer is converted into a category probability distribution through a function such as Softmax. Since there are 5 categories set for the materials in this application, the probabilities between the target features and each category can be obtained. The total probability corresponding to the 5 categories is set to 1, that is, the probability distribution of the target features in each category is determined. After obtaining multiple category probabilities, they need to be sorted in descending order for subsequent selection of the category with the highest probability as the classification result. The sorted category probabilities and their corresponding category labels are output as the sorting result. In the sorting result, the category probability ranked first usually indicates that this category is most likely to be the true category of the target object. Therefore, this category probability can be used as the target category probability for subsequent processing. Finally, the target label corresponding to the target category probability is output as the final classification result.This label represents the material category of the target object (plastic bottle) in the image.

[0048] For example, after determining the target feature, calculate the probabilities between the target feature and 5 categories in sequence. If the probability corresponding to category A is 0.1, the probability corresponding to category B is 0.1, the probability corresponding to category C is 0.1, the probability corresponding to category D is 0.6, and the probability corresponding to category E is 0.1, and the total probability of the 5 categories is determined to be 1. At this time, select the label corresponding to category D with the highest probability for outputting the classification result.

[0049] S104: Determine the material category according to the classification result, and determine the stacking area corresponding to the material category.

[0050] In the above S104, determining the material category according to the classification result specifically includes: determining the target label according to the classification result; determining the material category based on the target label according to the preset mapping relationship, where one target label corresponds to one material category. Specifically, it is the classification result obtained through the classification processing of the fully connected layer. This classification result is usually one or more category probabilities, and each category probability corresponds to a possible material category. From the classification result, select the category with the highest probability as the target label. This is because in most cases, the category with the highest probability is most likely to be the true material category of the target object. It is necessary to establish a mapping relationship in advance, that is, the correspondence between the target label and the material category. This mapping relationship is usually a lookup table or a database, which stores the material category information corresponding to each target label. Since the materials in this application are set to 5 categories, assign a label to each category in sequence. Therefore, the classification result is the label corresponding to the material category. After determining the target label, use the preset mapping relationship to find the corresponding material category. This is usually a simple lookup process, by comparing the target label with the entries in the mapping relationship to find the matching material category. After determining the material category corresponding to the first sample, according to the preset correspondence between the material category and the stacking area, find and determine the stacking area corresponding to this material category, and each material category corresponds to a stacking area.

[0051] In addition, after determining the stacking area, when controlling the robotic arm to grasp the first sample, in order to accurately grasp the first sample, it is also necessary to determine the grasping coordinates corresponding to the first sample so that the robotic arm can grasp the first sample according to the grasping coordinates. Specifically, it includes: inputting the first image into a preset rotated rectangle detection model for processing to obtain vertex coordinates; calculating the vertex coordinates to obtain the target center coordinates; determining the target shape corresponding to the first sample according to the first image; judging whether the target shape is a centrally symmetric shape; when the target shape is a centrally symmetric shape, confirm to integrate the target center coordinates and the stacking area to obtain a grasping instruction; send the grasping instruction to the robotic arm so as to control the robotic arm to grasp the first sample according to the target center coordinates. Specifically, after obtaining the first image, the obtained first image is input into a preset rotated rectangle detection model. This model is usually based on deep learning technologies such as YOLOv11 and can identify and locate the rotated rectangle in the image. The rotated rectangle can more accurately describe the inclination and rotation state of the target compared with the traditional horizontal rectangle. After the model processes the image, it will output the vertex coordinates of the rotated rectangle. At this time, the vertex coordinates include four, such as the coordinates corresponding to the four top corners of the first image. These coordinates define the position and shape of the target sample in the image. According to the four vertex coordinates of the rotated rectangle, the center point coordinates can be obtained through geometric calculations. This usually involves calculating the average value of the vertex coordinates or performing specific geometric transformations. After calculation, the obtained target center coordinates will be used for subsequent robotic arm grasping operations. Since the shapes of some plastic bottles in renewable resource plastic bottles are irregular, it may not be possible to grasp the first sample through the target center coordinates. Therefore, it is necessary to obtain the shape of the first sample in the first image in order to judge the shape. Further analyze the first image to identify the shape characteristics of the target sample. Determine the shape of the target sample through image processing technologies (such as image segmentation, feature extraction, etc.). This may involve preprocessing the image, such as denoising, enhancement, etc., to improve the accuracy of shape recognition. According to the characteristics of the target shape, judge whether it is a centrally symmetric shape. This usually involves observing whether the shape is centrally symmetric about a certain point and checking whether it satisfies the condition of completely coinciding after rotating 180 degrees. For shapes with obvious centrally symmetric characteristics such as circles, they can be directly judged as centrally symmetric shapes. For other complex shapes, more complex algorithms or models may be required for judgment. If the target shape is judged to be a centrally symmetric shape, then proceed to the next step. Integrate the target center coordinates with the information of the stacking area. The stacking area is usually a predefined area for placing the samples to be grasped. Integrating the information may involve calculating the relative position between the target center coordinates and the center of the stacking area to determine the grasping path and posture of the robotic arm. Generate specific grasping instructions according to the integrated information. These instructions will guide the robotic arm how to move to the target center position and grasp the sample in an appropriate posture.Send the generated grasping instruction to the control system of the robotic arm. The robotic arm moves to the target center coordinate position according to the received instruction and grasps the first sample in an appropriate posture. When the robotic arm successfully grasps the sample, the entire grasping process is completed. At this time, the robotic arm can move the sample to the designated stacking area or other designated positions.

[0052] Furthermore, when the target shape is not a centrally symmetric shape, confirm to perform a center point compensation algorithm calculation on the target center coordinates to obtain the center of gravity coordinates; integrate the center of gravity coordinates and the stacking area to obtain a grasping instruction, and send the grasping instruction to the robotic arm so that the robotic arm can grasp the first sample according to the center of gravity coordinates. Specifically, use image processing technology to carefully analyze the target shape in the first image. Identify the features of the target shape, including edges, contours, symmetry, etc. According to the features of the target shape, determine whether it is a non-centrally symmetric shape. A non-centrally symmetric shape refers to a shape that cannot be made to coincide with itself completely through any rotation or translation operation. The center point compensation algorithm is an algorithm used to adjust the target center coordinates to be closer to its actual center of gravity. This algorithm is based on the features of the target shape and determines the position of the center of gravity by calculating the mass distribution or pixel distribution of the shape. Extract the edge or contour features of the target shape from the first image. According to the features of the shape, calculate the mass distribution of each pixel or region (in image processing, the mass distribution can be estimated based on pixel values or specific algorithms). Use the mass distribution information to calculate the center of gravity coordinates of the target through weighted average or other mathematical methods. Compare the calculated center of gravity coordinates with the initial target center coordinates, calculate the compensation amount, and apply it to the target center coordinates to obtain the adjusted center of gravity coordinates. Obtain the position information of the stacking area, including coordinates, direction, size, etc. Integrate the adjusted center of gravity coordinates with the position information of the stacking area. Calculate the best path or posture from the center of gravity coordinates to the stacking area. Generate specific grasping instructions according to the integrated information. The grasping instructions should include parameters such as the movement path of the robotic arm, the grasping posture, and the grasping force. Send the generated grasping instruction to the control system of the robotic arm. The robotic arm moves to the center of gravity coordinate position according to the received instruction. Grasp the sample to be detected with an appropriate posture and force. When the robotic arm successfully grasps the sample, the entire grasping process is completed.

[0053] S105: Send a grasping instruction to the robotic arm to control the robotic arm to grasp the first sample to the stacking area.

[0054] In the above S105, according to the determined stacking area, a grasping instruction for the robotic arm is generated, including information such as the grasping position, grasping force, and movement path. At this time, the grasping position is determined according to the shape of the first sample. If the shape of the first sample is a centrosymmetric shape, the center coordinates are used as the grasping position for output. For a plastic bottle with a centrosymmetric shape, grasping its center position can ensure the stability and balance of the grasping process, reducing the risk of grasping failure or plastic bottle damage caused by improper grasping position. If the shape of the first sample is not a centrosymmetric shape, the center of gravity coordinates are used as the grasping position for output. According to the center of gravity coordinates, the actual weight distribution of the plastic bottle can be more accurately reflected, thus ensuring the stability during grasping. Regardless of the shape of the plastic bottle, the sorting and processing platform can ensure the accuracy and stability of grasping by calculating the center of gravity coordinates. The grasping instruction is sent to the control system of the robotic arm. The robotic arm moves to the position where the plastic bottle (the first sample) is located according to the received instruction, performs a grasping operation, and then moves the plastic bottle to the designated stacking area for stacking. The entire process realizes the automated operation from image acquisition, image processing, classification and recognition to robotic arm grasping and stacking, greatly improving the classification efficiency and accuracy of renewable resource plastic bottles.

[0055] The embodiment of the present application also provides an AI-based recyclable sorting and processing device. Figure 2 FIG. is a schematic structural diagram of an AI-based recyclable sorting and processing device provided by the embodiment of the present application. Refer to Figure 2 The device is a sorting and processing platform, and the sorting and processing platform includes an acquisition unit 201, a processing unit 202, and a sending unit 203.

[0056] The acquisition unit 201 acquires a first image corresponding to the first sample, and the first sample is any renewable resource plastic bottle in the recyclables.

[0057] The processing unit 202 processes the first image to obtain a second image; inputs the second image into a preset classification and recognition model for processing to obtain a classification result; determines the material category according to the classification result, and determines the stacking area corresponding to the material category, where the stacking area is the area for stacking renewable resource plastic bottles corresponding to the material category.

[0058] The sending unit 203 sends a grasping instruction to the robotic arm to control the robotic arm to grasp the first sample to the stacking area.

[0059] In a possible implementation, the acquisition unit 201 is configured to receive a sorting request sent by a user device, scan a conveyor belt according to the sorting request to obtain a third image, where the conveyor belt is used to convey renewable resource plastic bottles to be detected; the processing unit 202 is configured to perform polygon annotation on the target contour of the second sample in the third image to obtain a fourth image, where the second sample is any renewable resource plastic bottle that appears in the third image, and the target contour is the outer contour corresponding to the second sample; if it is determined that the fourth image is in a completely annotated state, the fourth image is output as the first image.

[0060] In a possible implementation, the processing unit 202 is configured to input the first image into a preset rotated rectangle detection model for processing to obtain vertex coordinates; calculate the vertex coordinates to obtain target center coordinates; determine the target outer shape corresponding to the first sample according to the first image; determine whether the target outer shape is a centrally symmetric shape; when the target outer shape is a centrally symmetric shape, confirm to integrate the target center coordinates and the stacking area to obtain a grasping instruction; the sending unit 203 is configured to send the grasping instruction to a robotic arm so as to control the robotic arm to grasp the first sample according to the target center coordinates.

[0061] In a possible implementation, when the target outer shape is not a centrally symmetric shape, the processing unit 202 is configured to confirm to perform a center point compensation algorithm calculation on the target center coordinates to obtain center of gravity coordinates; the sending unit 203 is configured to integrate the center of gravity coordinates and the stacking area to obtain a grasping instruction, and send the grasping instruction to the robotic arm so that the robotic arm grasps the first sample according to the center of gravity coordinates.

[0062] In a possible implementation, the processing unit 202 is configured to scale the first image to obtain a scaled first image; perform pixel value filling on the scaled first image to obtain a second image.

[0063] In a possible implementation, the processing unit 202 is configured to extract features from the second image to obtain target features; input the target features into a fully connected layer for classification processing to obtain multiple class probabilities, where one class probability corresponds to one material; sort the multiple class probabilities from largest to smallest to obtain a sorting result; the acquisition unit 201 is configured to acquire a target class probability, where the target class probability is the class probability corresponding to the top position in the sorting result; the processing unit 202 is configured to output the target label corresponding to the target class probability as a classification result.

[0064] In a possible implementation, the processing unit 202 is configured to determine a target label according to the classification result; determine a material category based on the target label according to a preset mapping relationship, where one target label corresponds to one material category.

[0065] It should be noted that when the device provided in the above embodiments realizes its functions, only the division of the above functional modules is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept. For the specific implementation process, please refer to the method embodiments and will not be elaborated here.

[0066] This application also discloses an electronic device. Referring to Figure 3 , Figure 3 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of this application. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 302, and at least one communication bus 305.

[0067] Among them, the communication bus 305 is used to realize the connection and communication between these components.

[0068] Among them, the user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may further include a standard wired interface and a wireless interface.

[0069] Among them, the network interface 304 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0070] Among them, the processor 301 may include one or more processing cores. The processor 301 connects various parts within the entire server through various interfaces and lines, and executes various functions of the server and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 302, and by calling the data stored in the memory 302. Optionally, the processor 301 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 301 may integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application requests, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301 and may be implemented separately by a single chip.

[0071] Among them, the memory 302 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory 302 includes a non-transitory computer-readable storage medium. The memory 302 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 302 may include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area can store the data involved in the above-mentioned method embodiments. Optionally, the memory 302 may also be at least one storage device located far from the aforementioned processor 301.

[0072] As Figure 3 shown, the memory 302, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for AI-based recyclable sorting processing.

[0073] In Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an interface for the user to input and obtain the data input by the user; and the processor 301 can be used to call the application program stored in the memory 302 for sorting and processing recyclables based on AI. When executed by one or more processors, the electronic device is caused to execute one or more of the methods as described in the foregoing embodiments.

[0074] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0075] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0076] In the several embodiments provided by this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0077] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0078] In addition, in each embodiment of this application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0079] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned memory includes various media that can store program codes, such as USB flash drives, mobile hard disks, magnetic disks, or optical discs.

[0080] The above are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited thereby. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. After considering the specification and the disclosure of the practical truth, those skilled in the art will easily think of other implementation schemes of the present disclosure. The present application aims to cover any variations, uses, or adaptive changes of the present disclosure, and these variations, uses, or adaptive changes follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure.

Claims

1. A method for sorting and processing recyclables based on AI, characterized in that: Applied to a sorting and processing platform, the method comprises: Acquire a first image corresponding to a first sample, where the first sample is any one plastic bottle made of renewable resources among recyclables; Processing the first image to obtain a second image; Inputting the second image into a preset classification recognition model for processing to obtain a classification result; Determine the material category according to the classification result, and determine the stacking area corresponding to the material category, the stacking area is an area for stacking the renewable resource plastic bottles corresponding to the material category; A grabbing instruction is sent to the robotic arm so as to control the robotic arm to grab the first sample to the stacking area.

2. The method according to claim 1, characterized in that Before acquiring the first image corresponding to the first sample, the method further includes: receiving a sorting request sent by a user device, and scanning a conveyor belt according to the sorting request to obtain a third image, wherein the conveyor belt is used to convey the renewable resource plastic bottles to be inspected; Performing polygon annotation on the target contour of the second sample in the third image to obtain a fourth image, wherein the second sample is any one of the renewable resource plastic bottles appearing in the third image, and the target contour is the outer contour corresponding to the second sample; If it is determined that the fourth image is in a complete annotation state, the fourth image is output as the first image.

3. The method according to claim 1, characterized in that Before sending the grasping instruction to the robot arm, the method further includes: Inputting the first image into a preset rotating rectangular frame detection model for processing to obtain vertex coordinates; Calculating the vertex coordinates to obtain the target center coordinates; determining a target shape corresponding to the first sample according to the first image; Determining whether the target shape is a centrally symmetrical shape; When the target shape is the central symmetrical shape, confirming that the target center coordinates and the stacking area are integrated to obtain the grabbing instruction; The grasping instruction is sent to the robotic arm so as to control the robotic arm to grasp the first sample according to the target center coordinates.

4. The method according to claim 3, characterized in that: After determining whether the target shape is a centrally symmetrical shape, the method further includes: When the target shape is not the centrally symmetrical shape, confirming that the target center coordinates are calculated using a center point compensation algorithm to obtain the center of gravity coordinates; The center-of-gravity coordinates and the stacking area are integrated to obtain the grabbing instruction, and the grabbing instruction is sent to the robotic arm so that the robotic arm grabs the first sample according to the center-of-gravity coordinates.

5. The method according to claim 1, characterized in that The processing of the first image to obtain the second image specifically includes: Scaling the first image to obtain a scaled first image; The scaled first image is filled with pixel values ​​to obtain the second image.

6. The method according to claim 1, characterized in that The inputting the second image into a preset classification recognition model for processing to obtain a classification result specifically includes: performing feature extraction on the second image to obtain target features; Input the target features into a fully connected layer for classification processing to obtain multiple category probabilities, where each category probability corresponds to one material; Sorting the plurality of category probabilities from largest to smallest to obtain a sorting result; Obtaining target category probability, where the target category probability is the category probability corresponding to the first one in the sorting result; The target label corresponding to the target category probability is output as the classification result.

7. The method according to claim 6, characterized in that Determining the material category according to the classification result specifically includes: Determine the target label according to the classification result; The material category is determined based on the target tag according to a preset mapping relationship, and one target tag corresponds to one material category.

8. An AI-based recyclables sorting and processing device, characterized in that: The device is a sorting and processing platform, comprising an acquisition unit (201), a processing unit (202) and a sending unit (203). The acquisition unit (201) acquires a first image corresponding to a first sample, wherein the first sample is any one plastic bottle made of renewable resources among recyclable materials; The processing unit (202) processes the first image to obtain a second image; inputs the second image into a preset classification recognition model for processing to obtain a classification result; determines a material category according to the classification result, and determines a stacking area corresponding to the material category, the stacking area being an area for stacking renewable resource plastic bottles corresponding to the material category; The sending unit (203) sends a grabbing instruction to the robotic arm, so as to control the robotic arm to grab the first sample to the stacking area.

9. An electronic device, characterized in that: The electronic device (300) comprises a processor (301), a memory (302), a user interface (303) and a network interface (304), wherein the memory (302) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (302) so that the electronic device (300) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is performed.

Images