A vision intelligence-based packaging material intelligent sorting method and system

Abstract

The application relates to the technical field of material sorting, and discloses a packaging material intelligent sorting method and system based on visual intelligence. The method comprises the following steps: performing an initial classification operation, performing a physical sorting operation after determining category information of the article through an image comparison operation, determining at least one classification detection article, and performing reclassification verification by taking the appearance image of the classification detection article as a verification sample; when the verification fails, performing a sorting termination procedure; the application can improve long-term sorting accuracy and robustness in response to dynamic changes in materials by learning from sorting errors and updating classification standards; and the verification resource can be concentrated on the classification detection articles by judging that the articles are more likely to cause sorting errors based on sorting time consumption abnormalities, so that the verification efficiency is improved.

Description

Technical Field

[0001] This invention belongs to the field of material sorting technology, specifically relating to a visual intelligence-based intelligent sorting method and system for packaging materials. Background Technology

[0002] Automated material handling systems are now widely used in modern warehousing and production environments. In the management of packaging materials, automated sorting can ensure production line continuity and optimize inventory management. Currently, intelligent sorting technology based on machine vision is widely used in high-throughput material classification.

[0003] Existing vision-based packaging material sorting technology completes classification through one-time image recognition and database matching. This can easily lead to confusion between packaging materials that are similar in appearance, have subtle features, or have printing defects, resulting in false alarms and missed alarms. Furthermore, the existing technology uses high-resolution image acquisition equipment and complex recognition models to improve the accuracy of the initial recognition, which increases the time delay of data processing, resulting in low sorting efficiency and directly affecting the overall throughput of the sorting line.

[0004] In view of this, this application provides a method and system for intelligent sorting of packaging materials based on visual intelligence. Summary of the Invention

[0005] The purpose of this invention is to provide a visual intelligence-based intelligent sorting method and system for packaging materials, in order to solve the problems of easily confused packaging materials being misjudged in the prior art, and the slow verification time.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A visual intelligence-based intelligent sorting method for packaging materials includes the following steps: Perform initial sorting operations on multiple items that have entered the sorting area; After the initial classification operation is completed, in response to the generated classification end signal, a closed-loop classification verification operation is performed; The initial classification operation for multiple items to be sorted entering the sorting area includes: acquiring an appearance image of each item to be sorted, setting any one of the appearance images as a reference image, determining the category information of other items to be sorted that are different from the corresponding items in the reference image through an image comparison operation, and performing physical sorting operations on the other items to be sorted. The closed-loop classification verification operation includes: identifying at least one classification detection item from the items that have completed physical sorting; using the appearance image of the classification detection item as a verification sample to perform reclassification verification on the classification detection item; and executing the sorting termination procedure when the result of the reclassification verification is verification failure. The sorting termination procedure includes: terminating the sorting operation, outputting an alarm prompt, and creating a new reference image based on the appearance image of the sorted and detected items for subsequent sorting tasks.

[0007] Preferably, at least one item for classification and inspection is identified from the items that have undergone physical sorting, including: The system acquires the classification times of multiple items that have completed physical sorting operations within a preset time interval; sets the first classification time after sorting as the reference time; when the difference between the classification time of any subsequent item and the reference time is greater than a preset difference threshold, the item corresponding to that classification time is identified as a classification detection item.

[0008] Preferably, the preset difference threshold is dynamically determined based on the duration of the preset time interval, and the preset difference threshold is a preset percentage of the duration of the preset time interval.

[0009] Preferably, when the difference between the classification time of any subsequent item and the reference time is not greater than a preset difference threshold, the reference time is updated to the classification time of any subsequent item, and the calculation and judgment of the difference continue until the item to be classified and detected is determined.

[0010] Preferably, the appearance image of the classified inspection item is used as a verification sample to perform reclassification verification on the classified inspection item, including: Extract evaluation parameters from the verification samples; obtain preset verification parameters associated with the category information of the classified and tested items; calculate the absolute difference between the evaluation parameters and the preset verification parameters; compare the absolute difference with the preset verification threshold to determine whether the reclassification verification result is successful or unsuccessful.

[0011] A visual intelligence-based intelligent sorting system for packaging materials includes the following modules: Image acquisition module, used to acquire appearance images of items to be sorted; The item classification module is used to call the appearance image to perform the initial classification operation, determine the category information of the items to be sorted and perform the physical sorting operation, and generate a classification end signal after completing the initial classification operation of a batch of items. The closed-loop verification module is used to respond to the classification end signal generated by the item classification module and perform closed-loop classification verification operations.

[0012] Preferably, the closed-loop classification verification operation includes: Identify at least one item for classification inspection from the items that have completed physical sorting, and use the appearance image of the item for classification inspection as a verification sample to perform reclassification verification; if the result of reclassification verification is verification failure, execute the sorting termination procedure.

[0013] Preferably, the execution of the sorting termination procedure includes: The sorting operation is terminated, an alarm is output, and a new baseline image is created based on the appearance image of the sorted and inspected items for subsequent sorting tasks.

[0014] Preferably, at least one item for classification and inspection is identified from the items that have undergone physical sorting, including: The system acquires the classification times of multiple items that have completed physical sorting operations within a preset time interval; sets the first classification time after sorting as the reference time; when the difference between the classification time of any subsequent item and the reference time is greater than a preset difference threshold, the item corresponding to that classification time is identified as a classification detection item.

[0015] Preferably, at least one item for classification inspection is identified from the items that have undergone physical sorting, and a reclassification verification is performed using an image of the item for classification inspection as a verification sample, including: Extract evaluation parameters from the verification samples; obtain preset verification parameters associated with the category information of the classified and tested items; calculate the absolute difference between the evaluation parameters and the preset verification parameters; compare the absolute difference with the preset verification threshold to determine whether the reclassification verification result is successful or unsuccessful. Beneficial effects

[0016] After the initial sorting operation, this invention identifies the items to be sorted and performs a re-sorting verification. When the verification fails, an alarm is output, and the appearance image of the sorted and detected items is newly created as a reference image for subsequent sorting tasks. By learning from sorting errors and updating the sorting standards, the accuracy of long-term sorting in response to dynamic changes in materials is improved.

[0017] This invention employs a two-stage process of sorting and verification. It performs uninterrupted physical sorting operations on a batch of items, and only performs classification verification operations on the identified items after responding to the classification end signal. This avoids interrupting the initial classification operation due to simultaneous verification, ensuring the high speed and continuity of the main sorting process, and improving the overall sorting efficiency and throughput.

[0018] This invention analyzes the sorting time of sorted items and identifies items whose sorting time difference with the previous item is greater than a preset difference threshold as sorting detection items. Based on the judgment that items with abnormally long sorting times are more likely to be sorted incorrectly, verification resources are concentrated on such sorting detection items, thereby improving verification efficiency. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method provided by the present invention; Figure 2 This is a system module diagram provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely for explaining the invention and are not intended to limit the scope of protection of the invention. Example 1

[0021] Please refer to Figure 1 This embodiment provides a visual intelligence-based intelligent sorting method for packaging materials, suitable for the automated identification and classification of different types of packaging materials in warehousing or production processes, including the following steps: In the automated conveyor belt or material storage area waiting for sorting, the image acquisition device captures the items entering the area, marks all items that can be processed independently as items to be sorted, and assigns them a unique number; it acquires the appearance image of all items to be sorted under the current lighting and posture, and associates and stores the appearance image with its number to form a batch of items to be processed. Furthermore, select any appearance image from the appearance images of the current batch of items and set it as the reference image. The first appearance image captured in the current batch of items can be used as the initial reference image. Perform image comparison operation on the reference image and other appearance images in the current batch of items, excluding the reference image, one by one to obtain a similarity score. Image comparison operations quantify the visual similarity between images of different appearances through a numerical processing flow; by using preset image analysis rules, which can be based on scale-invariant feature transformation or accelerated robust feature rules, a set of key feature points and their descriptors are extracted from each appearance image; the entire image or its key regions can also be converted into a set of multi-dimensional numerical sequences that can characterize its core visual content as feature vectors. A similarity score is obtained by calculating numerical distances such as Euclidean distance or angular differences such as cosine similarity between feature point descriptors or feature vectors corresponding to two images.

[0022] Furthermore, a judgment is made based on the similarity score obtained from the image comparison operation; If the similarity score between another appearance image and the reference image is lower than the preset similarity threshold, then the item to be sorted corresponding to it is determined to be a different item from the item to be sorted corresponding to the reference image; the item is then classified into a new category, and its category information is determined. The process of classifying the item into a new category and determining its category information includes: assigning a unique category identifier to the new category, which is associated with a subsequent physical sorting target location such as a specific collection box or conveyor belt branch; and controlling a drive robotic arm or pneumatic push rod to perform a physical sorting operation via control signals to sort the item to the location corresponding to its new category. If the images are determined to be the same item, the image comparison operation continues for the next appearance image until the comparison of all appearance images in the same batch of items is completed.

[0023] Furthermore, after all items to be sorted have completed the physical sorting operation, a sorting end signal is generated to indicate that the processing of this batch of items has ended; To ensure the continuous accuracy of the sorting process and proactively identify potential errors, a dynamic verification process is initiated based on the end-of-sorting signal. Specific items are selected for image acquisition and reclassification verification. From all items that have completed physical sorting operations, at least one item is identified for classification inspection.

[0024] Furthermore, a time-based sampling strategy identifies at least one item for classification and testing, selecting key node items that may represent batch changes in the production or feeding process; Determining at least one category detection item includes: acquiring outbound records of multiple items that have completed physical sorting operations within a preset time interval, wherein the preset time interval can be set to the most recent 30 minutes; the outbound records contain the sorting time corresponding to the completion of the physical sorting operation for each item, sorting these outbound records in ascending order according to the sorting time, and setting the first sorting time in the sorting result as the reference time; Set the classification time of any other item that has completed physical sorting as the test time, and calculate the classification time difference between the test time and the reference time. If the classification time difference is not greater than the preset difference threshold, it means that the items are sorted continuously and stably. Then update the reference time to the current test time and continue to compare it with the classification time of the next item. The essence of this iterative process is to identify the first interruption time in the sorting flow. When the calculated difference in sorting time is greater than the preset difference threshold for the first time, it is considered that an interval that may cause changes in sorting has occurred. The time to be measured is then determined as the verification time. The items that have completed physical sorting operations corresponding to the verification time are finally determined as the sorting detection items. To make the detection mechanism more adaptive, the preset difference threshold is not fixed, but dynamically determined according to the duration of the preset time interval. The preset difference threshold can be set as a percentage of 10% of the duration of the preset time interval. When the preset time interval is 30 minutes, the preset difference threshold is 3 minutes. Through the design of the dynamic threshold, normal time fluctuations can be tolerated when dealing with high-throughput continuous production, while abnormal pauses can be captured when dealing with low-speed or intermittent production, thus improving the effectiveness of the selection of verification points.

[0025] Furthermore, after identifying the items to be classified and tested, an image of the item's appearance is acquired again, and this image is used as a verification sample to review the classification results of the item. According to the preset classification verification rules, the classified detection items are reclassified and verified using the verification samples. The classification verification rules include quantitative evaluation criteria. Using the same image analysis rules as during the initial classification operation, a new set of feature vectors and other evaluation parameters are extracted from the appearance image of the verification sample. The preset verification parameters corresponding to the category to which the item has been assigned are retrieved from the memory. The preset verification parameters are the image feature parameters of the reference image of the category determined by its first member when the category is first established. The standard visual features representing the category are calculated and stored. The absolute difference is obtained by calculating the numerical difference between the Euclidean distance evaluation parameter between two feature vectors and the preset verification parameter. This absolute difference is then compared with the preset verification threshold to determine whether the verification passes. If the calculated absolute difference is greater than the verification threshold, causing the reclassification verification result to fail, it indicates that the characteristics of the item deviate significantly from the baseline characteristics of its category, and a sorting error may have occurred. At this time, the subsequent sorting task is terminated, an alarm prompt is output to notify the management personnel, and the classified and detected item is marked as an item to be reviewed, isolated and manually reviewed or specially handled. Then, the appearance image of the classified and detected item is set as the new baseline image, and new category information is created for it; if an item with similar visual characteristics to the item to be checked appears in a subsequent sorting task, it will be compared and classified according to this newly established baseline; When a potential sorting error is detected, not only is the error marked, but new category benchmarks that may have been missed in the initial sorting operation are also actively defined using the error samples, so as to avoid the recurrence of the same type of error in subsequent sorting tasks and achieve adaptive correction. If the re-classification verification result is successful, that is, the absolute difference is not greater than the verification threshold, it proves that the sorting result is reliable, and a sorting completion signal is output, and the entire sorting and verification process is successfully completed. Example 2

[0026] Please refer to Figure 2 This embodiment provides a vision-based intelligent packaging material sorting system that automatically identifies potential sorting errors or new materials during the sorting process and adaptively updates its classification criteria, thereby achieving accurate and intelligent management of the packaging material sorting process. This system can be deployed on a vision-based intelligent packaging material sorting terminal and includes the following modules: The image acquisition module is used to acquire appearance images of multiple items to be sorted entering the sorting area, which is usually a running conveyor belt. The image acquisition module includes at least one industrial camera set above the conveyor belt. When packaging materials such as cartons, plastic bottles and other packaging materials of different types, sizes or printed patterns pass under the camera, it captures top-view or multi-angle appearance images of all items to be sorted. The acquired images have sufficient resolution to clearly present the key visual features of the items.

[0027] The item classification module is used to perform the initial classification operation and generate trigger signals for the verification process; When the current batch of items begins to be processed, the first appearance image of the item to be sorted captured by the image acquisition module is called and set as the baseline image. The baseline image represents the main material types expected to appear in the current batch and is stored in the system's memory, associated with the initial category information; For other items to be sorted that subsequently enter the sorting area, the image acquisition module is continuously called to capture their appearance images and perform image comparison operations. The newly acquired appearance images are compared with the stored baseline images. The image comparison operation here can be achieved based on feature point matching, template matching, or similarity scores calculated by deep learning models. If the similarity of the comparison result is lower than the preset similarity threshold, it indicates that the current item is different from the item corresponding to the reference image. Then, new category information is determined for it, and physical sorting devices such as sorting baffles or robotic arms are controlled to perform physical sorting operations to guide the item to the corresponding collection area. After completing the initial classification of the preset number of items in the current batch, or after the preset running cycle ends, a classification end signal is generated to initiate the dynamic verification process.

[0028] The closed-loop verification module is used to respond to the classification end signal and perform closed-loop classification verification operation; determine the classification detection items; after receiving the classification end signal, determine at least one classification detection item from the items that have just completed the physical sorting operation; analyze the classification time spent by the item classification module in processing each item within a preset time interval, and set the classification time of the first item sorted in chronological order within the time interval as the reference time. Calculate the difference between the classification time of any subsequent item and the current reference time. When the difference is greater than the preset difference threshold, it indicates that there is an anomaly in the classification process of the item, such as the image comparison operation taking too long. This may be a potential misclassified item or a new material. The items corresponding to the abnormal classification time are identified as the classified detection items. The preset difference threshold can be dynamically determined according to the duration of the preset time interval. Specifically, it can be set to a preset percentage of the duration to adapt to normal fluctuations under different production cycles. If the calculated classification time difference is not greater than the preset difference threshold, it indicates that the classification time is within the normal range. The reference time will be updated to the classification time of the current item, and the calculation and judgment of the classification time difference will continue to be performed on the next item until the item to be classified is found or all items in the entire time interval are traversed. After identifying the items to be classified and tested, their appearance images are used as verification samples, and a more rigorous reclassification verification is performed on them. The reclassification verification process includes: extracting a set of predefined quantitative indicators such as color histograms, texture descriptors, and contour moments from the verification samples as evaluation parameters, and obtaining the standard parameter set of the category associated with the category information assigned to the item from the system database as preset verification parameters. Calculate the absolute difference between the evaluation parameter and the preset verification parameter, and compare the absolute difference with the preset verification threshold. If the absolute difference is less than or equal to the verification threshold, the result of the reclassification verification is considered successful; if the absolute difference is greater than the verification threshold, the verification is considered unsuccessful. When the re-classification verification result is a verification failure, the sorting termination procedure is executed. The sorting termination procedure includes: sending an instruction to the central control system to terminate the operation of the entire sorting line, thereby terminating the sorting operation to prevent the spread of errors; and outputting alarm prompts through human-machine interfaces such as displays or audible and visual alarms to notify on-site operators to intervene. Then, the appearance image of the category detection item that caused the verification failure is saved and created as a new baseline image for subsequent sorting tasks, realizing adaptive correction so that when the same type of item is encountered again, it can be correctly identified as a new category.

[0029] Through a classification-verification-learning process, it performs routine visual sorting tasks, automatically discovers and adapts to new types of packaging materials, and improves sorting accuracy and robustness without human intervention. It is suitable for modern logistics, recycling and manufacturing scenarios with a wide variety of packaging materials and rapid updates.

[0030] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from it. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for intelligent sorting of packaging materials based on visual intelligence, characterized in that, Includes the following steps: Perform initial sorting operations on multiple items that have entered the sorting area; After the initial classification operation is completed, in response to the generated classification end signal, a closed-loop classification verification operation is performed; The initial classification operation for multiple items to be sorted entering the sorting area includes: acquiring an appearance image of each item to be sorted, setting any one of the appearance images as a reference image, determining the category information of other items to be sorted that are different from the corresponding items in the reference image through an image comparison operation, and performing physical sorting operations on the other items to be sorted. The closed-loop classification verification operation includes: identifying at least one classification detection item from the items that have completed physical sorting; using the appearance image of the classification detection item as a verification sample to perform reclassification verification on the classification detection item; and executing the sorting termination procedure when the result of the reclassification verification is verification failure. The sorting termination procedure includes: terminating the sorting operation, outputting an alarm prompt, and creating a new reference image based on the appearance image of the sorted and detected items for subsequent sorting tasks.

2. The intelligent sorting method for packaging materials based on visual intelligence according to claim 1, characterized in that, Identify at least one item for classification inspection from the items that have undergone physical sorting, including: The system acquires the classification times of multiple items that have completed physical sorting operations within a preset time interval; sets the first classification time after sorting as the reference time; when the difference between the classification time of any subsequent item and the reference time is greater than a preset difference threshold, the item corresponding to that classification time is identified as a classification detection item.

3. The intelligent sorting method for packaging materials based on visual intelligence according to claim 2, characterized in that, The preset difference threshold is dynamically determined based on the duration of the preset time interval, and the preset difference threshold is a preset percentage of the duration of the preset time interval.

4. The intelligent sorting method for packaging materials based on visual intelligence according to claim 2, characterized in that: If the difference between the classification time of any subsequent item and the reference time is not greater than the preset difference threshold, the reference time will be updated to the classification time of any subsequent item, and the calculation and judgment of the difference will continue until the item to be classified and detected is determined.

5. The intelligent sorting method for packaging materials based on visual intelligence according to claim 1, characterized in that, Using the appearance images of the classified inspection items as verification samples, a reclassification verification is performed on the classified inspection items, including: Extract evaluation parameters from the verification samples; obtain preset verification parameters associated with the category information of the classified and tested items; calculate the absolute difference between the evaluation parameters and the preset verification parameters; compare the absolute difference with the preset verification threshold to determine whether the reclassification verification result is successful or unsuccessful.

6. A visual intelligence-based intelligent sorting system for packaging materials, characterized in that, Includes the following modules: Image acquisition module, used to acquire appearance images of items to be sorted; The item classification module is used to call the appearance image to perform the initial classification operation, determine the category information of the items to be sorted and perform the physical sorting operation, and generate a classification end signal after completing the initial classification operation of a batch of items. The closed-loop verification module is used to respond to the classification end signal generated by the item classification module and perform closed-loop classification verification operations.

7. The intelligent sorting system for packaging materials based on visual intelligence according to claim 6, characterized in that, The closed-loop classification verification operation includes: Identify at least one item for classification inspection from the items that have completed physical sorting, and use the appearance image of the item for classification inspection as a verification sample to perform reclassification verification; if the result of reclassification verification is verification failure, execute the sorting termination procedure.

8. The intelligent sorting system for packaging materials based on visual intelligence according to claim 7, characterized in that, The sorting termination procedure includes: The sorting operation is terminated, an alarm is output, and a new baseline image is created based on the appearance image of the sorted and inspected items for subsequent sorting tasks.

9. The intelligent sorting system for packaging materials based on visual intelligence according to claim 7, characterized in that, Identify at least one item for classification inspection from the items that have undergone physical sorting, including: The system acquires the classification times of multiple items that have completed physical sorting operations within a preset time interval; sets the first classification time after sorting as the reference time; when the difference between the classification time of any subsequent item and the reference time is greater than a preset difference threshold, the item corresponding to that classification time is identified as a classification detection item.

10. A visual intelligence-based intelligent sorting system for packaging materials according to claim 7, characterized in that, From the items that have completed physical sorting, identify at least one item for classification inspection, and use the appearance image of the item for classification inspection as a verification sample to perform reclassification verification, including: Extract evaluation parameters from the verification samples; obtain preset verification parameters associated with the category information of the classified and tested items; calculate the absolute difference between the evaluation parameters and the preset verification parameters; compare the absolute difference with the preset verification threshold to determine whether the reclassification verification result is successful or unsuccessful.

Images