Intelligent sorting system and method based on machine vision

Abstract

The application provides an intelligent sorting system and method based on machine vision, wherein the method comprises: collecting video stream information of an object set to be sorted, and judging whether the object set in the video stream information enters a sorting area; when the object set enters the sorting area, determining a sorting level of the object set, and sending a sorting instruction to each sorting entrance defined by the sorting level; splitting each object in the object set, and pushing each object to a corresponding sorting entrance according to the object type obtained by splitting; collecting an object image of a target object in the sorting entrance, and judging whether the target object has a defect; and if the target object has no defect, driving a conveying belt at the sorting entrance to place the target object in a preset packaging bag. The technical scheme provided by the application can be compatible with an existing logistics sorting production line while reducing costs.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to an intelligent sorting system and method based on machine vision. Background Technology

[0002] In the current logistics field, cargo sorting is a very labor-intensive process. Currently, intelligent sorting can be achieved through sorting robots, but sorting robots usually require significant investment in research and development and deployment, and they are often incompatible with existing logistics sorting production lines.

[0003] Therefore, there is a need for a low-cost intelligent sorting system that is compatible with existing logistics sorting lines. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent sorting system and method based on machine vision, which can reduce costs and be compatible with existing logistics sorting production lines.

[0005] This invention provides an intelligent sorting system based on machine vision, the system comprising:

[0006] The video stream recognition unit is used to collect video stream information of the set of objects to be sorted and to determine whether the set of objects in the video stream information has entered the sorting area.

[0007] The hierarchical identification unit is used to determine the sorting level of the object set when the object set enters the sorting area, and send sorting instructions to each sorting entrance defined by the sorting level;

[0008] The splitting and pushing unit is used to split each object in the object set and push each object to the corresponding sorting entry according to the type of the split object.

[0009] The defect identification unit is used to acquire an object image of a target object at any sorting inlet, and determine whether the target object has a defect based on the object image. If the target object does not have a defect, the unit drives the conveyor belt at the sorting inlet to place the target object into a preset packaging bag via the conveyor belt.

[0010] In one embodiment, the video stream recognition unit is specifically used to determine the direction of travel of the object set and, based on the direction of travel, identify a buffer area in the sorting area; wherein, if the object set crosses the first boundary of the sorting area, enters the buffer area, and crosses the second boundary of the sorting area from the buffer area, it is determined that the object set has entered the sorting area.

[0011] In one embodiment, the splitting and pushing unit is specifically used to identify the sorting entry corresponding to the object when splitting any object from the object set, plan the transmission path between the sorting entry and the area to be sorted, and push the object to the corresponding sorting entry according to the transmission path.

[0012] In one embodiment, the defect identification unit is specifically used to: extract multiple dimensional features of the target object from the object image and generate quality parameters for each dimensional feature; fuse the multiple dimensional features based on the quality parameters to obtain fused features of the target object; match the fused features with multiple preset base library features to determine target base library features that match the fused features; obtain potential defects corresponding to the target base library features, and identify whether the potential defects exist in the object image.

[0013] The present invention also provides an intelligent sorting method based on machine vision, the method comprising:

[0014] Collect video stream information of the set of objects to be sorted, and determine whether the set of objects in the video stream information has entered the sorting area;

[0015] When the object set enters the sorting area, the sorting level of the object set is determined, and a sorting instruction is sent to each sorting entrance defined by the sorting level.

[0016] The objects in the object set are split, and each object is pushed to the corresponding sorting entry according to the type of the split object.

[0017] For any sorting entrance, an object image of the target object in the sorting entrance is acquired, and the object image is used to determine whether the target object has defects. If the target object does not have defects, the conveyor belt at the sorting entrance is driven to place the target object into a preset packaging bag.

[0018] In one implementation, determining whether the set of objects in the video stream information has entered the sorting area includes:

[0019] The direction of travel of the object set is determined, and a buffer area is identified in the sorting area based on the direction of travel; wherein, if the object set crosses the first boundary of the sorting area, enters the buffer area, and crosses the second boundary of the sorting area from the buffer area, it is determined that the object set has entered the sorting area.

[0020] In one implementation, pushing each object to its corresponding sorting inlet includes:

[0021] When any object is separated from the object set, the sorting entry corresponding to the object is identified, and a transmission path between the sorting entry and the sorting area is planned. The object is then pushed to the corresponding sorting entry according to the transmission path.

[0022] In one implementation, determining whether the target object has defects based on the object image includes:

[0023] Extract multiple dimensional features of the target object from the object image and generate quality parameters for each dimensional feature;

[0024] Based on the quality parameters, the multiple dimensional features are fused to obtain the fused features of the target object;

[0025] The fused feature is matched with multiple preset base database features to determine the target base database feature that matches the fused feature;

[0026] Obtain the potential defects corresponding to the target base features, and identify whether the potential defects exist in the object image.

[0027] In one embodiment, the method further includes:

[0028] Step 1: Let the conveyor speed at a sorting entrance be V, and the standard length of the sorted object be l. Then, within a unit time T, the theoretical maximum sorting capacity at this sorting entrance is:

[0029]

[0030] Where S T This represents the theoretical maximum sorting capacity of the sorting entrance within a unit time T. Indicates to The value is rounded down;

[0031] Step 2: Let V be the actual operating speed of the conveyor belt within a unit time T observed at the sorting entrance. s The actual sorting volume at this sorting inlet is S. s Then the congestion coefficient of the sorting entrance is:

[0032]

[0033] Where k is the congestion coefficient;

[0034] Step 3: Based on the results of Step 1 and Step 2, calculate the congestion index of the sorting entrance. The calculation formula is as follows:

[0035]

[0036] Where z is the congestion index of the sorting entrance. When z is greater than 0.8, it indicates that the sorting entrance has started to be congested and intervention is needed to relieve the congestion. When z is greater than 0.9, it indicates that the sorting entrance is already congested and necessary intervention is needed.

[0037] The technical solution provided by this invention enables intelligent sorting by deploying image acquisition devices on existing logistics sorting lines. When a collection of objects enters the sorting area, they can be separated according to sorting levels and sent to the corresponding sorting entrances. At the sorting entrances, machine vision technology can be used to control the quality of the objects, placing those that meet the quality inspection criteria into packaging bags. This not only completes the intelligent sorting process but also ensures quality control. Therefore, by using machine vision, there is no need to redeploy the logistics sorting line, saving costs and improving sorting efficiency. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0039] Figure 1 This is a schematic diagram of the structure of an intelligent sorting system based on machine vision provided by the present invention.

[0040] Figure 2 This is a schematic diagram illustrating the steps of an intelligent sorting method based on machine vision provided by the present invention. Detailed Implementation

[0041] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, it should be noted that, for ease of description, only the parts relevant to this application are shown in the accompanying drawings, not the entire structure. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0042] The terms “comprising” and “having”, and any variations thereof, used in this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0043] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0044] Please see Figure 1 As shown, one embodiment of this application provides an intelligent sorting system based on machine vision, the system comprising:

[0045] The video stream recognition unit is used to collect video stream information of the set of objects to be sorted and to determine whether the set of objects in the video stream information has entered the sorting area.

[0046] The hierarchical identification unit is used to determine the sorting level of the object set when the object set enters the sorting area, and send sorting instructions to each sorting entrance defined by the sorting level;

[0047] The splitting and pushing unit is used to split each object in the object set and push each object to the corresponding sorting entry according to the type of the split object.

[0048] The defect identification unit is used to acquire an object image of a target object at any sorting inlet, and determine whether the target object has a defect based on the object image. If the target object does not have a defect, the unit drives the conveyor belt at the sorting inlet to place the target object into a preset packaging bag via the conveyor belt.

[0049] At the sorting entrance, some items may be too large and block the entrance, causing congestion. Foreign objects may also be stuck in the entrance, leading to blockage. If blockages are not detected promptly, the accumulated items may damage the system and jeopardize operational safety. To promptly detect blockages at the sorting entrance, the following algorithm is used:

[0050] Step 1: Let the conveyor speed at a sorting entrance be V, and the standard length of the sorted object be l. Then, within a unit time T, the theoretical maximum sorting capacity at this sorting entrance is:

[0051]

[0052] Where S T This represents the theoretical maximum sorting capacity of the sorting entrance within a unit time T. Indicates to The value is rounded down.

[0053] Step 2: Let V be the actual operating speed of the conveyor belt within a unit time T observed at the sorting entrance. s The actual sorting volume at this sorting inlet is S. s Then the congestion coefficient of the sorting entrance is:

[0054]

[0055] Where k is the congestion coefficient.

[0056] Step 3: Based on the results of Step 1 and Step 2, calculate the congestion index of the sorting entrance. The calculation formula is as follows:

[0057]

[0058] Where z is the congestion index of the sorting entrance. When z is greater than 0.8, it indicates that the sorting entrance has started to be congested and may need to be intervened to relieve the congestion. When z is greater than 0.9, it indicates that the sorting entrance is already quite congested and necessary intervention is required.

[0059] The algorithm accurately calculates the congestion level of the conveyor belt at the sorting entrance based on its operation and the sorting volume per unit time. It then formulates different strategies based on the degree of congestion to prevent the sorting entrance from becoming blocked, effectively preventing the accumulation of sorting objects from damaging the system and endangering operational safety.

[0060] In one embodiment, the video stream recognition unit is specifically used to determine the direction of travel of the object set and, based on the direction of travel, identify a buffer area in the sorting area; wherein, if the object set crosses the first boundary of the sorting area, enters the buffer area, and crosses the second boundary of the sorting area from the buffer area, it is determined that the object set has entered the sorting area.

[0061] The buffer area can be part of the sorting area. Different buffer areas can be determined based on the direction the object set travels. Generally, the object set passes through the buffer area first and then enters other areas of the sorting area. Therefore, the buffer area should be set facing the direction the object set is traveling. The object set crosses the first boundary, enters the buffer area, and then leaves the buffer area, crossing the second boundary, at which point it has truly entered the sorting area. This design is intended to address situations where the object set may backtrack during transport. For example, the object set might backtrack immediately after crossing the first boundary. In such cases, determining whether to trigger the subsequent intelligent sorting process solely based on whether the first boundary has been crossed would lead to false triggers. The intelligent sorting process is only considered necessary after the object set has passed through the buffer area and crossed the second boundary, thus avoiding repeated false triggers.

[0062] The sorting level can represent the order in which each object is sent into the sorting entrance. The closer the sorting entrance is to the sorting area, the higher the sorting level it can correspond to, and the earlier the corresponding object will be sorted out.

[0063] In one embodiment, the splitting and pushing unit is specifically used to identify the sorting entry corresponding to the object when splitting any object from the object set, plan the transmission path between the sorting entry and the area to be sorted, and push the object to the corresponding sorting entry according to the transmission path.

[0064] In one embodiment, the defect identification unit is specifically used to: extract multiple dimensional features of the target object from the object image and generate quality parameters for each dimensional feature; fuse the multiple dimensional features based on the quality parameters to obtain fused features of the target object; match the fused features with multiple preset base library features to determine target base library features that match the fused features; obtain potential defects corresponding to the target base library features, and identify whether the potential defects exist in the object image.

[0065] In this embodiment, the multiple dimensional features of the target object can describe its characteristics from different perspectives, such as shape, material, and color. Since different shooting environments result in varying shooting effects for different dimensional features, corresponding quality parameters can be determined based on these differences. Higher shooting effects correspond to higher quality parameters. Subsequently, the quality parameters can be used as weights for the dimensional features, resulting in a weighted summation to obtain the fused features. The preset database features can be standard features collected under favorable shooting conditions and angles. By calculating the similarity between the fused features and the database features, matching target database features can be obtained. These target database features can correspond to a specific object model, and potential defects typically associated with that model can be pre-collected. By identifying these potential defects, it can be determined whether the target object has defects.

[0066] Please see Figure 2 The present invention also provides an intelligent sorting method based on machine vision, the method comprising:

[0067] S1: Collect video stream information of the set of objects to be sorted, and determine whether the set of objects in the video stream information has entered the sorting area;

[0068] S2: When the object set enters the sorting area, determine the sorting level of the object set and send sorting instructions to each sorting entrance defined by the sorting level;

[0069] S3: Split each object in the object set, and push each object to the corresponding sorting entry according to the type of the split object;

[0070] S4: For any sorting entrance, acquire an object image of the target object in the sorting entrance, and determine whether the target object has defects based on the object image. If the target object does not have defects, drive the conveyor belt at the sorting entrance to place the target object into a preset packaging bag via the conveyor belt.

[0071] In one implementation, determining whether the set of objects in the video stream information has entered the sorting area includes:

[0072] The direction of travel of the object set is determined, and a buffer area is identified in the sorting area based on the direction of travel; wherein, if the object set crosses the first boundary of the sorting area, enters the buffer area, and crosses the second boundary of the sorting area from the buffer area, it is determined that the object set has entered the sorting area.

[0073] In one implementation, pushing each object to its corresponding sorting inlet includes:

[0074] When any object is separated from the object set, the sorting entry corresponding to the object is identified, and a transmission path between the sorting entry and the sorting area is planned. The object is then pushed to the corresponding sorting entry according to the transmission path.

[0075] In one implementation, determining whether the target object has defects based on the object image includes:

[0076] Extract multiple dimensional features of the target object from the object image and generate quality parameters for each dimensional feature;

[0077] Based on the quality parameters, the multiple dimensional features are fused to obtain the fused features of the target object;

[0078] The fused feature is matched with multiple preset base database features to determine the target base database feature that matches the fused feature;

[0079] Obtain the potential defects corresponding to the target base features, and identify whether the potential defects exist in the object image.

[0080] The technical solution provided by this invention enables intelligent sorting by deploying image acquisition devices on existing logistics sorting lines. When a collection of objects enters the sorting area, they can be separated according to sorting levels and sent to the corresponding sorting entrances. At the sorting entrances, machine vision technology can be used to control the quality of the objects, placing those that meet the quality inspection criteria into packaging bags. This not only completes the intelligent sorting process but also ensures quality control. Therefore, by using machine vision, there is no need to redeploy the logistics sorting line, saving costs and improving sorting efficiency.

[0081] The above is only one specific embodiment of the present invention, and any improvements made based on the concept of the present invention shall be considered within the scope of protection of the present invention.

Claims

1. A machine vision-based intelligent sorting system, characterized in that, The system includes: The video stream recognition unit is used to collect video stream information of the set of objects to be sorted and to determine whether the set of objects in the video stream information has entered the sorting area. The hierarchical identification unit is used to determine the sorting level of the object set when the object set enters the sorting area, and send sorting instructions to each sorting entrance defined by the sorting level; The splitting and pushing unit is used to split each object in the object set and push each object to the corresponding sorting entry according to the type of the split object. The defect identification unit is used to acquire an object image of a target object in any sorting inlet, and determine whether the target object has a defect based on the object image. If the target object does not have a defect, the unit drives the conveyor belt at the sorting inlet to place the target object into a preset packaging bag via the conveyor belt. During the sorting process, the following methods can be used to determine if there is congestion at the sorting entrance: Step 1: Let the conveyor speed at a sorting entrance be V, and the standard length of the sorted object be... Therefore, the theoretical maximum sorting capacity of this sorting entrance within a unit time T is: in This represents the theoretical maximum sorting capacity of the sorting entrance within a unit time T. Indicates to The value is rounded down; Step 2: Let the actual operating speed of the conveyor belt be within a unit time T observed at the sorting entrance. The actual sorting volume at this sorting entrance is Then the congestion coefficient of the sorting entrance is: Where k is the congestion coefficient; Step 3: Based on the results of Step 1 and Step 2, calculate the congestion index of the sorting entrance. The calculation formula is as follows: in The congestion index of the sorting entrance is z. When z is greater than 0.8, it indicates that the sorting entrance has started to be congested and intervention is needed to relieve the congestion. When z is greater than 0.9, it indicates that the sorting entrance is already congested and necessary intervention is needed.

2. The system according to claim 1, characterized in that, The video stream recognition unit is specifically used to determine the direction of travel of the object set and, based on the direction of travel, identify a buffer area in the sorting area; wherein, if the object set crosses the first boundary of the sorting area, enters the buffer area, and crosses the second boundary of the sorting area from the buffer area, it is determined that the object set has entered the sorting area.

3. The system according to claim 1, characterized in that, The splitting and pushing unit is specifically used to identify the sorting entry corresponding to the object when splitting any object from the object set, plan the transmission path between the sorting entry and the area to be sorted, and push the object to the corresponding sorting entry according to the transmission path.

4. The system according to claim 1, characterized in that, The defect identification unit is specifically used to: extract multiple dimensional features of the target object from the object image and generate quality parameters for each dimensional feature; fuse the multiple dimensional features based on the quality parameters to obtain the fused features of the target object; and match the fused features with multiple preset base database features to determine the target base database features that match the fused features. Obtain the potential defects corresponding to the target base features, and identify whether the potential defects exist in the object image.

5. A machine vision-based intelligent sorting method, employing the system described in any one of claims 1-4, characterized in that, The method includes: Collect video stream information of the set of objects to be sorted, and determine whether the set of objects in the video stream information has entered the sorting area; When the object set enters the sorting area, the sorting level of the object set is determined, and a sorting instruction is sent to each sorting entrance defined by the sorting level. The objects in the object set are split, and each object is pushed to the corresponding sorting entry according to the type of the split object. For any sorting entrance, an object image of the target object in the sorting entrance is acquired, and the object image is used to determine whether the target object has defects. If the target object does not have defects, the conveyor belt at the sorting entrance is driven to place the target object into a preset packaging bag.

6. The method according to claim 5, characterized in that, Determining whether the set of objects in the video stream information has entered the sorting area includes: The direction of travel of the object set is determined, and a buffer area is identified in the sorting area based on the direction of travel; wherein, if the object set crosses the first boundary of the sorting area, enters the buffer area, and crosses the second boundary of the sorting area from the buffer area, it is determined that the object set has entered the sorting area.

7. The method according to claim 5, characterized in that, Pushing each object to its corresponding sorting entry point includes: When any object is separated from the object set, the sorting entry corresponding to the object is identified, and a transmission path between the sorting entry and the sorting area is planned. The object is then pushed to the corresponding sorting entry according to the transmission path.

8. The method according to claim 5, characterized in that, Determining whether the target object has defects based on the object image includes: Extract multiple dimensional features of the target object from the object image and generate quality parameters for each dimensional feature; Based on the quality parameters, the multiple dimensional features are fused to obtain the fused features of the target object; The fused feature is matched with multiple preset base database features to determine the target base database feature that matches the fused feature; Obtain the potential defects corresponding to the target base features, and identify whether the potential defects exist in the object image.

Images