A vision-based cargo sorting error recognition system and method

Abstract

The present invention discloses a vision-based cargo sorting error recognition system and method; the system includes a misclassification judgment module and an image acquisition module; the image acquisition module is used to collect images of each sorting grid; the misclassification judgment module is used to identify the grid image, determine whether there is cargo in the grid image, and select the position of each cargo. During the sorting process, when a cargo arrives and is sent to the target grid, the sorting machine system server generates a cargo drop-off message and sends the cargo drop-off message to the misclassification judgment module; the present invention communicates the misclassification judgment module with the sorting machine system server and realizes time synchronization through NTP; and then sends the cargo drop-off message to the misclassification judgment module through the sorting machine system server, so that the misclassification judgment module only recognizes the sorting grid image where the drop-off occurs, thereby effectively identifying the sorting error and greatly reducing the image processing pressure of the misclassification judgment module.

Description

Technical Field

[0001] The present invention belongs to the technical field of logistics sorting, and in particular relates to a vision-based cargo sorting error recognition system and method. Background Art

[0002] During the express sorting process, express parcels may be missorted to other grids, resulting in the express being sent to the wrong destination; logistics sorting accuracy is a key assessment indicator for express logistics transportation. The conventional sorting mode uses express cargo scanning and sorting line trolley binding to sort express parcels. The express barcode information is obtained at the loading station, and the barcode information is bound one-to-one to the express trolley controlled by the PLC. After the sorting line trolley arrives at the designated position, the trolley rotates to make the goods slide to the designated grid. If there are missorted items, it cannot be discovered. The missorted express goods can only be discovered after they are dispatched to the destination.

[0003] The use of image recognition algorithms can help to monitor the grid. However, the image recognition algorithm requires a lot of computing power; and the number of grids in the sorting machine, especially the express logistics sorting, is large; conventional servers are difficult to perform real-time image recognition on all grids at the same time, which greatly increases the cost of misidentification of goods sorting.

[0004] In addition, in the target detection task, one target may be detected by multiple detection frames. These detection frames may have different positions and sizes, but they actually represent the same target. In the prior art, this problem is often solved by the non-maximum suppression algorithm NMS. The main purpose of NMS is to remove redundant detection frames and retain only the detection frame with the highest confidence, so as to obtain the final target detection result. However, in the traditional NMS algorithm, when the overlap (IOU) between two detection frames is higher than a certain threshold, one of the detection frames will be removed to reduce duplicate detection. This method is prone to missed detection when dealing with complex scenes such as densely arranged or occluded objects.

[0005] During cargo sorting, if multiple cargoes are on the same sorting cart due to reasons such as cargo rolling, the multiple cargoes dropped into the grid may be close to each other or even partially overlap in the grid. At this time, the non-maximum suppression algorithm NMS can easily identify multiple cargoes as a single cargo, resulting in the omission of sorting errors. Summary of the invention

[0006] The object of the present invention is to provide a vision-based cargo sorting error recognition system and method.

[0007] In a first aspect, the present invention provides a vision-based cargo sorting error recognition system, which includes an error judgment module and an image acquisition module; the image acquisition module is used to collect images of each sorting grid; the error judgment module communicates with the image acquisition module; the error judgment module is used to recognize the grid image, determine whether there are goods in the grid image, and select the location of each cargo.

[0008] During the sorting process, when a cargo arrives at and is sent to the target grid, the sorting machine system server generates a cargo drop message and sends the cargo drop message to the misclassification judgment module; the cargo drop message includes the target grid number and the drop time of the cargo; the misclassification judgment module that receives the cargo drop message extracts the grid time sequence image group of the target grid of the cargo at the drop time; the grid time sequence image group includes multiple grid images collected after the drop time; the first grid image collected is collected after the package is sent out at the drop time; the visual recognition algorithm is used to determine whether there is cargo in the grid image, and whether the package is dropped normally is determined based on whether the cargo in each grid image moves.

[0009] Preferably, the image acquisition module identifies five types of bag dropping, namely normal bag dropping, hanging bag, full bag blocking, wrong classification and floating bag.

[0010] If a cargo is detected in each grid image in the grid time-series image group, and the cargo position in each grid image moves, it is determined that the cargo falls into the grid normally.

[0011] If a cargo is detected in multiple grid images and the cargo position in the multiple grid images does not move, it is determined whether the cargo boundary is connected to the bottom edge of the grid; if the cargo boundary is not connected to the bottom edge of the grid, it is determined that the cargo is hanging; otherwise, it is determined that the cargo is full and blocked.

[0012] If multiple goods are detected in each grid image in any grid time-series image group, it is determined that the target grid has misclassified goods.

[0013] If no goods are detected in any of the grid images in the grid time sequence image group, the grid time sequence image group of the two adjacent grids of the target grid at the grid drop time is retrieved and the goods are identified; if goods are detected in the adjacent grid and there is no corresponding goods drop message in the adjacent grid at the grid drop time, it is determined that the goods are floating; otherwise, it is determined that the goods are misclassified.

[0014] Preferably, it also includes an alarm module; the alarm module includes a grid alarm light; each sorting grid is equipped with a grid alarm light; when any of the following occurs at the sorting grid: hanging bags, full bags, or floating bags, the corresponding grid alarm light lights up.

[0015] Preferably, it also includes a display module; the display module is used to display the sorting errors detected by the identification system; when hanging bags or full and blocked bags are detected, the display module displays real-time video of the corresponding grid.

[0016] Preferably, the misclassification judgment module communicates with the sorting machine system server and performs time synchronization via a network time protocol.

[0017] Preferably, the image acquisition module comprises a plurality of cameras; each camera can capture images of one or more sorting grids corresponding to itself.

[0018] In a second aspect, the present invention provides a method for identifying errors in cargo sorting based on vision, which comprises the following steps:

[0019] Step 1: Continuously collect images of all the slots of the sorting machine.

[0020] Step 2: Whenever a cargo falls into a grid, extract the grid time-series image group corresponding to the target grid of the cargo at the grid-falling moment; the grid time-series image group includes multiple grid images of the target grid collected after the grid-falling moment; the first grid image collected among the multiple grid images is collected after the package delivery time at the grid-falling moment; the package delivery time is the time required for the cargo to enter the sorting grid from the sorting cart.

[0021] Step 3: According to the grid time-series image group collected in step 2, the grid type is determined.

[0022] 3-1. Cargo identification is performed on multiple grid images in the grid time-series image group respectively to determine whether there are goods, the quantity of goods and the location of goods in the multiple grid images.

[0023] 3-2. Determination of the type of drop; the types of drop include normal drop, hanging bag, full bag, wrong classification, and floating grid.

[0024] If a cargo is detected in each grid image in the grid time-series image group, and the cargo position in each grid image moves, it is determined that the cargo falls into the grid normally.

[0025] If a cargo is detected in multiple grid images and the cargo position in the multiple grid images does not move, it is determined whether the cargo boundary is connected to the bottom edge of the grid; if the cargo boundary is not connected to the bottom edge of the grid, it is determined that the cargo is hanging; otherwise, it is determined that the cargo is full and blocked.

[0026] If multiple goods are detected in each grid image in any grid time-series image group, it is determined that the target grid has misclassified goods.

[0027] If no goods are detected in any of the grid images in the grid time sequence image group, the grid time sequence image group of the two adjacent grids of the target grid at the grid drop time is retrieved and the goods are identified; if goods are detected in the adjacent grid and there is no corresponding goods drop message in the adjacent grid at the grid drop time, it is determined that the goods are floating; otherwise, it is determined that the goods are misclassified.

[0028] When a hanging bag is detected, the alarm light corresponding to the bag opening will light up, and the display module will be used to display the real-time video shot at the bag opening; when a full bag is detected, the alarm light corresponding to the bag opening will light up, and the display module will be used to display the real-time video shot at the full bag opening; when a misclassification is detected, the information of the misclassified goods is recorded; when a floating grid is detected, the alarm light corresponding to the floating grid opening will light up.

[0029] As a preferred embodiment, the specific process of identifying goods on the grid image is as follows:

[0030] Step (1): Use a deep learning target detection algorithm to perform target detection on the grid image to form multiple candidate detection frames; each candidate detection frame corresponds to a confidence level; all candidate detection frames are added to the candidate detection frame set.

[0031] Step (2): Select the candidate detection frame with the highest confidence in the candidate detection frame set as the reference frame, and the remaining candidate detection frames as optimized frames; calculate the intersection-over-union ratio between all optimized frames and the reference frame.

[0032] Step (3): Update the confidence; update the confidence of all optimized boxes whose intersection-over-union ratio is greater than or equal to the intersection-over-union threshold.

[0033] Updated confidence The expression is as follows:

[0034]

[0035] in, is the confidence before updating; IOU is the intersection-over-union ratio of the detection frame and the reference frame; is the standard deviation parameter of the Gaussian function.

[0036] Step (4): Take the reference frame as a final cargo detection frame; remove the reference frame or all optimized frames whose confidence is lower than the confidence threshold from the candidate detection frame set.

[0037] Step (5): Repeat steps (2) to (4) until the set of candidate detection boxes is empty.

[0038] Preferably, the intersection threshold is set at 0.1 to 0.7; and the confidence threshold is set at 0.1 to 0.8.

[0039] Preferably, the packet output time is 100ms to 800ms; the sampling time of two adjacent grid images in the grid sequential image group differs by 40ms to 250ms according to the sampling frame rate.

[0040] Preferably, the judgment criterion for determining whether the cargo position in the plurality of grid images is moved is that the offset between the center points of the cargo detection frames in any two adjacent grid images is greater than a preset number of pixels.

[0041] The present invention has the following beneficial effects.

[0042] The present invention detects whether a package appears in the grid through a deep learning target detection algorithm, and determines whether the goods are correctly placed in the grid according to the number of packages and whether the packages are moved; and when the goods are placed in the grid incorrectly, determines the specific type of the goods placing error (including hanging packages, full material blocking packages, wrong classification, and floating grid).

[0043] The present invention communicates the misclassification judgment module with the sorting machine system server and realizes time synchronization through NTP; then the sorting machine system server sends a message of goods falling into the grid to the misclassification judgment module, so that the misclassification judgment module only recognizes the image of the sorting grid where the grid has fallen into the grid, which effectively identifies the sorting error, greatly reduces the image processing pressure of the misclassification judgment module, and reduces the requirement of real-time detection.

[0044] After the present invention identifies multiple candidate detection frames in an image, it performs Gaussian attenuation on the candidate detection frames with a relatively high intersection with the reference frame instead of directly deleting them; thus, it is possible to retain both detection frames when they are highly overlapped, thereby improving the detection effect on dense targets and small targets, and effectively avoiding the missed detection of packages that are close to or overlapped with each other.

[0045] The present invention uses an alarm lamp and a display module to provide a sorting error prompt to a worker in real time, which helps the worker to quickly handle a sorting error event.

[0046] The present invention realizes centralized analysis and processing in the data center, effectively controls the cost at a low level, and can trace the goods in real time throughout the entire process, facilitating subsequent inquiry and tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of the identification system used in the present invention.

[0048] Figure 2 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described below in conjunction with the accompanying drawings.

[0050] A vision-based cargo sorting error recognition method is used to automatically recognize errors that occur when a sorting machine delivers cargo into a grid.

[0051] like Figure 1 As shown, the identification system used in the cargo sorting error identification method includes a misclassification judgment module 1, an image acquisition module 2 and an alarm module. The image acquisition module 2 is used to collect images of each sorting grid; the image acquisition module 2 includes multiple cameras. Each camera can capture images of one or more sorting grids corresponding to itself. By segmenting the images collected by the image acquisition module 2, an independent image of each sorting grid can be obtained.

[0052] The system server of the misclassification judgment module 1 is connected to the camera in the image acquisition module 2 through a switch. The misclassification judgment module 1 is used to use a cargo recognition algorithm to recognize the image of the sorting grid, determine whether there is cargo in the image, and select the position of each cargo, and then determine whether cargo has fallen into the grid. The cargo in this embodiment is specifically a courier parcel.

[0053] The wrong sorting judgment module 1 communicates with the sorting machine system server 3 (specifically, the WCS server, Warehouse Control System) and realizes time synchronization through NTP (Network Time Protocol); the sorting machine system server is used to control the sorting of the upper and lower packages. When controlling the sorting trolley to lower the package, the sorting machine system server 3 sends the time of lowering the package and the number of the target grid for lowering the package to the wrong sorting judgment module 1. The wrong sorting judgment module 1 calls the image acquisition module 2 before and after the time of lowering the package, and recognizes it to determine whether the target grid for lowering the package has any goods dropped before and after the time of lowering the package.

[0054] The alarm module includes a grid alarm light. Each sorting grid is equipped with a grid alarm light. The wrong sorting judgment module 1 corresponds to a display module 4. The display module 4 is used to display the detected sorting error.

[0055] like Figure 2 As shown, the vision-based cargo sorting error recognition method includes the following steps:

[0056] Step 1: The image acquisition module 2 continuously acquires video stream image data of all slots of the sorting machine; the system server of the sorting machine controls the circular cross-belt sorting machine to sort the goods; whenever a cargo arrives and is sent to the target slot, the system server generates a cargo drop-off message and sends the cargo drop-off message to the misclassification judgment module 1 of the recognition system. The cargo drop-off message includes the target slot number and drop-off time of the cargo.

[0057] Step 2, the misclassification judgment module 1 retrieves the grid time sequence image group corresponding to the target grid at the grid drop moment according to the cargo drop message. The grid time sequence image group includes multiple grid images of the target grid collected after the grid drop moment; the first grid image collected among the multiple grid images is collected after the package outgoing time at the grid drop moment. The package outgoing time is the time required for the cargo to enter the sorting grid from the sorting trolley; the sampling time of the adjacent grid images in the multiple grid images is 40ms to 250ms according to the difference range of the loop speed; in this embodiment, the sampling frame rate of the video stream image data is 1 to 25FPS according to the loop speed; the multiple images are respectively the images collected by the image acquisition module 2 at the time closest to the grid drop moment, and the previous and next frames of images.

[0058] Step 3: According to the grid mouth time sequence image group collected in step 2, the grid drop situation is judged. There are five types of grid drop situations, namely normal grid drop, hanging bag, full material blocking bag, wrong separation, and floating grid.

[0059] 3-1. Cargo identification is performed on multiple grid images in the grid time-series image group respectively to determine whether there are goods, the quantity of goods and the location of goods in the multiple grid images.

[0060] 3-2. If a product is detected in multiple grid images, and the position of the product in the multiple grid images is moved, the product is judged to be in the grid normally. The judgment standard for the movement of the position of the product in the multiple grid images is: the offset between the center points of the product detection frames in any multiple adjacent grid images is greater than the preset number of pixels.

[0061] If multiple goods are detected in any grid image, it is determined that the target grid has misclassified goods.

[0062] If the goods are not detected in multiple grid images, the first grid error judgment is performed. The process of the first grid error judgment is as follows: retrieve the images of the two grids adjacent to the target grid and perform cargo identification; if the goods are detected in the adjacent grids, and there is no corresponding cargo grid drop event at the adjacent grid drop time, it is judged that the goods are floating, and the grid where the goods are detected is marked as a floating grid; otherwise, it is judged that the goods are misclassified.

[0063] If a cargo is detected in multiple grid images and the cargo position in multiple grid images does not move, the second grid error judgment is performed. The second grid error judgment process is as follows: If the cargo boundary is not connected to the bottom edge of the grid, it is judged that the cargo is hanging, and the target grid where the cargo is located is marked as the hanging grid. If the cargo boundary is connected to the bottom edge of the grid, it is judged that the cargo is full and the target grid where the cargo is located is marked as the full and blocked grid.

[0064] When a bag is detected, the alarm light corresponding to the bag compartment lights up, and the misclassification judgment module 1 controls the display module to display the real-time image and video captured by the bag compartment.

[0065] When it is detected that the bag is full and blocked, the alarm light corresponding to the full material blocking the bag grid is lit, and the wrong classification judgment module 1 controls the display module to display the real-time image and video taken by the full material blocking the bag grid.

[0066] When misclassification is detected, the misclassification judgment module 1 records the information of the misclassified goods to facilitate the staff to find the misclassified goods later.

[0067] When a piaoge is detected, the alarm light corresponding to the piaoge opening will light up. Subsequent staff will find the piaoge goods in the piaoge opening according to the three-segment code on the goods. The three-segment code is a unique regional identification code of the express company. The numbers on it represent the salesperson in the city, outlet, and unit area.

[0068] The specific process of identifying goods in the grid image is as follows:

[0069] Step (1): Use a deep learning target detection algorithm to perform target detection on the grid image and form multiple candidate detection frames. Each candidate detection frame corresponds to a confidence level. All candidate detection frames are added to the candidate detection frame set.

[0070] Step (2): Sort the detection frames in the candidate detection frame set from high to low according to the confidence score. Select the candidate detection frame with the highest confidence in the candidate detection frame set as the reference frame, and the remaining candidate detection frames as the optimized frames; calculate the intersection and union ratio between all optimized frames and the reference frame. IOU .

[0071] Step (3): Update the confidence. IOU The optimized box that is greater than or equal to the intersection-and-union threshold is obtained according to the intersection-and-union ratio obtained in step (3). IOU , use Gaussian decay function to update the confidence. The intersection threshold is 0.1 to 0.7.

[0072] Updated confidence The expression is as follows:

[0073]

[0074] in, is the confidence before updating; IOU is the intersection-over-union ratio of the detection frame and the reference frame; is the standard deviation parameter of the Gaussian function, which is used to control the degree of attenuation.

[0075] Updated confidence The reduction is related to the intersection-over-combination ratio IOUThe Gaussian decay function is related to the updated confidence when IOU is close to 1. Almost dropped to 0, when IOU is small, the updated confidence Reduced less.

[0076] Step (4): Take the reference frame as a final cargo detection frame; remove the reference frame or all optimized frames whose confidence is lower than the confidence threshold from the candidate detection frame set. In this embodiment, the confidence threshold is 0.1 to 0.8.

[0077] Step (5): Repeat steps (2) to (4) until the candidate detection box set is an empty set (that is, there is no candidate detection box), and the process is complete.

[0078] By debugging the values ​​of the intersection threshold and the confidence threshold, the following technical effects can be achieved: each time a round of steps (2) to (4) is executed, a new cargo detection frame can be obtained, and all candidate detection frames corresponding to the cargo detection frame are deleted. If there is only one cargo in the grid image, only one round of steps (2) to (4) is executed to complete; if there are only multiple cargoes in the grid image, multiple rounds of steps (2) to (4) equal to the number of cargoes will be executed.

[0079] In the cargo identification provided in this embodiment, the attenuation function is introduced to smooth the overlapping detection frames, reduce the confidence of the detection frames whose IOU is greater than the threshold, and improve the recall rate. Compared with the traditional NMS algorithm, it is more gentle and effective. It can improve the accuracy and robustness of target detection while maintaining high efficiency, thereby improving the accuracy and robustness of target detection.

Claims

1. A vision-based cargo sorting error recognition system, comprising a missorting judgment module (1) and an image acquisition module (2); characterized in that: It also includes an alarm module and a display module (4); the image acquisition module (2) is used to acquire an image of each sorting grid; the wrong sorting judgment module (1) communicates with the image acquisition module (2); the wrong sorting judgment module (1) is used to identify the grid image, determine whether there is a product in the grid image, and select the location of each product; During the sorting process, when a cargo arrives and is sent to a target slot, the sorting machine system server (3) generates a cargo drop-off message and sends the cargo drop-off message to the misclassification judgment module (1); the cargo drop-off message includes the target slot number and the drop-off time of the cargo; the misclassification judgment module (1) that receives the cargo drop-off message extracts a slot time-series image group of the target slot of the cargo at the drop-off time; the slot time-series image group includes a plurality of slot images captured after the drop-off time; the firstly captured slot image is captured after the time of the package being sent out at the drop-off time; a visual recognition algorithm is used to judge whether there is cargo in the slot image, and whether the package is dropped normally according to whether the cargo in each slot image moves; The alarm module includes a grid alarm light; each sorting grid is equipped with a grid alarm light; when any of the following conditions occurs at the sorting grid: a bag hanging, a bag full of materials, or a floating grid, the corresponding grid alarm light lights up; The display module (4) is used to display the detected sorting error of the identification system; when a hanging bag or a full bag is detected, the display module (4) displays a real-time video of the corresponding grid; The image acquisition module (2) identifies five types of bag drop, namely normal bag drop, bag hanging, full bag blockage, misclassification and floating bag; If a cargo is detected in each grid image in the grid time-series image group, and the cargo position in each grid image moves, it is determined that the cargo falls into the grid normally; If a cargo is detected in multiple grid images and the cargo position in the multiple grid images does not move, it is determined whether the cargo boundary is connected to the bottom edge of the grid; if the cargo boundary is not connected to the bottom edge of the grid, it is determined that the cargo is hanging; otherwise, it is determined that the cargo is full and blocked; If multiple goods are detected in each grid image in any grid time-series image group, it is determined that the target grid has misclassified goods; If no goods are detected in any of the grid images in the grid time sequence image group, the grid time sequence image group of the two adjacent grids of the target grid at the grid drop time is retrieved and the goods are identified; if goods are detected in the adjacent grid and there is no corresponding goods drop message in the adjacent grid at the grid drop time, it is determined that the goods are floating; otherwise, it is determined that the goods are misclassified.

2. The visual-based cargo sorting error identification system according to claim 1, characterized in that: The misclassification judgment module (1) communicates with the sorting machine system server (3) and performs time synchronization via a network time protocol.

3. A method for identifying cargo sorting errors based on vision, characterized in that: The following steps are involved: Step 1: Continuously collect images of all the slots of the sorting machine; Step 2: Whenever a product is dropped into a grid, a grid time-series image group corresponding to the target grid of the product at the grid dropping moment is extracted; the grid time-series image group includes multiple grid images of the target grid collected after the grid dropping moment; the first grid image collected among the multiple grid images is collected after the package delivery time at the grid dropping moment; the package delivery time is the time required for the product to enter the sorting grid from the sorting cart; Step 3: judging the grid type according to the grid time-series image group collected in step 2; Step 3-1. Recognize goods in each of the multiple grid images in the grid time-series image group, and determine whether there are goods, the quantity of goods, and the location of goods in the multiple grid images; Step 3-2. Determine the type of drop; the types of drop include normal drop, hanging bag, full bag, wrong classification, and floating grid; If a cargo is detected in each grid image in the grid time-series image group, and the cargo position in each grid image moves, it is determined that the cargo falls into the grid normally; If a cargo is detected in multiple grid images and the cargo position in the multiple grid images does not move, it is determined whether the cargo boundary is connected to the bottom edge of the grid; if the cargo boundary is not connected to the bottom edge of the grid, it is determined that the cargo is hanging; otherwise, it is determined that the cargo is full and blocked; If multiple goods are detected in each grid image in any grid time-series image group, it is determined that the target grid has misclassified goods; If no goods are detected in any of the grid images in the grid time sequence image group, the grid time sequence image group of the two adjacent grids of the target grid at the grid drop time is retrieved and the goods are identified; if goods are detected in the adjacent grid and there is no corresponding goods drop message in the adjacent grid at the grid drop time, it is determined that the goods are floating; otherwise, it is determined that the goods are misclassified; When a hanging bag is detected, the alarm light corresponding to the bag opening will light up, and the display module will be used to display the real-time video shot at the bag opening; when a full bag is detected, the alarm light corresponding to the bag opening will light up, and the display module will be used to display the real-time video shot at the full bag opening; when a misclassification is detected, the information of the misclassified goods is recorded; when a floating grid is detected, the alarm light corresponding to the floating grid opening will light up.

4. The method for identifying errors in cargo sorting based on vision according to claim 3, characterized in that: The specific process of identifying goods in the grid image is as follows: Step (1): Use a deep learning target detection algorithm to perform target detection on the grid image to form multiple candidate detection frames; each candidate detection frame corresponds to a confidence level; all candidate detection frames are added to the candidate detection frame set; Step (2): Select the candidate detection frame with the highest confidence in the candidate detection frame set as the reference frame, and the remaining candidate detection frames as the optimized frames; Calculate the intersection-over-union ratio between all optimized frames and the reference frame; Step (3): Update confidence; Update the confidence of all optimized boxes whose intersection-over-union ratio is greater than or equal to the intersection-over-union threshold; Updated confidence The expression is as follows: ； in, is the confidence before updating; IOU is the intersection-over-union ratio of the detection frame and the reference frame; is the standard deviation parameter of the Gaussian function; Step (4): Use the reference frame as a final cargo detection frame; remove the reference frame or all optimized frames whose confidence is lower than the confidence threshold from the candidate detection frame set; Step (5): Repeat steps (2) to (4) until the set of candidate detection boxes is empty.

5. The method for identifying errors in goods sorting based on vision according to claim 4, characterized in that: The intersection threshold is between 0.1 and 0.7; the confidence threshold is between 0.1 and 0.

8.

6. The method for identifying errors in goods sorting based on vision according to claim 3, characterized in that: The packet output time is 100ms to 800ms; the sampling time difference between two adjacent grid images in the grid sequential image group is 40ms to 250ms.

7. The method for identifying errors in goods sorting based on vision according to claim 3, characterized in that: The judgment criterion for determining whether the cargo position in the plurality of grid images is moved is that the offset between the center points of the cargo detection frames in any two adjacent grid images is greater than a preset number of pixels.

Images