Parcel detection method and parcel handling apparatus

Abstract

The application provides a parcel detection method and a parcel processing device, and relates to the technical field of logistics.The parcel detection method provided by the application comprises the following steps: when detecting parcels on a conveying surface, determining feature information of each virtual parcel in a first image collected by a first visual monitoring device and feature information of each virtual parcel in a second image collected by a second visual monitoring device, correcting the feature information of the virtual parcel in the first image according to the feature information of the virtual parcel in the second image, and determining information of an entity parcel by performing strategy matching on the virtual parcel in the first image and the virtual parcel in the second image.The parcel detection method provided by the application can combine feature information of virtual parcels obtained based on two-dimensional images and three-dimensional images to determine information of entity parcels on the conveying surface, overcome the problem of missed detection of parcels, and improve the accuracy of parcel detection.The parcel processing device provided by the application can implement the above-described parcel detection method.

Description

Technical Field

[0001] This application relates to the field of logistics technology, and more specifically, to a parcel detection method and parcel processing equipment. Background Technology

[0002] Currently, in the domestic logistics industry, with the surge in parcel sorting volume, more and more logistics companies are deploying logistics automation systems in sorting sites. These systems replace traditional manual methods for parcel separation, measurement, and sorting, greatly improving parcel processing efficiency and reducing logistics companies' labor costs.

[0003] Existing logistics automation systems include sorting equipment and various parcel handling devices located upstream of the sorting equipment. These parcel handling devices sequentially transport parcels and perform processes such as separation and measurement. Multiple parcels received in a stack are ultimately transported sequentially to the sorting equipment at predetermined intervals. The sorting equipment then sorts the parcels according to a pre-set sorting strategy. During parcel handling, the parcels need to be detected and their status determined, and appropriate processing is performed based on this status.

[0004] The related technology discloses a parcel handling device, including a conveying device, a swing wheel sorting machine arranged downstream of the conveying device along the parcel conveying direction, and a vision recognition device arranged above the conveying device. The vision recognition device includes multiple 3D cameras. The parcel handling device takes images of the parcels conveyed by the conveying device through the vision recognition device, and obtains the three-dimensional dimensions and real-time position coordinates of each parcel based on the images. Based on the above recognition results, it determines whether there are abnormal parcels that are conveyed in overlapping manner. For abnormal parcels that are conveyed in overlapping manner, the swing wheel sorting machine changes the swing direction and conveys them to a set position for manual intervention.

[0005] However, the inventors discovered that the package processing equipment of the related technology has the problem of inaccurate package detection, which may result in the missed detection of abnormal packages that are transported in overlapping manner, thus leading to package sorting abnormalities. Summary of the Invention

[0006] The purpose of this application is to provide a package detection method and a package processing device that can prevent abnormal packages from being missed and improve the accuracy of package detection.

[0007] The embodiments of this application can be implemented as follows:

[0008] In a first aspect, this application provides a package detection method applied to a package processing device. The package processing device includes a conveying device and a first visual monitoring device and a second visual monitoring device disposed above the conveying device. The conveying device is used to convey packages along a set direction and has a conveying surface for carrying and conveying packages. The first visual monitoring device is used to acquire two-dimensional images of the conveying surface of the conveying device, and the second visual monitoring device is used to acquire three-dimensional images of the conveying surface of the conveying device. The package detection method includes:

[0009] A first image is determined from images acquired by a first visual monitoring device, and a second image is determined from images acquired by a second visual monitoring device, wherein the first image and the second image are images acquired at the same time.

[0010] Detect virtual packages in the first image and determine the feature information of each virtual package, and detect virtual packages in the second image and determine the feature information of each virtual package;

[0011] The feature information of the virtual package in the first image is corrected based on the feature information of the virtual package in the second image;

[0012] Strategy matching is performed on the virtual packages in the first image and the virtual packages in the second image to determine the information of the physical packages on the transport surface.

[0013] In an optional implementation, the step of detecting virtual packages in the first image and determining the feature information of each virtual package includes:

[0014] The virtual packages in the first image are detected based on the object detection algorithm, and coarse information of each virtual package is obtained. The coarse information of the virtual packages is described by the bounding box of the object detection algorithm.

[0015] Based on the rough information of each virtual package, determine whether the virtual package is a first type or a second type. The first type of virtual package is the one that is more likely to be falsely detected, while the second type of virtual package is the one that is less likely to be falsely detected.

[0016] For each virtual package of the first type, the bounding box describing the virtual package is detected based on the object segmentation algorithm to obtain the virtual package within the bounding box and the precise information of each virtual package. The precise information of the virtual package uses the outline of the virtual package to describe the virtual package.

[0017] The precise information of the first type of virtual package is determined as the feature information of the first type of virtual package, and the rough information of the second type of virtual package is determined as the feature information of the second type of virtual package.

[0018] In an optional implementation, the step of determining whether a virtual package is a first-type or second-type virtual package based on the coarse information of each virtual package includes:

[0019] Determine whether a virtual package is a virtual package containing similar stacked objects based on rough information about the virtual package;

[0020] When a virtual package is determined to be a virtual package containing similar stacked objects, it is classified as a first-type virtual package; when a virtual package is determined not to be a virtual package containing similar stacked objects, it is classified as a second-type virtual package.

[0021] In an optional implementation, the step of correcting the feature information of the virtual package in the first image based on the feature information of the virtual package in the second image includes:

[0022] A matching operation is performed on the virtual packages in the first image and the virtual packages in the second image to determine the virtual packages in the first image that can be matched and those that cannot be matched.

[0023] For a virtual package that can be matched in the first image, the length and / or width of the virtual package are corrected based on the height data of the virtual package that matches it in the second image.

[0024] In an optional implementation, the step of performing strategy matching on the virtual packages in the first image and the virtual packages in the second image to determine the information of the physical packages on the delivery surface includes:

[0025] Determine whether the first number of virtual packages in the first image is equal to the second number of virtual packages in the second image;

[0026] If the first quantity and the second quantity are equal, a matching operation is performed on the virtual packages in the first image and the virtual packages in the second image to determine whether the virtual packages in the first image and the virtual packages in the second image can be completely matched. If they can be completely matched, the information of the physical packages on the conveying surface is determined based on the feature information of the virtual packages in the second image; otherwise, the information of the physical packages on the conveying surface is determined based on the feature information of the virtual packages in the first image.

[0027] If the first quantity and the second quantity are not equal, determine whether the first quantity is less than or equal to one piece. If the first quantity is less than or equal to one piece, determine the information of the physical package on the conveyor surface based on the feature information of the virtual package in the second image. Otherwise, determine the information of the physical package on the conveyor surface based on the feature information of the virtual package in the first image.

[0028] In an optional implementation, the package detection method further includes:

[0029] Based on the information of the physical packages, determine whether the distance between each physical package and its adjacent physical packages is less than a set value;

[0030] If the distance between a physical package and its adjacent physical package is less than a set value, the physical package and its adjacent physical package are determined to be abnormal packages that are being transported in overlapping manner, and the abnormal packages are handled in a special way.

[0031] In an optional implementation, the step of special handling of abnormal packages includes one of the following:

[0032] Separate the stacked items from the abnormal packages; or,

[0033] Return the abnormal package; or,

[0034] The abnormal package is transported to the unloading station for manual handling.

[0035] Secondly, this application provides a parcel handling device, including a control device, a conveying device, and a first visual monitoring device and a second visual monitoring device disposed above the conveying device. The conveying device is used to convey parcels along a set direction and has a conveying surface for carrying and conveying parcels. The first visual monitoring device is used to acquire two-dimensional images of the conveying surface of the conveying device, and the second visual monitoring device is used to acquire three-dimensional images of the conveying surface of the conveying device. The conveying device, the first visual monitoring device, and the second visual monitoring device are all electrically connected to the control device, which is configured as follows:

[0036] A first image is determined from images acquired by a first visual monitoring device, and a second image is determined from images acquired by a second visual monitoring device, wherein the first image and the second image are images acquired at the same time.

[0037] Detect virtual packages in the first image and determine the feature information of each virtual package, and detect virtual packages in the second image and determine the feature information of each virtual package;

[0038] The feature information of the virtual package in the first image is corrected based on the feature information of the virtual package in the second image;

[0039] Strategy matching is performed on the virtual packages in the first image and the virtual packages in the second image to determine the information of the physical packages on the transport surface.

[0040] In an optional implementation, the control device detects virtual packages in the first image and determines the feature information of each virtual package in the following manner:

[0041] The virtual packages in the first image are detected based on the object detection algorithm, and coarse information of each virtual package is obtained. The coarse information of the virtual packages is described by the bounding box of the object detection algorithm.

[0042] Based on the rough information of each virtual package, determine whether the virtual package is a first type or a second type. The first type of virtual package is the one that is more likely to be falsely detected, while the second type of virtual package is the one that is less likely to be falsely detected.

[0043] For each virtual package of the first type, the bounding box describing the virtual package is detected based on the object segmentation algorithm to obtain the virtual package within the bounding box and the precise information of each virtual package. The precise information of the virtual package uses the outline of the virtual package to describe the virtual package.

[0044] The precise information of the first type of virtual package is determined as the feature information of the first type of virtual package, and the rough information of the second type of virtual package is determined as the feature information of the second type of virtual package.

[0045] In an optional implementation, the control device corrects the feature information of the virtual package in the first image based on the feature information of the virtual package in the second image in the following manner:

[0046] A matching operation is performed on the virtual packages in the first image and the virtual packages in the second image to determine the virtual packages in the first image that can be matched and those that cannot be matched.

[0047] For a virtual package that can be matched in the first image, the length and / or width of the virtual package are corrected based on the height data of the virtual package that matches it in the second image.

[0048] The beneficial effects of the embodiments of this application include, for example:

[0049] In the package detection method of this application embodiment, when detecting packages on the conveyor surface, virtual packages included in a first image acquired by a first visual monitoring device are detected and feature information of each virtual package is determined; virtual packages included in a second image acquired by a second visual monitoring device are also detected and feature information of each virtual package is determined; feature information of virtual packages in the first image is corrected based on feature information of virtual packages in the second image; and information of physical packages on the conveyor surface is determined by strategy matching of virtual packages in the first and second images. Therefore, the package detection method provided by this application can combine feature information of virtual packages obtained based on two-dimensional images and feature information of virtual packages obtained based on three-dimensional images to determine information of physical packages on the conveyor surface, overcoming the problem of missed package detection caused by the low height of packages when using a 3D camera alone, and improving the accuracy of package detection. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a block diagram of the package processing equipment in one embodiment of this application;

[0052] Figure 2 This is a schematic diagram of a logistics system in one embodiment of this application;

[0053] Figure 3 This is a flowchart of a package detection method in one embodiment of this application;

[0054] Figure 4 This is a flowchart illustrating the detection of virtual packages in a first image and the determination of feature information for each virtual package in one embodiment of this application;

[0055] Figure 5a This is a schematic diagram of the first image in one embodiment of this application;

[0056] Figure 5b This is a schematic diagram of the region of interest in the first image in one embodiment of this application;

[0057] Figure 6 This is a schematic diagram of a 2D camera's imaging capabilities.

[0058] Figure 7a and Figure 7b This is a schematic diagram showing the spacing between two entities in two different scenarios.

[0059] Icons: 010 - Logistics system; 100 - Parcel handling equipment; 110 - Control device; 120 - Conveying device; 121 - Conveying surface; 130 - First visual monitoring device; 131 - 2D camera; 132 - First image; 133 - Region of interest; 134 - Virtual parcel; 140 - Second visual monitoring device; 141 - 3D camera; 150 - Swing sorter; 200 - Sorting equipment; 300 - First import device; 400 - Second import device; 410 - Drop-off platform; 500 - Stacking separator; 600 - Parallel separator; 020 - Target parcel; 021 - First physical parcel; 022 - Second physical parcel. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0061] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0062] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0063] In the description of this application, it should be noted that if terms such as "upper," "lower," "inner," or "outer" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of the invention is usually placed during use, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0064] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0065] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0066] In the current logistics field, parcel handling equipment uses images captured by multiple 3D cameras to detect parcels on conveyor belts and their status. However, the inventors have discovered that 3D cameras inevitably have certain detection errors during actual use: when a parcel is low, the 3D camera often fails to detect it on the conveyor belt. Therefore, existing parcel detection methods may miss some parcels. Specifically, when detecting objects within its field of view, the 3D camera acquires the coordinate data of each pixel in its field of view in three dimensions: height, length, and width. It then determines whether any pixels exhibit abrupt changes in height relative to a reference plane, thereby determining whether an object is present on the reference plane. If an object is present, its height, length, and width are calculated using the coordinate values ​​of its individual pixels. The inventors have found that when using a 3D camera to detect parcels on a conveyor belt, the maximum detection error is larger when the distance from the top surface of the parcel to the 3D camera is significant. For example, when the distance from the top surface of a package to the 3D camera reaches 2 meters, if the 3D camera has a maximum detection error of ±1%, then the maximum detection error will reach ±2 cm. In this case, if the package is small in height, such as an envelope with a height of less than 2 cm, the 3D camera may not be able to detect pixels with a height abrupt change relative to the reference plane, thus failing to detect the package on the conveyor, resulting in missed detection. If two packages with similar heights overlap during transport, such as two envelopes, the 3D camera's detection results cannot determine that the two packages are being transported overlapping, causing the overlapping package to be transported to the sorting equipment, resulting in sorting anomalies.

[0067] To address the shortcomings of the aforementioned related technologies, this application provides a package detection method and a package processing device. The package detection method and package processing device of this application collect two-dimensional and three-dimensional images of a package, and combine the information from the two-dimensional and three-dimensional images to determine the information of the physical package. This improves the accuracy of package detection by addressing potential missed detections that may occur when using 3D cameras for package detection in related technologies.

[0068] Figure 1 This is a block diagram of the package handling device 100 in one embodiment of this application. Figure 1 As shown, the parcel handling equipment 100 provided in this embodiment includes a control device 110, a conveying device 120, and a first visual monitoring device 130 and a second visual monitoring device 140 disposed above the conveying device 120. The conveying device 120 is used to convey parcels along a set direction, and the conveying device 120 has a conveying surface 121 for carrying and conveying parcels (see...). Figure 2The first visual monitoring device 130 is used to acquire two-dimensional images of the conveying surface 121 of the conveying device 120, and the second visual monitoring device 140 is used to acquire three-dimensional images of the conveying surface 121 of the conveying device 120. The conveying device 120, the first visual monitoring device 130, and the second visual monitoring device 140 are all electrically connected to the control device 110. The package handling equipment 100 provided in this application embodiment is used to convey packages and obtain the position information of the conveyed packages through the first visual monitoring device 130 and the second visual monitoring device 140. Since in the entire logistics system, upstream of the sorting equipment, piles of packages are separated during the conveying process and are finally conveyed to the sorting equipment in sequence to meet the set spacing requirements, the package handling equipment 100 provided in this application embodiment is used to detect whether there are abnormal packages with overlapping conveying in the currently conveyed packages, so that the system can perform corresponding processing according to the detection results to ensure that there are no abnormal packages with overlapping conveying in the packages conveyed to the sorting equipment.

[0069] Furthermore, the package processing equipment 100 in this embodiment also includes a swing wheel sorter 150, which is electrically connected to the control device 110. The swing wheel sorter 150 can transport packages to different downstream devices according to whether the packages are abnormal, so that abnormal packages can be processed in a timely manner and avoid being transported to the sorting equipment 200 in an overlapping manner.

[0070] Figure 2 This is a schematic diagram of a logistics system 010 in one embodiment of this application. Figure 1As shown, the logistics system 010 provided in this embodiment includes a sorting device 200 and multiple processing devices disposed upstream of the sorting device 200. Each of the multiple processing devices upstream of the sorting device 200 has the function of conveying packages, and each also has different functions such as separating packages and detecting packages. In this embodiment, a first inlet device 300 and a second inlet device 400 are disposed upstream of the sorting device 200, forming two parallel channels for conveying packages to the sorting device 200. Upstream of the first inlet device 300 and the second inlet device 400, in accordance with the package conveying direction, a stacking separator 500, a parallel separator 600, and the package processing device 100 provided in this embodiment are disposed sequentially. The stacking separator 500 is used to separate overlapping packages, and the parallel separator 600 is used to separate packages conveyed side-by-side in the package conveying direction. The parcel processing device 100 provided in this embodiment is used to detect the position information of the parcels it is conveying, so as to determine whether the parcels meet the set spacing requirements after being processed by the stack separator 500 and the parallel separator 600. If the spacing between the parcels meets the requirements, that is, there are no abnormal parcels being conveyed overlapping, then the normally conveyed parcels are transferred by the swing wheel sorter 150 to the first import device 300, and the first import device 300 sequentially conveys these parcels to the sorting device 200; if the parcel processing device 100 detects abnormal parcels being conveyed overlapping, then the swing wheel sorter 150 transfers the abnormal parcels being conveyed overlapping to the second import device 400. The second import device 400 includes a drop table 410 for manual handling, where overlapping abnormal parcels can be manually separated and then conveyed to the sorting device 200.

[0071] In this embodiment, the conveying surface 121 of the conveying device 120 includes a region to be detected. The first visual monitoring device 130 includes at least one 2D camera 131, and the total field of view of the 2D cameras 131 in the first visual monitoring device 130 can cover the region to be detected on the conveying surface 121. When the first visual monitoring device 130 includes multiple 2D cameras 131, the images acquired by the multiple 2D cameras 131 at the same time can be stitched together to obtain a two-dimensional image of the region to be detected. The second visual monitoring device 140 includes at least one 3D camera 141, and the total field of view of the 3D cameras 141 included in the second visual monitoring device 140 can cover the region to be detected on the conveying surface 121. When the second visual monitoring device 140 includes multiple 3D cameras 141, the images acquired by the multiple 3D cameras 141 at the same time can be stitched together to obtain a three-dimensional image of the region to be detected.

[0072] In this embodiment, the control device 110 of the package handling device 100 is configured as follows:

[0073] A first image is determined from the images acquired by the first visual monitoring device 130, and a second image is determined from the images acquired by the second visual monitoring device 140, wherein the first image and the second image are images acquired at the same time; virtual packages in the first image are detected and feature information of each virtual package is determined, and virtual packages in the second image are detected and feature information of each virtual package is determined; feature information of virtual packages in the first image is corrected based on feature information of virtual packages in the second image; strategy matching is performed on virtual packages in the first image and virtual packages in the second image to determine information of physical packages on the transport surface.

[0074] Furthermore, the control device 110 detects the virtual packages in the first image and determines the feature information of each virtual package in the following manner:

[0075] Virtual packages in the first image are detected using an object detection algorithm, and coarse information for each virtual package is obtained. The coarse information uses bounding boxes from the object detection algorithm to describe the virtual package. Based on the coarse information, each virtual package is determined to be either a first-type or second-type virtual package. First-type virtual packages are more prone to false detection, while second-type virtual packages are less prone to false detection. For each first-type virtual package, the bounding box describing the virtual package is detected using an object segmentation algorithm, obtaining the virtual package within the bounding box and precise information for each virtual package. The precise information uses the virtual package's outline to describe it. The precise information for first-type virtual packages is determined as the feature information for first-type virtual packages, and the coarse information for second-type virtual packages is determined as the feature information for second-type virtual packages.

[0076] Furthermore, the control device 110 corrects the feature information of the virtual package in the first image based on the feature information of the virtual package in the second image in the following manner:

[0077] A matching operation is performed on the virtual packages in the first image and the virtual packages in the second image to determine the virtual packages that can be matched and those that cannot be matched in the first image; for the virtual packages that can be matched in the first image, the length and / or width of the virtual packages are corrected according to the height data of the virtual packages that match them in the second image.

[0078] The following describes in detail the package detection method provided in the embodiments of this application. Figure 3This is a flowchart of a package detection method in one embodiment of this application. It should be understood that the package detection method provided in this embodiment is applicable to the package processing device 100 of this embodiment. Optionally, each step of the package detection method can be implemented by the control device 110 of the package processing device 100. Figure 3 As shown, the package detection method provided in this application includes:

[0079] Step S100: Determine a first image from the images acquired by the first visual monitoring device, and determine a second image from the images acquired by the second visual monitoring device, wherein the first image and the second image are images acquired at the same time.

[0080] Taking the parcel processing device 100 provided in this application embodiment as an example, during the process of controlling the conveying device 120 to convey parcels along a set direction, the control device 110 of the parcel processing device 100 controls the first visual monitoring device 130 to collect a two-dimensional image of the conveying surface 121 every set time interval (e.g., 1ms, 2ms, 3ms), and controls the second visual monitoring device 140 to collect a three-dimensional image of the conveying surface 121 every set time interval (e.g., 1ms, 2ms, 3ms). The control device 110 determines one image from the multiple two-dimensional images acquired by the first visual monitoring device 130 as the first image, and determines one image from the multiple three-dimensional images acquired by the second visual monitoring device 140 as the second image. The first image and the second image are images acquired at the same time.

[0081] Step S200: Detect virtual packages in the first image and determine the feature information of each virtual package, and detect virtual packages in the second image and determine the feature information of each virtual package.

[0082] Taking the package processing device 100 provided in this application embodiment as an example, the control device 110 performs package detection based on a first image to determine the virtual packages included in the first image and the feature information of each virtual package. Furthermore, the control device 110 also performs package detection based on a second image to determine the virtual packages included in the second image and the feature information of each virtual package. The feature information of each virtual package includes its packaging category, location, size, outline, etc. The size information of the virtual packages in the first image includes length and width, and the size information of the virtual packages in the second image includes length, width, and height. Optionally, the control device 110 may use methods known in the prior art (such as deep learning methods or background modeling methods) to perform package detection on the image.

[0083] Figure 4 This is a flowchart illustrating the detection of virtual packages in a first image and the determination of feature information for each virtual package in one embodiment of this application. Figure 4As shown, step S200, which involves detecting virtual packages in the first image and determining the feature information of each virtual package, specifically includes:

[0084] Step S210: Detect virtual packages in the first image based on the object detection algorithm and obtain coarse information of each virtual package, wherein the coarse information of the virtual package uses the bounding box of the object detection algorithm to describe the virtual package.

[0085] Step S220: Determine whether a virtual package is a first type or a second type based on the rough information of each virtual package. The first type of virtual package is a virtual package that is prone to false detection, and the second type of virtual package is a virtual package that is not prone to false detection.

[0086] Step S230: For each virtual package of the first type, the bounding box describing the virtual package is detected based on the object segmentation algorithm to obtain the virtual package within the bounding box and the precise information of each virtual package. The precise information of the virtual package uses the outline of the virtual package to describe the virtual package.

[0087] Step S240: The precise information of the first type of virtual package is determined as the feature information of the first type of virtual package, and the rough information of the second type of virtual package is determined as the feature information of the second type of virtual package.

[0088] In this embodiment, a combination of object detection and object segmentation algorithms is used to determine the feature information of virtual packages in the first image. Since object detection algorithms are characterized by high detection efficiency but low detection accuracy, and object segmentation algorithms are characterized by high detection accuracy but low detection efficiency, in this embodiment, an object detection algorithm is first used to perform a highly efficient primary detection on the first image to determine the virtual packages contained in the first image and the coarse information of each virtual package. The coarse information of the virtual packages uses the bounding boxes from the object detection algorithm to describe the virtual packages. Then, based on the coarse information of each virtual package, the type of each virtual package is determined. When a virtual package is of the first type, which is prone to false detection, an object segmentation method is used to perform a highly accurate secondary detection on the bounding boxes describing the virtual package to determine the virtual packages within the bounding boxes and the precise information of each virtual package. The precise information of the virtual packages uses the outline of the package to describe the virtual package. Finally, the precise information of each first-type virtual package is determined as the feature information of that virtual package, and the coarse information of each second-type virtual package is determined as the feature information of that virtual package. Optionally, in this embodiment, a known object detection algorithm can be used to detect the virtual package in the first image once. For example, object detection algorithms that include two-stage detection, such as R-CNN, SPPNet, Fast R-CNN, Faster R-CNN, and R-FCN, can be used to detect the virtual package; or object detection algorithms that only include one-stage detection, such as YOLO, SSD, and RetinaNet, can be used to detect the virtual package.

[0089] When a virtual package is determined to be a first-type virtual package prone to false detection based on coarse information, a pixel-level precise object segmentation algorithm is used to perform secondary detection on the bounding box describing the virtual package. In this process, a seed point within the bounding box is set as the growth starting point. Then, pixels in a defined neighborhood around the seed point are grown and merged according to growth rules until no pixels satisfy the growth point, thus obtaining the outline of the virtual package. Optionally, in this embodiment, a known object segmentation algorithm can be used for secondary detection of the first-type virtual package. For example, a deep learning-based object segmentation algorithm can be used to detect the outline of the virtual package within the bounding box, where VGG, GoogleNet, or other networks can be used as the basic framework for deep learning.

[0090] It should be noted that when a virtual package is identified as a first-type virtual package, prone to false detection, based on coarse information, a secondary detection using an object segmentation method on the bounding box describing the virtual package may result in multiple virtual packages being detected within a single bounding box. That is, during the initial detection using the object detection algorithm, multiple virtual packages may be incorrectly detected as a single virtual package. In this case, all virtual packages detected within the same bounding box through secondary detection are considered first-type virtual packages, and the precise information of each virtual package detected through secondary detection is used as its feature information.

[0091] Because this embodiment uses a combination of two algorithms to detect virtual packages in the first image, the entire image is detected once using a high-efficiency target detection algorithm. Based on the detection results of the target detection algorithm, only local areas are detected a second time using an object segmentation algorithm to obtain the virtual packages in that local area (i.e., within the bounding box) and the accurate information of each virtual package, thereby avoiding false detection of virtual packages. Through the decision-level fusion of the two algorithms, the method of this embodiment can achieve both detection efficiency and detection accuracy by complementing the advantages of the two types of detection algorithms.

[0092] Optionally, the information of the virtual package includes the packaging category, location, size, outline, and status of the virtual package. The packaging category of the virtual package includes envelopes, soft packaging, and boxes. The status of the virtual package includes whether the virtual package overlaps with other virtual packages or not.

[0093] Optionally, the virtual packages in the first image are detected based on the object detection algorithm to obtain coarse information about each virtual package. Specifically, this may include: extracting the region of interest in the first image that corresponds to the area to be detected on the transport surface, performing object detection on the region of interest in the first image, and obtaining coarse information about each virtual package. Figure 5a This is a schematic diagram of the first image 132 in one embodiment of this application;

[0094] Figure 5b This is a schematic diagram of the region of interest 133 in the first image 132 in one embodiment of this application. Figure 5a As shown, to ensure the detection of packages located at any position on the conveyor surface 121, the field of view of the first visual monitoring device 130 and the second visual monitoring device 140 is larger than the area to be detected on the conveyor surface. Therefore, in this embodiment, the region of interest 133 in the first image 132 is first acquired, and then virtual package detection is performed based on the region of interest 133 to improve the efficiency of package detection. Figure 5a and Figure 5bAs shown, in this embodiment, the region of interest 133 is located within the conveying surface 121, and a virtual package 134 exists in the region of interest 133.

[0095] Optionally, known methods of the prior art can be used to detect the virtual packages included in the second image and determine the feature information of each virtual package. For example, the region of interest corresponding to the area to be detected on the conveying surface in the second image can be extracted, and the point cloud data corresponding to the region of interest can be obtained. The feature information of each virtual package in the second image can be obtained by analyzing and processing the point cloud data.

[0096] Furthermore, in step S220 above, the step of determining whether a virtual package is a first-type virtual package or a second-type virtual package based on the rough information of each virtual package may specifically include:

[0097] Based on the rough information of the virtual package, determine whether the virtual package is a virtual package where similar objects are stacked; if the virtual package is determined to be a virtual package where similar objects are stacked, classify the virtual package as a first type of virtual package; if the virtual package is determined not to be a virtual package where similar objects are stacked, classify the virtual package as a second type of virtual package.

[0098] The stacking of similar objects includes stacking flexible packaging with flexible packaging, stacking envelopes with envelopes, and stacking boxes with boxes. Optionally, in this embodiment, before using the object detection algorithm, samples of various packaging types such as flexible packaging, envelopes, and boxes are used for training. Based on the obtained samples, the object detection algorithm is used to detect the virtual packages in the first image. After detecting the virtual packages in the first image through the object detection algorithm, the packaging category of each virtual package can be obtained. Then, the state of each virtual package is determined. When it is determined that a virtual package overlaps with other virtual packages, it is determined whether the virtual package is a virtual package with similar object stacking. That is, it is determined whether the two overlapping virtual packages are both flexible packaging, or whether the two overlapping virtual packages are both envelopes, or whether the two overlapping virtual packages are both boxes. When it is determined that a virtual package is a virtual package with similar object stacking, the virtual package is identified as a first type of virtual package; otherwise, the virtual package is identified as a second type of virtual package.

[0099] Because object detection algorithms have relatively low accuracy, when virtual packages are stacked together with similar objects, using coarse information about the virtual packages as their feature information for package processing may lead to inaccurate information about the physical packages on the conveyor surface, resulting in abnormal processing of the physical packages. For example, if multiple overlapping virtual packages are detected as a single virtual package, multiple overlapping physical packages on the conveyor surface may be incorrectly sorted to the same destination. Therefore, when a virtual package is determined to be a virtual package with similar object stacking based on its coarse information, it is classified as a first type of virtual package that is prone to false detection. This allows for secondary detection of the bounding box describing the virtual package using an object segmentation algorithm to obtain accurate information about the virtual package. However, if the virtual package does not overlap with other virtual packages, or if the two overlapping virtual packages are of different types, even if coarse information is used for subsequent package detection and processing, no detection or processing errors will occur. Therefore, in the above cases, the virtual package is classified as a second type of virtual package that is less prone to false detection. That is, there is no need to perform secondary detection on the bounding box describing the virtual package, and the coarse information of the virtual package can be used as its feature information, thereby improving package detection efficiency.

[0100] Step S300: Correct the feature information of the virtual package in the first image based on the feature information of the virtual package in the second image.

[0101] Taking the parcel processing device 100 provided in this application embodiment as an example, this step may specifically include: performing a matching operation on the virtual parcels in the first image and the virtual parcels in the second image to determine the virtual parcels that can be matched and the virtual parcels that cannot be matched in the first image; for the virtual parcels that can be matched in the first image, correcting the length and / or width of the virtual parcels according to the height data of the virtual parcels that match them in the second image.

[0102] In this embodiment, a matching operation is performed on the virtual packages in the first image and the virtual packages in the second image to determine the matching and non-matching virtual packages in the first image. Specifically, when a virtual package in the first image can match a virtual package in the second image, the virtual package in the first image is determined as a matching virtual package, and a correspondence between the two virtual packages is established. When a virtual package in the first image cannot match any virtual package in the second image, the virtual package is determined as a non-matching virtual package. It is understood that since the first image is determined based on a two-dimensional image (i.e., the first image), and the second image is determined based on a three-dimensional image (i.e., the second image), for physical packages (e.g., envelopes) with low height that cannot be detected by the 3D camera 141 on the conveyor surface 121, there will be no corresponding virtual package in the second image. Therefore, there may be non-matching virtual packages in the first image. These virtual packages often correspond to envelopes with low height on the conveyor surface 121.

[0103] Optionally, the SIFT algorithm or the SURF algorithm can be used to perform matching operations on the virtual packages in the first image and the virtual packages in the second image.

[0104] Furthermore, before performing matching operations on the virtual packages in the first image and the second image, the method may further include converting the virtual packages in the first image and the second image into virtual packages based on the same world coordinate system. Specifically, when performing package detection on the first image and the second image, the first image and the second image each have corresponding image coordinate systems. The image coordinate system is a coordinate system established with the image as a reference, used to describe the position of pixels in the image. Optionally, the coordinate system corresponding to the first image is a two-dimensional coordinate system, and the coordinate system corresponding to the second image is a three-dimensional coordinate system. After detecting virtual packages in the first image and the second image, the control device 110 converts the virtual packages in the first image from the image coordinate system of the first image to the world coordinate system, and converts the virtual packages in the second image from the image coordinate system of the second image to the world coordinate system. The world coordinate system is an absolute coordinate system established with the conveying surface 121 of the conveying device 120 as a reference, used to describe the position of the object on the conveying surface 121. After transforming the virtual packages in the first and second images to the same world coordinate system, the positions of the virtual packages in the first and second images are represented by coordinate values ​​in the world coordinate system, and the size of each virtual package is the size of the virtual package in the world coordinate system. This ensures the accuracy of matching operations between the virtual packages in the first and second images.

[0105] Since the first visual monitoring device 130 captures a two-dimensional image, the size of the virtual package detected by the first image is affected by the shooting angle of the 2D camera 131. For example, if the lens of the 2D camera 131 is pointing downwards, when a physical package is located directly below the 2D camera 131, the lens of the 2D camera 131 is facing the upper surface of the physical package. When detecting the virtual package through the first image, the outline of the virtual package corresponding to the physical package corresponds to the upper surface of the physical package. At this time, the size of the physical package can be calculated more accurately from the outline of the virtual package. However, when a physical package is located on one side directly below the 2D camera 131 (e.g., the front, back, left, or right side), the 2D camera 131 will capture at least two surfaces of the physical package (the upper surface and at least one side) when capturing the image. When detecting the virtual package through the first image, the outline of the virtual package corresponding to the physical package corresponds to at least two surfaces of the physical package. At this time, the size of the physical package calculated from the outline of the virtual package is inaccurate. Figure 6 This is a schematic diagram of the imaging process using a 2D camera 131. Figure 6 As shown, target package 020 is a physical package on conveyor surface 121. The package is conveyed in a horizontal direction, and its length is in the direction of its conveying (i.e., ...). Figure 6 The dimensions along the horizontal axis (indicated by the middle arrow). Target package 020 includes four vertices B, C, E, and F. The actual length of target package 020 along the horizontal axis is the distance BC from vertex B to vertex C. When target package 020 is imaged within the field of view of 2D camera 131, the camera can capture the vertices E, F, and C of target package 020. Based on the corresponding virtual package in the first image, the measured length of target package 020 along the horizontal axis will be AC. That is, there is a certain error (AB) between the measured length of target package 020 obtained from the first image captured by 2D camera 131 and the actual length of target package 020. The higher the height of target package 020 and the farther the target package 020 is from directly below 2D camera 131 (such as point D in the figure), the greater this error will be.

[0106] In this embodiment, after obtaining the feature information of the virtual package through the first image, for virtual packages that can be matched in the first image, the length and / or width of the virtual package are modified according to the height data of the matching virtual package in the second image. For example, for Figure 6The target package 020 in the image corresponds to a first virtual package in the first image and a second virtual package in the second image. When the control device 110 determines that the first virtual package and the second virtual package match through matching calculation, let the height of the target package 020 calculated based on the second virtual package be H, and the distance from the point A on the target package 020 farthest from the center point D of the camera's field of view calculated based on the first virtual package to the center point D of the camera's field of view be L. As can be easily seen from the figure below, H corresponds to BE, and L corresponds to AD. The length L of the package calculated based on the first virtual package has an error, and this error value L 误差 This corresponds to the distance AB. Since AB / AD = BE / OD, where OD is the camera's mounting height, this value is the known setpoint (let's say H) after the camera is installed. 相 Therefore, it is easy to know that the error value AB = AD * BE / OD. By substituting the detected H and L into the above formula, the error value L of the target package 020 detected based on the first image can be calculated. 误差 =L*H / H 相 This allows us to calculate the coordinates of vertex E, which is actually furthest from center point D in the target package 020, along the package's transport direction. Figure 6 In the case of the cuboid shown, the coordinates of vertex E and vertex B are the same in the package conveying direction. Therefore, the coordinates of vertex B and the actual length L of the target package 020 can be obtained. 实际 , where L 实际 The corresponding length BC is given by BC = AC - AB, and AC corresponds to the length L of the first virtual package obtained from the first image. 测量 Therefore, it can be known that L 实际 =L 测量 -AB, the calculated error value L 误差 Substituting the values, we can obtain the actual length of the first virtual package as:

[0107] L 实际 =L 测量 -L 误差 =L 测量 -L*H / H 相 .

[0108] Among them, L 测量 L is the length of the first virtual package along the transport direction of the package, obtained from the first image; L is the distance from the point on the first virtual package farthest from the center of the camera's field of view to the center of the camera's field of view, obtained from the first image; H is the height of the second virtual package, obtained from the second image. 相The installation height of the 2D camera 131 of the first visual monitoring device 130; the first virtual package and the second virtual package are matched with each other, and they correspond to the same physical package on the conveyor surface 121, that is, Figure 6 The target package is 020.

[0109] Therefore, based on the above calculation results, the actual length of the first virtual package can be corrected to the calculated L. 实际 The value of .

[0110] Similarly, the actual width W of the target package 020 can be calculated using the method described above. 实际 The actual width of the first virtual package is corrected to the calculated W. 实际 The value. Specifically,

[0111] W 实际 =W 测量 -W*H / H 相 .

[0112] Among them, W 测量 W is the width of the first virtual package along the width direction of the conveying surface 121 (the width direction of the conveying surface 121 is the direction perpendicular to the conveying direction of the package) obtained from the first image; W is the distance from the point farthest from the center of the camera's field of view on the first virtual package along the width direction of the conveying surface 121 obtained from the first image to the center of the camera's field of view; H is the height of the second virtual package obtained from the second image. 相 The installation height of the 2D camera 131 of the first visual monitoring device 130; the first virtual package and the second virtual package are matched with each other, and they correspond to the same physical package on the conveyor surface 121, that is, Figure 6 The target package is 020.

[0113] Step S400: Perform strategy matching on the virtual packages in the first image and the virtual packages in the second image to determine the information of the physical packages on the transport surface.

[0114] In this embodiment, step S400, which involves performing strategy matching on the virtual packages in the first image and the second image to determine the information of the physical packages on the transport surface, specifically includes:

[0115] Determine whether the first number of virtual packages in the first image is equal to the second number of virtual packages in the second image;

[0116] If the first quantity and the second quantity are equal, a matching operation is performed on the virtual packages in the first image and the virtual packages in the second image to determine whether the virtual packages in the first image and the virtual packages in the second image can be completely matched. If they can be completely matched, the information of the physical packages on the conveying surface is determined based on the feature information of the virtual packages in the second image; otherwise, the information of the physical packages on the conveying surface is determined based on the feature information of the virtual packages in the first image.

[0117] If the first quantity and the second quantity are not equal, determine whether the first quantity is less than or equal to one piece. If the first quantity is less than or equal to one piece, determine the information of the physical package on the conveyor surface based on the feature information of the virtual package in the second image. Otherwise, determine the information of the physical package on the conveyor surface based on the feature information of the virtual package in the first image.

[0118] In this embodiment, the first quantity is the number of virtual packages in the first image, and the second quantity is the number of virtual packages in the second image. First, it is determined whether the first quantity of virtual packages in the first image and the second quantity of virtual packages in the second image are equal. If these two quantities are equal, a matching operation is performed on the virtual packages in the first image and the second image. If the virtual packages in the first image and the second image can be completely matched, it indicates that virtual packages corresponding to each physical package on the conveyor surface have been detected in both the first and second images, and the information of a physical package on the conveyor surface is basically consistent with the information of virtual packages in the two images. Since the feature information of virtual packages in the second image includes height data, in this case, the information of physical packages on the conveyor surface is determined based on the feature information of virtual packages in the second image. If the virtual packages in the first image and the second image cannot be completely matched, it indicates that although the number of virtual packages detected by the two images is the same, a false detection of virtual packages has occurred in at least one of the images. Because images captured by the 3D camera 141 are prone to failing to detect envelopes on the transport surface, and when physical packages are irregularly shaped, they are easily misdetected as multiple virtual packages (for example, if a physical package is shaped like an inverted stool, all "legs" of the "stool" might be detected as separate packages), and because images captured by the 2D camera 131 include data from the R, G, and B channels, meaning they contain richer information, images acquired by the 2D camera 131 are more likely to determine differences between adjacent pixels, especially for overlapping envelopes, which are more easily detected. Furthermore, since the size information of the virtual package in the first image is corrected based on the height data of the matching virtual package in the second image, the problem of inaccurate package information caused by the shooting angle of the 2D camera 131 is overcome. Therefore, the virtual package in the first image can more realistically reflect the state of the physical packages on the transport surface. Thus, when the virtual packages in the first image and the virtual packages in the second image do not perfectly match, the information of the physical packages on the transport surface is determined based on the feature information of the virtual packages in the first image. Similarly, when the first number of virtual packages in the first image and the second number of virtual packages in the second image are not equal, and the first number is greater than one, the information of the physical packages on the transport surface is also determined based on the feature information of the virtual packages in the first image for the same reason mentioned above.When the number of virtual packages differs between the two images, and no virtual packages are detected or only one virtual package is detected in the first image, it often indicates a missed detection when detecting virtual packages using the first image (e.g., two overlapping transported packages are very similar in color, and the upper package completely overlaps the lower package). In this case, the information of the physical packages on the transport surface is determined based on the feature information of the virtual packages in the second image. In this embodiment, by combining the feature information of the virtual packages in the first image determined based on the two-dimensional image and the feature information of the virtual packages in the second image determined based on the three-dimensional image, and comprehensively considering the advantages and disadvantages of the 2D camera 131 and the 3D camera 141, the information of the physical packages on the final transport surface is determined, thus improving the accuracy of package detection.

[0119] Steps S100 to S400 described above illustrate how to obtain information about physical packages on the conveyor surface. Based on this, the control device 110 can determine the status of the physical packages and perform corresponding processing according to the information. In an optional embodiment, after step S400, the package detection method may further include:

[0120] Based on the information of the physical packages, determine whether the distance between each physical package and its adjacent physical packages is less than a set value; if the distance between a physical package and its adjacent physical packages is less than the set value, determine that the physical package and its adjacent physical packages are abnormal packages of overlapping delivery and perform special processing on the abnormal packages.

[0121] Figure 7a and Figure 7b This diagram illustrates the spacing between two entity packages in two scenarios. For two adjacent entity packages, the distance between the lowest vertex of the upstream package and the highest vertex of the downstream package can be used as the spacing between the two entity packages, based on the determined information of the entity packages (see reference). Figure 7a Alternatively, based on the information of the defined entity packages, the vertex closest to the other entity package in each of the two entity packages can be determined, and the distance between these two vertices can be used as the spacing between the two entity packages (see reference). Figure 7b ).like Figure 7aAs shown, the conveying surface has a first physical package 021 located upstream and a second physical package 022 located downstream. The downstreammost vertex of the first physical package 021 is the first point A1, and the upstreammost vertex of the second physical package 022 is the second point A2. The positive direction of the x-axis in the world coordinate system is the same as the conveying direction of the packages, that is, the coordinate value of the downstream point is greater than the coordinate value of the upstream point. The distance between the first physical package 021 and the second physical package 022 can be obtained by subtracting the coordinate value of the second point A2 from the coordinate value of the first point A1. If this distance is less than a set value, it is determined that the first physical package 021 and the second physical package 022 are being conveyed overlappingly, that is, both the first physical package 021 and the second physical package 022 are abnormal packages being conveyed overlappingly. Optionally, the set value can be equal to zero or greater than zero. In another embodiment, as... Figure 7b As shown, the closest point between the first physical package 021 located upstream and the second physical package 022 located downstream is point A1, and the closest point between the second physical package 022 located downstream and the first physical package 021 located upstream is point A2. The distance between the first physical package 021 and the second physical package 022 can be obtained by subtracting the coordinates of point A2 from the coordinates of point A1. It should be understood that when determining two physical packages as abnormal packages of overlapping transport, the two packages are not necessarily actually overlapping vertically; there may still be a certain gap between them. However, the equipment determines that the two physical packages are too close, which would cause sorting abnormalities, and therefore classifies them as abnormal packages of overlapping transport.

[0122] In an optional implementation, the step of special handling of abnormal packages includes one of the following:

[0123] The abnormal package is separated from the stack; or the abnormal package is returned; or the abnormal package is transported to the unloading platform 410 for manual handling.

[0124] Since the logistics system 010 provided in this application embodiment includes a second import device 400, which includes a drop-off table 410 for manual processing, when the package processing device 100 detects that the package it is transporting is an abnormal package, the abnormal package can be guided to the second import device 400 by the swing wheel sorter 150, so that the abnormal package can enter the drop-off table 410 for manual processing (such as separating overlapping packages). After the package is restored to normal, it continues to be transported to the sorting device 200 through the second import device 400.

[0125] In another embodiment of the logistics system provided in this application, a package processing device and a stacking separation device are included. The package processing device is located upstream of the stacking separation device. When the package processing device detects that the package it is transporting is an abnormal package, it can send a package abnormality signal to the stacking separation device so that the stacking separation device can perform overlapping package separation processing when the abnormal package arrives. In yet another embodiment of the logistics system provided in this application, a package processing device is included, which also has the function of separating overlapping packages. When it detects that the package it is transporting is an abnormal package, the package processing device performs overlapping package separation processing. In yet another embodiment of the logistics system provided in this application, a package processing device, a stacking separation device, and a return device are included. The inlet of the return device is connected to the outlet of the package processing device, and the outlet of the return device is connected to the inlet of the stacking separation device. When the package processing device detects that the package it is transporting is an abnormal package, it transports the abnormal package to the return device so that the abnormal package can be re-transported to the stacking separation device, thereby performing package separation processing again through the stacking separation device.

[0126] In summary, the package detection method of this application embodiment detects virtual packages included in the first image acquired by the first visual monitoring device 130 and determines the feature information of each virtual package, and detects virtual packages included in the second image acquired by the second visual monitoring device 140 and determines the feature information of each virtual package. The feature information of the virtual packages in the first image is then corrected based on the feature information of the virtual packages in the second image, and the information of the physical packages on the transport surface is determined by strategy matching between the virtual packages in the first image and the virtual packages in the second image. Therefore, the package detection method provided by this application can combine the feature information of virtual packages obtained based on two-dimensional images and the feature information of virtual packages obtained based on three-dimensional images to determine the information of physical packages on the transport surface, overcoming the problem of missed package detection caused by the low height of packages when using the 3D camera 141 alone, and improving the accuracy of package detection.

[0127] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A parcel detection method, applied to parcel processing equipment, characterized in that, The package handling equipment includes a conveying device and a first visual monitoring device and a second visual monitoring device disposed above the conveying device. The conveying device is used to convey packages along a predetermined direction and has a conveying surface for carrying and conveying packages. The first visual monitoring device is used to acquire a two-dimensional image of the conveying surface of the conveying device, and the second visual monitoring device is used to acquire a three-dimensional image of the conveying surface of the conveying device. The package detection method includes: A first image is determined from the images acquired by the first visual monitoring device, and a second image is determined from the images acquired by the second visual monitoring device, wherein the first image and the second image are images acquired at the same time. Detecting virtual packages in the first image and determining the feature information of each virtual package, and detecting virtual packages in the second image and determining the feature information of each virtual package; The feature information of the virtual package in the first image is corrected based on the feature information of the virtual package in the second image; Strategy matching is performed on the virtual packages in the first image and the virtual packages in the second image to determine the information of the physical packages on the transport surface; The step of detecting virtual packages in the first image and determining the feature information of each virtual package includes: The object detection algorithm is used to detect virtual packages in the first image and obtain coarse information for each virtual package, wherein the bounding box of the object detection algorithm is used to describe the virtual package in the coarse information of the virtual package. Based on the rough information of each virtual package, it is determined whether the virtual package is a first type of virtual package or a second type of virtual package, wherein the first type of virtual package is a virtual package that is prone to false detection, and the second type of virtual package is a virtual package that is not prone to false detection. For each virtual package of the first type, the bounding box describing the virtual package is detected based on the object segmentation algorithm to obtain the virtual package within the bounding box and the precise information of each virtual package, wherein the precise information of the virtual package uses the outline of the virtual package to describe the virtual package; The precise information of the first type of virtual package is determined as the feature information of the first type of virtual package, and the rough information of the second type of virtual package is determined as the feature information of the second type of virtual package.

2. The package detection method according to claim 1, characterized in that, The step of determining whether a virtual package is a first type or a second type based on the rough information of each virtual package includes: Based on the rough information of the virtual package, determine whether the virtual package is a virtual package where similar objects are stacked; When the virtual package is determined to be a virtual package where similar objects are stacked, the virtual package is classified as a virtual package of the first type; when the virtual package is determined not to be a virtual package where similar objects are stacked, the virtual package is classified as a virtual package of the second type.

3. The package detection method according to claim 1, characterized in that, The step of correcting the feature information of the virtual package in the first image based on the feature information of the virtual package in the second image includes: A matching operation is performed on the virtual packages in the first image and the virtual packages in the second image to determine the virtual packages in the first image that can be matched and those that cannot be matched. For a virtual package that can be matched in the first image, the length and / or width of the virtual package is corrected based on the height data of the virtual package that matches it in the second image.

4. The package detection method according to claim 1, characterized in that, The step of performing strategy matching on the virtual packages in the first image and the virtual packages in the second image to determine the information of the physical packages on the transport surface includes: Determine whether the first number of virtual packages in the first image is equal to the second number of virtual packages in the second image; If the first quantity and the second quantity are equal, a matching operation is performed on the virtual packages in the first image and the virtual packages in the second image to determine whether the virtual packages in the first image and the virtual packages in the second image can be completely matched. If they can be completely matched, the information of the physical packages on the conveying surface is determined based on the feature information of the virtual packages in the second image; otherwise, the information of the physical packages on the conveying surface is determined based on the feature information of the virtual packages in the first image. If the first quantity and the second quantity are not equal, determine whether the first quantity is less than or equal to one piece. If the first quantity is less than or equal to one piece, determine the information of the physical package on the conveyor surface based on the feature information of the virtual package in the second image. Otherwise, determine the information of the physical package on the conveyor surface based on the feature information of the virtual package in the first image.

5. The package detection method according to claim 1, characterized in that, The package detection method also includes: Based on the information of the physical packages, determine whether the distance between each physical package and its adjacent physical packages is less than a set value; If the distance between a physical package and its adjacent physical package is less than the set value, the physical package and its adjacent physical package are determined to be abnormal packages of overlapping delivery and the abnormal packages are handled in a special way.

6. The package detection method according to claim 5, characterized in that, The steps for special handling of the abnormal package include one of the following: Separate the stacked items from the abnormal packages; or, Return the abnormal package; or, The abnormal package is transported to the unloading station for manual handling.

7. A parcel processing device, characterized in that, The system includes a control device, a conveying device, and a first visual monitoring device and a second visual monitoring device disposed above the conveying device. The conveying device is used to convey packages along a predetermined direction and has a conveying surface for carrying and conveying packages. The first visual monitoring device is used to acquire two-dimensional images of the conveying surface of the conveying device, and the second visual monitoring device is used to acquire three-dimensional images of the conveying surface of the conveying device. The conveying device, the first visual monitoring device, and the second visual monitoring device are all electrically connected to the control device, which is configured as follows: A first image is determined from the images acquired by the first visual monitoring device, and a second image is determined from the images acquired by the second visual monitoring device, wherein the first image and the second image are images acquired at the same time. Detecting virtual packages in the first image and determining the feature information of each virtual package, and detecting virtual packages in the second image and determining the feature information of each virtual package; The feature information of the virtual package in the first image is corrected based on the feature information of the virtual package in the second image; Strategy matching is performed on the virtual packages in the first image and the virtual packages in the second image to determine the information of the physical packages on the transport surface; The control device detects virtual packages in the first image and determines the feature information of each virtual package in the following manner: The object detection algorithm is used to detect virtual packages in the first image and obtain coarse information for each virtual package, wherein the bounding box of the object detection algorithm is used to describe the virtual package in the coarse information of the virtual package. Based on the rough information of each virtual package, it is determined whether the virtual package is a first type of virtual package or a second type of virtual package, wherein the first type of virtual package is a virtual package that is prone to false detection, and the second type of virtual package is a virtual package that is not prone to false detection. For each virtual package of the first type, the bounding box describing the virtual package is detected based on the object segmentation algorithm to obtain the virtual package within the bounding box and the precise information of each virtual package, wherein the precise information of the virtual package uses the outline of the virtual package to describe the virtual package; The precise information of the first type of virtual package is determined as the feature information of the first type of virtual package, and the rough information of the second type of virtual package is determined as the feature information of the second type of virtual package.

8. The parcel processing equipment according to claim 7, characterized in that, The control device corrects the feature information of the virtual package in the first image based on the feature information of the virtual package in the second image in the following manner: A matching operation is performed on the virtual packages in the first image and the virtual packages in the second image to determine the virtual packages in the first image that can be matched and those that cannot be matched. For a virtual package that can be matched in the first image, the length and / or width of the virtual package is corrected based on the height data of the virtual package that matches it in the second image.

Images