Parcel detection method and parcel handling apparatus

Abstract

The application provides a parcel detection method and a parcel processing device, and relates to the technical field of logistics.In the parcel detection method of the application, virtual parcel detection is performed based on a single image obtained by each camera, and a reference virtual parcel corresponding to each entity parcel on a conveying surface is obtained through deduplication processing of the virtual parcel, so that the entity parcel on the conveying surface is determined according to the reference virtual parcel.The method can save the process of splicing multiple images, and the parcel detection method of the embodiment of the application can reduce the data operation amount during parcel detection, shorten the operation time of parcel detection, improve the parcel detection efficiency, and further improve the accuracy of parcel detection and sorting.The parcel processing device provided by the application can implement the above-mentioned parcel detection method, and thus has corresponding beneficial effects.

Description

Package inspection methods and package handling equipment Technical Field

[0001] This application relates to the field of logistics technology, and more specifically, to a parcel detection method and parcel processing equipment. Background Technology

[0002] Currently, in the domestic logistics industry, with the surge in parcel volume, more and more logistics companies are deploying automated sorting systems in their sorting areas. Automated sorting systems include conveying mechanisms and identification mechanisms. The conveying mechanism transports parcels according to the parcel information identified by the identification mechanism and follows a set strategy to complete the automated sorting of parcels. Compared with traditional manual sorting methods, using automated sorting systems for parcel sorting greatly improves sorting efficiency and reduces the labor costs of logistics companies.

[0003] In automated sorting systems, accurate package location detection is crucial for proper sorting. Existing automated sorting systems typically use cameras positioned above conveyor belts to detect the location of packages being transported. Related technologies disclose a method for detecting package location using camera images in automated sorting systems, which involves stitching together package images acquired by multiple cameras and detecting the package's location information based on the stitched image. However, this method involves a large amount of data processing and a relatively long processing time, potentially leading to a discrepancy between the detected package location and the actual location of the package, thus affecting the accuracy of package sorting. Therefore, the package location detection methods in related automated sorting systems suffer from inaccurate package location detection, which in turn affects the accuracy of package sorting. Summary of the Invention

[0004] The purpose of this application includes providing a package detection method and a package processing device that can accurately detect the location of packages, thereby improving the accuracy of package sorting.

[0005] The embodiments of this application can be implemented as follows:

[0006] In a first aspect, this application provides a package detection method applied to a package processing device. The package processing device includes a conveying mechanism and multiple cameras disposed above the conveying mechanism. The conveying mechanism has a conveying surface for conveying packages, and the conveying surface includes multiple sub-regions. The multiple cameras correspond one-to-one with the multiple sub-regions of the conveying surface, and each camera is used to capture an image of its corresponding sub-region. Adjacent sub-regions partially overlap. The package detection method includes:

[0007] Multiple images from multiple cameras at the same time are acquired, and each image is detected to identify the virtual package in each image;

[0008] The second coordinate position of the virtual package in the world coordinate system is determined based on the first coordinate position of each virtual package in the corresponding image coordinate system. The image coordinate system is used to describe the position of the pixel in the image, and the world coordinate system is used to describe the position of the object on the transport surface.

[0009] Based on the second coordinate position of each virtual package, the virtual packages are deduplicated to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface.

[0010] The physical packages on the transport surface are determined based on the reference virtual packages.

[0011] In an optional implementation, the virtual packages are deduplicated based on their second coordinate position to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface. Specifically, this includes:

[0012] Based on the second coordinate position of each virtual package, the virtual packages that fall into the overlapping area and the virtual packages that fall into the non-overlapping area are counted. The overlapping area refers to the area on the conveyor surface that belongs to at least two sub-areas, and the non-overlapping area refers to the area on the conveyor surface that belongs to only one sub-area.

[0013] The virtual packages in each overlapping region are deduplicated, and the remaining virtual packages in each overlapping region after deduplication and the virtual packages that fall into the non-overlapping region are determined as reference virtual packages.

[0014] In an optional implementation, the overlapping and non-overlapping regions are determined according to the following steps:

[0015] Multiple cameras are used to capture images of marker samples pre-set on the conveyor surface, wherein multiple detection units are arranged in multiple rows and columns in the marker samples;

[0016] Based on the images captured by each camera, determine the areas on the conveyor surface that were repeatedly photographed and those that were not.

[0017] Areas that are photographed repeatedly are defined as overlapping areas, and areas that are not photographed repeatedly are defined as non-overlapping areas.

[0018] In an optional implementation, deduplication is performed on the virtual packages in each overlapping region, specifically including:

[0019] Step S01: Determine multiple combinations of virtual packages in the overlapping area, wherein each combination includes two virtual packages;

[0020] Step S02: Determine a target combination from multiple combinations according to a set order;

[0021] Step S03: Process the target combination to determine whether the two virtual packages in the target combination can match. If the two virtual packages can match, proceed to step S04; otherwise, proceed to step S05.

[0022] Step S04: Delete one of the two virtual packages, and then proceed to step S05;

[0023] Step S05: Determine if there are any unprocessed combinations; if there are, execute step S02 again; otherwise, the process ends.

[0024] In an optional implementation, step S04 further includes:

[0025] After deleting one of the two virtual packages, all combinations containing the deleted virtual package will be deleted.

[0026] In an optional implementation, multiple combinations of virtual packages in the overlapping area are determined, including:

[0027] Based on the statistics of virtual packages falling into the overlapping area, N combinations are determined by pairwise correspondence of the virtual packages.

[0028] In an optional implementation, multiple combinations of virtual packages in the overlapping area are determined, including:

[0029] Based on the correspondence between virtual packages and cameras, the virtual packages that fall into the overlapping area are divided into P groups, where each group of virtual packages corresponds to one camera.

[0030] Determine Q combinations of virtual packages in group P, where each combination consists of two virtual packages, and the two virtual packages in each combination belong to different groups.

[0031] In an optional implementation, deleting one of the two virtual packages includes:

[0032] Get the area occupied by each virtual package in the corresponding overlapping area, and compare the size of the area occupied by two virtual packages in the corresponding overlapping area;

[0033] Delete the virtual package that occupies the least space.

[0034] In an optional implementation, the target combination is processed to determine whether the two virtual packages in the target combination can match, including:

[0035] Determine the baseline virtual package from the two virtual packages of the target combination;

[0036] Determine whether the baseline virtual package is located on the boundary of the overlapping area;

[0037] When the reference virtual package is located on the boundary of the overlapping area, the first formula is used to calculate the matching degree of the two virtual packages in the target combination. When the reference virtual package is not located on the boundary of the overlapping area, the second formula is used to calculate the matching degree of the two virtual packages in the target combination.

[0038] Determine whether the matching degree of the two virtual packages in the target combination is greater than a preset value. If so, determine that the two virtual packages in the target combination can match; otherwise, determine that the two virtual packages in the target combination cannot match.

[0039] in,

[0040] The first formula is:

[0041] The second formula is:

[0042] th represents the matching degree between the two virtual packages, x1 and y1 represent the coordinates of the first virtual package in the world coordinate system, x2 and y2 represent the coordinates of the second virtual package in the world coordinate system, w1 represents the width of the first virtual package, w2 represents the width of the second virtual package, l1 represents the length of the first virtual package, and l2 represents the length of the second virtual package.

[0043] Secondly, this application provides a parcel processing device, which includes a control device, a conveying mechanism, and multiple cameras disposed above the conveying mechanism. The conveying mechanism has a conveying surface for conveying parcels, and the conveying surface includes multiple sub-regions. The multiple cameras correspond one-to-one with the multiple sub-regions of the conveying surface, and each camera is used to capture an image of its corresponding sub-region. Adjacent sub-regions partially overlap. All the multiple cameras are electrically connected to the control device, which is configured as follows:

[0044] Multiple images from multiple cameras at the same time are acquired, and each image is detected to identify the virtual package in each image;

[0045] The second coordinate position of the virtual package in the world coordinate system is determined based on the first coordinate position of each virtual package in the corresponding image coordinate system. The image coordinate system is used to describe the position of the pixel in the image, and the world coordinate system is used to describe the position of the object on the transport surface.

[0046] Based on the second coordinate position of each virtual package, the virtual packages are deduplicated to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface.

[0047] The physical packages on the transport surface are determined based on the reference virtual packages.

[0048] The beneficial effects of the embodiments of this application include:

[0049] In the package detection method provided in this application embodiment, virtual package detection is performed based on a single image acquired by each camera. The first coordinate position of each virtual package acquired by the single image is uniformly converted into a second coordinate position in the world coordinate system. Based on the second coordinate position of each virtual package, the virtual packages are deduplicated to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface. The physical packages on the conveyor surface are then determined based on the reference virtual packages, thereby determining the position information of the physical packages on the conveyor surface. The package detection method of this application embodiment can eliminate the process of stitching multiple images. Compared with the stitched image obtained by stitching multiple images, the size of a single image acquired by a single camera is smaller and the amount of data is lower. Therefore, the amount of data computation when detecting a package from a single image will be much less than the amount of data computation when detecting a package from a stitched image. Furthermore, since the amount of data of a package is much smaller than the amount of data of an image, the amount of data computation for deduplication of virtual packages will also be much less than the amount of data computation when performing image fusion on the overlapping areas of multiple images. Therefore, the package detection method of this application embodiment can reduce the amount of data computation during package detection, shorten the computation time of package detection, thereby improving package detection efficiency and thus improving the accuracy of package detection and sorting.

[0050] The package processing equipment provided in this application embodiment can implement the above-mentioned package detection method, and therefore also has corresponding beneficial effects. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 is a block diagram of a package processing device in one embodiment of this application;

[0053] Figure 2 is a flowchart of a package detection method in one embodiment of this application;

[0054] Figure 3 is a flowchart of step S300 in the embodiment of Figure 2;

[0055] Figure 4 is a schematic diagram of a marked sample in one embodiment of this application;

[0056] Figure 5 is a flowchart of the deduplication process for virtual packages in each overlapping area in one embodiment of this application;

[0057] Figures 6a to 6e are schematic diagrams of the combination of virtual packages under different quantities;

[0058] Figure 7 is a schematic diagram of grouping and then combining all virtual packages falling into the overlapping area in one embodiment of this application;

[0059] Figure 8 is a block diagram of a package processing device in one embodiment of this application.

[0060] Icons: 010-Package handling equipment; 100-Conveying mechanism; 200-Camera; 300-Control device; 400-Bus; 500-Memory; 020-Virtual package. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0063] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0064] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0065] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0066] Currently, automated sorting systems typically place cameras above the conveyor mechanism to detect the position of packages being transported by the conveyor. Related automated sorting systems use stitched images from multiple cameras to detect package position. However, automated sorting systems usually include multiple different devices, such as stacking separation devices, single-item separation devices, and sorting devices. Each device has multiple cameras above its conveyor mechanism. When using related methods to detect package position, especially with a large number of cameras (e.g., a dozen, dozens, or even hundreds), the resulting stitched image data is often very large. Therefore, the data processing required to detect packages from the stitched image is very demanding and time-consuming. Since automated sorting systems have high requirements for the timeliness of package position detection, if the processing time for package position detection is long, the final detected package position may not match the actual position of the package, which could affect the accuracy of package sorting.

[0067] To address the issue of inaccurate package location detection in the aforementioned related technologies, which affects package sorting accuracy, this application provides a package detection method. This method transforms the first coordinate position of each virtual package obtained from a single image into a second coordinate position in the world coordinate system. Based on the second coordinate position of each virtual package, duplicate virtual packages are deduplicated to obtain reference virtual packages that correspond one-to-one with physical packages on the conveyor surface. The physical packages on the conveyor surface are then determined based on these reference virtual packages, thereby determining their position information. This application reduces the amount of data computation during package detection, shortens the computation time, and thus improves package detection efficiency, ultimately enhancing the accuracy of package detection and sorting.

[0068] The package detection method provided in this application is applied to a package processing device. Figure 1 is a block diagram of the package processing device 010 in one embodiment of this application. As shown in Figure 1, the package processing device 010 includes a control device 300, a conveying mechanism 100, and multiple cameras 200 disposed above the conveying mechanism 100. The conveying mechanism 100 has a conveying surface for conveying packages, and the conveying surface includes multiple sub-regions. The multiple cameras 200 correspond one-to-one with the multiple sub-regions of the conveying surface, and each camera 200 is used to capture an image of the corresponding sub-region. Adjacent sub-regions partially overlap. The conveying mechanism 100 and the multiple cameras 200 are all electrically connected to the control device 300. It should be understood that the conveying mechanism 100 may include various mechanisms capable of realizing conveying functions, such as a stacking separation mechanism, a single-item separation mechanism, a sorting mechanism, etc.

[0069] In this embodiment of the application, the control device 300 is configured as follows:

[0070] Multiple images from multiple cameras at the same time are acquired. Each image is inspected to identify virtual packages. Based on the first coordinate position of each virtual package in the corresponding image coordinate system, the second coordinate position of the virtual package in the world coordinate system is determined. The image coordinate system describes the position of pixels in the image, and the world coordinate system describes the position of the object on the conveyor surface. The virtual packages are deduplicated based on their second coordinate positions to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface. The physical packages on the conveyor surface are then determined based on these reference virtual packages. Specifically, determining the physical packages on the conveyor surface includes identifying their information, such as position, outline, size, and status.

[0071] Figure 2 is a flowchart of a package detection method in one embodiment of this application. This detection method can be implemented by a control device of the package processing equipment provided in this embodiment. As shown in Figure 2, the package detection method provided in this embodiment includes:

[0072] Step S100: Acquire multiple images from multiple cameras at the same time, detect each image, and determine the virtual package in each image.

[0073] Taking the parcel processing device 010 provided in this application as an example, the control device 300 acquires images of multiple sub-regions on the conveyor mechanism 100 through multiple cameras 200. Optionally, the control device 300 of the parcel processing device 010 controls each camera 200 to capture an image of the corresponding sub-region at set intervals (e.g., 1ms, 2ms), and obtains an image captured by each camera 200 at a set time, thereby acquiring multiple images from multiple cameras 200 at the same time. In the embodiments of this application, for each image, a known method of the prior art (such as deep learning method, background modeling method) can be used to process the image to detect virtual parcels present in the image. Virtual parcels are represented as graphics with a certain outline in the image.

[0074] Step S200: Determine the second coordinate position of the virtual package in the world coordinate system based on the first coordinate position of each virtual package in the corresponding image coordinate system. The image coordinate system is used to describe the position of the pixel in the image, and the world coordinate system is used to describe the position of the object on the conveying surface.

[0075] Taking the parcel processing device 010 provided in this application as an example, the first coordinate position of each virtual parcel in the image coordinate system corresponding to its image can be determined using known methods in the prior art. In this embodiment, each image has a corresponding image coordinate system, which is a coordinate system established with reference to the image captured by the camera 200, used to describe the position of pixels in the image. Optionally, the image coordinate system is a two-dimensional coordinate system, with a set point in the image as the origin, and two mutually perpendicular directions in the image as the x-axis and y-axis directions. For example, if the image captured by the camera 200 is rectangular, the image coordinate system can be selected with the center point of the image as the origin, and the extension directions of the two adjacent sides of the image as the x-axis and y-axis directions, respectively. Of course, if the camera 200 is a 3D camera, the image coordinate system can also be a three-dimensional coordinate system. After detecting a virtual parcel in the image, the control device 300 calculates the information of each detected virtual parcel, wherein the information of the virtual parcel includes the first coordinate position of the virtual parcel in the image coordinate system, the outline of the virtual parcel, the size of the virtual parcel, and other information. Optionally, the first coordinate position of the virtual package in the image coordinate system can be described by the center point of the virtual package, or by setting the coordinate values ​​of the vertices in the image coordinate system, such as using the coordinate position of the minimum x-coordinate value of the virtual package as the first coordinate position of the virtual package.

[0076] After determining the first coordinate position of the virtual package in the corresponding image coordinate system, a second coordinate position of the virtual package in the world coordinate system is determined based on this first coordinate position. The world coordinate system is an absolute coordinate system established with reference to the conveying surface of the conveying mechanism 100, used to describe the position of the object on the conveying surface. Optionally, the world coordinate system is a two-dimensional coordinate system, with a set point on the conveying surface as the origin, and two mutually perpendicular directions on the conveying surface as the x-axis and y-axis directions. For example, the world coordinate system may have a set point on the conveying surface as the origin, the package conveying direction as the x-axis, and the width direction of the conveying surface as the y-axis. Of course, if the camera 200 is a 3D camera and the image coordinate system is a three-dimensional coordinate system, the world coordinate system can also be a three-dimensional coordinate system.

[0077] Optionally, determining the second coordinate position of the virtual package in the world coordinate system based on the first coordinate position of each virtual package in the corresponding image coordinate system includes: obtaining the coordinate transformation matrix corresponding to the image coordinate system, wherein the coordinate transformation matrix is ​​used to convert the coordinate values ​​in the image coordinate system to the coordinate values ​​in the world coordinate system; and calculating the second coordinate position of the virtual package in the world coordinate system based on the first coordinate position of the virtual package in the corresponding image coordinate system and the coordinate transformation matrix.

[0078] Optionally, in this embodiment, each camera 200 corresponds to a coordinate transformation matrix. The memory 500 of the package processing device 010 pre-stores the coordinate transformation matrix corresponding to each camera 200. The image captured by each camera 200 is converted and calculated using the coordinate transformation matrix corresponding to that camera 200, thereby obtaining the second coordinate position of the virtual package based on the first coordinate position of the virtual package. The coordinate transformation matrix corresponding to the camera 200 is related to the position of the sub-region corresponding to that camera 200 on the conveyor surface and the shooting angle. Optionally, the package detection method in this embodiment further includes: pre-calibrating the image captured by each camera 200 using a preset checkerboard grid to obtain the coordinate transformation matrix corresponding to that camera 200. For example, a preset checkerboard pattern is set on a sub-region on the conveyor surface corresponding to a camera 200. The coordinates of each corner point in the checkerboard pattern in the world coordinate system are measured. An image of the sub-region is captured by the camera 200, and the coordinates of each corner point in the image coordinate system are detected. The coordinate mapping relationship of each corner point is obtained through the coordinates of each corner point in the two coordinate systems. The coordinate transformation matrix corresponding to the camera 200 is calculated using the coordinate mapping relationship of multiple corner points. Here, a corner point is a pixel that differs significantly from a sufficiently large number of its surrounding pixels.

[0079] Step S300: Based on the second coordinate position of each virtual package, perform deduplication on the virtual packages to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface.

[0080] Optionally, after determining the second coordinate position of the virtual package in the world coordinate system, this embodiment of the application further calculates the contour and size of the virtual package in the world coordinate system based on the contour and size of the virtual package in the image and the coordinate transformation matrix, and performs deduplication processing on the virtual packages based on the position and contour of each virtual package in the world coordinate system to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveying surface.

[0081] Step S400: Determine the physical package on the conveyor surface based on the reference virtual package.

[0082] After obtaining reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface, the information of the corresponding physical packages on the conveyor surface can be obtained based on the information of each reference virtual package. The information of the physical packages includes their position, outline, size, and status. Specifically, the position of the corresponding physical package on the conveyor surface is determined based on the second coordinate position of the reference virtual package in the world coordinate system; the outline and size of the corresponding physical package are determined based on the outline and size of the reference virtual package in the corresponding image and the corresponding scaling ratio of the image; and the status of the corresponding physical package is determined based on the status of the reference virtual package. Optionally, the status of the package includes whether the packages are being conveyed overlapping and whether the packages are tilted relative to the conveyor surface.

[0083] Optionally, in this embodiment, a package queue is used to record information about physical packages on the conveyor surface. After determining the physical packages on the conveyor surface, the information of each physical package is added to the package queue. The control device 300 then performs status detection, stack separation, and single-item separation on the packages based on the information of the physical packages recorded in the package queue.

[0084] In the package detection method of this application embodiment, virtual package detection is performed based on a single image acquired by each camera 200. The first coordinate position of each virtual package acquired through a single image is uniformly converted into a second coordinate position in the world coordinate system. Based on the second coordinate position of each virtual package, deduplication processing is performed on the virtual packages to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface. The physical packages on the conveyor surface are then determined based on the reference virtual packages, thereby determining the position information of the physical packages on the conveyor surface. This application embodiment eliminates the process of stitching multiple images. Compared to the stitched image obtained by stitching multiple images, the single image acquired by a single camera 200 is smaller in size and has a lower data volume. Therefore, the data computation amount when detecting packages from a single image is much less than that when detecting packages from a stitched image. Furthermore, since the data volume of a package is much smaller than the data volume of an image, the data computation amount for deduplication of virtual packages is also much less than the data computation amount when performing image fusion on the overlapping areas of multiple images. Therefore, through the embodiments of this application, the amount of data computation during package detection can be reduced, the computation time of package detection can be shortened, thereby improving the efficiency of package detection and thus improving the accuracy of package detection and sorting.

[0085] Furthermore, Figure 3 is a flowchart of step S300 in the embodiment of Figure 2. As shown in Figure 3, step S300 in this embodiment may specifically include:

[0086] Step S310: Based on the second coordinate position of each virtual package, count the virtual packages that fall into the overlapping area and the virtual packages that fall into the non-overlapping area. The overlapping area refers to the area on the conveyor surface that belongs to at least two sub-areas, and the non-overlapping area refers to the area on the conveyor surface that belongs to only one sub-area.

[0087] Step S320: Perform deduplication on the virtual packages in each overlapping region, and determine the remaining virtual packages in each overlapping region after deduplication and the virtual packages that fall into the non-overlapping region as reference virtual packages.

[0088] In this embodiment, the memory 500 of the package processing device 010 pre-stores the coordinate range of each overlapping region in the world coordinate system and the coordinate range of each non-overlapping region in the world coordinate system. After obtaining the second coordinate position of the virtual package in the world coordinate system, the virtual package is determined to fall into the overlapping region or the non-overlapping region based on the second coordinate position of each virtual package, the outline of the virtual package, and the coordinate range of each overlapping region and the non-overlapping region. Optionally, when it is determined from the outline of the virtual package that part of the virtual package is located in the overlapping region and part in the non-overlapping region, the virtual package is considered to have fallen into the overlapping region.

[0089] In this embodiment, virtual packages in each overlapping region are deduplicated so that the remaining reference virtual packages in each overlapping region correspond to different physical packages on the conveyor surface. For virtual packages falling into non-overlapping regions, these virtual packages are directly identified as reference virtual packages corresponding to physical packages on the conveyor surface. Because this embodiment divides the conveyor surface into overlapping and non-overlapping regions, and only deduplicates virtual packages falling into overlapping regions are processed, while virtual packages falling into non-overlapping regions are directly identified as reference virtual packages corresponding to multiple physical packages, the amount of data computation required for deduplicating virtual packages is greatly reduced, thereby further shortening the computation time for package detection and improving the efficiency of package detection.

[0090] In an optional embodiment of this application, the overlapping and non-overlapping regions are determined according to the following steps:

[0091] Multiple cameras 200 are used to capture images of pre-set marker samples on the conveyor surface, wherein multiple detection units are arranged in multiple rows and columns in the marker samples; based on the images captured by each camera 200, the areas on the conveyor surface that are repeatedly captured and those that are not repeatedly captured are determined; the areas that are repeatedly captured are determined as overlapping areas, and the areas that are not repeatedly captured are determined as non-overlapping areas.

[0092] Figure 4 is a schematic diagram of a marker sample in one embodiment of this application. The marker sample can adopt a preset checkerboard pattern as shown in Figure 4. The preset checkerboard pattern includes multiple squares arranged in M ​​rows and N columns, and each detection unit is one square in the preset checkerboard pattern. Optionally, in this embodiment, each detection unit of the marker sample is provided with a unique identifier, and the marker sample can cover multiple sub-regions of the conveying surface. For example, the marker sample can cover the entire conveying surface. After capturing images of the marker sample using multiple cameras 200, the detection units that are repeatedly captured and those that are not repeatedly captured in the preset checkerboard pattern are determined according to the number of detection units and identifiers included in the images captured by each camera 200. The area where the repeatedly captured detection units are located is determined as the overlapping area, and the area where the non-repeated detection units are located is determined as the non-overlapping area. In other schemes, the detection unit of the marked sample does not need to be marked. The conveying surface is divided into multiple square basic units, and the marked sample is a checkerboard with the same area as the basic unit. The marked sample is set in each basic unit in turn, and multiple cameras 200 are controlled to capture images simultaneously. If the images captured by multiple cameras 200 all contain the marked sample, then the basic unit where the marked sample is located is an overlapping area or part of an overlapping area. If only one camera 200 captures an image containing the marked sample, then the basic unit corresponding to the marked sample is a non-overlapping area or part of a non-overlapping area. After the above operation traverses all basic units, multiple interconnected basic units belonging to the overlapping area are combined into one overlapping area, and multiple interconnected basic units belonging to the non-overlapping area are combined into one non-overlapping area.

[0093] Figure 5 is a flowchart of the deduplication process for virtual packages in each overlapping region according to one embodiment of this application. As shown in Figure 5, the deduplication process for virtual packages in each overlapping region includes the following steps:

[0094] Step S01: Determine multiple combinations of virtual packages in the overlapping area, wherein each combination includes two virtual packages;

[0095] Step S02: Determine a target combination from multiple combinations according to a set order;

[0096] Step S03: Process the target combination to determine whether the two virtual packages in the target combination can match. If the two virtual packages can match, proceed to step S04; otherwise, proceed to step S05.

[0097] Step S04: Delete one of the two virtual packages, and then proceed to step S05;

[0098] Step S05: Determine if there are any unprocessed combinations; if there are, execute step S02 again; otherwise, the process ends.

[0099] In this embodiment, the virtual packages in each overlapping area are paired up, and it is determined whether the two virtual packages in each pair can match. When it is determined that the two virtual packages can match, it means that the two virtual packages correspond to a physical package in the overlapping area. Therefore, in this case, one of the virtual packages is deleted. In this way, after deduplication, the remaining virtual packages in each overlapping area do not match each other, and the remaining virtual packages in each overlapping area are reference virtual packages that correspond one-to-one with the actual physical packages existing in the overlapping area.

[0100] Optionally, in this embodiment, multiple virtual packages in each overlapping area are numbered. After determining the various combinations of virtual packages in an overlapping area, a virtual package is selected as the target virtual package in ascending or descending order of its virtual package number. Then, the combinations containing the target virtual package are sequentially determined as target combinations. For example, virtual package number 1 is first determined as the target virtual package, then all combinations containing virtual package number 1 are sequentially determined as target combinations, then virtual package number 2 is determined as the target package, and then all combinations containing virtual package number 2 that have not yet been processed to determine whether the two virtual packages match are sequentially determined as target combinations. In this way, each virtual package can be sequentially checked for matching with other virtual packages according to its number, avoiding inaccurate package detection due to missed virtual packages.

[0101] Furthermore, step S04 may also include: after deleting one of the two virtual packages, deleting all combinations containing the deleted virtual package.

[0102] In this embodiment, when two virtual packages in a combination match, not only is one virtual package deleted, but all combinations related to the deleted virtual package are also deleted. This eliminates the need to subsequently check if the two virtual packages in the deleted combination match, avoiding duplicate or invalid checks that might occur when checking each determined combination individually. Therefore, this embodiment significantly reduces the computational workload of deduplicating virtual packages in overlapping areas, shortens the time spent on deduplication, and improves package detection efficiency.

[0103] In an optional embodiment, step S01, determining multiple combinations of virtual packages in the overlapping area, may specifically include: based on the count of virtual packages falling into the overlapping area, determining N combinations by pairwise matching of the virtual packages. Here, N = M(M-1) / 2, where M is the count of virtual packages falling into the overlapping area, and M is an integer greater than or equal to 2. It can be understood that the N combinations determined in the above steps are the N ways to combine two virtual packages together from all virtual packages falling into the overlapping area. Figures 6a to 6e are schematic diagrams of the combination methods of virtual package 020 under different quantities. In Figures 6a to 6e, a box represents a virtual package 020 falling into the overlapping area, and each line connecting two boxes represents a combination method. As shown in Figures 6a to 6e, if the number of virtual packages 020 falling into the overlapping area is 2, then there is only one combination, that is, combining these two virtual packages 020 together; if the number of virtual packages 020 falling into the overlapping area is 3, then there are three combinations; if the number of virtual packages 020 falling into the overlapping area is 4, then there are six combinations; if the number of virtual packages 020 falling into the overlapping area is 5, then there are ten combinations; if the number of virtual packages 020 falling into the overlapping area is 6, then there are fifteen combinations. Therefore, the number of combinations N can be calculated according to the combination formula C(M, 2), which means the number of ways to select 2 virtual packages 020 from M virtual packages 020 to form a combination, and C(M, 2) is equal to M! / [(M-2)! * 2! ], that is, M(M-1) / 2.

[0104] The method described above for determining multiple combinations of virtual packages 020 in an overlapping region pairs all virtual packages 020 falling into the overlapping region together. This means that in some combinations, two virtual packages 020 are actually captured by the same camera 200, i.e., they exist in the same image. These two virtual packages 020 cannot possibly correspond to the same physical package 020. Comparing the two virtual packages 020 in such combinations to determine if they match would waste computational resources. Therefore, in optional embodiments, determining multiple combinations of virtual packages 020 in an overlapping region may include:

[0105] Based on the correspondence between virtual package 020 and camera 200, the virtual packages 020 that fall into the overlapping area are divided into P groups, where each group of virtual packages 020 corresponds to one camera 200.

[0106] Determine Q combinations of virtual packages 020 in group P, where each combination includes two virtual packages 020, and the two virtual packages 020 in each combination belong to different groups.

[0107] It is understandable that the number of combinations mentioned above... c j Let c be the number of virtual packages 020 in the j-th group. k Let P be the number of virtual packages 020 in the k-th group, where P, Q, j, and k are all positive integers, and P is greater than or equal to 2.

[0108] Figure 7 is a schematic diagram illustrating the grouping and recombining of all virtual packages 020 falling into the overlapping area in one embodiment of this application. Figure 7 shows three images taken by three cameras 200. In each image, two virtual packages 020 fall into the overlapping area (in reality, there may be more). The virtual packages 020 falling into the overlapping area captured by each camera 200 are grouped into a group of virtual packages 020 (not a single combination), so there are a total of three groups of virtual packages 020. In other words, the P value mentioned above is 3. One virtual package 020 in each group only establishes a combination relationship with virtual packages 020 in other groups, that is, forms a combination. It can be seen that although there are still six virtual packages 020 falling into the overlapping area, the combination method provided in this embodiment ultimately only establishes 12 combinations, which is 3 fewer combinations than the combination method in the embodiment of Figure 6e. It is understandable that only when two virtual packages 020 in a combination belong to two different images can these two virtual packages 020 possibly correspond to the same physical package, and only then is it necessary to determine whether deduplication processing is needed. Two different virtual packages 020 in the same image cannot correspond to the same physical package. That is to say, only when two virtual packages 020 are captured by two different cameras 200 is it necessary to compare and determine whether they can match. Therefore, in this embodiment, the virtual packages 020 are first grouped according to the correspondence between the virtual packages 020 and the cameras 200. By establishing a correspondence between each virtual package 020 in one group and each virtual package 020 in other groups, the combination of virtual packages 020 is determined. In this way, there is no need to combine virtual packages 020 in the same group again, which reduces the number of determined combinations of virtual packages 020, thereby reducing the amount of data computation when performing deduplication processing on virtual packages 020 in overlapping areas, shortening the data computation time, improving computational efficiency, and thus improving the efficiency of package detection. It can be understood that the number of combinations Q described above is reduced compared to the number of combinations N in the previous embodiment. The number of combinations has been reduced, which means... Number of combinations.

[0109] Furthermore, deleting one of the two virtual packages includes: obtaining the area occupied by each virtual package in the corresponding overlapping area, comparing the size of the areas occupied by the two virtual packages in the corresponding overlapping area, and deleting the virtual package with the smaller area.

[0110] In this embodiment, when two virtual packages are determined to match, the area occupied by each virtual package in the overlapping region is obtained, and the virtual package with the smaller area is deleted, that is, the virtual package with the larger area is retained. Since the area occupied by a virtual package corresponds to the size of the physical package it corresponds to, a larger area indicates more accurate package information in the image containing the virtual package. Conversely, a smaller area may mean that the image only contains a portion of the physical package on the transport surface; for example, part of the physical package may be in the overlapping region, and another part in the non-overlapping region. A camera 200 may only capture an image of a portion of the physical package located in the overlapping region. If the physical package is determined based on the virtual package falling within the overlapping region, the resulting information will be incomplete. Therefore, in this embodiment, the virtual package with the smaller area is deleted, while the virtual package with the larger area is retained to improve the accuracy of physical package detection, providing more accurate information for subsequent package processing and thus improving overall package processing accuracy.

[0111] In this embodiment of the application, optionally, processing the target combination to determine whether the two virtual packages in the target combination can match includes:

[0112] Determine the baseline virtual package from the two virtual packages in the target combination; determine whether the baseline virtual package is located on the boundary of the overlapping area; when the baseline virtual package is located on the boundary of the overlapping area, use the first formula to calculate the matching degree of the two virtual packages in the target combination; when the baseline virtual package is not located on the boundary of the overlapping area, use the second formula to calculate the matching degree of the two virtual packages in the target combination.

[0113] Determine whether the matching degree of the two virtual packages in the target combination is greater than a preset value. If so, determine that the two virtual packages in the target combination can match; otherwise, determine that the two virtual packages in the target combination cannot match.

[0114] in,

[0115] The first formula is:

[0116] The second formula is:

[0117] `th` represents the matching degree between the two virtual packages. `x1` and `y1` are the coordinates of the first virtual package in the world coordinate system, `x2` and `y2` are the coordinates of the second virtual package in the world coordinate system, `w1` is the width of the first virtual package, `w2` is the width of the second virtual package, `l1` is the length of the first virtual package, and `l2` is the length of the second virtual package. It is evident that the greater the difference in coordinates and dimensions between the two virtual packages, the lower the calculated matching degree `th`.

[0118] Optionally, the coordinates of the virtual package can be described using the coordinates of its center point or a designated vertex. When a virtual package is located on the boundary of an overlapping area, the corresponding physical package may be partially in the overlapping area and partially in the non-overlapping area. That is, although the physical package corresponding to the virtual package may be captured simultaneously by multiple cameras 200, some cameras 200 may capture a complete image of the physical package, while others may not. For cameras 200 that cannot capture a complete image of the physical package, the size of the virtual package determined from their image does not match the size of the physical package. Therefore, in this case, only the coordinates of the virtual package are used to calculate the matching degree between the two virtual packages. However, when the physical package is entirely located in the overlapping area, all multiple cameras 200 will be able to capture a complete image of the physical package. Therefore, in this case, introducing dimensional parameters such as the width and length of the virtual package when calculating the matching degree between the two virtual packages will help improve the accuracy of determining whether the virtual packages match.

[0119] Optionally, the reference virtual package is the virtual package used to determine the target combination. For example, in this embodiment, multiple virtual packages in the overlapping area are numbered, and after determining multiple combinations of the virtual packages, a virtual package is selected as the target virtual package in ascending or descending order of the virtual package numbers. The combination containing the target virtual package is then determined as the target combination. When calculating the matching degree between two virtual packages in the target combination, the reference virtual package is the target virtual package. For example, first, virtual package No. 1 is determined as the target virtual package, and then all combinations containing virtual package No. 1 are determined as the target combination. When calculating the matching degree between two virtual packages in the target combination, the reference virtual package is virtual package No. 1. Then, virtual package No. 2 is determined as the target virtual package, and then all combinations containing virtual package No. 2 that have not yet been processed are determined as the target combination. When calculating the matching degree between two virtual packages in the target combination, the reference virtual package is virtual package No. 2.

[0120] In this embodiment, the control device 300 of the package processing device 010 can execute an executable program to implement the package detection method in the above embodiments of this application. Figure 8 is a block diagram of the package processing device 010 in one embodiment of this application. As shown in Figure 8, the package processing device 010 also includes a bus 400 and a memory 500. The control device 300 is communicatively connected to the memory 500 through the bus 400. The memory 500 can store an executable program for the control device 300 to call to implement the above-mentioned package detection method. The memory 500 can also store relevant data for implementing the above-mentioned package detection method for use. For example, the memory 500 stores the coordinate transformation matrix corresponding to each camera 200, the coordinate range of each overlapping area in the world coordinate system, and the coordinate range of each non-overlapping area in the world coordinate system. The memory 500 can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other medium capable of storing program code. In some embodiments, the memory 500 can also be integrated with the control device 300.

[0121] In summary, in the package detection method provided in this application embodiment, virtual package detection is performed based on a single image acquired by each camera. The first coordinate position of each virtual package acquired through the single image is uniformly converted into a second coordinate position in the world coordinate system. Based on the second coordinate position of each virtual package, the virtual packages are deduplicated to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface. Thus, the physical packages on the conveyor surface are determined based on the reference virtual packages, and the position information of the physical packages on the conveyor surface is determined. The package detection method of this application embodiment eliminates the process of stitching multiple images together. Furthermore, compared to a stitched image, a single image acquired by a single camera is smaller in size and has less data volume. Therefore, the computational workload for detecting packages from a single image is significantly less than that for detecting packages from a stitched image. Also, since the data volume of a package is much smaller than that of an image, the computational workload for deduplicating virtual packages is significantly less than that for image fusion of overlapping areas of multiple images. Therefore, the package detection method of this application embodiment reduces the computational workload and shortens the processing time for package detection, thereby improving package detection efficiency and ultimately enhancing the accuracy of package detection and sorting. The package processing equipment provided in this application embodiment can implement the above-described package detection method and thus also has corresponding beneficial effects.

[0122] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A parcel detection method, applied to parcel processing equipment, characterized in that, The package handling equipment includes a conveying mechanism and multiple cameras disposed above the conveying mechanism. The conveying mechanism has a conveying surface for conveying packages, and the conveying surface includes multiple sub-regions. Each of the multiple cameras corresponds one-to-one with one of the multiple sub-regions of the conveying surface. Each camera is used to capture an image of its corresponding sub-region, and adjacent sub-regions partially overlap. The package detection method includes: acquiring multiple images from the multiple cameras at the same time; detecting in each image to determine a virtual package in each image; and determining a second coordinate position of the virtual package in the world coordinate system based on a first coordinate position of each virtual package in the corresponding image coordinate system. The coordinate system is used to describe the position of pixels in the image, and the world coordinate system is used to describe the position of objects on the conveyor surface. The virtual packages are deduplicated based on their second coordinate positions to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface. The physical packages on the conveyor surface are determined based on the reference virtual packages. Specifically, the process of deduplicating virtual packages based on their second coordinate positions to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface includes: counting virtual packages falling into overlapping areas and virtual packages falling into non-overlapping areas based on their second coordinate positions, wherein the overlapping areas... A region refers to a region on the conveying surface that belongs to at least two of the sub-regions, and a non-overlapping region refers to a region on the conveying surface that belongs to only one of the sub-regions. The deduplication process is performed on the virtual packages in each overlapping region, and the remaining virtual packages in each overlapping region after the deduplication process, along with the virtual packages falling into the non-overlapping region, are determined as reference virtual packages. The deduplication process for the virtual packages in each overlapping region specifically includes: step S01, determining multiple combinations of virtual packages in the overlapping region, wherein each combination includes two virtual packages; step S02, determining a target combination from the multiple combinations according to a set order; step... Step S03: Process the target combination to determine whether the two virtual packages in the target combination can match. If the two virtual packages can match, proceed to step S04; otherwise, proceed to step S05. Step S04: Delete one of the two virtual packages, and then proceed to step S05. Step S05: Determine whether there are any unprocessed combinations. If it is determined that there are still unprocessed combinations, proceed to step S02 again; otherwise, the process ends. Processing the target combination to determine whether the two virtual packages in the target combination can match includes: determining a reference virtual package from the two virtual packages in the target combination; determining whether the reference virtual package is located on the boundary of the overlapping area.When the reference virtual package is located on the boundary of the overlapping area, the matching degree of the two virtual packages in the target combination is calculated using a first formula. When the reference virtual package is not located on the boundary of the overlapping area, the matching degree of the two virtual packages in the target combination is calculated using a second formula. The first formula calculates the matching degree of the two virtual packages in the target combination using their coordinates in the world coordinate system, while the second formula calculates the matching degree of the two virtual packages in the target combination using their length, width, and coordinates in the world coordinate system. It is then determined whether the matching degree of the two virtual packages in the target combination is greater than a preset value. If so, the two virtual packages in the target combination are considered to be matched; otherwise, they are considered to be unmatched.

2. The package detection method according to claim 1, characterized in that, The overlapping region and the non-overlapping region are determined according to the following steps: using multiple cameras to capture images of pre-set marker samples on the conveyor surface, wherein the marker samples are provided with multiple detection units arranged in multiple rows and columns; determining the repeatedly captured region and the non-repeated region on the conveyor surface based on the images captured by each camera; determining the repeatedly captured region as the overlapping region and the non-repeated region as the non-overlapping region.

3. The package detection method according to claim 1, characterized in that, Step S04 further includes: after deleting one of the two virtual packages, deleting all combinations containing the deleted virtual package.

4. The package detection method according to claim 1, characterized in that, Determining multiple combinations of the virtual packages in the overlapping area includes: based on the statistics of the virtual packages falling into the overlapping area, determining N combinations by pairwise correspondence of the virtual packages.

5. The package detection method according to claim 1, characterized in that, Determining multiple combinations of the virtual packages in the overlapping area includes: dividing the counted virtual packages falling into the overlapping area into P groups according to the correspondence between the virtual packages and the cameras, wherein each group of virtual packages corresponds to one camera; determining Q combinations of the P groups of virtual packages, wherein each combination includes two virtual packages, and the two virtual packages in each combination belong to different groups.

6. The package detection method according to claim 1, characterized in that, Deleting one of the two virtual packages includes: obtaining the area occupied by each virtual package in the corresponding overlapping area, comparing the size of the areas occupied by the two virtual packages in the corresponding overlapping area, and deleting the virtual package with the smaller area.

7. The package detection method according to claim 1, characterized in that, The first formula is: The second formula is: Where th is the matching degree of the two virtual packages, x1 and y1 are the coordinate values ​​of the first virtual package in the world coordinate system, x2 and y2 are the coordinate values ​​of the second virtual package in the world coordinate system, w1 is the width of the first virtual package, w2 is the width of the second virtual package, l1 is the length of the first virtual package, and l2 is the length of the second virtual package.

8. A parcel processing device, characterized in that, The package handling equipment includes a control device, a conveying mechanism, and multiple cameras disposed above the conveying mechanism. The conveying mechanism has a conveying surface for conveying packages, and the conveying surface includes multiple sub-regions. Each of the multiple cameras corresponds one-to-one with one of the multiple sub-regions of the conveying surface. Each camera is used to capture an image of its corresponding sub-region. Adjacent sub-regions partially overlap. All cameras are electrically connected to the control device. The control device is configured to: acquire multiple images from the multiple cameras at the same time; detect each image to determine a virtual package in each image; and determine the virtual package's position in the world coordinate system based on the first coordinate position of each virtual package in the corresponding image coordinate system. The second coordinate position in the image is used to describe the position of a pixel in the image, and the world coordinate system is used to describe the position of an object on the conveyor surface. The virtual packages are deduplicated based on their second coordinate positions to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface. The physical packages on the conveyor surface are determined based on the reference virtual packages. Specifically, the process of deduplicating virtual packages based on their second coordinate positions to obtain reference virtual packages that correspond one-to-one with the physical packages on the conveyor surface includes: counting virtual packages falling into overlapping areas and virtual packages falling into non-overlapping areas based on their second coordinate positions. The virtual packages are defined as follows: the overlapping area refers to the area on the conveyor surface that belongs to at least two of the sub-regions, and the non-overlapping area refers to the area on the conveyor surface that belongs to only one of the sub-regions; the virtual packages in each overlapping area are deduplicated, and the virtual packages remaining in each overlapping area after the deduplication process and the virtual packages falling into the non-overlapping area are determined as reference virtual packages; the deduplication process for the virtual packages in each overlapping area specifically includes: step S01, determining multiple combinations of multiple virtual packages in the overlapping area, wherein each combination includes two virtual packages; step S02, determining a target package from the multiple combinations according to a set order. Target combination; Step S03, process the target combination to determine whether the two virtual packages in the target combination can match. If the two virtual packages can match, proceed to step S04; otherwise, proceed to step S05. Step S04, delete one of the two virtual packages, and then proceed to step S05. Step S05, determine whether there are any unprocessed combinations. If it is determined that there are still unprocessed combinations, proceed to step S02 again; otherwise, the process ends. Processing the target combination to determine whether the two virtual packages in the target combination can match includes: determining a reference virtual package from the two virtual packages in the target combination; determining whether the reference virtual package is located on the boundary of the overlapping area.When the reference virtual package is located on the boundary of the overlapping area, the matching degree of the two virtual packages in the target combination is calculated using a first formula. When the reference virtual package is not located on the boundary of the overlapping area, the matching degree of the two virtual packages in the target combination is calculated using a second formula. The first formula calculates the matching degree of the two virtual packages in the target combination using their coordinates in the world coordinate system, while the second formula calculates the matching degree of the two virtual packages in the target combination using their length, width, and coordinates in the world coordinate system. It is then determined whether the matching degree of the two virtual packages in the target combination is greater than a preset value. If so, the two virtual packages in the target combination are considered to be matched; otherwise, they are considered to be unmatched.

Images