A package automatic sorting method based on three-dimensional vision and related equipment

Abstract

The present invention discloses a method for automatic sorting of packages based on three-dimensional vision and related equipment, the method comprising: acquiring a first image, wherein the first image is a depth image taken by a three-dimensional camera above a conveyor belt toward the conveyor belt; binarizing the first image according to a preset depth threshold to obtain multiple target contours, and obtaining a minimum bounding rectangle corresponding to each target contour; determining the grasping point information of a target package according to the target rectangles in each of the minimum bounding rectangles; and controlling a grasping device according to the grasping point information to grasp the target package to a target position.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a three-dimensional vision-based automatic parcel sorting method and related equipment. Background Art

[0002] In the prior art, parcel sorting is usually done on a cross-belt sorter. The cross-belt sorter uses a trolley on a circular line to transport parcels to a transfer container outside the circular conveyor line to achieve parcel sorting. After the transfer container is full, it is transported to the buffer area. In this process, parcels are lined up on the front belt line and sequentially transported on the trolley on the circular line. Each trolley can only carry one parcel, and the parcels need to be lined up on the conveyor belt and transported sequentially, which is inefficient.

[0003] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0004] In view of the above-mentioned defects of the prior art, the present invention provides a three-dimensional vision-based automatic parcel sorting method and related equipment, aiming to solve the problem of low parcel sorting efficiency in the prior art.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] A first aspect of the present invention provides a method for automatic parcel sorting based on three-dimensional vision, the method comprising:

[0007] Acquire a first image, wherein the first image is a depth image captured by a three-dimensional camera above the conveyor belt in the direction of the conveyor belt;

[0008] Binarizing the first image according to a preset depth threshold to obtain multiple target contours, and obtaining a minimum bounding rectangle corresponding to each target contour;

[0009] Determine the grabbing point information of the target package according to the target rectangle in each of the minimum circumscribed rectangles;

[0010] The grabbing device is controlled according to the grabbing point information to grab the target package to the target position.

[0011] The automatic parcel sorting method based on three-dimensional vision, wherein, after obtaining the minimum circumscribed rectangle corresponding to each target contour, the method further comprises:

[0012] Convert pixel coordinate values ​​of a first middle point, a second middle point, a third middle point, and a fourth middle point in the second image into three-dimensional coordinate values ​​in a camera coordinate system, wherein the first middle point, the second middle point, the third middle point, and the fourth middle point are points at one quarter and three quarters on two diagonals of the target rectangle, respectively;

[0013] The size of the target package is obtained according to the three-dimensional coordinate values ​​of the first middle point, the second middle point, the third middle point and the fourth middle point in the camera coordinate system.

[0014] The automatic parcel sorting method based on three-dimensional vision, wherein the step of obtaining the size of the target parcel according to the three-dimensional coordinate values ​​of the first intermediate point, the second intermediate point, the third intermediate point and the fourth intermediate point in the camera coordinate system, comprises:

[0015] Obtain the length, width and height of the target package according to a first preset formula and a second preset formula;

[0016] The first preset formula is:

[0017]

[0018] The second preset formula is:

[0019] H=Z 0 -0.25*(Z 5 +Z 6 +Z 7 +Z 8 );

[0020] Among them, (X 5 ,Y 5 ,Z 5 )、(X 6 ,Y 6 ,Z 6 )、(X 7 ,Y 7 ,Z 7 )、(X 8 ,Y 8 ,Z 8 ) respectively represent the three-dimensional coordinate values ​​of the first middle point, the second middle point, the third middle point and the fourth middle point in the camera coordinate system, Z 0 Represents the distance from the conveyor belt plane to the 3D camera.

[0021] The automatic parcel sorting method based on three-dimensional vision, wherein the method further comprises:

[0022] Acquire a second image, wherein the second image is an image captured by a two-dimensional camera above the parcel conveyor belt in the direction of the parcel conveyor belt;

[0023] Determine the area enclosed by the target in the second image according to each vertex of the target rectangle;

[0024] Acquire a barcode corresponding to the target package according to the area of ​​the target package;

[0025] The size of the target package and the barcode corresponding to the target package are bound.

[0026] The automatic parcel sorting method based on three-dimensional vision, wherein the first image is binarized according to a preset depth threshold to obtain multiple target contours, includes:

[0027] Binarizing the first image according to a preset depth threshold to obtain a plurality of initial contours;

[0028] When the initial contour includes n barcodes in the corresponding area in the second image, the initial contour is divided into n target contours according to the position of each barcode, where n is a positive integer and n>1.

[0029] The automatic parcel sorting method based on three-dimensional vision, wherein the grab point information includes the coordinate value of at least one vertex of the target rectangle in the conveyor belt coordinate system; determining the grab point information of the target parcel according to the target rectangle in each of the minimum circumscribed rectangles includes:

[0030] Acquire a three-dimensional coordinate value of at least one vertex of the target rectangle in a camera coordinate system according to a pixel coordinate value of the at least one vertex of the target rectangle in the first image;

[0031] The coordinate value of the at least one vertex in the conveyor belt coordinate system is obtained according to the three-dimensional coordinate value of the at least one vertex in the camera coordinate system and the preset coordinate difference.

[0032] The automatic parcel sorting method based on three-dimensional vision, wherein the step of controlling the grabbing device to grab the target parcel to the target position according to the grabbing point information, comprises:

[0033] Determine the delivery location corresponding to the target package according to the barcode of the target package, and determine the target location according to the delivery location;

[0034] The grabbing device is controlled according to the grabbing point information and the conveying speed of the conveyor belt to grab the target package to the target position.

[0035] A second aspect of the present invention provides a three-dimensional vision-based automatic parcel sorting system, comprising:

[0036] An image acquisition module, the image acquisition module is used to acquire a first image, wherein the first image is a depth image captured by a three-dimensional camera above the conveyor belt in the direction of the conveyor belt;

[0037] A contour processing module, the contour processing module is used to perform binarization processing on the first image according to a preset depth threshold, obtain multiple target contours, and obtain a minimum circumscribed rectangle corresponding to each target contour;

[0038] A position information module, the position information module is used to determine the grab point information of the target package according to the target rectangle in each of the minimum circumscribed rectangles;

[0039] A grabbing module, the grabbing module is used to control a grabbing device to grab the target package to a target position according to the grabbing point information.

[0040] According to a third aspect of the present invention, a terminal is provided, comprising a processor and a computer-readable storage medium communicatively connected to the processor, wherein the computer-readable storage medium is suitable for storing a plurality of instructions, and the processor is suitable for calling the instructions in the computer-readable storage medium to execute the steps of implementing any one of the above-mentioned three-dimensional vision-based automatic parcel sorting methods.

[0041] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any of the above-mentioned three-dimensional vision-based automatic parcel sorting methods.

[0042] Compared with the prior art, the present invention provides a method for automatic sorting of parcels based on three-dimensional vision and related equipment. The method for automatic sorting of parcels based on three-dimensional vision adopts a three-dimensional camera to shoot from above the conveyor belt toward the conveyor belt to obtain a depth image, and binarizes the depth image according to a depth threshold to obtain multiple target contours, where the target contour is the contour of a single parcel, and determines the gripping point position of the parcel according to the minimum circumscribed rectangle corresponding to the target contour, and controls the gripping device to grip the parcel to the target position according to the gripping point position. In the scheme of the present invention, shooting is performed by a three-dimensional camera, and the image obtained by shooting is contoured according to the depth value, so that all parcels in the camera's field of view can be located, and the gripping device can be controlled to automatically grip. Compared with the single-piece queuing package supply adopted in the traditional sorting scheme, the conveyor belt space can be effectively utilized and the sorting efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of an embodiment of the method for automatic parcel sorting based on three-dimensional vision provided by the present invention;

[0044] Figure 2 A schematic diagram of a sorting operation in an embodiment of the automatic parcel sorting method based on three-dimensional vision provided by the present invention;

[0045] Figure 3 A structural principle diagram of an embodiment of an automatic parcel sorting system based on three-dimensional vision provided by the present invention;

[0046] Figure 4 A schematic diagram of the principles of an embodiment of a terminal provided by the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] Embodiment 1

[0049] like Figure 1 As shown, in one embodiment of the automatic parcel sorting method based on three-dimensional vision, the steps include:

[0050] S100: Acquire a first image, wherein the first image is a depth image captured by a three-dimensional camera above a conveyor belt toward the conveyor belt.

[0051] like Figure 2 As shown, a camera module 100 can be set above a conveyor belt 200 for conveying packages, and the camera module includes a three-dimensional camera. The three-dimensional camera shoots above the conveyor belt in the direction of the conveyor belt to obtain a depth image as the first image, that is, the pixel value of each pixel in the first image is the distance between the actual position corresponding to the pixel and the three-dimensional camera.

[0052] S200: binarize the first image according to a preset depth threshold to obtain multiple target contours, and acquire a minimum bounding rectangle corresponding to each target contour.

[0053] Binarizing the first image according to a preset depth threshold means that the pixel values ​​of the first image that are not less than the depth threshold are modified to a uniform first value, and the pixel values ​​of the first image that are less than the depth threshold are modified to the same second value. In this way, a new image is generated, and the pixel values ​​of all pixels in the image are one of the first value and the second value.

[0054] The depth threshold can be the distance from the plane of the conveyor belt to the depth camera or slightly less than the distance from the plane of the conveyor belt to the depth camera, that is, the distance from the plane of the conveyor belt to the depth camera is not less than the depth threshold and the difference between the distance from the plane of the conveyor belt to the depth camera and the depth threshold is less than a preset value, and the preset value should be less than the minimum height of the package. In this way, in the new image generated after binarizing the first image by the depth threshold, the pixel values ​​corresponding to the package on the conveyor belt and the conveyor belt are different, thereby realizing the segmentation of the foreground and the background. As long as the package is laid flat on the conveyor belt, the package on the conveyor belt can be distinguished from the conveyor belt, and the package area will form a contour in the conveyor belt area.

[0055] It is not difficult to see that if the packages are very close to each other or directly in contact, there will be a situation where only one contour will be identified for the adjacent packages. This situation can be avoided by placing each package on the conveyor belt at a certain distance to ensure that each contour corresponds to one package, but this will generate more operations. In this embodiment, the camera module also includes a two-dimensional camera, and the two-dimensional camera is used to shoot a two-dimensional image as a second image above the conveyor belt toward the conveyor belt. After acquiring the second image, the two-dimensional code in the second image can be identified. The three-dimensional camera and the two-dimensional camera are photographed at the same position, so there should be a corresponding relationship between the pixels in the first image and the pixels in the second image. Therefore, the barcode in the second image can be used to determine how many packages are included in the single contour of the second image after binary processing, and then perform contour segmentation. The first image is binarized according to a preset depth threshold to obtain multiple target contours, including:

[0056] Binarizing the first image according to a preset depth threshold to obtain a plurality of initial contours;

[0057] When the initial contour includes n barcodes in the corresponding area in the second image, the initial contour is divided into n target contours according to the position of each barcode, where n is a positive integer and n>1.

[0058] For each target contour obtained, the corresponding minimum bounding rectangle is obtained. According to each minimum bounding rectangle, the corresponding grasping point information of each package can be determined, that is, according to the target rectangle in each minimum bounding rectangle, the grasping point information of the target package can be obtained.

[0059] In a possible implementation, in this embodiment, when sorting parcels, the volume of each parcel is also determined to provide information for subsequent parcel packing, that is, after obtaining the minimum circumscribed rectangle corresponding to each target contour, the method provided in this embodiment further includes:

[0060] Convert pixel coordinate values ​​of a first middle point, a second middle point, a third middle point, and a fourth middle point in the second image into three-dimensional coordinate values ​​in a camera coordinate system, wherein the first middle point, the second middle point, the third middle point, and the fourth middle point are points at one quarter and three quarters on two diagonals of the target rectangle, respectively;

[0061] The size of the target package is obtained according to the three-dimensional coordinate values ​​of the first middle point, the second middle point, the third middle point and the fourth middle point in the camera coordinate system.

[0062] In the process of three-dimensional imaging, due to the uneven surface of the package and inaccurate depth information of the edge of the package, there will be certain errors in directly using the vertices of the target rectangle to determine the size of the target package. In this embodiment, the size of the target package is determined based on the points at one quarter and three quarters of the diagonal of the target rectangle.

[0063] Specifically, the pixel coordinate values ​​of the first middle point, the second middle point, the third middle point and the fourth middle point in the second image are converted into three-dimensional coordinate values ​​in the camera coordinate system by the formula: To obtain, where (u, v) represents the pixel coordinate value of the point, (X, Y, Z) represents the three-dimensional coordinate value of the point in the camera coordinate system, Z is the distance from the point to the camera, and can be directly obtained through the depth value in the second image taken by the three-dimensional camera, (u x ,v y ) is a pixel reference coordinate value, and the pixel reference coordinate value may be a pixel coordinate value of the center point of the two-dimensional image, x 、f y is the focal length of the camera, is the camera intrinsic parameter, and can be obtained through camera calibration.

[0064] The acquiring the size of the target package according to the three-dimensional coordinate values ​​of the first intermediate point, the second intermediate point, the third intermediate point, and the fourth intermediate point in the camera coordinate system includes:

[0065] Obtain the length, width and height of the target package according to a first preset formula and a second preset formula;

[0066] The first preset formula is:

[0067]

[0068] The second preset formula is:

[0069] H=Z 0 -0.25*(Z 5 +Z 6 +Z 7 +Z 8 );

[0070] Among them, (X 5 ,Y 5 ,Z 5 )、(X 6 ,Y 6 ,Z 6 )、(X 7 ,Y 7 ,Z 7 )、(X 8 ,Y 8 ,Z 8 ) respectively represent the three-dimensional coordinate values ​​of the first middle point, the second middle point, the third middle point and the fourth middle point in the camera coordinate system, Z 0 Represents the distance from the conveyor belt plane to the 3D camera.

[0071] After obtaining the size of the target package, the size of the target package is stored for later use in packing calculations. Specifically, the size of the target package can be bound to the barcode of the target package, so that the size of the target package can be obtained by scanning the barcode later. That is, the method also includes:

[0072] Acquire a second image, wherein the second image is an image captured by a two-dimensional camera above the parcel conveyor belt in the direction of the parcel conveyor belt;

[0073] Determine the area enclosed by the target in the second image according to each vertex of the target rectangle;

[0074] Acquire a barcode corresponding to the target package according to the area of ​​the target package;

[0075] The size of the target package and the barcode corresponding to the target package are bound.

[0076] The second image is taken by a two-dimensional camera, and the barcode of each package can be identified by a barcode recognition algorithm. Binding the size of the target package and the barcode corresponding to the target package is to establish a corresponding relationship between the size of the target package and the barcode of the target package, and store the corresponding relationship in a preset storage address, and other devices can access the storage address to obtain the corresponding size according to the barcode.

[0077] S300: Determine the grabbing point information of the target package according to the target rectangle in each of the minimum circumscribed rectangles.

[0078] In this embodiment, the pixel coordinates of the center point or vertex of the target rectangle on the image can be converted into coordinates on the conveyor coordinate system as the grabbing point information of the target package, so that the grabbing device 300 grabs the target package according to the grabbing point information. Figure 2 As shown, the grasping device 300 may be a drone provided with a grasping structure, and the grasping device 300 may also be a six-axis robotic arm, a delta robot, etc.

[0079] The grab point information includes the coordinate value of at least one vertex of the target rectangle in the conveyor belt coordinate system; the grab point information of the target package is determined according to the target rectangle in each of the minimum circumscribed rectangles, including:

[0080] Acquire a three-dimensional coordinate value of at least one vertex of the target rectangle in a camera coordinate system according to a pixel coordinate value of the at least one vertex of the target rectangle in the first image;

[0081] The coordinate value of the at least one vertex in the conveyor belt coordinate system is obtained according to the three-dimensional coordinate value of the at least one vertex in the camera coordinate system and the preset coordinate difference.

[0082] Specifically, the three-dimensional coordinate values ​​of the vertices of the target rectangle in the camera coordinate system are converted into coordinate values ​​in the conveyor belt coordinate system by the following formula:

[0083]

[0084] Among them, (X w ,Y w ,Z w ) represents the coordinate value in the conveyor coordinate system, (X,Y) represents the coordinate value of the X-axis and Y-axis in the camera coordinate system, H is the corresponding package height, X bias , Y bias The offsets of the origin of the conveyor coordinate system relative to the origin of the camera coordinate system on the X-axis and the Y-axis can be obtained by measuring the positions of the three-dimensional camera and the conveyor in advance.

[0085] S400: Controlling a grasping device to grasp the target package to a target position according to the grasping point information.

[0086] The controlling the grabbing device to grab the target package to the target position according to the grabbing point information specifically includes:

[0087] Determine the delivery location corresponding to the target package according to the barcode of the target package, and determine the target location according to the delivery location;

[0088] The grabbing device is controlled according to the grabbing point information and the conveying speed of the conveyor belt to grab the target package to the target position.

[0089] Packages shipped to different destinations have different storage locations. The delivery location corresponding to the target package is determined according to the barcode of the target package, and then the storage location corresponding to the target package can be determined as the target location. The grabbing point information is the position of the target package in the conveyor coordinate system when the three-dimensional camera captures the first image. There is a gap between the time when the grabbing device actually grabs and the shooting time. Therefore, the grabbing position can be determined according to the grabbing point information, the conveying speed of the conveyor belt and the shooting time of the first image, so that the grabbing load grabs the target package at the grabbing position. Specifically, the grabbing position can be determined first and then sent to the grabbing device, or the grabbing point information, the conveying speed of the conveyor belt and the shooting time of the first image can be sent to the grabbing device, and the grabbing device determines the grabbing position according to its own movement to grab.

[0090] In summary, this embodiment provides a method for automatic sorting of parcels based on three-dimensional vision. By shooting with a three-dimensional camera and dividing the image into contours according to the depth value, all parcels in the camera's field of view can be located, and the grasping device can be controlled to grasp automatically. Compared with the single-piece queuing package supply adopted in traditional sorting solutions, it can effectively utilize the conveyor belt space and improve the sorting efficiency.

[0091] It should be understood that, although the various steps in the flowcharts given in the accompanying drawings of the present specification are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0092] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0093] Embodiment 2

[0094] Based on the above embodiments, the present invention also provides a three-dimensional vision-based automatic parcel sorting system, such as Figure 3 As shown, the automatic parcel sorting system based on three-dimensional vision includes:

[0095] An image acquisition module, the image acquisition module is used to acquire a first image, wherein the first image is a depth image captured by a three-dimensional camera above the conveyor belt in the direction of the conveyor belt, as specifically described in the first embodiment;

[0096] A contour processing module, the contour processing module is used to perform binarization processing on the first image according to a preset depth threshold to obtain multiple target contours, and obtain a minimum circumscribed rectangle corresponding to each target contour, as described in the first embodiment;

[0097] A position information module, the position information module is used to determine the grab point information of the target package according to the target rectangle in each of the minimum circumscribed rectangles, as described in the first embodiment;

[0098] A grabbing module, the grabbing module is used to control a grabbing device to grab the target package to a target position according to the grabbing point information, as specifically described in the first embodiment.

[0099] Embodiment 3

[0100] Based on the above embodiments, the present invention also provides a terminal, such as Figure 4 As shown, the terminal includes a processor 10 and a memory 20. Figure 4 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0101] The memory 20 may be an internal storage unit of the terminal in some embodiments, such as a hard disk or memory of the terminal. The memory 20 may also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a three-dimensional vision-based automatic parcel sorting program 30 is stored on the memory 20, and the three-dimensional vision-based automatic parcel sorting program 30 can be executed by the processor 10, thereby realizing the three-dimensional vision-based automatic parcel sorting method in the present application.

[0102] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other chip, used to run the program code or process data stored in the memory 20, such as executing the three-dimensional vision-based automatic parcel sorting method.

[0103] In one embodiment, when the processor 10 executes the three-dimensional vision-based package automatic sorting program 30 in the memory 20, the following steps are implemented:

[0104] Acquire a first image, wherein the first image is a depth image captured by a three-dimensional camera above the conveyor belt in the direction of the conveyor belt;

[0105] Binarizing the first image according to a preset depth threshold to obtain multiple target contours, and obtaining a minimum bounding rectangle corresponding to each target contour;

[0106] Determine the grabbing point information of the target package according to the target rectangle in each of the minimum circumscribed rectangles;

[0107] The grabbing device is controlled according to the grabbing point information to grab the target package to the target position.

[0108] After obtaining the minimum bounding rectangle corresponding to each target contour, the method further includes:

[0109] Convert pixel coordinate values ​​of a first middle point, a second middle point, a third middle point, and a fourth middle point in the second image into three-dimensional coordinate values ​​in a camera coordinate system, wherein the first middle point, the second middle point, the third middle point, and the fourth middle point are points at one quarter and three quarters on two diagonals of the target rectangle, respectively;

[0110] The size of the target package is obtained according to the three-dimensional coordinate values ​​of the first middle point, the second middle point, the third middle point and the fourth middle point in the camera coordinate system.

[0111] Wherein, acquiring the size of the target package according to the three-dimensional coordinate values ​​of the first intermediate point, the second intermediate point, the third intermediate point and the fourth intermediate point in the camera coordinate system includes:

[0112] Obtain the length, width and height of the target package according to a first preset formula and a second preset formula;

[0113] The first preset formula is:

[0114]

[0115] The second preset formula is:

[0116] H=Z 0 -0.25*(Z 5 +Z 6 +Z 7 +Z 8 );

[0117] Among them, (X 5 ,Y 5 ,Z 5 )、(X 6 ,Y 6 ,Z 6 )、(X 7 ,Y 7 ,Z 7 )、(X 8 ,Y 8 ,Z 8 ) respectively represent the three-dimensional coordinate values ​​of the first middle point, the second middle point, the third middle point and the fourth middle point in the camera coordinate system, Z 0 Represents the distance from the conveyor belt plane to the 3D camera.

[0118] Wherein, the method further comprises:

[0119] Acquire a second image, wherein the second image is an image captured by a two-dimensional camera above the parcel conveyor belt in the direction of the parcel conveyor belt;

[0120] Determine the area enclosed by the target in the second image according to each vertex of the target rectangle;

[0121] Acquire a barcode corresponding to the target package according to the area of ​​the target package;

[0122] The size of the target package and the barcode corresponding to the target package are bound.

[0123] The binarization of the first image according to a preset depth threshold to obtain multiple target contours includes:

[0124] Binarizing the first image according to a preset depth threshold to obtain a plurality of initial contours;

[0125] When the initial contour includes n barcodes in the corresponding area in the second image, the initial contour is divided into n target contours according to the position of each barcode, where n is a positive integer and n>1.

[0126] The grab point information includes the coordinate value of at least one vertex of the target rectangle in the conveyor belt coordinate system; the grab point information of the target package is determined according to the target rectangle in each of the minimum circumscribed rectangles, including:

[0127] Acquire a three-dimensional coordinate value of at least one vertex of the target rectangle in a camera coordinate system according to a pixel coordinate value of the at least one vertex of the target rectangle in the first image;

[0128] The coordinate value of the at least one vertex in the conveyor belt coordinate system is obtained according to the three-dimensional coordinate value of the at least one vertex in the camera coordinate system and the preset coordinate difference.

[0129] Wherein, controlling the grabbing device to grab the target package to the target position according to the grabbing point information includes:

[0130] Determine the delivery location corresponding to the target package according to the barcode of the target package, and determine the target location according to the delivery location;

[0131] The grabbing device is controlled according to the grabbing point information and the conveying speed of the conveyor belt to grab the target package to the target position.

[0132] Embodiment 4

[0133] The present invention also provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the automatic parcel sorting method based on three-dimensional vision as described above.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatic parcel sorting based on three-dimensional vision, characterized in that: The method comprises: Acquire a first image, wherein the first image is a depth image captured by a three-dimensional camera above the conveyor belt in the direction of the conveyor belt; acquire a second image, wherein the second image is a two-dimensional image captured by a two-dimensional camera above the conveyor belt in the direction of the conveyor belt; The first image is binarized according to a preset depth threshold, and contour segmentation is performed according to the barcode of the second image to obtain multiple target contours, and a minimum circumscribed rectangle corresponding to each target contour is obtained; the depth threshold is not greater than the distance from the plane of the conveyor belt to the three-dimensional camera, and the difference between the depth threshold and the distance from the plane of the conveyor belt to the three-dimensional camera is less than a preset value, and the preset value is less than the minimum height of the package; Determine the grab point information of the target package according to the target rectangle in each of the minimum circumscribed rectangles; obtain the size of the target package according to the points at one quarter and three quarters on the two diagonals of the target rectangle; The grabbing device is controlled according to the grabbing point information to grab the target package to the target position.

2. The automatic parcel sorting method based on three-dimensional vision according to claim 1 is characterized in that: After obtaining the minimum bounding rectangle corresponding to each target contour, the method further includes: Convert pixel coordinate values ​​of a first middle point, a second middle point, a third middle point, and a fourth middle point in the second image into three-dimensional coordinate values ​​in a camera coordinate system, wherein the first middle point, the second middle point, the third middle point, and the fourth middle point are points at one quarter and three quarters on two diagonals of the target rectangle, respectively; The size of the target package is obtained according to the three-dimensional coordinate values ​​of the first middle point, the second middle point, the third middle point and the fourth middle point in the camera coordinate system.

3. The automatic parcel sorting method based on three-dimensional vision according to claim 2 is characterized in that: The acquiring the size of the target package according to the three-dimensional coordinate values ​​of the first intermediate point, the second intermediate point, the third intermediate point, and the fourth intermediate point in the camera coordinate system includes: Obtain the length, width and height of the target package according to a first preset formula and a second preset formula; The first preset formula is: ； The second preset formula is: ； in, , , , represent the three-dimensional coordinate values ​​of the first middle point, the second middle point, the third middle point and the fourth middle point in the camera coordinate system respectively, Represents the distance from the conveyor belt plane to the 3D camera.

4. The automatic parcel sorting method based on three-dimensional vision according to claim 2 is characterized in that: The method further comprises: Acquire a second image, wherein the second image is an image captured by a two-dimensional camera above the parcel conveyor belt in the direction of the parcel conveyor belt; Determine the area enclosed by the target in the second image according to each vertex of the target rectangle; Acquire a barcode corresponding to the target package according to the area of ​​the target package; The size of the target package and the barcode corresponding to the target package are bound.

5. The automatic parcel sorting method based on three-dimensional vision according to claim 4 is characterized in that: The binarization process is performed on the first image according to a preset depth threshold to obtain a plurality of target contours, including: Binarizing the first image according to a preset depth threshold to obtain a plurality of initial contours; When the initial contour includes n barcodes in the corresponding area in the second image, the initial contour is divided into n target contours according to the position of each barcode, where n is a positive integer and n>1.

6. The automatic parcel sorting method based on three-dimensional vision according to claim 1 is characterized in that: The grab point information includes the coordinate value of at least one vertex of the target rectangle in the conveyor belt coordinate system; the grab point information of the target package is determined according to the target rectangle in each of the minimum circumscribed rectangles, including: Acquire a three-dimensional coordinate value of at least one vertex of the target rectangle in a camera coordinate system according to a pixel coordinate value of the at least one vertex of the target rectangle in the first image; The coordinate value of the at least one vertex in the conveyor belt coordinate system is obtained according to the three-dimensional coordinate value of the at least one vertex in the camera coordinate system and the preset coordinate difference.

7. The automatic parcel sorting method based on three-dimensional vision according to claim 4 is characterized in that: The controlling the grabbing device to grab the target package to the target position according to the grabbing point information includes: Determine the delivery location corresponding to the target package according to the barcode of the target package, and determine the target location according to the delivery location; The grabbing device is controlled according to the grabbing point information and the conveying speed of the conveyor belt to grab the target package to the target position.

8. An automatic parcel sorting system based on three-dimensional vision, characterized in that: include: An image acquisition module, the image acquisition module is used to acquire a first image, wherein the first image is a depth image captured by a three-dimensional camera above the conveyor belt in the direction of the conveyor belt; and acquire a second image, wherein the second image is a two-dimensional image captured by a two-dimensional camera above the conveyor belt in the direction of the conveyor belt; A contour processing module, the contour processing module is used to perform binarization processing on the first image according to a preset depth threshold, and perform contour segmentation according to the barcode of the second image to obtain multiple target contours, and obtain a minimum circumscribed rectangle corresponding to each target contour; the depth threshold is not greater than the distance from the plane of the conveyor belt to the three-dimensional camera, and the difference between the depth threshold and the distance from the plane of the conveyor belt to the three-dimensional camera is less than a preset value, and the preset value is less than the minimum height of the package; A position information module, the position information module is used to determine the grab point information of the target package according to the target rectangle in each of the minimum circumscribed rectangles; and obtain the size of the target package according to the points at one quarter and three quarters on the two diagonals of the target rectangle; A grabbing module, the grabbing module is used to control a grabbing device to grab the target package to a target position according to the grabbing point information.

9. A terminal, characterized in that: The terminal includes: a processor, a computer-readable storage medium communicatively connected to the processor, the computer-readable storage medium being suitable for storing a plurality of instructions, and the processor being suitable for calling the instructions in the computer-readable storage medium to execute the steps of the automatic parcel sorting method based on three-dimensional vision as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the three-dimensional vision-based automatic parcel sorting method as described in any one of claims 1-7.

Images