Parcel separation device and parcel separation method

Abstract

The present application provides a parcel separation device and a parcel separation method, which relate to the field of logistics technology. The parcel separation device and method provided in the present application can, when it is determined that there is a monitoring blind spot on the conveying plane based on the image taken by the visual monitoring device, and there is a target parcel parallel to the monitoring blind spot along the second direction, determine the first conveying component in the monitoring blind spot and the second conveying component that carries the target parcel, and set the first conveying component and the second conveying component to have different conveying speeds, so that when there is a parcel in the monitoring blind spot, the parcel in the monitoring blind spot is separated from the target parcel. Therefore, through the method and device of the present application, the target parcel can be separated from the parcel in the monitoring blind spot, avoiding the phenomenon of parcel separation failure caused by the parallel conveyance of the target parcel and the parcel in the monitoring blind spot, thereby improving the success rate of the parcel separation device in separating the parcels.

Description

Technical Field

[0001] The present application relates to the field of logistics technology, and in particular to a parcel separation device and a parcel separation method. Background Art

[0002] At present, in the domestic logistics industry, with the surge in parcel sorting volume, more and more logistics companies are deploying logistics automation systems in sorting sites, using logistics automation systems to replace traditional manual methods to achieve parcel separation, measurement, and sorting, greatly improving parcel processing efficiency and reducing the labor costs of logistics companies.

[0003] A logistics automation system includes a parcel separation device that receives multiple parcels conveyed by a higher-level device (e.g., an evacuation device) and separates the parcels being conveyed in parallel, allowing the multiple parcels to leave the parcel processing device one by one along the conveying direction and enter the lower-level conveying device. Related art discloses a parcel separation device that includes a control device, a conveying device, and a camera disposed above the conveying device. The conveying device includes multiple conveyor belts arranged in M rows and N columns. The control device controls the camera to capture images of the parcels on the conveying device and, based on the images of the parcels, determines whether there are any parcels being conveyed in parallel. If there are any parcels being conveyed in parallel, the control device controls the conveying assembly carrying the parcels to convey the parcels at different speeds, thereby separating the parcels being conveyed in parallel.

[0004] However, the inventors have discovered that, when separating parcels, the parcel separation device of the related art may fail to separate the parcels when there is a tall parcel among the parcels being transported in parallel. Summary of the Invention

[0005] The purpose of the present application includes providing a package separation device and a package separation method, which can improve the success rate of package separation.

[0006] The embodiments of the present application can be implemented as follows:

[0007] In a first aspect, the present application provides a parcel separation device, comprising a control device, a conveying device for conveying parcels in a first direction, and a visual monitoring device disposed above the conveying device, wherein the conveying device comprises a plurality of conveying assemblies arranged in multiple rows along the first direction and multiple columns along a second direction, the first direction being perpendicular to the second direction, the visual monitoring device comprising at least one camera, each camera being configured to capture an image of a set area of a conveying plane of the conveying device, the conveying device and the visual monitoring device being electrically connected to the control device, and the control device being configured to:

[0008] Determine whether there is a monitoring blind spot on the conveying plane based on the images captured by the visual monitoring device;

[0009] In the case where it is determined that a monitoring blind spot exists on the conveying plane, determining whether there is a target package that is parallel to the monitoring blind spot along the second direction;

[0010] When it is determined that there is a target package parallel to the monitoring blind spot along the second direction, determining a first conveying assembly located in the monitoring blind spot and a second conveying assembly carrying the target package;

[0011] The first conveying component and the second conveying component are set to have different conveying speeds so that when there is a package in the monitoring blind area, the package in the monitoring blind area is separated from the target package.

[0012] In an optional embodiment, the step of determining whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device includes:

[0013] Determine whether there is an obstructed area on the conveying plane based on the image captured by the visual monitoring device, wherein the obstructed area refers to an area whose image cannot be captured by all cameras of the visual monitoring device;

[0014] When it is determined that there is a blocked area on the conveying plane, the blocked area is determined as a monitoring blind area on the conveying plane.

[0015] In an optional embodiment, the step of determining whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device includes:

[0016] Determine whether there is an obstructed area on the conveying plane based on the image captured by the visual monitoring device, wherein the obstructed area refers to an area whose image cannot be captured by all cameras of the visual monitoring device;

[0017] When it is determined that there is an obstructed area on the conveying plane, the package is tracked based on the historical images captured by the visual monitoring device to determine whether the status of the package in the obstructed area can be determined based on the package tracking results;

[0018] When it is determined that the state of whether there is a package in the blocked area cannot be determined based on the package tracking result, the blocked area is determined as a monitoring blind area on the conveying plane.

[0019] In an optional embodiment, the visual monitoring device includes multiple cameras, and the step of determining whether there is an obstructed area on the conveying plane based on images captured by the visual monitoring device includes:

[0020] Determine overlapping areas and non-overlapping areas on the conveying plane; wherein the overlapping area refers to an area on the conveying plane whose images can be captured by multiple cameras, and the non-overlapping area refers to an area on the conveying plane whose image can be captured by only one camera;

[0021] It is determined whether there is an obscured area in the overlapping area according to the first images taken by all cameras corresponding to each overlapping area, and it is determined whether there is an obscured area in the non-overlapping area according to the second images taken by the cameras corresponding to each non-overlapping area.

[0022] In an optional embodiment, the multiple cameras are all 3D cameras, and the step of determining whether there is an obscured area in the overlapping area based on the first images captured by all cameras corresponding to each overlapping area includes:

[0023] Obtaining the first image captured by all cameras corresponding to the overlapping area at a set time;

[0024] fusing the plurality of first images to obtain a fused image, and detecting whether there is a blank area in the fused image;

[0025] When a blank area is detected in the fused image, an area on the conveying plane corresponding to the blank area is determined as the blocked area in the overlapping area.

[0026] In an optional embodiment, the multiple cameras are all 2D cameras, and the step of determining whether there is an obscured area in the overlapping area based on the first images captured by all cameras corresponding to each overlapping area includes:

[0027] Obtaining the first image captured by all cameras corresponding to the overlapping area at a set time;

[0028] determining, based on the plurality of first images, packages present in the overlapping region and the height of each package;

[0029] Determine the blind spot of each camera in the overlapping area based on the location and height of each package and the location and height of each corresponding camera;

[0030] Determine whether the blind areas of the corresponding multiple cameras have an intersection;

[0031] When it is determined that the blind areas of the fields of view of the corresponding multiple cameras have an intersection, the intersection is determined as the blocked area of the overlapping area.

[0032] In an optional embodiment, when it is determined that a monitoring blind spot exists on the conveying plane, determining whether there is a target package that is aligned with the monitoring blind spot along the second direction includes:

[0033] Obtaining a first coordinate range of the monitoring blind area along the first direction and a second coordinate range along the second direction;

[0034] Determine whether there is a package that intersects a first coordinate range of the monitoring blind spot along a first direction and is adjacent to a second coordinate range of the monitoring blind spot along a second direction;

[0035] If there is a package that intersects the first coordinate range of the monitoring blind spot along the first direction and is adjacent to the second coordinate range of the monitoring blind spot along the second direction, the package is determined as a target package.

[0036] In an optional embodiment, the step of setting the first conveying assembly and the second conveying assembly to have different conveying speeds includes:

[0037] The conveying speed of the second conveying assembly is set to be greater than the conveying speed of the first conveying assembly.

[0038] In an optional embodiment, the step of setting the first conveying assembly and the second conveying assembly to have different conveying speeds includes:

[0039] Determine the area of monitoring blind spots;

[0040] Determine the difference in conveying speed between the first conveying assembly and the second conveying assembly according to the area of the monitoring blind zone;

[0041] The conveying speeds of the first conveying component and the second conveying component are set according to the difference.

[0042] In a second aspect, the present application provides a parcel separation method, which is applied to parcel separation equipment. The parcel separation equipment includes a conveyor device for conveying parcels in a first direction and a visual monitoring device disposed above the conveyor device. The conveyor device includes a plurality of conveyor assemblies arranged in multiple rows along the first direction and multiple columns along a second direction, where the first direction is perpendicular to the second direction. The visual monitoring device includes at least one camera, each camera being configured to capture an image of a set area of a conveying plane of the conveyor device. The parcel separation method includes:

[0043] Determine whether there is a monitoring blind spot on the conveying plane based on the images captured by the visual monitoring device;

[0044] In the case where it is determined that a monitoring blind spot exists on the conveying plane, determining whether there is a target package that is parallel to the monitoring blind spot along the second direction;

[0045] When it is determined that there is a target package parallel to the monitoring blind spot along the second direction, determining a first conveying assembly located in the monitoring blind spot and a second conveying assembly carrying the target package;

[0046] The first conveying component and the second conveying component are set to have different conveying speeds so that when there is a package in the monitoring blind area, the package in the monitoring blind area is separated from the target package.

[0047] The beneficial effects of the embodiments of the present application include:

[0048] The parcel separation device and method provided in the embodiments of the present application can, when it is determined that there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device, determine whether there is a target parcel parallel to the monitoring blind spot along the second direction. When it is determined that there is a target parcel parallel to the monitoring blind spot along the second direction, the first conveying component in the monitoring blind spot and the second conveying component carrying the target parcel are determined, and the first conveying component and the second conveying component are set to have different conveying speeds, so that when there is a parcel in the monitoring blind spot, the parcel in the monitoring blind spot is separated from the target parcel. Therefore, through the embodiments of the present application, when there is a higher parcel on the conveying plane of the conveying device, if the parcel conveyed in parallel with the higher parcel cannot be monitored due to the obstruction caused by the higher parcel, the higher parcel can be separated from the parcels in the monitoring blind spot by setting the first conveying component in the monitoring blind spot and the second conveying component carrying the higher parcel to have different conveying speeds, thereby avoiding the phenomenon of parcel separation failure caused by the higher parcel being conveyed in parallel with the parcel in the monitoring blind spot, thereby improving the success rate of parcel separation of the parcel separation device. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 This is a schematic diagram of a package separation device in one embodiment of the present application;

[0051] Figure 2 This is a block diagram of the composition of a package separation device in one embodiment of the present application;

[0052] Figure 3 This is a flow chart of a package separation method in one embodiment of the present application;

[0053] Figure 4 This is a flow chart for determining whether there is a monitoring blind spot on the conveying plane based on images captured by a visual monitoring device in one embodiment of the present application;

[0054] Figure 5 This is a flow chart for determining whether there is a monitoring blind spot on the conveying plane based on images captured by a visual monitoring device in another embodiment of the present application;

[0055] Figure 6 This is a schematic diagram of the shooting principle of a visual monitoring device in one embodiment of the present application;

[0056] Figure 7This is a schematic diagram of the working principle of a 2D camera in one embodiment of the present application;

[0057] Figure 8 This is a flowchart of determining whether there is a target package side by side with the monitoring blind spot along the second direction in one embodiment of the present application;

[0058] Figure 9 This is a flow chart of configuring the first conveying assembly and the second conveying assembly to have different conveying speeds in one embodiment of the present application.

[0059] Icons: 010-package separation equipment; 100-conveyor device; 110-conveyor assembly; 200-visual monitoring device; 210-first camera; 220-second camera; 300-buffer device; 400-control device. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0061] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0062] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0063] In the description of this application, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the accompanying drawings, or is the orientation or position relationship in which the invented product is usually placed when in use. It is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on this application.

[0064] In addition, the terms "first", "second", etc., if used, are merely used to distinguish and describe, and should not be understood as indicating or implying relative importance.

[0065] It should be noted that, in the absence of conflict, the features in the embodiments of this application can be combined with each other.

[0066] The inventors discovered that when conveying packages in conventional parcel separation equipment, if a tall parcel is not directly below the camera, or if the parcel is directly below the camera but the camera lens is angled with the conveying plane of the conveyor, the parcel will obstruct a portion of the conveying plane, preventing the camera from capturing the obscured area. If a shorter parcel is located in this obscured area, it may be completely cast into the shadow of the taller parcel, preventing the shorter parcel from being captured by the camera and, in other words, causing it to be missed. If the shorter parcel is conveyed parallel to the taller parcel, the two parcels cannot be properly separated, resulting in parcel separation failure.

[0067] Therefore, in order to improve the above-mentioned problems faced by the package separation equipment in the related art, the embodiments of the present application provide a package separation device and a package separation method, which determines the monitoring blind spot and sets the conveying component corresponding to the monitoring blind spot and the conveying component carrying the target package to different conveying speeds, so that when there is a package in the monitoring blind spot, the package in the monitoring blind spot can be separated from the target package.

[0068] Figure 1 This is a schematic diagram of a package separation device 010 in one embodiment of the present application; Figure 2 This is a block diagram of the composition of the package separation device 010 in one embodiment of the present application. Figure 1 and Figure 2 As shown, in the embodiment of the present application, the package separation device 010 includes a control device 400, a conveying device 100 for conveying packages in a first direction, and a visual monitoring device 200 arranged above the conveying device 100. The conveying device 100 includes a plurality of conveying components 110 arranged in multiple rows along the first direction and multiple columns along the second direction, and the first direction is perpendicular to the second direction. Figure 1 , the first direction is the direction indicated by arrow a, and the second direction is the direction indicated by arrow c or d. The visual monitoring device 200 includes at least one camera, each of which is configured to capture an image of a set area of the conveying plane of the conveying device 100. The conveying device 100 and the visual monitoring device 200 are both electrically connected to the control device 400.

[0069] exist Figure 1In the embodiment, the conveyor device 100 includes four rows and six columns, totaling twenty-four conveyor assemblies 110. Each conveyor assembly 110 includes a conveyor belt and a drive motor that drives the conveyor belt. The upper surfaces of the conveyor belts of the multiple conveyor assemblies 110 constitute the entire conveying plane. When a package is conveyed by the conveyor device 100, it is supported by one or more conveyor assemblies 110. By setting the conveying speeds of the conveyor assemblies 110 carrying two different packages to different levels, the two packages can be caused to move relative to each other in the package conveying direction. Based on this principle, package separation can be achieved. It should be understood that in other optional embodiments, the number of conveyor assemblies 110 can be adjusted as needed.

[0070] In this embodiment, the visual monitoring device 200 includes two cameras: a first camera 210 and a second camera 220. The first camera 210 and the second camera 220 are supported by a bracket and positioned above the entire conveyor device 100, respectively, to capture images of the conveyor plane. In this embodiment, the first camera 210 is positioned upstream of the second camera 220 along the package conveying direction, i.e., the first direction. The field of view of the first camera 210 covers the front half of the conveyor plane, located upstream, while the field of view of the second camera 220 covers the rear half of the conveyor plane, located downstream.

[0071] Optionally, the first camera 210 and the second camera 220 are positioned diagonally above the conveyor device 100. For example, the first camera 210 and the second camera 220 are spaced apart along the diagonal direction of the conveyor plane, with each camera's lens facing the other. In the absence of packages obstructing the conveyor plane, each camera can capture an image of the entire conveyor plane. This configuration reduces the number of cameras required for the visual monitoring device 200 and reduces the overall cost of the package separation device 010.

[0072] It should be understood that in other optional embodiments, only one camera may be provided to capture images of the entire conveying plane. When multiple cameras are used, the multiple cameras may be arranged in multiple rows along the first direction and multiple columns along the second direction. The conveying plane of the conveying device 100 may also be divided into multiple areas, with each camera used to capture images of one area. For example, if the visual monitoring device 200 includes nine cameras arranged in three rows and three columns, the conveying plane of the conveying device 100 may be evenly divided into nine areas, with each camera used to capture images of one area.

[0073] In the embodiment of the present application, the camera included in the visual monitoring device 200 may be a 2D camera, a 3D camera, or even include both a 2D camera and a 3D camera.

[0074] In this embodiment, the package separation device 010 further includes a buffer device 300, which is electrically connected to the control device 400. The buffer device 300 is disposed upstream of the conveying device 100 and is used to convey packages to the conveying device 100. Figure 1 As shown, in this embodiment, the conveying surface of the buffer device 300 gradually widens along the first direction and eventually connects with the conveying plane of the conveyor device 100. The buffer device 300 has the function of evacuating packages. In other optional embodiments, the buffer device 300 may also include multiple conveying assemblies arranged side by side. The structure and size of these conveying assemblies may be the same or similar to the structure and size of the conveying assemblies 110 of the conveyor device 100. They are arranged along the width direction of the conveyor device 100 (i.e., the second direction). The multiple conveying assemblies of the buffer device 300 correspond one-to-one with the upstreammost row of conveying assemblies 110 of the conveyor device 100.

[0075] In the embodiment of the present application, the control device 400 is configured to:

[0076] Determine whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device 200; if it is determined that there is a monitoring blind spot on the conveying plane, determine whether there is a target package side by side with the monitoring blind spot along the second direction; if it is determined that there is a target package side by side with the monitoring blind spot along the second direction, determine a first conveying component located in the monitoring blind spot and a second conveying component carrying the target package; set the first conveying component and the second conveying component to have different conveying speeds so that when there is a package in the monitoring blind spot, the package in the monitoring blind spot is separated from the target package.

[0077] Through the embodiments of the present application, when there is a higher package on the conveying plane of the conveying device 100, if the package conveyed in parallel with the higher package cannot be monitored due to the obstruction caused by the higher package, then by setting the first conveying component in the monitoring blind spot and the second conveying component carrying the higher package to have different conveying speeds, the higher package can be separated from the packages in the monitoring blind spot, avoiding the failure of package separation caused by the parallel conveyance of the higher package and the packages in the monitoring blind spot, thereby improving the success rate of package separation of the package separation device 010.

[0078] The following describes the parcel separation method provided in the present embodiment of the present application, based on the parcel separation device 010 provided in the present embodiment. The parcel separation method provided in the present embodiment of the present application can be applied to the parcel separation device 010 provided in the present embodiment of the present application and implemented by the control device 400 of the parcel separation device 010. Figure 3 This is a flow chart of a package separation method according to an embodiment of the present application. Figure 3 As shown, the package separation method includes:

[0079] Step S100: Determine whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device.

[0080] Taking the parcel separation device 010 provided in the embodiment of the present application as an example, the control device 400 controls the camera of the visual monitoring device 200 to capture an image of the conveyor plane and, based on the image information fed back by the visual monitoring device 200, determines whether there is a monitoring blind spot on the conveyor plane. Furthermore, the control device 400 also determines the location information of parcels outside the monitoring blind spot based on the image information fed back by the visual monitoring device 200.

[0081] Figure 4 This is a flow chart of determining whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device 200 in one embodiment of the present application. Figure 4 As shown, in an optional embodiment, step S100 specifically includes:

[0082] Step S101: determining whether there is an obstructed area on the conveying plane based on the image captured by the visual monitoring device, wherein the obstructed area refers to an area whose image cannot be captured by all cameras of the visual monitoring device;

[0083] Step S102: When it is determined that there is a blocked area on the conveying plane, the blocked area is determined as a monitoring blind area on the conveying plane.

[0084] In this embodiment, if a portion of the conveying plane is blocked from view by the camera due to a higher package, the control device 400 cannot determine whether there is a package in the area (the blocked area) based on the image captured by the visual monitoring device 200. In some embodiments, because the package separation device 010 only uses the currently captured image of the conveying plane to determine the presence and location of a package in a certain area at the current moment, if there is an blocked area on the conveying plane, the control device 400 cannot confirm whether there is a package in the blocked area at that moment. In this case, the blocked area is determined as a monitoring blind spot, and subsequent control of the conveying component 110 in the monitoring blind spot can ensure that even if there are packages that need to be separated, the separation will not fail due to omission.

[0085] Figure 5 This is a flow chart of another embodiment of the present application for determining whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device 200. Figure 5 As shown, in another optional embodiment, step S100 specifically includes:

[0086] Step S111, judging whether there is an obstructed area on the conveying plane based on the image captured by the visual monitoring device, wherein the obstructed area refers to an area whose image cannot be captured by all cameras of the visual monitoring device;

[0087] Step S112: If it is determined that there is an obstructed area on the conveying plane, the package is tracked based on the historical images captured by the visual monitoring device to determine whether the status of the package in the obstructed area can be determined based on the package tracking results;

[0088] Step S113 : When it is determined that the state of whether there is a package in the blocked area cannot be determined based on the package tracking result, the blocked area is determined as a monitoring blind spot on the conveying plane.

[0089] It will be appreciated that in this embodiment, in addition to determining the current presence and location of a package on the conveyor plane based on the image currently captured by the visual monitoring device 200, the control device 400 may also perform package tracking based on historical images captured by the visual monitoring device 200, attempting to determine whether a package is currently in an obstructed area. In this case, even if an obstructed area is present in the image currently captured by the visual monitoring device 200, it does not mean that it is impossible to determine whether a package is present in the obstructed area. Therefore, in this embodiment, if an obstructed area is determined to exist on the conveyor plane, package tracking is performed based on historical images captured by the visual monitoring device 200 to determine whether the presence of a package in the obstructed area can be determined based on the package tracking results. If it is determined that the presence of a package in the obstructed area cannot be determined based on the package tracking results, then the control device 400 cannot determine the presence of a package in the obstructed area. In this case, the control device 400 determines the obstructed area as a monitoring blind spot. By determining the obstructed area as a monitoring blind spot, subsequent control of the conveyor assembly 110 in the monitoring blind spot ensures that even if a package that needs to be separated exists in the obstructed area, the package will not be missed, resulting in a separation failure.

[0090] In an optional embodiment of the present application, the camera of the visual monitoring device 200 captures an image of the corresponding area on the conveying plane once every set time interval (for example, 1ms, 2ms), and the control device 400 determines the status of the package based on a historical image (for example, the previous frame image) captured at a set time before the current moment (for example, a moment with a preset time interval from the current moment, hereinafter referred to as a historical moment), and tracks and predicts the current package status based on the status of the package at the historical moment, the time interval between the historical moment and the current moment, and the operation status of the conveying component 110 of the conveying device 100 within the time interval. When it is determined that there is an obstructed area on the conveying plane according to the current image, if the obstructed area at the current moment can be determined according to the package tracking result If the status of the package in the obscured area at the current moment cannot be determined according to the package tracking result, the obscured area will be determined as the monitoring blind spot on the conveying plane, that is, the existence of the monitoring blind spot on the conveying plane is determined according to the image taken by the visual monitoring device 200, and the conveying component 110 of the monitoring blind spot is subsequently controlled accordingly to ensure that the packages are effectively separated when there are packages that need to be separated in the monitoring blind spot.

[0091] Figure 6 FIG. 2 is a schematic diagram of the shooting principle of the visual monitoring device 200 in one embodiment of the present application. Figure 6 As shown, the conveying device 100 conveys packages in the first direction indicated by the arrow X. The visual monitoring device 200 includes a first camera 210 and a second camera 220 arranged diagonally. The field of view of the first camera 210 covers the front half of the conveying plane near the upstream, and the field of view of the second camera 220 covers the rear half of the conveying plane near the downstream. Assume that at the current moment, packages A, B, C, and D on the conveying plane are at Figure 6 In the state shown, the control device 400 determines the blocked area A1 and the blocked area A2 (respectively Figure 6), for the obscured area A1, it is impossible to determine the status of the package in the area at the current moment by tracking the package on the historical image. Therefore, the obscured area A1 will be determined as a monitoring blind spot. For the obscured area A2, the existence of the package D can be determined based on the historical image taken by the first camera 210, and based on the position of the package D at the historical moment determined based on the historical image, the time interval between the historical moment and the current moment, and the operation status of the conveying component 110 of the conveying device 100 during the time interval, it is calculated that the package D is currently located in the obscured area A2. Therefore, the control device 400 determines that the status of whether there is a package in the obscured area A2 can be determined based on the package tracking result, and the obscured area A2 will not be determined as a monitoring blind spot on the conveying plane. Optionally, when the status of whether there is a package in the obscured area can be determined based on the package tracking results, if it is determined that there is a package in the obscured area, the package in the obscured area is transported according to the determined status of the package and the preset package separation strategy. The specific separation method can refer to the relevant technology and will not be repeated here; if it is determined that there is no package in the obscured area, the target package will continue to be transported downstream.

[0092] It should be noted that when tracking a package based on the operating conditions of the conveying components 110 of the conveying device 100, it is necessary to determine the specific operating conditions of each conveying component 110 within the time interval between the two images on which the package tracking is based (hereinafter referred to as the tracking time interval). For example, if each conveying component 110 moves at the same uniform speed within the tracking time interval, the displacement of each package in the first direction is calculated based on the duration of the tracking time interval and the uniform motion speed of the conveying component 110, and the position of the package at the current moment is determined based on the position of each package at the historical moment on which the package tracking is based and the displacement of the package along the first direction within the tracking time interval. When multiple conveying components 110 of the conveying device 100 move at different speeds within the tracking time interval, the displacement of each package in the first direction is calculated based on the duration of the tracking time interval and the specific movement conditions of the conveying component 110 carrying each package within the tracking time interval, and the position of the package at the current moment is determined based on the position of each package at the historical moment and the displacement of the package along the first direction within the tracking time interval.

[0093] In the above-mentioned embodiments of determining whether a monitoring blind spot exists on the conveying plane based on images captured by the visual monitoring device 200, if the visual monitoring device 200 includes multiple cameras, optionally, determining whether an obstructed area exists on the conveying plane based on images captured by the visual monitoring device 200 may further include:

[0094] Determine overlapping areas and non-overlapping areas on the conveying plane; wherein the overlapping area refers to an area on the conveying plane whose images can be captured by multiple cameras, and the non-overlapping area refers to an area on the conveying plane whose image can be captured by only one camera;

[0095] It is determined whether there is an obscured area in the overlapping area according to the first images taken by all cameras corresponding to each overlapping area, and it is determined whether there is an obscured area in the non-overlapping area according to the second images taken by the cameras corresponding to each non-overlapping area.

[0096] In this embodiment, the control device 400 controls each camera of the visual monitoring device 200 to obtain a real-time image of the corresponding area once every preset time interval. For each overlapping area, since the area can be captured by multiple cameras, only when all cameras cannot capture an image of a certain area of the overlapping area at the set time, the area can be determined as an obscured area. Therefore, when determining whether there is an obscured area in the overlapping area, it is necessary to refer to the real-time image (i.e., the first image) captured by each camera corresponding to the overlapping area at the same time; and for each non-overlapping area, since the area can only be captured by one camera, the area of the non-overlapping area whose image cannot be captured by the camera can be determined as an obscured area. Therefore, based on the real-time image (i.e., the second image) captured by the camera corresponding to each non-overlapping area, it can be determined whether there is an obscured area in the overlapping area.

[0097] On this basis, if the multiple cameras of the visual monitoring device 200 are all 3D cameras, optionally, determining whether there is an obstructed area in the overlapping area based on the first images captured by all cameras corresponding to each overlapping area may specifically include the following steps:

[0098] Obtaining the first image captured by all cameras corresponding to the overlapping area at a set time;

[0099] fusing the plurality of first images to obtain a fused image, and detecting whether there is a blank area in the fused image;

[0100] When a blank area is detected in the fused image, an area on the conveying plane corresponding to the blank area is determined as the blocked area in the overlapping area.

[0101] In this embodiment, when the visual monitoring device 200 includes multiple 3D cameras, for each overlapping area, multiple first images captured simultaneously by all cameras corresponding to the overlapping area are fused to produce a fused image. Since a 3D camera acquires point cloud data for all points within its field of view when detecting an object within its field of view, for each physical point on the conveying plane that can be captured, a pixel corresponding to the physical point exists in the image captured by the 3D camera. The pixel has coordinate data (i.e., point cloud data) in three dimensions: height, length, and width. However, when a certain area cannot be captured by the 3D camera, the image captured by the 3D camera does not include the pixel corresponding to the physical point in the area. In other words, the pixel corresponding to the area in the image does not have coordinate data, resulting in a blank area with no data. Therefore, after the multiple first images are fused to produce a fused image, the fused image is detected to determine whether there is a blank area. If there is a blank area in the fused image, it indicates that none of the cameras corresponding to the overlapping area captured an image of the area on the conveying plane corresponding to the blank area. Therefore, in this case, the area on the conveying plane corresponding to the blank area is determined to be an obscured area.

[0102] Optionally, the above-mentioned steps of fusing the multiple first images to obtain a fused image may further include: transforming the pixels of each first image from the corresponding image coordinate system to the world coordinate system; and fusing the pixels in the multiple first images to generate a fused image. Each first image has a corresponding image coordinate system, which is a coordinate system established with the image as a reference and is used to describe the position of the pixels in the image. The world coordinate system is an absolute coordinate system established with the conveying plane of the conveying device 100 as a reference and is used to describe the position of the object on the conveying plane. By transforming the pixels of each first image from the corresponding image coordinate system to the world coordinate system, the pixels in the image can be analyzed based on the same reference datum to obtain the actual status of the package on the conveying plane.

[0103] If all cameras included in the visual monitoring device 200 are 2D cameras, optionally, determining whether there is an obstructed area in the overlapping area based on the first images captured by all cameras corresponding to each overlapping area may specifically include the following steps:

[0104] Obtaining the first image captured by all cameras corresponding to the overlapping area at a set time;

[0105] determining, based on the plurality of first images, packages present in the overlapping region and the height of each package;

[0106] Determine the blind spot of each camera in the overlapping area based on the location and height of each package and the location and height of each corresponding camera;

[0107] Determine whether the blind areas of the corresponding multiple cameras have an intersection;

[0108] When it is determined that the blind areas of the fields of view of the corresponding multiple cameras have an intersection, the intersection is determined as the blocked area of the overlapping area.

[0109] It can be understood that the blind areas of the fields of view of the corresponding multiple cameras intersect, that is, the blind areas of the fields of view of the multiple cameras have overlapping areas. That is, if the blind areas of the fields of view of the cameras corresponding to the overlapping areas intersect, it means that each camera corresponding to the overlapping area cannot obtain the status information of the package in the intersection. Therefore, the intersection is determined as the blocked area of the overlapping area.

[0110] In this embodiment, for each overlapping area, after obtaining multiple first images acquired by all corresponding cameras at the same time, the pixel points in the multiple first images are first converted from the corresponding image coordinate system to the world coordinate system, and then the packages in the multiple first images are matched to determine the multiple packages in the overlapping area. For specific methods, please refer to the Chinese patent application with application number 2021106324663. After determining the packages in the overlapping area, the height of each package is determined based on the multiple first images, wherein the height of each package can be determined by a known technology. For example, the two adjacent cameras corresponding to the overlapping area are regarded as binocular cameras, and the binocular camera ranging method is used to obtain the height of each package. After determining the height of each package, the blind spot of the field of view of each camera in the overlapping area is determined based on the position and height of each package and the position and height of each camera corresponding to the overlapping area. Figure 7 This is a schematic diagram of the working principle of a 2D camera in one embodiment of the present application. Figure 7 As shown in the figure, points E, F, B, and C are the four corners of the package being inspected, point O represents the position of the 2D camera, and points A, B, C, and D are on the conveying plane. When the 2D camera captures images of the conveying plane, the package being inspected has a certain height, which causes the 2D camera to have a blind spot, namely the area between points A and B in the figure. Let the height of the package being inspected in the overlapping area determined by the first image be H, and the distance from the point A on the package being inspected that is farthest from the center point D of the camera's field of view to the center point of the camera's field of view be L, calculated based on the first image. Figure 7 It is easy to know that H corresponds to BE and L corresponds to AD. Figure 7It can be seen that the area within the range AB caused by the detected package cannot be captured by the camera, that is, it is the blind spot of the camera's field of view. Since AB / AD = BE / OD, where OD is the installation height of the camera, this value is the known set value after the camera is installed. From the above, it is easy to get AB = AD*BE / OD. By substituting the detected values of H, L and the known OD into the above formula, the length AB of the blind spot of the camera's field of view in the overlapping area that captured the first image can be calculated.

[0111] Step S200: When it is determined that there is a monitoring blind spot on the conveying plane, it is determined whether there is a target package that is aligned with the monitoring blind spot along the second direction.

[0112] Taking the package separation device 010 provided in the embodiment of the present application as an example, after determining that there is a monitoring blind spot on the conveying plane and determining the position of the monitoring blind spot, the control device 400 determines whether there is a target package side by side with the monitoring blind spot along the second direction.

[0113] Figure 8 This is a flow chart for determining whether there is a target package that is aligned with the monitoring blind zone along the second direction in one embodiment of the present application. Figure 8 As shown, optionally, step S200 specifically includes:

[0114] Step S210, obtaining a first coordinate range of the monitoring blind area along the first direction and a second coordinate range along the second direction;

[0115] Step S220, determining whether there is a package that intersects the coordinate range of the monitoring blind spot along the first direction and is adjacent to the second coordinate range of the monitoring blind spot along the second direction;

[0116] Step S230: If there is a package that intersects the coordinate range of the monitoring blind spot along the first direction and is adjacent to the second coordinate range of the monitoring blind spot along the second direction, the package is determined as a target package.

[0117] by Figure 6 Taking the conveying plane shown as an example, the blocked area A1 is a monitoring blind spot. The horizontal coordinate values of the two endpoints of the monitoring blind spot in the x direction (i.e., the first direction) are x1 and x2 respectively; the vertical coordinate values of the two endpoints of the monitoring blind spot in the y direction (i.e., the second direction) are y1 and y2 respectively. When judging whether package A is the target package, it is judged whether there are points on package A with horizontal coordinates in the interval [x1, x2], and whether there are points on package A with vertical coordinates adjacent to the interval [y1, y2]. If there are points on package A with horizontal coordinates in the interval [x1, x2] and points with vertical coordinates adjacent to the interval [y1, y2], then package A is the target package. Figure 6It is obvious that package A intersects with the first coordinate range of the monitoring blind spot in the first direction and is adjacent to the second coordinate range of the monitoring blind spot in the second direction. Therefore, package A is determined as the target package in this embodiment.

[0118] It should be understood that in an optional embodiment, package A being adjacent to the second coordinate range of the monitoring blind spot along the second direction may mean that package A and the monitoring blind spot are connected in the second direction, or it may mean that the distance between package A and the monitoring blind spot in the second direction is less than a set value.

[0119] Step S300: When it is determined that there is a target package parallel to the monitoring blind spot along the second direction, a first conveying assembly located in the monitoring blind spot and a second conveying assembly carrying the target package are determined.

[0120] Taking the package separation device 010 of an embodiment of the present application as an example, in the process of determining the first conveying component in the monitoring blind spot and the second conveying component carrying the target package, the determined first conveying component in the monitoring blind spot may be one or more, and the determined second conveying component carrying the target package may also be one or more.

[0121] Step S400: The first conveying component and the second conveying component are set to have different conveying speeds, so that when there is a package in the monitoring blind area, the package in the monitoring blind area is separated from the target package.

[0122] In an optional embodiment, the conveying speed of the second conveying assembly is set to be greater than the conveying speed of the first conveying assembly.

[0123] If the target package is the one that causes the blind spot, by setting the first conveyor assembly and the second conveyor assembly to different conveying speeds, the target package can be quickly separated from the blind spot, exposing the original blind spot to the camera's field of view. This allows the visual monitoring device 200 to capture an image of a package obscured by the target package in the original blind spot, allowing the package separation device 010 to determine the package's status based on the image and convey it downstream according to a preset separation strategy. If the target package is not the one that causes the blind spot, by setting the first conveyor assembly in the blind spot and the second conveyor assembly carrying the target package to different conveying speeds, it is also possible to avoid the situation where a package that may be in the blind spot is not separated from the target package.

[0124] Optionally, when it is determined that the monitoring blind spot includes multiple first conveying components, when the first conveying components and the second conveying components are set to have different conveying speeds, the conveying speeds of the multiple first conveying components are set to have the same value; when it is determined that the target package is carried by multiple second conveying components, when the first conveying components and the second conveying components are set to have different conveying speeds, the conveying speeds of the multiple second conveying components are set to have the same value.

[0125] Figure 9 This is a flow chart of setting the first conveying component and the second conveying component to have different conveying speeds in one embodiment of the present application. Figure 9 As shown, in another optional embodiment, step S400 specifically includes:

[0126] Step S410, determining the area of the monitoring blind spot;

[0127] Step S420, determining the difference between the conveying speeds of the first conveying assembly and the second conveying assembly based on the area of the monitoring blind spot;

[0128] Step S430 , setting the conveying speeds of the first conveying assembly and the second conveying assembly according to the difference.

[0129] In one embodiment, a relationship table is pre-set between the area of the monitoring blind zone and the difference in conveying speed between the first conveyor assembly and the second conveyor assembly. Optionally, the relationship table satisfies the following conditions: the larger the area of the monitoring blind zone, the larger the difference. Thus, when the area of the monitoring blind zone is large, the first conveyor assembly and the second conveyor assembly have a larger difference in conveying speed, so that the target package can be separated from the monitoring blind zone relatively quickly, so that the original monitoring blind zone is exposed to the camera's field of view as soon as possible, so as to quickly determine whether there is a package in the area. This avoids the situation where the target package takes a long time to separate from the monitoring blind zone when the area of the monitoring blind zone is large (usually due to the large volume of the target package), thereby affecting the conveying of other packages and improving the package conveying efficiency. Optionally, in other embodiments, the difference in conveying speed between the first conveyor assembly and the second conveyor assembly can also be calculated based on the area of the monitoring blind zone and a preset formula. For example, the difference DV between the conveying speeds of the first conveyor assembly and the second conveyor assembly can be calculated by the following formula: DV=a*S, where a is a fixed coefficient and S is the area of the monitoring blind zone.

[0130] In summary, the parcel separation device 010 and method provided in the embodiments of the present application can, when determining the presence of a monitoring blind spot on the conveying plane based on images captured by the visual monitoring device 200, determine whether there is a target parcel that is parallel to the monitoring blind spot along the second direction. If it is determined that there is a target parcel that is parallel to the monitoring blind spot along the second direction, the first conveying assembly in the monitoring blind spot and the second conveying assembly carrying the target parcel are determined, and the first conveying assembly and the second conveying assembly are set to have different conveying speeds. Therefore, when there is a parcel in the monitoring blind spot, the parcel in the monitoring blind spot is separated from the target parcel. Therefore, through the embodiments of the present application, when there is a tall parcel on the conveying plane of the conveying device 100, if the tall parcel obstructs the parcel being conveyed parallel to the tall parcel and cannot be monitored, by setting the first conveying assembly in the monitoring blind spot and the second conveying assembly carrying the tall parcel to have different conveying speeds, the tall parcel can be separated from the parcels in the monitoring blind spot, avoiding the phenomenon of parcel separation failure caused by the tall parcel being conveyed parallel to the parcels in the monitoring blind spot, thereby improving the success rate of parcel separation by the parcel separation device 010.

[0131] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A package separation device, characterized in that: The system comprises a control device, a conveying device for conveying packages in a first direction, and a visual monitoring device disposed above the conveying device. The conveying device comprises a plurality of conveying assemblies arranged in multiple rows along the first direction and multiple columns along a second direction, the first direction being perpendicular to the second direction. The visual monitoring device comprises at least one camera, each of which is configured to capture an image of a set area of a conveying plane of the conveying device. The conveying device and the visual monitoring device are both electrically connected to the control device. The control device is configured to: Determining whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device; In the case where it is determined that the monitoring blind spot exists on the conveying plane, determining whether there is a target package parallel to the monitoring blind spot along the second direction; When it is determined that the target package exists along the second direction and is parallel to the monitoring blind spot, determining a first conveying assembly located in the monitoring blind spot and a second conveying assembly carrying the target package; The first conveying assembly and the second conveying assembly are configured to have different conveying speeds, so that when there is a package in the monitoring blind area, the package in the monitoring blind area is separated from the target package; The step of determining whether there is a target package aligned with the monitoring blind area along the second direction when it is determined that the monitoring blind area exists on the conveying plane includes: Obtaining a first coordinate range of the monitoring blind area along the first direction and a second coordinate range along the second direction; Determining whether there is a package that intersects the first coordinate range of the monitoring blind spot along the first direction and is adjacent to the second coordinate range of the monitoring blind spot along the second direction; If there is a package that intersects the first coordinate range of the monitoring blind spot along the first direction and is adjacent to the second coordinate range of the monitoring blind spot along the second direction, the package is determined as the target package.

2. The package separation device according to claim 1, characterized in that: The step of determining whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device includes: Determining whether there is an obstructed area on the conveying plane based on the image captured by the visual monitoring device, wherein the obstructed area refers to an area whose image cannot be captured by all cameras of the visual monitoring device; When it is determined that the blocked area exists on the conveying plane, the blocked area is determined as the monitoring blind area on the conveying plane.

3. The package separation device according to claim 1, characterized in that: The step of determining whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device includes: Determining whether there is an obstructed area on the conveying plane based on the image captured by the visual monitoring device, wherein the obstructed area refers to an area whose image cannot be captured by all cameras of the visual monitoring device; When it is determined that the obstructed area exists on the conveying plane, performing package tracking based on the historical images captured by the visual monitoring device, and determining whether the presence of a package in the obstructed area can be determined based on the package tracking results; When it is determined that the state of whether there is a package in the blocked area cannot be determined based on the package tracking result, the blocked area is determined as the monitoring blind area on the conveying plane.

4. The package separation device according to claim 2 or 3, characterized in that: The visual monitoring device includes a plurality of cameras, and the step of determining whether there is an obstructed area on the conveying plane based on the image captured by the visual monitoring device includes: Determining an overlapping area and a non-overlapping area on the conveying plane; wherein the overlapping area refers to an area on the conveying plane whose images can be captured by multiple cameras, and the non-overlapping area refers to an area on the conveying plane whose image can be captured by only one camera; Determine whether the overlapping area has the blocked area according to the first images taken by all the cameras corresponding to each overlapping area, and determine whether the non-overlapping area has the blocked area according to the second images taken by the cameras corresponding to each non-overlapping area.

5. The package separation device according to claim 4, characterized in that: The plurality of cameras are all 3D cameras, and the step of determining whether the overlapping area includes the blocked area based on the first images captured by all the cameras corresponding to each overlapping area includes: Acquire the first images captured by all the cameras corresponding to the overlapping area at a set time; fusing the plurality of first images to obtain a fused image, and detecting whether there is a blank area in the fused image; When it is detected that the blank area exists in the fused image, the area on the conveying plane corresponding to the blank area is determined as the blocked area in the overlapping area.

6. The package separation device according to claim 4, characterized in that: The plurality of cameras are all 2D cameras, and the step of determining whether the overlapping area includes the blocked area based on the first images captured by all the cameras corresponding to each overlapping area includes: Acquire the first images captured by all the cameras corresponding to the overlapping area at a set time; determining, based on the plurality of first images, the packages present in the overlapping area and the height of each of the packages; Determining a blind spot of each camera in the overlapping area according to the position and height of each package and the position and height of each corresponding camera; Determine whether the blind areas of the fields of view of the corresponding multiple cameras intersect; When it is determined that the blind areas of the corresponding plurality of cameras have an intersection, the intersection is determined as the blocked area of the overlapping area.

7. The package separation device according to claim 1, characterized in that: The step of setting the first conveying assembly and the second conveying assembly to have different conveying speeds includes: The conveying speed of the second conveying assembly is set to be greater than the conveying speed of the first conveying assembly.

8. The package separation device according to claim 1, characterized in that: The step of setting the first conveying assembly and the second conveying assembly to have different conveying speeds includes: Determining the area of the monitoring blind zone; Determining a difference in conveying speed between the first conveying assembly and the second conveying assembly according to an area of the monitoring blind zone; The conveying speeds of the first conveying assembly and the second conveying assembly are set according to the difference.

9. A package separation method, applied to a package separation device, characterized in that: The parcel separation device includes a conveying device for conveying parcels in a first direction and a visual monitoring device disposed above the conveying device. The conveying device includes a plurality of conveying assemblies arranged in multiple rows along the first direction and multiple columns along a second direction, the first direction being perpendicular to the second direction. The visual monitoring device includes at least one camera, each of the cameras being configured to capture an image of a set area of a conveying plane of the conveying device. The parcel separation method includes: Determining whether there is a monitoring blind spot on the conveying plane based on the image captured by the visual monitoring device; In the case where it is determined that the monitoring blind spot exists on the conveying plane, determining whether there is a target package parallel to the monitoring blind spot along the second direction; When it is determined that the target package exists along the second direction and is parallel to the monitoring blind spot, determining a first conveying assembly located in the monitoring blind spot and a second conveying assembly carrying the target package; The first conveying assembly and the second conveying assembly are configured to have different conveying speeds, so that when there is a package in the monitoring blind area, the package in the monitoring blind area is separated from the target package; The step of determining whether there is a target package aligned with the monitoring blind area along the second direction when it is determined that the monitoring blind area exists on the conveying plane includes: Obtaining a first coordinate range of the monitoring blind area along the first direction and a second coordinate range along the second direction; Determining whether there is a package that intersects the first coordinate range of the monitoring blind spot along the first direction and is adjacent to the second coordinate range of the monitoring blind spot along the second direction; If there is a package that intersects the first coordinate range of the monitoring blind spot along the first direction and is adjacent to the second coordinate range of the monitoring blind spot along the second direction, the package is determined as the target package.

Images