Method and Device for Calculating Carriage Loading and Unloading Rate

Abstract

The present invention provides a method and device for calculating the loading and unloading rate of a carriage. The method includes: obtaining the point cloud data inside the carriage and the size information of the carriage; dividing the point cloud data into multiple voxels according to the preset voxel size information; determining the type of each voxel according to whether the voxel includes valid point cloud and the position of the voxel in the depth direction; calculating the total volume of the multiple voxels according to the preset volume calculation method corresponding to the type of the voxel; and calculating the loading and unloading rate of the carriage according to the total volume and the size information of the carriage. The present invention can divide the point cloud data of the carriage into multiple voxels, classify each voxel according to different cargo stacking methods, and preset different volume calculation methods for different types of voxels. After obtaining the total volume, the loading rate of the carriage can be calculated, which can improve the operation efficiency and the accuracy of the loading and unloading rate.

Description

Technical Field

[0001] The present invention relates to the technical field of depth cameras, and more particularly, to a method and device for calculating the loading and unloading rate of a carriage. Background Art

[0002] Driven by the continuous development of e-commerce, the competition in the express logistics industry has become increasingly fierce. Accurately and real-time tracking of the change in the loading and unloading rate of express transport vehicles has become one of the necessary means to improve operational efficiency. Currently, enterprises generally measure the change in the loading and unloading rate by installing sensors in the carriage. Among them, depth cameras stand out due to their advantages such as being unaffected by environmental light changes, having three-dimensional ranging information, and being easy to install. The existing measurement schemes for the loading and unloading rate of a carriage based on depth cameras generally directly adopt the grid projection method and the point cloud triangulation method, without considering the interference of the difference in the placement of goods in the carriage on the measurement method, resulting in a large error in the loading and unloading rate result. Summary of the Invention

[0003] The problem solved by the present invention is that the error of the measurement result of the loading and unloading rate in the existing measurement scheme for the loading and unloading rate of a carriage based on a depth camera is relatively large.

[0004] To solve the above problem, the present invention provides a method for calculating the loading and unloading rate of a carriage, including: obtaining the point cloud data in the carriage and the size information of the carriage; dividing the point cloud data into a plurality of voxels according to the preset voxel size information; the preset voxel size information includes the height, width, and depth of the voxel; determining the type of each voxel according to whether the voxel includes valid point cloud and the position of the voxel in the depth direction; the types include compact cargo voxels, occluded cargo voxels, outlier cargo voxels, full cargo voxels, and idle voxels; calculating the total volume of the plurality of voxels according to the preset volume calculation method corresponding to the type of the voxel; calculating the loading and unloading rate of the carriage according to the total volume and the size information of the carriage.

[0005] Optionally, determining the type of each voxel according to whether there is valid point cloud in the voxel and the position of the voxel in the depth direction includes: traversing each voxel channel in the height direction and width direction; the voxel channel is a voxel queue along the depth direction; determining the voxel containing the farthest valid point cloud on each voxel channel as a compact cargo voxel; determining the voxel with the number of channels greater than that of the compact cargo voxel on each voxel channel as an occluded cargo voxel; determining the voxel with the number of channels less than that of the compact cargo voxel and containing valid point cloud on each voxel channel as an outlier cargo voxel; wherein, if the voxels in the same column as the outlier cargo voxel and with the column number greater than that of the outlier cargo voxel are all idle voxels, then this outlier cargo voxel is determined as the first type of outlier cargo voxel, otherwise, it is determined as the second type of outlier cargo voxel; if there are no voxels containing valid point cloud on a certain voxel channel, then determining the first voxel on this channel as a full cargo voxel, and determining the types of the remaining voxels as occluded cargo voxels; determining the voxels that do not belong to the above four types as idle voxels.

[0006] Optionally, the volume V of the compact cargo voxel 1 The calculation formula is as follows:

[0007]

[0008] wherein, c i represents the number of channels of the i-th compact cargo voxel, w represents the width of a single voxel, h represents the height of a single voxel, d i represents the depth mean value of the point cloud data in the voxel, and m represents the total number of voxels belonging to the compact cargo voxel;

[0009] The volume V of the outlier cargo voxel 2 The calculation formula is as follows:

[0010]

[0011] wherein, k 1 , k 2 respectively represent the total numbers of the first type of discrete cargo voxels and the second type of discrete cargo voxels; w i , w j respectively represent the absolute values of the width differences of the point cloud sets in the i-th and j-th voxels belonging to the first type of discrete cargo voxels and the second type of discrete cargo voxels, h i represents the absolute value of the height difference of the point cloud set in the i-th voxel belonging to the first type of discrete cargo voxels, ΔH j represents the actual height value of the j-th voxel of the second type of discrete cargo voxels from the carriage floor, row j represents the row number where the j-th second type of discrete cargo voxel is located; Δd i , Δd irespectively represent the absolute value of the depth difference of the point cloud sets within the i-th and j-th voxels belonging to the first type of discrete cargo voxels and the second type of discrete cargo voxels;

[0012] the volume V of the voxels in the full cargo state 3 The calculation formula is as follows:

[0013]

[0014] where n represents the total number of voxels belonging to the full cargo voxels, w represents the width of a single voxel, h represents the height of a single voxel, and D device represents the distance from the point cloud data acquisition device to the front end face of the carriage.

[0015] Optionally, the calculation formula for the loading and unloading rate r of the carriage is as follows:

[0016]

[0017] where V 1 represents the volume of the compact cargo voxels, V 2 represents the volume of the discrete cargo voxels, V 3 represents the volume of the full cargo voxels, D device represents the distance from the point cloud data acquisition device to the front end face of the carriage, H represents the height of the carriage calibrated based on the point cloud data, and W represents the width of the carriage calibrated based on the point cloud data.

[0018] Optionally, determining the type of each of the voxels according to whether the voxel includes valid point cloud and the position of the voxel in the depth direction further includes: if the voxels in the same column as the full cargo voxels and with a larger column number are all in the full cargo voxel state, then determine that the full cargo voxels are caused by full cargo, otherwise they are caused by cargo occlusion; if it is caused by cargo occlusion, then fill the voxels on each voxel channel with the compact cargo voxels belonging to the adjacent voxel channels on both sides in the width direction of the voxel channel; determine the type of each voxel on the filled voxel channel.

[0019] Optionally, filling the voxels on each voxel channel with the compact cargo voxels belonging to the adjacent voxel channels on both sides in the width direction of the voxel channel includes: if there is a compact cargo voxel on one side in the width direction, then fill the voxel on the voxel channel with the same depth as the compact cargo voxel with the first point cloud mean value of the compact cargo voxel; if there are compact cargo voxels on both sides in the width direction, then calculate the second point cloud mean value of the two compact cargo voxels, and fill the voxel on the voxel channel with the same depth as the compact cargo voxel with the second point cloud mean value; if there are no voxels containing valid point cloud on both sides, then fill in the next iteration.

[0020] Optionally, the obtaining of the size information of the carriage includes: projecting the point cloud data in the depth direction to obtain corresponding projection data; intercepting the projection data of a partial area near the image center point of the projection data; and determining the distance from the point cloud data acquisition device to the front end face of the carriage according to the projection data of the partial area near the image center point.

[0021] Optionally, the obtaining of the size information of the carriage further includes: intercepting the point cloud data at a preset position in the depth direction; the preset position does not include the front end face of the carriage; projecting the point cloud data at the preset position in the depth direction, and counting the number of pixels of the carriage wall in the height direction and the width direction; and determining the height and width of the carriage respectively according to the differences in the height direction and the width direction corresponding to the maximum value of the number of pixels of the carriage wall.

[0022] The present invention provides a carriage loading and unloading rate calculation device, including: an acquisition module for acquiring point cloud data inside the carriage and the size information of the carriage; a voxel segmentation module for segmenting the point cloud data into a plurality of voxels according to preset voxel size information; the preset voxel size information includes the height, width and depth of the voxel; a voxel type determination module for determining the type of each voxel according to whether the voxel includes valid point cloud and the position of the voxel in the depth direction; the types include compact cargo voxels, occluded cargo voxels, outlier cargo voxels, full cargo voxels, and idle voxels; a cargo volume calculation module for calculating the total volume of the plurality of voxels according to a preset volume calculation method corresponding to the type of the voxel; and a loading and unloading rate calculation module for calculating the loading and unloading rate of the carriage according to the total volume and the size information of the carriage.

[0023] Optionally, the voxel type determination module is specifically configured to: traverse each voxel channel in the height direction and the width direction; the voxel channel is a voxel queue in the depth direction; determine the voxel containing the farthest valid point cloud on each voxel channel as a compact cargo voxel; determine the voxel with the number of channels greater than that of the compact cargo voxel on each voxel channel as an occluded cargo voxel; determine the voxel with the number of channels less than that of the compact cargo voxel and containing valid point cloud on each voxel channel as an outlier cargo voxel; wherein, if the voxels in the same column as the outlier cargo voxel and with the number of columns greater than that of the outlier cargo voxel are all idle voxels, then this outlier cargo voxel is determined as a first type of outlier cargo voxel, otherwise, it is determined as a second type of outlier cargo voxel; if there are no voxels containing valid point cloud on a certain voxel channel, then determine the first voxel on that channel as a full cargo voxel, and determine the types of the remaining voxels as occluded cargo voxels; and determine the voxels that do not belong to the above four types as idle voxels.

[0024] The method and device for calculating the loading and unloading rate of a carriage provided by the present invention divide the point cloud data of the carriage into multiple voxels, classify each voxel according to different cargo stacking methods, and preset different volume calculation methods for different types of voxels. After obtaining the total volume, the loading rate of the carriage can be calculated, which can improve the operation efficiency and the accuracy of the loading and unloading rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0026] Figure 1 It is a schematic diagram of the installation effect of the depth camera in the embodiment of the present invention;

[0027] Figure 2 It is a schematic flowchart of a method for calculating the loading and unloading rate of a carriage in an embodiment of the present invention;

[0028] Figure 3 It is a schematic structural diagram of the voxelization of point cloud data in the embodiment of the present invention;

[0029] Figure 4 It is a schematic diagram of the front-end face projection of the carriage in the embodiment of the present invention;

[0030] Figure 5 It is a schematic diagram of the slice projection of the carriage in the embodiment of the present invention;

[0031] Figure 6 It is a schematic structural diagram of a device for calculating the loading and unloading rate of a carriage in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the following will give a detailed description of the specific embodiments of the present invention with reference to the drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0033] Figure 1 It is a schematic diagram of the installation effect of the depth camera, and the depth camera is used to collect the point cloud data or depth map inside the carriage. As Figure 1As shown in the figure, a depth camera 102 is installed at the top corner inside the carriage 101. Some depth cameras can directly output point cloud data, and some depth cameras can output depth maps. Then, the depth values can be converted into point cloud data by combining the internal parameters of the depth camera. Point cloud data refers to a dataset of points in a certain coordinate system, usually including three-dimensional coordinates X, Y, and Z; a depth map refers to an image in 3D computer graphics and computer vision that contains information related to the distance from the surface of the scene object to the viewpoint.

[0034] In Figure 1 it is shown that the X-axis represents the width direction of the carriage, the Y-axis represents the height direction of the carriage, and the Z-axis represents the depth direction of the carriage. The positive direction of the Z-axis is defined as the far end, and the negative direction of the Z-axis is defined as the near end.

[0035] Figure 2 FIG. is a schematic flowchart of a method for calculating the loading and unloading rate of a carriage in an embodiment of the present invention. The method includes:

[0036] S202, obtaining the point cloud data inside the carriage and the dimension information of the carriage.

[0037] Since the point cloud data acquisition device (such as a depth camera) is installed inside the carriage, the distance from the device to the front end face of the carriage is not equal to the length of the carriage at this time. Therefore, the three-dimensional dimensions of the carriage can be calibrated based on the point cloud data inside the carriage to determine the dimension information of the carriage.

[0038] S204, dividing the point cloud data into multiple voxels according to the preset voxel size information.

[0039] Among them, a voxel is a volume element, which is a cubic block with a preset size. Considering that the point cloud data is sparse and large in quantity, for this reason, the point cloud is voxelized to form a dense and regular data structure. To ensure that the point cloud is dense after voxelization, the height, width, and depth values of a single voxel block need to be estimated according to the distance from the device to the front end face of the carriage, the resolution, and the field of view angle. The preset voxel size information includes the height, width, and depth of the voxel.

[0040] After the point cloud data is voxelized, a three-dimensional data with a data structure of height, width, and depth (or called channels) is obtained. Figure 3 FIG. shows a schematic structural diagram of the voxelization of the point cloud data. Among them, the X-axis is the width direction of the voxel, the Y-axis is the height direction of the voxel, and the Z-axis is the depth direction of the voxel. In Figure 3 multiple channels are formed by voxels along the Z-axis.

[0041] For example, when the resolution of the depth camera is 320*240, the field of view angle is 60°*45°, and the carriage of 9.6 meters is photographed, by calculation, the physical size represented by the point cloud at the farthest distance is about 4 cm. Therefore, a voxel block of 4 cm*4 cm*10 cm can be adopted, and the point cloud within each voxel is classified into the same target.

[0042] S206. Determine the type of each voxel according to whether the voxel includes valid point cloud and the position of the voxel in the depth direction.

[0043] During the actual process of loading and unloading goods, the goods in the carriage are not closely connected from the inside to the outside, and there may even be a large gap between the goods. As the goods are continuously stacked, there may be void data in the point cloud, which is caused by goods occlusion, the invisible internal space of the target, or the goods being fully stacked and entering the detection blind area. Based on the above analysis, it is necessary to classify the voxels. In this embodiment, the classification criteria for the goods point cloud are defined, and specifically, the following five types of voxels are defined: compact goods voxels, occluded goods voxels, outlier goods voxels, full occupancy goods voxels, and idle voxels.

[0044] Optionally, first, traverse each voxel channel in the above-mentioned height direction and width direction. The voxel channel is a voxel queue along the depth direction; then, determine the type of each voxel in the following manner:

[0045] (1) Determine the voxel containing the farthest valid point cloud on each voxel channel as a compact goods voxel. Specifically, define the set of voxels containing valid point cloud data at the farthest position on any voxel channel as a compact goods voxel.

[0046] (2) Determine the voxel with the number of channels greater than that of the compact goods voxel on each voxel channel as an occluded goods voxel. Define the set of voxels located on the same voxel channel and with the number of channels greater than that of the first type of voxels as an occluded goods voxel.

[0047] (3) Determine the voxel with the number of channels less than that of the compact goods voxel and containing valid point cloud on each voxel channel as an outlier goods voxel; among them, if all the voxels in the same column as the outlier goods voxel and with the number of columns greater than that of the outlier goods voxel are idle voxels, then this outlier goods voxel is determined as the first type of outlier goods voxel, otherwise, it is determined as the second type of outlier goods voxel. Define the set of voxels located on the same voxel channel and with the number of channels less than that of the compact goods voxel channel and containing valid point cloud as an outlier goods voxel, and specifically divide it into the first type of outlier goods voxel and the second type of outlier goods voxel.

[0048] (4) If there are no voxels containing valid point clouds on a certain voxel channel, then the first voxel on this channel is determined as a voxel filled with goods, and the types of the remaining voxels are determined as occluded goods voxels. If there are no voxels containing valid point clouds when traversing a voxel channel, the first voxel on this voxel channel is defined as a voxel filled with goods, and the remaining voxels are defined as occluded goods voxels.

[0049] (5) Determine the voxels that do not belong to the above four types as idle voxels.

[0050] S208. Calculate the total volume of the above-mentioned multiple voxels according to the preset volume calculation method corresponding to the type of the voxel.

[0051] Based on the above voxel types, this embodiment provides different preset volume calculation methods. Specifically, for compact goods voxels and occluded goods voxels located on the same voxel channel, the volume of the goods on this voxel channel can be calculated based on the quantity of the two and the volume of the compact goods voxels; for outlier goods voxels, the volume of the goods on this voxel channel can be calculated based on its quantity and the occupied volume of the point cloud in each voxel; for voxels filled with goods and voxels in the occluded goods state located on the same voxel channel, the volume of the goods on this voxel channel can be calculated based on the overall length, width, and height of the voxel channel. It should be noted that for idle voxels that do not belong to any of the above types, they are not occupied by goods, that is, the volume of the corresponding goods is zero.

[0052] Based on the above method, the volume of the goods on each voxel channel can be calculated, and then the sum is obtained to get the total volume of the goods in the carriage.

[0053] S210. Calculate the loading and unloading rate of the carriage according to the above total volume and the dimension information of the carriage.

[0054] The carriage loading and unloading rate calculation method provided by the embodiment of the present invention divides the point cloud data of the carriage into multiple voxels, classifies each voxel according to different goods stacking methods, and presets different volume calculation methods for different types of voxels. After obtaining the total volume, the loading rate of the carriage can be calculated, which can improve the operation efficiency and the accuracy of the loading and unloading rate.

[0055] Optionally, based on the classification result of the above voxels, the following four calculation methods are defined.

[0056] (1) The volume V of the compact goods voxel 1 The calculation formula is as follows:

[0057]

[0058] Among them, among them, c irepresents the number of channels of the i-th compact cargo voxel, w represents the width of a single voxel, h represents the height of a single voxel, and d i represents the average depth value of the point cloud data within the voxel, and m represents the total number of compact cargo voxels. w, h, d i When multiplied, the volume of a single voxel is obtained, and d i That is, the average depth value of the point cloud data of the compact cargo voxels on the voxel channel, c i , w, h, d i When multiplied, the volume occupied by the cargo in one channel is obtained. Summing up the volumes occupied by the cargo in m channels gives the total volume occupied by the cargo in this type of channel.

[0059] (2) The volume V of the outlier cargo voxels 2 The calculation formula is as follows:

[0060]

[0061] Among them, k 1 , k 2 respectively represent the total numbers of the first type of discrete cargo voxels and the second type of discrete cargo voxels; w i , w j respectively represent the absolute values of the width differences of the point cloud sets within the i-th and j-th voxels of the first type of discrete cargo voxels and the second type of discrete cargo voxels, h i represents the absolute value of the height difference of the point cloud set within the i-th voxel of the first type of discrete cargo voxels, ΔH j represents the actual height value of the j-th voxel of the second type of discrete cargo voxels from the carriage floor, row j represents the row number where the j-th second type of discrete cargo voxel is located; Δd i , Δd i respectively represent the absolute values of the depth differences of the point cloud sets within the i-th and j-th voxels of the first type of discrete cargo voxels and the second type of discrete cargo voxels.

[0062] (3) The volume V of the full cargo voxels 3 The calculation formula is as follows:

[0063]

[0064] Among them, n represents the total number of full cargo voxels, w represents the width of a single voxel, h represents the height of a single voxel, and D device represents the distance from the point cloud data acquisition device to the front end face of the carriage.

[0065] (4) The volume of the occluded cargo voxels is already included in the volume calculation formulas of the compact cargo voxels or the full cargo voxels, so there is no need to calculate it repeatedly.

[0066] Based on the above calculation formula, the calculation formula for the loading and unloading rate r of the carriage is as follows:

[0067]

[0068] Among them, V 1 represents the volume of the compact cargo voxels, V 2 represents the volume of the discrete cargo voxels, V 3 represents the volume of the full cargo voxels, D device represents the distance from the point cloud data acquisition device to the front end face of the carriage, H represents the height of the carriage calibrated based on the point cloud data, and W represents the width of the carriage calibrated based on the point cloud data.

[0069] Considering that as the goods are continuously stacked, there may be void data in the point cloud. This phenomenon is caused by cargo occlusion, the invisible internal space of the target, or the goods filling up and entering the detection blind area. For example, since the above point cloud data acquisition device is installed in the corner, the set of void voxels identified as full cargo status may be generated by the goods protruding from the left and right sides blocking or the goods exceeding the field of view angle of the device. Therefore, it is necessary to identify the reasons for the generation of void voxels.

[0070] Optionally, the classification rule is as follows: If the voxels traversed downward along the column of the full cargo voxels are all voxels in the full cargo state, then the void voxel is caused by the full cargo, otherwise the void voxel is caused by cargo occlusion. Based on this, the above S206 includes the following steps: If the voxels in the same column as the full cargo voxels and with a larger column number are all in the full cargo voxel state, then determine that the full cargo voxel is caused by the full cargo, otherwise it is caused by cargo occlusion; if it is caused by cargo occlusion, then fill the voxels on each voxel channel belonging to the compact cargo voxels on both sides in the width direction of the voxel channel; determine the types of the voxels on the filled voxel channel.

[0071] The void voxels caused by cargo occlusion are filled by the compact cargo voxels in the left and right neighborhoods. The rule is as follows: If there are compact cargo voxels on only one side, directly use the point cloud mean value in this voxel to fill the void voxel. If there are compact cargo voxels on both sides, calculate the point cloud mean value of the two voxels to fill the void voxel. If there are no voxels containing valid point clouds on both sides, enter the next iteration, and the void voxels in the occlusion state can be filled up through continuous iteration.

[0072] Based on this, the above filling step can be performed in the following manner: If there is a compact cargo voxel on one side in the width direction, fill the voxel on the voxel channel with the same depth as the compact cargo voxel with the first point cloud mean value of the compact cargo voxel; if there are compact cargo voxels on both sides in the width direction, calculate the second point cloud mean value of the two compact cargo voxels, and fill the voxel on the voxel channel with the same depth as the compact cargo voxel with the second point cloud mean value; if there are no voxels containing valid point clouds on both sides, fill in the next iteration.

[0073] Among them, if there is a compact cargo voxel in the adjacent channel on either the left or right side in the width direction, for the voxel on the current channel with the same depth as the compact cargo voxel, that is, the adjacent voxel to it, perform filling. Based on the above classification principle, the type of the filled voxel is determined as a compact cargo voxel.

[0074] Considering that there is a certain deviation between the point cloud data of the carriage collected by the device and the ideal carriage size, therefore, perform three-dimensional size calibration based on the point cloud data of the empty carriage collected by the device, and use it to calculate the loading and unloading rate instead of using the volume of the ideal carriage size, so as to improve the accuracy of the loading and unloading rate. Based on this, the above S202 can determine the length of the carriage in the following manner, that is, the above D device : Project the point cloud data along the depth direction to obtain the corresponding projection data; intercept the projection data of a partial area near the image center point of the projection data; determine the distance from the point cloud data acquisition device to the front end face of the carriage according to the projection data of the partial area near the image center point.

[0075] When calibrating the length of the carriage, it is necessary to project the carriage point cloud data along the Z axis to estimate the true distance from the device to the front end face of the carriage. Figure 4 This is a schematic diagram of the projection of the front end face of the carriage. To avoid the interference of the data of the surrounding carriage walls on the distance estimation, the projection data of a partial area near the image center point can be used for calculation. Exemplarily, only take Figure 4 the projection data of the quarter area in the projection plane divided by the image center point in , and obtain the mean value or median according to the projection data of the intercepted area, which is used to represent the approximate distance from the device to the front end face of the carriage.

[0076] The above S202 can determine the height and width of the carriage in the following manner, that is, the above H and W: Intercept the point cloud data at a preset position in the depth direction of the point cloud data; this preset position does not include the front end face of the carriage; project the point cloud data at the preset position along the depth direction, and count the number of pixels of the carriage wall in the height direction and the width direction; determine the height and width of the carriage respectively according to the differences in the height direction and the width direction corresponding to the maximum value of the number of pixels of the carriage wall.

[0077] When calibrating the height and width of the carriage, the point cloud data at the middle in the length direction of the carriage can be used to avoid the influence of the front end face of the carriage. Exemplarily, the point cloud data of the first n - 1 meters in front of the carriage can be intercepted, where n represents the length of the carriage, and the point cloud is projected onto the Z - axis direction. Figure 5 It is a schematic diagram of the projection of the carriage slice.

[0078] In Figure 5 Among them, white pixels represent the point cloud of the empty carriage wall, and black pixels represent the point cloud without targets. Optionally, count the number of white pixels by rows and columns. When processing images, the row refers to the height direction, and the column refers to the width direction, so that two histograms can be generated. The horizontal axes of the two histograms respectively represent the width position and the height position, and the vertical axis represents the number of white pixels. There are two maximum values in the row histogram, corresponding to the two side walls of the carriage respectively, and there are two maximum values in the column histogram, corresponding to the top wall and the bottom plate of the carriage respectively. By searching for the indexes of the two highest values on both sides of the histogram, the x min , x max , y min , y max of the carriage wall can be located, so as to calculate that the height value of the carriage is y min - y max and the width value is x max - x min .

[0079] The embodiment of the present invention provides a method for identifying the point cloud data of goods in different placement states in a carriage, designing different calculation methods for the loading and unloading rate to improve the self - adaptability of the loading and unloading rate; and performing three - dimensional size calibration according to the point cloud data of the empty carriage collected by the device, and using the calibrated volume of the carriage instead of the volume of the ideal carriage to calculate the loading and unloading rate. According to different stacking methods of goods, a classification standard for goods point clouds is defined, and an adaptive loading and unloading rate calculation formula is designed. When calculating the loading and unloading rate, the volume of the carriage calibrated above is used instead of the volume of the ideal carriage, which improves the operation efficiency and the accuracy of the loading and unloading rate.

[0080] Figure 6 is a schematic structural diagram of a carriage loading and unloading rate calculation device in an embodiment of the present invention. The device includes:

[0081] An acquisition module 601, configured to acquire the point cloud data in the carriage and the size information of the carriage;

[0082] A voxel segmentation module 602, configured to segment the point cloud data into a plurality of voxels according to the preset voxel size information; the preset voxel size information includes the height, width and depth of the voxel;

[0083] A voxel type determination module 603, configured to determine the type of each voxel according to whether the voxel includes valid point cloud and the position of the voxel in the depth direction; the types include compact cargo voxels, occluded cargo voxels, outlier cargo voxels, full cargo voxels, and idle voxels;

[0084] A cargo volume calculation module 604, configured to calculate the total volume of the multiple voxels according to a preset volume calculation method corresponding to the type of the voxel;

[0085] A loading and unloading rate calculation module 605, configured to calculate the loading and unloading rate of the carriage according to the total volume and the dimension information of the carriage.

[0086] Optionally, as an embodiment, the voxel type determination module is specifically configured to:

[0087] Determine the voxel containing the farthest valid point cloud on each voxel channel as a compact cargo voxel;

[0088] Determine the voxel with the number of channels greater than that of the compact cargo voxel on each voxel channel as an occluded cargo voxel;

[0089] Determine the voxel with the number of channels less than that of the compact cargo voxel and containing valid point cloud on each voxel channel as an outlier cargo voxel; wherein, if the voxels in the same column as the outlier cargo voxel and with the number of columns greater than that of the outlier cargo voxel are all idle voxels, then this outlier cargo voxel is determined as the first type of outlier cargo voxel, otherwise, it is determined as the second type of outlier cargo voxel;

[0090] If there is no voxel containing valid point cloud on a certain voxel channel, then determine the first voxel on this channel as a full cargo voxel, and determine the types of the remaining voxels as occluded cargo voxels;

[0091] Determine the voxels that do not belong to the above four types as idle voxels.

[0092] Optionally, as an embodiment, the volume V of the compact cargo voxel 1 The calculation formula is as follows:

[0093]

[0094] Wherein, c i represents the number of channels of the i-th compact cargo voxel, w represents the width of a single voxel, h represents the height of a single voxel, d i represents the depth mean value of the point cloud data in the voxel, and m represents the total number of compact cargo voxels;

[0095] The volume V of the outlier cargo voxel 2 The calculation formula is as follows:

[0096]

[0097] Among them, k 1 and k 2 respectively represent the total numbers of discrete cargo voxels of the first type and the second type; w i and w j respectively represent the absolute values of the width differences of the point cloud sets within the i-th and j-th voxels of the discrete cargo voxels of the first type and the second type, h i represents the absolute value of the height difference of the point cloud set within the i-th voxel of the discrete cargo voxels of the first type, ΔH j represents the actual height value of the j-th voxel of the discrete cargo voxels of the second type from the carriage floor, row j represents the row number where the j-th discrete cargo voxel of the second type is located; Δd i and Δd i respectively represent the absolute values of the depth differences of the point cloud sets within the i-th and j-th voxels of the discrete cargo voxels of the first type and the second type;

[0098] The volume V of the voxels in the full cargo state 3 The calculation formula is as follows:

[0099]

[0100] Among them, n represents the total number of voxels belonging to the full cargo voxels, w represents the width of a single voxel, h represents the height of a single voxel, D device represents the distance from the point cloud data acquisition device to the front end face of the carriage.

[0101] Optionally, as an embodiment, the calculation formula of the loading and unloading rate r of the carriage is as follows:

[0102]

[0103] Among them, V 1 represents the volume of the compact cargo voxels, V 2 represents the volume of the discrete cargo voxels, V 3 represents the volume of the full cargo voxels, D device represents the distance from the point cloud data acquisition device to the front end face of the carriage, H represents the height of the carriage calibrated based on the point cloud data, and W represents the width of the carriage calibrated based on the point cloud data.

[0104] Optionally, as an embodiment, the voxel type determination module is further configured to: if the voxels in the same column as the occupied cargo voxels and with a larger column number are all in the occupied cargo voxel state, determine that the occupied cargo voxels are caused by the occupied cargo, otherwise they are caused by cargo occlusion; if they are caused by cargo occlusion, then fill the voxels on each voxel channel of the compact cargo voxel filling voxel channel according to the adjacent voxel channels on both sides in the width direction of the voxel channel; determine the types of the voxels on the filled voxel channel.

[0105] Optionally, as an embodiment, the voxel type determination module is further configured to: if there are compact cargo voxels on one side in the width direction, fill the voxels on the voxel channel with the same depth as the compact cargo voxels with the first point cloud mean value of the compact cargo voxels; if there are compact cargo voxels on both sides in the width direction, calculate the second point cloud mean value of the two compact cargo voxels, and fill the voxels on the voxel channel with the same depth as the compact cargo voxels with the second point cloud mean value; if there are no voxels containing valid point clouds on both sides, fill them in the next iteration.

[0106] Optionally, as an embodiment, the acquisition module is specifically configured to: project the point cloud data along the depth direction to obtain corresponding projection data; intercept the projection data of a partial area near the image center point of the projection data; determine the distance from the point cloud data acquisition device to the front end face of the carriage according to the projection data of the partial area near the image center point.

[0107] Optionally, as an embodiment, the acquisition module is specifically configured to: intercept the point cloud data at a preset position in the depth direction; the preset position does not include the front end face of the carriage; project the point cloud data at the preset position along the depth direction, and count the number of pixels of the carriage wall in the height direction and the width direction; determine the height and width of the carriage respectively according to the differences in the height direction and the width direction corresponding to the maximum value of the number of pixels of the carriage wall.

[0108] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements each process of the above embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here. Among them, the computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0109] Of course, those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing a control device through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above embodiments of the carriage loading and unloading rate calculation method. The storage medium can be a memory, a disk, an optical disc, etc.

[0110] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0111] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0112] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for calculating the loading and unloading rate of a carriage, characterized in that, it includes: Obtain the point cloud data inside the carriage and the dimension information of the carriage; Divide the point cloud data into multiple voxels according to the preset voxel dimension information; The preset voxel dimension information includes the height, width and depth of the voxel; Determine the type of each voxel according to whether the voxel contains valid point cloud and the position of the voxel in the depth direction; the types include compact cargo voxels, occluded cargo voxels, outlier cargo voxels, full cargo voxels, and idle voxels; Calculate the total volume of the multiple voxels according to the preset volume calculation method corresponding to the type of the voxel; Calculate the loading and unloading rate of the carriage according to the total volume and the dimension information of the carriage; The volume of the compact cargo voxel V 1 The calculation formula is as follows: Among them, c i represents the number of channels of the i th compact cargo voxel, w represents the width of a single voxel, h represents the height of a single voxel, d i represents the depth mean of the point cloud data within the voxel, m represents the total number of compact cargo voxels; The volume of the outlier cargo voxel V 2 The calculation formula is as follows: Among them, k 1 and k 2 respectively represent the total numbers of the first - type discrete cargo volume elements and the second - type discrete cargo volume elements; w i and w j respectively represent the absolute value of the width difference of the point - cloud sets within the i and j th volume elements of the first - type discrete cargo volume elements and the second - type discrete cargo volume elements; h i represents the absolute value of the height difference of the point - cloud set within the i th volume element of the first - type discrete cargo volume elements, Δ H j represents the actual height value of the j th volume element of the second - type discrete cargo volume elements from the carriage floor; row j represents the row number where the j th second - type discrete cargo volume element is located; Δ d i and Δ d i respectively represent the absolute value of the depth difference of the point - cloud sets within the i and j th volume elements of the first - type discrete cargo volume elements and the second - type discrete cargo volume elements; The volume of the voxel in the full-load cargo state V 3 The calculation formula is as follows: Among them, n represents the total number of voxels occupied by the goods, w represents the width of a single voxel, h represents the height of a single voxel, D device represents the distance from the point cloud data acquisition device to the front end face of the carriage.

2. The method according to claim 1, characterized in that, The step of determining the type of each voxel according to whether the voxel contains valid point cloud and the position of the voxel in the depth direction includes: Traverse each voxel channel in the height direction and width direction; the voxel channel is a voxel queue along the depth direction; Determine the voxel containing the farthest valid point cloud on each voxel channel as a compact cargo voxel; Determine the voxel with the number of channels greater than that of the compact cargo voxel on each voxel channel as an occluded cargo voxel; Determine the voxel with the number of channels less than that of the compact cargo voxel and containing valid point cloud on each voxel channel as an outlier cargo voxel; wherein, if the voxels in the same column as the outlier cargo voxel and with a greater number of columns than the outlier cargo voxel are all idle voxels, then this outlier cargo voxel is determined as the first type of outlier cargo voxel, otherwise, it is determined as the second type of outlier cargo voxel; If there are no voxels containing valid point cloud on a certain voxel channel, then determine the first voxel on this channel as a full cargo voxel, and determine the types of the remaining voxels as occluded cargo voxels; Determine the voxels that do not belong to the above four types as idle voxels.

3. The method according to claim 1, characterized in that, The loading and unloading rate of the carriage r The calculation formula is as follows: Among them, V 1 represents the volume belonging to the compact cargo voxel, V 2 represents the volume belonging to the discrete cargo voxel, V 3 represents the volume belonging to the full cargo voxel, D device represents the distance from the point cloud data acquisition device to the front end face of the carriage, H represents the height of the carriage calibrated based on the point cloud data, W represents the width of the carriage calibrated based on the point cloud data.

4. The method according to claim 2, characterized in that, The step of determining the type of each voxel according to whether the voxel contains valid point cloud and the position of the voxel in the depth direction further includes: If the voxels in the same column as the full cargo voxel and with a greater number of columns are all in the state of full cargo voxels, then determine that the full cargo voxel is caused by full cargo, otherwise it is caused by cargo occlusion; If it is caused by cargo occlusion, then fill each voxel on the voxel channel with the compact cargo voxels on the adjacent voxel channels on both sides in the width direction of the voxel channel; Determine the type of each voxel on the filled voxel channel.

5. The method according to claim 4, characterized in that, The step of filling each voxel on the voxel channel with the compact cargo voxels on the adjacent voxel channels on both sides in the width direction of the voxel channel includes: If there is a compact cargo voxel on one side in the width direction, then fill the voxel with the same depth as the compact cargo voxel on the voxel channel with the first point cloud mean value of the compact cargo voxel; If there are compact cargo voxels on both sides in the width direction, calculate the second point cloud mean of the two compact cargo voxels, and fill the voxels on the voxel channel with the same depth as the compact cargo voxels with the second point cloud mean; If there are no voxels containing valid point clouds on both sides, fill them in the next iteration.

6. The method according to any one of claims 1-5, characterized in that, the obtaining the size information of the carriage includes: project the point cloud data along the depth direction to obtain corresponding projection data; intercept the projection data of a partial area near the image center point of the projection data; determine the distance from the point cloud data acquisition device to the front end face of the carriage according to the projection data of the partial area near the image center point.

7. The method according to claim 6, characterized in that, the obtaining the size information of the carriage further includes: intercept the point cloud data at a preset position in the depth direction of the point cloud data; the preset position does not include the front end face of the carriage; project the point cloud data at the preset position along the depth direction, and count the number of carriage wall pixels in the height direction and the width direction; determine the height and width of the carriage respectively according to the differences in the height direction and the width direction corresponding to the maximum value of the number of carriage wall pixels.

8. A carriage loading and unloading rate calculation device, characterized in that, comprising: an acquisition module for acquiring point cloud data in the carriage and the size information of the carriage; a voxel segmentation module for segmenting the point cloud data into a plurality of voxels according to preset voxel size information; the preset voxel size information includes the height, width and depth of the voxels; a voxel type determination module for determining the type of each voxel according to whether the voxel includes valid point clouds and the position of the voxel in the depth direction; the types include compact cargo voxels, occluded cargo voxels, outlier cargo voxels, full cargo voxels, and idle voxels; a cargo volume calculation module for calculating the total volume of the plurality of voxels according to a preset volume calculation method corresponding to the type of the voxels; a loading and unloading rate calculation module for calculating the loading and unloading rate of the carriage according to the total volume and the size information of the carriage; The volume of the compact cargo voxel V 1 The calculation formula is as follows: Among them, c i represents the number of channels of the i th compact cargo voxel, w represents the width of a single voxel, h represents the height of a single voxel, d i represents the depth mean of the point cloud data within the voxel, m represents the total number of compact cargo voxels; The volume of the outlier cargo voxel V 2 The calculation formula is as follows: Among them, k 1 and k 2 respectively represent the total numbers of the first - type discrete cargo volume elements and the second - type discrete cargo volume elements; w i and w j respectively represent the absolute values of the width differences of the point - cloud sets within the i th j individual volume elements of the first - type discrete cargo volume elements and the second - type discrete cargo volume elements; h i represents the absolute value of the height difference of the point - cloud set within the i th H j individual volume element of the first - type discrete cargo volume elements, Δ j represents the actual height value of the row j th j individual volume element of the second - type discrete cargo volume elements from the carriage floor; d i and Δ d i respectively represent the absolute values of the depth differences of the point - cloud sets within the i th j individual volume elements of the first - type discrete cargo volume elements and the second - type discrete cargo volume elements; The volume of the voxel in the full cargo state V 3 The calculation formula is as follows: Among them, n represents the total number of voxel occupied by the goods, w represents the width of a single voxel, h represents the height of a single voxel, D device represents the distance from the point cloud data acquisition device to the front end face of the carriage.

9. The device according to claim 8, characterized in that, the voxel type determination module is specifically used for: traverse each voxel channel in the height direction and the width direction; the voxel channel is a voxel queue along the depth direction; determine the voxel containing the farthest valid point cloud on each voxel channel as a compact cargo voxel; determine the voxel with the number of channels greater than that of the compact cargo voxel on each voxel channel as an occluded cargo voxel; determine the voxel with the number of channels less than that of the compact cargo voxel and containing valid point clouds on each voxel channel as an outlier cargo voxel; wherein, if the voxels in the same column as the outlier cargo voxel and with the number of columns greater than that of the outlier cargo voxel are all idle voxels, then this outlier cargo voxel is determined as a first type of outlier cargo voxel, otherwise, it is determined as a second type of outlier cargo voxel; If there is no voxel containing valid point cloud on a certain voxel channel, the first voxel on this channel is determined as a voxel occupied by goods, and the types of the remaining voxels are determined as voxels of goods being occluded. Voxels that do not belong to the above four types are determined as free voxels.

Images