Method and system for static volume measurement of a package

Abstract

The application relates to a parcel static volume measurement method and system. The method comprises the following steps: obtaining a depth map and an RGB image of a logistics parcel; performing semantic segmentation on the RGB image of the logistics parcel by using a constructed logistics parcel semantic segmentation model to extract a parcel detection basic segmentation mask of the logistics parcel; converting depth data of the parcel detection basic segmentation mask into 3D point cloud coordinates, and projecting each point to a calibration belt surface to obtain a projection surface segmentation mask after projection; determining a parcel height of the logistics parcel based on the projection distance of the 3D point cloud coordinates to the calibration belt surface for the projection surface segmentation mask, and determining a parcel length and a parcel width of the logistics parcel according to the minimum circumscribed rectangle of the projection surface segmentation mask. The application can quickly and effectively realize parcel volume measurement, and improve the accuracy and reliability of parcel volume measurement.

Description

TECHNICAL FIELD

[0001] The present application relates to a volume measurement method and system, in particular to a static volume measurement method and system of a package. BACKGROUND

[0002] In recent years, with the rapid development of logistics technology, the requirement for package volume measurement technology is also increasing in express sorting business. In thousands of packages, classifying packages of different volumes is an important technical link, that is, the accuracy of the package volume measurement result directly affects the subsequent process of express sorting.

[0003] When there is a large error in the package volume measurement, the accuracy of express sorting will be greatly reduced, thereby reducing the efficiency of express package delivery. In addition, in logistics, sometimes the volume of the delivered package needs to be counted, and a more accurate measurement result is required.

[0004] At present, the accuracy of package volume measurement in the express industry is not high enough, and high-precision volume measurement still needs to rely on manual measurement of the size of the package, but manual measurement is difficult to cope with the massive packages to be measured. Therefore, how to quickly and accurately measure the volume of the package has become a problem to be solved. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide a static volume measurement method and system for a package, which can quickly and effectively measure the size of the package and improve the accuracy and reliability of the package size measurement.

[0006] According to the technical scheme provided by the present application, a static volume measurement method for a package, the volume measurement method comprising:

[0007] A logistics package camera is provided for acquiring a depth map and an RGB image, wherein the logistics package camera is calibrated based on the acquired depth data to determine the calibration belt surface of the logistics package conveyor after calibration;

[0008] The depth map and the RGB image of the logistics package acquired based on the logistics package camera are segmented using a constructed logistics package semantic segmentation model to extract a package detection basic segmentation mask of the logistics package;

[0009] The depth data of the package detection basic segmentation mask is converted into 3D point cloud coordinates, and each point is projected to the calibration belt surface to obtain a projection surface segmentation mask after projection;

[0010] The package height of the logistics package is determined based on the projection distance of the 3D point cloud coordinates to the calibration belt surface, and the package length and the package width of the logistics package are determined according to the minimum circumscribed rectangle of the projection surface segmentation mask.

[0011] The static volume of the logistics package is determined based on the determined package height, package length and package width.

[0012] The constructed logistics package semantic segmentation model comprises:

[0013] The logistics package semantic segmentation base model is constructed, wherein the logistics package semantic segmentation base model is based on a network framework of DeepLabv3+, and a network stem part of the network framework adopts an Xception model.

[0014] The constructed logistics package semantic segmentation base model is trained, wherein the training of the logistics package semantic segmentation base model comprises a pre-training stage and a semantic segmentation training stage in sequence.

[0015] In the pre-training stage, the logistics package semantic segmentation base model is trained by using an ImageNet-1k data set, so as to obtain pre-training weight parameters of the logistics package semantic segmentation base model after the pre-training stage ends.

[0016] In the semantic segmentation training stage, the package RGB training image used for making training samples is collected by using a logistics package camera, the package RGB training image is labeled and the ROI region is selected, so as to generate a semantic segmentation training data set.

[0017] Based on the generated semantic segmentation training data set, the logistics package semantic segmentation base model after the pre-training stage is subjected to semantic segmentation training, so as to generate the logistics package semantic segmentation model from the logistics package semantic segmentation base model after a target training state is reached.

[0018] In the pre-training stage, the weight is initialized by using a Kaiming method, and the pre-training is performed by using a fixed number of iterations.

[0019] The learning rate decay strategy is a segmented decay, and the weight with the highest average accuracy is selected as the pre-training weight parameter.

[0020] In the semantic segmentation training stage, the learning rate decay strategy adopts a multi-step decay, and the number of iterations of network training is not less than 3000 times.

[0021] The evaluation indexes are evaluated by using the average intersection over union and the prediction accuracy, the evaluation indexes are calculated once every 50 generations, and the change curve of each evaluation index data is recorded.

[0022] When the projection screen segmentation mask is obtained based on the parcel detection basic segmentation mask, the method comprises:

[0023] The depth data corresponding to the parcel detection basic segmentation mask is converted into corresponding 3D point cloud coordinates, and each point cloud is projected towards the calibration belt surface to determine the projection distance and the 3D point cloud coordinates of the projection point after projection;

[0024] The 3D point cloud coordinates of the projection point are converted into 2D coordinates on the calibration belt surface based on the depth camera internal parameter of the logistics parcel camera, and the projection surface segmentation mask is obtained based on the converted 2D coordinates on the calibration belt surface.

[0025] When the point cloud is projected towards the calibration belt surface to determine the projection distance, the method comprises:

[0026] A projection distance threshold is configured, and the projection distance is screened using the configured projection distance threshold to delete the projection distance less than the configured projection distance threshold.

[0027] For all projection points, a point cloud radius filter is used for filtering to delete outliers around the current projection point.

[0028] After the 3D point cloud coordinates of the projection point are converted into 2D coordinates on the calibration belt surface, the projection point located outside the depth map range is deleted.

[0029] A height filtering threshold is configured to filter the height of the projection point based on the configured height filtering threshold.

[0030] After the height of the projection point is filtered, the height of all remaining projection points is histogram counted, the number of each height in the histogram is accumulated from small to large according to the height value, and when the accumulated height number is greater than a preset total number proportion threshold of all height totals, the corresponding height is configured as the parcel height of the logistics parcel.

[0031] For the projection surface segmentation mask, the maximum area connected region of the projection screen segmentation mask and the contour points of the maximum area connected region are determined.

[0032] Based on the contour points of the maximum area connected region, the vertices of the minimum circumscribed rectangle of the maximum area connected region are determined, the vertices of the minimum circumscribed rectangle are converted into 3D point cloud coordinates, and the Euclidean distance between adjacent vertices is determined, wherein

[0033] Among the determined Euclidean distances, the smaller distance is configured as the width of the logistics parcel, and the larger distance is configured as the length of the logistics parcel.

[0034] A parcel static volume measurement system comprises a volume measurement controller, wherein

[0035] The depth map and the RGB image of any logistics package are measured by using the above method.

[0036] The advantages of the present application are as follows: the depth map and the RGB image of the logistics package are obtained based on the logistics package camera, the RGB image of the logistics package is segmented by using the constructed logistics package semantic segmentation model to extract the package detection basic segmentation mask of the logistics package; the depth data of the package detection basic segmentation mask is converted into 3D point cloud coordinates, and each point is projected to the calibration belt surface to obtain the projection surface segmentation mask after projection; the package height of the logistics package is determined based on the projection distance of the 3D point cloud coordinates to the calibration belt surface, and the package length and the package width of the logistics package are determined according to the minimum circumscribed rectangle of the projection surface segmentation mask; that is, the package size can be measured quickly and effectively, and the accuracy and reliability of the package size measurement are improved. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The flowchart of an embodiment of the present application.

[0038] Figure 2 The schematic diagram of an embodiment of the 3D point cloud projection of the present application. DETAILED DESCRIPTION

[0039] The present application will be further described below in combination with specific drawings and embodiments.

[0040] In order to quickly and effectively realize the measurement of the package volume and improve the accuracy and reliability of the package volume measurement, the volume measurement method of the package, in an embodiment of the present application, comprises the following steps:

[0041] A logistics package camera for acquiring a depth map and an RGB image is provided, wherein the logistics package camera is calibrated based on the acquired depth map to determine the calibration belt surface of the logistics package conveying after calibration;

[0042] The depth map and the RGB image of the logistics package are acquired based on the logistics package camera, and the RGB image of the logistics package is segmented by using the constructed logistics package semantic segmentation model to extract the package detection basic segmentation mask of the logistics package;

[0043] The depth data of the package detection basic segmentation mask is converted into 3D point cloud coordinates, and each point is projected to the calibration belt surface to obtain the projection surface segmentation mask after projection;

[0044] The projection surface segmentation mask is used to determine the package height of the logistics package based on the projection distance of the 3D point cloud coordinates to the calibration belt surface, and the minimum circumscribed rectangle of the projection surface segmentation mask is used to determine the package length and the package width of the logistics package.

[0045] The static volume of the logistics package is determined based on the determined package height, the package length and the package width.

[0046] Generally, the logistics package is conveyed by a belt. During the conveying of the logistics package by the belt, an image of the conveying of the logistics package by the belt is acquired by using a logistics package camera. The image acquired by the logistics package camera includes a depth map and an RGB image. The logistics package camera can adopt a form of an existing commonly used industrial camera, and specifically, the logistics package camera can acquire the depth map and the RGB image of the logistics package.

[0047] When the depth map and the RGB image are acquired by using the logistics package camera, the logistics package camera needs to be calibrated. After the calibration, the belt surface in the RGB image can be determined. During the calibration of the logistics package camera, an existing commonly used form can be adopted. An embodiment of the calibration is given below.

[0048] During the calibration, depth data (the depth data specifically refers to the depth value of each pixel in the depth map) in a depth map of a region of interest is acquired. The depth data is converted into 3D point cloud coordinates by using an internal parameter matrix of a depth camera of the logistics package camera. The internal parameter matrix of the depth camera of the logistics package camera is a characteristic parameter of the logistics package camera. The way and process of converting the depth data in the region of interest into 3D point cloud coordinates based on the internal parameter matrix of the depth camera can be consistent with existing ones, and specifically, the depth data can be converted into 3D point cloud coordinates.

[0049] The converted 3D point cloud coordinates are subjected to plane fitting by using a random consensus algorithm (RANSAC, Random Sample Consensus). During the plane fitting by using the random consensus algorithm, three points are first randomly selected from the 3D point cloud coordinates. A plane equation can be calculated by using the selected three points by using a commonly used technical means in the technical field. After the plane equation is calculated, the distances of all the remaining points to the plane are counted, and a distance fitting threshold is set to divide other 3D points into in-plane points and out-of-plane points.

[0050] The distance fitting threshold can be selected according to the point cloud distribution of the belt surface. If the point cloud fluctuation of the belt surface is small, the distance threshold is set to be small. For example, when the distance fitting threshold is set to 1500, the points with a distance less than 1500 to the plane are regarded as in-plane sample points, and otherwise, the points are regarded as out-of-plane points.

[0051] In a specific implementation, the calibration process of the above belt surface fitting can be completed using the computeModel function in the PCL library. The optimal plane parameters that meet the requirements are finally selected, and the calibrated belt surface is obtained at this time.

[0052] After calibrating the logistics package camera, when the logistics package is normally conveyed using the belt, the depth map and the RGB image of the region of interest can be obtained. The region of interest is generally the region used in the calibration described above. After obtaining the RGB image of the logistics package, the logistics package semantic segmentation model is used to perform semantic segmentation on the RGB image of the logistics package. When there is a logistics package in the RGB image, the package detection basic segmentation mask of the logistics package can be extracted.

[0053] When performing semantic segmentation on the logistics package, the existing common target detection method can be used. Generally, when performing semantic segmentation, the position of the logistics package and the region of the position of the logistics package can be obtained. An embodiment of semantic segmentation target detection based on the DeepLabv3+ network framework is given below. The logistics package semantic segmentation model based on the DeepLabv3+ network framework is as follows:

[0054] The constructed logistics package semantic segmentation model includes:

[0055] The logistics package semantic segmentation basic model is constructed, wherein the logistics package semantic segmentation basic model is based on the DeepLabv3+ network framework, and the network backbone part of the network framework uses the Xception model.

[0056] The constructed logistics package semantic segmentation basic model is trained, wherein the training of the logistics package semantic segmentation basic model includes a pre-training stage and a semantic segmentation training stage performed in sequence.

[0057] In the pre-training stage, the ImageNet-1k dataset is used to train the logistics package semantic segmentation basic model, so that the pre-training weight parameters of the logistics package semantic segmentation basic model are obtained after the pre-training stage ends.

[0058] In the semantic segmentation training stage, the logistics package camera is used to collect package RGB training images for making training samples, and the package RGB training images are labeled and the ROI region is selected to generate a semantic segmentation training dataset.

[0059] Based on the generated semantic segmentation training dataset, the logistics package semantic segmentation basic model after the pre-training stage is subjected to semantic segmentation training, so that the logistics package semantic segmentation model is generated by the logistics package semantic segmentation basic model after the target training state is reached.

[0060] In specific implementation, for the logistics package semantic segmentation model based on the network framework of DeepLabv3+ and using the Xception model for the network backbone part, the introduction of the hollow convolution can reduce the loss of information while increasing the receptive field. In order to obtain the final logistics package semantic segmentation model, the pre-training phase and the semantic segmentation training phase need to be performed on the logistics package semantic segmentation base model.

[0061] In the pre-training phase and the semantic segmentation training phase, the corresponding training conditions need to be configured. The training conditions configured in the pre-training phase and the semantic segmentation training phase are exemplified as follows. Specifically, the pre-training phase uses the ImageNet-1k dataset for training. In order to improve the convergence speed and prevent the occurrence of gradient explosion, the weight is initialized using the Kaiming method in the pre-training phase, the logistics package semantic segmentation base model is trained for 400,000 fixed iterations, and the weight with the highest average accuracy is selected as the pre-training weight parameter; the initial learning rate is set to 0.001, and the learning rate decay strategy is stepwise decay.

[0062] Specifically, the ImageNet-1k dataset is a commonly used public dataset. After configuring the training conditions of the above pre-training phase, the logistics package semantic segmentation base model can be pre-trained. The specific training method and process based on the configured training conditions can be consistent with the existing ones, and will not be repeated here.

[0063] In the semantic segmentation training phase, the weight of the logistics package semantic segmentation base model uses the pre-training weight parameter; the training initial learning rate is set to 0.05, and the learning rate decay strategy uses multi-step decay. The number of iterations for training is not less than 3000, the evaluation index uses the average intersection over union and the prediction accuracy for evaluation, the evaluation index is calculated every 50 generations, and the change curve of each evaluation index data is recorded. According to the results of the evaluation index, the training parameters are adjusted for multiple training, the weight with the best accuracy is selected as the final weight parameter of the logistics package semantic segmentation base model, and thus the logistics package semantic segmentation model can be obtained.

[0064] In specific implementation, the learning rate decay strategy using stepwise decay and the multi-step decay strategy are consistent with the existing ones. Generally, the configured training conditions are commonly used conditions in the technical field, and the training method and process according to the configured training conditions can be consistent with the existing ones. Specifically, the training method and process according to the configured training conditions can be consistent with the existing ones.

[0065] For the semantic segmentation training phase, the training data set includes a plurality of training samples, and the training samples in the training data set can be prepared by the following method: different sizes of packages are placed on the belt surface in turn, and the package depth map and the RGB image are collected by the logistics package camera installed above the belt, and the collected depth map and RGB image are placed in different folders respectively; then the collected package RGB image is labeled by the data set labeling tool Labelme, and the gray values of the package area and the background area are set to 255 and 0 respectively by coding, so as to obtain a package standard segmentation mask (i.e. the actual position of the logistics package in each RGB image); after the labeling processing, all the data sets are randomly divided into a training set and a verification set according to the required proportion.

[0066] The interested region of the package RGB image is manually selected, and the interested region is mainly the region where the package may appear on the belt surface. The pre-selected interested region can avoid the interference of other regions on the package processing; the findContours and minAreaRect functions in Opencv are used to find the maximum circumscribed rectangle of the interested region, and then the corresponding region of the RGB image is obtained according to the maximum circumscribed rectangle as the input image of the semantic segmentation model. Specifically, the interested region is consistent with the interested region selected during calibration, and the interested region refers to the region where the package appears on the belt surface. When the logistics package appears in the interested region during the conveying of the logistics package by the belt, the package in the interested region is the package region, and the other regions are regarded as the background region.

[0067] When working, the RGB image of the interested region is obtained by using the logistics package camera, and the logistics package semantic segmentation model is used for semantic segmentation of the RGB image to obtain the package detection basic segmentation mask of the logistics package. The package detection basic segmentation mask of the logistics package is the position of the logistics package on the belt surface and the region of the position on the belt.

[0068] In an embodiment of the present application, when the projection screen segmentation mask is obtained based on the package detection basic segmentation mask, it includes:

[0069] The depth data corresponding to the package detection basic segmentation mask is converted into the corresponding 3D point cloud coordinates, and each point cloud is projected to the calibration belt surface to determine the projection distance and the 3D point cloud coordinates of the projection point after projection;

[0070] The 3D point cloud coordinates of the projection point are converted into 2D coordinates on the calibration belt surface based on the depth camera internal parameter of the logistics package camera, and the projection surface segmentation mask is obtained based on the converted 2D coordinates on the calibration belt surface.

[0071] Specifically, when the point cloud is projected to the calibration belt surface to determine the projection distance, it includes:

[0072] configure a projection distance threshold, filter the projection distance with the configured projection distance threshold to delete the point cloud coordinates less than the configured projection distance threshold;

[0073] For all projection points, use a point cloud radius filter to filter to delete outliers around the current projection point.

[0074] In specific implementation, as shown in the formula (1), by calculating the distance from the 3D point cloud coordinates to the calibrated belt surface, and judging the size of the distance, the configured projection distance threshold is selected according to the height range of the measured package, for example, 1.2 times of the maximum height of the measured package is selected as the projection distance threshold. The point cloud coordinates less than the projection distance threshold are deleted, and the points with projection distance greater than the projection distance threshold are retained. Figure 2

[0075] Figure 2 In the formula (1), ABCD refers to the top surface of the logistics package, A'B'C'D' refers to the bottom surface of the logistics package. h refers to the height difference between the top surface of the package and the bottom surface of the logistics package, that is, the height of the logistics package. Figure 2 The belt plane in the formula (1) is the calibrated belt surface described above. Since the logistics package is placed on the belt surface, the bottom surface A'B'C'D' of the logistics package coincides with the calibrated belt surface. When measuring the static volume of the package, the height h needs to be determined, which can be determined according to the following description.

[0076] For 2D coordinates, which are generally pixel coordinates in a pixel coordinate system, the pixel coordinate system can be constructed in a conventional manner. Since the 2D coordinates and the 3D point cloud coordinates have a one-to-one correspondence, they can be directly converted based on the depth camera internal parameters of the logistics package camera. The conversion method and process can be consistent with the existing method.

[0077] For all projection points, use a point cloud radius filter to filter, set a radius threshold and a number of adjacent points threshold for each point. The radius threshold and the number of adjacent points threshold can be set according to the requirements for the edge points of the package. The radius threshold can be set as the maximum distance of adjacent points from each point as the center, and the points exceeding the distance will not be considered as adjacent points of the projection point. The configured number of adjacent points threshold is the minimum number of adjacent points for each point, and the center point less than the number will be considered as an outlier.

[0078] After setting the radius threshold and the number of adjacent points threshold, count the number of adjacent points of the projection point within the radius threshold range with each point as the center point. For the center point with the number of adjacent points less than the number of adjacent points threshold, it is judged as an outlier and is deleted.

[0079] ​After the initial deletion of outliers, the radius threshold and the number of adjacent points threshold are adjusted, and then the radius filtering is performed again.

[0080] In specific implementation, to reduce the sensitivity of the second radius filtering, the radius threshold and the number of adjacent points threshold of the second point cloud radius filter should be smaller than the radius threshold and the number of adjacent points threshold of the first point cloud radius filter.

[0081] Further, after converting the 3D point cloud coordinates of the projection points into 2D coordinates of the calibration belt surface, the projection points located outside the depth map range are deleted.

[0082] Specifically, the 2D coordinates of the projection surface are obtained by converting the 3D point cloud coordinates of the projection points, and whether the horizontal and vertical coordinate values of the 2D coordinates are within the dimension range of the depth map is determined, that is, whether the 2D coordinate values are negative or greater than the maximum value of the length and width of the depth map is determined, only the projection points within the dimension range of the depth map are retained, and the pixel position of the projection point in the depth map is obtained according to the inverse operation of the projection point calculation formula, so as to further obtain the segmentation mask of the wrapped depth map.

[0083] The 0.8 times of the minimum wrapping area is set as the wrapping area threshold, the connected domains in the depth map segmentation mask with an area less than the wrapping area threshold are filtered out, and then the corresponding coordinates of the connected domains meeting the area threshold in the projection surface are obtained, and the segmentation mask of the wrapped projection surface is obtained.

[0084] In specific implementation, for a belt conveying a logistics package, the type of the conveyed package generally remains stable, that is, the minimum wrapping area can be determined.

[0085] In an embodiment of the present application, a height filtering threshold is configured to filter the height of the projection point based on the configured height filtering threshold.

[0086] After filtering the height of the projection point, the height of all the remaining projection points is histogram counted, the number of each height in the histogram is accumulated from small to large according to the height value, and when the accumulated height number is greater than the preset total number ratio threshold of the total number of all heights, the corresponding height is configured as the wrapping height of the logistics package.

[0087] Specifically, the height filtering threshold is configured, including the minimum wrapping height threshold and the maximum wrapping height threshold, wherein the minimum wrapping height threshold can generally be 0.5 times the minimum height of the to-be-tested package, and the maximum wrapping height threshold can be 1.2 times the maximum height of the to-be-tested package.

[0088] Filter out the projection points whose height is less than the minimum wrapping height threshold and greater than the maximum wrapping height threshold in all positions of the projection surface segmentation mask.

[0089] For all projection points remaining after the above processing, histogram statistics of the height are performed, and the height when the cumulative height number is greater than 90% of the total number of all heights is taken as the wrapping height, that is, at this time, the preset total number ratio threshold is 90%, and the preset total number ratio threshold is 90%. Of course, the preset total number ratio threshold can also be other conditions, and the specific selection can be based on actual needs, so as to meet the value of the wrapping height.

[0090] As can be seen from the above description, the type of the logistics package is determined, so the height range of the logistics package is generally known, and therefore the minimum wrapping height threshold and the maximum wrapping height threshold can be determined.

[0091] In an embodiment of the present application, the projection surface segmentation mask is determined, the maximum area connected region of the projection screen segmentation mask is determined, and the contour points of the maximum area connected region are determined.

[0092] Based on the contour points of the maximum area connected region, the vertices of the minimum circumscribed rectangle of the maximum area connected region are determined, the vertices of the minimum circumscribed rectangle are converted into 3D point cloud coordinates, and the Euclidean distance between adjacent vertices is determined.

[0093] Among the determined Euclidean distances, the smaller distance is configured as the width of the logistics package, and the larger distance is configured as the length of the logistics package.

[0094] Specifically, for the projection surface segmentation mask, the maximum area connected region is determined, and the findContours function in Opencv is used to determine the contour points of the maximum connected region. Subsequently, the contour points are processed by downsampling and Gaussian filtering, which can further reduce the interference of protruding points around the contour and smooth the contour. Therefore, the maximum area connected region of the projection surface segmentation mask can be determined by using the commonly used technical means in the technical field.

[0095] In specific implementation, through the filtering of the Gaussian filtering and the point cloud radius filter, the wrapping edge damage processing can be realized, and the wrapping shape after processing is approximately regarded as a cuboid.

[0096] The projection segmentation mask contour is determined, each vertex is converted into 3D point cloud coordinates according to the depth camera intrinsic matrix, and the Euclidean distance between adjacent vertices is obtained by using the two norm, wherein the smaller distance is the width of the logistics package, and the larger distance is the length of the logistics package.

[0097] In specific implementation, the existing common technical means can be used to determine the vertexes of the minimum circumscribed rectangle of the mask profile of the projection surface. The specific way and process of obtaining the Euclidean distance between adjacent vertexes by using the two-norm can be consistent with the existing ones, and the required Euclidean distance can be determined.

[0098] In summary, the static size measurement of the logistics package can be realized, that is, the package height, the package length and the package width of the logistics package can be obtained, and the volume measurement of the package can be realized according to the size of the logistics package.

[0099] In specific implementation, most of the measured logistics packages are regular cuboid packages, and after the package height, the package length and the package width are determined, the volume of the logistics package can be determined according to the volume calculation method of the cuboid, that is, the size of the space occupied by the logistics package can be determined. In addition, the volume measurement of the irregular special-shaped object can not be considered.

[0100] In summary, the static volume measurement system of the package is obtained, and in an embodiment of the present application, the volume measurement controller comprises a volume measurement controller, wherein,

[0101] For the depth map and the RGB image of any logistics package, the above-mentioned method is used to measure the volume of the logistics package.

[0102] Specifically, the volume measurement controller can use the existing common computer device, that is, the calibration and the construction of the logistics package semantic segmentation model are performed by using the computer device, and the specific implementation method and process of the volume measurement of the logistics package can refer to the above description, which will not be described here.

[0103] As can be seen from the above description, according to the working process of the volume measurement controller, the volume measurement controller can be divided into the following parts:

[0104] The plane calibration module is used to demarcate the belt plane region of interest and calibrate the calibration belt surface.

[0105] The semantic segmentation module is used to construct a logistics package semantic segmentation model and perform semantic segmentation on the logistics package to be measured to obtain a package detection basic segmentation mask.

[0106] The 3D point cloud data processing module is used to convert the depth map corresponding to the package detection basic segmentation mask into 3D point cloud coordinates, project the point cloud coordinates to the belt surface to obtain the projection distance and the projection point coordinates, and filter the projection point coordinates by using a radius filter to obtain the projection point coordinates with outliers removed.

[0107] The parcel static volume measurement module is used to obtain the parcel height by the projection distance, to obtain the minimum circumscribed rectangle of the projection segmentation mask profile and to obtain the length and width of the parcel, and to calculate the static volume of the parcel according to the length, width and height by using the volume formula.

[0108] The above is only the preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with the preferred embodiment, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content without departing from the technical solution of the present application, and the equivalent embodiments with equivalent changes are equivalent. Any simple modification, equivalent change and modification of the above embodiments according to the technical essence of the present application are still within the scope of the technical solution of the present application.

Claims

1. A method of measuring the static volume of a parcel, characterized by, The volume measurement method comprises: providing a logistics package camera for acquiring a depth map and an RGB image, wherein the logistics package camera is calibrated based on the acquired depth map to determine a calibration belt surface of the logistics package conveyor after calibration; acquiring a depth map and an RGB image of a logistics package based on the logistics package camera, performing semantic segmentation on the RGB image of the logistics package by using a constructed logistics package semantic segmentation model to extract a package detection basic segmentation mask of the logistics package; converting depth data of the package detection basic segmentation mask into 3D point cloud coordinates, and projecting each point to the calibration belt surface to obtain a projection surface segmentation mask after projection; determining the package height of the logistics package based on the projection distance of the 3D point cloud coordinates to the calibration belt surface for the projection surface segmentation mask, and determining the package length and the package width of the logistics package according to the minimum circumscribed rectangle of the projection surface segmentation mask; determining the static volume of the logistics package based on the determined package height, package length and package width; the constructed logistics package semantic segmentation model comprises: constructing a logistics package semantic segmentation base model, wherein the logistics package semantic segmentation base model is based on a DeepLabv3+ network framework, and the network backbone part of the network framework adopts an Xception model; training the constructed logistics package semantic segmentation base model, wherein the training of the logistics package semantic segmentation base model comprises a pre-training stage and a semantic segmentation training stage in sequence; in the pre-training stage, the logistics package semantic segmentation base model is trained by using an ImageNet-1k data set, so as to obtain pre-training weight parameters of the logistics package semantic segmentation base model after the pre-training stage ends; in the semantic segmentation training stage, a package RGB training image used for making a training sample is collected by using the logistics package camera, the package RGB training image is labeled and the ROI region is selected to generate a semantic segmentation training data set; based on the generated semantic segmentation training data set, the logistics package semantic segmentation base model after the pre-training stage is subjected to semantic segmentation training, so as to generate a logistics package semantic segmentation model from the logistics package semantic segmentation base model after a target training state is reached; when the projection surface segmentation mask is obtained based on the package detection basic segmentation mask, it comprises: converting the depth data corresponding to the package detection basic segmentation mask into corresponding 3D point cloud coordinates, and projecting each point cloud to the calibration belt surface to determine the projection distance and the 3D point cloud coordinates of the projection point after projection; converting the 3D point cloud coordinates of the projection point into 2D coordinates on the calibration belt surface based on the depth camera intrinsic parameter of the logistics package camera, and obtaining the projection surface segmentation mask based on the converted 2D coordinates on the calibration belt surface.

2. The wrapped static volumetric measurement method of claim 1, wherein, in in the pre-training stage, the weight is initialized by using the Kaiming method, and the pre-training is performed by using a fixed number of iterations; the learning rate decay strategy is a segmented decay, and the weight with the highest average accuracy is selected as the pre-training weight parameter.

3. The wrapped static volumetric measurement method of claim 1, wherein, in In the semantic segmentation training phase, the learning rate decay strategy adopts multi-step decay, and the number of iterations of network training is not less than 3000 times; The evaluation indexes are evaluated by using the average intersection over union and the prediction accuracy, and the evaluation index data change curve is recorded by calculating the evaluation index every 50 generations.

4. The wrapped static volume measurement method of claim 1, wherein, When projecting the point cloud to the calibration belt surface to determine the projection distance, the following steps are included: A projection distance threshold is configured, and the projection distance is filtered using the configured projection distance threshold to delete the projection distance less than the configured projection distance threshold. For all projection points, a point cloud radius filter is used for filtering to delete outliers around the current projection point.

5. The method of claim 1, wherein, After converting the 3D point cloud coordinates of the projection points to 2D coordinates of the calibration belt surface, the projection points located outside the depth map range are deleted.

6. The method of claim 1 to 3, wherein, A height filtering threshold is configured to filter the height of the projection points based on the configured height filtering threshold. After filtering the height of the projection points, the height of all remaining projection points is histogram counted, and the number of each height in the histogram is accumulated from small to large according to the height value. When the accumulated height number is greater than the preset total number ratio threshold of the total number of all heights, the corresponding height is configured as the package height of the logistics package.

7. The method of claim 1 to 3, wherein, The projection surface segmentation mask is divided, the maximum area connected region of the projection surface segmentation mask is determined, and the contour points of the maximum area connected region are determined. Based on the contour points of the maximum area connected region, the vertices of the minimum bounding rectangle of the maximum area connected region are determined, the vertices of the minimum bounding rectangle are converted to 3D point cloud coordinates, and the Euclidean distance between adjacent vertices is determined. Among the determined Euclidean distances, the small distance is configured as the width of the logistics package, and the large distance is configured as the length of the logistics package.

8. A system for static volume measurement of a package, characterized by The volume measurement controller includes a volume measurement controller, wherein For any logistics package depth map and RGB image, the method of any one of the above claims 1~7 is used to measure the volume of the logistics package.

Images