Three-dimensional volume measurement method, device, storage medium and electronic device

Abstract

This application discloses a method for volume measurement using a three-dimensional point cloud, comprising: obtaining current three-dimensional point cloud data of a measured object; determining the maximum length of the current three-dimensional point cloud data in a preset first dimension; when the difference between the maximum length in the first dimension and a second length is greater than an error threshold, using the current three-dimensional point cloud data to perform point cloud modeling to obtain a current modeling result; fusing the current modeling result with the point cloud model of the target object to obtain an updated point cloud model of the target object; and when volume measurement is required, determining the volume of the target object based on the latest point cloud model of the target object. Application of this application can improve the volume measurement accuracy of various objects, including irregular objects.

Description

Technical Field

[0001] The present application relates to three-dimensional point cloud technology, and in particular to a method, device, storage medium and electronic device for three-dimensional volume measurement using three-dimensional point cloud. Background Art

[0002] With the advancement of 3D image shooting and processing technology, the application of 3D point clouds is becoming more and more extensive.

[0003] Automatic volume measurement of large objects is necessary in many scenarios. Recently, there has been increasing research on technologies related to automatic volume measurement. Generally speaking, there are two main approaches to volume measurement: measuring the length, width, and height of a box, and directly integrating 3D point clouds.

[0004] Among them, the first type of method is only applicable to rectangular boxes and is usually used to measure the volume of a single box. It is mainly used for handheld box volume measurement and box volume measurement on conveyor belts. It cannot measure the volume of irregular objects (such as stacked goods).

[0005] Although the second type of method can be used to measure the volume of irregular objects (such as stacked goods), the current methods of using three-dimensional point clouds for volume measurement all require that the three-dimensional point clouds meet the condition that there is no self-occlusion or other occlusion. Due to this limitation, the current processing of volume measurement using three-dimensional point clouds can usually only be used for the volume measurement of boxes with only one height plane, because multiple height planes will cause point cloud loss due to self-occlusion, which will inevitably lead to a significant decrease in volume measurement accuracy. Summary of the Invention

[0006] The present application provides a three-dimensional volume measurement method, device, storage medium and electronic device, which can improve the volume measurement accuracy of various objects including irregular objects.

[0007] To achieve the above objectives, this application adopts the following technical solutions:

[0008] A method for volume measurement using a three-dimensional point cloud, comprising:

[0009] Obtain the current three-dimensional point cloud data of the measured object;

[0010] Determining a maximum length of the current three-dimensional point cloud data in a preset first dimension;

[0011] When the difference between the maximum length in the first dimension and the second length is greater than an error threshold, performing point cloud modeling using the current three-dimensional point cloud data to obtain a current modeling result; wherein the second length is the length in the first dimension of the point cloud model of the target object determined based on the historical three-dimensional point cloud data of the target object;

[0012] Fusing the current modeling result with the target object point cloud model to obtain an updated target object point cloud model;

[0013] When volume measurement is required, the volume of the target object is determined based on the latest point cloud model of the target object.

[0014] Preferably, in an initial state, the target object is modeled according to current three-dimensional point cloud data of the target object to obtain an initial point cloud model of the target object.

[0015] Preferably, after obtaining the current three-dimensional point cloud data of the measured object, the method further comprises:

[0016] Performing plane clustering on the current three-dimensional point cloud data according to the coordinate values ​​of the first dimension to obtain planes corresponding to different coordinate values ​​of the first dimension;

[0017] When the coordinate fluctuation in the first dimension of all points clustered to the same plane is lower than a fluctuation threshold, when it is determined based on the current three-dimensional point cloud data that the target object is self-occluded, the point cloud data of the occluded portion is completed using the coordinate data in the first dimension of the occluded plane in the current three-dimensional point cloud data, and the target object is modeled based on the completed three-dimensional point cloud data to obtain the updated target object point cloud model;

[0018] When the coordinate fluctuation of the point clouds clustered on the same plane in the first dimension is not less than the fluctuation threshold, the step of determining the length of the current three-dimensional point cloud data in the first dimension continues.

[0019] Preferably, the method of determining whether the target object is blocked includes:

[0020] Fitting each of the planes into a rectangle, and determining the area ratio covered by the point cloud within the rectangle; wherein the point cloud within the rectangle includes all points corresponding to the current three-dimensional point cloud data whose coordinates in the first dimension are greater than or equal to the coordinates of the corresponding plane in the first dimension;

[0021] When the proportion is not higher than the proportion threshold, it is determined that the corresponding plane and the target object are self-occluded; when the proportion is higher than the proportion threshold, it is determined that the corresponding plane is not self-occluded.

[0022] Preferably, the completing of the obscured point cloud data includes:

[0023] In the occluded plane, after fitting the plane into a rectangle, point cloud data is inserted into the portion of the rectangle not covered by the point cloud, and the coordinates of the inserted point cloud data in the first dimension are the same as the depth coordinates of the point cloud in the occluded plane.

[0024] Preferably, the method further comprises:

[0025] The timing starts each time an updated point cloud model of the target object is obtained. After the set timing time is up, the current three-dimensional point cloud data is used to perform point cloud modeling to obtain the current modeling result; the current modeling result is fused with the target object point cloud model to obtain an updated point cloud model of the target object.

[0026] Preferably, the operation of obtaining the current three-dimensional point cloud data of the measured object is performed once every set time.

[0027] Preferably, every time the operation of acquiring the current three-dimensional point cloud data is performed N times, the operation of performing planar clustering on the current three-dimensional point cloud data in the first dimension is performed once; wherein N is a preset natural number.

[0028] A device for volume measurement using a three-dimensional point cloud, comprising: an acquisition unit, a length determination unit, a modeling unit, a model updating unit, and a volume measurement unit;

[0029] The acquisition unit is used to acquire the current three-dimensional point cloud data of the measured object;

[0030] The length determination unit is configured to determine the maximum length of the current three-dimensional point cloud data in a preset first dimension;

[0031] The modeling unit is configured to perform point cloud modeling using the current three-dimensional point cloud data to obtain a current modeling result when a difference between a maximum length in the first dimension and a second length is greater than an error threshold; wherein the second length is a length in the first dimension of the point cloud model of the target object determined based on historical three-dimensional point cloud data of the target object;

[0032] The model updating unit is configured to fuse the current modeling result with the target object point cloud model to obtain an updated target object point cloud model;

[0033] The volume measurement unit is configured to determine the volume of the target object based on the latest point cloud model of the target object when volume measurement is required.

[0034] Preferably, the modeling unit is used to model the target object according to the current three-dimensional point cloud data of the target object in an initial state to obtain an initial point cloud model of the target object.

[0035] Preferably, the device further comprises: a plane clustering unit and a point cloud data completion unit;

[0036] The plane clustering unit is configured to perform plane clustering on the current three-dimensional point cloud data according to the coordinate values ​​of the first dimension to obtain planes corresponding to different coordinate values ​​of the first dimension;

[0037] The point cloud data completion unit is configured to, when it is determined based on the current three-dimensional point cloud data that the target object is self-occluded, use the coordinate data of the occluded plane in the current three-dimensional point cloud data in the first dimension to complete the point cloud data of the occluded portion, and model the target object based on the completed three-dimensional point cloud data to obtain the updated target object point cloud model;

[0038] The length determination unit is configured to, when the coordinate fluctuation of the point clouds clustered on the same plane in the first dimension is not less than the fluctuation threshold, perform the operation of determining the length of the current three-dimensional point cloud data in the first dimension.

[0039] Preferably, in the point cloud data completion unit, a method of determining whether the target object is blocked includes:

[0040] Fitting each of the planes into a rectangle, and determining the area ratio covered by the point cloud within the rectangle; wherein the point cloud within the rectangle includes all points corresponding to the current three-dimensional point cloud data whose coordinates in the first dimension are greater than or equal to the coordinates of the corresponding plane in the first dimension;

[0041] When the proportion is not higher than the proportion threshold, it is determined that the corresponding plane and the target object are self-occluded; when the proportion is higher than the proportion threshold, it is determined that the corresponding plane is not self-occluded.

[0042] Preferably, in the point cloud data completion unit, completing the obscured point cloud data includes:

[0043] In the occluded plane, after fitting the plane into a rectangle, point cloud data is inserted into the portion of the rectangle not covered by the point cloud, and the coordinates of the inserted point cloud data in the first dimension are the same as the depth coordinates of the point cloud in the occluded plane.

[0044] Preferably, the device further comprises a timer;

[0045] After each time the model updating unit obtains the updated target object point cloud model, or after each time the point cloud data completing unit obtains the updated target object point cloud model, starting a timer;

[0046] After the timing time set by the timer expires, the modeling unit is further used to perform point cloud modeling using the current three-dimensional point cloud data to obtain a current modeling result.

[0047] A computer-readable storage medium stores computer instructions, which, when executed by a processor, can implement any of the three-dimensional volume measurement methods described above.

[0048] An electronic device comprising at least a computer-readable storage medium and a processor;

[0049] The processor is configured to read the executable instructions from the computer-readable storage medium and execute the instructions to implement any one of the above three-dimensional volume measurement methods.

[0050] As can be seen from the above technical solution, in this application, the current three-dimensional point cloud data of the measured target is obtained; the maximum length of the current three-dimensional point cloud data in the first dimension is determined; when the difference between the maximum length in the first dimension and the length of the target point cloud model in the first dimension determined based on the historical three-dimensional point cloud data of the target is greater than the error threshold, it is determined that the length direction of the target in the first dimension has changed and it is necessary to update the target point cloud model, then the current three-dimensional point cloud data is used to perform point cloud modeling, and the modeling result is fused with the original target point cloud model to obtain an updated target point cloud model, so that the target point cloud model is updated to include the latest current three-dimensional point cloud data; when volume measurement is required, the volume of the target is determined based on the latest target point cloud model. Through the above method, when there is no occlusion, the target point cloud model is constructed in time using the historical three-dimensional point cloud data. When the target changes and occlusion may occur, the current three-dimensional point cloud data obtained in real time is fused with the historically constructed target point cloud model, on the one hand, reflecting the changed part of the target, and on the other hand, the target point cloud model constructed when it is not occluded is used to compensate for the currently occluded part. As a result, the point cloud of the occluded part is always included in the updating process of the target object point cloud model, which greatly improves the volume measurement accuracy of the target object and is particularly suitable for volume measurement of irregular objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1a 、 1b 1c and 1d are the RGB image, depth image and point cloud diagram of the target object obtained at a certain moment respectively;

[0052] Figure 2Schematic diagram of the basic process of the volume measurement method using three-dimensional point cloud in this application;

[0053] Figure 3 This is a schematic diagram of a specific process of a volume measurement method using a three-dimensional point cloud in a specific embodiment of the present application;

[0054] Figure 4a 、 4b 4c are the bird's-eye view point cloud diagrams of the cargo at time T0, time T1, and time T2 in Example 1 respectively;

[0055] Figure 4d This is a schematic diagram of the point cloud after the point cloud model is updated based on the point cloud data of the cargo at time T2;

[0056] Figure 5a and Figure 5b They are respectively a bird's-eye view of the cargo point cloud in Example 2 and a bird's-eye view diagram of the cargo point cloud after completion;

[0057] Figure 6 Schematic diagram of the basic structure of the volume measurement device using three-dimensional point cloud in this application;

[0058] Figure 7 This is a schematic diagram of the basic structure of an electronic device in this application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical means and advantages of this application more clear, this application is further described in detail below with reference to the accompanying drawings.

[0060] In the existing method of using 3D point cloud for volume measurement, the 3D point cloud is required to meet the condition of no self-occlusion or other occlusion. This is because: taking the case of a target object consisting of multiple stacked goods with sub-occlusion as an example, the goods are stacked from low to high. When the stack is high, there is self-occlusion of the lower goods, such as Figure 1a This is a schematic diagram of the target point cloud obtained at a certain moment. Figure 1b and Figure 1c are the corresponding RGB images and depth images respectively, Figure 1a The part surrounded by the gray box is the self-occluded part. This part of the point cloud is missing. Figure 1a If volume measurement is performed using the point cloud data shown, the measured volume will be underestimated due to missing points in the self-occluded area. Of course, in addition to self-occlusion, external occlusions other than cargo objects can also cause missing points and underestimate the measured volume. Therefore, current methods for volume measurement using 3D point clouds require that the 3D point cloud be free of self-occlusion or other occlusions.

[0061] It can be seen that the volume measurement of regularly placed targets or rectangular boxes can currently achieve high-precision measurement levels. However, actual logistics transportation lines often have irregularly shaped goods or goods placed randomly, and there are also interference problems such as self-occlusion of goods and obstruction by other targets during cargo transportation. Therefore, the volume measurement technology for self-occlusion of goods or obstruction by other non-cargo targets still has the problem of low measurement accuracy.

[0062] The basic idea of ​​this application is to combine the three-dimensional point cloud data of the target object obtained at different times to construct a point cloud model of the target object including the point cloud of the obscured part.

[0063] Figure 2 This is a basic flow chart of the volume measurement method using 3D point cloud in this application. Figure 2 As shown, the method includes:

[0064] Step 201: Acquire current three-dimensional point cloud data of the target object.

[0065] The current three-dimensional point cloud data may be the latest three-dimensional point cloud data of the target object captured by a shooting device including a depth sensor (e.g., a binocular camera based on stereo vision, a camera based on structured light technology or TOF technology, etc.). For example, it may be the three-dimensional point cloud data corresponding to the latest image frame data. In practical applications, the acquisition operation of this step may be performed multiple times, and after each acquisition of the current three-dimensional point cloud data, the target object point cloud model may be modeled through subsequent steps 202-204. The specific multiple acquisition operations may be performed at set time intervals, or may be triggered by an event (e.g., performed after receiving a volume measurement instruction), and this application does not limit this.

[0066] Step 202: Determine the maximum length of the current three-dimensional point cloud data in the first dimension.

[0067] Among them, the first dimension can be a certain direction in which the size of the preset target object changes. For example, in a scenario where multiple pieces of goods are stacked to form a target object, when the camera shoots downward from the air (such as when the goods are placed in sequence from bottom to top in a warehouse, and the camera is installed on the roof of the warehouse to shoot the estimated volume of the goods), the first dimension can be the height direction of the target object. When the camera shoots parallel from the side (such as when the goods are placed in sequence from the inside to the outside in a truck compartment, and the camera is installed on the door to shoot the estimated volume of the goods), the first dimension can be the side direction.

[0068] The maximum length in the first dimension may be the maximum height difference in the current three-dimensional point cloud data or the maximum side length difference in the current three-dimensional point cloud data.

[0069] In step 203 , when the difference between the maximum length in the first dimension and the second length is greater than the error threshold, point cloud modeling of the target object is performed using the current three-dimensional point cloud data to obtain a current modeling result.

[0070] The second length refers to the length of the target object point cloud model in the first dimension determined based on the historical three-dimensional point cloud data of the target object.

[0071] Target object three-dimensional point cloud modeling refers to the process of describing the appearance contour of the target object using discrete points in three-dimensional space.

[0072] Before step 201, the three-dimensional point cloud data of the target object may have been obtained, and the target object has been modeled using these historical three-dimensional point cloud data to obtain a point cloud model of the target object. The size of the target object during modeling is characterized by the length of the target object point cloud model in the first dimension. In this step, the maximum length A determined in step 202 is compared with the length B of the target object point cloud model modeled based on the historical data in the first dimension. If the difference between A and B is less than the error threshold, it means that the size of the target object has changed in the first dimension. For example, in the scenario of stacking goods, it may be that one layer of goods has been added or reduced. In this case, the point cloud model of the target object needs to be updated. The specific update processing includes the processing of this step and the processing of step 204. Among them, the processing in this step is used to prepare for the update, use the current three-dimensional point cloud data to perform point cloud modeling, and obtain the current modeling result.

[0073] In addition, the above error threshold is set to prevent the difference between A and B caused by processing errors when the target size does not change. The error threshold can be set as needed.

[0074] Step 204 : Fusing the current modeling result with the target object point cloud model to obtain an updated target object point cloud model.

[0075] Since the three-dimensional point cloud data used for point cloud modeling in step 203 is the latest one, and is obtained when the target object changes in size in the first dimensional direction, there may be occlusion problems in the current modeling result C (for example, self-occlusion caused when the target object increases in size in the first dimensional direction). However, the target point cloud model D obtained in the previous modeling was obtained when the target object did not change in size in the first dimensional direction, so there was no self-occlusion in the top layer of the target object during modeling. The top layer here refers to the highest layer of the target object in the first dimensional direction. Based on this, the current modeling result is fused with the target point cloud model in this application to obtain an updated target point cloud model. In this way, the updated target point cloud model includes both the point cloud of the target object with size changes included in the current modeling result C, and the point cloud of the possibly occluded part is retained through the target point cloud model D, that is, the updated target point cloud model can reflect the overall outline of the current target object, including the occluded part.

[0076] The target object point cloud model is updated once through the processing of the aforementioned steps 201-204. This process can be performed repeatedly. A specific example is given below. During the cargo handling process, there will be self-occlusion of the cargo, interference from people and other moving targets, which manifests as non-cargo point clouds and occluded point clouds in the single-frame point cloud cargo data. However, since the positions of non-cargo point clouds and occluded point clouds are different in each frame, the information of multiple frames can be used to model and update the three-dimensional point cloud model of the cargo using the valid cargo point clouds of different frames at the beginning of cargo placement.

[0077] For example, when goods are stacked from low to high, when the goods are stacked higher, there is self-occlusion of the goods on the lower layer. If only the volume of the goods point cloud of the current frame is measured, the measured volume will be smaller due to the missing point cloud of the self-occluded part. If the point cloud during the stacking process is repeatedly modeled and updated in real time through steps 201-204, specifically, if the height of the current frame point cloud position is different from the height of the modeled goods, it will be updated. In this way, even if the self-occluded point cloud information of the current frame is missing, this part of the point cloud information is complete in the latest goods point cloud model. By measuring the volume of the three-dimensional point cloud model of the goods, the volume of the goods can be accurately obtained.

[0078] Step 205 : When volume measurement is required, the volume of the target object is determined based on the latest point cloud model of the target object.

[0079] After each execution of step 204, the latest target point cloud model can be obtained, reflecting the overall outline of the current target object. When volume measurement is required, this step is executed to determine the volume of the target object based on the latest target point cloud model. The volume calculated in this way includes the occluded part, which greatly improves the accuracy of volume measurement.

[0080] So far, Figure 2 This concludes the basic flow of the volume measurement method. Through the above processing, the target object's 3D point cloud data (i.e., sequential point cloud data) acquired at different times is used to model and update the target object. This ensures that the updated point cloud model of the target object always includes the occluded point cloud, improving the accuracy of volume measurement.

[0081] The above method of measuring volume by acquiring point cloud data at multiple different times can be applied to both regular and irregular objects. Furthermore, for regularly arranged rectangular objects, in addition to updating the target object point cloud model using the above method, another method can be used to determine the latest target object point cloud model. Specifically, the following processing can be performed after step 201:

[0082] Step 206 : performing plane clustering on the current three-dimensional point cloud data according to the coordinate values ​​of the first dimension to obtain planes corresponding to different coordinate values ​​of the first dimension.

[0083] Plane clustering is used to distinguish the different planes to which the current three-dimensional point cloud data belongs. Plane clustering is performed according to the coordinate values ​​of the first dimension, and each clustered plane corresponds to a coordinate value of the first dimension. The coordinate fluctuations of the point clouds belonging to the same plane are used to determine whether another method can be used to complete the obscured part of the point cloud. If the coordinate fluctuations of the point clouds clustered on the same plane in the first dimension are lower than the fluctuation threshold, step 207 is executed to use another method to complete the point cloud. Otherwise, step 208 is executed to return to step 202, and the aforementioned method of acquiring point cloud data at multiple different times to model and update the target object is used to perform volume measurement.

[0084] Step 207: When the coordinate fluctuation in the first dimension of all points clustered on the same plane is lower than the fluctuation threshold, and it is determined based on the current three-dimensional point cloud data that the target object is self-occluded, the point cloud data of the occluded part is completed using the coordinate data in the first dimension of the point cloud data of the occluded plane in the current three-dimensional point cloud data, and the target object is modeled based on the completed three-dimensional point cloud data to obtain an updated point cloud model of the target object.

[0085] When the coordinate fluctuation of the point cloud clustered on the same plane in the first dimension is lower than the fluctuation threshold, it is determined whether the target object is self-occluded based on the current three-dimensional point cloud data. If so, the point cloud is completed, and the target object is modeled using the completed three-dimensional point cloud data. The modeling result is used as the updated target object point cloud model.

[0086] After updating the point cloud model of the target object in this step, the process may return to step 202 to wait for the next acquisition of the current three-dimensional point cloud data of the target object, or may execute step 205 to perform volume calculation.

[0087] That is, after the target point cloud model is updated in step 207, the next target point cloud model update can be continued through steps 202-204 or steps 206-207. When volume measurement is required, step 205 is executed to calculate the volume using the latest target point cloud model.

[0088] Step 208 : If the coordinate fluctuation of the point clouds clustered on the same plane in the first dimension is not less than the fluctuation threshold, continue to execute step 202 .

[0089] It can be seen from the above-mentioned newly added processing that in this application, when performing different rounds of target object point cloud model updates, steps 202-204 or steps 206-207 can be used to update the target object point cloud model. The two different point cloud model update methods can also be used alternately, and this application does not limit this.

[0090] The following describes the specific implementation of this application through a specific embodiment. In the following specific embodiment, the volume of the goods stored in a warehouse is measured, and the goods are stacked layer by layer from bottom to top. A camera equipped with a depth sensor is used to capture the target object, generating three-dimensional point cloud data of the target object.

[0091] Figure 3 This is a specific flow chart of the volume measurement method using three-dimensional point cloud in a specific embodiment of this application. Figure 3 As shown, the method includes:

[0092] Step 301 : in an initial state, initial three-dimensional point cloud data of a target object is obtained, and the target object is modeled using the initial three-dimensional point cloud data to obtain an initial point cloud model of the target object.

[0093] After acquiring the 3D point cloud data of the measured object for the first time, the object is modeled to obtain an initial point cloud model of the object. At this point, the initial point cloud model of the object is the latest point cloud model of the object. The next time the point cloud model of the object is updated using the first point cloud model update method, the latest point cloud model of the object is used as the input model.

[0094] Step 302: Acquire current three-dimensional point cloud data of the target object.

[0095] This step can be repeated at set time intervals, or it can be performed under event triggering. The three-dimensional point cloud data obtained here is the point cloud data belonging to the target object. When the cargo is blocked by people or other moving objects, this application can exclude these external blocking factors (excluding self-blocking factors) by default and only obtain the point cloud data belonging to the target object. For example, when a person blocks a part of the target object, the point cloud data of the person is removed from the overall point cloud data obtained by the camera, and the point cloud data of the target object is retained. The specific processing of excluding these external blocking factors can adopt various existing methods, and this application does not limit this.

[0096] Step 303: Determine the maximum height of the current three-dimensional point cloud data.

[0097] In this embodiment, the first dimension refers to the height direction. Therefore, the current height of the target object is determined based on the current three-dimensional point cloud data of the target object, that is, the maximum difference in height coordinates in the three-dimensional point cloud data, that is, the maximum height difference.

[0098] Step 304 , determining whether the difference between the maximum height of the current 3D point cloud data and the height of the latest target object point cloud model is greater than an error threshold; if so, executing step 305 , otherwise returning to step 302 .

[0099] The current height E of the target object is determined by the maximum height in step 303. The height of the latest target object point cloud model is the height F of the target object point cloud model after the previous update. When the height difference between the two exceeds the error threshold, it is considered that the height of the target object has changed, and there is new or reduced cargo. Step 305 needs to be executed to update the target object point cloud model. When the height difference between E and F does not exceed the error threshold, it is considered that the height of the target object has not changed, and the process returns to step 302 to wait for the next acquisition of the target object's three-dimensional point cloud data.

[0100] Step 305: Model the target object using the current three-dimensional point cloud data to obtain the current modeling result.

[0101] Step 306 : Fusing the current modeling result with the latest target object point cloud model to obtain an updated target object point cloud model.

[0102] As mentioned above, the current modeling result includes the point cloud of the changed part after the height of the target object changes. At the same time, the latest target object point cloud model includes the point cloud of the part that may be blocked. The current modeling result is fused with the latest target object point cloud model to obtain an updated target object point cloud model, so that the updated target object point cloud model includes the complete point cloud of the current target object, reflecting the overall outline of the current target object. At the same time, the updated target object point cloud model serves as the latest target object point cloud model. The specific fusion processing can be carried out in accordance with the existing method, and different processing can be adopted according to whether the height of the target object increases or decreases, which will not be repeated here. After completing the update of the target object point cloud model, you can return to step 302 to wait for the next acquisition of the three-dimensional point cloud data of the target object, or you can execute step 307 to calculate the volume of the target object.

[0103] Step 307 : When volume measurement is required, the volume of the target object is determined based on the latest point cloud model of the target object.

[0104] Through the aforementioned steps 303 - 306 , the target object point cloud model is updated once using the time-series point cloud data (hereinafter referred to as the first point cloud model updating method). This process can be performed repeatedly.

[0105] In addition, the above-mentioned updating of the target object point cloud model is explained using self-occlusion as an example. In fact, the same is also applicable to other external occlusions. Specifically, since the three-dimensional point cloud data of the target object obtained through the processing of step 302 directly excludes external occlusions, if there is a moving external occlusion, then the currently occluded part of the target object may be unoccluded at other times. Therefore, the point cloud of the currently occluded target object may be included in the three-dimensional point cloud data of the target object obtained at other times, and the point cloud of the occluded part can be compensated by fusion processing of the point cloud models obtained at different times.

[0106] After each execution of step 306, the latest target object point cloud model can be obtained, reflecting the overall outline of the current target object. When volume measurement is required, this step is executed. Based on the latest target object point cloud model, the volume measurement of the target object can be achieved according to the triangulated mesh or grid method to determine the volume of the target object. The volume calculated in this way includes the self-occluded part, which greatly improves the accuracy of the volume measurement.

[0107] As mentioned above, in addition to the first point cloud model updating method, a second point cloud model updating method can also be used for regular objects. Specifically, the following steps can be continued after step 302:

[0108] Step 308 : Cluster the current three-dimensional point cloud data into different planes according to their heights, and obtain planes corresponding to different height values.

[0109] First, the current 3D point cloud data is used to determine whether the cargo is a regular object. The point cloud of a regular object exhibits stratification by height plane, with point clouds within the same height plane having essentially consistent heights and minimal fluctuations. This characteristic does not exist in the point cloud of an irregular object. Based on this, in this step, the current 3D point cloud data is clustered into planes based on height, and then, in step 309, the point cloud heights of each plane are determined to be consistent.

[0110] In addition, this step may be performed once after executing step 302 N times, where N is a preset natural number, for example, N=2.

[0111] Step 309 , determining whether the height fluctuation in the point cloud data within the same plane is less than a fluctuation threshold; if so, executing step 310 ; otherwise, executing step 303 .

[0112] If the height fluctuation in the point cloud data within the same plane is less than the fluctuation threshold, it is considered to be a regular target object, such as a target object composed of regularly placed rectangular blocks. In this case, the second method is used to update the target object point cloud model; if the height fluctuation in the point cloud data within the same plane is not less than the fluctuation threshold, it is considered not to be a regular target object, such as messily placed goods or irregularly shaped goods. In this case, return to step 303 and use the first method to update the target object point cloud model.

[0113] Step 310 , based on the current 3D point cloud data, determine whether the target object is self-occluded. If so, execute step 311 ; otherwise, return to step 302 .

[0114] For regular objects, we can use the prior information that the top plane of the cargo is a rectangle to determine whether the cargo is obstructed. Specific processing may include:

[0115] 1) Fitting each plane obtained by the clustering operation in step 308 into a rectangle, and determining the area ratio covered by the point cloud within the rectangle;

[0116] For regular objects, it is assumed that they are composed of regular cuboids, so the top plane of the cargo is assumed to be a rectangle. Based on this, each plane obtained by clustering is fitted into a rectangle. Among them, the point cloud within the rectangle includes all points corresponding to the current 3D point cloud data whose height coordinates are greater than or equal to the height coordinates corresponding to the corresponding plane. For example, Figure 1a After the midpoint cloud is clustered into different planes, the second-highest plane is fitted into a rectangle, and the point cloud within the rectangle includes all points of the second-highest plane and all points of the top plane.

[0117] 2) When the area ratio is not higher than the ratio threshold, it is determined that the corresponding plane and the target object are self-occluded; when the area ratio is higher than the ratio threshold, it is determined that the corresponding plane is not self-occluded.

[0118] After the clustered plane is fitted into a rectangle, this rectangle represents the top surface of the cargo on that layer. Portions of this top surface not covered by the point cloud are considered to be self-occluded by the cargo on the upper layer. A percentage threshold is introduced to account for computational and measurement errors. Based on this, if the area percentage exceeds the threshold, the entire top surface of the cargo on that layer is considered covered by the point cloud, indicating that the corresponding plane is not self-occluded. If the area percentage is not higher than the threshold, it is considered that some portions of the top surface of the cargo on that layer are not covered by the point cloud, indicating that the corresponding plane is self-occluded.

[0119] Step 311 : Use the height data of the blocked plane in the current three-dimensional point cloud data to complete the point cloud data of the blocked portion.

[0120] For the plane X identified in step 310 as partially obscured, the area within the fitted rectangle not covered by the point cloud is interpolated based on the values ​​of the adjacent point clouds to complete the point cloud of the regular plane. During point cloud completion, the heights of the supplementary points in the area not covered by the point cloud are set to the same height as that of the corresponding plane X. This completes the missing portion of plane X.

[0121] The processing of the above steps 309-311 can be performed on the entire target object; or, it can be performed on a target plane after clustering. In this case, step 310 is to determine whether the target plane has self-occlusion. If the target plane has self-occlusion, step 311 is executed for the target plane. If the target plane does not have self-occlusion, return to step 302.

[0122] Step 312 : Modeling the target object based on the completed three-dimensional point cloud data to obtain an updated point cloud model of the target object.

[0123] The completion of the obscured point cloud is completed through step 311, and the completed three-dimensional point cloud data includes the point cloud data of the obscured part, which can reflect the overall appearance of the current target object. The target object point cloud model obtained by modeling using the completed three-dimensional point cloud data also includes the point cloud data of the obscured part, which can reflect the overall outline of the current target object. At the same time, the updated target object point cloud model is used as the latest target object point cloud model. At this point, through the processing of steps 308-312, the update of the target object point cloud model is completed by completing the obscured part (called the second target object point cloud model update method). After completing this update, you can return to step 302 to wait for the next acquisition of the three-dimensional point cloud data of the target object, or you can execute step 207 to calculate the target object volume.

[0124] The above is the detailed processing process of the specific embodiment of the present application. Two specific examples are given below to illustrate the above two target object point cloud model updating methods.

[0125] Example 1 uses the first method to update the target object point cloud model. Figure 4a This is a bird's-eye view point cloud diagram of the cargo at time T0. At this time, the cargo has not yet been stacked on the ground, that is, there is no cargo point cloud model. The point cloud model will be initialized based on the current cargo point cloud to generate a cargo point cloud model, which is used for volume measurement at the current time and as input for the cargo point cloud model at time T1.

[0126] Figure 4b The figure below is a bird's-eye view of the cargo point cloud at time T1. As can be seen, at time T1, the first piece of cargo is placed on the ground, and part of the ground is obscured. However, the ground point cloud used as the input for the cargo point cloud model is the complete point cloud model obtained at time T0, including the obscured part. The two can be fused to update the cargo point cloud model, which is then used for volume measurement at time T1 and as input for the cargo point cloud model at time T2.

[0127] Figure 4c This is a bird's-eye view point cloud diagram of the cargo at time T2. It can be seen that at time T2, when the second layer of cargo is placed on the first layer of cargo, not only the ground area is blocked, but there is also an area on the first layer of cargo blocked by the second layer of cargo. At this time, the ground point cloud and the first layer of cargo point cloud in the input of the cargo point cloud model are complete. Based on this, the cargo point cloud model is updated to obtain point clouds with no obstructions on the ground, the first layer of cargo, and the second layer of cargo, as shown in Figure 2. Figure 4d As shown, it is used for volume measurement at time T2 and for input of cargo point cloud model at time T3.

[0128] Example 2 uses the second method to update the target object point cloud model. Figure 5a This is a bird's-eye view of the cargo point cloud, which includes the visible ground area, the ground area blocked by the cargo, the cargo point cloud of the nth layer, the cargo point cloud of the n-1th layer, and the self-occluded area of ​​the cargo of the n-1th layer that is blocked by the cargo of the nth layer.

[0129] In the process of updating the target object point cloud model, the cargo plane of the n-1 layer and the cargo plane of the n layer will be identified, and the corresponding rectangles will be fitted. Since the ratio of the point cloud area of ​​the cargo plane of the n-1 layer to the area of ​​the corresponding rectangle will be less than the set threshold, it means that the plane is self-occluded; and the ratio of the point cloud area of ​​the cargo plane of the n layer to the area of ​​the corresponding rectangle will be very close to 1, indicating that the plane is not self-occluded; it is necessary to complete the occluded point cloud in the n-1 layer plane, that is, to complete the points that are not covered in the fitted rectangular area, and use the average point cloud height of the n-1 layer plane to assign the height of the supplementary points. The effect after completion is as follows Figure 5b shown.

[0130] The above introduction to the specific implementation of this application takes self-occlusion as an example to illustrate the implementation process and effect examples of occlusion completion and target object point cloud modeling. In fact, the processing method of this application is also effective for point cloud missing caused by other factors similar to self-occlusion (such as: personnel occlusion).

[0131] It can be seen from the specific implementation of the volume measurement method in the above-mentioned present application that the present application can complete the point cloud of the obscured part when a regular target object is self-occluded, and can be used for volume measurement of self-occluded goods, without being affected by the placement of regular targets; for regular or irregular targets that are self-occluded or obscured by other non-goods, the target point cloud information of multiple frames can be used to model the target object, and the target point cloud information of the unobstructed area in the time series can be fused to realize the volume measurement of the target object that is self-occluded or obscured by other non-goods, which will not be affected by the transportation process, and the volume of the target object in the transportation process can also be dynamically measured.

[0132] This application also provides a device for volume measurement using three-dimensional point clouds. Figure 6 The basic structure diagram of the device is shown in Figure 2. Figure 6 As shown, the device includes: an acquisition unit, a length determination unit, a modeling unit, a model updating unit and a volume measurement unit.

[0133] The acquisition unit is used to obtain the current three-dimensional point cloud data of the measured object;

[0134] a length determination unit, configured to determine the maximum length of the current three-dimensional point cloud data in a preset first dimension;

[0135] a modeling unit configured to perform point cloud modeling using the current three-dimensional point cloud data to obtain a current modeling result when a difference between a maximum length in the first dimension and a second length is greater than an error threshold; wherein the second length is a length in the first dimension of a point cloud model of the target object determined based on historical three-dimensional point cloud data of the target object;

[0136] The model updating unit is used to fuse the current modeling result with the target object point cloud model to obtain an updated target object point cloud model;

[0137] The volume measurement unit is used to determine the volume of the target object based on the latest point cloud model of the target object when volume measurement is required.

[0138] Optionally, the modeling unit may be configured to model the target object according to current three-dimensional point cloud data of the target object in an initial state to obtain an initial point cloud model of the target object.

[0139] To support the second method to update the target object point cloud model, optionally, Figure 6 The device shown may further include: a plane clustering unit and a point cloud data completion unit.

[0140] The plane clustering unit can be used to perform plane clustering on the current three-dimensional point cloud data according to the coordinate values ​​of the first dimension to obtain planes corresponding to different coordinate values ​​of the first dimension;

[0141] The point cloud data completion unit may be configured to, when the coordinate fluctuation in the first dimension of all points clustered to the same plane is below a fluctuation threshold, determine based on the current three-dimensional point cloud data that the target object is self-occluded, use the coordinate data in the first dimension of the occluded plane in the current three-dimensional point cloud data to complete the point cloud data of the occluded portion, and model the target object based on the completed three-dimensional point cloud data to obtain an updated point cloud model of the target object;

[0142] The length determination unit can be used to perform the operation of determining the length of the current three-dimensional point cloud data in the first dimension when the coordinate fluctuation of the point clouds clustered on the same plane in the first dimension is not lower than the fluctuation threshold.

[0143] Optionally, in the point cloud data completion unit, a method for determining whether the target object is blocked may specifically include:

[0144] Fit each plane obtained from the clustering into a rectangle, and determine the area ratio covered by the point cloud within the rectangle; wherein the point cloud within the rectangle includes all points corresponding to the current three-dimensional point cloud data whose coordinates in the first dimension are greater than or equal to the coordinates of the corresponding plane in the first dimension;

[0145] When the determined area ratio is not higher than the ratio threshold, it is determined that the corresponding plane and the target object are self-occluded; when the determined area ratio is higher than the ratio threshold, it is determined that the corresponding plane is not self-occluded.

[0146] Optionally, in the point cloud data completion unit, the process of completing the obscured portion of the point cloud data may specifically include:

[0147] In the occluded plane, after fitting the plane into a rectangle, point cloud data is inserted into the part not covered by the point cloud in the rectangle. The coordinates of the inserted point cloud data in the first dimension are the same as the depth coordinates of the point cloud in the occluded plane.

[0148] Optionally, Figure 6 The device shown may further include a timer;

[0149] After each time the model updating unit obtains the updated target object point cloud model, or after each time the point cloud data completion unit obtains the updated target object point cloud model, starting the timer;

[0150] After the timing time set by the timer expires, the modeling unit can be further used to perform point cloud modeling using the current three-dimensional point cloud data to obtain the current modeling result.

[0151] This application also provides a computer-readable storage medium storing instructions that, when executed by a processor, can perform the steps of the method for volume measurement using a three-dimensional point cloud. In practical applications, the computer-readable medium can be included in the devices / apparatuses / systems described in the above embodiments, or it can exist independently and not be incorporated into the devices / apparatuses / systems. The instructions stored in the computer-readable storage medium, when executed by a processor, can perform the steps of the volume measurement method described above.

[0152] According to the embodiments disclosed in the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof, but is not intended to limit the scope of protection of the present application. In the embodiments disclosed in the present application, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0153] Figure 7 This application also provides an electronic device. Figure 7 , which shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0154] The electronic device may include a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 702 is executed, the aforementioned method for volume measurement using a three-dimensional point cloud can be implemented.

[0155] Specifically, in actual applications, the electronic device may further include components such as a power supply 703 and an input / output unit 704. Those skilled in the art will appreciate that Figure 7 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0156] The processor 701 is the control center of the electronic device. It uses various interfaces and lines to connect the various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 702 and calling data stored in the memory 702, it executes various functions of the server and processes data, thereby controlling the electronic device as a whole.

[0157] The memory 702 can be used to store software programs and modules, that is, the computer-readable storage medium mentioned above. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application required for a function, etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0158] The electronic device also includes a power supply 703 for supplying power to various components. This power supply 703 can be logically connected to the processor 701 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 703 can also include any of one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other components.

[0159] The electronic device may further include an input / output unit 704. The input / output unit 704 may be used to receive input digital or character information and generate keyboard, mouse, joystick, or optical signal input related to user settings and function control. The input / output unit 704 may also be used to display information input by the user or information provided to the user, as well as various graphical user interfaces. These graphical user interfaces may be composed of graphics, text, icons, videos, or any combination thereof.

[0160] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for volume measurement using a three-dimensional point cloud, characterized in that: include: Obtain the current three-dimensional point cloud data of the measured object; Determining a maximum length of the current three-dimensional point cloud data in a preset first dimension; When the difference between the maximum length in the first dimension and the second length is greater than an error threshold, performing point cloud modeling using the current three-dimensional point cloud data to obtain a current modeling result; wherein the second length is the length in the first dimension of the point cloud model of the target object determined based on the historical three-dimensional point cloud data of the target object; Fusing the current modeling result with the target object point cloud model to obtain an updated target object point cloud model; When volume measurement is required, determining the volume of the target object based on the latest point cloud model of the target object; After obtaining the current three-dimensional point cloud data of the measured object, the method further includes: Performing plane clustering on the current three-dimensional point cloud data according to the coordinate values ​​of the first dimension to obtain planes corresponding to different coordinate values ​​of the first dimension; When the coordinate fluctuation of all points clustered on the same plane in the first dimension is lower than the fluctuation threshold, when it is determined based on the current three-dimensional point cloud data that the target object is self-occluded, the point cloud data of the occluded portion is completed, and the target object is modeled using the completed three-dimensional point cloud data, and the modeling result is used as an updated point cloud model of the target object; when the coordinate fluctuation of the point cloud clustered on the same plane in the first dimension is not lower than the fluctuation threshold, the step of determining the length of the current three-dimensional point cloud data in the first dimension is continued; The method of determining whether the target object is self-occluded includes: Fitting each of the planes into a rectangle, and determining the area ratio covered by the point cloud within the rectangle; wherein the point cloud within the rectangle includes all points corresponding to the current three-dimensional point cloud data whose coordinates in the first dimension are greater than or equal to the coordinates of the corresponding plane in the first dimension; When the proportion is not higher than the proportion threshold, it is determined that the corresponding plane and the target object are self-occluded; when the proportion is higher than the proportion threshold, it is determined that the corresponding plane is not self-occluded.

2. The method according to claim 1, characterized in that In an initial state, the target object is modeled according to the current three-dimensional point cloud data of the target object to obtain an initial point cloud model of the target object.

3. The method according to claim 1, characterized in that The point cloud data of the obscured portion is completed using the coordinate data of the obscured plane in the current three-dimensional point cloud data in the first dimension.

4. The method according to claim 3, characterized in that The step of completing the obscured point cloud data includes: In the occluded plane, after fitting the plane into a rectangle, point cloud data is inserted into the portion of the rectangle not covered by the point cloud, and the coordinates of the inserted point cloud data in the first dimension are the same as the depth coordinates of the point cloud in the occluded plane.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The timing starts each time an updated point cloud model of the target object is obtained. After the set timing time is up, the current three-dimensional point cloud data is used to perform point cloud modeling to obtain the current modeling result; the current modeling result is fused with the target object point cloud model to obtain an updated point cloud model of the target object.

6. The method according to any one of claims 1 to 4, characterized in that The operation of obtaining the current three-dimensional point cloud data of the measured object is performed once at set time intervals.

7. The method according to claim 3 or 4, characterized in that Every time the operation of acquiring the current three-dimensional point cloud data is performed N times, an operation of plane clustering the current three-dimensional point cloud data in the first dimension is performed once; wherein N is a preset natural number.

8. A device for volume measurement using a three-dimensional point cloud, characterized in that: include: Acquisition unit, length determination unit, modeling unit, model updating unit, volume measurement unit, plane clustering unit and point cloud data completion unit; The acquisition unit is used to acquire the current three-dimensional point cloud data of the measured object; The plane clustering unit is configured to perform plane clustering on the current three-dimensional point cloud data according to the coordinate values ​​of the first dimension to obtain planes corresponding to different coordinate values ​​of the first dimension; The point cloud data completion unit is configured to, when the coordinate fluctuation of all points clustered to the same plane in the first dimension is lower than a fluctuation threshold, complete the point cloud data of the occluded portion when it is determined based on the current three-dimensional point cloud data that the target object is self-occluded, and use the completed three-dimensional point cloud data to model the target object, and use the modeling result as an updated point cloud model of the target object; The length determining unit is configured to determine the maximum length of the current three-dimensional point cloud data in the preset first dimension when the coordinate fluctuation of the point clouds clustered on the same plane in the first dimension is not less than the fluctuation threshold; The modeling unit is configured to perform point cloud modeling using the current three-dimensional point cloud data to obtain a current modeling result when a difference between a maximum length in the first dimension and a second length is greater than an error threshold; wherein the second length is a length in the first dimension of the point cloud model of the target object determined based on historical three-dimensional point cloud data of the target object; The model updating unit is configured to fuse the current modeling result with the target object point cloud model to obtain an updated target object point cloud model; The volume measurement unit is configured to determine the volume of the target object based on the latest point cloud model of the target object when volume measurement is required; Wherein, in the point cloud data completion unit, a method of determining whether the target object is blocked includes: Fitting each of the planes into a rectangle, and determining the area ratio covered by the point cloud within the rectangle; wherein the point cloud within the rectangle includes all points corresponding to the current three-dimensional point cloud data whose coordinates in the first dimension are greater than or equal to the coordinates of the corresponding plane in the first dimension; When the proportion is not higher than the proportion threshold, it is determined that the corresponding plane and the target object are self-occluded; when the proportion is higher than the proportion threshold, it is determined that the corresponding plane is not self-occluded.

9. The device according to claim 8, characterized in that The modeling unit is used to model the target object according to the current three-dimensional point cloud data of the target object in an initial state to obtain an initial point cloud model of the target object.

10. The device according to claim 8, characterized in that In the point cloud data completion unit, the point cloud data of the blocked portion is completed using the coordinate data of the blocked plane in the first dimension in the current three-dimensional point cloud data.

11. The device according to claim 10, characterized in that In the point cloud data completion unit, completing the obscured point cloud data includes: In the occluded plane, after fitting the plane into a rectangle, point cloud data is inserted into the portion of the rectangle not covered by the point cloud, and the coordinates of the inserted point cloud data in the first dimension are the same as the depth coordinates of the point cloud in the occluded plane.

12. The device according to any one of claims 8 to 11, characterized in that The apparatus further comprises a timer; After each time the model updating unit obtains the updated target object point cloud model, or after each time the point cloud data completing unit obtains the updated target object point cloud model, starting a timer; After the timing time set by the timer expires, the modeling unit is further used to perform point cloud modeling using the current three-dimensional point cloud data to obtain a current modeling result.

13. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by a processor, the three-dimensional volume measurement method described in any one of claims 1 to 7 can be implemented.

14. An electronic device, characterized in that: The electronic device includes at least a computer-readable storage medium and also includes a processor; The processor is configured to read executable instructions from the computer-readable storage medium and execute the instructions to implement the three-dimensional volume measurement method according to any one of claims 1 to 7.

Images