Method, device, system and equipment for measuring volume of goods in carriage

Abstract

The invention relates to the technical field of logistics transportation, and discloses a method, a device, a system and equipment for measuring the volume of goods in a carriage, and the method comprises the steps: further generating three-dimensional point cloud data through depth map data collected by a TOF camera, correcting the three-dimensional point cloud data, and carrying out the real-time detection in a target carriage. According to the method, whether the target compartment belongs to the static scene currently or not is determined, and when the target compartment belongs to the static scene, the volume of the target cargo is calculated, so that the burden of measurement personnel is greatly reduced, the time measurement cost is reduced, and meanwhile, the possibility and convenience of real-time information mastering are provided for remote scheduling personnel; the accuracy of measuring the volume of target goods can be improved.

Description

Technical Field

[0001] The present invention relates to the field of logistics and transportation technology, and in particular to a method, device, system and equipment for measuring the volume of goods in a carriage. Background Art

[0002] In the field of logistics and transportation technology, timely obtaining and mastering the volume occupancy rate of cargo in the carriage is of great significance to logistics scheduling management. Effective logistics scheduling based on the remaining volume in the carriage can effectively reduce transportation costs.

[0003] The volume measurement methods currently used mainly include manual estimation and monocular camera image recognition. Among them, the manual estimation method refers to the measurement and estimation of the cargo by measurement personnel using tape measures, rangefinders and other equipment. The monocular camera image recognition method refers to the use of a mobile monocular camera to obtain the front image of the carriage, and to optimize the objective function at the binary image level with the pre-stored three-dimensional model of the truck, and estimate the average depth value of the carriage in the direction of the cargo depth to complete the calculation.

[0004] The manual estimation method relies on the operator's measurement experience, especially when measuring irregular cargo. It is not only time-consuming but also has low accuracy. The monocular camera image recognition method essentially compares pixel values ​​at the two-dimensional image level and fits a plane that describes the overall depth information of the cargo in the compartment. This greatly increases the measurement error when the depth of the cargo in the loading direction is not a strict plane, which still leads to low accuracy. Summary of the invention

[0005] In view of this, the present invention provides a method, device, system and equipment for measuring the volume of cargo in a carriage to solve the problems of high measurement time cost and low measurement accuracy.

[0006] According to a first aspect, an embodiment of the present disclosure provides a method for measuring the volume of cargo in a compartment, the method comprising:

[0007] Obtain the three-dimensional point cloud data of the target carriage in an empty state;

[0008] Calculate the coordinate axis deflection angle of the 3D point cloud data in the world coordinate system;

[0009] Correct the 3D point cloud data based on the deflection angle of the coordinate axis;

[0010] Obtain the current depth map data of the target car in the current state;

[0011] Based on the current depth map data of the target compartment in the current state, detecting whether the target compartment belongs to a static scene in the current state;

[0012] If the target compartment is a static scene in the current state, the volume of the target cargo in the target compartment is calculated based on the corrected 3D point cloud data and the current depth map data.

[0013] The disclosed embodiment uses the depth map data collected by the TOF camera to further calibrate it into three-dimensional point cloud data, corrects the three-dimensional point cloud data, and detects static target cargo in real time inside the target compartment. After the target cargo is detected, the volume of the target cargo is calculated, which greatly reduces the burden on measurement personnel and reduces the time measurement cost. At the same time, it provides remote dispatch personnel with the possibility and convenience of grasping information in real time. Most importantly, it can improve the accuracy of measuring the volume of the target cargo.

[0014] In some optional implementations, calculating the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system includes:

[0015] Downsampling the 3D point cloud data;

[0016] Sort the downsampled 3D point cloud data according to the X-axis coordinate value in the world coordinate system;

[0017] Acquire the three-dimensional point cloud data of the mutually opposing side walls of the target carriage from the sorted three-dimensional point cloud data;

[0018] Downsampling the side wall three-dimensional point cloud data in an equidistant manner;

[0019] Based on the side wall three-dimensional point cloud data that has been downsampled in an equidistant manner, derivative data between adjacent point clouds are obtained;

[0020] Calculate the average value of derivative data between adjacent point clouds;

[0021] Extract the maximum and minimum values ​​of the X-axis coordinate values ​​of the point cloud group corresponding to the average value from the derivative data between adjacent point clouds;

[0022] Extracting first target point cloud data between the minimum value and the maximum value from the downsampled three-dimensional point cloud data, and using the three-dimensional point cloud data that does not meet the minimum value and the maximum value as second target point cloud data;

[0023] Based on the first target point cloud data and the second target point cloud data, the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system is calculated by the RANSAC algorithm.

[0024] By executing the above implementation, the three-dimensional point cloud data of the target empty compartment is corrected, which is conducive to the subsequent accurate calculation of the volume of the target cargo.

[0025] In some optional implementations, based on the first target point cloud data and the second target point cloud data, calculating the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system by using a RANSAC algorithm includes:

[0026] Acquire the two-dimensional point cloud data between the Y axis and the Z axis from the first target point cloud data to generate a first fitting straight line through the RANSAC algorithm, and calculate the X-axis deflection angle of the three-dimensional point cloud data in the world coordinate system based on the slope value of the first fitting straight line, the X-axis deflection angle is used to correct the bottom surface depth map data of the target compartment, and the X-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target compartment relative to the X-axis;

[0027] Acquire the two-dimensional point cloud data between the X-axis and the Z-axis from the second target point cloud data to generate a second fitting straight line and a third fitting straight line through the RANSAC algorithm, and calculate the Y-axis deflection angle of the three-dimensional point cloud data in the world coordinate system based on the slope value of the second fitting straight line and the slope value of the third fitting straight line. The Y-axis deflection angle is used to correct the depth map data of the two sides of the target car, and the Y-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target car relative to the Y-axis;

[0028] Acquire the two-dimensional point cloud data between the X-axis and the Y-axis from the first target point cloud data to generate a fourth fitting straight line through the RANSAC algorithm, and calculate the Z-axis deflection angle of the three-dimensional point cloud data in the world coordinate system based on the slope value of the fourth fitting straight line, wherein the Z-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target car relative to the Z-axis;

[0029] By executing the above implementation mode, the correction data of the three-dimensional point cloud data of the target empty compartment is further specifically calculated, and the three-dimensional point cloud data of the target empty compartment is corrected, which is conducive to the subsequent accurate calculation of the volume of the target cargo.

[0030] In some optional implementations, based on the current depth map data of the target compartment in the current state, detecting whether the target compartment in the current state belongs to a static scene includes:

[0031] Get the actual number of frames of the current depth map data of the target car in the current state;

[0032] Acquire data of a first preset number of frames for detecting a static number of frames from current depth map data of an actual number of frames;

[0033] Performing median filtering on the current depth map data of the actual frame number according to the second preset frame number;

[0034] Convert the current depth map data after median filtering into three-dimensional point cloud data;

[0035] Correct the three-dimensional point cloud data corresponding to the current depth map data using the coordinate axis deflection angle;

[0036] Projecting the three-dimensional point cloud data corresponding to the current depth map data in the cargo depth direction to obtain the current voxelized depth map data;

[0037] Performing differential processing on the current voxelized depth map data according to a third preset frame number, and outputting a differential processing result of a target frame number generated based on the actual frame number and the first preset frame number;

[0038] Determine whether each frame of data in the differential processing result is static frame data;

[0039] If yes, obtain the total number of static frames of the static frame data from the difference processing result, and calculate the ratio of the total number of static frames to the target number of frames;

[0040] If the ratio is greater than the first preset threshold, the target car belongs to a static scene in the current state.

[0041] By executing the above implementation, the current depth map data of the target compartment in the current state is sequentially subjected to median filtering and differential processing, which is conducive to accurately locking the static target object to prevent dynamic interference objects from interfering with the target object and affecting the volume calculation of the target cargo.

[0042] In some optional implementations, determining whether each frame of data in the differential processing result is static frame data includes:

[0043] Obtain current frame data of current voxelized depth map data from the difference processing result;

[0044] Obtain the voxel difference between two frames of data consisting of a frame of data adjacent to the current frame of data at the same position, as well as the maximum number of voxels in the X-axis direction and the maximum number of voxels in the Y-axis direction;

[0045] Calculate the objective function value for detecting static frame data based on the voxel difference, the maximum number of voxels in the X-axis direction, and the maximum number of voxels in the Y-axis direction;

[0046] If the objective function value is greater than the second preset threshold, the current frame data in the difference processing result is static frame data.

[0047] By executing the above implementation, further determining the static frame data in the differential processing result is helpful to accurately lock the static target object to prevent the dynamic interference object from interfering with the target object and affecting the volume calculation of the target cargo.

[0048] In some optional implementations, calculating the volume of the target cargo in the target compartment includes:

[0049] Obtain the average cargo depth value of the target compartment in the cargo depth direction;

[0050] Obtain the width voxel size value and height voxel size value of the target carriage;

[0051] The volume of the target cargo in the target compartment is calculated based on the average depth value, width voxel size value and height voxel size value of the cargo.

[0052] By executing the above implementation, it is helpful to accurately calculate the volume of the target acquisition.

[0053] In some optional implementations, in the process of calculating the target volume of the target cargo in the current state, it further includes: determining whether the target cargo is blocked by a blocking object, including:

[0054] Obtaining the inner wall thickness threshold of the target compartment and the average depth value of each voxel of the target compartment in the cargo depth direction;

[0055] Obtain the Z-axis coordinate value from the corrected 3D point cloud data;

[0056] Based on the Z-axis coordinate value, obtain the estimated value of the inner wall of the target compartment;

[0057] Based on the average depth value of each voxel in the carriage, the estimated value of the inner wall and the inner wall thickness threshold, it is determined whether some voxel areas of the target cargo are blocked by blocking objects.

[0058] By executing the above embodiment, during the process of calculating the volume of the target cargo, the target cargo is prevented from being disturbed by the blocking objects.

[0059] In some optional implementations, if a portion of a voxel area of ​​the target cargo is blocked by a blocking object, the blocked area is filled by an adjacent area of ​​the blocked area.

[0060] By executing the above implementation, the blocking area is filled, which is conducive to the accurate calculation of the target goods.

[0061] According to a second aspect, an embodiment of the present disclosure further provides a device for measuring the volume of cargo in a compartment, the device comprising:

[0062] A three-dimensional data acquisition module is used to acquire three-dimensional point cloud data of the target carriage in an empty state;

[0063] The deflection angle calculation module calculates the deflection angle of the coordinate axis of the three-dimensional point cloud data in the world coordinate system;

[0064] 3D data correction module, used to correct 3D point cloud data based on the deflection angle of the coordinate axis;

[0065] A current data acquisition module is used to obtain the current depth map data of the target compartment in the current state;

[0066] A target cargo detection module, used to detect whether there is target cargo in the target compartment in the current state based on the current depth map data of the target compartment in the current state;

[0067] The static scene detection module is used to calculate the volume of the target cargo in the target compartment based on the corrected three-dimensional point cloud data and the current depth map data if the target compartment belongs to a static scene in the current state.

[0068] According to a third aspect, an embodiment of the present disclosure provides a system for measuring the volume of cargo in a compartment, comprising:

[0069] A mobile terminal, used for sending a trigger signal;

[0070] A router, installed inside the target car, for transmitting the trigger signal;

[0071] A TOF camera installed inside the target compartment, used to collect current depth map data of the target compartment in a current state and receive a trigger signal through a router, and execute the volume measurement method of the cargo in the compartment according to the first aspect or any embodiment of the first aspect based on the trigger signal, and output the volume of the target cargo in the target compartment;

[0072] The server receives the volume of the target cargo in the target compartment output by the TOF camera through the router.

[0073] According to a fourth aspect, an embodiment of the present disclosure provides a computer device, including:

[0074] The memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for measuring the volume of cargo in a vehicle compartment in the first aspect or any embodiment of the first aspect by executing the computer instructions.

[0075] According to the fifth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable the computer to implement the method for measuring the volume of cargo in a vehicle compartment in the first aspect or any embodiment of the first aspect.

[0076] According to the sixth aspect, an embodiment of the present disclosure provides a computer program product, comprising computer instructions, and the computer instructions are used to enable a computer to execute the method for measuring the volume of cargo in a vehicle compartment in the first aspect or any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0078] Figure 1 is a schematic flow chart of a method for measuring the volume of cargo in a carriage according to an embodiment of the present invention;

[0079] Figure 2 is a schematic flow chart of another method for measuring the volume of cargo in a carriage according to an embodiment of the present invention;

[0080] Figure 3A is the three-dimensional point cloud data of the target carriage after downsampling according to an embodiment of the present invention;

[0081] Figure 3B is a schematic diagram of the distribution of point clouds of the left and right side walls of the target compartment on the X-axis according to an embodiment of the present invention;

[0082] Figure 3C is a schematic diagram of derivative distribution between adjacent points according to an embodiment of the present invention;

[0083] Figure 3D is a schematic diagram of yz plane projection and fitting straight line according to an embodiment of the present invention;

[0084] Figure 3E is a schematic diagram of xz plane projection and fitted straight line according to an embodiment of the present invention;

[0085] Figure 3F is a schematic diagram of xy plane projection and fitted straight line according to an embodiment of the present invention;

[0086] Figure 4 is a flow chart of another method for measuring the volume of cargo in a carriage according to an embodiment of the present invention;

[0087] Figure 5 is a schematic diagram of three-dimensional point cloud data correction and voxelization according to an embodiment of the present invention;

[0088] Figure 6 is a flow chart of another method for measuring the volume of cargo in a carriage according to an embodiment of the present invention;

[0089] Figure 7 is a flow chart of another method for measuring the volume of cargo in a carriage according to an embodiment of the present invention;

[0090] Fig. 8A is a schematic diagram of a point cloud depth map according to an embodiment of the present invention;

[0091] Figure 8B is a schematic diagram of blind spot completion according to an embodiment of the present invention;

[0092] Figure 8C is a schematic diagram of voxelization after blind area completion according to an embodiment of the present invention;

[0093] Fig. 9 is a structural block diagram of a system for measuring the volume of cargo in a carriage according to an embodiment of the present invention;

[0094] Fig.10 is a schematic diagram of the use process and related task processing of the method for measuring the volume of cargo in a carriage according to an embodiment of the present invention;

[0095] Fig.11 is a schematic diagram of the framework of an Internet of Things system according to an embodiment of the present invention;

[0096] Fig.12 is a structural block diagram of a device for measuring the volume of cargo in a carriage according to an embodiment of the present invention;

[0097] Fig.13 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0098] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0099] According to an embodiment of the present invention, an embodiment of a method for measuring the volume of cargo in a vehicle compartment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0100] In this embodiment, a method for measuring the volume of cargo in a compartment is provided, which can be used in mobile terminals, such as mobile phones, tablet computers, etc. Figure 1 FIG. 4 is a flow chart of a method for measuring the volume of cargo in a carriage according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0101] Step S101, obtaining three-dimensional point cloud data of the target compartment in an empty state.

[0102] Specifically, the target carriage can be a truck with a body, and the unloaded state indicates an empty carriage state. The three-dimensional point cloud data is converted based on the two-dimensional depth map data collected by the TOF camera. TOF is the abbreviation of "time-of-flight". The TOF camera is a depth imaging camera that illuminates the target object and measures the distance by measuring the transmission time of light between the lens and the object, thereby realizing a depth map through multiple light rays and achieving 3D stereo depth sensing.

[0103] In a specific example, obtaining three-dimensional point cloud data of a target carriage in an empty state includes:

[0104] Step a1, obtaining the initial depth map data of the target compartment in an empty state.

[0105] Step a2: calibrate the three-dimensional point cloud data of the target car in an empty state based on the initial depth map data.

[0106] Specifically, step a1-step a2 is equivalent to converting the depth information collected by the TOF camera in the empty car state to generate three-dimensional point cloud data. Since the depth data obtained by the TOF camera can be used to calculate the three-dimensional point cloud data of the spatial object, the three-dimensional point cloud data more accurately describes the relative distance of objects in the physical space.

[0107] Step S102, calculating the deflection angle of the coordinate axis of the three-dimensional point cloud data in the world coordinate system.

[0108] Step S103, correcting the three-dimensional point cloud data based on the coordinate axis deflection angle.

[0109] Specifically, since the installation of the TOF camera cannot guarantee that the photographed object is always horizontal to the world coordinate system, in order to ensure the accuracy of the subsequent calculation of the cargo volume in the target compartment, it is necessary to calculate the coordinate axis deflection angle based on the three-dimensional point cloud data in advance, that is, the posture correction parameter, and then correct the three-dimensional point cloud data of each subsequent frame to coincide with the world coordinate system, so as to facilitate the voxelization calculation of the target object in the target compartment. The coordinate axis deflection angle can be expressed as θ x ,θ y ,θ z Indicates that, where θ x It represents the X-axis deflection angle, θ y It represents the Y-axis deflection angle, θ z Indicates the Z-axis deflection angle.

[0110] The disclosed embodiment calculates the deflection angle of the coordinate axis of the three-dimensional point cloud data in the world coordinate system and corrects the three-dimensional point cloud data, which is beneficial to ensuring the measurement accuracy of the target object in the target compartment.

[0111] Step S104, obtaining current depth map data of the target vehicle compartment in the current state.

[0112] Specifically, the current state of the target compartment indicates the state in which the target compartment is being loaded with goods and the state in which the target compartment has been fully loaded with goods. The current depth map data indicates the current depth map data collected in real time by the TOF camera in the above state.

[0113] Step S105 , based on the current depth map data of the target compartment in the current state, detecting whether the target compartment in the current state belongs to a static scene.

[0114] Specifically, here, it is equivalent to performing static scene detection on the target compartment in real time, that is, realizing the volume calculation of the target object without the interference of dynamic targets.

[0115] Step S106: If the target compartment is a static scene in the current state, the volume of the target cargo in the target compartment is calculated based on the corrected three-dimensional point cloud data and the current depth map data.

[0116] Specifically, for a static scene of the target compartment, the volume of the target cargo in the target compartment is calculated based on the corrected 3D point cloud data and the current depth map data through the volume calculation formula. Figure 1 If the target car in the current state does not belong to a static scene, return to step S104 to continue to obtain the current depth map data of the target car in the current state in real time.

[0117] Furthermore, based on the corrected three-dimensional point cloud data, the three-dimensional point cloud data is projected in the cargo depth direction to obtain empty car voxelized depth map data of the target compartment. The current depth map data also needs to be further processed to obtain current voxelized depth map data. Therefore, the volume of the target cargo in the target compartment is calculated based on the empty car voxelized depth map data of the target compartment and the current voxelized depth map data.

[0118] The disclosed embodiment executes the above steps S101 to S106, further converts the depth map data collected by the TOF camera into three-dimensional point cloud data, corrects the three-dimensional point cloud data, and performs real-time detection inside the target compartment to determine whether the target compartment currently belongs to a static scene. When the target compartment belongs to a static scene, the volume of the target cargo is calculated, which greatly reduces the burden on the measurement personnel and reduces the time measurement cost. At the same time, it provides the remote dispatching personnel with the possibility and convenience of grasping information in real time. Most importantly, it can improve the accuracy of measuring the volume of the target cargo.

[0119] In this embodiment, a method for measuring the volume of cargo in a compartment is provided, which can be used in mobile terminals, such as mobile phones, tablet computers, etc. Figure 2FIG. 4 is a flow chart of a method for measuring the volume of cargo in a carriage according to an embodiment of the present invention. Figure 2 As shown, step S102, calculating the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system, the process includes the following steps:

[0120] Step S1021, downsampling the three-dimensional point cloud data.

[0121] Specifically, downsampling of three-dimensional point cloud data is performed to improve computational efficiency.

[0122] Step S1022, sorting the downsampled three-dimensional point cloud data according to the X-axis coordinate value in the world coordinate system.

[0123] Specifically, Figure 3A As shown in the figure, in order to improve the calculation efficiency, the three-dimensional point cloud data is arranged in the column direction to obtain the three-dimensional point cloud data P of the empty compartment after downsampling. n},in, Represents the coordinate values ​​of the X-axis, Y-axis, and Z-axis of the point cloud with index n.

[0124] Step S1023, obtaining the three-dimensional point cloud data of the mutually opposing side walls of the target compartment from the sorted three-dimensional point cloud data.

[0125] Specifically, since the point clouds of the left and right side walls of the target compartment are more concentrated on the X-axis, the point clouds are sorted based on the size of the X-axis coordinate values, and the following can be obtained: Figure 3B The distribution diagram shown.

[0126] Step S1024, downsampling the side wall three-dimensional point cloud data in an equidistant manner.

[0127] Step S1025 , based on the sidewall three-dimensional point cloud data that has been downsampled in an equidistant manner, derivative data between adjacent point clouds are obtained.

[0128] Specifically, in Figure 3B It can be found that the X value difference of the point cloud on both sides of the target compartment is small, while the distribution of the point cloud in the middle of the left and right side walls shows an upward trend. Figure 3B After equally spaced downsampling of the data points, we can find the derivatives between adjacent points. Figure 3C The derivative distribution diagram is shown.

[0129] Step S1026, calculating the average value of the derivative data between adjacent point clouds.

[0130] Step S1027, extracting the maximum and minimum X-axis coordinate values ​​of the point cloud group corresponding to the average value from the derivative data between adjacent point clouds.

[0131] Step S1028: extract first target point cloud data between the minimum value and the maximum value from the down-sampled three-dimensional point cloud data, and use the three-dimensional point cloud data that does not meet the minimum value and the maximum value as the second target point cloud data.

[0132] Specifically, Figure 3C The derivative data between adjacent point clouds in the , and the maximum and minimum values ​​of the derivative greater than the average value are extracted, and then the corresponding X-axis coordinate values ​​are traced back, and the minimum coordinate value is selected from the X-axis coordinate values, which is recorded as x min , select the maximum coordinate value in the X-axis coordinate value, recorded as x max , extract the x in the 3D point cloud data min <x n <x max The point cloud data of the condition is recorded as P I , P I That is, the first target point cloud data mentioned above will not satisfy x min <x n <x max The point cloud data of the condition is taken as the second target point cloud data, denoted as P o .

[0133] Step S1029: Based on the first target point cloud data and the second target point cloud data, the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system is calculated by using the RANSAC algorithm.

[0134] Specifically, based on the selected first target point cloud data and the second target point cloud data, the coordinate axis deflection angle θ of the three-dimensional point cloud data in the world coordinate system is calculated according to the RANSAC algorithm. x ,θ y ,θ z RANSAC algorithm is the abbreviation of Random Sample Consensus Algorithm.

[0135] The embodiment of the present disclosure is conducive to accurately obtaining the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system by executing the above steps S1021 to S1029.

[0136] In this embodiment, a method for measuring the volume of cargo in a compartment is provided, which can be used in mobile terminals, such as mobile phones, tablet computers, etc. Figure 4 FIG. 4 is a flow chart of a method for measuring the volume of cargo in a carriage according to an embodiment of the present invention. Figure 4 As shown, step S1029, based on the first target point cloud data and the second target point cloud data, the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system is calculated by the RANSAC algorithm. The process includes the following steps:

[0137] Step S401, obtain the two-dimensional point cloud data between the Y axis and the Z axis from the first target point cloud data to generate a first fitting straight line through the RANSAC algorithm, and calculate the X-axis deflection angle of the three-dimensional point cloud data in the world coordinate system based on the slope value of the first fitting straight line. The X-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target car relative to the X-axis.

[0138] Specifically, the first target point cloud data is P I , for the first target point cloud data P I The RANSAC algorithm is used for the two-dimensional point cloud data between the Y axis and the Z axis. The purpose is to use the feature of the large number of points on the bottom surface of the target car to extract its first fitting straight line y=k1×z+b1 in the YZ plane. In the first fitting straight line function, y and z are the y and z coordinate values ​​of any point on the fitting straight line, respectively, and k1 and b1 are the slope and intercept of the first fitting straight line function on the yz plane, respectively. By executing step S401, in order to calculate the X-axis deflection angle θ of the three-dimensional point cloud data in the world coordinate system x , X-axis deflection angle θ x It is used to correct the bottom depth map data of the target car in the world coordinate system, such as Figure 3D shown.

[0139] Step S402, obtaining the two-dimensional point cloud data between the X-axis and the Z-axis from the second target point cloud data to generate a second fitting straight line and a third fitting straight line through the RANSAC algorithm, and based on the slope value of the second fitting straight line and the slope value of the third fitting straight line, calculating the Y-axis deflection angle of the three-dimensional point cloud data in the world coordinate system, the Y-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target car relative to the Y-axis.

[0140] Specifically, the second target point cloud data is P o , for the second target point cloud data P o The RANSAC algorithm is used on the two-dimensional point cloud data between the X-axis and the Z-axis to use the feature of the large number of point clouds on the two side walls of the car to extract the second fitting straight line z=k on the left and right side walls in the x and z planes respectively. 2，l ×x+b 2，l And the third fitting straight line z = k 2，r ×x+b 2，r , where x and z are the x and z coordinates of any point on the two fitting lines, respectively, and k 2，l , b 2，l are the slope and intercept of the second fitting straight line function of the left wall on the xz plane, respectively. The subscript l represents the left wall. Similarly, k 2，r , b 2，rare the slope and intercept of the third fitting straight line function on the xz plane, respectively, and the subscript r represents the right side wall. By executing step S402, in order to calculate the Y-axis deflection angle θ of the three-dimensional point cloud data in the world coordinate system, y It is used to correct the side depth map data of the target car in the world coordinate system, such as Figure 3E shown.

[0141] Step S403, obtain the two-dimensional point cloud data between the X-axis and the Y-axis from the first target point cloud data to generate a fourth fitting straight line through the RANSAC algorithm, and calculate the Z-axis deflection angle of the three-dimensional point cloud data in the world coordinate system based on the slope value of the fourth fitting straight line. The Z-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target car relative to the Z axis.

[0142] Specifically, the first target point cloud data is P I , for the first target point cloud data P I The RANSAC algorithm is used for the two-dimensional point cloud data between the X-axis and the Y-axis, with the purpose of utilizing the feature of the large number of point clouds on the inner wall of the car to extract the fourth fitting straight line y=k3×x+b3 on the XY plane. In the fourth fitting straight line function, x and y are respectively the x and y coordinate values ​​of any point on the fitting straight line, k3 and b3 are respectively the slope and intercept of the fourth fitting straight line function on the xy plane. By executing step S403, in order to calculate the Z-axis deflection angle θ of the three-dimensional point cloud data in the world coordinate system, the Z-axis deflection angle θ is used to realize the correction of the bottom depth map data of the target car in the world coordinate system, such as Figure 3F shown.

[0143] In the specific example, k1 in the first fitting straight line and k in the second fitting straight line are 2，l , k 2，r , k3 in the third fitting line can be calculated by the following formula (1) to obtain the coordinate axis deflection angle θ x ,θ y ,θ z ,θ x ,θ y ,θ z The corresponding radian value is ω x ,ω y ,ω z The original three-dimensional point cloud data of each frame can be transformed into Corrected to coincide with the world coordinate system to obtain the corrected 3D point cloud data Among them, P f and P c are all 3-row n-column point cloud coordinate value matrices, x fn ,y fn , zfn Represents three-dimensional point cloud data P f The x, y, and z coordinates of the nth point cloud in cn ,y cn , z cn Represents three-dimensional point cloud data P c The x, y and z coordinate values ​​of the nth point cloud in the rotation matrix R is a 3-row 3-column matrix used to transform the point cloud coordinate value matrix P of the original 3D point cloud data f The point cloud coordinate value matrix P converted to the three-dimensional point cloud depth data rotated to coincide with the world coordinate system c , which is the corrected three-dimensional point cloud coordinate value matrix. In formula (2), R(u, v) represents the value of the element in the uth row and vth column of the matrix, ω x ,ω y ,ω z That is, the deflection angle θ in equation (1) x ,θ y ,θ z The corresponding radian value.

[0144] θ x = atan(k1)

[0145] θ y = atan(k 2，l )+atan(k 2，r )

[0146] θ z = atan(k3) (1)

[0147] P c =R×P f

[0148] R(1,1)=cos(ω z )cos(ω y )

[0149] R(1,2)=sin(ω x )sin(ω y )cos(ω z )-cos(ω x )sin(ω z )

[0150] R(1,3)=sin(ω y )cos(ω x )cos(ω z )+sin(ω x )sin(ω z )

[0151] R(2,1)=cos(ωy )sin(ω z )

[0152] R(2,2)=sin(ω x )sin(ω y )sin(ω z )+cos(ω x )cos(ω z )

[0153] R(2,3)=sin(ω y )sin(ω z )cos(ω x )-sin(ω x )cos(ω z )

[0154] R(3,1)=-sin(ω y )

[0155] R(3,2)=sin(ω x )cos(ω y )

[0156] R(3,3)=cos(ω x )cos(ω y ) (2)

[0157] like Figure 5 As shown in the figure, it is a specific schematic diagram for correcting the three-dimensional point cloud data. Since the installation of the TOF camera cannot be guaranteed to be horizontal to the world coordinate system, it is necessary to calculate the posture correction parameters (coordinate axis deflection angle) based on the three-dimensional point cloud data, and correct the three-dimensional point cloud data of subsequent frames to coincide with the world coordinate system, so as to facilitate the volume calculation of the target cargo in the target compartment.

[0158] In this embodiment, a method for measuring the volume of cargo in a compartment is provided, which can be used in mobile terminals, such as mobile phones, tablet computers, etc. Figure 6 FIG. 4 is a flow chart of a method for measuring the volume of cargo in a carriage according to an embodiment of the present invention. Figure 6 As shown, step S105, based on the current depth map data of the target compartment in the current state, detect whether the target compartment in the current state belongs to a static scene, the process includes the following steps:

[0159] Step S1051, obtaining the actual number of frames of the current depth map data of the target vehicle compartment in the current state.

[0160] Specifically, the current state of the target compartment includes the state in which the target compartment is being loaded and the state in which the target compartment has been loaded. The current depth map data represents the current depth map data collected in real time by the TOF camera in the above state. Here, the current depth map data collected in real time by the TOF camera is used to detect whether the target compartment belongs to a static scene, and the actual frame number represents the total frame number of the current depth map data. The actual frame number can be represented by k2.

[0161] Step S1052: acquiring data of a first preset frame number for detecting a static frame number from the current depth map data of the actual frame number.

[0162] Specifically, the first preset frame number can be represented by k1, and the depth map data of the previous k1 frames is obtained from the current depth map data of the k2 frame for the preparation stage of static frame detection. The static frame detection preparation stage includes the dust filtering process and the pre-reserve data required for the inter-frame voxel depth value difference processing process.

[0163] Step S1053 , performing median filtering processing on the current depth map data of the actual frame number according to the second preset frame number.

[0164] Specifically, the second preset frame number can be expressed as w t Indicates that median filtering is also a dust filtering method, for a time domain window size of w t The corresponding pixels of each depth map data are subjected to median filtering in order to remove the influence of dynamic data such as dust in the car on the volume calculation.

[0165] Step S1054: convert the current depth map data after median filtering into three-dimensional point cloud data.

[0166] Specifically, this is equivalent to converting the current depth map data after median filtering into three-dimensional point cloud data.

[0167] Step S1055, using the coordinate axis deflection angle to correct the three-dimensional point cloud data corresponding to the current depth map data.

[0168] Specifically, the coordinate axis deflection angle here is further calculated through the above steps S1021 to S1029. The above coordinate axis deflection angle is used to correct the three-dimensional point cloud data of the current depth map data to the horizontal state of the real space (that is, it is equivalent to aligning the relative relationship between the current depth map data and the three-dimensional point cloud data of the target empty car in space).

[0169] Step S1056, projecting the three-dimensional point cloud data corresponding to the current depth map data in the cargo depth direction to obtain the current voxelized depth map data.

[0170] Specifically, the three-dimensional point cloud data corresponding to the current depth map data is projected in the cargo depth direction, which is equivalent to performing a two-dimensional projection.

[0171] Step S1057 , performing differential processing on the current voxelized depth map data according to the third preset frame number, and outputting a differential processing result of a target frame number generated based on the actual frame number and the first preset frame number.

[0172] Specifically, the third preset frame number is w d In order to improve the computational efficiency, the voxel depth map of the actual number of frames after median filtering and voxelization is further processed by using the inter-frame voxel depth value difference method, and the window size of the differential frame is w d At the same time, a differential processing result of a target frame number generated based on the actual frame number and the first preset frame number is output. The target frame number can be represented by Q.

[0173] In a specific example, based on the difference processing result of the target frame number generated by the actual frame number and the first preset frame number, the formula is used to calculate K2-K1+1=Q. For example, K2 is 12 frames, K1 is 7 frames, and Q is 6 frames.

[0174] Step S1058, determining whether each frame of data in the differential processing result is static frame data.

[0175] Specifically, the data in the difference processing result is Q frame voxel depth map data. At this time, the Q frame voxel depth map data is judged to determine whether the current depth map data of the k2 frame includes a static frame.

[0176] Step S1059: If yes, obtain the total number of static frames of the static frame data from the differential processing result, and calculate the ratio of the total number of static frames to the target number of frames.

[0177] Specifically, for example, the 1st frame data, the 4th frame data, and the 5th frame data in the Q frame depth map data are static frame data, the total number of static frames is represented by T, and the ratio of the total number of static frames to the target number of frames is represented by M, T=3 frames, Q=6 frames, T / Q=M, M=3 / 6=0.5.

[0178] Step S1060: If the ratio is greater than the first preset threshold, the target car belongs to a static scene in the current state.

[0179] Specifically, for example, the first preset threshold can be 0.8, and the first preset threshold can be flexibly set in combination with the actual scenario. When the ratio M of the total number of static frames to the target number of frames is greater than the first preset threshold, it means that there are more static frame data, that is, the target car belongs to a static scene in the current state, that is, there is no dynamic target interference. On the contrary, the target car does not belong to a static scene in the current state, and there is dynamic target interference.

[0180] In a specific example, if the ratio is less than or equal to the first preset threshold, the target vehicle compartment does not belong to a static scene in the current state, that is, there is dynamic target interference. In this scenario, it is not the focus of the embodiment of the present disclosure.

[0181] In some optional implementations, the above step S1058, determining whether each frame of data in the difference processing result is static frame data, includes:

[0182] Step b1, obtaining current frame data of current voxelized depth map data from the difference processing result;

[0183] Step b2, obtaining the voxel difference between two frames of data consisting of a frame of data adjacent to the current frame of data at the same position and the maximum number of voxels in the X-axis direction and the maximum number of voxels in the Y-axis direction.

[0184] Specifically, the voxel difference between two adjacent frames of data at the same position can be expressed as D x，y Indicates that the maximum number of voxels in the X-axis direction can be expressed as N X Indicates that the maximum number of voxels in the Y-axis direction can be expressed as N Y express.

[0185] Step b3, calculating the objective function value for detecting static frame data based on the voxel difference, the maximum number of voxels in the X-axis direction and the maximum number of voxels in the Y-axis direction.

[0186] Specifically, the objective function value for detecting static frame data can be represented by δ.

[0187] In a specific example, the calculation formula of the objective function value is expressed by the following formula:

[0188]

[0189] Among them, in the above formula (1), D x，y It can be expressed by the following formula (4):

[0190]

[0191] Wherein, α in the above formula (1) is expressed by the following formula (5):

[0192]

[0193] In the above formula, are the depth values ​​of the voxels (x, y) of frame i and the previous Δi frame, and the window size w of Δi and the difference frame. d The relationship is w d =Δi+1, a is the gain value, L is the target car length, Bi , B i-Δi are the number of voxels without value in the i-th frame and the previous Δi frames, m i，X , m i，Y is the average position of the voxels without value in the i-th frame in the X-axis direction and the average position in the Y-axis direction, N X 、N Y They are the maximum number of voxels in the X-axis direction and the maximum number of voxels in the Y-axis direction respectively.

[0194] Step b4: if the objective function value is greater than the second preset threshold, the current frame data in the differential processing result is static frame data.

[0195] Further, if the objective function value is less than or equal to the second preset threshold, it is determined that each frame of data in the differential processing result is dynamic frame data.

[0196] Specifically, the embodiment of the present disclosure executes the above steps b1 to b4 to verify whether each frame of the current depth map data after differential processing is static frame data, so as to verify whether there are static objects in the target compartment in the current state, so as to eliminate the interference of dynamic objects, and finally realize the accurate calculation of the volume of the target cargo in the target compartment.

[0197] In this embodiment, a method for measuring the volume of cargo in a compartment is provided, which can be used in mobile terminals, such as mobile phones, tablet computers, etc. Figure 7 FIG. 4 is a flow chart of a method for measuring the volume of cargo in a carriage according to an embodiment of the present invention. Figure 7 As shown, in step S106, if the target compartment is a static scene in the current state, the volume of the target cargo in the target compartment is calculated based on the corrected three-dimensional point cloud data and the current depth map data. The process includes the following steps:

[0198] Step S1061, obtaining the average depth value of the target cargo in the cargo loading depth direction.

[0199] Specifically, the average depth value of the target cargo in the cargo depth direction represents the depth value between the door and the front wall of the target cargo. The average depth value of the cargo is obtained by projecting and voxelizing the three-dimensional point cloud data in the cargo depth direction. Specifically, the average depth value of the target cargo is the difference between the voxels generated by the data collected for the target cargo and the voxels generated by the three-dimensional point cloud data of the target compartment. The difference is the depth of the target cargo in the space corresponding to the voxel size. The average depth value of the target cargo can be expressed by v x，y express.

[0200] Step S1062, obtaining the width voxel size value and the height voxel size value of the target compartment.

[0201] Specifically, the width voxel size value of the target compartment can be represented by S1, and the height voxel size value of the target compartment can be represented by S2, wherein the width voxel size value of the target compartment and the height voxel size value of the target compartment can be set according to the resolution of the TOF camera, or the width voxel size value and the height voxel size value of the target compartment can be input in advance.

[0202] Step S1063 , calculating the volume of the target cargo in the target compartment based on the average depth value, width voxel size value and height voxel size value of the cargo.

[0203] In a specific example, the volume of the target cargo in the target compartment can be expressed by the following formula (6):

[0204]

[0205] Among them, V represents the volume of the target cargo in the target compartment, S1 represents the voxel size value of the width of the target compartment, S2 represents the voxel size value of the height of the target compartment, and v x，y is the average depth value of the target cargo in the cargo depth direction. Therefore, the disclosed embodiment can convert the volume occupancy rate of the target compartment into the volume occupancy rate of the target compartment according to the width and height dimensions of the target compartment input in advance and the average depth value of the target cargo in the cargo depth direction.

[0206] When the TOF camera and the front and rear goods are in certain positions, the front goods will block the laser from hitting the rear goods, causing a blind spot, such as Fig. 8A As shown, therefore, for such cases, the blind area voxels need to be completed.

[0207] Therefore, in some optional implementations, in the process of calculating the target volume of the target cargo in the current state, it further includes: determining whether the target cargo is blocked by a blocking object, including:

[0208] Step c1, obtaining the inner wall thickness threshold of the target compartment and the average depth value of each voxel of the target compartment in the cargo depth direction.

[0209] Specifically, the inner wall thickness threshold of the target compartment can be expressed as T t Indicates that the average depth value of the target compartment in the cargo depth direction can be expressed by the average depth value of the compartment on the (x, y) voxel, which can be expressed by E x，y The inner wall thickness threshold of the target compartment can be preset, and the average depth value of the target compartment in the cargo depth direction can be obtained by real-time detection.

[0210] Step c2, obtaining the Z-axis coordinate value from the corrected three-dimensional point cloud data.

[0211] Step c3, based on the Z-axis coordinate value, obtain the estimated value of the inner wall of the target compartment.

[0212] Specifically, the Z-axis coordinate values ​​in the corrected three-dimensional point cloud data are sorted in ascending order, and the Z values ​​of 90% to 95% are extracted and the average value is calculated as the estimated value of the inner wall of the target compartment. The estimated value of the inner wall of the target compartment is expressed as d w express.

[0213] Step c4, based on the average depth value of each voxel in the carriage, the estimated value of the inner wall and the inner wall thickness threshold, it is determined whether a part of the voxel area of ​​the target cargo is blocked by the blocking object.

[0214] Specifically, in order to magnify the difference in the objective function value under the static condition of the target car, the characteristic that dynamic objects block the TOF camera light path when moving and form blind spots with position changes on the voxels can be used to set the blind spots as valueless voxels and the non-blind spots as valued voxels.

[0215] In a specific example, the average depth value of the target compartment in the cargo depth direction is calculated by the following formula (7).

[0216]

[0217] The above formula (7) is used to calculate the voxelized data E of the target cargo volume. x，y Perform the inner wall data invalidation operation, where E x，y is the average depth of the target compartment in the cargo depth direction, NaN represents an invalid value, d w is the estimated value of the inner wall of the target compartment, T t is the inner wall thickness threshold of the target compartment.

[0218] In some optional implementations, the method further includes: if a partial voxel area of ​​the target cargo is blocked by a blocking object, the blocked area is filled by an adjacent area of ​​the blocked area.

[0219] Specifically, if E x，y With d w The absolute difference between the inner wall thickness threshold T of the target compartment is set. t If the voxel is within the range of the inner wall of the carriage, its value is not used for the following downward growth operation, so the voxel (x, y) is set to invalid and its value is NaN. Otherwise, it is set to valid and its value is E. x，y Then, the following formula (8) is used to perform a downward growth operation on the effective voxels in the blind area between the target goods to complete the voxels in the blind area.

[0220]

[0221] Among them, E x，y-1 is the average depth of voxel (x, y-1), then voxel (x, y-1) is the upper adjacent voxel of voxel (x, y). The completion effect is as follows Figure 8B and Figure 8C Finally, the result can be obtained by calculating the formula (6) for the non-invalid value voxel area.

[0222] In this embodiment, a system for measuring the volume of cargo in a compartment is provided, which is used in the method for measuring the volume of cargo in a compartment in the above embodiment. Fig. 9 As shown, the system includes: a mobile terminal 91, a router 92, a TOF camera 93 and a server 94.

[0223] The mobile terminal 91 is installed inside the target compartment and is used to send a trigger signal, and the trigger signal is used to instruct the TOF camera 93 to calculate the volume of the target cargo in the target compartment.

[0224] The router 92 is installed inside the target compartment and is used to transmit the trigger signal. The router is a 4G wireless router. The router can be installed at the top position near the compartment door and in the center.

[0225] The TOF camera 93 is used to collect the current depth map data of the target compartment in the current state and receive the trigger signal through the router 92, and execute the volume measurement method of the cargo in the compartment in the above embodiment based on the trigger signal, and output the volume of the target cargo in the target compartment;

[0226] The server 94 is used to receive the volume of the target cargo in the target compartment output by the TOF camera 93 through the router 92 .

[0227] Specifically, the mobile terminal sends a trigger signal for calculating the cargo in the target compartment, and the router forwards the signal to the TOF camera. The TOF camera returns the calculation result according to the corresponding trigger signal and sends it to the server via the router.

[0228] like Fig.10 FIG. 1 is a schematic diagram of a method for measuring the volume of cargo in a carriage according to an embodiment of the present disclosure. Fig.10In the process, the mobile terminal is the execution subject. When the mobile terminal is turned on, it first checks whether there are calibration files and posture correction files. If not, it will prompt "empty vehicle calibration". When the "empty vehicle calibration" mode is selected, N frames of data will be continuously acquired, and the calibration file and posture correction file of the empty vehicle will be calculated and output and saved locally on the device. When the above files exist, the volume calculation mode can be selected, a total of K2 frames of data will be acquired, and target object detection and volume calculation will be performed. If the current data is judged not to belong to a static scene (there is interference from dynamic objects), it will prompt "recalculate", otherwise the volume calculation result will be output and the current data will be saved in the global storage, providing data guarantee for subsequent expansion algorithms that may need to use historical data. In addition, the volume calculation mode can be divided into a continuous trigger mode and a single trigger mode. The continuous trigger mode starts from the receipt of a trigger signal and continuously triggers the volume calculation mode, outputting the calculation results in real time until a stop trigger signal is received. This allows a better grasp of the volume changes in the target compartment. The single trigger mode starts from the receipt of a trigger signal and performs a volume calculation. This can support calculations when volume information is needed, saving power resources.

[0229] The method for measuring the volume of cargo in a carriage in the embodiment of the present disclosure is specifically implemented by relying on an Internet of Things system, as follows: Fig.11 As shown, the process includes the following:

[0230] First, a 4G box and a voltage-stabilized power supply are integrated into the cab. The voltage-stabilized power supply is used to power other devices in the system, and the 4G box is used to build a local area network and transmit data.

[0231] Secondly, an external antenna is installed outside the cab for communication between local devices and cloud license servers, and between local devices and third-party business platforms.

[0232] Again, a TOF camera is installed at the rear of the car, connected to the 4G box via a network cable, and an http server is built based on the 4G box (router). The web page can be used to operate the device to perform "empty car calibration" and "volume measurement".

[0233] Again, the business personnel hold a mobile device (a mobile phone or PC with Internet access) connected to the Internet of Things to execute the business content described in 3.

[0234] Finally, a license server is built in the cloud to verify whether the local device (TOF camera) has the authority to perform business operations.

[0235] Fig.11 The usage process of the Internet of Things system in is as follows:

[0236] 1. Connect the 4G box, TOF camera and regulated power supply and turn on the power;

[0237] 2. After the device indicator light is on normally, use your mobile device to connect to the wireless network of the 4G box, open the web page and log in to the specified http server;

[0238] 3. The device is connected to the Internet through the 4G box and sends the device ID to the license server. If the license is valid, the permission is issued and the following contents 4-5 are executed; otherwise, the use of the above 3 business contents is prohibited and a prompt is given on the web page;

[0239] 4. Click the "Empty Car Calibration" button on the mobile device. After the task is completed, the success or failure status and empty car information will be displayed on the web page;

[0240] 5. After the empty truck is calibrated successfully, the business personnel stacks the goods and clicks "Volume Measurement" to calculate the volume. The calculation result is transmitted to the third-party business platform via the network or displayed on the local web page.

[0241] In addition, the business functions and business scenarios of the business system in the IoT system framework can be replaced. For example, the carriage scenario is replaced with a courier sorting conveyor belt, and the business functions are replaced with courier box size measurement and volume measurement; the carriage scenario is replaced with a gate channel, and the business functions are replaced with pedestrian counting, etc.

[0242] In this embodiment, a device for measuring the volume of goods in a compartment is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0243] This embodiment provides a device for measuring the volume of cargo in a compartment. Fig.12 As shown, including:

[0244] A three-dimensional data acquisition module 121 is used to acquire three-dimensional point cloud data of the target carriage in an empty state;

[0245] A deflection angle calculation module 122 calculates the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system;

[0246] A three-dimensional data correction module 123, used for correcting three-dimensional point cloud data based on a deflection angle of a coordinate axis;

[0247] The current data acquisition module 124 is used to acquire the current depth map data of the target compartment in the current state;

[0248] A static scene detection module 125, for detecting whether the target compartment in the current state belongs to a static scene based on the current depth map data of the target compartment in the current state;

[0249] The target compartment calculation module 126 is used to calculate the volume of the target cargo in the target compartment based on the corrected three-dimensional point cloud data and the current depth map data if the target compartment is a static scene in the current state.

[0250] In some optional embodiments, the target car calculation module 126 includes:

[0251] The data sampling and processing submodule is used to downsample the three-dimensional point cloud data;

[0252] The coordinate data sorting submodule is used to sort the downsampled 3D point cloud data according to the X-axis coordinate value in the world coordinate system;

[0253] The side wall data acquisition submodule acquires the three-dimensional point cloud data of the mutually opposing side walls of the target compartment from the sorted three-dimensional point cloud data;

[0254] The side wall data processing submodule is used to downsample the side wall three-dimensional point cloud data in an equidistant manner;

[0255] A derivative data acquisition submodule, for acquiring derivative data between adjacent point clouds based on the sidewall three-dimensional point cloud data that has been downsampled in an equidistant manner;

[0256] The derivative data calculation submodule is used to calculate the average value of the derivative data between adjacent point clouds;

[0257] The derivative data extraction submodule is used to extract the maximum and minimum values ​​of the X-axis coordinate values ​​of the point cloud group corresponding to the average value from the derivative data between adjacent point clouds;

[0258] A target data determination submodule is used to extract first target point cloud data between the minimum value and the maximum value from the downsampled three-dimensional point cloud data, and use the three-dimensional point cloud data that does not meet the minimum value and the maximum value as the second target point cloud data;

[0259] The correction data calculation submodule is used to calculate the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system through the RANSAC algorithm based on the first target point cloud data and the second target point cloud data.

[0260] In some optional implementations, the correction data calculation submodule includes:

[0261] a first correction angle calculation unit, used for acquiring two-dimensional point cloud data between the Y axis and the Z axis from the first target point cloud data, so as to generate a first fitting straight line through a RANSAC algorithm, and based on the slope value of the first fitting straight line, calculating an X-axis deflection angle of the three-dimensional point cloud data in the world coordinate system, wherein the X-axis deflection angle is used for correcting the deflection of the three-dimensional point cloud data of the target carriage relative to the X axis;

[0262] a second correction angle calculation unit, used for acquiring two-dimensional point cloud data between the X-axis and the Z-axis from the second target point cloud data, so as to generate a second fitting straight line and a third fitting straight line through a RANSAC algorithm, and calculating a Y-axis deflection angle of the three-dimensional point cloud data in a world coordinate system based on a slope value of the second fitting straight line and a slope value of the third fitting straight line, wherein the Y-axis deflection angle is used for correcting a deflection of the three-dimensional point cloud data of the target carriage relative to the Y-axis;

[0263] The third correction angle calculation unit is used to obtain the two-dimensional point cloud data between the X-axis and the Y-axis from the first target point cloud data, so as to generate a fourth fitting straight line through the RANSAC algorithm, and calculate the Z-axis deflection angle of the three-dimensional point cloud data in the world coordinate system based on the slope value of the fourth fitting straight line. The Z-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target car relative to the Z-axis.

[0264] In some optional embodiments, the target cargo detection module includes:

[0265] A data acquisition submodule is used to obtain the actual number of frames of the current depth map data of the target compartment in the current state;

[0266] A data detection submodule, used to obtain data of a first preset number of frames for detecting a static number of frames from current depth map data of an actual number of frames;

[0267] A filtering processing submodule, used for performing median filtering on the current depth map data of the actual frame number according to the second preset frame number;

[0268] A data conversion submodule, used to convert the current depth map data after median filtering into three-dimensional point cloud data;

[0269] The data correction submodule is used to correct the three-dimensional point cloud data corresponding to the current depth map data using the coordinate axis deflection angle;

[0270] A data projection submodule is used to project the three-dimensional point cloud data corresponding to the current depth map data in the cargo depth direction to obtain the current voxelized depth map data;

[0271] A differential output submodule, used to perform differential processing on the current voxelized depth map data according to a third preset frame number, and output a differential processing result of a target frame number generated based on the actual frame number and the first preset frame number;

[0272] The data judgment submodule is used to judge whether each frame of data in the difference processing result is static frame data;

[0273] A second calculation submodule is used for obtaining the total number of static frames of the static frame data from the difference processing result, and calculating the ratio of the total number of static frames to the target number of frames;

[0274] The static determination submodule is used to determine that if the ratio is greater than a first preset threshold, the target car belongs to a static scene in the current state.

[0275] In some optional implementations, the data judgment submodule includes:

[0276] A first acquisition unit, configured to acquire current frame data of current voxelized depth map data from a differential processing result;

[0277] The second acquisition unit is used to acquire the voxel difference between two frames of data formed by a frame of data adjacent to the current frame of data at the same position, as well as the maximum number of voxels in the X-axis direction and the maximum number of voxels in the Y-axis direction;

[0278] A data calculation unit, used for calculating an objective function value for detecting static frame data based on the voxel difference, the maximum number of voxels in the X-axis direction, and the maximum number of voxels in the Y-axis direction;

[0279] The static determination unit is used to determine that the current frame data in the differential processing result is static frame data if the objective function value is greater than a second preset threshold.

[0280] In some optional implementations, the target compartment calculation module includes:

[0281] The depth data acquisition submodule is used to obtain the average depth value of the cargo in the cargo depth direction of the target compartment;

[0282] A carriage data acquisition submodule is used to obtain a width voxel size value and a height voxel size value of a target carriage;

[0283] The target cargo calculation submodule is used to calculate the volume of the target cargo in the target compartment based on the average depth value, width voxel size value and height voxel size value of the cargo.

[0284] In some optional implementations, the target compartment calculation module includes:

[0285] A data acquisition module, used to obtain a threshold value of the inner wall thickness of the target compartment and an average depth value of each voxel of the compartment in the cargo depth direction of the target compartment;

[0286] A data correction module is used to obtain the Z-axis coordinate value from the corrected three-dimensional point cloud data;

[0287] A data acquisition module, used to obtain an estimated value of the inner wall of the target compartment based on the Z-axis coordinate value;

[0288] The data judgment module is used to judge whether a part of the voxel area of ​​the target cargo is blocked by the blocking object based on the average depth value of each voxel in the carriage, the estimated value of the inner wall and the inner wall thickness threshold.

[0289] In some optional embodiments, the system further includes: a data filling module, which is used to fill in a blocked area with an adjacent area of ​​the blocked area if a part of the voxel area of ​​the target cargo is blocked by a blocking object.

[0290] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0291] The volume measuring device for cargo in the carriage in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0292] An embodiment of the present invention further provides a computer device having the above-mentioned device for measuring the volume of cargo in a carriage.

[0293] See also Fig.13 , Fig.13 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.13 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.13 A processor 10 is taken as an example.

[0294] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0295] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0296] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0297] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0298] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig.13 The example of connecting through bus is taken in the following.

[0299] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0300] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0301] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0302] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for measuring the volume of cargo in a carriage, characterized in that: The method comprises: Obtain the three-dimensional point cloud data of the target carriage in an empty state; Calculating the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system; Correcting the three-dimensional point cloud data based on the coordinate axis deflection angle; Obtain the current depth map data of the target car in the current state; Based on the current depth map data of the target compartment in the current state, detecting whether the target compartment belongs to a static scene in the current state; If the target compartment belongs to a static scene in the current state, the volume of the target cargo in the target compartment is calculated based on the corrected three-dimensional point cloud data and the current depth map data.

2. The method according to claim 1, characterized in that Calculating the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system includes: Performing downsampling processing on the three-dimensional point cloud data; Sorting the downsampled three-dimensional point cloud data according to the X-axis coordinate values ​​in the world coordinate system; Acquire the three-dimensional point cloud data of the mutually opposing side walls of the target compartment from the sorted three-dimensional point cloud data; Downsampling the side wall three-dimensional point cloud data in an equidistant manner; Based on the sidewall three-dimensional point cloud data that has been downsampled in an equidistant manner, obtaining derivative data between adjacent point clouds; Calculating the average value of the derivative data between the adjacent point clouds; Extracting the maximum and minimum values ​​of the X-axis coordinate values ​​of the point cloud group corresponding to the average value from the derivative data between the adjacent point clouds; Extracting first target point cloud data between the minimum value and the maximum value from the three-dimensional point cloud data after the downsampling process, and using the three-dimensional point cloud data that does not satisfy the range between the minimum value and the maximum value as second target point cloud data; Based on the first target point cloud data and the second target point cloud data, a coordinate axis deflection angle of the three-dimensional point cloud data in a world coordinate system is calculated by a RANSAC algorithm.

3. The method according to claim 2, characterized in that Based on the first target point cloud data and the second target point cloud data, calculating the coordinate axis deflection angle of the three-dimensional point cloud data in the world coordinate system by using a RANSAC algorithm, including: Acquire two-dimensional point cloud data between the Y axis and the Z axis from the first target point cloud data to generate a first fitting straight line through the RANSAC algorithm, and calculate an X-axis deflection angle of the three-dimensional point cloud data in a world coordinate system based on the slope value of the first fitting straight line, wherein the X-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target car relative to the X axis; Acquire the two-dimensional point cloud data between the X-axis and the Z-axis from the second target point cloud data, so as to generate a second fitting straight line and a third fitting straight line through the RANSAC algorithm, and calculate the Y-axis deflection angle of the three-dimensional point cloud data in the world coordinate system based on the slope value of the second fitting straight line and the slope value of the third fitting straight line, wherein the Y-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target car relative to the Y-axis; The two-dimensional point cloud data between the X-axis and the Y-axis is obtained from the first target point cloud data to generate a fourth fitting straight line through the RANSAC algorithm, and based on the slope value of the fourth fitting straight line, the Z-axis deflection angle of the three-dimensional point cloud data in the world coordinate system is calculated, and the Z-axis deflection angle is used to correct the deflection of the three-dimensional point cloud data of the target car relative to the Z-axis.

4. The method according to claim 1, characterized in that Based on the current depth map data of the target compartment in the current state, detecting whether the target compartment belongs to a static scene in the current state includes: Get the actual number of frames of the current depth map data of the target car in the current state; Acquire data of a first preset number of frames for detecting a static number of frames from current depth map data of an actual number of frames; Performing median filtering on the current depth map data of the actual frame number according to a second preset frame number; Convert the current depth map data after median filtering into three-dimensional point cloud data; Correcting the three-dimensional point cloud data corresponding to the current depth map data using the coordinate axis deflection angle; Projecting the three-dimensional point cloud data corresponding to the current depth map data in the cargo depth direction to obtain current voxelized depth map data; Performing differential processing on the current voxelized depth map data according to a third preset frame number, and outputting a differential processing result of a target frame number generated based on the actual frame number and the first preset frame number; Determining whether each frame of data in the differential processing result is static frame data; If yes, obtaining the total number of static frames of the static frame data from the difference processing result, and calculating the ratio of the total number of static frames to the target number of frames; If the ratio is greater than a first preset threshold, the target vehicle compartment belongs to a static scene in the current state.

5. The method according to claim 4, characterized in that Determining whether each frame of data in the differential processing result is static frame data includes: Obtaining current frame data of the current voxelized depth map data from the differential processing result; Obtaining the voxel difference between two frames of data consisting of a frame of data adjacent to the current frame of data at the same position and the maximum number of voxels in the X-axis direction and the maximum number of voxels in the Y-axis direction; Calculating an objective function value for detecting static frame data based on the voxel difference, the maximum number of voxels in the X-axis direction, and the maximum number of voxels in the Y-axis direction; If the objective function value is greater than a second preset threshold, the current frame data in the differential processing result is static frame data.

6. The method according to claim 1, characterized in that Calculating the volume of the target cargo in the target compartment, including: Obtain the average cargo depth value of the target compartment in the cargo depth direction; Obtaining a width voxel size value and a height voxel size value of the target carriage; The volume of the target cargo in the target compartment is calculated based on the average depth value of the cargo, the width voxel size value, and the height voxel size value.

7. The method according to claim 1, characterized in that In the process of calculating the target volume of the target cargo in the current state, it further includes: determining whether the target cargo is blocked by a blocking object, including: Obtaining a threshold value of the inner wall thickness of the target compartment and an average depth value of each voxel of the target compartment in a cargo depth direction; Obtaining a Z-axis coordinate value from the corrected three-dimensional point cloud data; Based on the Z-axis coordinate value, obtaining an estimated value of the inner wall of the target compartment; Based on the average depth value of each voxel of the carriage, the inner wall estimation value and the inner wall thickness threshold, it is determined whether a partial voxel area of ​​the target cargo is blocked by a blocking object.

8. The method according to claim 7, characterized in that If a part of the voxel area of ​​the target cargo is blocked by a blocking object, the blocked area is filled by an adjacent area of ​​the blocked area.

9. A device for measuring the volume of cargo in a carriage, characterized in that: The device comprises: A three-dimensional data acquisition module is used to acquire three-dimensional point cloud data of the target carriage in an empty state; A deflection angle calculation module calculates the deflection angle of the coordinate axis of the three-dimensional point cloud data in the world coordinate system; A three-dimensional data correction module, used for correcting the three-dimensional point cloud depth map data based on the deflection angle of the coordinate axis; A current data acquisition module is used to obtain the current depth map data of the target compartment in the current state; A target cargo detection module, configured to detect whether the target carriage belongs to a static scene in the current state based on the current depth map data of the target carriage in the current state; The static scene detection module is used to calculate the volume of the target cargo in the target compartment based on the corrected three-dimensional point cloud data and the current depth map data if the target compartment belongs to a static scene in the current state.

10. A system for measuring the volume of cargo in a carriage, characterized in that: include: A mobile terminal, used for sending a trigger signal; A router, installed inside the target carriage, for transmitting the trigger signal; A TOF camera installed inside the target compartment, used to collect current depth map data of the target compartment in a current state and receive the trigger signal through the router, and execute the volume measurement method of the cargo in the compartment according to any one of claims 1 to 8 based on the trigger signal, and output the volume of the target cargo in the target compartment; The server receives the volume of the target cargo in the target compartment output by the TOF camera through the router.

11. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for measuring the volume of cargo in a vehicle compartment as claimed in any one of claims 1 to 8 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for measuring the volume of cargo in a vehicle compartment according to any one of claims 1 to 8.

13. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method for measuring the volume of cargo in a vehicle compartment according to any one of claims 1 to 8.

Images