Method and device for detecting remaining cargo volume

Abstract

The embodiments of the present disclosure disclose a method and device for detecting the remaining cargo volume. The specific implementation of the method includes: obtaining point cloud data in the storage box; selecting a reference point from the point cloud data, and performing the following point cloud triangulation steps: searching for a predetermined number of candidate points closest to the reference point; projecting the predetermined number of candidate points onto the projection plane of the reference point to obtain a projection point set; connecting the projection points in the projection point set to construct a non-overlapping triangle set; selecting points near the reference point from points that have never been used as reference points as reference points, and continuing to perform the above-mentioned point cloud triangulation steps; finally, for each triangle in the triangle set, constructing a tetrahedron based on the coordinates of the three points of the triangle before projection and the position of the laser radar, and calculating the volume of the tetrahedron; summing the volumes of each tetrahedron as the remaining cargo volume. This implementation method realizes the rapid and accurate detection of the cargo volume of the storage box.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly, to a method and apparatus for detecting remaining cargo volume. Background Art

[0002] With the rapid development of science and technology, industrial production processes are moving towards automation, intelligence, and speed. Ports play a vital role in connecting inland economic hinterlands and promoting the development of maritime transportation. Their level of intelligence is directly related to the economic development of the entire region. The key to automated production between ports is the high-precision inspection of bulk cargo train cars entering and leaving the port. LiDAR detection technology has attracted widespread attention as a high-precision detection method. The development of this technology in the field of surveying and mapping has provided new ideas for obtaining surface information of measured objects and achieving high-precision detection. By acquiring and processing the point cloud data of the measured object, the surface morphology information of the measured object can be obtained, and then a series of operations such as geometric parameter extraction and volume calculation processing can be completed according to actual engineering needs. Summary of the Invention

[0003] Embodiments of the present disclosure provide a method and apparatus for detecting remaining cargo volume.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for detecting the remaining cargo volume, wherein a laser radar is installed at the rear of a storage box and completely scans the entire point cloud inside the storage box toward the inside of the storage box, the method comprising: obtaining point cloud data inside the storage box; selecting a reference point from the point cloud data, and performing the following point cloud triangulation steps: searching for a predetermined number of candidate points closest to the reference point; projecting the predetermined number of candidate points onto a projection plane of the reference point to obtain a projection point set; connecting the projection points in the projection point set to construct a non-overlapping triangle set; selecting points near the reference point from points that have never been used as reference points as reference points, and repeating the above point cloud triangulation steps until all points are traversed, wherein the projection direction is the direction in which the position of the laser radar points to the reference point, and the projection plane is a plane passing through the reference point and perpendicular to the projection direction; for each triangle in the triangle set, constructing a tetrahedron according to the coordinates of the three points of the triangle before projection and the position of the laser radar, and calculating the volume of the tetrahedron; and summing the volumes of each tetrahedron as the remaining cargo volume.

[0005] In some embodiments, obtaining point cloud data within a storage box includes: obtaining initial point cloud data within the storage box; dividing the initial point cloud data into grids according to the angular resolution of the lidar; calculating the coordinates of the center point of the grid based on the average value of the coordinates of the points within each grid; and generating point cloud data based on the center point of each grid.

[0006] In some embodiments, connecting the projection points in the projection point set to construct a non-overlapping triangle set includes: connecting the projection points in the projection point set and performing a visibility check to retain visible points; and triangulating the visible points to construct a non-overlapping triangle set.

[0007] In some embodiments, before connecting the projection points in the projection point set to construct a non-overlapping triangle set, the method further includes: filtering out the projection points in the projection point set whose distance from the reference point exceeds a predetermined threshold, wherein, in the projection point set, the distance obtained by multiplying the distance to the nearest point to the reference point by a predetermined parameter is used as the predetermined threshold, and the predetermined threshold is greater than the predetermined side length threshold.

[0008] In some embodiments, the method further includes: establishing a coordinate system on the projection plane of the reference point with the reference point as the origin, and obtaining the coordinates of each projection point; calculating the angle between the line segment formed by each projection point and the origin and the horizontal axis based on the coordinates of each projection point, and calculating the angle of each angle in the triangle based on the angle.

[0009] In some embodiments, connecting the projection points in the projection point set to construct a non-overlapping triangle set includes: connecting the projection points in the projection point set based on a greedy projection triangulation algorithm to construct non-overlapping candidate triangles; if the candidate triangle meets the angle constraint condition and the length constraint condition, the candidate triangle is added to the triangle set.

[0010] In some embodiments, the angle restriction condition includes that the angle of any angle is less than a predetermined maximum angle and greater than a predetermined minimum angle, and the length restriction condition includes that the length of any side is less than a predetermined side length threshold, wherein the predetermined side length threshold is positively correlated with the distance from the reference point to the laser radar and the angular resolution of the laser radar.

[0011] In a second aspect, an embodiment of the present disclosure provides a device for detecting the remaining cargo volume, wherein a laser radar is installed at the rear of a storage box and scans the entire point cloud inside the storage box toward the inside of the storage box. The device includes: an acquisition unit configured to acquire point cloud data inside the storage box; a triangulation unit configured to select a reference point from the point cloud data and perform the following point cloud triangulation steps: searching for a predetermined number of candidate points closest to the reference point; projecting the predetermined number of candidate points onto the projection plane of the reference point to obtain a set of projection points; and connecting the projection points in the set of projection points to construct a triangle. a set of non-overlapping triangles is generated; a point near the reference point is selected from points that have never been used as a reference point, and the above point cloud triangulation steps are repeated until all points are traversed, wherein the projection direction is the direction in which the position of the laser radar points to the reference point, and the projection plane is the plane passing through the reference point and perpendicular to the projection direction; a construction unit is configured to, for each triangle in the triangle set, construct a tetrahedron according to the coordinates of the three points of the triangle before projection and the position of the laser radar, and calculate the volume of the tetrahedron; a calculation unit is configured to sum the volumes of each tetrahedron as the remaining cargo volume.

[0012] In some embodiments, the acquisition unit is further configured to: acquire initial point cloud data within the storage box; divide the initial point cloud data into grids according to the angular resolution of the lidar; calculate the coordinates of the center point of the grid based on the average value of the coordinates of the points within each grid; and generate point cloud data based on the center point of each grid.

[0013] In some embodiments, the triangulation unit is further configured to: connect the projection points in the projection point set and perform visibility check to retain visible points; and triangulate the visible points to construct a non-overlapping triangle set.

[0014] In some embodiments, the triangulation unit is further configured to: before connecting the projection points in the projection point set to construct a non-overlapping triangle set, filter out the projection points in the projection point set whose distance from the reference point exceeds a predetermined threshold, wherein, in the projection point set, the distance obtained by multiplying the distance to the nearest point to the reference point by a predetermined parameter is used as the predetermined threshold, and the predetermined threshold is greater than the predetermined side length threshold.

[0015] In some embodiments, the triangulation unit is further configured to: establish a coordinate system on the projection plane of the reference point with the reference point as the origin, and obtain the coordinates of each projection point; calculate the angle between the line segment formed by each projection point and the origin and the horizontal axis according to the coordinates of each projection point, and calculate the angle of each corner in the triangle based on the angle.

[0016] In some embodiments, the triangulation unit is further configured to: connect the projection points in the projection point set based on a greedy projection triangulation algorithm to construct non-overlapping candidate triangles; if the candidate triangle meets the angle constraint and the length constraint, the candidate triangle is added to the triangle set.

[0017] In some embodiments, the angle restriction condition includes that the angle of any angle is less than a predetermined maximum angle and greater than a predetermined minimum angle, and the length restriction condition includes that the length of any side is less than a predetermined side length threshold, wherein the predetermined side length threshold is positively correlated with the distance from the reference point to the laser radar and the angular resolution of the laser radar.

[0018] In a third aspect, an embodiment of the present disclosure provides an electronic device for detecting remaining cargo volume, comprising: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method as described in any one of the first aspects.

[0019] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method as described in any one of the first aspects is implemented.

[0020] The embodiments of the present disclosure provide a method and apparatus for detecting the remaining cargo volume. A laser radar device is installed at the rear of a storage box to collect a point cloud image of the storage box after the cargo is loaded. The point cloud image can be used to calculate the volume occupied by the cargo in the current storage box without affecting the workers loading and unloading the cargo. The problem of poor triangulation results due to the sparsity of point clouds at long distances. According to the principle of laser radar imaging, the direction of laser emission is selected as the normal direction of the point cloud, and the point cloud near the reference point cloud is projected onto a plane perpendicular to the normal vector, and triangulation is performed on this plane. This can solve the problem that the sparsity of the point cloud on the side of the deep storage box caused by the angular resolution makes it impossible to form triangles better, which ultimately leads to errors in volume calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following drawings:

[0022] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

[0023] Figure 2 is a flow chart of an embodiment of a method for detecting remaining cargo volume according to the present disclosure;

[0024] Figure 3a 、 3b is a schematic diagram of point cloud data of a method for detecting remaining cargo volume according to the present disclosure;

[0025] Figure 4 is a flow chart of another embodiment of a method for detecting remaining cargo volume according to the present disclosure;

[0026] Figure 5 is a schematic structural diagram of an embodiment of a device for detecting remaining cargo volume according to the present disclosure;

[0027] Figure 6 It is a schematic diagram of the structure of a computer system of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] The present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0029] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] Figure 1 An exemplary system architecture is shown to which an embodiment of the method for detecting a remaining cargo volume or the apparatus for detecting a remaining cargo volume of the present disclosure can be applied.

[0031] like Figure 1 As shown, the system architecture includes lidar, processing chips and servers.

[0032] As an example, during the implementation of this solution, the lidar can be, for example, Ouster's 128-line lidar OS0-128, which is used to collect point cloud data; the processing chip can be, for example, NXP's i.MX8M chip, which is used to upload the point cloud collected by the OS0-128 to the cloud and process the data in the cloud. The entire solution operates as follows:

[0033] 1. The OS0-128 is installed at the rear of the storage box, with the radar facing the interior. Because the OS0-128 has a 90-degree vertical field of view and a 360-degree horizontal field of view, it can fully scan the entire point cloud inside the storage box. During operation, the radar continuously scans the point cloud inside the storage box.

[0034] 2. Connect the i.MX8M embedded board to OS0-128 using the network port to obtain the point cloud data generated by OS0-128, and then package the point cloud data and upload it to the server.

[0035] 3. The server processes the transmitted data, that is, it uses the triangulation method to reconstruct the point cloud model and calculates the volume of the current point cloud after reconstruction.

[0036] LiDAR is used to measure the volume inside the storage box. The general processing method is to triangulate the collected point cloud to form a mesh surface, and then measure the volume inside the storage box. After the point cloud is triangulated, a tetrahedron is formed with the installation position of the LiDAR. The tetrahedron formed by each triangle on the surface of the triangulated model and the point of the LiDAR installation position is calculated. The sum of each tetrahedron is the volume of the remaining space in the current storage box.

[0037] It should be noted that a server can be either hardware or software. When a server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When a server is software, it can be implemented as multiple software programs or software modules (for example, multiple software programs or software modules used to provide distributed services), or as a single software program or software module. This is not specifically limited here. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0038] It should be noted that the method for detecting the remaining cargo volume provided in the embodiments of the present disclosure is generally executed by a server. Accordingly, the device for detecting the remaining cargo volume is generally provided in the server.

[0039] It should be understood that Figure 1 The number of LiDARs, i.MX8M chips, and servers in the examples is for illustrative purposes only. Any number of LiDARs, i.MX8M chips, and servers may be used as needed. Furthermore, the models of LiDARs and chips are not limited to those listed in the examples of this application.

[0040] Continue to refer Figure 2 , shows a process 200 of an embodiment of a method for detecting remaining cargo volume according to the present disclosure. The method for detecting remaining cargo volume includes the following steps:

[0041] Step 201: Acquire point cloud data in the storage box.

[0042] In this embodiment, the execution subject of the method for detecting the remaining cargo volume (eg Figure 1The server shown in the figure can obtain point cloud data in the storage box from the laser radar used by the user to detect the remaining cargo volume through a wired connection or a wireless connection. This application can directly use the original point cloud data.

[0043] In some optional implementations of this embodiment, point cloud data after mean filtering can also be used. The mean filtering process is as follows: obtain the initial point cloud data in the storage box; divide the initial point cloud data into grids according to the angular resolution of the laser radar; calculate the coordinates of the center point of the grid based on the average value of the coordinates of the points in each grid; generate point cloud data based on the center point of each grid. The initial point cloud is divided into grids according to the angular resolution of the laser radar, which is divided into horizontal division and vertical division. The center of gravity of the current point cloud is calculated based on the point cloud coordinates in each angular grid, and mean filtering is performed. Mean filtering of the input point cloud according to the angular resolution of the laser radar can solve the problem of complex connection caused by the laser radar scanning multiple point clouds at the same angle and the subsequent triangulation of the local point cloud projection, thereby simplifying the calculation process and shortening the detection time.

[0044] Step 202: Select a reference point from the point cloud data.

[0045] In this embodiment, an arbitrary initial point is selected as the reference point R.

[0046] Step 203: Search for a predetermined number of candidate points closest to the reference point.

[0047] In this embodiment, the distance here is the distance in three-dimensional space calculated based on the actual coordinates of the point cloud. The predetermined number k can be an integer such as 100. The predetermined number of points closest to the reference point in the point cloud data are used as candidate points.

[0048] Step 204 : Project a predetermined number of candidate points onto the projection plane of the reference point to obtain a set of projection points.

[0049] In this embodiment, the k nearest points on point R that meet the distance requirements are projected. The projection direction is the direction in which the laser radar points to point R. All points are projected onto a plane perpendicular to this direction to obtain the plane coordinates of each nearest point, as shown in the following example: Figure 3a As shown. Figure 1 In practice, this installation method can lead to poor triangulation due to the sparseness of distant point clouds. Based on the principles of lidar imaging, the laser emission direction is selected as the point cloud normal. Point clouds near the reference point cloud are projected onto a plane perpendicular to this normal vector, and greedy projection triangulation is performed on this plane. This solves the problem of greedy point cloud triangulation failing to form triangles due to the sparsity of the point cloud on the side of the storage box due to angular resolution.

[0050] Projection is performed using the direction between the current reference point and the origin as the perpendicular direction to the projection plane. Due to the fixed shooting characteristics of LiDAR and the angular resolution scanning characteristics of mechanical radar, selecting this projection plane eliminates the need for normal vector calculations, and the density of the projected point cloud is uniform, allowing for better triangulation. After selecting this projection plane, the direction of each local point cloud from the radar origin is very close to the direction from the reference point to the origin. This is equivalent to only performing depth compression on the points on the projected plane in the projection direction, preventing errors in connecting the projected point cloud.

[0051] In some optional implementations of this embodiment, the method further includes: establishing a coordinate system on the projection plane of the reference point with the reference point as the origin, and obtaining the coordinates of each projection point; and calculating the angle between the line segment formed by each projection point and the origin and the horizontal axis based on the coordinates of each projection point, as the angle of each projection point. The calculated angle of each projection point is used for subsequent angle constraint determination. In this way, when calculating the angle of a triangle, the angles of the two projection points can be directly subtracted. This improves calculation speed and shortens detection time.

[0052] Step 205: Connect the projection points in the projection point set to construct a non-overlapping triangle set.

[0053] In this embodiment, the existing 3D point reconstruction technology can be used to connect the projection points in the projection point set to construct a non-overlapping triangle set, such as Figure 3b The specific steps are as follows:

[0054] (1) Initialize a triangle:

[0055] 1. Calculate the center of gravity of all points;

[0056] 2. Find the point closest to the center of gravity and set it as ptn0;

[0057] 3. In the area of ptn0, calculate the farthest distance and find a point 1 / 3len away from it (but still greater than the minimum distance) as ptn1;

[0058] 4. Find a point ptn2 in the domain such that the radius of the circumscribed circle they form is the smallest;

[0059] These three points form a triangle and are marked as boundary points.

[0060] (2) Constructing Mesh:

[0061] 1. First find all the boundary half edges from the Mesh;

[0062] 2. For each half-edge: If the endpoint can be extended, do the following; otherwise, do the next one:

[0063] 1) First, expand from the upper and lower adjacent edges of this edge. If the expansion is successful, the newly added edge is also triangulated and reconstructed; and the new half-edge constructed by the reconstruction is added to the half-edge set.

[0064] 2) If the extension cannot be performed on the upper and lower edges, find all unused neighboring points of the two endpoints;

[0065] 3) For each adjacent point above, form the third endpoint of the triangle, but it must meet several requirements:

[0066] A. The lengths of the two newly added edges must be less than the threshold;

[0067] B. The interior angles of a triangle are all within a certain range: (20 to 100 degrees)

[0068] C. The dihedral angle formed must be greater than the threshold.

[0069] D. The radius of the circumscribed circle of the formed triangular facet is the smallest.

[0070] E. The new surface cannot contain boundary points or points that cannot be expanded.

[0071] 4) After finding the points that meet the reconstruction requirements, reconstruct a triangle patch and reconstruct the two newly added edges at the same time (reconstruct only the front and back edges). Finally, reconstruct the two newly added edges and put them into the half-edge queue.

[0072] 5) If both points of a half-edge cannot be reconstructed, delete it until there are no edges in the half-edge queue.

[0073] (3) Delete and rebuild overlapping triangles:

[0074] 1. First find the boundary points but not isolated points;

[0075] 2. For each such point, find the boundary faces of its area; if the number of boundary faces is less than 2, there is no overlap; if the angle between any of the faces is less than 30°, there is overlap.

[0076] 3. If there is overlap, delete these faces.

[0077] (IV) Update the usage of each point

[0078] Traverse each point in the Mesh and assign corresponding flags to the corresponding points: Boundary points: PT_BOUND Isolated points: PT_NOT_USED Points inside the surface: PT_IN_MESH Other points: PT_TRI_USED

[0079] 5. Fill the holes

[0080] 1. Find the holes:

[0081] A. Find all the boundary edges and put them in a queue;

[0082] B. Start from one of the half edges and find the half edge that ends at the starting point of this edge. Continue searching until you find the half edge that starts at the end point of the first half edge. You will find a hole, and the rest are similar.

[0083] C. Determine the validity of a hole: 1. If there are only three edges, it may be an isolated surface

[0084] 2. If there are 4 edges, they may be two isolated faces.

[0085] 3. If the number of edges is between 2 and 8, it will not work even if there are common vertices.

[0086] In the above three situations, you need to delete the face.

[0087] 4. Fill the holes:

[0088] 1).*If the number of edges is less than 20, delete the surrounding faces and start over; if it is a general hole, fill it, otherwise do not fill it.

[0089] 2) After the above operations, extract the holes twice and fill them.

[0090] 5. Update the usage information of the point again, the same as above

[0091] Step 206: Select a point near the reference point from the points that have never been used as reference points as a reference point, and repeat steps 203-206 until all points are traversed.

[0092] In this embodiment, points near the reference point are added to the sequence and a point is randomly selected as the reference point for the next cycle, and the above steps are repeated until all points are completed.

[0093] Step 207: For each triangle in the triangle set, construct a tetrahedron based on the coordinates of the three points of the triangle before projection and the position of the laser radar, and calculate the volume of the tetrahedron.

[0094] In this embodiment, after triangulating the point cloud, the points on the two-dimensional projection surface are matched to the points in the three-dimensional space before projection. A tetrahedron is constructed using the triangles formed by the points in the three-dimensional space and the position of the lidar. The formula for calculating the tetrahedron volume is well-known in the art and will not be described in detail here.

[0095] In step 208 , the volumes of each tetrahedron are summed up as the remaining cargo volume.

[0096] In this embodiment, the remaining space in the storage box is calculated by summing the volumes of each tetrahedron to determine the remaining cargo volume. The cargo is arranged closely from the front to the rear of the storage box. Alternatively, if the lidar is mounted at the front of the vehicle, the boxes can be arranged closely from the rear to the front. This allows calculation of both the remaining cargo volume and the loaded volume based on the total cargo volume.

[0097] Further references Figure 4 , which shows a process 400 of another embodiment of a method for detecting remaining cargo volume. The process 400 of the method for detecting remaining cargo volume includes the following steps:

[0098] Step 401: Acquire point cloud data in the storage box.

[0099] Step 402: Select a reference point from the point cloud data.

[0100] Step 403: Search for a predetermined number of candidate points closest to the reference point.

[0101] Step 404: Project a predetermined number of candidate points onto the projection plane of the reference point to obtain a set of projection points.

[0102] Steps 401-404 are substantially the same as steps 201-204, and therefore are not described in detail.

[0103] Step 405: Filter out the projection points in the projection point set whose distances from the reference point exceed a predetermined threshold.

[0104] In this embodiment, the distance to the nearest point R multiplied by the parameter u is used to limit the distance of the k closest points. Points exceeding this distance are not included in the triangulation. This process is to adapt to the point cloud density near R, limiting points that are too far away in areas with sparse point clouds. The predetermined threshold must be greater than the predetermined edge length threshold.

[0105] Step 406: Connect the projection points in the projection point set and perform a visibility check.

[0106] In this embodiment, a line is drawn between the reference point and the nearest point and a visibility check is performed. First, the side of the triangle to be formed in the projection plane is checked for visibility. That is, if the side of the triangle that has been formed intersects with the side of the triangle to be formed, the formation of the triangle is abandoned. Figure 3b The points connected to point R by the dotted line will be blocked by the edges of the triangle and will be invisible.

[0107] Step 407: Determine the angle threshold and side length limit of the triangle to be formed.

[0108] In this embodiment, the angle restriction condition includes that the angle of any angle is less than a predetermined maximum angle and greater than a predetermined minimum angle, so that the angle will not be very small or very large, and it is easier to construct a tetrahedron whose volume is easy to calculate. The points that have been verified for visibility are triangulated and it is determined that the angle of the formed triangle is between the maximum angle and the minimum angle, and meets the adaptive triangle side length restriction, then it is considered that this triangle can constitute a triangulated network. In actual application, the density of point clouds at different distances is different, and the triangle side length restriction threshold needs to be adaptively processed to solve the problem that the same triangle side length restriction cannot adapt to the different point cloud densities in different areas, resulting in poor triangulation effect. By judging the distance of the point cloud and the angular resolution characteristics of the lidar, the distance to the nearest point of the point cloud on the projection surface of the previous step can be estimated, and the side length threshold of the formed triangle can be adaptively adjusted.

[0109] The length restriction condition includes: the length of any side is less than a predetermined side length threshold, wherein the predetermined side length threshold is positively correlated with the distance from the reference point to the laser radar and the angular resolution of the laser radar. For example,

[0110] L=b*μ*tan(θ / 2)

[0111] Where L is the predetermined edge length threshold, μ is the distance from the reference point to the lidar, and θ is the angular resolution of the lidar. The hyperparameter b is greater than or equal to 2 and can be set to 5 or higher.

[0112] In the greedy projection triangulation process, it is necessary to limit the side length of the triangle during triangulation to prevent the reference point and the farther point from forming a larger triangle that affects the triangulation of other points in the area. The density of the points scanned by the radar in different areas is different. The original greedy projection triangulation method will only limit the triangle side length of the fixed threshold, and cannot adapt to point clouds with different densities. This method is based on the scanning characteristics of the lidar. When the radar is installed in a fixed position, the point cloud is sparse at a distance. Ideally, assuming that the point cloud scanned by the lidar is limited by the angular resolution, the angles of the point clouds between the two are evenly spaced, then the distance between the point clouds at different distances and their nearest point clouds on the projection surface can be calculated. Assume that the angular resolution of the radar scan is θ, and the distance between the current point cloud and the radar origin is μ. Then the distance L between the current point and its nearest point is as follows:

[0113] L=2*μ*tan(θ / 2)

[0114] Therefore, points in different areas can estimate the closest distance L of surrounding points on the projection surface by calculating the above formula, and the threshold for limiting the side length of the triangle is set to ω*L, where ω is a fixed parameter, that is, the threshold for limiting the side length of the triangle is limited by a multiple of L under the projection surface, thereby realizing an adaptive side length threshold.

[0115] Adaptively adjusting the triangle edge length limit threshold based on point cloud depth can better limit triangle edge length under varying point cloud densities. Due to the fixed LiDAR shooting position and the scanning characteristics of mechanical LiDAR, point cloud density is higher in areas closer to the LiDAR and lower in areas farther away. Ideally, the distance between point clouds at different distances can be calculated to adapt the triangle edge length threshold.

[0116] In step 408 , a point near the reference point is selected from the points that have not been used as reference points as a reference point, and the above point cloud triangulation steps 403 - 408 are repeated until all points are traversed.

[0117] Step 409 : For each triangle in the triangle set, construct a tetrahedron based on the coordinates of the three points of the triangle before projection and the position of the laser radar, and calculate the volume of the tetrahedron.

[0118] In step 410 , the volumes of each tetrahedron are summed up as the remaining cargo volume.

[0119] Steps 408-410 are substantially the same as steps 206-208, and thus are not described in detail.

[0120] The quality of triangulation of point clouds is a key factor affecting the final calculation of the storage box volume. The commonly used point cloud triangulation algorithm is greedy projection triangulation. Greedy projection triangulation first calculates the normal vector of each point cloud, and projects each point based on the tangent plane perpendicular to the normal vector in the local area of each point. After the projection, the triangulation connection is completed. If the lidar is installed inside the storage box, it will affect the workers' loading and unloading of goods. If the lidar equipment is installed at the tail of the storage box and fixed, it can capture the volume of goods inside the storage box throughout the entire process without affecting the normal loading and unloading of goods in the storage box. Since the scanning method of the mechanical lidar is based on angle division, different devices have different angular resolutions. When the mechanical lidar is installed at the tail of the storage box, when scanning to the farthest point of the storage box, due to the angular resolution characteristics of the radar, the point cloud finally generated on the side of the storage box will form a shape like this Figure 1 situation. Figure 1 The right side of the middle is the installation position of the laser radar. When scanning the circle position, the radar point cloud is as follows Figure 1 As shown in the figure, in this case, when using the greedy projection triangulation algorithm to triangulate the point cloud, the triangulated mesh cannot be formed well due to the threshold judgment problem.

[0121] In greedy projection triangulation, the local point cloud is projected onto the tangent plane to form a triangulation. This is to prevent different triangles from intersecting after the point cloud is triangulated. However, unordered point clouds have errors in the calculation of normal vectors. Even if the local normal vector is correctly found, the irregularities of the point cloud will make it difficult to triangulate properly when triangulating on the projection plane. Therefore, based on the characteristics of lidar imaging, a projection plane is selected for a local point cloud with the direction from the local point cloud to the lidar radar origin (i.e., the installation location) as the normal vector. In this way, since the lidar installation location is fixed, all point clouds collected by the radar in the local area can only have one point in this normal direction. After selecting this projection plane, the direction of each local point cloud from the lidar origin is very close to the direction from the reference point to the origin. This is equivalent to the point on the projected plane only performing depth compression on the current point cloud in the projection direction. And due to the angular resolution of the lidar, the local point cloud projected onto this plane is evenly distributed. Therefore, when triangulating on this projection plane, there will be no point cloud chaos and poor triangulation. Its effect is as follows Figure 3a As shown, it can be seen that the point cloud is evenly arranged under this projection surface.

[0122] Based on the above method, greedy triangulation is applied to the volume calculation of storage boxes in special scenarios to achieve better triangulation model and volume calculation accuracy.

[0123] Further references Figure 5As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for detecting the remaining cargo volume. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0124] like Figure 5 As shown, the device 500 for detecting the remaining cargo volume of this embodiment, wherein a laser radar is installed at the rear of the storage box and completely scans the entire point cloud inside the storage box toward the inside of the storage box, and the device includes: an acquisition unit 501, a triangulation unit 502, a construction unit 503 and a calculation unit 504. Among them, the acquisition unit 501 is configured to acquire point cloud data in the storage box; the triangulation unit 502 is configured to select a reference point from the point cloud data and perform the following point cloud triangulation steps: search for a predetermined number of candidate points closest to the reference point; project the predetermined number of candidate points onto the projection plane of the reference point to obtain a projection point set; connect the projection points in the projection point set to construct a non-overlapping triangle set; select points near the reference point from points that have never been used as reference points as reference points, and repeat the above point cloud triangulation steps until all points are traversed, wherein the projection direction is the direction in which the position of the laser radar points to the reference point, and the projection plane is a plane passing through the reference point and perpendicular to the projection direction; the construction unit 503 is configured to construct a tetrahedron for each triangle in the triangle set according to the coordinates of the three points of the triangle before projection and the position of the laser radar, and calculate the volume of the tetrahedron; the calculation unit 504 is configured to sum the volume of each tetrahedron as the remaining cargo volume.

[0125] In this embodiment, the specific processing of the acquisition unit 501, the triangulation unit 502, the construction unit 503 and the calculation unit 504 of the device 500 for detecting the remaining cargo volume can be referred to. Figure 2 This corresponds to steps 201-208 in the embodiment.

[0126] In some optional implementations of this embodiment, the acquisition unit 501 is further configured to: acquire initial point cloud data within the storage box; divide the initial point cloud data into grids according to the angular resolution of the lidar; calculate the coordinates of the center point of the grid based on the average value of the coordinates of the points within each grid; and generate point cloud data based on the center point of each grid.

[0127] In some optional implementations of this embodiment, the triangulation unit 502 is further configured to: connect the projection points in the projection point set and perform visibility check to retain visible points; and triangulate the visible points to construct a non-overlapping triangle set.

[0128] In some optional implementations of this embodiment, the triangulation unit 502 is further configured to: before connecting the projection points in the projection point set to construct a non-overlapping triangle set, filter out the projection points in the projection point set whose distance from the reference point exceeds a predetermined threshold, wherein, in the projection point set, the distance obtained by multiplying the distance to the nearest point to the reference point by a predetermined parameter is used as the predetermined threshold, and the predetermined threshold is greater than the predetermined side length threshold.

[0129] In some optional implementations of this embodiment, the triangulation unit 502 is further configured to: establish a coordinate system on the projection plane of the reference point with the reference point as the origin, and obtain the coordinates of each projection point; calculate the angle between the line segment formed by each projection point and the origin and the horizontal axis based on the coordinates of each projection point, and calculate the angle of each angle in the triangle based on the angle.

[0130] In some optional implementations of this embodiment, the triangulation unit 502 is further configured to: connect the projection points in the projection point set based on a greedy projection triangulation algorithm to construct non-overlapping candidate triangles; if the candidate triangle meets the angle constraint condition and the length constraint condition, the candidate triangle is added to the triangle set.

[0131] In some optional implementations of this embodiment, the angle restriction condition includes that the angle of any angle is less than a predetermined maximum angle and greater than a predetermined minimum angle, and the length restriction condition includes: the length of any side is less than a predetermined side length threshold, wherein the predetermined side length threshold is positively correlated with the distance from the reference point to the laser radar and the angular resolution of the laser radar.

[0132] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device and a readable storage medium.

[0133] An electronic device for detecting remaining cargo volume comprises: one or more processors; a storage device having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method described in process 200 or 400.

[0134] A computer-readable medium stores a computer program thereon, wherein the computer program implements the method described in process 200 or 400 when executed by a processor.

[0135] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0136] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0137] Various components in device 600 are connected to I / O interface 605, including an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, optical disk, etc.; and a communication unit 609, such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0138] The computing unit 601 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the method for detecting the remaining cargo volume. For example, in some embodiments, the method for detecting the remaining cargo volume can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method for detecting the remaining cargo volume described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the method for detecting the remaining cargo volume via any other suitable means (e.g., via firmware).

[0139] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0143] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0144] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a server in a distributed system or a server integrated with blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server may be a server in a distributed system or a server integrated with blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0145] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0146] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for detecting remaining cargo volume, wherein: The laser radar is installed at the rear of the storage box and scans the entire point cloud inside the storage box. The method includes: Get the point cloud data in the storage box; A reference point is selected from the point cloud data, and the following point cloud triangulation steps are performed: searching for a predetermined number of candidate points closest to the reference point; projecting the predetermined number of candidate points onto a projection plane of the reference point to obtain a projection point set; connecting the projection points in the projection point set to construct a non-overlapping triangle set; selecting points near the reference point from points that have never been used as reference points as reference points, and repeating the above point cloud triangulation steps until all points are traversed, wherein the projection direction is the direction in which the laser radar position points to the reference point, and the projection plane is a plane passing through the reference point and perpendicular to the projection direction; For each triangle in the triangle set, construct a tetrahedron based on the coordinates of the three points of the triangle before projection and the position of the lidar, and calculate the volume of the tetrahedron; The remaining cargo volume is calculated by summing the volumes of each tetrahedron.

2. The method according to claim 1, wherein The step of obtaining point cloud data in the storage box includes: Obtain the initial point cloud data in the storage box; Dividing the initial point cloud data into grids according to the angular resolution of the laser radar; Calculate the coordinates of the center point of the grid based on the average of the coordinates of the points within each grid; Generate point cloud data based on the center point of each grid.

3. The method according to claim 1, wherein Connecting the projection points in the projection point set to construct a non-overlapping triangle set includes: Connecting the projection points in the projection point set and performing a visibility check, and retaining visible points; Triangulate the visible points to construct a set of non-overlapping triangles.

4. The method according to claim 1, wherein Before connecting the projection points in the projection point set to construct a non-overlapping triangle set, the method further includes: The projection points in the projection point set whose distance from the reference point exceeds a predetermined threshold are filtered out, wherein, in the projection point set, the distance obtained by multiplying the distance of the nearest point to the reference point by a predetermined parameter is used as the predetermined threshold, and the predetermined threshold is greater than a predetermined side length threshold, wherein the predetermined side length threshold is positively correlated with the distance from the reference point to the laser radar and the angular resolution of the laser radar.

5. The method according to claim 1, wherein The method further comprises: Establish a coordinate system on the projection plane of the reference point with the reference point as the origin, and obtain the coordinates of each projection point; The angle between the line segment formed by each projection point and the origin and the horizontal axis is calculated according to the coordinates of each projection point, and the angle of each angle in the triangle is calculated according to the angle.

6. The method according to claim 1, wherein Connecting the projection points in the projection point set to construct a non-overlapping triangle set includes: Connecting the projection points in the projection point set based on a greedy projection triangulation algorithm to construct non-overlapping candidate triangles; If the candidate triangle meets the angle restriction and the length restriction, the candidate triangle is added to the triangle set.

7. The method according to claim 6, wherein: The angle restriction condition includes that the angle of any angle is smaller than a predetermined maximum angle and larger than a predetermined minimum angle, and the length restriction condition includes that the length of any side is smaller than a predetermined side length threshold.

8. A device for detecting remaining cargo volume, wherein: The laser radar is installed at the rear of the storage box and scans the entire point cloud inside the storage box. The device includes: an acquisition unit configured to acquire point cloud data in the storage box; A triangulation unit is configured to select a reference point from the point cloud data and perform the following point cloud triangulation steps: searching for a predetermined number of candidate points closest to the reference point; projecting the predetermined number of candidate points onto a projection plane of the reference point to obtain a projection point set; connecting the projection points in the projection point set to construct a non-overlapping triangle set; selecting points near the reference point from points that have never been used as reference points as reference points, and repeating the above point cloud triangulation steps until all points are traversed, wherein the projection direction is the direction in which the laser radar position points to the reference point, and the projection plane is a plane passing through the reference point and perpendicular to the projection direction; A construction unit is configured to construct a tetrahedron for each triangle in the triangle set according to the coordinates of the three points of the triangle before projection and the position of the laser radar, and calculate the volume of the tetrahedron; The calculation unit is configured to sum the volumes of each tetrahedron as the remaining cargo volume.

9. An electronic device for detecting remaining cargo volume, comprising: one or more processors; a storage device having one or more computer programs stored thereon, When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method according to any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images