A target volume automatic measurement method and device, and a storage medium

Abstract

The application discloses a target volume automatic measurement method and device and a storage medium, and comprises the following steps: S1, constructing a point cloud map according to a laser radar single-frame point cloud, wherein the point cloud map contains unmanned vehicle pose information; S2, obtaining an absolute position of a target in the point cloud map by using visual relative positioning and the unmanned vehicle pose information in the point cloud map, and obtaining the target by ground point elimination; S3, performing clustering processing and noise reduction processing on the target, and adopting a point cloud slicing method to perform real-time calculation on the volume of the target. The technical scheme of the application can realize real-time and automatic measurement of the target object by the unmanned vehicle, and has a positive significance for improving the autonomous ability of the unmanned vehicle.

Description

Technical Field

[0001] This invention belongs to the technical field of autonomous driving, and particularly relates to a method and device for automatic measurement of target volume for unmanned vehicles, as well as a storage medium. Background Technology

[0002] In recent years, unmanned vehicles have been widely used in industrial scenarios such as park inspections, security checks, and transportation. LiDAR and visual sensors are used in various fields of unmanned vehicles, including target recognition, path planning, and autonomous localization. By fusing LiDAR and visual information, reliable perception of the environment and targets can be achieved, which is of great significance for the autonomous operation of unmanned vehicles.

[0003] In recent years, object volume measurement methods based on stereo vision have attracted the attention of scholars both domestically and internationally. A method for measuring the volume of small coal piles based on binocular stereo vision has been proposed. This method extracts SURF features from binocular vision-acquired images and achieves non-contact measurement of the volume of small coal piles through matching. However, vision-based object volume measurement suffers from drawbacks such as significant susceptibility to illumination and long computation time. Laser scanners, with their dense scanning beams and high ranging accuracy, are widely used in surveying and mapping. LiDAR offers advantages such as all-weather operation and high precision. Currently, LiDAR is widely used for real-time localization and mapping (SLAM) of unmanned vehicles. Commonly used 2D LiDAR SLAM algorithms include Hector SLAM, Gmapping, and Cartography. Common 3D LiDAR SLAM algorithms include Loam, Lego-loam, and Suma++, which have been widely applied in both indoor and outdoor scenarios. LiDAR can also be used to scan the contours of objects, thereby estimating their volume. Currently, implicit surface reconstruction algorithms are used to construct 3D point cloud network models to calculate spatial volume. This method is mainly used for indoor room measurements, and requires the prior installation of positioning rails to obtain point cloud positions, which limits its autonomy to some extent. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for automatic measurement of target volume and a storage medium, which can realize the real-time and automatic measurement of target objects by unmanned vehicles, and has positive significance for improving the autonomous capabilities of unmanned vehicles.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] An automatic measurement method for target volume includes the following steps:

[0007] Step S1: Construct a point cloud map based on a single frame of point cloud data from the LiDAR, wherein the point cloud map contains the pose information of the unmanned vehicle.

[0008] Step S2: Using visual relative positioning and the unmanned vehicle pose information in the point cloud map, obtain the absolute position of the target in the point cloud map, and obtain the target by removing ground points;

[0009] Step S3: Perform clustering and noise reduction processing on the target, and use the point cloud slicing method to calculate the volume of the target in real time.

[0010] Preferably, step S1 includes:

[0011] Step S11: Perform distortion compensation processing on single frame points of the lidar;

[0012] Step S12: Use the Lego-Loam algorithm to extract features from the single frame points of the LiDAR after distortion compensation to obtain a feature set;

[0013] Step S13: Construct a point cloud map based on the feature set.

[0014] Preferably, in step S11, distortion compensation processing is performed on the single-frame point cloud of the lidar using an IMU and an on-board odometer.

[0015] Preferably, in step S2, a ground plane fitting method is used to remove ground points, and a plane fitting algorithm is used to perform plane fitting on the ground points to achieve the segmentation and removal of ground points.

[0016] The present invention also provides an automatic target volume measuring device, comprising:

[0017] The construction module is used to construct a point cloud map based on a single frame of point cloud data from a LiDAR radar, wherein the point cloud map contains the pose information of the unmanned vehicle.

[0018] The processing module is used to obtain the absolute position of the target in the point cloud map by using visual relative positioning and the position and pose information of the unmanned vehicle in the point cloud map, and to obtain the target by removing ground points;

[0019] The calculation module is used to perform clustering and noise reduction on the target, and to calculate the volume of the target in real time using the point cloud slicing method.

[0020] Preferably, the construction module uses an IMU and an on-board odometer to perform distortion compensation processing on the single-frame point cloud of the lidar.

[0021] Preferably, the processing module uses a ground plane fitting method to remove ground points, and simultaneously uses a plane fitting algorithm to perform plane fitting on the ground points to achieve the segmentation and removal of ground points.

[0022] The present invention also provides a storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement an automatic target volume measurement method.

[0023] This invention uses an autonomous vehicle as a platform and leverages LiDAR to construct a global point cloud map to estimate the vehicle's real-time pose. Simultaneously, it fuses visual and LiDAR data to acquire the point cloud contour of a target object, and then uses point cloud slicing to segment the target object's point cloud and estimate its volume. Compared to existing technologies, this invention enables real-time, automatic measurement of target objects by the autonomous vehicle, significantly enhancing its autonomy. Attached Figure Description

[0024] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of the automatic target volume measurement method according to an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the automatic target volume measurement method according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram illustrating the principle of the lidar point cloud slicing method. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] Example 1:

[0031] like Figure 1 , 2 As shown, this embodiment of the invention provides an automatic target volume measurement method, which uses a lidar and a vision sensor mounted on an unmanned vehicle, and includes the following steps:

[0032] Step S1: Construct a point cloud map based on a single frame of point cloud data from the LiDAR, wherein the point cloud map contains the pose information of the unmanned vehicle.

[0033] Step S2: Using visual relative positioning and the unmanned vehicle pose information in the point cloud map, obtain the absolute position of the target in the point cloud map, and obtain the target by removing ground points;

[0034] Step S3: Cluster and noise reduction are performed on the target, and the volume of the target is calculated in real time using the point cloud slicing method.

[0035] As one embodiment of the present invention, step S1 includes:

[0036] Step S11: Perform distortion compensation processing on single frame points of the lidar;

[0037] Step S12: Use the Lego-Loam algorithm to extract features from the single frame points of the LiDAR after distortion compensation to obtain a feature set;

[0038] Step S13: Construct a point cloud map based on the feature set.

[0039] Furthermore, in step S11, when dynamically measuring the volume of an object, real-time motion information of the autonomous vehicle and the target outline are required. Lego-LOAM is a commonly used SLAM method for autonomous vehicles, which adds loop closure detection on top of LoAM and is more lightweight. Lego-LOAM performs feature extraction and frame matching on the LiDAR point cloud, but the point cloud map will be distorted under maneuvering conditions (such as turning). In order to improve positioning accuracy and map quality, this embodiment of the invention uses a high-frequency IMU and odometry to perform distortion compensation processing on the point cloud, and then uses the distortion-compensated point cloud for feature extraction.

[0040] Traditional LiDAR distortion compensation algorithms rely solely on IMU data, performing linear interpolation on each point in a frame of point cloud to obtain its pose change relative to the start of the scan. However, the relative position calculation based solely on IMU data has a large error. Adding onboard odometer information can effectively improve the accuracy of relative position estimation. Therefore, this invention utilizes IMU angular velocity information and onboard odometer data for dead reckoning and performs distortion compensation processing on the LiDAR point cloud.

[0041] Let p i Let i = 1, 2, ..., l be a point in a frame of point cloud, and let t0 be the start time of the point cloud scan. i The relative time t to t0 i Before scanning to this point, n IMU data points have been generated, where the interval between each data point and t0 is t. jLet j = 1, 2, ..., n. Then the change in angle relative to t0 when the point is scanned is:

[0042]

[0043] Where, ω m =[ω mx ω my ω mz ] represents the angular velocity of the m-th IMU data point, ω mx ω my ω mz For ω m The components are on three axes. By fusing IMU data with odometry data and using dead reckoning, the relative displacement change T between two frames of point cloud can be obtained. j =[T x T y T z ]. Scanned to p i The displacement change at time is:

[0044]

[0045] The scanned value p can be obtained i Relative pose transformation of time-of-flight lidar [θ] i T i ]=[θ ix θ iy θ iz T ix T iy T iz ], where θ i =[θ ix θ iy θ iz [To scan p] i The change in angle of the lidar, T i =[T ix T iy T iz [To scan p] i The state transition matrix can be obtained from the displacement change of the lidar. Then p i After distortion compensation, it can be expressed as:

[0046]

[0047] Furthermore, in step S12, this embodiment of the invention employs the Lego-Loam algorithm for feature extraction to extract the distortion-compensated point cloud. Projected onto distance image D a×bIn the diagram, 'a' represents the radar beam, and 'b' represents the horizontal resolution of the lidar. Range image D a×b Each pixel in the image corresponds to Each point p in t Its pixel value r i p t The Euclidean distance to the center of the lidar is calculated. The distance image is then clustered and segmented to remove interfering points that could cause errors, resulting in the segmented point cloud L. t Feature extraction is performed on the segmented point cloud, and the curvature c of a row of point clouds in the distance image is calculated:

[0048]

[0049] Where N is the number of consecutive points in a row, r is the distance from the point cloud to the lidar, and points with larger c values ​​are considered edges. Points with smaller c values ​​are taken as plane points. n is extracted using the curvature of the point cloud. e The edge point with the largest c value As edge features n p The plane point with the smallest c value As edge features Obtain the feature set

[0050] Furthermore, in step S13, the feature set of each frame's point cloud is obtained. The feature set of the next moment versus the previous moment The following formulas are used to calculate the distance from an edge point to a line and the distance from a plane point to a surface for feature matching:

[0051]

[0052]

[0053] in, express edge points in the middle, express Plane points in, and express edge points in the middle, express Plane points. First, match. and Estimate [t] z θ roll θ pitch ], then with [t z θ roll θ pitch [] is a constraint, matching and Estimate the remaining [t] x t y θ yaw This yields the initial pose based on feature matching. A local map is then formed from a set of the most recent feature sets. Where s is Q t-1 The size of the keyframe is determined. The current keyframe features are matched with the local map to achieve frame-to-image matching. After LM optimization, the pose of the keyframe is obtained. The pose of each keyframe is stored in the pose map, along with the feature results extracted from the point cloud. When loop closure detection occurs, the keyframe is matched with the previous feature set using ICP, and new constraints are added to update the sensor pose and add the keyframe to the global map, thus obtaining the point cloud map.

[0054] In one embodiment of the present invention, step S2 first involves obtaining the position of the target object in the image through the target detection module, and then using visual imaging to calculate the coordinates to solve for the absolute position of the object; subsequently, the point cloud containing the target object is removed by removing ground points, as follows:

[0055] LiDAR can provide spatial scale information of an object, and visual sensors can identify the object. Combining the two can solve for the target's position. Let the coordinate system of the LiDAR at time t be F. L,t The visual sensor coordinate system is F C,t The coordinate transformation matrix between the lidar and the vision sensor is: For each lidar point p L = (x,y,z), F L,t With F C,t The conversion relationship is as follows:

[0056]

[0057] To calculate the 3D position of the target, its 2D position (x, y, w, h) on the planar image is first obtained using a target detection algorithm, where (x, y) represents the target's center coordinates, and (w, h) represents the target's width and height. The specific process can be referenced from the Yolo (You Only Look Once) algorithm. Let Ouv be the image coordinate system, fixed on the imaging plane, with point O located at the top left corner of the image. The u-axis is parallel to the x-axis to the right, and the v-axis is parallel to the y-axis downwards. Based on the detection box information and the aligned depth map, (x, y) is taken as the target's 2D planar position, corresponding to its coordinates (u, v) in the image coordinate system. Substituting this into the intrinsic parameter matrix K of the vision sensor, the detected target's 3D position is obtained as follows:

[0058]

[0059] Among them, (u0,v0,f x ,f y Let be the intrinsic parameter of the visual sensor, obtained through intrinsic parameter calibration, and d be the depth information of (u,v). Let the global frame at time t be F. W,t Using the radar's pose information at this time The target object can be obtained at F W,t The coordinates below are:

[0060]

[0061] When clustering target point clouds, ground point clouds may be mistakenly identified as object point clouds. Therefore, ground points must be removed before clustering target point clouds. This paper employs a ground plane fitting method for ground point removal. By fitting ground points to a plane using a plane fitting algorithm, effective segmentation and removal of ground points can be achieved.

[0062] First, at the distortion compensation point cloud P distor Select the points with the lowest height from the set of points P as the seed point set. seeds Then, based on the RANSAC (RANdom SAmple Consensus) algorithm, P was analyzed. seeds By performing plane fitting, a ground planar model can be obtained. Then, the distance (dist) from each point in the original point cloud to the plane is calculated. i i = 1, 2, ..., l, where l is the number of point clouds. When the distance from a point to the plane is less than a set distance threshold Th... dist When the angle between the normal vector of the currently fitted plane and the normal vector of the previously fitted plane is less than a certain threshold, it is used as the new seed point. This process is repeated until the difference θ between the normal vector of the currently fitted plane and the normal vector of the previously fitted plane is less than a certain threshold. The seed point at this point is the ground point, denoted as P. ground The remaining points are non-ground points, denoted as P. no_ground .

[0063] In one embodiment of the present invention, in step S3, after obtaining the target point cloud and the global pose of the target, the point cloud contour of the object can be obtained through point cloud stitching and clustering, at which point the volume of the object can be calculated. Euclidean clustering centered on the target object's position yields the surface point cloud of the target object. By utilizing the normal vector information of the point cloud and removing noise from the target point cloud, the target point cloud contour can be obtained. The slicing method is an improved method for measuring the volume of an object based on the traditional segmentation method. It calculates the volume by slicing the object's point cloud contour, offering advantages such as high measurement accuracy and strong measurement stability. In the slicing method, when the slices are sufficiently small, the volume of a slice can be considered equal to the product of the slice's area and thickness. The volume of the object can be obtained by summing the volumes of each slice.

[0064] like Figure 3As shown, in this embodiment of the invention, the point cloud of the target is cut into b segments from top to bottom. Let the target height be H, then the thickness of each segment is:

[0065]

[0066] At this point, the calculation of the target volume can be converted into the area S of each segment. i The calculation of i = 1, 2, ..., b. Each segment is a single point p. j =(x j ,y j Polygon A composed of j = 1, 2, ..., u i , where u is A i The number of points in A. In calculating A i When calculating the area, the point cloud first needs to be sorted according to a certain direction. In A i Choose a point p s Search for its nearest point to obtain p e Connecting two points gives you one edge of the polygon. Then, from the remaining points, search for the distance from point p. s (p e Find the nearest point p and calculate its distance to p. s (p e The distance d) s (d e If d s ≤d e Then point p along p s Direction update p s Conversely, move point p along p e Direction update p e And so on, until A. i All points in the array have been searched. At this point, polygon A, arranged in sequence, can be obtained. i Point cloud. Assume A i There are u point clouds p0, p1, ..., p u =p0 arranged counterclockwise, then A i area S i for:

[0067]

[0068] Then, based on the area of ​​each polygon, the volume of the target object can be calculated as follows:

[0069]

[0070] This invention employs an automatic target volume measurement method based on LiDAR / vision. First, an improved Lego-loam algorithm for point cloud distortion correction is used to construct a high-precision point cloud map at high speeds. Then, LiDAR and vision fusion is used to achieve automatic target identification and localization. Finally, the object volume is measured using a slicing method. By removing ground points from the point cloud map, clustering, downsampling, sorting the point cloud, and calculating the volume, real-time estimation of the target volume is achieved.

[0071] Example 2:

[0072] This invention provides an automatic target volume measuring device, comprising:

[0073] The construction module is used to construct a point cloud map based on a single frame of point cloud data from a LiDAR radar, wherein the point cloud map contains the pose information of the unmanned vehicle.

[0074] The processing module is used to obtain the absolute position of the target in the point cloud map by using visual relative positioning and the position and pose information of the unmanned vehicle in the point cloud map, and to obtain the target by removing ground points;

[0075] The calculation module is used to perform clustering and noise reduction on the target, and to calculate the volume of the target in real time using the point cloud slicing method.

[0076] As one embodiment of the present invention, the construction module uses an IMU and an on-board odometer to perform distortion compensation processing on a single frame point cloud of a lidar.

[0077] As one embodiment of the present invention, the processing module uses a ground plane fitting method to remove ground points, and at the same time uses a plane fitting algorithm to perform plane fitting on the ground points to achieve the segmentation and removal of ground points.

[0078] Example 3:

[0079] This invention also provides a storage medium storing machine-executable instructions. When these machine-executable instructions are invoked and executed by a processor, they cause the processor to implement an automatic target volume measurement method.

[0080] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. An automatic method for measuring target volume, characterized in that, Includes the following steps: Step S1: Construct a point cloud map based on a single frame of point cloud data from the LiDAR, wherein the point cloud map contains the pose information of the unmanned vehicle. Step S2: Using visual relative positioning and the unmanned vehicle pose information in the point cloud map, obtain the absolute position of the target in the point cloud map, and obtain the target by removing ground points; Step S3: Perform clustering and noise reduction processing on the target, and use the point cloud slicing method to calculate the volume of the target in real time; Step S1 includes: Step S11: Perform distortion compensation processing on single frame points of the lidar; Step S12: Use the Lego-Loam algorithm to extract features from the single frame points of the LiDAR after distortion compensation to obtain a feature set; Step S13: Construct a point cloud map based on the feature set; In step S11, distortion compensation processing is performed on the single-frame point cloud of the lidar using an IMU and an on-board odometer. Dead reckoning is performed using IMU angular velocity information and on-board odometer, and distortion compensation is applied to the lidar point cloud. set up For a single point in a point cloud frame, the start time of the point cloud scan is... , arrive relative time ; This was generated before the scan reached that point. IMU data, where each data point is relative to The interval time is Then, when the point is scanned, the value relative to the target point is obtained. The change in angle is: in, For the first angular velocity of IMU data , , for The components are on three axes; by fusing IMU data with odometry data, the relative displacement change between two frames of point cloud can be obtained through dead reckoning. ; Scanned The displacement change at time is: The scanned image can be obtained Relative pose transformation of time-of-flight lidar ,in To scan The change in angle of the lidar at that time To scan The displacement change of the lidar is used to obtain its state transition matrix. ,but After distortion compensation, it is represented as follows: .

2. The automatic target volume measurement method as described in claim 1, characterized in that, In step S2, a ground plane fitting method is used to remove ground points. At the same time, a plane fitting algorithm is used to perform plane fitting on the ground points to achieve the segmentation and removal of ground points.

3. An automatic target volume measuring device for implementing the automatic target volume measuring method of claim 1, characterized in that, include: The construction module is used to construct a point cloud map based on a single frame of point cloud data from a LiDAR radar, wherein the point cloud map contains the pose information of the unmanned vehicle. The processing module is used to obtain the absolute position of the target in the point cloud map by using visual relative positioning and the position and pose information of the unmanned vehicle in the point cloud map, and to obtain the target by removing ground points; The calculation module is used to perform clustering and noise reduction on the target, and to calculate the volume of the target in real time using the point cloud slicing method.

4. A storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the automatic target volume measurement method as described in claim 1 or 2.

Images