Mining area map construction method and device, equipment and storage medium

Abstract

The invention relates to the field of map construction, and provides a mining area map construction method and device, equipment and a storage medium, and the method comprises the steps: obtaining a ground point cloud in a key frame collected in a moving process of a vehicle; projecting a ground point cloud in the key frame into a preset grid, and determining a grid point cloud projected into the grid; according to the ground plane parameter consistency of the grid point cloud in the grid, the poses of all key frames in the grid are optimized; and converting and splicing the key frames according to the optimized poses, and constructing the mining area map. The converted and spliced key frames are subjected to pose optimization through the ground plane parameter consistency of the grid point clouds in the grids, so that the influence of jitter on the map precision can be effectively reduced, and the precision of the constructed map can be improved.

Description

Technical Field

[0001] The present application relates to the field of map construction, and in particular to a method, device, equipment and storage medium for constructing a mining area map. Background Art

[0002] The three-dimensional high-precision map of the mining area can provide more comprehensive information for autonomous vehicles, and is also important for mineral material calculation and production planning. When constructing a mining area map, you can use drones to collect data, but the collection cost is high. When constructing a three-dimensional map of a mining area by manual collection, it is difficult to ensure the freshness of the map. Using unmanned vehicles to update the map during operation can effectively reduce the collection cost and ensure the freshness of the map to a large extent.

[0003] However, since the open-pit mining area is relatively open and the terrain is uneven, the unmanned vehicle will experience severe bumps during operation, the sensors of the unmanned vehicle will vibrate greatly, and the mining scene is a degraded scene. Due to the lack of features, it is difficult to achieve high-precision map construction. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a method, device, equipment and storage medium for constructing a mining area map to solve the problem in the prior art that it is difficult to achieve high-precision map construction in an open-pit mining area.

[0005] A first aspect of an embodiment of the present application provides a method for constructing a mining area map, the method comprising:

[0006] Obtain the ground point cloud in the key frames collected during the movement of the vehicle;

[0007] Projecting the ground point cloud in the key frame into a preset grid, and determining the grid point cloud projected into the grid;

[0008] Optimizing the poses of all key frames in a single grid according to the consistency of ground plane parameters of the grid point cloud in the single grid;

[0009] The key frames are converted and spliced ​​according to the optimized posture to construct the mining area map.

[0010] In combination with the first aspect, in a first possible implementation manner of the first aspect, projecting the ground point cloud in the key frame into a preset grid, and determining the grid point cloud projected into the grid includes:

[0011] According to the preset position of the grid, the ground point cloud is projected to the corresponding grid to obtain the projection area of ​​the ground point cloud in the grid;

[0012] Determine the number of sub-grids covered by the projection area of ​​the ground point cloud in the key frame in a single grid, wherein the sub-grids are generated according to the grid division;

[0013] When the number of sub-grids covered in a single grid is less than a predetermined threshold, the projection of the key frame in the grid is deleted.

[0014] In combination with the first aspect, in a second possible implementation manner of the first aspect, optimizing the poses of all key frames in the grid according to the consistency of ground plane parameters of the grid point cloud in a single grid includes:

[0015] Determining a minimum eigenvalue of a grid point cloud within a single grid;

[0016] All key frame poses in the grid are optimized according to the minimum eigenvalue.

[0017] In combination with the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, optimizing all key frame poses in the grid according to the minimum eigenvalue includes:

[0018] According to the minimum eigenvalue of the grid point cloud in the grid, the postures of all key frames included in the grid point cloud in the grid are optimized, and the postures include a pitch angle, a roll angle, and a height in a direction vertical to a horizontal plane.

[0019] In combination with any one of the first aspect to the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, determining a grid point cloud of the ground point cloud within the grid includes:

[0020] Performing eigenvalue decomposition on the grid point cloud within the grid;

[0021] Determine a first ratio of the second largest eigenvalue to the smallest eigenvalue after eigenvalue decomposition;

[0022] When the first ratio is less than a predetermined ratio threshold, the grid point cloud in the grid is deleted.

[0023] In combination with any one of the first aspect to the third possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, determining a grid point cloud of the ground point cloud within the grid includes:

[0024] Get the angle between the ground normal vector in the grid and the Z axis direction;

[0025] When the angle is greater than a predetermined angle threshold, the grid point cloud in the grid is deleted.

[0026] In combination with any one of the first aspect to the third possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, before projecting the ground point cloud in the key frame into a preset grid, the method further includes:

[0027] Obtain the start time and end time of the point cloud of the key frame, and determine a first time difference between the start time and the end time;

[0028] Determine a second ratio of the second time difference to the first time difference according to a second time difference between an acquisition time and a start time of a point in the point cloud;

[0029] The pose corresponding to the point in the point cloud is determined according to the second ratio, and motion compensation is performed on the point in the point cloud according to the pose corresponding to the point in the point cloud.

[0030] A second aspect of an embodiment of the present application provides a mining area map construction device, the device comprising:

[0031] A key frame acquisition unit, used to acquire key frames of ground point clouds from key frames collected during the movement of the vehicle;

[0032] A key frame acquisition unit, used to acquire ground point clouds in key frames collected during the movement of the vehicle;

[0033] A grid division unit, used for projecting the ground point cloud in the key frame into a preset grid, and determining the grid point cloud projected into the grid;

[0034] A posture optimization unit, used for optimizing the postures of all key frames in the grid according to the consistency of ground plane parameters of the grid point cloud in the grid;

[0035] The conversion and splicing unit is used to convert and splice the key frames according to the optimized posture to construct the mining area map.

[0036] The third aspect of an embodiment of the present application provides a mining area map construction device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in any one of the first aspects are implemented.

[0037] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0038] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application obtain the ground point cloud in the key frame collected by the vehicle during movement, project the ground point cloud of the key frame into the grid, and determine the grid point cloud projected into the grid. According to the consistency of the ground plane parameters of the grid point cloud in the grid, the posture of all key frames in the grid is optimized, and the key frames are converted and spliced ​​based on the optimized posture to construct a mining area map. Since the converted and spliced ​​key frames are optimized in posture through the consistency of the ground plane parameters of the grid point cloud in the grid, the influence of jitter on the map accuracy can be effectively reduced, which is beneficial to improving the accuracy of the constructed map. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 This is a schematic diagram of the implementation process of a mining area map construction method provided in an embodiment of the present application;

[0041] Figure 2 It is a schematic diagram of the implementation process of a point cloud dedistortion processing method provided in an embodiment of the present application;

[0042] Figure 3 It is a schematic diagram of an implementation process of optimizing the position and posture of a key frame provided in an embodiment of the present application;

[0043] Figure 4 is a schematic diagram of a mining area map construction device provided in an embodiment of the present application;

[0044] Figure 5 It is a schematic diagram of a mining area map construction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0046] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0047] The three-dimensional high-precision map of the mining area is of great significance to unmanned mines. It can provide more comprehensive information for autonomous driving vehicles and is also important for mineral material calculation and production planning. When constructing the mining area map, the three-dimensional map of the mining area is constructed by drone collection, but the collection cost is high. It is difficult to ensure the freshness of the map when constructing the three-dimensional map of the mining area by manual collection. The use of unmanned vehicles can update the map during operation, which can effectively reduce the collection cost and ensure the freshness of the map to a large extent. However, due to the relatively open open-pit mining area and the uneven terrain, the unmanned vehicle is more bumpy during operation, the sensor of the unmanned vehicle is greatly shaken, and the mining scene is a degraded scene. Due to the lack of features, it is difficult to achieve high-precision map construction.

[0048] Based on this, an embodiment of the present application proposes a method for constructing a mining map, which can be applied to a mapping vehicle, or can also be applied to other terminal devices or servers located in the background. This method uses the ground data of the mining area to repair the problem of low map accuracy caused by vehicle jitter. In order to solve the problem that laser features are difficult to correspond, the point cloud is gridded. According to the fact that the ground point clouds in the same grid are in the same plane, BA (the full name of English is Bundle Adjustment, and the full name of Chinese is bundle adjustment) is used to optimize the plane point cloud in the grid to achieve the purpose of optimizing road signs. By optimizing the plane road signs, the posture parameters in the multi-frame point cloud can be optimized, including the roll angle roll, the pitch angle pitch, and the vertical ground direction coordinate z, so as to achieve the observable point cloud matching and eliminate the influence of vehicle jitter.

[0049] like Figure 1 The figure is a schematic diagram of the implementation process of a mining area map construction method provided in an embodiment of the present application, the method comprising:

[0050] In S101, a ground point cloud in a key frame collected during the movement of the vehicle is obtained.

[0051] The embodiment of the present application can obtain key frames through vehicles and extract ground point clouds included in the key frames. For example, sensing equipment such as laser radars and distance sensors can be set on the vehicle to collect ground point clouds in the mining area. The vehicle can include an unmanned vehicle or a manually driven vehicle, and the vehicle can include a mining truck, etc.

[0052] During the movement of the vehicle, the point cloud in the environment can be acquired through the point cloud acquisition device. In the acquired point cloud, the key frames included in the point cloud can be determined according to predetermined rules. The acquisition can be performed according to a predetermined distance interval or time interval. For example, during the movement of the vehicle, the key frames in the point cloud can be determined according to a predetermined distance interval of any length between 0.5 and 2 meters. In a possible implementation, the key frames can also be determined based on the repetition rate of the features in the acquired point cloud. For example, when the repetition rate of the features of the acquired point cloud and the point cloud in the previous key frame is less than a predetermined ratio, the newly acquired point cloud is determined to be a new key frame.

[0053] The ground point cloud is a point cloud composed of points located on the ground in the point cloud included in the key frame.

[0054] The acquired ground point cloud can select a predetermined number of key frames for pose optimization calculation according to the time sequence of the acquired point clouds. For example, the point cloud used for pose optimization calculation can be determined by a sliding window of a predetermined time length, or by a sliding window of a predetermined number of key frames. For example, it can be set to collect key frames at a predetermined time interval Tr, and the sliding window can be n*Tr, and n can be a natural number greater than 2, such as 10, 20, etc. For example, the value of n can be 10, and the point clouds of the 1st to 10th frames can be determined to be located in the same sliding window through a sliding window of a predetermined time length, and pose optimization calculations are performed on the 1st to 10th frames of point clouds. When the 11th frame of point cloud is collected, the point cloud in the sliding window is updated to the 2nd to 11th frames, that is, the earliest key frame in the sliding window is deleted or solidified, and so on. The point cloud in the sliding window can also delete the earliest frame 2 and update to include key frames from the 3rd to 12th frames, etc.

[0055] The key frames determined according to the sliding window may include an initial frame. The initial frame may include the first frame when the point cloud is first acquired. As the key frames in the sliding window change, the initial frame is the first frame in the sliding window, that is, the earliest key frame.

[0056] In a possible implementation, the embodiment of the present application may further perform a dedistortion process on the collected point cloud before determining the key frame to obtain a point cloud with motion distortion removed, thereby improving the accuracy of the point cloud.

[0057] Among them, the process of point cloud dedistortion can be as follows: Figure 2 As shown, including:

[0058] In S201, the start time and the end time of the point cloud of the key frame are obtained, and a first time difference between the start time and the end time is determined.

[0059] It takes a certain amount of time for a mapping vehicle to collect a frame of point cloud. The time when the key frame point cloud collection starts can be used as the start time, and the time when the key frame point cloud collection is completed can be used as the end time. Between the start time and the end time, the mapping vehicle can collect point clouds in different directions by rotating, moving, and other motions. Therefore, in the same frame of point cloud, point clouds in different directions may have different collection times and collection postures.

[0060] In order to facilitate determination of the acquisition time corresponding to a single point in the point cloud, the duration of acquiring the frame of point cloud, that is, the time interval between the start time and the end time, may be used as the first time difference.

[0061] In S202, a second ratio of the second time difference to the first time difference is determined according to a second time difference between the acquisition time and the start time of the point in the point cloud.

[0062] Since the acquisition time of the non-orientation point cloud in the same frame is different, the second time difference can be determined according to the acquisition time of the point cloud and the start time of the frame. The second time difference can be used to determine the duration of the movement of the point when the point cloud of the frame is first acquired.

[0063] The second time difference is compared with the first time difference to determine a second ratio of the second time difference to the first time difference, thereby facilitating determination of the accurate position and posture of the point.

[0064] In S203, a posture corresponding to the point is determined according to the second ratio, and motion compensation is performed on the point according to the posture corresponding to the point.

[0065] After determining the second ratio, the exact posture corresponding to the point can be determined in combination with the amount of movement of the frame point cloud from the start time to the end time, so as to facilitate posture adjustment of the point according to the posture, including rotation transformation or position transformation according to the posture of the point to compensate for the position change and rotation change of the point cloud caused by the movement, thereby obtaining a point cloud with distortion removed.

[0066] The amount of motion of a frame of point cloud can be determined based on the pose sequence corresponding to the frame. For example, the pose sequence L1 of the combined navigation can be interpolated to obtain the poses corresponding to the start and end time of any frame of point cloud. and T1 G The external parameter matrix from laser radar to integrated navigation is T1 G , the position of the laser radar corresponding to the start time for: The position of the laser radar corresponding to the end time T1 L for

[0067] The amount of motion in the corresponding frame is T10 for:

[0068] According to the above steps, the second time difference dt of each point relative to the start time of the point cloud and the first time difference t of the frame point cloud are calculated, and the second ratio of the second time difference to the first time difference is dt / t. The position and posture of each point are interpolated using the second ratio to obtain the position transformation and rotation transformation of each point. After compensating for the position change and rotation change caused by the motion of each point, the point cloud after the motion distortion is removed can be obtained.

[0069] Based on the same method, the coordinate system of the first frame can be used as the basis, and according to the position of the lidar in each key frame, the point cloud of the key frame can be transformed into the coordinate system of the first key frame, so as to facilitate the optimization calculation of the point cloud in the same coordinate system.

[0070] In S102, the ground point cloud in the key frame is projected into a preset grid, and a grid point cloud projected into the grid is determined.

[0071] When the point cloud is grid-divided in the embodiment of the present application, the position of the grid division can be determined according to the point cloud of the initial frame. The predetermined range can be divided according to the predetermined grid size based on the vehicle position of the initial frame as the center. For example, the predetermined range can be a circular range, and the radius of the circular range can include any value in the range of 20-40 meters, such as 20 meters, 25 meters, 30 meters, 35 meters or 40 meters. The predetermined grid size can be a square grid, for example, the size of the grid can include any value in the range of 1.5 meters to 4 meters.

[0072] Of course, when performing grid division, it is not necessary to limit the grid division to the initial frame. The range and position of the grid can also be determined based on any key frame in the sliding window. For example, the range and position of the grid division can be determined based on the key frame in the middle position of the sliding window.

[0073] After the position of the grid is determined, the point clouds of two or more key frames used for calculation can be projected into the grid to obtain the point clouds corresponding to each grid.

[0074] When projecting a keyframe into the grid, you can project a point cloud to be optimized or a solidified point cloud according to the type of keyframe. A fixed point cloud is a point cloud that does not need to be optimized. The pose of point clouds of other frames projected into the grid can be optimized based on the solidified point cloud. When determining the solidified point cloud, the first frame of point cloud collected by the vehicle can be set as a solidified point cloud, which does not need pose optimization.

[0075] In a possible implementation, the number of keyframe optimization calculations can be recorded. When the number of optimizations reaches a predetermined threshold, the keyframe is set as a solidified keyframe. The point cloud projected to the grid area is a solidified point cloud, and no pose optimization calculation is required. For example, when selecting keyframes according to the sliding window, the earliest keyframe deleted can be set as a solidified keyframe.

[0076] For non-solidified keyframes, the point cloud in the keyframe can be projected to the area where the grid is located to determine the point cloud of the keyframe projected in each grid. When the sliding window includes multiple keyframes, the multiple keyframes can be projected to the area where the grid is located in sequence to obtain the point clouds from different keyframes projected in each grid.

[0077] For example, a sliding window of fixed duration includes 10 keyframes, and the point cloud projected into the grid X may include the point clouds and poses of the 10 keyframes. The point clouds in the 10 keyframes may include solidified point clouds, for example, the point cloud in the first keyframe is the solidified keyframe. Of course, the point cloud projected into the grid X may also include only the point clouds in some of the keyframes.

[0078] In order to simplify calculation and improve calculation efficiency, when determining the grid point cloud within the grid, the embodiment of the present application can filter the grid point cloud within the grid to obtain a grid point cloud whose plane features meet preset requirements, or obtain a grid point cloud whose inclination angle meets preset requirements.

[0079] For example, RANSAC (random sampling consensus in Chinese) ground extraction can be performed on each grid point cloud to obtain the ground plane parameters of the grid, and the angle between the ground normal vector in the ground plane parameters and the Z axis (the world coordinate system, that is, the vertical horizontal plane direction) is calculated. If it is greater than a predetermined angle, such as greater than any angle between 8 and 15 degrees, the current grid is discarded, that is, there is no need to perform pose optimization calculations based on the grid point cloud in the grid.

[0080] Alternatively, a predetermined range of ground plane parameters in the current grid may be extracted, such as all point clouds near any value within 20cm-50cm, and eigenvalue decomposition may be performed on the point clouds extracted in each grid. If the ratio of the second largest eigenvalue to the minimum eigenvalue after eigenvalue decomposition is greater than a predetermined ratio threshold, such as greater than any value in the range of 2-4, including greater than 3, the grid point cloud in the current grid is used; if it is less than the predetermined ratio threshold, the current grid is discarded, and there is no need to perform pose optimization calculations based on the grid point cloud in the grid, and the grid with better plane parameters is retained for pose optimization calculations.

[0081] The ground point cloud in the grid can be determined according to one or both of the above two methods. The ground point clouds in the grid of all key frames in the sliding window can be superimposed to obtain the final ground point cloud.

[0082] In S103, according to the consistency of ground plane parameters of the grid point cloud in a single grid, the poses of all key frames in the single grid are optimized.

[0083] When performing pose optimization on key frames in the same sliding window, pose optimization calculations can be performed on all key frames in the sliding window. When the sliding window includes solidified key frames and non-solidified key frames, pose optimization calculations can be performed on the non-solidified key frames based on the consistency between the point cloud parameters of the solidified key frames and the point cloud parameters of the non-solidified key frames. Alternatively, the pose of the non-solidified key frames can also be optimized based on the consistency of the point cloud parameters of the non-solidified key frames. When the point cloud projected by a key frame includes multiple grid point clouds, the pose parameters of the key frame can be further determined using the least squares method or other methods based on the optimization parameters of the multiple grids.

[0084] In order to improve the optimization accuracy, before optimizing the pose of the key frame, the pose of the key frame can be transformed first, and the coordinates of the point cloud in the key frame can be transformed to the coordinate system of the initial frame (such as the first frame), so as to facilitate the optimization calculation of the point cloud in the key frame and the initial frame. For example, the point cloud in the key frame can be transformed to the coordinate system of the initial frame according to the position transformation and rotation transformation of the constructed vehicle.

[0085] In order to optimize the deviations in pitch angle, roll angle and height z in the vertical horizontal plane direction caused by the jitter of the key frame, the bundle adjustment method can be used to solve the ground point cloud, that is, the ground point cloud in the same grid is considered to be the same plane landmark. Based on the plane landmark, the roll angle roll, pitch angle pitch and height z of different key frames are optimized. For the ground plane of different key frames in the same grid, the optimization result can obtain a plane point cloud with the minimum error, that is, minimize the minimum eigenvalue of the point cloud in the current grid. Therefore, the embodiment of the present application can decouple the plane parameters from the posture, and directly optimize the posture to be optimized (roll angle roll, pitch angle pitch, and height z) without optimizing the plane parameters, thereby reducing the dimension of the unknowns. The distance error of the minimized point to the plane can be converted into minimizing the minimum eigenvalue of the plane point cloud in the current grid, thereby achieving the purpose of simplifying the unknowns to be solved.

[0086] Among them, the implementation process of optimizing the pose of the key frame in the sliding window can be as follows: Figure 3 As shown, including:

[0087] In S301, a solidified key frame in a sliding window is determined, and a solidified point cloud and a solidified pose corresponding to the solidified key frame are stored in the grid.

[0088] After acquiring the point cloud frame, we can first determine whether the point cloud frame is the first frame point cloud. If it is the first frame point cloud, we can determine that the frame is a fixed key frame. That is, the parameters in the frame are key frames that do not need to be optimized, including the point cloud and posture of the frame do not need to be optimized, including the posture parameters roll, pitch and z of the frame do not need to be optimized, and the point cloud of the fixed key frame can be projected and stored in the corresponding grid.

[0089] The solidified key frame may include the key frame of the first frame, and may also include the key frame whose optimization times are greater than a predetermined times threshold.

[0090] The grid can be determined based on the first frame, for example, the first frame can be taken as the center, and each box of a predetermined size (such as a 2m*2m square) can be used as a grid. The data stored in the grid can include the position and posture of the key frame corresponding to the solidified key frame, and the ground point cloud in the key frame (for the point cloud of the first key frame, the position and posture of the first key frame and the ground point cloud in the first key frame are stored).

[0091] The sliding window can refer to maintaining a sliding window of a predetermined time, and only optimizing the posture of each key frame in the sliding window each time, including roll, pitch and z value. If it exceeds the set sliding window size, the point cloud of the past historical frame can be fixed and the posture can be kicked out of the sliding window.

[0092] In S302, the point cloud of the non-solidified key frame is projected to the corresponding grid, and the non-solidified key frame point cloud and key frame pose in the grid are determined.

[0093] If the current frame is not a non-solidified keyframe such as the first frame data, the ground point cloud in the current keyframe can be projected into the current grid. If the grid at the corresponding position is not constructed when projecting to the current grid, the grid can be constructed first, and then the point cloud can be projected into the grid, and the pose of the current frame can be added to the grid.

[0094] In a possible implementation, when projecting a key frame into a grid, in order to determine the validity of the projection, each grid may be further divided into a predetermined number of sub-grids, such as dividing each grid into 9 sub-grids. For the divided sub-grids, it is possible to count whether each key frame effectively covers the corresponding sub-grid when projected into the grid area, that is, to determine whether the sub-grid is a valid sub-grid. If the projection area of ​​the point cloud of the key frame effectively covers the sub-grid, the sub-grid is a valid sub-grid, otherwise it is an invalid sub-grid.

[0095] If the number of valid sub-grids in the projection area is less than a predetermined threshold when the key frame is projected into the grid, for example, less than 3, it is considered that the number of grid point clouds in the grid is small, and the point cloud of the key frame in the grid can be deleted. If the number of valid sub-grids in the projection area is greater than the predetermined threshold, the point cloud of the key frame is allowed to be projected into the grid and the projection is successful. The state of the successfully projected grid is set as the grid to be optimized, and the successfully projected point cloud is the key frame point cloud.

[0096] In S303, the key frame poses in the grid are optimized according to the consistency of the ground plane parameters of the point cloud of the key frame.

[0097] After the projection is successful, the grid to be optimized can be optimized. The pose optimization of the key frame can be achieved by determining the minimum eigenvalue of the solidified point cloud and the key frame point cloud, or by determining the minimum eigenvalue of the key frame point cloud, that is, assuming that the ground points in the same grid are in the same plane. For example, the pose parameters of all key frames in the current sliding window, including roll angle roll, pitch angle pitch, and height z, can be derived to obtain the first-order and second-order levels of the pose parameters. Then the residual is optimized by the LM (Levenberg-Marquardt in English and Levenberg-Marquardt in Chinese) algorithm, and the roll angle roll, pitch angle pitch, and height z of all key frames in the current sliding window are optimized at the same time.

[0098] In a possible implementation, the maximum number of key frames allowed to be included in the sliding window can be preset. If the number of key frames in the sliding window is greater than the set maximum number, the point cloud of the oldest historical frame in the sliding window can be marginalized, and the pose and point cloud information of the historical frame can be fixed. When the point cloud information is fixed, the point cloud information can be added to the fixed point cloud, and the fixed point cloud can be re-downsampled to obtain an updated fixed point cloud, and the updated fixed point cloud can be used as an optimization prior to participate in subsequent optimization.

[0099] In S104, the key frames are converted and spliced ​​according to the optimized posture to construct the mining area map.

[0100] After optimizing the pose parameters of the key frame, the point cloud of the key frame can be transformed based on the optimized pose parameters, including roll angle, pitch angle, and height z, and the transformed point cloud can be spliced ​​to obtain a high-precision mining area map.

[0101] In order to optimize the deviations in roll angle, pitch angle, and height z caused by key frame jitter, the ground point cloud in the same grid can be considered as the same plane landmark, and the roll angle, pitch angle, and height z of different key frames under the plane landmark are optimized. For the ground plane of different key frames in the same grid, the optimization result is to obtain a plane point cloud with the minimum error, that is, to minimize the minimum eigenvalue of the point cloud in a grid. The plane parameters can be decoupled from the posture, and the posture to be optimized, including the roll angle, pitch angle, and height z, can be directly optimized without optimizing the plane parameters.

[0102] When minimizing the minimum eigenvalue of a point cloud within a grid, the following formula can be used:

[0103]

[0104] in, can be converted to: T represents transpose, N represents the number of points in the grid, n represents the normal vector of the plane, q represents the center point of the grid plane, Represents the average value of points in the grid plane, p i represents the i-th point in the grid, λ min () represents the minimization function.

[0105] The embodiment of the present application takes the grid point cloud in the same grid as the starting point and performs multi-frame optimization and solution on the point cloud of the ground plane, thereby repairing the accumulated error between multiple frames and significantly eliminating the problem of map construction accuracy caused by vehicle jitter. In addition, by dividing the ground plane into grids and assuming that the plane parameters in a single grid are consistent, the problem of difficult data correspondence in the traditional laser light speed adjustment method can be effectively solved, making the posture optimization easier to solve.

[0106] In addition, since most of the mining scene is an open scene, the scene lacks corresponding features, and direct matching calculation will be degraded. However, this application only needs to use the ground plane to optimize the posture parameters of roll, pitch, and z, which have a greater impact on performance, to ensure the effectiveness of removing the impact of jitter after system optimization.

[0107] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0108] Figure 4 A schematic diagram of a mining area map construction device provided in an embodiment of the present application, the device comprising:

[0109] The key frame acquisition unit 401 is used to acquire the ground point cloud in the key frame collected during the movement of the vehicle.

[0110] The grid division unit 402 is used to project the ground point cloud in the key frame into a preset grid and determine the grid point cloud projected into the grid.

[0111] The posture optimization unit 403 is used to optimize the postures of all key frames in the grid according to the consistency of the ground plane parameters of the grid point cloud in a single grid.

[0112] The conversion and splicing unit 404 is used to convert and splice the key frames according to the optimized posture to construct the mining area map.

[0113] Figure 4 The mining area map construction device shown in FIG. Figure 1 The mining area map construction method shown corresponds to the above.

[0114] Figure 5 Schematic diagram of a mining area map construction device provided in an embodiment of the present application. Figure 5 As shown, the mining area map construction device 5 of this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50, such as a mining area map construction program. When the processor 50 executes the computer program 52, the steps in the above-mentioned mining area map construction method embodiments are implemented. Alternatively, when the processor 50 executes the computer program 52, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0115] Exemplarily, the computer program 52 may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 52 in the mining area map construction device 5.

[0116] The mining area map construction device 5 can be a computing device such as a mapping vehicle, a desktop computer, a notebook, a PDA, and a cloud server. The mining area map construction device can include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art can understand that Figure 5 It is only an example of the mining area map construction device 5 and does not constitute a limitation of the mining area map construction device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the mining area map construction device may also include input and output devices, network access devices, buses, etc.

[0117] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0118] The memory 51 may be an internal storage unit of the mining area map construction device 5, such as a hard disk or memory of the mining area map construction device 5. The memory 51 may also be an external storage device of the mining area map construction device 5, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the mining area map construction device 5. Further, the memory 51 may also include both an internal storage unit of the mining area map construction device 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the mining area map construction device. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0119] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0120] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0122] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0125] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0126] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for constructing a mining area map, characterized in that: The method comprises: Obtain the ground point cloud in the key frames collected during the movement of the vehicle; Projecting the ground point cloud in the key frame into a preset grid, and determining the grid point cloud projected into the grid; Optimizing the poses of all key frames in a single grid according to the consistency of ground plane parameters of the grid point cloud in the single grid; The key frames are converted and spliced ​​according to the optimized posture to construct the mining area map.

2. The method according to claim 1, characterized in that: Projecting the ground point cloud in the key frame into a preset grid, and determining the grid point cloud projected into the grid, comprises: According to the preset position of the grid, the ground point cloud is projected to the corresponding grid to obtain the projection area of ​​the ground point cloud in the grid; Determine the number of sub-grids covered by the projection area of ​​the ground point cloud in the key frame in a single grid, wherein the sub-grids are generated according to the grid division; When the number of sub-grids covered in a single grid is less than a predetermined threshold, the projection of the key frame in the grid is deleted.

3. The method according to claim 1, characterized in that According to the consistency of the ground plane parameters of the grid point cloud in a single grid, the poses of all key frames in the grid are optimized, including: Determining a minimum eigenvalue of a grid point cloud within a single grid; All key frame poses in the grid are optimized according to the minimum eigenvalue.

4. The method according to claim 3, characterized in that Optimizing all key frame poses in the grid according to the minimum eigenvalue, including: According to the minimum eigenvalue of the grid point cloud in the grid, the postures of all key frames included in the grid point cloud in the grid are optimized, and the postures include a pitch angle, a roll angle, and a height in a direction vertical to a horizontal plane.

5. The method according to any one of claims 1 to 4, characterized in that: Determining a grid point cloud of the ground point cloud within the grid includes: Performing eigenvalue decomposition on the grid point cloud within the grid; Determine a first ratio of the second largest eigenvalue to the smallest eigenvalue after eigenvalue decomposition; When the first ratio is less than a predetermined ratio threshold, the grid point cloud in the grid is deleted.

6. The method according to any one of claims 1 to 4, characterized in that: Determining a grid point cloud of the ground point cloud within the grid includes: Get the angle between the ground normal vector in the grid and the Z axis direction; When the angle is greater than a predetermined angle threshold, the grid point cloud in the grid is deleted.

7. The method according to any one of claims 1 to 4, characterized in that: Before projecting the ground point cloud in the key frame into a preset grid, the method further includes: Obtain the start time and end time of the point cloud of the key frame, and determine a first time difference between the start time and the end time; Determine a second ratio of the second time difference to the first time difference according to a second time difference between an acquisition time and a start time of a point in the point cloud; The pose corresponding to the point in the point cloud is determined according to the second ratio, and motion compensation is performed on the point in the point cloud according to the pose corresponding to the point in the point cloud.

8. A mining area map construction device, characterized in that: The device comprises: A key frame acquisition unit, used to acquire ground point clouds in key frames collected during the movement of the vehicle; A grid division unit, used for projecting the ground point cloud in the key frame into a preset grid, and determining the grid point cloud projected into the grid; A posture optimization unit, used for optimizing the postures of all key frames in the grid according to the consistency of ground plane parameters of the grid point cloud in the grid; The conversion and splicing unit is used to convert and splice the key frames according to the optimized posture to construct the mining area map.

9. A mining area map construction device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images