System and method for constructing three-dimensional map of mining area

Through the coordinated work of drones, mine vehicles and roadside perception ends, the three-dimensional point cloud maps are obtained and integrated with lidar and navigation equipment, the problems of difficulty in building three-dimensional maps in mining areas are solved and the problem of untimely updates are achieved, and efficient and safe three-dimensional Map build and update.

CN113706702BActive Publication Date: 2024-11-01CHONGQING JIUZHOU XINGYI NAVIGATION EQUIP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110920940.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-11
Publication Date
2024-11-01
Estimated Expiration
2041-08-11

AI Technical Summary

Technical Problem

It is difficult, cumbersome and untimely updated three-dimensional maps in the mining environment, especially due to the high dust density and rapid changes in the terrain environment, it is difficult for traditional methods to build a full-view map and grasp the road conditions in real time.

Method used

UAVs, mine cars and roadside sensing ends are all equipped with lidar, differential GNSS navigation equipment and IMU inertial measurement units. The global three-dimensional point cloud map is obtained through the drone as the base map, and it is connected to the mine cars and roadside sensing ends. The obtained local three-dimensional point cloud map is fused, and the ICP algorithm is used to match and splice point clouds to build a global three-dimensional map.

Benefits of technology

It realizes the rapid and timely construction and update of three-dimensional maps in the mining area environment, improves the operating efficiency and safety of the mining area, and can build a larger range of three-dimensional maps in the dusty environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113706702B_ABST
    Figure CN113706702B_ABST
Patent Text Reader

Abstract

The present invention provides a system and method for constructing a three-dimensional map of a mining area. A three-dimensional point cloud map of the entire mining area is conveniently and quickly obtained as a base map by a drone, and then the map is integrated with the local three-dimensional point cloud map obtained in real time by a roadside sensing terminal and a mining vehicle to obtain a global three-dimensional point cloud map. Compared with the method in the prior art of constructing a three-dimensional map by scanning with equipment carried by the vehicle itself, the three-dimensional map is easier to construct and the construction speed is faster. The three-dimensional map can be updated more timely according to changes in the terrain of the mining area. Moreover, compared with the method in the prior art of constructing a three-dimensional map by scanning with equipment carried by the vehicle itself, a larger three-dimensional map can be constructed in a dusty environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a system and method for constructing a three-dimensional map of a mining area. Background Art

[0002] Autonomous driving technology can not only avoid or reduce the harm or threat to the health and safety of drivers, but also greatly improve efficiency, reduce costs, and be more economical, energy-saving and environmentally friendly. Therefore, it has a good application prospect in open-pit mines and can be applied on mine cars to realize unmanned transportation of materials. To realize the autonomous driving of mine cars, it is necessary to first build a three-dimensional high-precision map of the mine for path planning; the existing three-dimensional high-precision map construction method is generally developed for urban roads, and only relies on the equipment carried by the vehicle itself to scan and construct a three-dimensional map. Compared with the urban environment, the mining area has the characteristics of high dust density, rapid changes in terrain environment with different mining progress, and frequent spills on the road and falling rocks on the slope, which leads to the following problems:

[0003] 1. Due to the rapid changes in terrain and environment, the 3D map needs to be rebuilt frequently. The traditional 3D map construction method is time-consuming and labor-intensive, which reduces the operating efficiency of the mining area;

[0004] 2. Due to the high density of dust, the effective distance of the sensor is greatly reduced. In the traditional method, point cloud data collection only relies on sensors on the vehicle, which makes it difficult to build a three-dimensional map of the entire mining area;

[0005] 3. Since spilled objects and falling rocks on slopes often cause damage to the roads, it is difficult to grasp the real-time road conditions by relying solely on the sensors on the vehicle to collect point cloud data. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the purpose of the embodiments of the present application is to provide a system and method for constructing a three-dimensional map of a mining area, aiming to solve the problems of difficulty, complexity and untimely updating of three-dimensional maps in a mining environment.

[0007] In a first aspect, an embodiment of the present application provides a mining area three-dimensional map construction system, including a drone, a mining car, and multiple roadside sensing terminals, wherein the drone, the mining car, and the roadside sensing terminal all include a laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit;

[0008] The drone is used to obtain a global three-dimensional point cloud map of the mining area as a base map through corresponding laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit, and send it to the mining car;

[0009] The roadside sensing terminal is arranged on the roadside of the mining area road, and is used to obtain a first local three-dimensional point cloud map of the area where the roadside sensing terminal is located through a corresponding laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit, and send it to the mining vehicle;

[0010] The mine car is used to obtain a second local three-dimensional point cloud map of the area where the mine car is located through the corresponding laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit, fuse the first local three-dimensional point cloud map and the second local three-dimensional point cloud map to obtain a preliminary fused local three-dimensional point cloud map, and then fuse the preliminary fused local three-dimensional point cloud map with the base map to obtain a fused global three-dimensional point cloud map.

[0011] The mining area three-dimensional map construction system of the embodiment of the present application can use a drone to quickly and conveniently obtain a global three-dimensional point cloud map of the mining area as a base map when the mining area terrain changes, and then fuse it with the local three-dimensional point cloud map obtained in real time by the roadside sensing terminal and the mining car to obtain a global three-dimensional point cloud map. Compared with the method in the prior art that only relies on the equipment carried by the vehicle itself to scan and construct a three-dimensional map, the three-dimensional map construction is easier and faster, and the three-dimensional map can be updated more timely according to the changes in the mining area terrain; and compared with the method in the prior art that only relies on the equipment carried by the vehicle itself to scan and construct a three-dimensional map, a larger three-dimensional map can be constructed in a dusty environment.

[0012] Preferably, when the drone, mining car and roadside sensing terminal obtain the corresponding three-dimensional point cloud map through the corresponding laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit:

[0013] Use laser radar to obtain the coordinates of the scanning point in the laser radar coordinate system;

[0014] The coordinates of the scanning point in the geocentric rectangular coordinate system are calculated according to the following formula:

[0015] ;

[0016] in, is the coordinate of the scanning point in the geocentric rectangular coordinate system, is the coordinate of the center of the corresponding IMU inertial measurement unit in the geocentric rectangular coordinate system, is the rotation matrix from the corresponding IMU inertial measurement unit coordinate system to the geocentric rectangular coordinate system, is the component of the offset from the center of the laser radar coordinate system of the laser radar to the center of the IMU inertial measurement unit in the IMU inertial measurement unit coordinate system, is the rotation matrix from the laser radar coordinate system to the IMU inertial measurement unit coordinate system, are the coordinates of the scanning point in the laser radar coordinate system.

[0017] Preferably, when the mining vehicle fuses the first local three-dimensional point cloud image with the second local three-dimensional point cloud image:

[0018] Based on the coordinates of each point in the geocentric rectangular coordinate system, the first local three-dimensional point cloud image and the second local three-dimensional point cloud image are spliced.

[0019] Preferably, when the mining vehicle fuses the initially fused local three-dimensional point cloud image with the base map to obtain a fused global three-dimensional point cloud image:

[0020] The point cloud matching method based on the ICP algorithm is used to perform point cloud splicing on the preliminary fused local three-dimensional point cloud map and the base map.

[0021] Preferably, when the mining vehicle uses the point cloud matching method based on the ICP algorithm to perform point cloud splicing on the preliminary fused local three-dimensional point cloud image and the base map, the following steps are performed:

[0022] C1. Select the feature points with the same name in the preliminary fused local 3D point cloud image and the base image to obtain the first feature point set with the same name corresponding to the preliminary fused local 3D point cloud image. The second feature point set with the same name as the base map ;

[0023] C2. Repeat the following iterative process until the error Less than a preset threshold:

[0024] Make the center point of the first feature point set with the same name and the center point of the second feature point set with the same name The center points of the feature points coincide with each other, and the translation vector of the center point of the first feature point set with the same name in the coincidence operation is calculated. ;

[0025] For each point in the first feature point set with the same name, Search for its nearest neighbor in space to establish the corresponding relationship and calculate the rotation matrix between the nearest points;

[0026] The total rotation matrix, rotation vector, and the first set of feature points with the same name after rotation for the nth iteration are calculated according to the following formula:

[0027] ;

[0028] in, is the total rotation matrix, is the rotation matrix between the closest points calculated in the i-th iteration, is the rotation vector, is the first set of feature points with the same name after rotation for the nth iteration. is the initial first set of feature points with the same name;

[0029] The error is calculated by the following formula :

[0030] ;

[0031] in, for The position of the i-th point in is the second feature point set with the same name The position of the i-th point in ;

[0032] Determine the error Is it less than the preset threshold?

[0033] C3. transforming each point of the preliminary fused local 3D point cloud image according to the following formula, and splicing the transformed preliminary fused local 3D point cloud image with the base map:

[0034] ;

[0035] in, is the position of the i-th point in the preliminary fused local 3D point cloud after conversion, is the position of the i-th point in the preliminary fused local 3D point cloud before conversion.

[0036] Preferably, step C1 comprises:

[0037] Select multiple feature points in the preliminary fused local three-dimensional point cloud image to obtain a first feature point set with the same name corresponding to the preliminary fused local three-dimensional point cloud image. ;

[0038] For the first set of feature points with the same name For each feature point in the base map, the Kd-tree algorithm is used to search for the closest point in the base map as the corresponding feature point with the same name, and the second feature point set with the same name corresponding to the base map is obtained. .

[0039] In a second aspect, the present application provides a method for constructing a three-dimensional map of a mining area, comprising the steps of:

[0040] A1. Obtain a global three-dimensional point cloud map of the mining area obtained by the drone through the laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit set on the drone as a base map;

[0041] A2. Obtaining a first local three-dimensional point cloud map of the area where the roadside sensing terminal is located, obtained by the roadside sensing terminal through a laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit arranged on the roadside sensing terminal; the roadside sensing terminal is arranged on the side of the road in the mining area;

[0042] A3. Obtaining a second local three-dimensional point cloud map of the area where the mine car is located by a laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit provided on the mine car;

[0043] A4. Fusing the first local three-dimensional point cloud image and the second local three-dimensional point cloud image to obtain a preliminary fused local three-dimensional point cloud image;

[0044] A5. Fusing the initially fused local three-dimensional point cloud image with the base image to obtain a fused global three-dimensional point cloud image.

[0045] Preferably, step A4 comprises:

[0046] Based on the coordinates of each point in the geocentric rectangular coordinate system, the first local three-dimensional point cloud image and the second local three-dimensional point cloud image are spliced.

[0047] Preferably, step A5 comprises:

[0048] The point cloud matching method based on the ICP algorithm is used to perform point cloud splicing on the preliminary fused local three-dimensional point cloud map and the base map.

[0049] Preferably, the step of performing point cloud stitching on the preliminary fused local three-dimensional point cloud image and the base map by using a point cloud matching method based on an ICP algorithm comprises:

[0050] C1. Select the feature points with the same name in the preliminary fused local 3D point cloud image and the base image to obtain the first feature point set with the same name corresponding to the preliminary fused local 3D point cloud image. The second feature point set with the same name as the base map ;

[0051] C2. Repeat the following iterative process until the error Less than a preset threshold:

[0052] Make the center point of the first feature point set with the same name and the center point of the second feature point set with the same name The center points of the feature points coincide with each other, and the translation vector of the center point of the first feature point set with the same name in the coincidence operation is calculated. ;

[0053] For each point in the first feature point set with the same name, Search for its nearest neighbor in space to establish the corresponding relationship and calculate the rotation matrix between the nearest points;

[0054] The total rotation matrix, rotation vector, and the first set of feature points with the same name after rotation for the nth iteration are calculated according to the following formula:

[0055] ;

[0056] in, is the total rotation matrix, is the rotation matrix between the closest points calculated in the i-th iteration, is the rotation vector, is the first set of feature points with the same name after rotation for the nth iteration. is the initial first feature point set with the same name;

[0057] The error is calculated by the following formula :

[0058] ;

[0059] in, for The position of the i-th point in is the second feature point set with the same name The position of the i-th point in ;

[0060] Determine the error Is it less than the preset threshold?

[0061] C3. transforming each point of the preliminary fused local 3D point cloud image according to the following formula, and splicing the transformed preliminary fused local 3D point cloud image with the base map:

[0062] ;

[0063] in, is the position of the i-th point in the preliminary fused local 3D point cloud after conversion, is the position of the i-th point in the preliminary fused local 3D point cloud before conversion.

[0064] Beneficial effects:

[0065] The system and method for constructing a three-dimensional map of a mining area provided in the embodiments of the present application can, when the terrain of the mining area changes, conveniently and quickly obtain a global three-dimensional point cloud map of the mining area as a base map through a drone, and then fuse it with the local three-dimensional point cloud map obtained in real time by the roadside sensing terminal and the mining car to obtain a global three-dimensional point cloud map. Compared with the method in the prior art of constructing a three-dimensional map by scanning with the equipment carried by the vehicle itself, the three-dimensional map construction is easier and faster, and the three-dimensional map can be updated more timely according to the changes in the terrain of the mining area; and compared with the method in the prior art of constructing a three-dimensional map by scanning with the equipment carried by the vehicle itself, a larger three-dimensional map can be constructed in a dusty environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of the structure of a mining area three-dimensional map construction system provided in an embodiment of the present application.

[0067] Figure 2 A flowchart of a system method for constructing a three-dimensional map of a mining area provided in an embodiment of the present application.

[0068] Figure 3 This is the schematic diagram of laser scanning positioning.

[0069] Figure 4 This is the effect of point cloud stitching based on the point cloud matching method of the ICP algorithm.

[0070] Figure 5 This is the first point cloud image used for point cloud stitching.

[0071] Figure 6 This is the second point cloud image used for point cloud stitching. DETAILED DESCRIPTION

[0072] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0073] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0074] The embodiments or examples provided in the disclosure below are used to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the present invention. In addition, the present invention may repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides examples of various specific processes and materials, but a person skilled in the art can be aware of the application of other processes and / or the use of other materials.

[0075] See also Figure 1 , a mining area three-dimensional map construction system provided in an embodiment of the present application includes a drone 1, a mining car 2 and a plurality of roadside sensing terminals 3, wherein the drone 1, the mining car 2 and the roadside sensing terminal 3 all include a laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit;

[0076] The drone 1 is used to obtain a global three-dimensional point cloud map of the mining area as a base map through the corresponding (i.e., its own) laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit, and send it to the mining car 2;

[0077] The roadside sensing terminal 3 is arranged on the roadside of the mining area road, and is used to obtain a first local three-dimensional point cloud map of the area where the roadside sensing terminal is located (the size of the area is determined by the effective detection range of the laser radar) through the corresponding (i.e., its own) laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit, and send it to the mining car 2;

[0078] The mining car 2 is used to obtain a second local three-dimensional point cloud map of the area where the mining car 2 is located (the size of the area is determined by the effective detection range of the laser radar) through the corresponding (i.e., its own) laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit, and fuse the first local three-dimensional point cloud map with the second local three-dimensional point cloud map to obtain a preliminary fused local three-dimensional point cloud map, and then fuse the preliminary fused local three-dimensional point cloud map with the base map to obtain a fused global three-dimensional point cloud map.

[0079] Among them, the number and specific setting positions of the roadside sensing terminals 3 can be set according to actual needs. For example, the roadside sensing terminals 3 can be set at equal intervals along the extension direction of the mining road; the roadside sensing terminals 3 can also be set at important local locations such as road bends, mine car loading points, and mine car unloading points to ensure that the mine car can more accurately follow the planned route when turning, loading, and unloading.

[0080] By using the mining area three-dimensional map construction system, when the terrain of the mining area changes, the drone 1 can quickly and conveniently obtain a global three-dimensional point cloud map of the mining area as a base map, and then fuse it with the local three-dimensional point cloud map obtained in real time by the roadside sensing terminal 3 and the mining car 2 to obtain a global three-dimensional point cloud map. Compared with the method in the prior art that only relies on the equipment carried by the vehicle itself to scan and construct a three-dimensional map, the three-dimensional map construction is easier and faster, and the three-dimensional map can be updated more timely according to the changes in the mining area terrain; and compared with the method in the prior art that only relies on the equipment carried by the vehicle itself to scan and construct a three-dimensional map, a larger range of three-dimensional maps can be constructed in a dusty environment.

[0081] In fact, although the accuracy of the base map obtained by the drone 1 is relatively low, the accuracy of the local 3D point cloud map obtained by the roadside sensing terminal 3 and the mine car 2 is high, and the local 3D point cloud map obtained by the roadside sensing terminal 3 and the mine car 2 is a 3D point cloud map of important local positions related to the driving environment. The global 3D point cloud map after the fusion of the three is sufficient to meet the automatic driving requirements of the mine car 2. The mining area 3D map construction system has the following advantages:

[0082] 1. In view of the fact that the construction of three-dimensional maps in mining areas requires the use of vehicles to collect and construct three-dimensional maps repeatedly from time to time, a solution is proposed that mainly uses local mapping of mining vehicles, supplemented by the collaboration of drones and roadside sensing terminals. This can effectively reduce the cycle of three-dimensional map construction and significantly improve the efficiency of mining operations.

[0083] 2. The base map data obtained by the drone is updated in quasi-real time through the mining car and the roadside sensing terminal, and the construction of the high-precision three-dimensional map of the mining area is divided into three parts for fusion, which can effectively avoid the limitations of using a single channel for three-dimensional map construction; in dusty environments, the complementary fusion of the mining car three-dimensional map and the roadside sensing terminal three-dimensional map can effectively expand the scope of three-dimensional map construction;

[0084] 3. Relying on the hourly updates of the roadside sensing terminal and the real-time updates of the mine car, the complementary integration can dynamically grasp the situation of the mining area in quasi-real time. Compared with the mode relying on vehicle mapping, it can effectively detect the falling rocks of the front vehicle and the surrounding landslides during the operation of the mine car, which can effectively improve the safety of the mine car operation.

[0085] It should be noted that the drone 1, the mining car 2 and the roadside sensing terminal 3 are all provided with a communication module (such as a 4G / 5G communication module, but not limited thereto), and the drone 1, the roadside sensing terminal 3 and the mining car 2 are communicatively connected via the communication module.

[0086] In some preferred embodiments, reference Figure 3 , when the UAV 1, the mining car 2 and the roadside sensing terminal 3 obtain the corresponding three-dimensional point cloud map through the corresponding (respective) laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit:

[0087] Use laser radar to obtain the coordinates of the scanning point in the laser radar coordinate system (hereinafter referred to as S system);

[0088] The coordinates of the scanning point in the geocentric rectangular coordinate system (hereinafter referred to as the E system) are calculated according to the following formula:

[0089] ;

[0090] in, is the coordinate of the scanning point in the geocentric rectangular coordinate system (E system); is the coordinate of the center of the corresponding IMU inertial measurement unit in the geocentric rectangular coordinate system, which can be obtained by the differential GNSS navigation device and the IMU inertial measurement unit through combined navigation solution; The rotation matrix of the corresponding IMU inertial measurement unit coordinate system (hereinafter referred to as I system) to the geocentric rectangular coordinate system can be formed by the attitude (Pitch, Roll, Yaw) measured by the differential GNSS navigation device and the IMU inertial measurement unit; The component of the offset from the center of the laser radar coordinate system of the laser radar to the center of the IMU inertial measurement unit in the IMU inertial measurement unit coordinate system can be measured in advance; is the rotation matrix from the laser radar coordinate system to the IMU inertial measurement unit coordinate system, which can be calibrated in advance; are the coordinates of the scanning point in the laser radar coordinate system.

[0091] The coordinates of the scanning points in the three-dimensional point cloud image constructed in the above manner are unified into coordinates in the E system to facilitate subsequent fusion.

[0092] According to the characteristic of the laser radar that the smaller the distance, the higher the accuracy, the local 3D point cloud map constructed by the mine car and the road test sensing end has a relatively small observation range, and can reach the centimeter to millimeter level. The coordinates of the scanning points in each 3D point cloud map are unified as coordinates in the E system. Therefore, the local 3D point cloud map obtained by the roadside sensing end and the 3D point cloud map obtained by the mine car can be directly spliced ​​and fused; therefore, in some embodiments, when the mine car 2 fuses the first local 3D point cloud map and the second local 3D point cloud map:

[0093] Based on the coordinates of each point in the geocentric rectangular coordinate system, the first local three-dimensional point cloud image and the second local three-dimensional point cloud image are spliced.

[0094] The coordinate values ​​of each scanning point in the E system remain unchanged after splicing.

[0095] Furthermore, due to the large scope of the mining area, the longer the laser scanning distance, the lower the accuracy of the base map construction by the drone. In addition, the angular error of the drone's flight attitude also makes it impossible to directly stitch point clouds through direct geographic reference information (the coordinates of the scanning points in the E system); therefore, in some embodiments, when the mining car 2 fuses the preliminary fused local three-dimensional point cloud map with the base map to obtain a fused global three-dimensional point cloud map:

[0096] The point cloud matching method based on the ICP algorithm is used to perform point cloud splicing on the preliminary fused local three-dimensional point cloud map and the base map.

[0097] The ICP algorithm is a least squares iterative optimization process. The main method is: first, find the feature points with the same name in the two frames of point cloud data, iteratively solve the motion parameters between the corresponding points based on the feature points, and convert the laser point clouds of different reference frames to the same coordinate system according to the motion parameters to achieve complete overlap of the parts of the two point clouds that should overlap.

[0098] Specifically, when the mining vehicle 2 uses the point cloud matching method based on the ICP algorithm to perform point cloud splicing on the preliminary fused local three-dimensional point cloud image and the base map, the following steps are performed:

[0099] C1. Select the feature points with the same name in the preliminary fused local 3D point cloud image and the base image to obtain the first feature point set with the same name corresponding to the preliminary fused local 3D point cloud image. The second feature point set with the same name as the base map ;

[0100] C2. Repeat the following iterative process until the error Less than a preset threshold:

[0101] Make the center point of the first feature point set with the same name and the center point of the second feature point set with the same name The center points of the feature points coincide with each other, and the translation vector of the center point of the first feature point set with the same name in the coincidence operation is calculated. ; The coordinates of the center point can be calculated using the following formula:

[0102] ;

[0103] Among them, X, Y, and Z are the three coordinate values ​​of the center point. , , are the three coordinate values ​​of the i-th point in the corresponding feature point set with the same name (the first feature point set with the same name or the second feature point set with the same name);

[0104] For each point in the first feature point set with the same name, Search for its nearest neighbor in space to establish a corresponding relationship and calculate the rotation matrix between the nearest points ( );

[0105] The total rotation matrix, rotation vector, and the first set of feature points with the same name after rotation for the nth iteration are calculated according to the following formula:

[0106] ;

[0107] in, is the total rotation matrix, is the rotation matrix between the closest points calculated in the i-th iteration, is the rotation vector, is the first set of feature points with the same name after rotation for the nth iteration. is the initial first feature point set with the same name;

[0108] The error is calculated by the following formula :

[0109] ;

[0110] in, for The position of the i-th point in is the second feature point set with the same name The position of the i-th point in ;

[0111] Determine the error Is it less than the preset threshold?

[0112] C3. transforming each point of the preliminary fused local 3D point cloud image according to the following formula, and splicing the transformed preliminary fused local 3D point cloud image with the base map:

[0113] ;

[0114] in, is the position of the i-th point in the preliminary fused local 3D point cloud after conversion, is the position of the i-th point in the preliminary fused local 3D point cloud before conversion.

[0115] The effect of point cloud matching based on ICP algorithm for point cloud splicing and fusion can be referred to Figure 4-6 In this example, EagleEye2000 laser radar is used to scan a corner of the same factory at different angles, and the Figure 5 , 6 The two frames of point cloud images are obtained by splicing and fusion of point cloud images using the point cloud matching method based on ICP algorithm Figure 4 The point cloud image shown has a good fusion effect.

[0116] In some embodiments, step C1 comprises:

[0117] Select multiple feature points in the preliminary fused local three-dimensional point cloud image to obtain a first feature point set with the same name corresponding to the preliminary fused local three-dimensional point cloud image. ;

[0118] For the first set of feature points with the same name For each feature point in the base map, the Kd-tree algorithm is used to search for the closest point in the base map as the corresponding feature point with the same name, and the second feature point set with the same name corresponding to the base map is obtained. .

[0119] Among them, some corner points, special marking points (for example, some spherical markers, sharp-angle markers, etc. can be set in the mining area, and the center of the spherical marker and the tip of the sharp-angle marker are special marking points) and other points can be selected as feature points. The Kd-tree algorithm is an existing technology that can accurately search for the corresponding feature points with the same name, and its detailed steps are not described here.

[0120] In some preferred embodiments, the mining vehicle 2 is further used to perform point cloud filtering and noise reduction processing on the fused global three-dimensional point cloud image, so as to further improve the accuracy of the global three-dimensional point cloud image.

[0121] Please refer to Figure 2 The present application embodiment provides a method for constructing a three-dimensional map of a mining area, comprising the steps of:

[0122] A1. Obtain a global three-dimensional point cloud map of the mining area obtained by the drone through the laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit set on the drone as a base map;

[0123] A2. Obtaining a first local three-dimensional point cloud map of the area where the roadside sensing terminal is located, obtained by the roadside sensing terminal through a laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit arranged on the roadside sensing terminal; the roadside sensing terminal is arranged on the side of the road in the mining area;

[0124] A3. Obtaining a second local three-dimensional point cloud map of the area where the mine car is located by a laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit provided on the mine car;

[0125] A4. Fusing the first local three-dimensional point cloud image and the second local three-dimensional point cloud image to obtain a preliminary fused local three-dimensional point cloud image;

[0126] A5. Fuse the initially fused local three-dimensional point cloud image with the base image to obtain a fused global three-dimensional point cloud image.

[0127] In some embodiments, the method for constructing a three-dimensional map of a mining area may be applied to the mining vehicle 2 in the above-mentioned three-dimensional map construction system of a mining area.

[0128] By using the method for constructing a three-dimensional map of a mining area, when the terrain of the mining area changes, a global three-dimensional point cloud map of the mining area can be conveniently and quickly obtained as a base map through a drone, and then fused with the local three-dimensional point cloud map obtained in real time by the roadside sensing terminal and the mining car to obtain a global three-dimensional point cloud map. Compared with the method in the prior art of constructing a three-dimensional map by scanning with the equipment carried by the vehicle itself, the three-dimensional map is easier to construct and the construction speed is faster, and the three-dimensional map can be updated more timely according to the changes in the terrain of the mining area; and compared with the method in the prior art of constructing a three-dimensional map by scanning with the equipment carried by the vehicle itself, a larger three-dimensional map can be constructed in a dusty environment.

[0129] In fact, although the accuracy of the base map obtained by the drone is relatively low, the accuracy of the local 3D point cloud map obtained by the roadside sensing terminal and the mine car is relatively high. Moreover, the local 3D point cloud map obtained by the roadside sensing terminal and the mine car is a 3D point cloud map of important local positions related to the driving environment. The global 3D point cloud map after the fusion of the three is sufficient to meet the automatic driving needs of the mine car. The mining area 3D map construction system has the following advantages:

[0130] 1. In view of the fact that the construction of three-dimensional maps in mining areas requires the use of vehicles to collect and construct three-dimensional maps repeatedly from time to time, a solution is proposed that mainly uses local mapping of mining vehicles, supplemented by the collaboration of drones and roadside sensing terminals. This can effectively reduce the cycle of three-dimensional map construction and significantly improve the efficiency of mining operations.

[0131] 2. The base map data obtained by the drone is updated in quasi-real time through the mining car and the roadside sensing terminal, and the construction of the high-precision three-dimensional map of the mining area is divided into three parts for fusion, which can effectively avoid the limitations of using a single channel for three-dimensional map construction; in dusty environments, the complementary fusion of the mining car three-dimensional map and the roadside sensing terminal three-dimensional map can effectively expand the scope of three-dimensional map construction;

[0132] 3. Relying on the hourly updates of the roadside sensing terminal and the real-time updates of the mine car, the complementary integration can dynamically grasp the situation of the mining area in quasi-real time. Compared with the mode relying on vehicle mapping, it can effectively detect the falling rocks of the front vehicle and the surrounding landslides during the operation of the mine car, which can effectively improve the safety of the mine car operation.

[0133] In some preferred embodiments, reference Figure 3 , when the UAV, mining car and roadside sensing end obtain the corresponding three-dimensional point cloud map through the corresponding (respective) laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit:

[0134] Use laser radar to obtain the coordinates of the scanning point in the laser radar coordinate system (hereinafter referred to as S system);

[0135] The coordinates of the scanning point in the geocentric rectangular coordinate system (hereinafter referred to as the E system) are calculated according to the following formula:

[0136] ;

[0137] in, is the coordinate of the scanning point in the geocentric rectangular coordinate system (E system); is the coordinate of the center of the corresponding IMU inertial measurement unit in the geocentric rectangular coordinate system, which can be obtained by the differential GNSS navigation device and the IMU inertial measurement unit through combined navigation solution; The rotation matrix of the corresponding IMU inertial measurement unit coordinate system (hereinafter referred to as I system) to the geocentric rectangular coordinate system can be formed by the attitude (Pitch, Roll, Yaw) measured by the differential GNSS navigation device and the IMU inertial measurement unit; The component of the offset from the center of the laser radar coordinate system of the laser radar to the center of the IMU inertial measurement unit in the IMU inertial measurement unit coordinate system can be measured in advance; is the rotation matrix from the laser radar coordinate system to the IMU inertial measurement unit coordinate system, which can be calibrated in advance; are the coordinates of the scanning point in the laser radar coordinate system.

[0138] The coordinates of the scanning points in the three-dimensional point cloud image constructed in the above manner are unified into coordinates in the E system to facilitate subsequent fusion.

[0139] According to the characteristic of the laser radar that the smaller the distance, the higher the accuracy, the local 3D point cloud map constructed by the mine car and the road test sensing end has a relatively small observation range and can reach the centimeter to millimeter level, and the coordinates of the scanning points in each 3D point cloud map are unified as coordinates in the E system, so the local 3D point cloud map obtained by the roadside sensing end and the 3D point cloud map obtained by the mine car can be directly spliced ​​and fused; therefore, in some embodiments, step A4 includes:

[0140] Based on the coordinates of each point in the geocentric rectangular coordinate system, the first local three-dimensional point cloud image and the second local three-dimensional point cloud image are spliced.

[0141] Furthermore, due to the large scope of the mining area, the longer the laser scanning distance, the lower the accuracy of the base map construction by the drone. In addition, the angular error of the drone's flight attitude also makes it impossible to directly stitch the point cloud through direct geographic reference information (the coordinates of the scanning points in the E system); therefore, in some embodiments, step A5 includes:

[0142] The point cloud matching method based on the ICP algorithm is used to perform point cloud splicing on the preliminary fused local three-dimensional point cloud map and the base map.

[0143] The ICP algorithm is a least squares iterative optimization process. The main method is: first, find the feature points with the same name in the two frames of point cloud data, iteratively solve the motion parameters between the corresponding points based on the feature points, and convert the laser point clouds of different reference frames to the same coordinate system according to the motion parameters to achieve complete overlap of the parts of the two point clouds that should overlap.

[0144] Specifically, the step of performing point cloud splicing on the preliminary fused local three-dimensional point cloud image and the base map by using a point cloud matching method based on an ICP algorithm includes:

[0145] C1. Select the feature points with the same name in the preliminary fused local 3D point cloud image and the base image to obtain the first feature point set with the same name corresponding to the preliminary fused local 3D point cloud image. The second feature point set with the same name as the base map ;

[0146] C2. Repeat the following iterative process until the error Less than a preset threshold:

[0147] Make the center point of the first feature point set with the same name and the center point of the second feature point set with the same name The center points of the feature points coincide with each other, and the translation vector of the center point of the first feature point set with the same name in the coincidence operation is calculated. ; The coordinates of the center point can be calculated using the following formula:

[0148] ;

[0149] Among them, X, Y, and Z are the three coordinate values ​​of the center point. , , are the three coordinate values ​​of the i-th point in the corresponding feature point set with the same name (the first feature point set with the same name or the second feature point set with the same name);

[0150] For each point in the first feature point set with the same name, Search for its nearest neighbor in space to establish a corresponding relationship and calculate the rotation matrix between the nearest points ( );

[0151] The total rotation matrix, rotation vector, and the first set of feature points with the same name after rotation for the nth iteration are calculated according to the following formula:

[0152] ;

[0153] in, is the total rotation matrix, is the rotation matrix between the closest points calculated in the i-th iteration, is the rotation vector, is the first set of feature points with the same name after rotation for the nth iteration. is the initial first feature point set with the same name;

[0154] The error is calculated by the following formula :

[0155] ;

[0156] in, for The position of the i-th point in is the second feature point set with the same name The position of the i-th point in ;

[0157] Determine the error Is it less than the preset threshold?

[0158] C3. transforming each point of the preliminary fused local 3D point cloud image according to the following formula, and splicing the transformed preliminary fused local 3D point cloud image with the base map:

[0159] ;

[0160] in, is the position of the i-th point in the preliminary fused local 3D point cloud after conversion, is the position of the i-th point in the preliminary fused local 3D point cloud before conversion.

[0161] In some embodiments, step C1 comprises:

[0162] Select multiple feature points in the preliminary fused local three-dimensional point cloud image to obtain a first feature point set with the same name corresponding to the preliminary fused local three-dimensional point cloud image. ;

[0163] For the first set of feature points with the same name For each feature point in the base map, the Kd-tree algorithm is used to search for the closest point in the base map as the corresponding feature point with the same name, and the second feature point set with the same name corresponding to the base map is obtained. .

[0164] Among them, some corner points, special marking points (for example, some spherical markers, sharp-angle markers, etc. can be set in the mining area, and the center of the spherical marker and the tip of the sharp-angle marker are special marking points) and other points can be selected as feature points. The Kd-tree algorithm is an existing technology that can accurately search for the corresponding feature points with the same name, and its detailed steps are not described here.

[0165] In some preferred embodiments, after step A5, the method further comprises the following steps:

[0166] The fused global three-dimensional point cloud image is subjected to point cloud filtering and denoising processing.

[0167] To further improve the accuracy of the global three-dimensional point cloud map.

[0168] In summary, although the present invention has been disclosed as above in terms of preferred embodiments, the above preferred embodiments are not intended to limit the present invention. A person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and the solutions are substantially the same as the present invention.

Claims

1. A mining area three-dimensional map construction system, characterized in that: It includes a drone, a mining car and multiple roadside sensing terminals, wherein the drone, the mining car and the roadside sensing terminal all include a laser radar, a differential GNSS navigation device and an IMU inertial measurement unit; The drone is used to obtain a global three-dimensional point cloud map of the mining area as a base map through corresponding laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit, and send it to the mining car; The roadside sensing terminal is arranged on the roadside of the mining area road, and is used to obtain a first local three-dimensional point cloud map of the area where the roadside sensing terminal is located through a corresponding laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit, and send it to the mining vehicle; The mine car is used to obtain a second local three-dimensional point cloud map of the area where the mine car is located through a corresponding laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit, fuse the first local three-dimensional point cloud map and the second local three-dimensional point cloud map to obtain a preliminary fused local three-dimensional point cloud map, and then fuse the preliminary fused local three-dimensional point cloud map with the base map to obtain a fused global three-dimensional point cloud map; When the UAV, mining car and roadside sensing end obtain the corresponding three-dimensional point cloud map through the corresponding laser radar, differential GNSS navigation equipment and IMU inertial measurement unit: Use laser radar to obtain the coordinates of the scanning point in the laser radar coordinate system; The coordinates of the scanning point in the geocentric rectangular coordinate system are calculated according to the following formula: ; in, is the coordinate of the scanning point in the geocentric rectangular coordinate system, is the coordinate of the center of the corresponding IMU inertial measurement unit in the geocentric rectangular coordinate system, which is obtained by the differential GNSS navigation device and the IMU inertial measurement unit through combined navigation solution. is the rotation matrix from the corresponding IMU inertial measurement unit coordinate system to the geocentric rectangular coordinate system, which is formed by the attitude measured by the differential GNSS navigation device and the IMU inertial measurement unit. is the component of the offset from the center of the laser radar coordinate system of the laser radar to the center of the IMU inertial measurement unit in the IMU inertial measurement unit coordinate system, is the rotation matrix from the laser radar coordinate system to the IMU inertial measurement unit coordinate system, is the coordinate of the scanning point in the laser radar coordinate system; When the mine car fuses the first local three-dimensional point cloud image with the second local three-dimensional point cloud image: Based on the coordinates of each point in the geocentric rectangular coordinate system, splicing the first local three-dimensional point cloud image and the second local three-dimensional point cloud image; When the mine car fuses the initially fused local three-dimensional point cloud map with the base map to obtain a fused global three-dimensional point cloud map: The point cloud matching method based on the ICP algorithm is used to perform point cloud splicing on the preliminary fused local three-dimensional point cloud map and the base map.

2. The mining area three-dimensional map construction system according to claim 1, characterized in that: When the mine car uses the point cloud matching method based on the ICP algorithm to perform point cloud splicing on the preliminary fused local three-dimensional point cloud map and the base map, the following steps are performed: C1. Select the feature points with the same name in the preliminary fused local 3D point cloud image and the base image to obtain the first feature point set with the same name corresponding to the preliminary fused local 3D point cloud image. The second feature point set with the same name as the base map ; C2. Repeat the following iterative process until the error Less than a preset threshold: Make the center point of the first feature point set with the same name and the center point of the second feature point set with the same name The center points of the feature points coincide with each other, and the translation vector of the center point of the first feature point set with the same name in the coincidence operation is calculated. ; For each point in the first feature point set with the same name, Search for its nearest neighbor in space to establish the corresponding relationship and calculate the rotation matrix between the nearest points; The total rotation matrix, rotation vector, and the first set of feature points with the same name after rotation for the nth iteration are calculated according to the following formula: ; in, is the total rotation matrix, is the rotation matrix between the closest points calculated in the i-th iteration, is the rotation vector, is the first set of feature points with the same name after rotation for the nth iteration. is the initial first set of feature points with the same name; The error is calculated by the following formula : ; in, for The position of the i-th point in is the second feature point set with the same name The position of the i-th point in ; Determine the error Is it less than the preset threshold? C3. transforming each point of the preliminary fused local 3D point cloud image according to the following formula, and splicing the transformed preliminary fused local 3D point cloud image with the base map: ; in, is the position of the i-th point in the preliminary fused local 3D point cloud after conversion, is the position of the i-th point in the preliminary fused local 3D point cloud before conversion.

3. The mining area three-dimensional map construction system according to claim 2, characterized in that: Step C1 includes: Select multiple feature points in the preliminary fused local three-dimensional point cloud image to obtain a first feature point set with the same name corresponding to the preliminary fused local three-dimensional point cloud image. ; For the first set of feature points with the same name For each feature point in the base map, the Kd-tree algorithm is used to search for the closest point in the base map as the corresponding feature point with the same name, and the second feature point set with the same name corresponding to the base map is obtained. .

4. A method for constructing a three-dimensional map of a mining area, characterized in that: Includes steps: A1. Obtain a global three-dimensional point cloud map of the mining area obtained by the drone through the laser radar, differential GNSS navigation equipment, and IMU inertial measurement unit set on the drone as a base map; A2. Obtaining a first local three-dimensional point cloud map of the area where the roadside sensing terminal is located, obtained by the roadside sensing terminal through a laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit arranged on the roadside sensing terminal; the roadside sensing terminal is arranged on the side of the road in the mining area; A3. Obtaining a second local three-dimensional point cloud map of the area where the mine car is located by a laser radar, a differential GNSS navigation device, and an IMU inertial measurement unit provided on the mine car; A4. Fusing the first local three-dimensional point cloud image and the second local three-dimensional point cloud image to obtain a preliminary fused local three-dimensional point cloud image; A5. Fusing the preliminary fused local three-dimensional point cloud map with the base map to obtain a fused global three-dimensional point cloud map; When the UAV, mining car and roadside sensing end obtain the corresponding three-dimensional point cloud map through the corresponding laser radar, differential GNSS navigation equipment and IMU inertial measurement unit: Use laser radar to obtain the coordinates of the scanning point in the laser radar coordinate system; The coordinates of the scanning point in the geocentric rectangular coordinate system are calculated according to the following formula: ; in, is the coordinate of the scanning point in the geocentric rectangular coordinate system, is the coordinate of the center of the corresponding IMU inertial measurement unit in the geocentric rectangular coordinate system, which is obtained by the differential GNSS navigation device and the IMU inertial measurement unit through combined navigation solution. is the rotation matrix from the corresponding IMU inertial measurement unit coordinate system to the geocentric rectangular coordinate system, which is formed by the attitude measured by the differential GNSS navigation device and the IMU inertial measurement unit. is the component of the offset from the center of the laser radar coordinate system of the laser radar to the center of the IMU inertial measurement unit in the IMU inertial measurement unit coordinate system, is the rotation matrix from the laser radar coordinate system to the IMU inertial measurement unit coordinate system, is the coordinate of the scanning point in the laser radar coordinate system; Step A4 includes: Based on the coordinates of each point in the geocentric rectangular coordinate system, splicing the first local three-dimensional point cloud image and the second local three-dimensional point cloud image; Step A5 includes: The point cloud matching method based on the ICP algorithm is used to perform point cloud splicing on the preliminary fused local three-dimensional point cloud image and the base map.

5. The method for constructing a three-dimensional map of a mining area according to claim 4, characterized in that: The step of performing point cloud splicing on the preliminary fused local three-dimensional point cloud image and the base map by using a point cloud matching method based on the ICP algorithm comprises: C1. Select the feature points with the same name in the preliminary fused local 3D point cloud image and the base image to obtain the first feature point set with the same name corresponding to the preliminary fused local 3D point cloud image. The second feature point set with the same name as the base map ; C2. Repeat the following iterative process until the error Less than a preset threshold: Make the center point of the first feature point set with the same name and the center point of the second feature point set with the same name The center points of the feature points coincide with each other, and the translation vector of the center point of the first feature point set with the same name in the coincidence operation is calculated. ; For each point in the first feature point set with the same name, Search for its nearest neighbor in space to establish the corresponding relationship and calculate the rotation matrix between the nearest points; The total rotation matrix, rotation vector, and the first set of feature points with the same name after rotation for the nth iteration are calculated according to the following formula: ; in, is the total rotation matrix, is the rotation matrix between the closest points calculated in the i-th iteration, is the rotation vector, is the first set of feature points with the same name after rotation for the nth iteration. is the initial first feature point set with the same name; The error is calculated by the following formula : ; in, for The position of the i-th point in is the second feature point set with the same name The position of the i-th point in ; Determine the error Is it less than the preset threshold? C3. transforming each point of the preliminary fused local 3D point cloud image according to the following formula, and splicing the transformed preliminary fused local 3D point cloud image with the base map: ; in, is the position of the i-th point in the preliminary fused local 3D point cloud after conversion, is the position of the i-th point in the preliminary fused local 3D point cloud before conversion.