A mine area drivable area boundary updating method based on visual three-dimensional reconstruction

Abstract

The application discloses a mine area drivable area boundary updating method based on visual three-dimensional reconstruction, and collects mine area boundary data rapidly through visual three-dimensional reconstruction. Since the boundary of the mine area has the characteristics of high frequency change, the unmanned driving of the mine area requires that the boundary data has certain real-time performance. Therefore, in order to improve the collection efficiency of the boundary of the loading area, the dump and other working areas in the mine area, a new collection updating scheme is provided. A high-resolution camera is erected around the working area as a data collection end, the original data collected is a multi-view image, a visual three-dimensional reconstruction is combined as a spatial relationship restoration post-processing scheme, and real coordinates are converted through aerial survey data to realize data positioning.

Description

Technical Field

[0001] This invention relates to the field of high-precision map acquisition technology for unmanned mining areas, and in particular to a method for updating the boundary of drivable areas in mining areas based on visual 3D reconstruction. Background Technology

[0002] Autonomous driving refers to intelligent driverless control technology where vehicles perceive the road environment through onboard sensor systems, rely on computer systems to autonomously plan routes, and reach their destination. It utilizes onboard sensors to perceive the vehicle's surroundings and, based on the perceived road, vehicle position, and obstacle information, controls the vehicle's steering and speed, enabling safe and reliable driving on roads. Autonomous driving integrates numerous technologies such as automatic control, system architecture, artificial intelligence, and visual computing, and is a product of the advanced development of computer science, pattern recognition, and intelligent control technology. Unmanned driving in mining areas is a type of autonomous driving, applied to specific locations and with specific vehicles to achieve specific purposes. Mining areas specifically refer to open-pit mines, such as iron ore and coal mines, and utilize large mining trucks and wide-body vehicles to achieve unmanned driving and unmanned mineral transportation in mining areas.

[0003] Photogrammetry is a science and technology that uses optical cameras to capture images and then uses these images to study and determine the shape, size, position, and relationships of objects. It includes acquiring images of objects, studying methods for processing single or multiple images (including theory, equipment, and techniques), and methods and equipment for outputting the measured results in graphical or digital form. Its main tasks are to produce topographic maps of various scales, establish topographic databases, and provide basic surveying data for geographic information systems and various engineering applications.

[0004] 3D reconstruction refers to establishing a mathematical model of a 3D object suitable for computer representation and processing. It is the foundation for processing, manipulating, and analyzing the properties of objects in a computer environment, and a key technology for creating virtual reality representations of the objective world within a computer. In computer vision, 3D reconstruction refers to the process of reconstructing 3D information from single-view or multi-view images. Because single-view information is incomplete, 3D reconstruction requires the use of empirical knowledge. Multi-view 3D reconstruction (similar to human binocular positioning) is relatively easier. The method involves first calibrating the camera, i.e., calculating the relationship between the camera's image coordinate system and the world coordinate system, and then reconstructing the 3D information using information from multiple 2D images.

[0005] Visual SLAM (Simultaneous Localization and Mapping) refers to a method of determining the position of robots, vehicles, or moving cameras in an environment using only external information such as images, while simultaneously creating a representation (map) of the explored area. Compared to photogrammetry and 3D reconstruction, visual SLAM places greater emphasis on real-time performance.

[0006] The three technologies mentioned above share the same underlying theory, all requiring camera models, multi-view feature point matching, and business intelligence (BA) optimization, but with different focuses.

[0007] While traditional aerial surveying techniques can yield good data quality, they are limited by acquisition conditions, primarily flight weather and the timeliness of post-processing, thus failing to meet requirements. Visual SLAM solutions also face practical challenges, such as the need for continuous movement of the camera carrier and the need for further development of the technology in this area, making it difficult to achieve ideal results.

[0008] For example, RTK measurement solutions require continuous instrument setup and data collection from unmanned vehicles traveling on-site, resulting in poor safety performance. Vehicle-mounted GPS boundary-finding data collection solutions require the data collection vehicle to be ready at all times, and data collection takes time. The mixing of data collection vehicles and unmanned mining trucks in the same area poses challenges to the safety and controllability of unmanned vehicles. Vehicle-mounted LiDAR scanning solutions suffer from the same drawbacks as vehicle-mounted GPS solutions, and the relatively high cost of LiDAR is also a factor to consider.

[0009] To address this issue, a method for updating the boundary of drivable areas in mining areas based on visual 3D reconstruction is proposed. Summary of the Invention

[0010] In view of this, the present invention provides a method for updating the boundary of drivable areas in mining areas based on visual 3D reconstruction, so as to improve the acquisition efficiency of the boundary of working areas such as loading areas and spoil heaps in mining areas.

[0011] This invention provides a method for updating the boundary of drivable areas in mining areas based on visual 3D reconstruction, comprising the following steps:

[0012] S1: Set up multiple high-resolution cameras around the work area to be measured, and calibrate the interior and exterior orientation elements of each camera.

[0013] S2: Transmit multi-view image data collected by multiple cameras to the cloud data processing platform in real time, and store the multi-view image data on the cloud data processing platform according to the collection time and camera ID;

[0014] S3: Eliminate dynamic objects in multi-view images from the same viewpoint, and retain static objects in multi-view images from the same viewpoint;

[0015] S4: Perform sparse reconstruction (SFM), dense reconstruction (MVS), and point cloud modeling sequentially on multi-view images containing only static objects.

[0016] S5: Using the point cloud method to set elevation thresholds, solve the boundary of the work area to be measured, and transform the coordinate system of the obtained boundary into the geographic coordinate system;

[0017] S6: Merge the existing boundary of the work area to be measured with the boundary of the work area to be measured in the geographic coordinate system obtained in step S5, and update the boundary of the work area to be measured; wherein, the existing boundary of the work area to be measured is the result of the previous update.

[0018] In one possible implementation, in the above-mentioned method for updating the boundary of drivable mining area based on visual 3D reconstruction provided by the present invention, step S4, sparse reconstruction SFM is performed on the multi-view image containing only static objects, specifically including:

[0019] S41: Use the SIFT algorithm to perform feature point detection and feature point matching on multi-view images containing only static objects;

[0020] S42: Retain the condition that satisfies x1 T Feature point matching pairs with the condition Fx2=0 are used to construct an epipolar geometry; where matrix F is the fundamental matrix in computer vision, and x1 and x2 represent the homogeneous coordinates of the feature points in the two paired images, respectively;

[0021] S43: Obtain the pose of each camera based on the epipolar geometry;

[0022] S44: Perform bundled adjustment and optimization of the pose of each camera.

[0023] In one possible implementation, in the above-mentioned method for updating the boundary of drivable mining area based on visual 3D reconstruction provided by the present invention, step S41, which uses the SIFT algorithm to perform feature point detection and feature point matching on a multi-view image containing only static objects, specifically includes:

[0024] SS1: In multi-view images containing only static objects, a Gaussian difference scale space is created for each image, and key points are detected at each scale in the scale space of each image using the difference of Gaussians function.

[0025] SS2: Remove keypoints with contrast below the threshold and keypoints with unstable principal curvature in the edge gradient direction. By fitting a fine model, determine the location and scale of the remaining keypoints.

[0026] SS3: Calculate the orientation of the retained keypoints based on the local gradient, and assign one or more orientations to the retained keypoints;

[0027] SS4: Using the position, scale, and orientation of the retained keypoints, calculate the keypoint descriptors to describe the keypoints. The retained keypoints and their corresponding keypoint descriptors constitute feature points.

[0028] In one possible implementation, in the above-mentioned method for updating the boundary of a drivable area in a mining area based on visual 3D reconstruction provided by the present invention, step S5, which uses a point cloud to set an elevation threshold, solves for the boundary of the work area to be measured, specifically includes:

[0029] S51: Use dense point clouds as input data;

[0030] S52: Obtain the minimum elevation value of the input point cloud data;

[0031] S53: Based on the obtained minimum elevation value, calculate the difference between the elevation values ​​of other points and the minimum elevation value;

[0032] S54: Determine whether the elevation difference between each point and its surrounding points is greater than the threshold; if yes, then the point has undergone a sudden change; if no, then the point has not undergone a sudden change.

[0033] S55: Connect the points where abrupt changes occur to form a line, which serves as the boundary of the work area to be measured.

[0034] The above-mentioned method for updating the boundary of drivable areas in mining areas based on visual 3D reconstruction provided by this invention rapidly collects boundary data of mining areas through visual 3D reconstruction. Since the boundaries of mining areas change frequently, unmanned driving in mining areas requires boundary data to have a certain degree of real-time performance. Therefore, to improve the efficiency of boundary data collection for working areas such as loading areas and spoil heaps in mining areas, this invention proposes a new data collection and update scheme. High-resolution cameras are set up around the work area as data acquisition terminals. The raw data collected are multi-view images. Visual 3D reconstruction is used as a post-processing scheme to restore spatial relationships, and aerial survey data is used to convert to real coordinates to achieve data positioning.

[0035] This solution has the advantages of low cost, rapid boundary updates, non-invasive data collection, and high security, and can well meet the needs of mining area boundaries. Attached Figure Description

[0036] Figure 1 This is a flowchart of a method for updating the boundary of a drivable area in a mining area based on visual 3D reconstruction according to the present invention.

[0037] Figure 2 This is a schematic diagram showing the camera of the present invention mounted in the loading area.

[0038] Figure 3 This is a schematic diagram of the field of view of the camera installed according to the present invention;

[0039] Figure 4 This is a schematic diagram illustrating the four coordinate systems involved in the image processing of this invention;

[0040] Figure 5 This is a schematic diagram illustrating the transformation between the image coordinate system and the camera coordinate system of the present invention;

[0041] Figure 6 This is a schematic diagram illustrating the transformation between the pixel coordinate system and the image coordinate system of the present invention;

[0042] Figure 7 This is a schematic diagram of the photo data transmission link between the camera, signal tower, and data processing platform of the present invention.

[0043] Figure 8 This is a before-and-after comparison image of filtering out moving obstacles in the image processing of this invention;

[0044] Figure 9 This is a multi-view geometric schematic diagram of the present invention;

[0045] Figure 10 This is a flowchart illustrating the process of obtaining the boundary in this invention;

[0046] Figure 11 This is a schematic diagram comparing the old and new boundaries of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely illustrative and are not intended to limit the present invention.

[0048] To improve the efficiency of boundary data acquisition for working areas such as loading areas and spoil heaps within the mining area, two specific requirements are put forward for the boundary data acquisition process of these areas: first, the acquisition should be characterized by high frequency of changes; second, the boundary data acquisition should have a certain degree of real-time capability.

[0049] Based on this, the present invention provides a method for updating the boundary of drivable areas in mining areas based on visual 3D reconstruction, such as... Figure 1 As shown, it includes the following steps:

[0050] S1: Set up multiple high-resolution cameras around the work area to be measured, and calibrate the interior and exterior orientation elements of each camera.

[0051] S2: Transmit multi-view image data collected by multiple cameras to the cloud data processing platform in real time, and store the multi-view image data on the cloud data processing platform according to the collection time and camera ID;

[0052] S3: Eliminate dynamic objects in multi-view images from the same viewpoint, and retain static objects in multi-view images from the same viewpoint;

[0053] S4: Perform sparse reconstruction (SFM), dense reconstruction (MVS), and point cloud modeling sequentially on multi-view images containing only static objects.

[0054] S5: Using the point cloud method to set elevation thresholds, solve the boundary of the work area to be measured, and transform the coordinate system of the obtained boundary into the geographic coordinate system;

[0055] S6: Merge the existing boundary of the work area to be measured with the boundary of the work area to be measured in the geographic coordinate system obtained in step S5, and update the boundary of the work area to be measured; wherein, the existing boundary of the work area to be measured is the result of the previous update.

[0056] In specific implementation, when performing step S4 of the above-mentioned method for updating the boundary of drivable mining area based on visual 3D reconstruction provided by the present invention, sparse reconstruction SFM is performed on the multi-view image containing only static objects, which may specifically include the following steps:

[0057] S41: Use the SIFT algorithm to perform feature point detection and feature point matching on multi-view images containing only static objects;

[0058] S42: Retain the condition that satisfies x1 T Feature point matching pairs with the condition Fx2=0 are used to construct an epipolar geometry; where matrix F is the fundamental matrix in computer vision, and x1 and x2 represent the homogeneous coordinates of the feature points in the two paired images, respectively;

[0059] S43: Obtain the pose of each camera based on the epipolar geometry;

[0060] S44: Perform bundled adjustment and optimization of the pose of each camera.

[0061] In specific implementation, when performing step S41 of the above-mentioned method for updating the boundary of drivable mining area based on visual 3D reconstruction provided by the present invention, the following steps may be included:

[0062] SS1: In multi-view images containing only static objects, a Gaussian difference scale space is created for each image, and key points are detected at each scale in the scale space of each image using the difference of Gaussians function.

[0063] SS2: Remove keypoints with contrast below the threshold and keypoints with unstable principal curvature in the edge gradient direction. By fitting a fine model, determine the location and scale of the remaining keypoints.

[0064] SS3: Calculate the orientation of the retained keypoints based on the local gradient, and assign one or more orientations to the retained keypoints;

[0065] SS4: Using the position, scale, and orientation of the retained keypoints, calculate the keypoint descriptors to describe the keypoints. The retained keypoints and their corresponding keypoint descriptors constitute feature points.

[0066] In specific implementation, when performing step S5 of the above-mentioned method for updating the boundary of a drivable mining area based on visual 3D reconstruction provided by the present invention, the boundary of the work area to be measured is solved by using the point cloud to set an elevation threshold. Specifically, this may include the following steps:

[0067] S51: Use dense point clouds as input data;

[0068] S52: Obtain the minimum elevation value of the input point cloud data;

[0069] S53: Based on the obtained minimum elevation value, calculate the difference between the elevation values ​​of other points and the minimum elevation value;

[0070] S54: Determine whether the elevation difference between each point and its surrounding points is greater than the threshold; if yes, then the point has undergone a sudden change; if no, then the point has not undergone a sudden change.

[0071] S55: Connect the points where abrupt changes occur to form a line, which serves as the boundary of the work area to be measured.

[0072] A working area refers to a specific area within a particular open-pit mine, serving as a site for different operational purposes during mineral extraction. Based on operational purpose, open-pit mines are divided into: a loading area (primarily for loading minerals); an unloading area (for unloading transported minerals from trucks); a crushing station (for breaking naturally mined mineral blocks into smaller pieces); a spoil heap (for dumping auxiliary soil); a parking lot (for parking vehicles); a repair shop (for vehicle repair); and a refueling area (for unmanned refueling). The working area boundary refers to the effective extent of the working area, used to distinguish between working and non-working areas. Working area boundaries are generally categorized by material into artificially constructed boundaries resembling walls and naturally formed, clearly distinguishable boundaries. Based on the degree of boundary variation, they can be further divided into fixed working area boundaries and variable working area boundaries. The boundaries of the loading area, partial unloading area, and spoil heap are dynamically variable working area boundaries. Maintaining the timeliness and real-time nature of this type of boundary data is the subject of this invention.

[0073] The following is a detailed description of the specific implementation of the above-mentioned method for updating the boundary of drivable mining area based on visual 3D reconstruction provided by the present invention through a specific embodiment.

[0074] Example 1: The loading area is a typical dynamic boundary scenario. This example uses the loading area as the working area to be measured to illustrate the concept in detail.

[0075] Step 1: Set up a high-resolution camera.

[0076] like Figure 2 The image shows the loading area of ​​a mining district. What needs to be collected and identified is the natural boundary continuously excavated by the electric shovel. Figure 2 The white dots 1-4 indicate the camera mounting locations. Camera mounting requirements: 1. The number of cameras should ideally cover the electric shovel's data acquisition and advancement line. This embodiment uses four cameras. Based on the camera's FOV (Field of View), it is sufficient to cover the entire data acquisition and advancement line. Figure 3 As shown; 2. Each camera must ensure 60% image overlap; 3. The camera should be mounted on one side of the fixed boundary of the loading area; 4. The camera should be mounted on a solid foundation and have a certain height, and be stable and reliable.

[0077] Step 2: Camera calibration.

[0078] Cameras primarily use the pinhole imaging principle to create images. To obtain the true three-dimensional coordinates from the image, a transformation between four coordinate systems is required: world coordinate system O... w -X w Y w Z w Camera coordinate system O c -X c Y c Z c Pixel coordinate system o-xy and image coordinate system o-uv, such as Figure 4 As shown. Figure 4 Midpoint P(X) w ,Y w Z w () represents a point in the world coordinate system, that is, a real point in real life. World coordinate system O w -X w Y w Z W Used to describe the camera's location, in geographic projected coordinates. Camera coordinate system O c -X c Y c Z c The origin of the o-uv coordinate system is the optical center of the camera, and the unit is meters (m). The origin of the o-uv pixel coordinate system is the top-left corner of the image, and the unit is pixels. The origin of the o-xy image coordinate system is the midpoint of the imaging plane, and the unit is millimeters (mm).

[0079] The formula for transforming the world coordinate system to the camera coordinate system is as follows:

[0080]

[0081] Among them, X c The x-coordinate of the camera coordinate system, Y... c Z represents the ordinate of the camera coordinate system. c The x-axis represents the height of the camera coordinate system. w The x-coordinate of the world coordinate system, y w The z-coordinate represents the ordinate in the world coordinate system. w R represents the height of the world coordinate system, R represents the camera's rotation matrix, and t represents the camera's translation vector.

[0082] like Figure 5 As shown, P(Xc,Yc,Zc) is a point in the camera coordinate system, and P(x,y) is a point in the image coordinate system. The formula for transforming the camera coordinate system into the image coordinate system is as follows:

[0083]

[0084] Where x represents the horizontal coordinate of the image coordinate system, y represents the vertical coordinate of the image coordinate system, and f represents the focal length of the camera.

[0085] like Figure 6 As shown, The formula for converting pixel coordinates to image coordinates is as follows:

[0086]

[0087] Where u represents the x-coordinate of the pixel in the pixel coordinate system, v represents the y-coordinate of the pixel in the pixel coordinate system, u0 represents the x-coordinate of the origin of the image coordinate system in the pixel coordinate system, and v0 represents the y-coordinate of the origin of the image coordinate system in the pixel coordinate system.

[0088] From the formula above, we can deduce that nine parameters are needed to complete the aforementioned series of transformations. Camera calibration involves determining these nine parameters. These nine parameters consist of three interior orientation elements and six exterior orientation elements. Camera calibration methods include: traditional camera calibration methods, active vision camera calibration methods, and camera self-calibration methods.

[0089] Step 3: Data is transmitted back to the platform.

[0090] Multi-view image data collected by multiple cameras is transmitted in real time to a cloud data processing platform. The multi-view image data is then stored on the cloud data processing platform according to the acquisition time and camera ID. Figure 7 As shown.

[0091] Step 4: Data preprocessing.

[0092] The process filters out moving objects from a series of photos taken from the same angle, retaining only static objects. The main principle is that for continuously captured time-lapse photos, moving vehicles will only appear in certain areas of multiple photos, while static objects will appear in every photo. By comparing the pixels of multiple photos, we can retain the static images, then composite the photos to output a photo containing only static images. The reason for this data preprocessing is primarily to remove moving objects. The images before and after preprocessing are shown below. Figure 8 As shown in (a) and (b) in the figure.

[0093] Step 5: Sparse reconstruction (SFM).

[0094] (1) Use the SIFT algorithm to perform feature point detection and feature point matching on multi-view images containing only static objects.

[0095] Specifically, from a multi-view geometry perspective, a single point of the same object will appear in images from multiple viewpoints. Finding points with the same name from multiple viewpoints is called feature point matching, such as... Figure 9 As shown.

[0096] Figure 9 Explanation: For Figure 9 The cube in the image should have 8 corners. Figure 9 Only 7 corners are visible, namely X 1 X 2 X 3 X 4 X 5 X 6 and X 7 Taking photos of the cube from different angles yields three images: image1, image2, and image3. For the X... 1 The corresponding points in the three photos are in image1. In image2 Image3 because They all correspond to the X of a cube in real-world objects. 1 Therefore, these three points are called points with the same name.

[0097] The specific steps are as follows: First, in multi-view images containing only static objects, a Gaussian difference scale space is created for each image. At each scale in the scale space of each image, keypoints are detected using the Gaussian difference function. Then, keypoints with contrast below a threshold and keypoints with unstable principal curvature in the edge gradient direction are deleted. By fitting a refined model, the position and scale of the retained keypoints are determined. Next, the direction of the retained keypoints based on the local gradient is calculated, and one or more directions are assigned to the retained keypoints. Finally, using the position, scale, and direction of the retained keypoints, keypoint descriptors are calculated to describe the keypoints. The retained keypoints and their corresponding keypoint descriptors constitute feature points.

[0098] (2) Retain those that satisfy x1 T Feature point matching pairs with the condition Fx2=0 are used to construct an epipolar geometry; where matrix F is the fundamental matrix in computer vision, and x1 and x2 represent the homogeneous coordinates of the feature points in the two paired images, respectively.

[0099] (3) Obtain the pose of each camera based on the epipolar geometry.

[0100] (4) Perform pose optimization (BA) on each camera.

[0101] Step 6: Densely rebuild MVS.

[0102] MVS stands for Multi View Stereo. SFM generates sparse point clouds and restores the geometric relationships between cameras, while MVS generates dense point clouds. SFM generates some camera parameters and geometric relationships between cameras, and MVS is a refinement of the above steps to produce so-called dense reconstruction.

[0103] MVS algorithms are mainly divided into three types: voxel-based MVS, feature point growing based MVS, and depth-map merging based MVS. This embodiment adopts the feature point growing based MVS method.

[0104] Step 7: Point cloud modeling.

[0105] The design principle of the elevation extraction algorithm for the work area boundary is that the boundary of the work area often has significant abrupt changes, which can be solved by setting an elevation threshold using point clouds, such as... Figure 10 As shown, the specific method and process are as follows:

[0106] (1) Use dense point clouds as input data.

[0107] (2) Obtain the minimum elevation value of the input point cloud data.

[0108] (3) Based on the obtained minimum elevation value, calculate the difference between the elevation values ​​of other points and the minimum elevation value.

[0109] (4) Determine whether the elevation difference between each point and the surrounding points is greater than the threshold; if yes, the point has undergone a sudden change; if no, the point has not undergone a sudden change.

[0110] (5) Connect the points where abrupt changes occur to form a line, which serves as the boundary of the work area to be measured.

[0111] Step 8: Coordinate system transformation.

[0112] Transform the coordinate system of the boundary obtained in step 7 into a geographic coordinate system.

[0113] Step 9: Data fusion and boundary update.

[0114] The existing boundaries of the work area to be measured (such as...) Figure 11 The solid line shown) and the boundary of the work area to be measured in the geographic coordinate system obtained in step eight (such as...) Figure 11 The dashed lines shown merge, as... Figure 11 As shown, the principle of the fusion algorithm is to cut line segments and update the boundary of the work area to be measured. The existing boundary of the work area to be measured is the result of the previous update.

[0115] The above-mentioned method for updating the boundary of drivable mining area based on visual 3D reconstruction provided by the present invention has the following seven advantages:

[0116] (1) Safety. The solution using computer vision is a non-contact data acquisition method. It does not require on-site positioning and measurement, nor does it require sending data acquisition vehicles to the site for tracking or radar scanning. Therefore, there is no risk of collision between mining trucks in normal operation and vehicles engaged in data acquisition, which greatly improves safety and significantly reduces the complexity of the overall unmanned driving scheme for the entire mining area.

[0117] (2) Data continuity (i.e., resistance to weather interference). This mainly refers to the stability of the collected data, which can be collected in most mining environments and has a certain degree of resistance to interference. Mining sites generally have harsh conditions, such as strong winds and low temperatures. UAVs cannot take off and collect data in mining sites where conditions are unsuitable. This invention is wind-resistant and low-temperature resistant.

[0118] (3) Data accuracy. This mainly refers to two aspects: first, data density, i.e., resolution; and second, the degree of closeness between the collected data values ​​and the true values. Since the drivable area boundary is a means of ensuring safe driving, in autonomous driving, the area within the drivable area boundary is a safe zone. The drivable area can provide prior map data for dynamic path planning in dynamic docking scenarios. Based on these objectives, the boundary data does not need to have very high accuracy or resolution. Based on this objective, this invention can effectively ensure the accuracy required for the data's intended use, without being too low or too high in accuracy.

[0119] (4) Boundary update speed. This mainly refers to the time spent from the change of boundary data on site to the generation of new boundary data. In existing solutions, manual data collection, UAV aerial photography, and lidar scanning all require the scheduling and dispatch of data collection equipment, and efforts should be made to avoid interfering with the unmanned mining trucks that are operating normally. Therefore, the real-time nature of the update cannot be guaranteed. In this invention, since the high-resolution camera is set up on site in advance, there are no dispatch issues or obstacle avoidance considerations. It is only necessary to transmit the digital image data before and after the change. Therefore, this invention is significantly better than existing solutions in terms of update speed, with a fast, controllable, and quantifiable update time.

[0120] (5) Cost. This mainly refers to the financial costs of the hardware and software required to achieve the target of the drivable area boundary. In existing solutions, the financial costs mainly consist of three parts: acquisition equipment costs, labor costs, and equipment operating costs. Acquisition equipment costs refer to the cost of purchasing acquisition equipment, such as RTK, drones, and LiDAR, since existing acquisition solutions all require the support of acquisition equipment. Labor costs refer to the fact that existing solutions require technicians with certain professional knowledge to operate the acquisition equipment, and these personnel need certain salaries. Equipment operating costs refer to the fact that the normal operation of the acquisition equipment requires certain auxiliary support, and it should be noted here that vehicle-mounted LiDAR needs to be refueled regularly. Considering all aspects, the camera of this invention has certain advantages in these three aspects.

[0121] (6) Measurement range. This mainly refers to the effective mapping range without moving the measuring equipment. UAVs are suitable for large-scale measurements, while RTK manual measurements require continuous point marking, which can guarantee the range but makes it difficult to guarantee data resolution. The effective detection range of lidar is generally 60m to 70m, while the vision technology of this invention can overcome the above problems, depending on the camera resolution and baseline distance.

[0122] (7) Automation level (i.e., human intervention). This mainly refers to the degree of human intervention. Since the existing data collection schemes involve a relatively large proportion of human intervention, it inevitably affects the real-time data update requirements of autonomous driving to a certain extent. This invention has very little human intervention, a high degree of automation, and greatly improves operational efficiency.

[0123] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for updating the boundary of drivable areas in mining areas based on visual 3D reconstruction, characterized in that, Includes the following steps: S1: Set up multiple high-resolution cameras around the work area to be measured, and calibrate the interior and exterior orientation elements of each camera. S2: Transmit multi-view image data collected by multiple cameras to the cloud data processing platform in real time, and store the multi-view image data on the cloud data processing platform according to the collection time and camera ID; S3: Eliminate dynamic objects in multi-view images from the same viewpoint, and retain static objects in multi-view images from the same viewpoint; S4: Perform sparse reconstruction (SFM), dense reconstruction (MVS), and point cloud modeling sequentially on multi-view images containing only static objects. S5: Using the point cloud method to set elevation thresholds, solve the boundary of the work area to be measured, and transform the coordinate system of the obtained boundary into the geographic coordinate system; S6: Merge the existing boundary of the work area to be measured with the boundary of the work area to be measured in the geographic coordinate system obtained in step S5, and update the boundary of the work area to be measured; wherein, the existing boundary of the work area to be measured is the result of the previous update; In step S4, sparse reconstruction SFM is performed on the multi-view image containing only static objects, specifically including: S41: Use the SIFT algorithm to perform feature point detection and feature point matching on multi-view images containing only static objects; S42: Retain satisfaction Conditional feature point matching pairs are used to construct the epipolar geometry; where the matrix F As a fundamental matrix in computer vision, and These represent the homogeneous coordinates of the feature points in the two paired images, respectively; S43: Obtain the pose of each camera based on the epipolar geometry; S44: Perform bundled adjustment and optimization of the pose of each camera; In step S5, the boundary of the work area to be measured is solved by setting an elevation threshold using point cloud data. Specifically, this includes: S51: Use dense point clouds as input data; S52: Obtain the minimum elevation value of the input point cloud data; S53: Based on the obtained minimum elevation value, calculate the difference between the elevation values ​​of other points and the minimum elevation value; S54: Determine whether the elevation difference between each point and its surrounding points is greater than the threshold; if yes, then the point has undergone a sudden change; if no, then the point has not undergone a sudden change. S55: Connect the points where abrupt changes occur to form a line, which serves as the boundary of the work area to be measured.

2. The method for updating the boundary of drivable areas in mining areas based on visual 3D reconstruction as described in claim 1, characterized in that, Step S41: Use the SIFT algorithm to perform feature point detection and feature point matching on multi-view images containing only static objects, specifically including: SS1: In multi-view images containing only static objects, a Gaussian difference scale space is created for each image, and key points are detected at each scale in the scale space of each image using the difference of Gaussians function. SS2: Remove keypoints with contrast below the threshold and keypoints with unstable principal curvature in the edge gradient direction. By fitting a fine model, determine the location and scale of the remaining keypoints. SS3: Calculate the orientation of the retained keypoints based on the local gradient, and assign one or more orientations to the retained keypoints; SS4: Using the position, scale, and orientation of the retained keypoints, calculate the keypoint descriptors to describe the keypoints. The retained keypoints and their corresponding keypoint descriptors constitute feature points.

Images