A real-time construction method of three-dimensional terrain of open-pit mine based on video monitoring

Abstract

The present application relates to a kind of open-pit mine three-dimensional terrain real-time construction method based on video monitoring, comprising: S1, in target mine layout fixed target control point, obtain target mine image;S2, to target mine image matching feature points, according to matching feature points and initial camera parameter, obtain initial sparse three-dimensional coordinate point;S3, according to target control point to initial sparse three-dimensional coordinate point is optimized, obtain sparse ground three-dimensional coordinate point;S4, according to sparse ground three-dimensional coordinate point constructs seed face sheet, to seed face sheet is diffused, obtain new seed face sheet;S5, to new seed face sheet is filtered, the scale of target mine image is reselected, obtain low resolution image, return S4 until reach preset iteration number, obtain reconstruction face sheet;S6, according to reconstruction face sheet, constructs open-pit mine three-dimensional terrain.The present application improves the accuracy and integrity of reconstruction face sheet.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of open-pit mine terrain information monitoring, in particular to a real-time open-pit mine three-dimensional terrain construction method based on video monitoring. BACKGROUND

[0002] Real-time monitoring of open-pit mine terrain is a key link to ensure the safe and stable operation of the mine. In terms of mine safety, real-time modeling of open-pit mine terrain can timely detect safety problems such as slope deformation and damage, and provide timely alarm information to take effective measures for slope control, processing and repair, thereby ensuring the safety and sustainability of the mining process; in terms of mine operation, real-time monitoring of open-pit mine terrain can timely calculate the amount of excavation, provide data support for open-pit mine design, and provide timely path planning for mine autonomous vehicles to improve vehicle operation efficiency.

[0003] Some existing methods for establishing open-pit mine three-dimensional terrain are as follows: 1. Satellite stereo image pairs and unmanned aerial photography, which use various carriers to carry cameras to obtain stereo pairs, and then use the principle of space triangulation to obtain the three-dimensional terrain of the photographed area. However, due to the influence of satellite operation period and unmanned aerial vehicle flight frequency, the time interval for obtaining three-dimensional terrain by this method is long. 2. Airborne laser radar, airborne laser radar can obtain dense point cloud data, and three-dimensional terrain of open-pit mine can be constructed using point cloud data. However, due to the influence of flight frequency, the time interval for obtaining three-dimensional terrain by airborne laser radar is long. 3. Real-time three-dimensional terrain reconstruction technology based on unmanned mine vehicle, which uses an unmanned mine vehicle platform to carry multiple sensors (such as laser radar, camera, inertial navigation system, etc.), collects mine terrain data in real time, and processes and analyzes the data to finally generate high-precision three-dimensional terrain. However, this method has the problem of small coverage scale, and is limited to updating key areas such as counter-slopes and retaining walls. Video monitoring has been used only for real scene image acquisition or AI algorithm carrier, and its potential for three-dimensional modeling has not been developed, but it has the characteristics of wide coverage and high monitoring frequency, and has great potential. SUMMARY

[0004] The purpose of the present application is to provide a real-time open-pit mine three-dimensional terrain construction method based on video monitoring, which improves the accuracy and integrity of the reconstructed patches.

[0005] To achieve the above purpose, the present application provides the following scheme:

[0006] A real-time open-pit mine three-dimensional terrain construction method based on video monitoring, comprising:

[0007] S1, fixed target control points are arranged in the target mine, and images of the target mine are obtained;

[0008] S2, matching feature points of the target mine image, obtaining initial sparse three-dimensional coordinate points according to the matching feature points and initial camera parameters;

[0009] S3, optimizing the initial sparse three-dimensional coordinate points according to the target control points, obtaining sparse ground three-dimensional coordinate points;

[0010] S4, constructing a seed patch according to the sparse ground three-dimensional coordinate points, diffusing the seed patch, and obtaining a new seed patch;

[0011] S5, filtering the new seed patch, reselecting the scale of the target mine image, obtaining a low-resolution image, returning to S4 until a preset iteration number is reached, and obtaining a reconstructed patch;

[0012] S6, constructing an open-pit mine three-dimensional terrain according to the reconstructed patch.

[0013] Optionally, obtaining initial sparse ground three-dimensional coordinate points comprises:

[0014] Detecting and matching feature points in the target mine image using a SIFT operator, and obtaining the initial sparse ground three-dimensional coordinate points according to the matching feature points and initial camera parameters, wherein the initial camera parameters include a camera center position obtained by GNSS technology and an IMU real-time attitude.

[0015] Optionally, optimizing the camera attitude and the initial sparse three-dimensional point coordinate points according to the target control points to obtain sparse ground three-dimensional coordinate points comprises:

[0016] Projecting three-dimensional points to the target mine image using current camera parameters, calculating the error between the projection points and the target control points in the two-dimensional pixel coordinates of the target mine image using a bundle adjustment model, optimizing the camera parameters by minimizing the error, and obtaining the sparse ground three-dimensional coordinate points according to the optimized camera parameters.

[0017] Optionally, constructing a seed patch according to the sparse ground three-dimensional coordinate points comprises: taking the sparse ground three-dimensional coordinate points as the center coordinates of the seed patch, and taking the direction of the sparse ground three-dimensional coordinate points pointing to the camera exposure point as the normal vector of the seed patch, and constructing the seed patch.

[0018] Optionally, diffusing the seed patch to obtain a new seed patch comprises:

[0019] S31, constructing the new seed patch in the neighborhood of the seed patch, wherein the initial three-dimensional coordinates of the new seed patch are obtained by coordinate direct calculation on the three-dimensional coordinates of the seed patch, and the initial normal vector of the new seed patch is the same as the normal vector of the seed patch;

[0020] S32, performing patch optimization algorithm of multi-view stereo matching algorithm on the new seed patch to obtain the pixel similarity of the new seed patch in the target mine image projection, if the similarity is higher than a threshold, the new seed patch is obtained, otherwise, the new seed patch is not obtained, and returning to S31 until the diffusion of all the seed patches is completed.

[0021] Optionally, obtaining the pixel similarity of the new seed patch in the image projection comprises:

[0022] Taking the seed patch in the target mine image projection coordinates as a center point, obtaining the pixel gray value in a preset window range, and calculating the multi-scale normalized cross-correlation coefficient through the pixel gray value in the preset window range to obtain the pixel similarity.

[0023] Optionally, filtering the new seed patch comprises: filtering the new seed patch by using visible consistency filtering, photometric error function filtering and geometric consistency filtering.

[0024] Optionally, before reselecting the scale of the target mine image, it comprises: processing the image by using Gaussian filtering and down-sampling to obtain low-resolution images of different scales.

[0025] The beneficial effects of the present application are: 1. A real-time three-dimensional terrain construction method for open-pit mines based on video monitoring is proposed, which uses fixed ground targets to automatically calculate the poses of multiple cameras in real time, constructs a bundle adjustment model with control points, and then uses multiple video monitoring corresponding key frame images at the same time and multi-view stereo matching reconstruction technology to realize real-time three-dimensional modeling of mine production scenes. 2. A multi-view stereo matching three-dimensional reconstruction technology based on pyramid strategy is proposed, which uses multi-scale rules to calculate the similarity in the patch diffusion stage, improving the accuracy and integrity of the reconstructed patches. 3. Based on the real-time three-dimensional modeling terrain of open-pit mines, a real-time monitoring and early warning method for key areas of mines is proposed; a real-time monitoring and early warning method for mine slope stability is proposed; a real-time calculation and statistics method for mine earthwork is proposed, which provides decision support for mine operation and mining design; a real-time map construction method for mines is proposed, which provides real-time path planning for mine autonomous vehicles. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below only illustrate some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0027] Figure 1 The flow chart of the method for constructing a three-dimensional terrain of an open-pit mine in real time based on video monitoring according to an embodiment of the present application;

[0028] Figure 2 The flow chart of the patch optimization algorithm according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0030] In order to make the above-mentioned objects, features and advantages of the present application more apparent, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0031] The present embodiment provides a method for constructing a three-dimensional terrain of an open-pit mine in real time based on video monitoring, comprising:

[0032] S1, arranging a fixed target control point in a target mine to obtain a target mine image;

[0033] S2, matching feature points of the target mine image, and obtaining initial sparse three-dimensional coordinate points according to the matched feature points and initial camera parameters;

[0034] S3, optimizing the initial sparse three-dimensional coordinate points according to the target control point to obtain sparse ground three-dimensional coordinate points;

[0035] S4, constructing a seed patch according to the sparse ground three-dimensional coordinate points, diffusing the seed patch, and obtaining a new seed patch;

[0036] S5, filtering the new seed patch, reselecting the scale of the target mine image, obtaining a low-resolution image, returning to S4 until a preset iteration number is reached, and obtaining a reconstructed patch;

[0037] S6, constructing a three-dimensional terrain of an open-pit mine according to the reconstructed patch.

[0038] The embodiment utilizes a ground fixed target to automatically solve a multi-camera pose in real time, constructs a bundle adjustment model with control points, and further utilizes a plurality of video monitors to realize real-time three-dimensional modeling of a mine production scene by corresponding key frame images and multi-view stereo matching reconstruction technology.

[0039] Further, acquiring the initial sparse ground three-dimensional coordinate points comprises:

[0040] SIFT operators are utilized to detect and match feature points in target mine images, and initial sparse ground three-dimensional coordinate points are acquired according to the matched feature points and initial camera parameters, wherein the initial camera parameters comprise a camera center position acquired by GNSS technology and an IMU real-time pose.

[0041] The camera pose and the initial sparse three-dimensional coordinate points are optimized according to the target control points, and the sparse ground three-dimensional coordinate points are acquired, comprising:

[0042] The three-dimensional points are projected to the target mine images by using the current camera parameters, a bundle adjustment model is used to calculate the error between the projection points and the target control points in the target mine images, the camera parameters are optimized by minimizing the error, and the sparse ground three-dimensional coordinate points are acquired according to the optimized camera parameters.

[0043] Specifically, SIFT operators are utilized to detect and match feature points in a plurality of images, and a corresponding relationship between the images is established by the matched feature points. The matched feature points and initial camera parameters are utilized to perform preliminary sparse three-dimensional reconstruction. The control points with known coordinates are introduced to correct the preliminary reconstruction results. A bundle adjustment model with control points is used to optimize the camera parameters and the three-dimensional point coordinates to minimize the re-projection error. Finally, accurate camera interior and exterior orientation elements are obtained, the camera space pose and accurate position are restored, and the sparse ground three-dimensional coordinate points are obtained.

[0044] Further, constructing seed patches according to the sparse ground three-dimensional coordinate points comprises: taking the sparse ground three-dimensional coordinate points as the central coordinates of the seed patches, and taking the directions of the sparse ground three-dimensional coordinate points pointing to the camera exposure points as the normal vectors of the seed patches, to construct the seed patches.

[0045] Further, the seed patches are diffused to acquire new seed patches, comprising:

[0046] S31, new seed patches are constructed in the neighborhood of the seed patches, wherein the initial three-dimensional coordinates of the new seed patches are obtained by coordinate direct calculation on the three-dimensional coordinates of the seed patches, and the initial normal vectors of the new seed patches are the same as the normal vectors of the seed patches;

[0047] S32, the patch optimization algorithm adopting the multi-view stereo matching algorithm is used to optimize the new seed patch, to obtain the pixel similarity of the new seed patch in image projection, if the similarity is higher than a threshold, the new seed patch is obtained, otherwise, the new seed patch is not obtained, and S31 is returned until the diffusion of all seed patches is completed.

[0048] Specifically, the embodiment performs multi-scale patch diffusion based on a pyramid strategy, including:

[0049] ①The seed patch is diffused by using the characteristics that adjacent patches have similar normal vectors and three-dimensional coordinates, and a new patch is constructed in the neighborhood of the seed patch. In this method, the new patch is constructed by using an eight-neighborhood diffusion method. The initial three-dimensional coordinates of the new patch are calculated by coordinate forward calculation from the three-dimensional coordinates of the seed patch. The initial normal vector of the new patch is consistent with the normal vector of the seed patch. Then, the optimization equation is built with the new patch. By using the principle that the pixel similarity of the patch in each image is the largest, the three-dimensional coordinates and the normal vector of the patch are constantly adjusted, so that the pixel similarity of the patch in each image is the largest. The newly constructed patch is optimized by the optimization algorithm. If the similarity is higher than a certain threshold, it is determined that the optimization is successful, and a new seed patch is generated. Otherwise, it is determined that the optimization fails, and the newly constructed patch is abandoned; ②The operation in ① is repeated until all seed patches complete one diffusion.

[0050] Further, obtaining the pixel similarity of the new seed patch in image projection includes: taking the projection coordinates of the seed patch in the image as a center point, obtaining the pixel gray value in a preset window range, calculating the multi-scale normalized cross-correlation coefficient by the pixel gray value in the preset window range, and obtaining the pixel similarity.

[0051] Further, filtering the new seed patch includes: filtering the new seed patch by using visible consistency filtering, photometric error function filtering and geometric consistency filtering.

[0052] Specifically, ① visibility consistency filtering: check the visibility consistency of each patch, and ensure that it is visible in all visible images. If the patch is not visible (blocked by other patches) in a certain visible image, it will be removed. ② photometric error function filtering: the photometric error function considers the gray difference and projection error (generally measured by the normalized cross-correlation coefficient) of the patch in different images. By setting a reasonable threshold, the patch with large photometric error is removed. ③ geometric consistency filtering: the principle of surface fitting filtering is based on the assumption that the surface of the object is usually smooth and continuous. Therefore, for a given patch, a series of adjacent patches (i.e. neighborhood patches) can be selected around it, and a local surface can be fitted using these neighborhood patches. Then, the normal vector of the patch can be optimized through the fitted surface, so that it more accurately reflects the true shape of the object surface, and the patch with large distance from the surface is removed.

[0053] Further, the step of reselecting the scale of the image comprises: processing the image by using Gaussian filtering and down-sampling to obtain low-resolution images of different scales.

[0054] The above and other objects of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings, in which: Figure 1 The method of the present embodiment is further described as follows:

[0055] Fixed high-definition cameras, IMUs (inertial navigation) and network transmission devices are deployed around the open-pit mine slope. A real-time solving system is deployed in the monitoring center (generally the cloud receiving and storage) to analyze and process the observation data in real time to obtain real-time terrain information.

[0056] 1. Data acquisition, transmission and storage: fixed target control points are arranged in the stable area of the mine, and high-definition cameras are deployed around the stable area of the open-pit mine slope to ensure that the overlap degree of adjacent camera frames is more than 70%. The deployment position should ensure that the cameras can capture the same scene from different angles, and the baseline distance between the cameras should be moderate to ensure the accuracy of three-dimensional reconstruction, and the camera center position is acquired by GNSS. The image data captured by the cameras needs to be transmitted to the data processing center through wired or wireless means. The data processing center is responsible for data storage, processing and analysis application.

[0057] 2. Data processing: data processing mainly extracts images obtained by cameras at different positions at the same time for multi-view stereo matching to construct a three-dimensional terrain. Multi-view stereo matching reconstruction can be regarded as the inverse process of taking photos of a determined scene. Taking photos maps a three-dimensional scene into a two-dimensional image, while multi-view stereo matching reconstruction is the opposite, which aims to restore the real three-dimensional scene from images taken from different viewpoints. The basic idea of the multi-view stereo matching algorithm is as follows:

[0058] Step 1, camera space pose recovery: in this embodiment, the camera is fixedly installed around the slope, the camera center position obtained by the GNSS technology is taken as the initial value X, Y, Z, and the IMU real-time pose is taken as the initial value, some ground fixed target control points are automatically identified, images obtained by different positions of the camera at the same time are extracted, and a bundle adjustment model with control points is constructed, wherein the bundle adjustment model with control points is constructed by combining the traditional bundle adjustment model with the control points.

[0059] SIFT operators are used to detect and match feature points in multiple images, and the corresponding relationship between the images is established through the matched feature points. The initial camera parameters are used to perform preliminary sparse three-dimensional reconstruction, and initial sparse ground three-dimensional coordinate points are obtained, wherein the initial camera parameters are the camera center position obtained by the GNSS technology and the IMU real-time pose.

[0060] The control points with known coordinates are introduced to correct the preliminary reconstruction results, including: in the bundle adjustment model, the three-dimensional coordinates of these control points and the current camera parameters are used to project the three-dimensional points to the projection points of the images, to construct an error function such as a re-projection error, that is, the three-dimensional points are projected to the images by using the current camera parameters, the error between the projection points and the fixed target projected on the image two-dimensional pixel coordinates is calculated, and then the camera parameters are optimized by minimizing all such errors.

[0061] Specifically, the control points provide absolute spatial constraints for the bundle adjustment, and the bundle adjustment model fuses the information of the control points with the observation data by minimizing the re-projection error, that is, the bundle adjustment model includes the collinearity equation constructed by the control points in the construction of the collinearity equation, and optimizes the camera parameters and scene structure jointly. The three form a closed loop: the control point constraint optimizes the direction, the bundle adjustment model provides a mathematical framework, and the camera parameter optimization is the output result. After the adjustment calculation is completed, the optimized camera parameters are obtained, and then the sparse ground three-dimensional point coordinates can be calculated by using the space forward intersection formula.

[0062] Step 2, constructing seed patches and image pyramids: ①construct seed patches by using the ground point coordinates obtained by the sparse three-dimensional reconstruction in the last step, the center coordinates of the seed patches are the sparse ground three-dimensional point coordinates, and the normal vectors of the seed patches are the directions in which the sparse ground three-dimensional point coordinates point to the camera exposure points. ②Gaussian filtering and down-sampling (usually row and column sampling) are used to generate a series of images with gradually reduced resolution, to prepare for subsequent multi-scale patch diffusion.

[0063] Step 3, multi-scale patch diffusion based on pyramid strategy: ①Diffusion is performed on the seed patch by using the feature that adjacent patches have similar normal vectors and three-dimensional coordinates. A new patch is constructed in the neighborhood of the seed patch. In this method, the new patch is constructed by using the eight-neighborhood method. The initial three-dimensional coordinates of the new patch are calculated by the normal calculation of the three-dimensional coordinates of the seed patch. The initial normal vector of the new patch is consistent with the normal vector of the seed patch. Then, the optimization equation is built with the new patch. By using the principle that the pixel similarity of the patch projection in each image is the largest, the three-dimensional coordinates and the normal vector of the patch are constantly adjusted, so that the pixel similarity of the patch projection in each image is the largest (here, the calculation method of the similarity is as follows: taking the projection coordinates of the patch in each image as the center point, the pixel gray value in the specified window range is obtained, and the multi-scale normalized cross-correlation coefficient is calculated as the similarity judgment standard). The new patch is optimized by using the optimization algorithm. If the similarity is higher than a certain threshold, it is determined that the optimization is successful to generate a new seed patch, otherwise, it is determined that the optimization fails and the new patch is abandoned; ②Repeat the operation in ① until all the seed patches complete one diffusion;

[0064] As shown in Figure 2 , the basic idea of the patch optimization algorithm of the multi-view stereo matching algorithm is as follows:

[0065] Step 1. Initialization of the patch: Since the newly constructed patch is adjacent to the seed patch, the initial three-dimensional coordinate information of the new patch is constructed by taking a certain offset of the three-dimensional coordinates of the seed patch in the plane. The initial normal vector information is consistent with the normal vector of the seed patch.

[0066] Step 2. Patch optimization: The purpose of patch optimization is to calculate the optimal three-dimensional coordinates and normal vector of the patch, so that the pixel correlation in the image window corresponding to the patch reaches the maximum. By constructing the optimization equation according to this rule, the three-dimensional coordinate information and the normal vector information are constantly adjusted, so that the optimization equation obtains the optimal solution, that is, the optimal three-dimensional coordinates and the normal vector of the patch are obtained.

[0067] The core of the patch optimization algorithm is the optimization equation. In order to simplify the operation, the algorithm fixes the three-dimensional coordinate information of the new patch on a light ray, and simplifies the six degrees of freedom of the patch (including three-dimensional coordinates (x, y, z) and normal vector (a, b, c)) to three, that is, the distance z along the light ray and the two normal vector direction angles a and b.

[0068] The process of optimizing the patch is to constantly adjust the three degrees of freedom of the patch, so that the correlation coefficient of the image window of the patch projection on the n images is the largest.

[0069]

[0070] In the above formula, n is the number of search images, f iis the correlation coefficient of the ith search image and the reference image (the correlation coefficient is calculated from the window pixel values). The process of patch optimization is converted to the process of finding the minimum of the function F(z, a, b).

[0071] Step 4, patch filtering: all generated patches are filtered using a filtering algorithm, and patches that do not meet the rules are removed. The filtering rules are as follows: ① visibility consistency filtering: the visibility consistency of each patch is checked to ensure that it is visible in all visible images. If the patch is not visible (blocked by other patches) in a certain visible image, it is removed. ② photometric error function filtering: the photometric error function takes into account the gray difference and projection error (generally measured by normalized cross-correlation coefficient) of the patch in different images. By setting a reasonable threshold, patches with large photometric error are removed. ③ geometric consistency filtering: the principle of curved surface fitting filtering is based on the assumption that the surface of the object is usually smooth and continuous. Therefore, for a given patch, a series of adjacent patches (i.e. neighborhood patches) can be selected around it, and these neighborhood patches can be used to fit a local curved surface. Then, the normal vector of the patch can be optimized through the fitted curved surface to more accurately reflect the true shape of the object surface, and patches with large distance from the curved surface are removed.

[0072] Step 5, iteration: the ground three-dimensional patch data formed by the patches filtered through the first patch diffusion will have some hollow areas. In order to fill the hollow areas, diffusion and filtering operations need to be performed again. The traditional algorithm directly repeats steps 2 and 3 three times to ensure the accuracy and integrity of the reconstructed patches. However, from the actual effect, hollow areas are still unavoidable. Especially in weak texture areas, calculating the normalized cross-correlation coefficient can face some challenges, because weak texture areas lack obvious features, leading the normalized cross-correlation coefficient to be very sensitive to noise and small gray changes, resulting in inaccurate matching results. In order to solve this problem and improve the reliability of the normalized cross-correlation coefficient, the matching window can be expanded to increase the amount of feature information involved in the calculation, but window expansion will increase the computational load and increase memory consumption. The method proposed in this paper is a multi-scale patch diffusion method based on pyramid strategy. When performing patch diffusion in iteration, the image resolution is gradually reduced to calculate the normalized cross-correlation coefficient. In this way, under the same matching window size, as the image resolution decreases, more image feature information is included, thereby improving the reliability of the normalized cross-correlation coefficient and the accuracy and success rate of matching in weak texture areas.

[0073] 3. Data analysis and application: Through multi-view stereo matching, a two-dimensional multi-view image set can be reversely constructed into three-dimensional terrain data. In the aspect of open-pit mine safety monitoring: through the multi-view image set obtained in real time, three-dimensional terrain data of the mine can be obtained in a time sequence of minutes, and the subsidence area range of the mine safety operation focus is recorded into the system in advance, the terrain subsidence amount can be calculated through the automatic superposition analysis of two periods of terrain data, and safety hazards can be found in time. By comparing the slope angle, platform width and platform height of the slope in the terrain data extracted by the automatic algorithm with the design value of the slope, the stability of the slope can be analyzed, and decision analysis can be provided for the early warning of safety problems such as slope deformation and destruction. In the aspect of open-pit mine operation: the three-dimensional terrain data of the mine can be used to obtain the earthwork amount of the day in time, and provide data support for subsequent mine exploitation design; at the same time, the three-dimensional terrain data is superimposed on the image data, the road information of the mine is extracted by the automatic algorithm, and a real-time dynamic map of the mine is constructed, which provides real-time path planning for the mine automatic driving vehicle, and can greatly improve the operation efficiency of the mine vehicle.

[0074] The above-described embodiments are only descriptions of the preferred modes of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements to the technical solutions of the present application made by those skilled in the art shall fall within the protection scope determined by the claims of the present application.

Claims

1. A method for real-time construction of three-dimensional terrain in open-pit mines based on video surveillance, characterized in that, include: S1. Set up fixed target control points at the target mine and acquire images of the target mine; S2. Match feature points to the target mine image, and obtain initial sparse three-dimensional coordinate points based on the matched feature points and initial camera parameters; Obtaining the initial sparse 3D ground coordinates includes: The SIFT operator is used to detect and match feature points in the target mine image. Based on the matched feature points and initial camera parameters, the initial sparse ground 3D coordinate points are obtained. The initial camera parameters include the camera center position and IMU real-time attitude obtained through GNSS technology. S3. Optimize the initial sparse three-dimensional coordinate points based on the target control points to obtain sparse ground three-dimensional coordinate points, including: Using the current camera parameters, 3D points are projected onto the target mine image. The bundle adjustment model is used to calculate the error between the projected points and the target control points on the target mine image, and the camera parameters are optimized by minimizing the error. Based on the optimized camera parameters, the sparse ground 3D coordinate points are obtained. S4. Construct a seed patch based on the sparse three-dimensional coordinate points of the ground, and diffuse the seed patch to obtain a new seed patch; S5. Filter the new seed patch, reselect the scale of the target mine image, obtain a low-resolution image, return to S4 until the preset number of iterations is reached, and obtain the reconstructed patch. Before re-selecting the scale of the target mine image, the process includes: processing the image using Gaussian filtering and downsampling to obtain low-resolution images at different scales; S6. Based on the reconstructed surface patch, construct the three-dimensional terrain of the open-pit mine.

2. The method for real-time construction of three-dimensional terrain in open-pit mines based on video surveillance according to claim 1, characterized in that, Constructing a seed patch based on the sparse 3D ground coordinate points includes: using the sparse 3D ground coordinate points as the center coordinates of the seed patch, and using the direction from the sparse 3D ground coordinate points to the camera exposure point as the normal vector of the seed patch, thereby constructing the seed patch.

3. The method for real-time construction of three-dimensional terrain in open-pit mines based on video surveillance according to claim 1, characterized in that, Diffusion of the seed facets to obtain new seed facets includes: S31. Construct the new seed patch in the neighborhood of the seed patch, wherein the initial three-dimensional coordinates of the new seed patch are obtained by performing forward coordinate calculation on the three-dimensional coordinates of the seed patch, and the initial normal vector of the new seed patch is the same as the normal vector of the seed patch. S32. The new seed facet is optimized using a multi-view stereo matching algorithm. The pixel similarity of the new seed facet projected onto the target mine image is obtained. If the similarity is higher than a threshold, the new seed facet is obtained. Otherwise, the new seed facet is not obtained, and the process returns to S31 until all the seed facets have completed diffusion.

4. The method for real-time construction of three-dimensional terrain in open-pit mines based on video surveillance according to claim 3, characterized in that, Obtaining the pixel similarity of the new seed patch projected onto the image includes: Using the projection coordinates of the seed patch onto the target mine image as the center point, obtain the pixel grayscale values ​​within a preset window range. Calculate the multi-scale normalized cross-correlation coefficient using the pixel grayscale values ​​within the preset window range to obtain the pixel similarity.

5. The method for real-time construction of three-dimensional terrain in open-pit mines based on video surveillance according to claim 1, characterized in that, Filtering the new seed patch includes filtering the new seed patch using visibility consistency filtering, photometric error function filtering, and geometric consistency filtering.

Images