Copper ore mining quantity estimation method based on binocular image three-dimensional reconstruction

Abstract

The invention discloses a copper mine mining quantity estimation method based on binocular image three-dimensional reconstruction. The method specifically comprises the following steps: acquiring binocular images of a camera of a satellite for shooting a target copper mine mining area at two positions; processing the binocular image by using distortion correction and stereo correction to obtain an overlapping region in the binocular image; calculating the depth of each pixel point in the overlapping region in the binocular image, obtaining the depth information of each pixel point, and generating a depth map; identifying the mining area of the binocular image by using a pre-trained yolk v8 model; calculating the mining amount by combining the depth information and the identified mining area; the method has the advantages that the binocular image is used for performing three-dimensional reconstruction on the copper ore area, compared with monocular and multi-view three-dimensional reconstruction, the method is simpler and more efficient, and more accurate depth information can be obtained; the binocular image is used for carrying out three-dimensional reconstruction on the copper ore area, compared with monocular and multi-view three-dimensional reconstruction, the method is simpler and more efficient, and more accurate depth information can be obtained.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision, and in particular to a method for estimating copper mine mining volume based on three-dimensional reconstruction of binocular images. Background Art

[0002] There are two main types of copper mining: open-pit mining and underground mining. Open-pit mining accounts for 90% of copper production. In open-pit mining, the copper ore raw materials include deep hole blasting, mechanical excavation, and vehicle transportation.

[0003] The estimation of the mining volume of open-pit copper mines is currently mainly based on the mine operators' own data. With the development of image algorithms, a method based on deep learning has recently emerged to estimate the area of ​​the mining area by identifying the closed circle of the mining area through satellite images, but the lack of depth information makes it impossible to obtain a more accurate mining volume.

[0004] In order to accurately estimate the amount of copper mined, it is necessary to perform 3D reconstruction of the mining area. 3D reconstruction is divided into active 3D reconstruction and passive 3D reconstruction according to different sensors. Active 3D reconstruction actively detects depth information based on sensors. Commonly used sensors include LiDAR, Structured Light and ToF (Time-of-Fight). The applicable scenarios of active 3D reconstruction are limited, and the hardware equipment is usually expensive. Passive 3D reconstruction usually only requires a camera and has a wider range of applicable scenarios, so it is valued / favored by researchers. Passive 3D reconstruction can be divided into three methods according to the number of input views of the algorithm: monocular depth estimation, binocular stereo matching and multi-view 3D reconstruction. Summary of the invention

[0005] The present invention aims to identify the mining area of ​​copper mines based on the existing copper mining volume identification method based on satellite images, but lacks depth information and cannot obtain a more accurate mining volume.

[0006] The present invention proposes a copper mining volume estimation method based on binocular image 3D reconstruction. The copper mining area is reconstructed in 3D using binocular images to obtain relatively accurate depth information. YOLO V8 is used to identify the copper mining area in the satellite image, thereby improving the accuracy of identifying the copper mining area and thus improving the accuracy of mining volume estimation.

[0007] The solutions adopted are:

[0008] A copper mining volume estimation method based on binocular image 3D reconstruction is as follows:

[0009] S1. Obtain binocular images of the target copper mining area taken by the satellite's built-in camera at two locations;

[0010] S2. Processing the binocular images by using distortion correction and stereo correction to obtain overlapping areas in the binocular images;

[0011] S3. Calculate the depth of each pixel in the overlapping area of ​​the binocular image, obtain the depth information of each pixel, and generate a depth map;

[0012] S4. Use the pre-trained yolov8 model to identify the mining area of ​​the binocular image;

[0013] S5. Calculate the mining volume by combining the depth information and the identified mining area.

[0014] In a further embodiment of the technical solution of the present invention, the Zhang Zhengyou calibration method is used to calculate the distortion coefficient and internal parameters of the camera that takes binocular images in the satellite to correct the image distortion, and the internal parameters include the focal length of the camera.

[0015] In a further preferred embodiment of the technical solution of the present invention, two satellite images are processed by stereo correction, and the process is as follows:

[0016] Use SIFT or ORB algorithm to detect feature points in binocular images respectively;

[0017] Use BRIEF or BEBLID to describe the feature points in the binocular image;

[0018] Traverse the feature points of the binocular image, perform feature point matching, and estimate the external parameters of the camera that took the binocular image.

[0019] In a further optimization of the technical solution of the present invention, the depth is calculated as follows:

[0020] Z is the depth;

[0021] B is the distance between the cameras that take binocular images;

[0022] f is the focal length of the camera;

[0023] x l -x r For parallax.

[0024] For further optimization of the technical solution of the present invention, the calculation of the mining volume is as follows: V d =∑ x,y S*(z t,x,y -z t-1,x,y );

[0025] z t,x,y is the depth of the pixel (x, y) when the tth group is photographed;

[0026] z t-1,x,yis the depth of the pixel (x, y) when the t-1th group was photographed;

[0027] S is the pixel area.

[0028] A copper mining volume estimation system based on satellite images reconstructed by binocular stereo matching 3D, comprising an image acquisition module, an image processing module, a data acquisition module, a mining area recognition module and a mining volume calculation module;

[0029] An image acquisition module, used for acquiring binocular images;

[0030] The image processing module is used to process the acquired binocular images and obtain the depth information of each pixel in the image;

[0031] Data acquisition module, used to obtain training data for the yolo v8 model;

[0032] The mining area recognition module uses the yolo v8 model to identify the mining area of ​​the processed binocular image;

[0033] The mining volume calculation module is used to calculate the mining volume of the mining area.

[0034] An electronic device comprises a processor and a memory, wherein the memory stores a computer program that can be run on the processor, and the processor executes the steps of the above method when running the computer program.

[0035] A computer-readable storage medium The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. The present invention uses binocular images to perform three-dimensional reconstruction of the copper mine area, which is simpler and more efficient than monocular and multi-view three-dimensional reconstruction and can obtain more accurate depth information.

[0038] 2. The present invention uses YOLO V8 to identify the copper mining area in the satellite image, which improves the accuracy of identifying the copper mining area, thereby improving the accuracy of mining volume estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a copper mining volume estimation method based on three-dimensional reconstruction of binocular images;

[0040] Figure 2 It is the corresponding relationship of one pixel in the binocular image. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the following Figure 1-2 The present invention is further described in detail. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] like Figure 1 As shown, the present invention provides a method for estimating copper mining volume based on binocular image three-dimensional reconstruction, and the specific method is as follows:

[0043] S1. Obtain binocular images of the target copper mining area taken by the satellite's built-in camera at two locations;

[0044] S2. Processing the binocular images by using distortion correction and stereo correction to obtain overlapping areas in the binocular images;

[0045] S3. Calculate the depth of each pixel in the overlapping area of ​​the binocular image, obtain the depth information of each pixel, and generate a depth map;

[0046] S4. Use the pre-trained yolo v8 model to identify the mining area of ​​the binocular image;

[0047] S5. Calculate the mining volume by combining the depth information and the identified mining area.

[0048] Obtaining binocular images of the target copper mining area, specifically: determining the location of the target copper mine, the satellite passes through the target copper mining area and uses the satellite's own camera to successively take two images of the copper mining area, so as to obtain a binocular image of the same target copper mining area.

[0049] When collecting binocular images, due to the imaging principle of the camera, the collected binocular images are prone to distortion, so it is necessary to perform distortion correction on the binocular images.

[0050] Distortion correction processes the binocular images separately, specifically:

[0051] The Zhang Zhengyou calibration method is used to calculate the distortion coefficient and internal parameters of the camera that takes binocular images in the satellite. The specific steps are as follows:

[0052] Use a camera to take at least 8 photos of the calibration plate from different angles.

[0053] Input the captured photos into the open source image computing software OpenCV, call the camera calibration model, and calculate the camera's distortion coefficient and internal parameters.

[0054] After acquiring the binocular image, call the distortion correction function in OpenCV and pass in the camera's distortion coefficient and internal parameters to perform distortion correction on the binocular image.

[0055] The intrinsic parameters include the focal length f of the camera.

[0056] Since the camera in the satellite that takes binocular images will inevitably rotate and translate when collecting images of the same target copper mining area at two locations, the captured binocular images will undergo radial distortion, so the binocular images need to be stereo corrected.

[0057] Stereo correction is used to process the binocular images respectively. Stereo correction processes the binocular images based on the image registration algorithm, including feature detection, feature description, feature matching, and transformation estimation. Specifically:

[0058] Feature detection: Use image feature detection algorithms such as SIFT or ORB to identify important or unique content in binocular images, i.e. feature points; for example, edges, corners, etc.

[0059] Feature description: The feature points detected by the binocular image feature are described using the feature description algorithm BRIEF or BEBLID. Each feature point in the binocular image is described as a vector. The feature vectors of the corresponding positions in the binocular image are very close.

[0060] Feature matching: traverse the feature points in the binocular image after feature detection and feature description, and use the feature points and the feature descriptions corresponding to the feature points to make each feature point in one image of the binocular image find the most similar feature point in the other image;

[0061] Transformation estimation: The RANSAC algorithm is used to combine two binocular satellite images after feature matching to estimate the translation and rotation of the camera that takes the binocular image in the satellite, i.e., the external parameters.

[0062] The external and internal parameters are input into OpenCV, and the stereo correction is performed on the other image based on the first image obtained in the binocular image, so that the target copper mining area has the same size in the two images and has a clear pixel correspondence, and the overlapping area in the binocular image is obtained.

[0063] The depth of each pixel in the overlapping part of the binocular image is calculated to generate a depth map.

[0064] The corresponding relationship of one pixel point in the overlapping part of the binocular image in the binocular image is as follows: Figure 2 As shown, the depth information of the pixel is calculated as follows:

[0065]

[0066] B is the distance between the cameras that take the binocular images, that is, l ,O r The distance between l ,Or They represent the positions of the cameras that took the binocular images when they were taken;

[0067] x l -x r is the parallax; x l ,x r are p and p' respectively. Figure 2 The x-axis coordinate in , p is the position of the pixel in one of the binocular images, and p' is the corresponding position of the pixel in the other binocular image;

[0068] f is the focal length of the camera.

[0069] The yolov8 model is used to identify the mining area of ​​the binocular image, as follows:

[0070] Obtain satellite images of a certain number of copper mining areas;

[0071] Marking the mining area according to the pixel area in the satellite image based on the mining area range information;

[0072] The satellite images and annotation information of the mining areas are input into the yolov8 model for training;

[0073] The overlapping area in the distortion-corrected and stereo-corrected binocular image is input into the trained yolov8 model, and the pixel range of the mining area in the overlapping area of ​​the identified binocular image is output.

[0074] Combined with the pixel range of the mining area output by the yolov8 model and the depth information of each pixel point in the depth map, the mining volume of the target copper mining area is calculated. The mining volume is calculated as follows:

[0075] V d =∑ x,y S*(z t,x,y -z t-1,x,y );

[0076] z t,x,y is the depth of the pixel (x, y) when the tth group is photographed;

[0077] z t-1,x,y is the depth of the pixel (x, y) when the t-1th group was photographed;

[0078] S is the pixel area.

[0079] A copper mining volume estimation system based on satellite images reconstructed by binocular stereo matching 3D, comprising an image acquisition module, an image processing module, a data acquisition module, a mining area recognition module and a mining volume calculation module;

[0080] An image acquisition module, used for acquiring binocular images;

[0081] The image processing module is used to process the acquired binocular images and obtain the depth information of each pixel in the image;

[0082] Data acquisition module, used to obtain training data of yolov8 model;

[0083] The mining area recognition module uses the yolov8 model to identify the mining area of ​​the processed binocular image;

[0084] The mining volume calculation module is used to calculate the mining volume of the mining area.

[0085] An electronic device comprises a processor and a memory, wherein the memory stores a computer program that can be run on the processor, and the processor executes the steps of the above method when running the computer program.

[0086] A computer-readable storage medium The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0087] The parts not involved in the present invention are the same as the prior art or can be implemented by using the prior art.

[0088] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for estimating copper mining volume based on three-dimensional reconstruction of binocular images, characterized in that: The specific method is as follows: S1. Obtain binocular images of the target copper mining area taken by the satellite's built-in camera at two locations; S2. Processing the binocular images by using distortion correction and stereo correction to obtain overlapping areas in the binocular images; S3. Calculate the depth of each pixel in the overlapping area of ​​the binocular image, obtain the depth information of each pixel, and generate a depth map; S4. Use the pre-trained yolo v8 model to identify the mining area of ​​the binocular image; S5. Calculate the mining volume by combining the depth information and the identified mining area.

2. The method for estimating copper mining volume based on binocular image 3D reconstruction according to claim 1, characterized in that: The Zhang Zhengyou calibration method is used to calculate the distortion coefficient and internal parameters of the camera that takes binocular images in the satellite to correct the image distortion. The internal parameters include the focal length of the camera.

3. The method for estimating copper mining volume based on binocular image 3D reconstruction according to claim 1, characterized in that: The two satellite images are processed using stereo correction. The process is as follows: Use SIFT or ORB algorithm to detect feature points in binocular images respectively; Use BRIEF or BEBLID to describe the feature points in the binocular image; Traverse the feature points of the binocular image, perform feature point matching, and estimate the external parameters of the camera that took the binocular image.

4. The method for estimating copper mining volume based on binocular image 3D reconstruction according to claim 1, characterized in that: The depth is calculated as follows: Z is the depth; B is the distance between the cameras that take binocular images; f is the focal length of the camera; x l -x r For parallax.

5. The method for estimating copper mining volume based on binocular image 3D reconstruction according to claim 1, characterized in that: The calculation of the mining volume is as follows: V d =Σ x,y S*(z t,x,y -z t-1,x,y ); z t,x,y is the depth of the pixel (x, y) when the tth group is photographed; z t-1,x,y is the depth of the pixel (x, y) when the t-1th group was photographed; S is the pixel area.

6. A copper mining volume estimation system based on satellite images reconstructed by binocular stereo matching, characterized in that: It includes an image acquisition module, an image processing module, a data acquisition module, an excavation area recognition module and an excavation volume calculation module; An image acquisition module, used for acquiring binocular images; The image processing module is used to process the acquired binocular images and obtain the depth information of each pixel in the image; Data acquisition module, used to obtain training data for the yolo v8 model; The mining area recognition module uses the yolo v8 model to identify the mining area of ​​the processed binocular image; The mining volume calculation module is used to calculate the mining volume of the mining area.

7. An electronic device comprising a processor and a memory, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor runs the computer program, the steps of the method according to any one of claims 1 to 5 are performed.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images