A low-cost and easy-to-deploy downhole positioning method based on depth images

Abstract

A low-cost and easily deployable underground positioning method based on depth images is disclosed. This method involves distributing n positioning base stations with known coordinates throughout an underground tunnel. Each base station is a passive, low-cost reflective sphere with a reflective coating. The target being located moves underground carrying a reflective sphere with the same coating. A depth camera capable of emitting infrared light is movably placed within the tunnel to detect the target's location nodes in real time. The depth camera transmits the captured images to an edge computing node connected to it. The real-time location information of the target is obtained through a sequence of calculations using a location node detection algorithm, a ranging algorithm based on multi-filter collaborative denoising, and an optimized weighted center point positioning algorithm based on polygonal positioning errors. This invention utilizes passive, low-cost reflective spheres for real-time positioning of moving targets underground, eliminating the need for power supply infrastructure, resulting in low cost and high positioning accuracy.

Description

Technical Field

[0001] This invention relates to a downhole positioning method, specifically a low-cost and easily deployable downhole positioning method based on depth images, belonging to the field of downhole intelligent positioning technology. Background Technology

[0002] Coal resources, as one of the energy mineral resources, occupy an important position in primary energy production and consumption. However, the harsh underground mining environment and constantly changing working faces make it crucial to effectively ensure the safety of underground personnel and maintain underground production efficiency. Underground coal mine positioning technology plays a vital role in safe mining, personnel monitoring and dispatching, and post-disaster rescue. Due to the complex underground environment and technological limitations, the effectiveness of wireless target positioning underground is not ideal. Therefore, accurate positioning of underground targets in coal mines is an urgent problem to be solved in coal mine safety production and intelligent construction.

[0003] Currently, downhole positioning technologies mainly include various wireless signals such as Wi-Fi, Bluetooth, UWB, Zigbee, and RFID, as well as infrared positioning technology. However, these technologies have obvious drawbacks and still face many limitations in practical downhole applications.

[0004] (1) It requires the deployment of active positioning base stations in the well in advance, and such base stations usually require external power supply, which poses a safety hazard to their application in the well environment;

[0005] (2) The underground environment is harsh and the tunnel structure is complex and varied. Therefore, the multipath effect of wireless signals is significant and the signal attenuation is severe in the underground environment, which causes the underground positioning technology based on such wireless signals to exhibit low positioning accuracy and poor positioning effect.

[0006] (3) The above-mentioned underground positioning technology has high deployment and maintenance costs when deploying active positioning base stations, making it difficult to apply on a large scale underground.

[0007] Downhole positioning technology based on visual sensors has become a recent research hotspot due to its advantages of no electromagnetic interference and low cost. However, traditional visual positioning technologies based on monocular or binocular cameras are easily limited by unfavorable lighting conditions downhole, resulting in low target recognition accuracy and large distance calculation errors. Summary of the Invention

[0008] The purpose of this invention is to provide a low-cost and easily deployable downhole positioning method based on depth images. It utilizes a passive, low-cost reflective sphere combined with a depth camera to perform real-time positioning of moving targets in the well. It does not require the deployment of power supply infrastructure, is low in cost, and has high positioning accuracy.

[0009] To achieve the above objectives, the present invention provides a low-cost and easily deployable downhole positioning method based on depth images, comprising the following steps:

[0010] Step 1: n positioning base stations with known coordinates are evenly distributed in the underground roadway. The positioning base station is a passive, low-cost reflective ball with a reflective coating on its surface. The target being positioned moves underground carrying a reflective ball with a reflective coating on its surface.

[0011] Step 2: A depth camera capable of emitting infrared light is movably placed in the alleyway to detect the location nodes of the target in real time. The depth camera transmits the captured images to the edge computing node connected to the depth camera, and obtains the real-time location information of the target by sequentially calculating the location node detection algorithm, the ranging algorithm based on multi-filter collaborative denoising, and the optimized weighted center point positioning algorithm based on multilateral positioning error.

[0012] The location node detection algorithm in step 2 of this invention consists of two parts:

[0013] 1) Data augmentation strategy based on small target reorganization includes two parts: histogram matching and copy-paste method. Histogram matching is to register the histogram of one image with the histogram of another image to make the histogram distribution of the two images consistent. Before copying and pasting the location node of the target, the image where the location node of the target is located is histogram matched with the image to be inserted, so that the location node of the target is better integrated into the inserted image.

[0014] 2) Based on a deep learning-based location node detection model, this invention uses deep learning YOLOv5 to detect the location nodes of the target. The deep learning YOLOv5 network structure is divided into four parts according to the processing stage: Input, Backbone, Neck, and Prediction. The Input part is used to implement basic processing tasks such as data augmentation, adaptive image scaling, and anchor box calculation. The Backbone part, as the backbone network, mainly uses the CSP (cross-stage partial) structure to extract the main information from the input samples in the image for use in subsequent stages. The Neck part uses the FPN (feature pyramid network) and PAN (path aggregation network) structures to enhance feature fusion using the information extracted by the Backbone part. The Prediction part makes predictions and calculates loss values ​​such as GLOU_loss.

[0015] The ranging algorithm based on multi-filter collaborative denoising in step 2 of this invention comprises two parts:

[0016] 1) Multi-filter collaborative denoising: First, a spatial filter is used to process the image, which can preserve the edge details of distant objects; second, a fill filter is used to process the image to fill the black spot areas in the depth map; finally, a temporal filter is used to process the image, using the exponential moving average method to "remember" the depth value of a pixel in previous frames and take the average value. After applying the above three filters in a coordinated manner, it can be seen that the quality of the depth image is significantly improved, the black hole part is significantly reduced, and thus the ranging accuracy can be improved.

[0017] 2) The depth image-based ranging method, after obtaining the absolute depth of the location node of the target, combines it with two-dimensional data information to obtain the relative distance between the positioning base station and the target. First, based on the simplest camera model, the 3D point P(X,Y,Z) is projected onto the 2D image as point p(u,v).

[0018] The following formula can be derived:

[0019] p = K[R|T]P

[0020] Where: p is a point in the 2D plane;

[0021] K is the intrinsic parameter matrix of the camera model;

[0022] [R|T] is the extrinsic parameter matrix of the camera model, used to describe the Euclidean transformation between the world coordinate system and the camera coordinate system;

[0023] R is a 3×3 rotation matrix;

[0024] T is a 3×1 translation vector;

[0025] P, which stands for [X,Y,Z,1], is used to describe the world coordinate information of a 3D point.

[0026] The matrix form of the above equation is:

[0027]

[0028] Where s is a scaling factor that varies with focal length. Without considering the conversion between world coordinates and camera coordinates, i.e., using the camera coordinate system, then:

[0029]

[0030] Substituting, we get:

[0031]

[0032] Where: f x and f y This is the zoomed-out focal length;

[0033] u0 and v0 are the translation amounts of the origin;

[0034] Then, using the Z information provided by the depth map, 2D and 3D coordinates can be converted to each other. After calculating the 3D coordinates of each target location node in the camera coordinate system, the location base station (x) can be calculated using the Euclidean distance formula. i ,y i ,z i The relative distance ρ between the target (x0, y0, z0) and the target being located. i The Euclidean distance formula is as follows:

[0035]

[0036] The optimized weighted center point localization algorithm based on multilateral positioning error is as follows:

[0037] There are n positioning base stations, namely I1, I2, ..., I... n The corresponding coordinates are (x1, y1, z1), (x2, y2, z2), ..., (x n ,y n ,z n The distances between the target node I0 and the location node I0 with coordinates (x0, y0, z0) are ρ1, ρ2, ..., ρ1, ρ2, ..., ρ3. n Then we have:

[0038]

[0039] It can be represented in matrix form:

[0040] AX = L

[0041] Where: A is the coefficient matrix;

[0042] X is an unknown vector (x0, y0, z0). T ;

[0043] L is a constant vector;

[0044]

[0045]

[0046] X = A can be obtained through matrix transformations. -1 L

[0047] The equation AX = L has equal left and right sides. However, the larger the error in L, the larger the error in solving the equation using the least squares method, and the lower the reliability of the data. Therefore, in the improved weighted centroid algorithm, the reciprocal of the obtained coordinate error value is used as the weight, as shown in the formula below:

[0048]

[0049]

[0050]

[0051] Where (x'0, y'0, z'0) represents the position coordinates of the target determined by the weighted centroid algorithm;

[0052] W i This represents the weights of each group of data;

[0053] This represents the coordinate value estimated using the polygonal positioning algorithm for the i-th data set;

[0054] A i and L i Represents the parameters of the linear equation determined by the i-th set of data;

[0055] norm(X) represents the second norm of X.

[0056] The depth camera is a RealSense series D455.

[0057] Compared with existing technologies, this invention uses n positioning base stations with known coordinates evenly distributed in the underground roadway. The positioning base stations are passive, low-cost reflective spheres with a reflective coating on their surface. The targets being located move underground carrying reflective spheres with a reflective coating. A depth camera, which is movably installed in the roadway, can perform active ranging compared to monocular and binocular cameras. It is robust to dark environments and therefore has good application prospects underground. The depth camera can emit infrared light to detect the position nodes of the targets in real time. The depth camera measures the distance between the positioning base stations and the targets in real time and obtains the real-time position information of the targets through edge computing nodes connected to the depth camera. This invention is a low-cost, battery-free positioning solution that can achieve accurate and safe positioning of moving targets underground. This invention can be deployed on many mobile devices (such as inspection robots) and can move with the underground working face, exhibiting good mobility. Attached Figure Description

[0058] Figure 1 This is an application scenario diagram of the present invention;

[0059] Figure 2 This is a flowchart of the process of the present invention;

[0060] Figure 3 Figure (a) shows the denoising effect of the present invention, and Figure (b) shows the original depth map of the present invention and the denoised depth map of the present invention.

[0061] Figure 4 This is a diagram showing the camera model and similarity relationships of the present invention;

[0062] Figure 5 This is a schematic diagram of the polygonal positioning of the present invention. Detailed Implementation

[0063] The invention will now be further described with reference to the accompanying drawings.

[0064] like Figure 1 As shown, a low-cost and easily deployable downhole positioning method based on depth images includes the following steps:

[0065] Step 1: n positioning base stations with known coordinates are evenly distributed in the underground roadway. The positioning base stations of the present invention can be deployed at any location in the roadway, and the location is not limited. In this embodiment, n positioning base stations are set on the roof of the roadway. The positioning base station is a passive low-cost reflective ball with a reflective coating on its surface. The target being located carries the reflective ball with a reflective coating on its surface and moves underground.

[0066] Step 2: A depth camera capable of emitting infrared light is movably placed in the alleyway to detect the location nodes of the target in real time. The depth camera transmits the captured images to the edge computing node connected to the depth camera, and obtains the real-time location information of the target by sequentially calculating the location node detection algorithm, the ranging algorithm based on multi-filter collaborative denoising, and the optimized weighted center point positioning algorithm based on multilateral positioning error.

[0067] like Figure 2 As shown, the location node detection algorithm in step 2 of this invention includes two parts:

[0068] 1) A data augmentation strategy based on small target reorganization includes two parts: histogram matching and copy-paste. The copy-paste method generates more location nodes in an image, increasing the number of location nodes in the image and the number of images containing location nodes, thus making the training more balanced. Considering the different lighting levels between images, simply inserting location nodes from one image into another will cause the added location nodes to mismatch with the background, introducing a lot of noise into the newly generated image. Based on this, this paper introduces an image histogram matching method on the basis of the "copy-paste" strategy. Histogram matching involves registering the histogram of one image with the histogram of another image to make the histogram distribution of the two images consistent. Before copying and pasting the location nodes of the located target, the image containing the location nodes of the located target is subjected to histogram matching with the image to be inserted, so that the location nodes of the located target are better integrated into the inserted image.

[0069] 2) A deep learning-based location node detection model. The main purpose of the location node detection model is to find all targets of interest in an image and determine their category and location. Due to the different appearances, shapes, and poses of various objects, coupled with interference from factors such as lighting and occlusion during imaging, target detection has always been one of the most challenging problems in the field of computer vision. Among them, the YOLO algorithm is currently the most widely used algorithm in target detection. This invention selects deep learning YOLOv5 to perform target detection on the location nodes of the target. The deep learning YOLOv5 network structure is divided into four parts according to the processing stage: Input, Backbone, Neck, and Prediction. The Input part is used to implement basic processing tasks such as data augmentation, adaptive image scaling, and anchor box calculation. The Backbone part, as the backbone network, mainly uses the CSP (cross-stage partial) structure to extract the main information from the input samples in the image for use in subsequent stages. The Neck part uses the FPN (feature pyramid network) and PAN (path aggregation network) structures to enhance feature fusion using the information extracted by the Backbone part. The Prediction part makes predictions and calculates loss values ​​such as GLOU_loss.

[0070] The ranging algorithm based on multi-filter collaborative denoising in step 2 of this invention comprises two parts:

[0071] 1) Multi-filter collaborative denoising: First, to preserve edge details while filtering out noise, an edge-preserving spatial filter is used. For depth maps derived from triangulation, noise increases quadratically with the distance of the object from the camera, leading to overly smoothed edge information in near images and insufficient smoothness in distant images. Therefore, the depth map is converted back to a disparity map. Since disparity and depth are approximately inversely proportional, edge details of distant objects can be preserved. Second, a filling filter is used to fill in black dot areas in the depth map. Since the main camera of the depth camera used in this invention is the left camera, and the number of objects seen by the left and right cameras is usually inconsistent, specifically, when an object seen by the left camera does not appear in the right camera, the depth at these locations cannot be calculated due to the lack of disparity information. In the depth map, this is represented by black dots with a depth value of 0. Furthermore, these black dots also appear when there is overexposure or underexposure, and when an object is too close to the camera, less than the minimum measurable distance. This invention uses the maximum value (farthest) of the gray pixels to fill in the black dot areas. Finally, a temporal filter is used to remove noise caused by environmental changes. Since the environment may change, such as changes in lighting, dynamic changes, and noise from infrared emitters, and since the frame information provided by the camera is unrelated (each frame is independent), it loses past appearance information and introduces too much noise. Therefore, this invention introduces a temporal filter using an exponential moving average method to "remember" the depth value of a pixel in previous frames and take the average. After applying the above three filters in conjunction, the effect is as follows: Figure 3 As shown in the figure, Figure (a) is the original depth map of the present invention, and Figure (b) is the denoised depth map of the present invention. It can be seen that the quality of the depth image is significantly improved and the black hole part is significantly reduced, thereby improving the ranging accuracy.

[0072] 2) Depth image-based ranging method: After obtaining the absolute depth of the location node of the target, the relative distance between the positioning base station and the target can be obtained by combining the two-dimensional data information. First, based on the simplest camera model, the 3D point P(X,Y,Z) is projected onto the point p(u,v) on the 2D image, such as... Figure 4 As shown.

[0073] according to Figure 4 The following formula can be derived:

[0074] p = K[R|T]P

[0075] Where: p is a point in the 2D plane;

[0076] K is the intrinsic parameter matrix of the camera model;

[0077] [R|T] is the extrinsic parameter matrix of the camera model, used to describe the Euclidean transformation between the world coordinate system and the camera coordinate system;

[0078] R is a 3×3 rotation matrix;

[0079] T is a 3×1 translation vector;

[0080] P, which stands for [X,Y,Z,1], is used to describe the world coordinate information of a 3D point.

[0081] The matrix form of the above equation is:

[0082]

[0083] Where s is a scaling factor that varies with focal length. Without considering the conversion between world coordinates and camera coordinates, i.e., using the camera coordinate system, then:

[0084]

[0085] Substituting, we get:

[0086]

[0087] Where: f x and f y This is the zoomed-out focal length;

[0088] u0 and v0 are the translation amounts of the origin;

[0089] Then, using the Z information provided by the depth map, 2D and 3D coordinates can be converted to each other. After calculating the 3D coordinates of each target location node in the camera coordinate system, the location base station (x) can be calculated using the Euclidean distance formula. i ,y i ,z i The relative distance ρ between the target (x0, y0, z0) and the target being located. i The Euclidean distance formula is as follows:

[0090]

[0091] The optimized weighted center point localization algorithm based on multilateral positioning error is as follows:

[0092] like Figure 5 As shown, assume there are n positioning base stations I1, I2, ..., I... n The corresponding coordinates are (x1, y1, z1, (x2, y2, z2), ..., (x n ,y n ,z nThe distances between the target node I0 and the location node I0 with coordinates (x0, y0, z0) are ρ1, ρ2, ..., ρ1, ρ2, ..., ρ3. n Then we have:

[0093]

[0094] It can be represented in matrix form:

[0095] AX = L

[0096] Where A is the coefficient matrix;

[0097] X is an unknown vector (x0, y0, z0). T ;

[0098] L is a constant vector;

[0099]

[0100]

[0101] X = A can be obtained through matrix transformations. -1 L

[0102] In range-based positioning algorithms, the positioning result using only one set of data may have a large error. Therefore, it is necessary to integrate multiple sets of data. In traditional centroid algorithms based on range-based positioning using multiple sets of data, the average of multiple estimated positions is generally taken as the final result. Traditional centroid algorithms assume that the weights of the estimated coordinates of each set of data are equal, failing to reflect the different influences of the data. Therefore, this invention integrates multiple sets of data and proposes an improved weighted centroid algorithm based on polygon positioning. Since when solving the position coordinate equation, when L is an accurate value, the correct estimated coordinate value X can be obtained, and the equation AX = L is equal on both sides, and the larger the error of L, the larger the error of the least squares method in solving the equation, and the lower the reliability of the data, the improved weighted centroid algorithm uses the reciprocal of the obtained coordinate error value as the weight, as shown in the following formula:

[0103]

[0104]

[0105] Where (x'0, y'0, z'0) represents the position coordinates of the target determined by the weighted centroid algorithm;

[0106] W i This represents the weights of each group of data;

[0107] This represents the coordinate value estimated using the polygonal positioning algorithm for the i-th data set;

[0108] A i and L i Represents the parameters of the linear equation determined by the i-th set of data;

[0109] norm(X) represents the second norm of X.

[0110] The depth camera is a RealSense series D455.

[0111] This invention utilizes a low-cost, battery-free reflective sphere as a positioning node and employs a series of algorithms to locate moving targets in underground wells. In particular, it designs a data augmentation strategy based on small target reassembly to improve the recognition accuracy of tiny positioning nodes. Furthermore, it proposes a novel ranging algorithm based on multi-filter collaborative denoising and designs an optimized weighted center point positioning algorithm based on polygonal positioning errors to minimize underground positioning errors. This invention has great potential to provide a low-cost, high-efficiency solution for accurate target positioning in complex underground environments.

Claims

1. A low cost, easy to deploy, downhole positioning method based on depth images, characterized in that, Comprising the following steps: Step 1: n positioning base stations with known coordinates are evenly distributed in the underground tunnel, and the positioning base stations are passive low-cost reflective balls with reflective coating on the surface; The positioned target carries a reflective ball with reflective coating on the surface and moves underground; Step 2: a depth camera capable of emitting infrared light is placed in the tunnel to detect the position nodes of the positioned target in real time, and the captured image is transmitted to the edge computing node connected to the depth camera, and the real-time position information of the positioned target is obtained through the position node detection algorithm, the ranging algorithm based on multi-filter collaborative denoising, and the optimized weighted center point positioning algorithm based on multi-lateral positioning error in the edge computing node; The position node detection algorithm in step 2 includes two parts: 1) Data enhancement strategy based on small target reorganization, including histogram matching and copy-paste method, through histogram matching, the histogram of one image is matched according to the histogram of another image, so that the histogram distribution of the two images remains consistent, and before copying and pasting the position nodes of the positioned target, the image where the position nodes of the positioned target are located is histogram matched with the image to be inserted, so that the position nodes of the positioned target are better integrated into the inserted image; 2) Position node detection model based on deep learning, through the deep learning detection model, the position nodes of the positioned target in the image after histogram matching and copy-paste are detected, and the deep learning network structure is divided into Input, Backbone, Neck, and Prediction four parts according to the processing stage.

2. A low cost and easy to deploy downhole positioning method based on depth images according to claim 1, characterized in that, The ranging algorithm based on multi-filter collaborative denoising in step 2 includes two parts: 1) Multi-filter collaborative denoising, first, an edge-preserving spatial filter is used to process the image, which is used to maintain the edge details of distant objects; second, a filling filter is used to process the image, which is used to fill the black point area in the depth map; finally, a time filter is used to process the image, which uses the exponential moving average method to "remember" the depth value of a certain pixel in the previous frame, and takes the average value; 2) The ranging method based on depth image, after obtaining the absolute depth of the position node of the positioned target, the relative distance between the positioning base station and the positioned target can be obtained by combining the two-dimensional data information. First, according to the simplest camera model, the 3D point projected onto the 2D image ; The following formula is derived: wherein: are points in a 2D plane; is the intrinsic matrix of the camera model; is the extrinsic matrix of the camera model, which describes the Euclidean transform between the world coordinate system and the camera coordinate system; is a 3x3 rotation matrix; is a 3 x 1 translation vector; i.e. , for describing the world coordinate information of the 3D point; The matrix form of the above formula is: wherein: is a scale factor that varies with focal length; then: Substitution gives: wherein: and is the scaled focal length; and is the origin translation; The Z information provided by the depth map can be used to convert 2D coordinates and 3D coordinates. After the 3D coordinates of each positioning target in the camera coordinate system are calculated, the relative distance between the positioning base station and the positioning target can be calculated by using the Euclidean distance formula. The Euclidean distance formula is:​​ 。 3. A low cost and easy to deploy downhole positioning method based on depth images according to claim 2, characterized in that, The optimized weighted center point positioning algorithm based on multi-lateral positioning error is as follows: The positioning base station is respectively The corresponding coordinate value is The position node of the positioned target is The coordinate value is The distance is Then, which can be expressed in matrix form as: wherein, is a coefficient matrix; unknown vector ; is a constant vector; By matrix transformation, we can get: , equation Left and right equal, at the same time, The greater the error of, the greater the error of the least squares method equation, the lower the reliability of the data, so in the improved weighted centroid algorithm, the reciprocal of the coordinate error value is taken as the weight, and the formula is as follows: , wherein: represents the position coordinates of the target determined by the weighted centroid algorithm; representing the weight of each set of data; representing the first coordinate values estimated by the multilateration algorithm for the set of data; and denotes the linear equation parameter determined by the set of data of data denotes the two-norm of 4. A low cost and easy to deploy downhole positioning method based on depth images according to claim 2, characterized in that, The model of the depth camera is RealSense series D455.

Images