Three-dimensional positioning method for heading machine based on mine environment monocular image

Abstract

The invention relates to a tunneling machine three-dimensional positioning method based on mine environment monocular images, and belongs to the technical field of target positioning. The method includes: acquiring an original image of a driving path of the heading machine; performing enhancement processing on the original image by using a low-illumination image enhancement algorithm based on a diffusion model to obtain a target image; the target image is input into a pre-trained heading machine recognition YOLOv10 network, and the heading machine recognition YOLOv10 network outputs a target frame where the heading machine is located; estimating the depth of the target image by using a depth estimation algorithm to obtain a depth value of the target image; calculating the three-dimensional coordinates of the target image in the world coordinate system according to the conversion relation between the world coordinate system and the image coordinate system and the depth value of the target image; and positioning the position of the heading machine in the world coordinate system according to the depth value of the target image and the target frame of the heading machine. According to the method, the low-illumination original image can be effectively enhanced, and the excessive brightness enhancement of the strong-illumination area can be avoided, so that the recognition accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target positioning, and in particular to a three-dimensional positioning method for a roadheader based on a monocular image of a mine environment. Background Art

[0002] Low light, high dust and strong light interference in mining environments make it extremely challenging to directly identify and locate roadheaders through images.

[0003] Existing methods for locating roadheaders usually rely on image processing techniques, such as histogram equalization, to adjust the image exposure to enhance the image quality. The enhanced image is then fed into a deep learning-based image recognition network to locate the roadheader.

[0004] However, due to the extreme lighting conditions in mines, these enhancement techniques often lead to unbalanced image processing results, resulting in color distortion and noise amplification. These problems make it difficult to apply the processed images to deep learning-based image recognition algorithms, ultimately resulting in low recognition and positioning accuracy of the roadheader. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a three-dimensional positioning method of a roadheader based on a monocular image of a mine environment. The technical solution of the present invention is as follows:

[0006] A three-dimensional positioning method for a roadheader based on a monocular image of a mine environment, comprising:

[0007] S1, obtaining the original image captured by the mine monitoring camera on the driving path of the roadheader;

[0008] S2, using a low-light image enhancement algorithm based on a diffusion model to enhance the original image to obtain a target image, wherein the low-light image enhancement algorithm based on a diffusion model is obtained by regularization-guided training in a strong-light region;

[0009] S3, input the target image into the pre-trained YOLOv10 network for tunnel boring machine recognition, and the YOLOv10 network for tunnel boring machine recognition outputs the target box where the tunnel boring machine is located;

[0010] S4, using a depth estimation algorithm to estimate the depth of the target image to obtain a depth value of the target image;

[0011] S5, calculating the three-dimensional coordinates of the target image in the world coordinate system according to the conversion relationship between the world coordinate system and the image coordinate system and the depth value of the target image;

[0012] S6, locating the position of the tunnel boring machine in the world coordinate system according to the depth value of the target image and the target frame where the tunnel boring machine is located.

[0013] Optionally, after S6, the step further includes:

[0014] S7, calculating the change amount of the three-dimensional coordinates of the tunnel boring machine within a preset time interval, and calculating the moving speed of the tunnel boring machine according to the change amount of the three-dimensional coordinates of the tunnel boring machine within the preset time interval.

[0015] Optionally, before S2, the step further includes: training a low-light image enhancement algorithm based on a diffusion model, specifically including:

[0016] S21, acquiring a low-light image dataset, where the low-light image dataset includes a plurality of low-light images of a mine environment;

[0017] S22, performing enhancement processing on each low-light image in the low-light image dataset to obtain an enhanced image corresponding to each low-light image;

[0018] S23, combining each low-light image in the low-light image dataset and its corresponding enhanced image to obtain a training dataset;

[0019] S24, training the low-light image enhancement algorithm based on the diffusion model through the training data set until the loss value of the loss function of the low-light image enhancement algorithm based on the diffusion model no longer decreases, thereby obtaining a trained low-light image enhancement algorithm based on the diffusion model.

[0020] Optionally, when S24 is specifically implemented, it includes:

[0021] S241, extracting a strong light region in each low light image using a strong light region extraction method;

[0022] S242, using strong illumination area regularization to guide the low-light image enhancement algorithm based on the diffusion model to perform regression training until the loss value of the loss function of the low-light image enhancement algorithm based on the diffusion model no longer decreases, obtaining the network weights of the low-light image enhancement algorithm based on the diffusion model, and obtaining a trained low-light image enhancement algorithm based on the diffusion model.

[0023] Optionally, the loss function of the low-light image enhancement algorithm based on the diffusion model is:

[0024]

[0025] w(x)=λM(x)+(1-M(x));

[0026] Among them, L total is the loss function, N is the number of pixels in the image, ε θ (x t,t) is the noise predicted by the low-light image enhancement algorithm based on the diffusion model at time step t, ε is the real noise obtained by regularized Gaussian noise sampling; ω(x) is the weight coefficient, which depends on the extraction result of the strong illumination area; λ is a constant less than 1, indicating that the gradient descent speed is slower in the strong illumination area, and the original enhancement speed is maintained in the non-strong illumination area, and M(x) is the mask of the strong illumination area.

[0027] Optionally, before S3, the method further includes: training a YOLOv10 network for roadheader identification, specifically including:

[0028] S41, constructing a low-light image dataset, where the low-light image dataset includes a plurality of low-light images of a mine environment;

[0029] S42, using a low-light image enhancement algorithm based on a diffusion model to enhance each low-light image in the low-light image dataset to obtain an enhanced image dataset;

[0030] S43, screening the enhanced image data set, selecting images including the tunnel boring machine operation scene as the target image data set, and marking the tunnel boring machine in each enhanced image in the target image data set to obtain a training data set;

[0031] S44, construct an initial YOLOv10 network for tunnel boring machine identification, and use the training data set to fine-tune the initial YOLOv10 network for tunnel boring machine identification until its loss function value no longer decreases, thereby obtaining a trained YOLOv10 network for tunnel boring machine identification.

[0032] Optionally, in a specific implementation, S4 uses a DPT depth estimation algorithm to estimate the depth of the target image to obtain a depth value of the target image.

[0033] Optionally, S5 is implemented by the following formula during specific implementation:

[0034] X=(x*d) / f

[0035] Y=(y*d) / f

[0036] Z = d;

[0037] Among them, X, Y, and Z are the three-dimensional coordinates of the target image in the world coordinate system, x and y are the two-dimensional coordinates of the target image in the image coordinate system, d is the depth value of the target image, and f is the focal length of the mine monitoring camera.

[0038] Optionally, in the specific implementation of S6, the three-dimensional coordinates of the center point of the tunnel boring machine in the world coordinate system are calculated according to the depth value of the target image and the target frame where the tunnel boring machine is located by the following formula, and the three-dimensional coordinates of the center point of the tunnel boring machine in the world coordinate system are determined as the located position of the tunnel boring machine:

[0039] X′=(x*d) / f+X c

[0040] Y'=((y*d) / f+Y c

[0041] Z′=d+Z c ;

[0042] Among them, X', Y', and Z' are the three-dimensional coordinates of the center point of the tunnel boring machine in the world coordinate system, x and y are the two-dimensional coordinates of the center point of the target frame where the tunnel boring machine is located in the image coordinate system, d is the depth value of the target image, f is the focal length of the mine monitoring camera, and X c , Y c , Z c is the three-dimensional coordinate of the mine monitoring camera in the world coordinate system.

[0043] Optionally, when S7 is specifically implemented, it includes:

[0044] S71, calculating the moving distance of the tunnel boring machine between preset time intervals by the following formula:

[0045] ΔX=X2-X1

[0046] ΔY=Y2-Y1

[0047] ΔZ=Z2-Z1;

[0048] Among them, X1, Y1, Z1 are the three-dimensional coordinates of the center point of the tunnel boring machine at the first time point in the world coordinate system, X2, Y2, Z2 are the three-dimensional coordinates of the center point of the tunnel boring machine at the second time point in the world coordinate system, ▽X, ▽Y, ▽Z are the moving distances of the center point of the tunnel boring machine between the preset time intervals, and the time interval between the first time point and the second time point is the preset time interval;

[0049] S72, calculating the moving speed of the tunnel boring machine according to the moving distance of the center point of the tunnel boring machine between the preset time intervals and the preset time interval by the following formula:

[0050]

[0051] Among them, ▽t is the preset time interval, and v is the moving speed of the tunnel boring machine.

[0052] All the above optional technical solutions can be combined arbitrarily, and the present invention does not provide detailed descriptions of the structures after the combinations.

[0053] By means of the above scheme, the beneficial effects of the present invention are as follows:

[0054] The original image is enhanced by using a low-light image enhancement algorithm based on a diffusion model, and the low-light image enhancement algorithm based on a diffusion model is trained by regularization guidance of the strong light area, so that when the original image is enhanced, the low-light original image can be effectively enhanced while avoiding excessive brightness enhancement of the strong light area, so that the obtained target image can be better applied to the target recognition network, thereby improving the recognition accuracy. By selecting the YOLOv10 network as the YOLOv10 network for tunnel boring machine recognition, the accuracy of tunnel boring machine recognition can be further improved compared with other neural network structures.

[0055] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of a method for three-dimensional positioning of a roadheader based on a monocular image of a mine environment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0058] The three-dimensional positioning method of a roadheader based on a monocular image of a mine environment provided by an embodiment of the present invention can be implemented by any electronic device with computing functions, such as a PC, a mobile terminal or a server. Figure 1 As shown, the three-dimensional positioning method of a roadheader based on a monocular image of a mine environment provided by an embodiment of the present invention includes the following steps S1 to S6:

[0059] S1, obtaining the original image captured by the mine monitoring camera on the driving path of the tunnel boring machine.

[0060] Specifically, a mine monitoring camera at the driving path of the tunnel boring machine will collect videos or images on the driving path of the tunnel boring machine in real time, and the electronic device can obtain the original image from the mine monitoring camera.

[0061] S2, using a low-light image enhancement algorithm based on a diffusion model to enhance the original image to obtain a target image.

[0062] The low-light image enhancement algorithm based on the diffusion model can stably output an enhanced image when a low-light image is input. It should be noted that before S2, the step of training the low-light image enhancement algorithm based on the diffusion model is also included, which specifically includes the following steps S21 to S24:

[0063] S21, acquiring a low-light image dataset, where the low-light image dataset includes a plurality of low-light images of a mine environment.

[0064] Specifically, it is preferred to select high-definition images with three RGB channels, each pixel storing no less than 16 bits, an image size of no less than 800x800, no strong light source around the camera, and able to observe the complete tunnel boring machine or most areas of the tunnel boring machine, and select low-light images at different times in the mine, especially low-light images in the tunnel boring machine operation scene, and try to cover multiple low-light images of the tunnel boring machine in the mine operation area to construct a low-light image dataset.

[0065] S22, performing enhancement processing on each low-light image in the low-light image dataset to obtain an enhanced image corresponding to each low-light image.

[0066] When the step S22 is implemented, it can be achieved through one or more of the following methods: image enhancement, noise removal, image sharpening, image noise reduction, image super-resolution, image alignment and image cropping, so as to ensure that the enhanced image has a better image effect than the low-light image.

[0067] S23, combining each low-light image in the low-light image dataset and its corresponding enhanced image to obtain a training dataset.

[0068] Each set of training images in the training data set includes a low-light image and its corresponding enhanced image.

[0069] S24, training the low-light image enhancement algorithm based on the diffusion model through the training data set until the loss value of the loss function of the low-light image enhancement algorithm based on the diffusion model no longer decreases, thereby obtaining a trained low-light image enhancement algorithm based on the diffusion model.

[0070] In a specific embodiment, the S24, when implemented, includes the following steps S241 and S242:

[0071] S241, extracting the strong light area in each low light image using a strong light area extraction method.

[0072] In mine tunnels, lights are the only light source. When shooting, the camera facing the lights is likely to produce high-exposure areas, affecting image quality. In the process of enhancing low-light images, although increasing exposure can improve the overall brightness, it may also lead to overexposure of the light area, causing a halo effect, thereby obscuring the details of surrounding objects and reducing their recognition. This is particularly critical in complex environments such as mines, affecting safety monitoring and operations. Therefore, strong light suppression becomes an important step in low-light image enhancement to ensure that the integrity of details is maintained while increasing image brightness. Effective strong light area detection and suppression technology can accurately identify and adjust strong light areas, reduce light imbalance, and ensure reasonable enhancement of other parts. Therefore, when training a low-light image enhancement algorithm based on a diffusion model, the embodiment of the present invention first extracts the strong light area in each low-light image.

[0073] For any low-light image, when extracting its strong light area, it is achieved in the following way: first, the color low-light image is converted into a grayscale image to simplify the image information and highlight the brightness difference. Then, the grayscale image is subjected to Gaussian filtering to eliminate random noise and enhance the smoothness of the image. Next, the threshold segmentation technology is used to set the brightness threshold, and the pixel area exceeding the brightness threshold is identified as the strong light area; specifically, the brightness threshold can be set by an adaptive threshold segmentation method or by manually setting a fixed threshold. In order to further optimize the detection effect of the strong light area, morphological operations can also be applied to eliminate isolated noise points and fill regional holes. After that, the connected domain analysis method is used to mark and contour analyze the extracted strong light area, calculate the area, shape and other parameters of the connected area, and obtain the geometric features of the strong light area. Finally, the outline of the strong light area is drawn on the original low-light image to realize the visual display of the strong light area, and the overexposed area can be repaired or filled as needed. This method of extracting the strong light area has high detection accuracy and can effectively deal with the problem of extracting the strong light area in the low-light image.

[0074] S242, using strong illumination area regularization to guide the low-light image enhancement algorithm based on the diffusion model to perform regression training until the loss value of the loss function of the low-light image enhancement algorithm based on the diffusion model no longer decreases, obtaining the network weights of the low-light image enhancement algorithm based on the diffusion model, and obtaining a trained low-light image enhancement algorithm based on the diffusion model.

[0075] Among them, the loss function of the low-light image enhancement algorithm based on the diffusion model is:

[0076]

[0077] w(x)=λM(x)+(1-M(x));

[0078] Among them, Ltotal is the loss function, N is the number of pixels in the image, ε θ (x t ,t) is the noise predicted by the low-light image enhancement algorithm based on the diffusion model at time step t, ε is the real noise obtained by regularized Gaussian noise sampling; ω(x) is the weight coefficient, which depends on the extraction result of the strong illumination area; λ is a constant less than 1, indicating that the gradient descent speed is slower in the strong illumination area, and the original enhancement speed is maintained in the non-strong illumination area, and M(x) is the mask of the strong illumination area.

[0079] That is to say, the embodiment of the present invention combines the diffusion model with the strong light area detection to propose a low-light image enhancement algorithm based on a diffusion model. It is an image enhancement method guided by regularization in the strong light area. In the loss function, the mask of the strong light area is used to weight the loss function and generate weights, so that the gradient descent speed in the strong light area is slower, thereby avoiding over-enhancement in the strong light area.

[0080] In the whole diffusion process, the goal of the low-light image enhancement algorithm based on the diffusion model is to minimize L total , so that the low-light image enhancement algorithm based on the diffusion model can effectively enhance the low-light image while avoiding excessive brightness enhancement in the strong light area. The mask weighting design of the strong light area in the loss function guides the low-light image enhancement algorithm based on the diffusion model to pay more attention to the enhancement of the non-strong light area, thereby optimizing the enhancement effect of the entire low-light image. After multiple iterations of optimization, the loss function makes the generated enhanced image gradually close to the real image, and the enhancement effect of the strong light area is effectively controlled.

[0081] In the process of specific regression training, the network weights of the diffusion model are actually trained, that is, ∈ θ (x t , t). The diffusion model in the embodiment of the present invention adopts a U-Net network, whose input is the noise image x at the current time step t. t , the output is the noise prediction ∈ at this time step θ (x t , t). The U-Net network mainly includes an input layer, an encoder, a skip connection layer and a decoder connected in sequence.

[0082] Input layer: used to receive the input of the noise image and time step t, and process the time step information through time embedding.

[0083] Encoder: used to extract multi-scale feature information of the image and gradually reduce the dimension to obtain the key structural information of the image.

[0084] Skip connection layer: used to connect the encoder and decoder parts, maintain the low-resolution feature information of the image, and help restore the details of the image.

[0085] Decoder: used to gradually restore the resolution of the image and generate noise predictions ∈ θ (x t , t), for use in the subsequent denoising process (backward diffusion).

[0086] In a specific embodiment, the S2, when implemented, includes the following steps:

[0087] 1. Input the original image into the low-light image enhancement algorithm based on the diffusion model, and perform forward diffusion processing to gradually add noise to the original image to generate a noisy image x t .

[0088] The forward diffusion process can be described by the following formula:

[0089]

[0090] Among them, q(x t |x t-1 ) indicates that x is known t-1 In the case of x t The distribution of x t represents the noise image generated in the tth step; α t is the noise attenuation factor, which is used to control the amount of noise increase at each step; I is Gaussian noise with unit variance; N(·) represents the Gaussian function.

[0091] Through this forward diffusion process, noise is gradually added to the original image until a highly noisy noisy image x is generated. T For any time step t, the noise image x t The relationship with the original image x0 can be expressed by the cumulative diffusion process as:

[0092]

[0093] in, represents the cumulative noise attenuation during the forward diffusion process, q(x t |x0) means x is known when x0 is known t The number of T is not specifically limited in the embodiment of the present invention, for example, T can be set to 500.

[0094] 2. Using the trained diffusion model, the noise is gradually removed from the noisy image through the reverse diffusion process to reconstruct a high-quality enhanced image. The specific steps are as follows: TStarting from the previous step, a less noisy image is gradually generated through back diffusion until a high-quality enhanced image is finally generated. The back diffusion process uses the following formula for step-by-step denoising:

[0095]

[0096] Among them, p θ (x t-1 |x t ) indicates that x is known t In the case of x t-1 The distribution of μ θ (x t , t) and ∑ θ (x t , t) are the mean and variance of the image during the denoising process. The trained diffusion model is used to predict μ θ (x t , t) and ∑ θ (x t , t), realize the conversion of noisy image to denoised image.

[0097] μ θ (x t , t) is calculated as follows:

[0098]

[0099] Through the above steps 1 and 2, the enhanced target image is finally output, and its brightness, contrast and details are effectively improved.

[0100] The embodiment of the present invention utilizes the diffusion model to gradually add and remove noise to achieve the enhancement of low-light images, which can effectively solve the problems of image blur and missing details under low-light conditions and is suitable for image processing tasks in low-light environments.

[0101] S3, input the target image into the pre-trained YOLOv10 network for tunnel boring machine recognition, and the YOLOv10 network for tunnel boring machine recognition outputs the target box where the tunnel boring machine is located.

[0102] It should be noted that, before S3, the step of training the tunnel boring machine to recognize the YOLOv10 network is also included, which specifically includes the following steps S41 to S44:

[0103] S41, constructing a low-light image dataset, where the low-light image dataset includes a plurality of low-light images of a mine environment.

[0104] Specifically, when constructing a low-light image dataset, the selected low-light images should be high-definition and color images with three RGB channels, each pixel of the image should be stored at least 16 bits, and the image size should be no less than 800x800; there should be no strong light source around the camera.

[0105] It should be noted that the low-light image dataset constructed in step S41 may be the same as or different from the low-light image dataset constructed in step S21.

[0106] S42, using a low-light image enhancement algorithm based on a diffusion model to enhance each low-light image in the low-light image dataset to obtain an enhanced image dataset.

[0107] Of course, this step may also be performed by enhancing the low-light image through other image enhancement algorithms, and the present invention does not specifically limit this.

[0108] S43, screening the enhanced image data set, selecting images including the tunnel boring machine operation scene as the target image data set, and marking the tunnel boring machine in each enhanced image in the target image data set to obtain a training data set.

[0109] The enhanced image should be able to observe the complete tunnel boring machine or most areas of the tunnel boring machine.

[0110] When annotating the tunnel boring machine in the enhanced image, it can be done manually or by using an annotation tool such as LabelImg. The training data set should contain the enhanced image and the annotation information of the tunnel boring machine, and the annotation information includes the location and category of the tunnel boring machine. For example, the annotation information is expressed as (x, y, w, h, class), where (x, y) is the center coordinate of the tunnel boring machine, (w, h) is the width and height of the tunnel boring machine, and class is the category of the tunnel boring machine.

[0111] It should be noted that the training data set should contain enough samples to ensure that the YOLOv10 network for tunnel boring machine recognition has good generalization ability.

[0112] S44, construct an initial YOLOv10 network for tunnel boring machine identification, and use the training data set to fine-tune the initial YOLOv10 network for tunnel boring machine identification until its loss function value no longer decreases, thereby obtaining a trained YOLOv10 network for tunnel boring machine identification.

[0113] Among them, the purpose of fine-tuning training is to enable the tunnel boring machine recognition YOLOv10 network to detect tunnel boring machines and improve its detection accuracy.

[0114] Furthermore, the training data set includes a training set and a test set. When the loss function value of the YOLOv10 network for tunnel boring machine identification no longer decreases, the embodiment of the present invention can use evaluation indicators such as mAP, precision, recall, etc. to evaluate the fine-tuned YOLOv10 network for tunnel boring machine identification to evaluate its detection accuracy and generalization ability. If the evaluation result does not meet the standard, the YOLOv10 network for tunnel boring machine identification is further optimized and fine-tuned until the evaluation result meets the standard, thereby obtaining a trained YOLOv10 network for tunnel boring machine identification.

[0115] Regarding the specific implementation method of using the training data set to fine-tune the initial roadheader identification YOLOv10 network, reference can be made to the existing neural network model training method, which is not elaborated in detail in the embodiment of the present invention.

[0116] In a specific embodiment, the target frame where the tunnel boring machine is located output by the tunnel boring machine identification YOLOv10 network may be multiple, and the tunnel boring machine identification YOLOv10 network also outputs the confidence corresponding to each target frame where the tunnel boring machine is located. In view of this situation, the embodiment of the present invention can select the target frame where the tunnel boring machine is located with the highest confidence as the final target frame where the tunnel boring machine is located.

[0117] By selecting the YOLOv10 network as the tunnel boring machine identification YOLOv10 network, the accuracy of tunnel boring machine identification can be improved compared with other neural network structures.

[0118] S4, using a depth estimation algorithm to estimate the depth of the target image to obtain a depth value of the target image.

[0119] In a specific embodiment, this step can be implemented using a DPT (Vision Transformer for Dense Prediction) depth estimation algorithm.

[0120] S5, calculating the three-dimensional coordinates of the target image in the world coordinate system according to the conversion relationship between the world coordinate system and the image coordinate system and the depth value of the target image.

[0121] In a specific embodiment, the S5 is implemented by the following formula:

[0122] X=(x*d) / f

[0123] Y=(y*d) / f

[0124] Z = d;

[0125] Among them, X, Y, and Z are the three-dimensional coordinates of the target image in the world coordinate system, x and y are the two-dimensional coordinates of the target image in the image coordinate system, d is the depth value of the target image, and f is the focal length of the mine monitoring camera.

[0126] When executing step S5, it is necessary to first determine the internal parameters or external parameters of the mine monitoring camera, such as the focal length. Specifically, when determining the internal parameters and external parameters of the mine monitoring camera, such as the focal length, it can be achieved by using a camera calibration algorithm such as the Zhang Zhengyou calibration method.

[0127] S6, locating the position of the tunnel boring machine in the world coordinate system according to the depth value of the target image and the target frame where the tunnel boring machine is located.

[0128] In a specific embodiment, when S6 is implemented, the three-dimensional coordinates of the center point of the tunnel boring machine in the world coordinate system are calculated according to the depth value of the target image and the target frame where the tunnel boring machine is located by the following formula, and the three-dimensional coordinates of the center point of the tunnel boring machine in the world coordinate system are determined as the position of the tunnel boring machine obtained by positioning:

[0129] X′=(x*d) / f+X c

[0130] Y′=(y*d) / f+Y c

[0131] Z′=d+Z c ;

[0132] Among them, X', Y', Z' are the three-dimensional coordinates of the center point of the tunnel boring machine in the world coordinate system, x, y are the two-dimensional coordinates of the center point of the target frame where the tunnel boring machine is located in the image coordinate system, d is the depth value of the target image, and f is the focal length of the mine monitoring camera; X c , Y c , Z c is the three-dimensional coordinate of the mine monitoring camera in the world coordinate system, which can be obtained through the positioning system of the mine monitoring camera or other methods.

[0133] Through the above steps, the three-dimensional coordinates of the tunnel boring machine in the world coordinate system can be obtained, and the position of the tunnel boring machine can be determined according to the three-dimensional coordinates. In another embodiment, in order to obtain more abundant information about the tunnel boring machine so as to monitor the tunnel boring machine, the step S6 may further include: S7, calculating the change amount of the three-dimensional coordinates of the tunnel boring machine within a preset time interval, and calculating the moving speed of the tunnel boring machine according to the change amount of the three-dimensional coordinates of the tunnel boring machine within the preset time interval. The preset time interval can be set as needed, and the embodiment of the present invention does not specifically limit this. For example, the preset time interval can be set to 1s.

[0134] In a specific embodiment, the S7, when implemented, includes:

[0135] S71, calculating the moving distance of the tunnel boring machine between preset time intervals by the following formula:

[0136] ΔX=X2-X1

[0137] ΔY=Y2-Y1

[0138] ΔZ=Z2-Z1;

[0139] Among them, X1, Y1, and Z1 are the three-dimensional coordinates of the center point of the tunnel boring machine at the first time point in the world coordinate system, X2, Y2, and Z2 are the three-dimensional coordinates of the center point of the tunnel boring machine at the second time point in the world coordinate system, ▽X, ▽Y, and ▽Z are the moving distances of the center point of the tunnel boring machine between preset time intervals, and the time interval between the first time point and the second time point is the preset time interval.

[0140] S72, calculating the moving speed of the tunnel boring machine according to the moving distance of the center point of the tunnel boring machine between the preset time intervals and the preset time interval by the following formula:

[0141]

[0142] Among them, ▽t is the preset time interval, and v is the moving speed of the tunnel boring machine.

[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A three-dimensional positioning method for a roadheader based on a monocular image of a mine environment, characterized in that: include: S1, obtaining the original image captured by the mine monitoring camera on the driving path of the roadheader; S2, using a low-light image enhancement algorithm based on a diffusion model to enhance the original image to obtain a target image, wherein the low-light image enhancement algorithm based on a diffusion model is obtained by regularization-guided training in a strong-light region; S3, input the target image into the pre-trained YOLOv10 network for tunnel boring machine recognition, and the YOLOv10 network for tunnel boring machine recognition outputs the target box where the tunnel boring machine is located; S4, using a depth estimation algorithm to estimate the depth of the target image to obtain a depth value of the target image; S5, calculating the three-dimensional coordinates of the target image in the world coordinate system according to the conversion relationship between the world coordinate system and the image coordinate system and the depth value of the target image; S6, locating the position of the tunnel boring machine in the world coordinate system according to the depth value of the target image and the target frame where the tunnel boring machine is located.

2. The three-dimensional positioning method of a roadheader based on a monocular image of a mine environment according to claim 1, characterized in that: The S6 also includes: S7, calculating the change amount of the three-dimensional coordinates of the tunnel boring machine within a preset time interval, and calculating the moving speed of the tunnel boring machine according to the change amount of the three-dimensional coordinates of the tunnel boring machine within the preset time interval.

3. The three-dimensional positioning method of a roadheader based on a monocular image of a mine environment according to claim 1, characterized in that: Before S2, the method further includes: training a low-light image enhancement algorithm based on a diffusion model, specifically including: S21, acquiring a low-light image dataset, where the low-light image dataset includes a plurality of low-light images of a mine environment; S22, performing enhancement processing on each low-light image in the low-light image dataset to obtain an enhanced image corresponding to each low-light image; S23, combining each low-light image in the low-light image dataset and its corresponding enhanced image to obtain a training dataset; S24, training the low-light image enhancement algorithm based on the diffusion model through the training data set until the loss value of the loss function of the low-light image enhancement algorithm based on the diffusion model no longer decreases, thereby obtaining a trained low-light image enhancement algorithm based on the diffusion model.

4. The method for three-dimensional positioning of a roadheader based on a monocular image of a mine environment according to claim 3, characterized in that: When S24 is specifically implemented, it includes: S241, extracting a strong light region in each low light image using a strong light region extraction method; S242, using strong illumination area regularization to guide the low-light image enhancement algorithm based on the diffusion model to perform regression training until the loss value of the loss function of the low-light image enhancement algorithm based on the diffusion model no longer decreases, obtaining the network weights of the low-light image enhancement algorithm based on the diffusion model, and obtaining a trained low-light image enhancement algorithm based on the diffusion model.

5. The method for three-dimensional positioning of a roadheader based on a monocular image of a mine environment according to claim 4, characterized in that: The loss function of the low-light image enhancement algorithm based on the diffusion model is: w(x)=λM(x)+(1-M(x)). Among them, L total is the loss function, N is the number of pixels in the image, ε θ (x t , t) is the noise predicted by the low-light image enhancement algorithm based on the diffusion model at time step t, ε is the real noise obtained by regularized Gaussian noise sampling; ω(x) is the weight coefficient, which depends on the extraction result of the strong illumination area; λ is a constant less than 1, indicating that the gradient descent speed is slower in the strong illumination area, and the original enhancement speed is maintained in the non-strong illumination area, and M(x) is the mask of the strong illumination area.

6. The method for three-dimensional positioning of a roadheader based on a monocular image of a mine environment according to claim 1, characterized in that: The S3 also includes: training the tunnel boring machine recognition YOLOv10 network, specifically including: S41, constructing a low-light image dataset, where the low-light image dataset includes a plurality of low-light images of a mine environment; S42, using a low-light image enhancement algorithm based on a diffusion model to enhance each low-light image in the low-light image dataset to obtain an enhanced image dataset; S43, screening the enhanced image data set, selecting images including the tunnel boring machine operation scene as the target image data set, and marking the tunnel boring machine in each enhanced image in the target image data set to obtain a training data set; S44, construct an initial YOLOv10 network for tunnel boring machine identification, and use the training data set to fine-tune the initial YOLOv10 network for tunnel boring machine identification until its loss function value no longer decreases, thereby obtaining a trained YOLOv10 network for tunnel boring machine identification.

7. The method for three-dimensional positioning of a roadheader based on a monocular image of a mine environment according to claim 1, characterized in that: In the specific implementation of S4, the depth of the target image is estimated using a DPT depth estimation algorithm to obtain a depth value of the target image.

8. The method for three-dimensional positioning of a roadheader based on a monocular image of a mine environment according to claim 1, characterized in that: In the specific implementation, S5 is realized by the following formula: X=(x*d) / f Y=(y*d) / f Z = d; Among them, X, Y, and Z are the three-dimensional coordinates of the target image in the world coordinate system, x and y are the two-dimensional coordinates of the target image in the image coordinate system, d is the depth value of the target image, and f is the focal length of the mine monitoring camera.

9. The method for three-dimensional positioning of a roadheader based on a monocular image of a mine environment according to claim 1, characterized in that: In the specific implementation of S6, the three-dimensional coordinates of the center point of the tunnel boring machine in the world coordinate system are calculated according to the depth value of the target image and the target frame where the tunnel boring machine is located by the following formula, and the three-dimensional coordinates of the center point of the tunnel boring machine in the world coordinate system are determined as the position of the tunnel boring machine obtained by positioning: X′=(x*d) / f+X c Y′=(y*d) / f+Y c Z′=d+Z c ; Among them, X', Y', and Z' are the three-dimensional coordinates of the center point of the tunnel boring machine in the world coordinate system, x and y are the two-dimensional coordinates of the center point of the target frame where the tunnel boring machine is located in the image coordinate system, d is the depth value of the target image, f is the focal length of the mine monitoring camera, and X c , Y c , Z c is the three-dimensional coordinate of the mine monitoring camera in the world coordinate system.

10. The method for three-dimensional positioning of a roadheader based on a monocular image of a mine environment according to claim 2, characterized in that: When S7 is specifically implemented, it includes: S71, calculating the moving distance of the tunnel boring machine between preset time intervals by the following formula: ΔX=X2-X1 ΔY=Y2-Y1 ΔZ=Z2-Z1; Among them, X1, Y1, and Z1 are the three-dimensional coordinates of the center point of the tunnel boring machine at the first time point in the world coordinate system, and X2, Y2, and Z2 are the three-dimensional coordinates of the center point of the tunnel boring machine at the second time point in the world coordinate system. is the moving distance of the center point of the tunnel boring machine between the preset time intervals, and the time interval between the first time point and the second time point is the preset time interval; S72, calculating the moving speed of the tunnel boring machine according to the moving distance of the center point of the tunnel boring machine between the preset time intervals and the preset time interval by the following formula: in, is the preset time interval, and v is the moving speed of the tunnel boring machine.

Images