Method and system for repairing mine depth image

Abstract

The invention relates to a mine depth image restoration method and system, and relates to the technical field of mine depth image restoration. The restoration method comprises the following steps: acquiring a to-be-processed mine color image and a to-be-restored mine depth image corresponding to the same mine environment object; carrying out defogging and enhancement processing on the mine color image to be processed to obtain a defogged color balance image; respectively extracting first edge information and second edge information of the to-be-restored mine depth image and a registered defogging color balance image corresponding to the defogging color balance image registered with the to-be-restored mine depth image; determining fusion edge information according to the first edge information and the second edge information; and based on the fused edge information and the mine gray level image corresponding to the registration defogging color balance image, restoring the to-be-restored pixels in the to-be-restored mine depth image by using a combined bilateral filter. According to the embodiment of the invention, mine depth map restoration can be realized.

Description

Technical Field

[0001] The present disclosure relates to the technical field of mine depth image restoration, and in particular to a mine depth image restoration method and system. Background Art

[0002] As coal mining gradually enters the deep stage, the mine environment is becoming increasingly complex. Devices such as Kinect cameras face multiple technical challenges when used to collect mine tunnel images (mine depth images). The complexity of the mine environment, especially the voids in the mine depth images caused by insufficient light, dust, and water mist, seriously affects the quality of the mine depth images and the accuracy of subsequent 3D reconstruction.

[0003] 1. Insufficient processing of mine color images: Mine color images in mine environments often have problems such as low brightness, high noise, and loss of details due to insufficient lighting, dust, and water mist. Although the traditional dark channel prior image processing algorithm based on mine color can effectively remove dust and water mist in most scenarios, in special scenarios such as mine tunnels, the application effect is significantly affected due to the limitations of the algorithm itself and complex scene conditions. Specifically:

[0004] (1) Mispositioning of atmospheric light values: In mine tunnels, artificial lighting is used and there may be strong specular reflections in the foreground area, which will cause the atmospheric light values in the dark channel prior algorithm based on the dark channel prior image to be mispositioned on the foreground object, resulting in color aberration or light spots in the mine dehazed image corresponding to the mine color image.

[0005] (2) Failure of bright white areas: The areas near the light sources in the mine tunnels will show a strong bright white effect. The dark channel theory will fail when estimating these areas, resulting in deviations in the estimation of transmittance and atmospheric light values, and color distortion of the mine color image.

[0006] (3) Brightness reduction: Since the dark channel prior algorithm based on the dark channel prior image uses the darkest channel in the mine color image when estimating the transmittance, in an environment such as a mine where the lighting is inherently poor, the brightness of the mine color image after dehazing is often dimmer than other methods, and it cannot effectively improve the quality of the mine color image.

[0007] (4) Low computational efficiency: The dark channel prior algorithm uses methods such as softmatting or guided filtering to obtain a refined transmission map. These methods involve a large number of floating-point operations, resulting in low dehazing efficiency and affecting the ability to process real-time mine color images.

[0008] Due to the deficiencies of the above (1)-(4), it is necessary to propose a method applicable to special environments containing dust and fog such as underground coal mines, and efficiently restore and enhance the mine depth image by combining the mine color image pair and the mine depth image pair. By improving the estimation of atmospheric light value, downsampling optimization, transmission map estimation, and Retinex algorithm processing, the present invention can significantly improve the restoration quality and processing speed of the mine depth image, while reducing color distortion and detail loss of the mine color image, and is applicable to multiple fields such as image restoration, enhancement, and three-dimensional reconstruction.

[0009] 2. Hole problems in mine depth images in the mine environment: Depth images captured by Kinect cameras usually have holes, especially in complex mine environments. Due to the limitations of the measurement principle of Kinect cameras and physical factors in the mine environment (such as irregular surfaces, multiple object occlusions, etc.), a large amount of missing data will appear in the mine depth image. In addition, due to depth value ambiguity and multi-path reflection of light, some pixel points cannot accurately obtain depth information, resulting in the generation of holes in the mine depth image. These holes seriously affect the integrity of the point cloud data and the accuracy of subsequent processing during the three-dimensional reconstruction process. Existing depth image repair methods, such as the depth map enhancement method based on joint bilateral filtering proposed by Feng et al., although they can effectively remove noise and enhance image details, still have some deficiencies: (1) Smoothing problem of edge information: Feng et al.'s method uses a joint bilateral filter to denoise, but in practical applications, bilateral filtering will smooth some edge information, resulting in the amplification of noise during the amplification process of the detail layer, thus affecting the quality of the depth image. (2) High time complexity: This method has a large amount of computation and high time complexity when performing image enhancement, and is not suitable for three-dimensional reconstruction tasks that require efficient processing.

[0010] To overcome the above deficiencies, this study proposes an improved method for repairing mine depth images, which uses the mine color image and the mine depth image collected by the Kinect camera to repair the holes in the depth image. This method transforms the pixel points of the mine color image into the coordinate system of the mine depth image, and combines the texture information of the mine depth image and the mine color image to perform more accurate hole repair, thereby improving the quality of the mine depth image and providing more accurate data for subsequent three-dimensional reconstruction. Summary of the Invention

[0011] The present disclosure proposes a technical solution for a method and system for repairing mine depth images.

[0012] According to one aspect of the present disclosure, a method for restoring a mine depth image is provided, including: obtaining a to-be-processed mine color image and a to-be-restored mine depth image corresponding to the same mine environment object; performing defogging and enhancement processing on the to-be-processed mine color image to obtain a defogged color-balanced image; respectively extracting first edge information of the to-be-restored mine depth image and second edge information of a registered defogged color-balanced image corresponding to the defogged color-balanced image registered with it; determining fused edge information according to the first edge information and the second edge information; and using a joint bilateral filter to restore to-be-restored pixels in the to-be-restored mine depth image based on the fused edge information and a mine grayscale image corresponding to the registered defogged color-balanced image.

[0013] Preferably, the method for performing defogging and enhancement processing on the to-be-processed mine color image to obtain a defogged color-balanced image includes: respectively extracting a bright channel prior image and a dark channel prior image corresponding to the to-be-processed mine color image; determining a transmittance image according to the to-be-processed mine color image, the bright channel prior image, and the dark channel prior image; performing defogging on the to-be-processed mine color image based on the to-be-processed mine color image, its corresponding transmittance image, and the atmospheric light value corresponding to each color channel to obtain a defogged color image; and performing image enhancement on the defogged color image to obtain a defogged color-balanced image.

[0014] Preferably, the method for extracting the dark channel prior image corresponding to the to-be-processed mine color image includes: respectively calculating multiple minimum values corresponding to each color channel in the to-be-processed mine color image within a set first local area; configuring the minimum value among the multiple minimum values corresponding to each color channel as the pixel corresponding to the dark channel prior image within the set first local area; and repeating the above method, traversing the to-be-processed mine color image with the set first local area to obtain the dark channel prior image corresponding to the to-be-processed mine color image.

[0015] Preferably, the set first local area is configured as 3×3.

[0016] Preferably, the method for extracting the bright channel prior image corresponding to the to-be-processed mine color image includes: respectively calculating multiple maximum values corresponding to each color channel in the to-be-processed mine color image within a set first local area; configuring the maximum value among the multiple maximum values corresponding to each color channel as the pixel corresponding to the bright channel prior image within the set first local area; and repeating the above method, traversing the to-be-processed mine color image with the set first local area to obtain the bright channel prior image corresponding to the to-be-processed mine color image.

[0017] Preferably, the set first local area is configured as 3×3.

[0018] Preferably, the method for determining the transmittance image according to the to-be-processed mine color image, the bright channel prior image, and the dark channel prior image includes: determining the atmospheric light value corresponding to each color channel according to the bright channel prior image and the dark channel prior image; performing downsampling processing on the bright channel prior image to obtain a mine color downsampled image; estimating a transmission map according to the mine color downsampled image and the atmospheric light value corresponding to each color channel; calculating the gradient corresponding to the transmission map estimation to obtain a corresponding gradient map; determining a gradient response weight map according to the gradient map; and determining the transmittance image based on the transmission map estimation by using a gradient domain weighted convolution filter operator.

[0019] Preferably, the method for determining the gradient response weight map according to the gradient map includes: normalizing the gradient map, and determining the gradient response weight map based on the normalized gradient map.

[0020] Preferably, the method for determining the atmospheric light value corresponding to each color channel according to the bright channel prior image and the prior image includes: configuring the first set of bright pixel points for the pixels corresponding to the top set percentage of the pixel values of each channel image in the dark channel prior image sorted from large to small; configuring the second set of bright pixel points for the pixels corresponding to the top set percentage of the pixel values of each channel image in the bright channel prior image sorted from large to small; and determining the atmospheric light value corresponding to each color channel based on the first set of bright pixel points and the second set of bright pixel points.

[0021] Preferably, the set percentage is configured as 0.1%.

[0022] Preferably, the method for determining the atmospheric light value corresponding to each color channel based on the first set of bright pixel points and the second set of bright pixel points includes: obtaining a first weight coefficient and a second weight coefficient; respectively summing and averaging the bright pixel points corresponding to each color channel in the first set of bright pixel points to obtain the atmospheric light value of each color channel of the dark channel corresponding to the first set of bright pixel points; respectively summing and averaging the bright pixel points corresponding to each color channel in the second set of bright pixel points to obtain the atmospheric light value of each color channel of the bright channel corresponding to the second set of bright pixel points; and determining the atmospheric light value corresponding to each color channel based on the atmospheric light value of the dark channel corresponding to each color channel and its corresponding first weight coefficient, the atmospheric light value of the bright channel corresponding to each color channel and its corresponding second weight coefficient.

[0023] Preferably, the sum of the first weight coefficient and the second weight coefficient is configured as 1.

[0024] Preferably, the first weight coefficient and the second weight coefficient are respectively configured to be 0.5.

[0025] Preferably, the method for determining the atmospheric light value corresponding to each color channel based on the dark channel atmospheric light value corresponding to each color channel and its corresponding first weight coefficient, and the bright channel atmospheric light value corresponding to each color channel and its corresponding second weight coefficient includes: multiplying the dark channel atmospheric light value corresponding to each color channel by the corresponding first weight coefficient to obtain the first weight dark channel atmospheric light value corresponding to each color channel; multiplying the bright channel atmospheric light value corresponding to each color channel by the corresponding second weight coefficient to obtain the second weight bright channel atmospheric light value corresponding to each color channel; respectively summing the first weight dark channel atmospheric light value corresponding to each color channel and the second weight bright channel atmospheric light value corresponding to each color channel to determine the atmospheric light value corresponding to each color channel.

[0026] Preferably, the method for dehazing the to-be-processed mine color image based on the to-be-processed mine color image, its corresponding transmittance image, and the atmospheric light value corresponding to each color channel to obtain a dehazed color image includes: obtaining a set protection factor; respectively calculating the difference between each color channel pixel in the to-be-processed mine color image and its corresponding atmospheric light value to obtain the pixel difference corresponding to each color channel; respectively determining the maximum value corresponding to each color channel between each color channel pixel in the transmittance image and the set protection factor; respectively calculating the ratio corresponding to each color channel between the pixel difference of each color channel and the maximum value of the corresponding color channel; adding the ratio corresponding to each color channel to the atmospheric light value of the corresponding color channel to obtain a dehazed color image.

[0027] Preferably, the method for performing image enhancement on the dehazed color image to obtain a dehazed color balanced image includes: converting the dehazed color image in the RGB space to a visible photon subset image corresponding to the HSV space; based on the visible photon subset image, determining the reflection component corresponding to the brightness information V(x) component in the visible photon subset image;

[0028] performing non-linear stretching on the reflection component to obtain a non-linearly stretched reflection component; updating the brightness information V(x) component based on the logarithmic illuminance corresponding to the brightness information V(x) component and the non-linearly stretched reflection component to obtain an updated brightness information V(x) component; fusing the updated brightness information V(x) component, the H(x) component, and the S(x) component in the visible photon subset image to obtain an HSV fused dehazed color balanced image; converting the HSV fused dehazed color balanced image corresponding to the HSV space to the RGB space to obtain a dehazed color balanced image.

[0029] Preferably, the method for determining the reflection component corresponding to the luminance information V(x) component in the visible photon set image based on the visible photon set image includes: extracting the luminance information V(x) component corresponding to the visible photon set image; using the guided filter technique to estimate the illuminance corresponding to the luminance information V(x) component; respectively converting the luminance information V(x) component and its corresponding illuminance into the logarithmic domain to obtain the logarithmic luminance information V(x) component and the logarithmic illuminance; subtracting the logarithmic illuminance corresponding to the logarithmic luminance information V(x) component to determine the reflection component corresponding to the luminance information V(x) component.

[0030] Preferably, the method for non-linearly stretching the reflection component to obtain the non-linearly stretched reflection component includes: performing normalization processing on the reflection component to obtain a normalized reflection component; performing non-linear stretching on the normalized reflection component to obtain a non-linearly stretched reflection component.

[0031] Preferably, the method for respectively extracting the first edge information and the second edge information of the to-be-restored mine depth image and the registered haze-removed color balance image corresponding thereto includes: using an edge extraction algorithm to respectively perform edge extraction on the to-be-restored mine depth image and the registered haze-removed color balance image to obtain the first edge information corresponding to the to-be-restored mine depth image and the second edge information corresponding to the registered haze-removed color balance image.

[0032] Preferably, the method for determining the fused edge information according to the first edge information and the second edge information includes: performing pixel-by-pixel comparison based on the first edge binary image and the second edge binary image corresponding to the first edge information and the second edge information to obtain the fused edge information.

[0033] Preferably, the method for performing pixel-by-pixel comparison based on the first edge binary image and the second edge binary image corresponding to the first edge information and the second edge information to obtain the fused edge information includes: performing a logical OR operation on the first edge binary image and the second edge binary image corresponding to the first edge information and the second edge information pixel by pixel to obtain the fused edge information.

[0034] Preferably, the method for repairing the to-be-repaired pixels in the to-be-repaired mine depth image by using a joint bilateral filter based on the fused edge information and the mine grayscale image corresponding to the registered haze-removed color-balanced image includes: obtaining a judgment function corresponding to the fused edge information; judging whether the pixels adjacent to the to-be-repaired pixels in the to-be-repaired mine depth image participate in the joint bilateral filtering based on the judgment function and the fused edge information; if the pixels adjacent to the to-be-repaired pixels participate in the joint bilateral filtering, repairing the to-be-repaired pixels in the to-be-repaired mine depth image by using the joint bilateral filter based on the pixels adjacent to the to-be-repaired pixels and the mine grayscale image corresponding to the registered haze-removed color-balanced image.

[0035] Preferably, the method for judging whether the pixels adjacent to the to-be-repaired pixels in the to-be-repaired mine depth image participate in the joint bilateral filtering based on the judgment function and the fused edge information includes: when the to-be-repaired pixel is a missing pixel or the pixels adjacent to the to-be-repaired pixel are not the edge pixels corresponding to the fused edge information and the to-be-repaired pixel is the edge pixel corresponding to the fused edge information, the pixels adjacent to the to-be-repaired pixel participate in the joint bilateral filtering; otherwise, the pixels adjacent to the to-be-repaired pixel do not participate in the joint bilateral filtering.

[0036] Preferably, the method for repairing the to-be-repaired pixels in the to-be-repaired mine depth image by using a joint bilateral filter based on the pixels adjacent to the to-be-repaired pixels and the mine grayscale image corresponding to the registered haze-removed color-balanced image includes: performing weighted averaging on the pixels corresponding to each color channel in the registered haze-removed color-balanced image to obtain the mine grayscale image corresponding to the registered haze-removed color-balanced image; respectively calculating the pixel difference weight and the Gaussian distribution distance weight value corresponding to the mine grayscale image based on the mine grayscale image; repairing the to-be-repaired pixels in the to-be-repaired mine depth image by using the joint bilateral filter based on the pixel difference weight and the Gaussian distribution distance weight value corresponding to the mine grayscale image and the pixels adjacent to the to-be-repaired pixels.

[0037] According to one aspect of the present disclosure, there is provided a device / system for repairing mine depth images, including: an acquisition unit configured to acquire a to-be-processed mine color image and a to-be-repaired mine depth image corresponding to the same mine environmental object; a defogging and enhancement processing unit configured to perform defogging and enhancement processing on the to-be-processed mine color image to obtain a defogged color-balanced image; an extraction unit configured to extract first edge information of the to-be-repaired mine depth image and second edge information of a registered defogged color-balanced image corresponding to the defogged color-balanced image registered with it; a determination unit configured to determine fused edge information according to the first edge information and the second edge information; and a repair unit configured to repair to-be-repaired pixels in the to-be-repaired mine depth image by using a joint bilateral filter based on the fused edge information and a mine grayscale image corresponding to the registered defogged color-balanced image.

[0038] Preferably, the defogging and enhancement processing unit includes: a prior image extraction unit configured to extract a bright channel prior image and a dark channel prior image corresponding to the to-be-processed mine color image respectively; a transmittance image unit configured to determine a transmittance image according to the to-be-processed mine color image, the bright channel prior image and the dark channel prior image; a defogged color image processing unit configured to perform defogging on the to-be-processed mine color image based on the to-be-processed mine color image, its corresponding transmittance image and the atmospheric light value corresponding to each color channel to obtain a defogged color image; and an image enhancement processing unit configured to perform image enhancement on the defogged color image to obtain a defogged color-balanced image.

[0039] Preferably, the transmittance image unit includes: an atmospheric light value determination unit configured to determine the atmospheric light value corresponding to each color channel according to the bright channel prior image and the prior image; wherein, the atmospheric light value determination unit includes: a first determination unit, a second determination unit and a third determination unit; the first determination unit is configured to multiply the dark channel atmospheric light value corresponding to each color channel by the corresponding first weight coefficient to obtain the first weighted dark channel atmospheric light value corresponding to each color channel; the second determination unit is configured to multiply the bright channel atmospheric light value corresponding to each color channel by the corresponding second weight coefficient to obtain the second weighted bright channel atmospheric light value corresponding to each color channel; and the third determination unit is configured to sum the first weighted dark channel atmospheric light value corresponding to each color channel and the second weighted bright channel atmospheric light value corresponding to each color channel respectively to determine the atmospheric light value corresponding to each color channel.

[0040] Preferably, the image enhancement processing unit includes: a reflection component determination unit; the reflection component determination unit is configured to determine a reflection component corresponding to a luminance information V(x) component in the visible photon subset image corresponding to the dehazed color image; wherein, the reflection component determination unit includes: a luminance information V(x) component extraction unit for extracting the luminance information V(x) component corresponding to the visible photon subset image; an illuminance estimation unit for estimating the illuminance corresponding to the luminance information V(x) component by using a guided filter technique; a logarithmic processing unit for respectively converting the luminance information V(x) component and its corresponding illuminance into the logarithmic domain to obtain a logarithmic luminance information V(x) component and a logarithmic illuminance; and a reflection component calculation unit for subtracting the corresponding logarithmic illuminance from the logarithmic luminance information V(x) component to determine the reflection component corresponding to the luminance information V(x) component.

[0041] Preferably, the dehazed color image processing unit includes: a set protection factor acquisition unit for acquiring a set protection factor; a pixel difference calculation unit for respectively calculating a difference between each color channel pixel in the to-be-processed mine color image and its corresponding atmospheric light value to obtain a pixel difference corresponding to each color channel; a maximum value determination unit for respectively determining a maximum value corresponding to each color channel between each color channel pixel in the transmittance image and the set protection factor; a ratio calculation unit for respectively calculating a ratio corresponding to each color channel between the pixel difference of each color channel and the maximum value of the corresponding color channel; and an addition unit for adding the ratio corresponding to each color channel to the atmospheric light value of the corresponding color channel to obtain a dehazed color image.

[0042] According to one aspect of the present disclosure, there is provided a system for repairing a mine depth image, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to: execute the above-mentioned method for repairing a mine depth image.

[0043] According to one aspect of the present disclosure, there is provided a system for repairing a mine depth image, including: an electronic device; a processor configured on the electronic device; and, a memory for storing processor-executable instructions; wherein, the processor is configured to call the instructions stored in the memory to execute the above-mentioned method for repairing a mine depth image.

[0044] According to one aspect of the present disclosure, there is provided a system for repairing a mine depth image, including: a computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions, when executed by a processor, implement the above-mentioned method for repairing a mine depth image.

[0045] According to one aspect of the present disclosure, a system for repairing mine depth images is provided, including: a computer program product, where the computer program product sets computer programs / instructions, and when the computer programs / instructions are executed by a processor, the above-mentioned method for repairing mine depth images is implemented.

[0046] In the embodiments of the present disclosure, aiming at the deficiencies in the processing of mine color images and the deficiencies in the processing of holes in mine depth maps, by combining mine depth images and mine color images, more accurate hole repair of mine depth images is carried out, thereby improving the quality of mine depth images and providing more accurate data for subsequent three-dimensional reconstruction.

[0047] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure.

[0048] According to the following detailed description of exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear. Description of the Drawings

[0049] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These drawings show embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0050] Figure 1 A flowchart showing a method for repairing mine depth images according to an embodiment of the present disclosure;

[0051] Figure 2 Effect diagrams showing a foggy underground passage color image and its dark channel prior image and bright channel prior image according to an embodiment of the present disclosure;

[0052] Figure 3 Effect diagrams showing a foggy mine color image (mine color image to be processed) and its dark channel prior map and bright channel prior image according to an embodiment of the present disclosure.

[0053] Figure 4 Effect diagrams showing a foggy underground passage color image and its corresponding dark channel prior image before algorithm improvement and dark channel prior image after algorithm improvement according to an embodiment of the present disclosure;

[0054] Figure 5 Effect diagrams showing a foggy mine color image (mine color image to be processed) and its corresponding dark channel prior image before algorithm improvement and dark channel prior image after algorithm improvement according to an embodiment of the present disclosure;

[0055] Figure 6 Effect diagrams showing the corresponding filtering window of the improved joint bilateral filter according to an embodiment of the present disclosure. Detailed Embodiments

[0056] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0057] As used herein, the term "exemplary" means "serving as an example, embodiment, or illustration". Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

[0058] As used herein, the term "and / or" merely describes an association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" as used herein means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set composed of A, B, and C.

[0059] In addition, for a better description of the present disclosure, numerous specific details are given in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be implemented without some specific details. In some instances, methods, means, elements, and circuits well-known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0060] It can be understood that the above-mentioned embodiments of the method and system for repairing the mine depth image according to the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not elaborate further.

[0061] In addition, the present disclosure also provides a system or device for repairing the mine depth image, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any one of the methods for repairing the mine depth image provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be elaborated further.

[0062] The present invention relates to the technical field of mine depth image repair, and specifically relates to an algorithm for processing mine depth images in a mine environment, especially an optimization method for noise processing, defogging, dust removal, and hole repair of mine color images and mine depth images, which is applied to the three-dimensional structure reconstruction of mine roadways.

[0063] Figure 1 A flowchart showing a method for repairing a mine depth image according to an embodiment of the present disclosure is as Figure 1As shown, the method for repairing the mine depth image includes: Step S101: Obtain the to-be-processed mine color image and the to-be-repaired mine depth image corresponding to the same mine environment object; Step S102: Perform defogging and enhancement processing on the to-be-processed mine color image to obtain a defogged color balance image; Step S103: Extract the first edge information and the second edge information of the to-be-repaired mine depth image and the registered defogged color balance image corresponding to the registered defogged color balance image respectively; Step S104: Determine the fused edge information according to the first edge information and the second edge information; Step S105: Based on the fused edge information and the mine gray image corresponding to the registered defogged color balance image, use a joint bilateral filter to repair the to-be-repaired pixels in the to-be-repaired mine depth image. Aiming at the insufficient processing of mine color images and the insufficient processing of the hole problems in mine depth maps, by combining mine depth images and mine color images, the holes in more accurate mine depth images are repaired, thereby improving the quality of mine depth images and providing more accurate data for subsequent 3D reconstruction.

[0064] Step S101: Obtain the to-be-processed mine color image and the to-be-repaired mine depth image corresponding to the same mine environment object.

[0065] In the embodiments of the present disclosure and other possible embodiments, a Kinect camera is used to synchronously obtain a foggy mine color image (to-be-processed mine color image) and a mine depth image (to-be-repaired mine depth image) corresponding to the same mine environment object.

[0066] Step S102: Perform defogging and enhancement processing on the to-be-processed mine color image to obtain a defogged color balance image.

[0067] In the embodiments of the present disclosure and other possible embodiments, based on the fusion algorithm of mine color fusion images: Since water mist (fog) in the air will affect imaging, the dark channel prior algorithm based on the dark channel prior image itself restores the color image from the prior theory angle of the atmospheric light scattering model. However, when shooting mine color images, there are problems such as dust in the mine, mine lighting environment, and uneven mine roadway lighting that affect the images. Therefore, when restoring mine color images using the dark channel prior algorithm based on the dark channel prior image, it is necessary to consider the influence of dust in the mine, mine lighting environment, and uneven mine roadway lighting on mine color images.

[0068] In the embodiments of the present disclosure and other possible embodiments, when restoring the mine color image, it is necessary to consider the technical problems of the influence of dust in the mine, the mine lighting environment, the uneven lighting in the mine roadway, etc. on the mine color image. It is necessary to improve the dark channel prior algorithm based on the dark channel prior image, fuse the dark channel prior image and the bright channel prior image (mine color fusion image) corresponding to the mine color image, realize a more accurate estimation of the atmospheric light value, reduce the influence of the restoration error on the restoration of the mine color image, and at the same time improve the processing method at the edge of the mine color image to obtain a smoother and more natural restored image of the mine color image. The improvement of this algorithm (fusion based on the mine color fusion image) mainly focuses on the following aspects: 1. Atmospheric light value estimation; 2. Downsampling optimization algorithm for the foggy mine color image (mine color image to be processed); 3. Transmittance image mapping algorithm.

[0069] In the embodiments of the present disclosure and other possible embodiments, for the atmospheric light value estimation: As can be seen from the following formula, if we want to obtain a fog-free mine color image, essentially only two unknown variables need to be obtained, namely the transmittance and the atmospheric light value. In the traditional algorithm, the top 0.1% of the dark channel pixels with the highest brightness in the dark channel prior image are selected, and then these dark channel pixels are mapped back to the original color image to obtain the mapped color image; and the three-channel images (R-channel image, G-channel image, B-channel image) corresponding to the mapped color image are respectively proposed; the three pixel averages corresponding to the pixels of the three-channel images are respectively calculated, and the three pixel averages are respectively configured as the atmospheric light estimation values corresponding to the three-channel images. However, this simple selection strategy is easily affected by special points in the color image, such as white vehicles, specular reflections, and vehicle lights in the color image.

[0070]

[0071] In the above formula, J(x) is the color restored image after defogging, I(x) is the foggy color image (color image to be processed), A is the atmospheric light estimation value, t(x) is the transmittance estimation value, and t0 is the protection factor; max() represents the maximum function. When the transmittance is too small, the problem of color restored image distortion is likely to occur, so t0 is used for improvement. According to the dark channel prior theory, specifically, t0 = 0.1.

[0072] Among them, I(x) - A means that each pixel corresponding to each channel image of the three-channel image in the color image to be processed is respectively subtracted from the atmospheric light estimation value corresponding to the corresponding channel. denotes each pixel corresponding to each channel image of the three-channel image in

[0073] For example, in A = [AR, AG, AB], AR, AG, and AB respectively represent the estimated values of the atmospheric light corresponding to the R channel, G channel, and B channel. Therefore, for each pixel in the R channel image of the three-channel image in the color image to be processed, subtract the estimated value AR of the atmospheric light corresponding to the R channel; for each pixel in the G channel image of the three-channel image in the color image to be processed, subtract the estimated value AG of the atmospheric light corresponding to the G channel; for each pixel in the B channel image of the three-channel image in the color image to be processed, subtract the estimated value AR of the atmospheric light corresponding to the B channel.

[0074] Similarly, represent for each pixel in the R / G / B channel image of the three-channel image, subtract the estimated value of the atmospheric light corresponding to the R / G / B channel. The present invention proposes a fusion method based on a mine color fusion image, which estimates the atmospheric light by fusing the channels of the dark channel prior image and the bright channel prior image. According to the dark channel theory, in the mine color image, there should be at least one of the red, yellow, and blue (RGB) color channels with the lowest intensity or almost zero.

[0075] In the implementation of the present disclosure, the method for performing defogging and enhancement processing on the mine color image to be processed to obtain a defogged color balance image includes: respectively extracting the bright channel prior image and the dark channel prior image corresponding to the mine color image to be processed; determining the transmittance image according to the mine color image to be processed, the bright channel prior image, and the dark channel prior image; performing defogging based on the mine color image to be processed, its corresponding transmittance image, and the atmospheric light value corresponding to each color channel to obtain a defogged color image; performing image enhancement on the defogged color image to obtain a defogged color balance image.

[0076] In the implementation of the present disclosure, the method for extracting the dark channel prior image corresponding to the mine color image to be processed includes: within a set first local area, respectively calculate multiple minimum values corresponding to each color channel in the mine color image to be processed; configure the minimum value among the multiple minimum values corresponding to each color channel as the pixel corresponding to the dark channel prior image within the set first local area; repeat the above method, and traverse the mine color image to be processed with the set first local area to obtain the dark channel prior image corresponding to the mine color image to be processed; where the set first local area can be configured as 3×3.

[0077] In the implementation of the present disclosure, the method for extracting the bright channel prior image corresponding to the to-be-processed mine color image includes: calculating the multiple maximum values corresponding to each color channel in the to-be-processed mine color image respectively within a set first local area; configuring the maximum value among the multiple maximum values corresponding to each color channel as the pixel corresponding to the bright channel prior image within the set first local area; repeating the above method, traversing the to-be-processed mine color image with the set first local area to obtain the bright channel prior image corresponding to the to-be-processed mine color image; wherein, the set first local area can be configured as 3×3.

[0078] In the embodiments of the present disclosure and other possible embodiments, those skilled in the art can configure the size corresponding to the set first local area according to actual needs.

[0079] In the embodiments of the present disclosure and other possible embodiments, therefore, the dark channel prior image of a foggy mine color image (to-be-processed mine color image) can be defined as:

[0080]

[0081] wherein, I dark (x) is the dark channel prior image corresponding to the foggy mine color image (to-be-processed mine color image) I(x), x represents the pixel value, and Ω(x) is the set first local area of 3×3 around the pixel x. I c (x) represents the channel image corresponding to any R channel, G channel, or B channel in the channel set C; min() represents the minimum function.

[0082] Similarly, in a foggy mine color image (to-be-processed mine color image), there is also at least one color channel whose pixel intensity values are very close to being uniform and the pixels are bright enough.

[0083] In the embodiments of the present disclosure and other possible embodiments, the bright channel prior image of a foggy mine color image (to-be-processed mine color image) can be defined as:

[0084]

[0085] wherein, I bright (x) is the dark channel prior image corresponding to the foggy mine color image (to-be-processed mine color image) I(x), x represents the pixel value, and Ω(x) is the set first local area of 3×3 around the pixel x. I c (x) represents the channel image corresponding to any R channel, G channel, or B channel in the channel set C; max() represents the maximum function.

[0086] Figure 2Show the effect diagrams of the foggy underground passage color image and its dark channel prior image and bright channel prior image according to the embodiments of the present disclosure; Figure 3 Show the effect diagrams of the foggy mine color image (the mine color image to be processed) and its dark channel prior image and bright channel prior image according to the embodiments of the present disclosure. With the defining formulas of the dark channel prior image and the bright channel prior image corresponding to the mine color image, the dark channel prior image and the bright channel prior image of a mine color image can be obtained. As Figure 2 and Figure 3 shown, the effect diagrams of different foggy underground passage color images and mine color images (the mine color images to be processed) and their dark channel prior images (DCP) and bright channel prior images (BCP). In Figure 2 , from left to right are the foggy color image of the underground passage, the dark channel prior image and the bright channel prior image; in Figure 3 , from left to right are the foggy mine color image of the underground tunnel, the dark channel prior image and the bright channel prior image. There are high levels of shadows, colored objects and dark objects in these dark channel prior images and bright channel prior images, which lead to changes in the dark channel values and bright channel values of the dark channel prior images and bright channel prior images.

[0087] In the implementation of the present disclosure, the method for determining the transmittance image according to the mine color image to be processed, the bright channel prior image and the dark channel prior image includes: determining the atmospheric light value corresponding to each color channel according to the bright channel prior image and the dark channel prior image; performing downsampling processing on the bright channel prior image to obtain a downsampled mine color image; performing transmission map estimation according to the downsampled mine color image and the atmospheric light value corresponding to each color channel; calculating the gradient corresponding to the transmission map estimation to obtain the corresponding gradient map; determining the gradient response weight map according to the gradient map; and determining the transmittance image based on the transmission map estimation by using the gradient domain weighted convolution filtering operator.

[0088] In the implementation of the present disclosure, the method for determining the gradient response weight map according to the gradient map includes: normalizing the gradient map, and determining the gradient response weight map based on the normalized gradient map.

[0089] In the implementation of the present disclosure, the method for determining the atmospheric light value corresponding to each color channel according to the bright channel prior image and the prior image includes: sorting the pixel values of each channel image in the dark channel prior image from large to small, and configuring the first set of bright pixel points for the top set percentage of the pixels; sorting the pixel values of each channel image in the bright channel prior image from large to small, and configuring the second set of bright pixel points for the top set percentage of the pixels; determining the atmospheric light value corresponding to each color channel based on the first set of bright pixel points and the second set of bright pixel points; wherein, the set percentage is configured to be 0.1%.

[0090] In the implementation of the present disclosure, the method for determining the atmospheric light value corresponding to each color channel based on the first set of bright pixel points and the second set of bright pixel points includes: obtaining a first weight coefficient and a second weight coefficient; respectively summing and averaging the bright pixel points corresponding to each color channel in the first set of bright pixel points to obtain the atmospheric light value of each color channel of the dark channel corresponding to the first set of bright pixel points; respectively summing and averaging the bright pixel points corresponding to each color channel in the second set of bright pixel points to obtain the atmospheric light value of each color channel of the bright channel corresponding to the second set of bright pixel points; determining the atmospheric light value corresponding to each color channel based on the atmospheric light value of the dark channel corresponding to each color channel and its corresponding first weight coefficient, the atmospheric light value of the bright channel corresponding to each color channel and its corresponding second weight coefficient; wherein, the sum of the first weight coefficient and the second weight coefficient is configured to be 1; wherein, the first weight coefficient and the second weight coefficient are respectively configured to be 0.5.

[0091] In the embodiments of the present disclosure and other possible embodiments, the method for determining the atmospheric light value corresponding to each color channel based on the atmospheric light value of the dark channel corresponding to each color channel and its corresponding first weight coefficient, the atmospheric light value of the bright channel corresponding to each color channel and its corresponding second weight coefficient includes: multiplying the atmospheric light value of the dark channel corresponding to each color channel by the corresponding first weight coefficient to obtain the first weight dark channel atmospheric light value corresponding to each color channel; multiplying the atmospheric light value of the bright channel corresponding to each color channel by the corresponding second weight coefficient to obtain the second weight bright channel atmospheric light value corresponding to each color channel; respectively summing the first weight dark channel atmospheric light value corresponding to each color channel and the second weight bright channel atmospheric light value corresponding to each color channel to determine the atmospheric light value corresponding to each color channel.

[0092] In the implementation of the present disclosure, the method for defogging based on the to-be-processed mine color image, its corresponding transmittance image, and the atmospheric light value corresponding to each color channel to obtain a defogged color image includes: obtaining a set protection factor; respectively calculating the difference between each color channel pixel in the to-be-processed mine color image and its corresponding atmospheric light value to obtain the pixel difference corresponding to each color channel; respectively determining the maximum value corresponding to each color channel between each color channel pixel in the transmittance image and the set protection factor; respectively calculating the ratio corresponding to each color channel between the pixel difference of each color channel and the maximum value of the corresponding color channel; adding the ratio corresponding to each color channel to the atmospheric light value of the corresponding color channel to obtain a defogged color image.

[0093] In the embodiments of the present disclosure and other possible embodiments, this algorithm improves the estimation of the atmospheric light value, selects the first group of bright pixel points Ω dark (x) in each channel image of the dark channel prior image I dark (x) that are located in the top 0.1% (the top 0.1% corresponding to the pixel values of each channel image in the dark channel prior image I dark (x) sorted from large to small), and respectively sums and averages these first group of bright pixel points Ω dark (x) in each channel image of the dark channel prior image I dark (x) to obtain the average value of the first group of bright pixel points corresponding to each channel; and configure the average value of the first group of bright pixel points corresponding to each channel as the dark channel atmospheric light estimation value A dark ; in the same way, select the second group of bright pixel points Ω bright (x) in each channel image of the bright channel prior image I bright (x) that are located in the top 0.1% (the top 0.1% corresponding to the pixel values of each channel image in the bright channel prior image I bright (x) sorted from large to small), sum and average these second group of bright pixel points Ω bright (x) to obtain the average value of the second group of bright pixel points corresponding to each channel; and configure the average value of the second group of bright pixel points corresponding to each channel as the bright channel atmospheric light estimation value A bright . Finally, by weighting the dark channel atmospheric light estimation value A dark and the bright channel atmospheric light estimation value A bright , obtain the atmospheric light value A0 corresponding to the foggy mine color image (the to-be-processed mine color image), and the definition formula is as follows:

[0094] A0 = αA dark + βA bright

[0095] Among them, the sum of the first weight coefficient α and the second weight coefficient β is 1. Here, the first weight coefficient α = 0.5 and the second weight coefficient β = 0.5. Specifically, the above atmospheric light value A0 includes the atmospheric light values corresponding to each channel respectively. Among them, C = {R, G, B} or {r, g, b}.

[0096] Therefore, in the embodiments of the present disclosure and other possible embodiments, the atmospheric light value A0 corresponding to the foggy mine color image (the to-be-processed mine color image) is estimated from the foggy mine color image (the to-be-processed mine color image) after adding the dark channel prior image DCP and the bright channel prior image BCP to the foggy mine color image (the to-be-processed mine color image). By characterizing the bright channel atmospheric light estimation value A corresponding to the atmospheric depth or coverage of the foggy mine color image (the to-be-processed mine color image). bright Thus, the atmospheric light value A0 can be estimated more accurately. In a night environment or in a mine roadway where there is a lack of natural light, due to the influence of various light sources in the mine, the brightness of the atmospheric light may not be as bright and constant as in the daytime. Therefore, only the first group of bright pixel points Ω corresponding to the dark channel prior image I datk (x) dark (x) for estimating the atmospheric light value A0 corresponding to the foggy mine color image (the to-be-processed mine color image) may cause noise. Furthermore, in this study, the dark channel atmospheric light estimation value A dark and the bright channel atmospheric light estimation value A bright corresponding weighted values are used to estimate the atmospheric light value A0. This method helps to reduce the noise problem brought by the first group of bright pixel points Ω corresponding to the dark channel prior image I dark (x), thereby improving the accuracy and robustness of the atmospheric light value A0 estimation. dark (x).

[0097] In the embodiments of the present disclosure and other possible embodiments, a downsampling optimization algorithm for foggy mine color images (mine color images to be processed): An inherent defect of the above-mentioned dark channel prior algorithm is its high time complexity, and a large number of color image acquisitions are required in the input stage of the 3D reconstruction process. To solve the problem of slow defogging speed caused by the high time complexity of the dark channel prior algorithm, a downsampling optimization algorithm for foggy mine color images (mine color images to be processed) is proposed. This downsampling optimization algorithm can provide the global features of foggy mine color images by downsampling foggy mine color images (mine color images to be processed), and the transmittance calculation on the corresponding downsampled images of foggy mine color images will significantly reduce the computational complexity. Through experimental comparison, this study proposes a downsampling optimization algorithm for foggy mine color images, which uses the bilinear interpolation algorithm to reduce the foggy mine color image to one-fourth of its original size to obtain the mine color downsampled image I L (x); On this downsampled mine color image I L (x), first calculate the transmission map estimate t(x) corresponding to the downsampled mine color image; Then, based on the transmission map estimate t(x), use a gradient domain weighted convolution filter operator (such as, sobel operator) to calculate the gradient map G(x) corresponding to the transmission map estimate t(x), and normalize the gradient map G(x) to obtain the normalized gradient map G~(x); Use the normalized gradient map G~(x) to construct the gradient response weight map w(x); Based on the gradient response weight map w(x) and the transmission map estimate t(x), calculate the refined transmittance t~(x); Finally, through the bicubic interpolation algorithm, upsample these calculated transmittances to map them to the original size corresponding to the foggy mine color image to obtain the complete transmittance image res(x).

[0098] In the embodiments of the present disclosure and other possible embodiments, the advantage of this downsampling optimization algorithm for foggy mine color images is that by performing calculations on the corresponding downsampled mine color images of foggy mine color images, the computational burden can be greatly reduced while maintaining the accuracy of the algorithm, especially for large-sized foggy mine color images. Based on the proposed downsampling optimization algorithm for foggy mine color images, the execution speed of the defogging algorithm for the color image to be processed / color image to be restored (foggy mine color image) can be effectively improved, and the process of obtaining the transmittance image res(x) corresponding to the mine color image through the bicubic interpolation algorithm helps to maintain the quality of the mine color image.

[0099] In the embodiments of the present disclosure and other possible embodiments, for the transmittance image mapping algorithm: in the process of estimating the transmission map t(x), there is an assumption that the transmission in the local region Ω(x) is constant, and the atmospheric light value A0 includes the atmospheric light values corresponding to each channel respectively. Where C = {R, G, B} or {r, g, b}, and it is always positive. Since the transmittance of the sky part or the intensity-uniform region tends to zero, the atmospheric light values corresponding to each channel When dealing with the regions where the transmittance of the sky and the intensity-uniform region tends to zero, problems such as color distortion are likely to occur. To avoid this situation, a weight constant ω is introduced. Finally, the calculation formula corresponding to the transmission map estimate t(x) for each channel in the three channels is:

[0100]

[0101] In the formula, 0 ≤ ω ≤ 1, the second local region Ω(x) = 20×20 is set, the mine color downsampled image I L (x), 1 represents the identity matrix; C = {R, G, B} or {r, g, b} represents the R / r channel, G / g channel, B / b channel, and min() represents the minimum function. The gradient map G(x) corresponding to the transmission map estimate t(x) is calculated by using the gradient domain weighted convolution filtering operator, and the gradient map G(x) is normalized to obtain the normalized gradient map Using the normalized gradient map Construct the gradient response weight map w(x); calculate the refined transmittance based on the gradient response weight map w(x) and the transmission map estimate t(x) Finally, these calculated transmittances are upsampled by the bicubic interpolation algorithm and mapped to the original size corresponding to the foggy mine color image to obtain the complete transmittance image res(x).

[0102] In the embodiments of the present disclosure and other possible embodiments, image defogging processing is performed on the transmittance image res(x), and the specific formula is as follows:

[0103]

[0104] In the above formula, I(x) is the foggy mine color image (the mine color image to be processed); t0 is the set protection factor. When the transmittance is too small, problems such as distortion of the restored image (foggy mine color image / mine color image to be processed / mine color image to be restored) are likely to occur. Therefore, t0 is used for improvement. According to the dark channel prior theory, specifically, t0 = 0.1 is taken.

[0105] In the embodiments of the present disclosure and other possible embodiments, the above-mentioned atmospheric light value A0 includes the atmospheric light values corresponding to each channel respectively. Among them, C = {R, G, B} or {r, g, b}. Furthermore, the above I(x) - A0 means that each pixel in the R-channel image of the three-channel image in the foggy mine color image / color image to be processed I(x) is respectively subtracted from the estimated value of the atmospheric light corresponding to the R channel. Each pixel in the G-channel image of the three-channel image in the color image to be processed I(x) is respectively subtracted from the estimated value of the atmospheric light corresponding to the G channel. Each pixel in the B-channel image of the three-channel image in the color image to be processed I(x) is respectively subtracted from the estimated value of the atmospheric light corresponding to the B channel. Similarly, max(res(x), t0) represents the maximum value between the pixel values of the R-channel transmittance image, G-channel transmittance image, and B-channel transmittance image corresponding to the transmittance image res(x) and the set protection factor.

[0106] In the embodiments of the present disclosure and other possible embodiments, similarly, represents the corresponding to the R channel, G channel, and B channels plus A0 corresponding to the respective channels (the estimated value of the atmospheric light for the R channel the estimated value of the atmospheric light for the G channel the estimated value of the atmospheric light for the G channel ).

[0107]

[0108] Figure 4 Shows a foggy underground passage color image according to an embodiment of the present disclosure and its corresponding dark channel prior image before algorithm improvement and dark channel prior image after algorithm improvement; Figure 5 Shows a foggy mine color image (mine color image to be processed) according to an embodiment of the present disclosure and its corresponding dark channel prior image before algorithm improvement and dark channel prior image after algorithm improvement. As Figure 4 and Figure 5 shown, in this study, foggy mine color images (color images to be processed) of underground passages and coal mine tunnels are selected for algorithm comparison. The leftmost one is the foggy mine color image, the middle one is the defogged color image obtained by using the traditional dark channel prior algorithm for defogging, and the rightmost one is the defogged color image obtained by using the improved dark channel prior algorithm (fusion algorithm based on mine color fusion image + downsampling optimization algorithm for foggy mine color image + transmittance image mapping algorithm) for defogging. From Figure 4 and Figure 5 it can be observed that the improved dark channel prior algorithm has a better defogging effect, and the defogging effect in the bright area of the foggy mine color image (color image to be processed) is more thorough. The problem of poor defogging effect in the bright area existing in the traditional dark channel prior algorithm has been improved.

[0109] In the implementation of the present disclosure, the method for performing image enhancement on the dehazed color image to obtain a dehazed color balanced image includes: converting the dehazed color image in the RGB space to a visible photon subset image corresponding to the HSV space; based on the visible photon subset image, determining a reflection component corresponding to the luminance information V(x) component in the visible photon subset image; performing non-linear stretching on the reflection component to obtain a non-linearly stretched reflection component; based on the logarithmic illuminance corresponding to the luminance information V(x) component and the non-linearly stretched reflection component, updating the luminance information V(x) component to obtain an updated luminance information V(x) component; fusing the updated luminance information V(x) component with the H(x) component and the S(x) component in the visible photon subset image to obtain an HSV fused dehazed color balanced image; and converting the HSV fused dehazed color balanced image corresponding to the HSV space to the RGB space to obtain a dehazed color balanced image.

[0110] In the embodiments of the present disclosure and other possible embodiments, the method for determining a reflection component corresponding to the luminance information V(x) component in the visible photon subset image based on the visible photon subset image includes: extracting the luminance information V(x) component corresponding to the visible photon subset image; using a guided filter technique to estimate the illuminance corresponding to the luminance information V(x) component; respectively converting the luminance information V(x) component and its corresponding illuminance to the logarithmic domain to obtain a logarithmic luminance information V(x) component and a logarithmic illuminance; and subtracting the corresponding logarithmic illuminance from the logarithmic luminance information V(x) component to determine the reflection component corresponding to the luminance information V(x) component.

[0111] In the implementation of the present disclosure, the method for performing non-linear stretching on the reflection component to obtain a non-linearly stretched reflection component includes: performing normalization processing on the reflection component to obtain a normalized reflection component; and performing non-linear stretching on the normalized reflection component to obtain a non-linearly stretched reflection component.

[0112] In the embodiments of the present disclosure and other possible embodiments, after performing the improved dark channel prior algorithm (fusion algorithm based on mine color fusion image + downsampling optimization algorithm for foggy mine color image + transmittance image mapping algorithm), this study uses the color image enhancement algorithm Retinex to perform color image enhancement on the dehazed color image after dehazing; wherein, the color image enhancement algorithm can be configured as the Retinex algorithm.

[0113] In the embodiments of the present disclosure and other possible embodiments, the Retinex algorithm corresponding to the color image enhancement algorithm includes: First, perform a conversion from the RGB subset to the visible photon subset HSV space on the dehazed color image to be processed, and then extract the luminance information V component from the visible photon subset HSV to ensure the protection of the color characteristics of the dehazed color image in subsequent processing; Next, to prevent halos and blurs in the dehazed color image, use an enhanced adaptive guided filtering algorithm to estimate the illuminance of the luminance information V component in the dehazed color image of the image, and convert the luminance information V component and its corresponding illuminance to the logarithmic domain (take the logarithm) respectively to obtain the logarithmic luminance information V component and the logarithmic illuminance; In the logarithmic domain, subtract the corresponding logarithmic illuminance from the logarithmic luminance information V component to determine the reflection component corresponding to the luminance information V component. According to the Retinex theory, the illuminance is mainly composed of low-frequency data in the dehazed color image, and these low-frequency data represent the object illumination intensity from the light source. Therefore, in order to reduce the illumination non-uniformity of the dehazed color image, it is necessary to perform a balance change process on the dehazed color image to obtain a dehazed color balance image. Specifically, the method for performing a balance change process on the dehazed color image to obtain a dehazed color balance image includes: performing non-linear stretching on the reflection component corresponding to the luminance information V component to obtain a non-linearly stretched reflection component to solve the problem of loss of details in mine photos; Further, in order to prevent misestimation of the enhancement effect by the non-linearly stretched reflection component, it is necessary to combine the reflection component and its corresponding non-linearly stretched reflection component together to create a new luminance information V component (update the luminance information V component); In order to achieve the final effect of dehazed color image enhancement, the new luminance information V component is fused with the original H component and S component to obtain an HSV fused dehazed color balance image, and then the HSV fused dehazed color balance image is converted back to the RGB space to obtain a dehazed color balance image.

[0114] In the embodiments of the present disclosure and other possible embodiments, for the overall process of the fusion algorithm based on the mine color fusion image: ① Select the dark channel prior image I dark (x), and for each channel image in the dark channel prior image I dark (x) (the first 0.1% corresponding to the pixel values of each channel image in the dark channel prior image I dark (x) sorted from largest to smallest), the first set of bright pixel points Ω dark (x), and sum and average the corresponding first set of bright pixel points Ω dark (x) of each channel image in these dark channel prior images I dark to obtain the average value of the first set of bright pixel points corresponding to each channel; and configure the average value of the first set of bright pixel points corresponding to each channel as the dark channel atmospheric light estimation value A dark ; In the same way, select the bright channel prior image I brightThe first 0.1% where each channel image in (x) is located (the first 0.1% corresponding to the bright channel prior image I broght of the pixel values of each channel image in (x) sorted from largest to smallest) corresponding to the second set of bright pixel points Ω bright (x), for these second set of bright pixel points Ω bright (x), sum and average them to obtain the average value of the second set of bright pixel points corresponding to each channel; and configure the average value of the second set of bright pixel points corresponding to each channel as the bright channel atmospheric light estimation value A bright . Finally, by weighting the dark channel atmospheric light estimation value A dark and the bright channel atmospheric light estimation value A bright , obtain the atmospheric light value A0 corresponding to the foggy mine color image. The above atmospheric light value A0 respectively includes the atmospheric light value corresponding to each channel where C = {R, G, B} or {r, g, b}. ② Use the bilinear interpolation algorithm to reduce the foggy mine color image to one-fourth of its original size to obtain the mine color downsampled image I L (x); on this size-reduced mine color downsampled image I L (x), first calculate the transmission map estimation t(x) corresponding to the mine color downsampled image; then, based on the transmission map estimation t(x), use an edge detection operator or a gradient calculation operator (such as, sobel operator) to calculate the gradient map G(x) corresponding to the transmission map estimation t(x), and normalize the gradient map G(x) to obtain the normalized gradient map Use the normalized gradient map to construct the gradient response weight map w(x); based on the gradient response weight map w(x) and the transmission map estimation t(x), calculate the refined transmittance Finally, upsample these calculated transmittances by the bicubic interpolation algorithm, map them to the original size corresponding to the foggy mine color image to obtain the complete transmittance image res(x). ③ Add a weight constant ω to the transmittance image mapping algorithm, and calculate the refined transmittance through the gradient domain weighted convolution filtering operator to reduce the color distortion problem. ④ Perform the Retinex algorithm on the dehazed color image corresponding to the processed transmittance image res(x) to obtain the dehazed color balanced image and improve problems such as uneven illumination.

[0115] Step S103: Extract the first edge information and the second edge information of the registered dehazed color balanced image corresponding to the to-be-repaired mine depth image and the registered dehazed color balanced image respectively.

[0116] In the implementation of the present disclosure, the method for separately extracting the first edge information and the second edge information of the registered haze-removed color-balanced image corresponding to the depth image of the mine to be repaired includes: using an edge extraction algorithm to separately extract the edges of the depth image of the mine to be repaired and the registered haze-removed color-balanced image, so as to obtain the first edge information corresponding to the depth image of the mine to be repaired and the second edge information corresponding to the registered haze-removed color-balanced image.

[0117] In the embodiments of the present disclosure and other possible embodiments, a mine depth image hole repair algorithm based on RGB and edge information-guided joint bilateral filtering: When obtaining the mine depth image (color image to be repaired) using a Kinect camera in this study, a hazy mine color image (color image to be processed) will be obtained at the same time. Therefore, the proposed algorithm takes the mine depth image as the benchmark, transforms the pixel points of the hazy mine color image into the coordinate system of the mine depth image, and at the same time uses the rich texture information of the haze-removed color-balanced image corresponding to the mine depth image and the hazy mine color image to repair the holes in the mine depth map. Based on this, a mine depth image hole repair based on RGB and edge information-guided joint bilateral filtering is proposed, and the improvements of this algorithm are as follows: 1. RGB guidance map and 2. edge information.

[0118] In the embodiments of the present disclosure and other possible embodiments, RGB guidance map: In the joint bilateral filtering algorithm, Gaussian filtering is usually used to process the mine depth map to obtain the low-frequency depth map corresponding to the mine depth image, and the low-frequency pixel information of the mine depth image is used to replace the pixel intensity value of the mine depth image. In the acquisition stage of the mine depth image in this study, a hazy mine color image and a mine depth image are obtained synchronously through a Kinect camera. By registering the haze-removed color-balanced image corresponding to the hazy mine color image with the mine depth image, the coordinate correspondence between the pixel points of the haze-removed color-balanced image corresponding to the hazy mine color image and the pixel points of the mine depth image can be determined.

[0119] In the embodiments of the present disclosure and other possible embodiments, in the dehazed color balance image corresponding to the foggy mine color image, each pixel is composed of three channels: red (R), green (G), and blue (B). By performing weighted averaging on the pixels of the three channels, the gray-scale information of the mine gray-scale image corresponding to the dehazed color balance image can be determined. Each pixel in the mine gray-scale image is a single-channel brightness value. Although each pixel in the mine gray-scale image no longer contains color detail information, the texture structure and brightness distribution of the dehazed color balance image can be retained. In this study, the dehazed color balance image is converted into a mine gray-scale image, and the gray-scale value p of the mine gray-scale image is used to configure the pixel intensity weight of the joint bilateral filtering algorithm to repair the holes in the mine depth image. The pixel intensity weight of the mine gray-scale image can be expressed by the following formula:

[0120]

[0121] In the embodiments of the present disclosure and other possible embodiments, the pixel intensity weight of the gray-scale image in the above formula is used to replace the pixel intensity weight in the original bilateral joint filtering.

[0122] Step S104: Determine the fused edge information according to the first edge information and the second edge information.

[0123] In the implementation of the present disclosure, the method for determining the fused edge information according to the first edge information and the second edge information includes: performing a pixel-by-pixel comparison on the first edge binary image and the second edge binary image corresponding to the first edge information and the second edge information to obtain the fused edge information.

[0124] In the implementation of the present disclosure, the method for performing a pixel-by-pixel comparison on the first edge binary image and the second edge binary image corresponding to the first edge information and the second edge information to obtain the fused edge information includes: performing a logical OR operation on the first edge binary image and the second edge binary image corresponding to the first edge information and the second edge information on a pixel-by-pixel basis to obtain the fused edge information.

[0125] Step S105: Based on the fused edge information and the mine gray-scale image corresponding to the registered dehazed color balance image, use a joint bilateral filter to repair the pixels to be repaired in the to-be-repaired mine depth image.

[0126] In the implementation of the present disclosure, the method for repairing the pixels to be repaired in the to-be-repaired mine depth image by using a joint bilateral filter based on the fused edge information and the mine gray-scale image corresponding to the registered haze-removed color-balanced image includes: obtaining a judgment function corresponding to the fused edge information; based on the judgment function and the fused edge information, judging whether the pixels adjacent to the pixels to be repaired in the to-be-repaired mine depth image participate in the joint bilateral filtering; if the pixels adjacent to the pixels to be repaired participate in the joint bilateral filtering, then based on the pixels adjacent to the pixels to be repaired and the mine gray-scale image corresponding to the registered haze-removed color-balanced image, using the joint bilateral filter to repair the pixels to be repaired in the to-be-repaired mine depth image.

[0127] In the implementation of the present disclosure, the method for judging whether the pixels adjacent to the pixels to be repaired in the to-be-repaired mine depth image participate in the joint bilateral filtering based on the judgment function and the fused edge information includes: when the pixel to be repaired is a missing pixel or the pixels adjacent to the pixel to be repaired are not the edge pixels corresponding to the fused edge information and the pixel to be repaired is the edge pixel corresponding to the fused edge information, the pixels adjacent to the pixel to be repaired participate in the joint bilateral filtering; otherwise, the pixels adjacent to the pixel to be repaired do not participate in the joint bilateral filtering.

[0128] In the implementation of the present disclosure, the method for repairing the pixels to be repaired in the to-be-repaired mine depth image by using a joint bilateral filter based on the pixels adjacent to the pixels to be repaired and the mine gray-scale image corresponding to the registered haze-removed color-balanced image includes: performing weighted averaging on the pixels corresponding to each color channel in the registered haze-removed color-balanced image to obtain the mine gray-scale image corresponding to the registered haze-removed color-balanced image; based on the mine gray-scale image, respectively calculating the pixel difference weight and the Gaussian distribution distance weight value corresponding to the mine gray-scale image; based on the pixel difference weight and the Gaussian distribution distance weight value corresponding to the mine gray-scale image, and the pixels adjacent to the pixels to be repaired, using the joint bilateral filter to repair the pixels to be repaired in the to-be-repaired mine depth image.

[0129] In the embodiments of the present disclosure and other possible embodiments, edge information: In the mine depth image collected by the RGB-D camera, most of the pixels lacking depth information exist in the edge area of the mine depth map. Many common algorithms repair the void pixels in the mine depth image by taking the weighted average of the neighboring pixels of the void pixels. However, for the edge area of the mine depth image, it often involves the neighboring pixels of the foreground image and the neighboring pixels of the background image. Without additional information, it is very difficult to determine whether the void pixel belongs to the foreground image or the background image. If the void pixels in the background image are restored through the pixels in the foreground image, it will result in the missing edge information corresponding to the edge area in the mine depth image after weighted average repair.

[0130] Therefore, in the embodiments of the present disclosure and other possible embodiments, the algorithm uses an edge extraction operator (such as, Canny operator) to extract edge information from the color image and the depth image respectively. The extracted edge information will be used as the pixel validity weight to participate in the repair of the void pixels in the mine depth image. The specific steps for edge information extraction of the defogged color balance color image and the mine depth image are as follows:

[0131] (1) Use the edge extraction operator (such as, Canny operator) to extract edge information from the mine depth image and the defogged color balance image registered with the mine depth image respectively, to obtain the first edge information and the second edge information corresponding to the defogged color balance color image and the mine depth image.

[0132] (2) Compare the first edge information and the second edge information of the two images, the defogged color balance color image and the mine depth image, with the first edge binary image and the second edge binary image pixel by pixel, and use the logical OR operation to obtain the fused edge information E.

[0133] (3) In the joint bilateral filter, add the extracted fused edge information E as the judgment information. For the pixel to be repaired in the mine depth image, when the pixel to be repaired i is a missing pixel or the neighboring pixel j of the pixel to be repaired i is not the edge pixel (valid pixel) corresponding to the fused edge information E and is the edge pixel corresponding to the same area as the pixel to be repaired i in the fused edge information E, the pixel j participates in the joint bilateral filtering process.

[0134] Therefore, in the embodiments of the present disclosure and other possible embodiments, based on the above two analyses, the joint bilateral filter is improved. The corresponding expression of the improved joint bilateral filter is as follows:

[0135]

[0136] In the formula, represents the mine depth image corresponding to the pixel j adjacent to the pixel to be repaired i; wd (i, j) represents the distance weight corresponding to the pixel i to be repaired in the Gaussian distribution of the Gaussian distribution distance weight image of the mine grayscale image and the pixel j adjacent to the pixel i to be repaired; is the pixel intensity weight corresponding to the pixel i to be repaired in the Gaussian distribution of the mine grayscale image and the pixel j adjacent to the pixel i to be repaired; H(E j ) is a judgment function for whether the pixel i participates in the joint bilateral filtering process based on the fused edge information E; Ω(i) represents the neighborhood centered on the pixel i to be repaired; K i is a normalization constant used to ensure that the sum of the weights of the filter is 1.

[0137] Figure 6 shows the filtering window corresponding to the improved joint bilateral filter according to an embodiment of the present disclosure. As Figure 6 shown, the filtering window corresponding to the improved joint bilateral filter. The improved joint bilateral filter proposed by this algorithm combines the trade-off between the grayscale difference of the mine grayscale image corresponding to the defogged color-balanced color image and the spatial distance corresponding to the pixel i to be repaired and the pixel j adjacent to the pixel i to be repaired in the mine depth image. At the same time, the introduction of the fused edge information E can further improve the ability of the joint bilateral filter to maintain the structure of the first edge binary image and the second edge binary image. The improved joint bilateral filter can more accurately identify the boundary of the hole in the mine depth image and avoid the blurring of the hole information.

[0138] In the embodiments of the present disclosure and other possible embodiments, through the above-mentioned improved joint bilateral filter, the processing of the edge pixels corresponding to the fused edge information E pays more attention to retaining the edge information, while the processing of the non-edge pixels is more inclined to overall smoothing, realizing the repair of large-scale holes in the mine depth image. The above-mentioned improved joint bilateral filter enables the hole repair algorithm of the mine depth image based on RGB and edge information-guided joint bilateral filtering to be flexibly adjusted in different texture and hole regions, and better repair the holes in the mine depth image.

[0139] For the fusion algorithm based on the mine color fusion image, objective indicators such as structural similarity (SSIM), peak signal-to-noise ratio (PSNR), and root mean square error (RMSE) are selected to quantitatively analyze the processing effect of the image. For the above four different algorithms, using the I-HAZE, O-HAZE, and self-made datasets, the average values of the image SSIM, PSNR, and RMSE are calculated respectively, and the results are shown in the following table.

[0140] Table 1 Structural similarity index of color image processing

[0141]

[0142] Table 2 Peak signal-to-noise ratio index of color image processing

[0143]

[0144] Table 3 Root Mean Square Error Index for Color Image Processing

[0145]

[0146] SSIM is used to measure the similarity between two images, comprehensively considering their similarity in terms of brightness, contrast, and structure. It measures the similarity between them by comparing the original image and the transformed image. The value of SSIM ranges from -1 to 1, and the closer the value is to 1, the more similar the images are. As can be seen from Table 1, compared with other existing algorithms, in the indoor I-HAZE dataset and the self-made underground roadway dataset, this algorithm achieves the highest value in the average of the structural similarity evaluation index, reflecting that higher image result integrity can be maintained after algorithm processing. However, in terms of the processing effect on the O-HAZE dataset, this algorithm is not as ideal as the original DCP algorithm. This is because the DCP algorithm was originally an algorithm for processing outdoor images, and the research improved the estimation of the atmospheric light value indoors. When processing the O-HAZE data in the outdoor environment, inaccurate estimation of the atmospheric light value will occur in this algorithm.

[0147] Peak Signal-to-Noise Ratio (PSNR) is used to measure the quality of signals such as images or audio, and is usually used to compare the differences between the original signal and the processed signal. A higher PSNR value indicates a smaller difference between the two and higher quality. As can be seen from Table 2 above, on the I-HAZE dataset and the self-made dataset, this algorithm has achieved a significant improvement in PSNR. By comparing the PSNR values, it can be observed that the image quality has been significantly improved under the processing of this algorithm. This is mainly due to the improved estimation of atmospheric light and the fusion of the Retinex algorithm after the defogging process. These improvements reduce the noise in the image and increase the ratio of the signal to noise in the image.

[0148] Root Mean Square Error (RMSE) is used to measure the difference between two images. By calculating the difference between the predicted value and the actual value of each pixel, then squaring, averaging, and taking the square root to obtain the final error value. The smaller the RMSE, the smaller the difference between the images, that is, the more similar they are. By comparing the RMSE in Table 3, this algorithm performs worse than the initial DCP algorithm when processing O-HAZE outdoor images, but when processing indoor images in the I-HAZE dataset and the self-built dataset, it can reduce the error generated during image processing and retain the detailed information of the image, meeting the requirements of subsequent 3D reconstruction.

[0149] For the objective quantitative analysis of the depth image hole repair algorithm based on improved joint bilateral filtering, structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) are used as the basis for the quantitative analysis results. Still use the SAMF algorithm, FMM algorithm, JBF algorithm and this algorithm to repair the four depth images of Aloe, Lampshade1, Rocks1, and Rocks2 in the Middlebury 2006 dataset, and calculate the structural similarity value and peak signal-to-noise ratio value of the repaired images. The experimental results are as follows.

[0150] Table 4 Structural Similarity Index for Depth Map Hole Repair

[0151]

[0152] Table 5 Peak Signal-to-Noise Ratio Index for Depth Map Hole Repair

[0153]

[0154]

[0155] As can be seen from the above table, the structural similarity and peak signal-to-noise ratio of the image repaired by the SAMF algorithm are the smallest, indicating that the structural change of the depth image repaired by the SAMF algorithm is the largest and the noise amplification is the most serious; the two attribute values of the structural similarity and peak signal-to-noise ratio of the FMM algorithm and the JBF algorithm are increased compared with the SAMF algorithm, indicating that there is an improvement in image texture distortion and noise amplification; while the structural similarity and peak signal-to-noise ratio of the image repaired by this algorithm are the largest, indicating that this algorithm has higher accuracy and better quality for depth image repair. Therefore, compared with the other three algorithms, this algorithm has the best repair effect on depth image holes.

[0156] In the embodiments of the present disclosure and other possible embodiments, the specific steps of the present invention are further described in detail.

[0157] Step 1: The input image I(x) is a foggy mine color image (the mine color image to be processed), with a corresponding size of 1920×1080. The foggy mine color image (the mine color image to be processed) contains low contrast and blurred details affected by dust, haze, etc. in the mine environment.

[0158] Step 2: The calculation formulas for the bright channel prior image I bright (x) and the dark channel prior image I dark (x) are as follows:

[0159]

[0160] Among them, I dark(x) is the dark channel prior image of the color image I(x) to be processed; I bright (x) is the bright channel prior image of the color image I(x) to be processed; I c (x) is the channel image corresponding to different color channels, that is, the channel image corresponding to any R channel, G channel, and B channel in the channel set C; Ω(x) is the set first local area of 3×3 around the pixel x.

[0161] Step three: Select the dark channel prior image I dark (x), and select the first group of bright pixel points Ω dark (x) corresponding to the top 0.1% (the top 0.1% corresponding to the pixel values of each channel image in the dark channel prior image I dark (x) sorted from large to small); in the same way, select the bright channel prior image I bright (x), and select the second group of bright pixel points Ω bright (x) corresponding to the top 0.1% (the top 0.1% corresponding to the pixel values of each channel image in the bright channel prior image I bright (x) sorted from large to small).

[0162] Ω dark (x) = {x ∈ I(x)|x ∈ top 0.1％ (I dark (x))}

[0163] Ω bright (x) = {x ∈ I(x)|x ∈ top 0.1％ (I bright (x))}

[0164] Step four: Sum and average the three channels (color channels) of the pixels in the dark channel of the first group of bright pixel points Ω dark (x) to obtain the atmospheric light value A dark of each color channel in the dark channel corresponding to the first group of bright pixel points. Sum and average the three channels (color channels) of the pixel points in the bright channel of the second group of bright pixel points Ω bright (x) to obtain the atmospheric light value A bright of each color channel in the bright channel corresponding to the second group of bright pixel points. Finally, obtain the new atmospheric light value A0 by weighting the atmospheric light value A dark of each color channel in the dark channel and the atmospheric light value A bright of each color channel in the bright channel. The definition formula is as follows:

[0165]

[0166] A0 = αA dark + βAbright

[0167] where size(Ω dark (x)) is the number of pixels contained in the set Ω dark (x). The sum of the first weight coefficient α and the first weight coefficient β is 1. Here, in the study, α = 0.5 and β = 0.5 are taken.

[0168] Specifically, for each channel image of these dark channel prior images I dark (x), the corresponding first group of bright pixel points Ω dark (x) are summed and averaged to obtain the average value of the first group of bright pixel points corresponding to each channel (the number of the first group of bright pixel points corresponding to each channel ∑ x∈Ωdark x) divided by the total number of the first pixel points size(Ω dark (x)) corresponding to the dark channel prior image I dark (x) of the corresponding channel; and the average value of the first group of bright pixel points corresponding to each channel is configured as the dark channel atmospheric light estimation value (the atmospheric light value of each color channel in the dark channel) A dark .

[0169] Specifically, for each channel, the corresponding second group of bright pixel points Ω bright (x) are summed and averaged to obtain the average value of the second group of bright pixel points corresponding to each channel (the number of the second group of bright pixel points corresponding to each channel ∑ x∈Ωbright x) divided by the total number of the second pixel points size(Ω dark (x)) corresponding to the dark channel prior image I bright (x) of the corresponding channel; and the average value of the second group of bright pixel points corresponding to each channel is configured as the bright channel atmospheric light estimation value (the atmospheric light value of each color channel in the bright channel) A bright .

[0170] Step 4: The foggy mine color image I(x) is downsampled by the bilinear interpolation method and reduced to one-fourth of its original size to obtain the mine color downsampled image I L (x). Then, the transmission map estimation t(x) is performed on the downsampled mine color downsampled image I L (x). The formula for the transmission map estimation is as follows:

[0171]

[0172] In the formula, the weight constant ω, 0 ≤ ω ≤ 1, Ω(x) = 20×20 is the set second local area of 20×20 around the pixel x, and A0 is the atmospheric light value corresponding to the three channels of r, g, and b.

[0173] Step 5: Use the Sobel operator to calculate the gradient corresponding to the transmission map estimate t(x), and obtain the gradient map G(x) corresponding to the transmission map estimate t(x); after the calculation is completed, normalize the gradient map G(x) to obtain the normalized gradient map Use the normalized gradient map The specific formula is as follows:

[0174] G x (x) = t(x i+1 , x i ) - t(x i-1 , x i )

[0175] G y (x) = t(x i , x + x i+1 ) - t(x i , x i-1 )

[0176]

[0177] where i represents the coordinate position corresponding to the pixel x; G x (x) represents the gradient of the transmission map estimate t(x) in the x direction; G y (x) represents the gradient of the transmission map estimate t(x) in the y direction; ε represents a set parameter greater than 0 but infinitely close to 0;

[0178] Step 6: The normalized gradient map obtained Construct the gradient response weight map w(x):

[0179]

[0180] where β = 10.0 is a set adjustment parameter to control the edge attenuation speed.

[0181] Step 7: Based on the transmission map estimate t(x), use the gradient domain weighted convolution filtering operator to obtain the refined transmittance (transmittance image)

[0182]

[0183] In the formula, w(x) represents the gradient response weight map; t(x) represents t(x); Ω(x) is configured as a set second local region with a size of 20×20, and ε = 0.001 is a reserved decimal to prevent the denominator from being zero.

[0184] Step 8: The transmittance The corresponding image size is one-fourth of the size of the foggy mine color image. Use bicubic interpolation for the refined transmittance Perform upsampling to map the image size corresponding to the transmittance to the original size corresponding to the hazy mine color image, so as to obtain the complete transmittance image res(x).

[0185] Step Nine: Perform image defogging processing on the transmittance image res(x), and the specific formula is as follows:

[0186]

[0187] In the above formula, I(x) is the hazy mine color image (the mine color image to be processed / the mine color image to be repaired); t0 is the set protection factor. When the transmittance is too small, there is a problem of distortion in the restored image (the hazy mine color image / the mine color image to be processed / the mine color image to be repaired). Therefore, t0 is used for improvement. According to the dark channel prior theory, specifically, t0 = 0.1.

[0188] The above atmospheric light value A0 respectively includes the atmospheric light value corresponding to each channel where C = {R, G, B} or {r, g, b}. Furthermore, the above I(x) - A0 means that each pixel of the R-channel image corresponding to the three-channel image of the hazy mine color image / to-be-processed color image I(x) is respectively subtracted from the estimated atmospheric light value of the corresponding R-channel each pixel of the G-channel image corresponding to the three-channel image of the to-be-processed color image I(x) is respectively subtracted from the estimated atmospheric light value of the corresponding G-channel each pixel of the B-channel image corresponding to the three-channel image of the to-be-processed color image I(x) is respectively subtracted from the estimated atmospheric light value of the corresponding B-channel Similarly, max(res(x), t0) represents the maximum value between the pixel values of the R-channel transmittance image, G-channel transmittance image, and B-channel transmittance image corresponding to the transmittance image res(x) and the set protection factor. Similarly, represents the corresponding to the R-channel, G-channel, and B-channels plus the A0 corresponding to the corresponding channel (the estimated atmospheric light value of the R-channel the estimated atmospheric light value of the G-channel the estimated atmospheric light value of the G-channel ).

[0189]

[0190]

[0191] Step Ten: After performing the improved dark channel prior algorithm (fusion algorithm based on mine color fusion images + downsampling optimization algorithm for mine color images + transmittance image mapping algorithm), this study uses the color image enhancement algorithm Retinex to enhance the dehazed color image. Among them, the color image enhancement algorithm can be configured as the Retinex algorithm.

[0192] Specific process: The dehazed color image J(x) is converted from RGB to HSV space to obtain the corresponding visible photon subset image HSV(x) of the dehazed color image. Subsequently, the luminance information V(x) component is extracted from the visible photon subset to ensure the protection of the color characteristics of the dehazed color image in subsequent processing.

[0193] The Retinex algorithm corresponding to the color image enhancement algorithm includes: First, the dehazed color image to be processed is converted from the RGB subset to the visible photon subset HSV space. Subsequently, the luminance information V component is extracted from the visible photon subset image HSV(x) corresponding to the visible photon subset HSV to ensure the protection of the color characteristics of the dehazed color image in subsequent processing.

[0194] Step Eleven: Use the enhanced adaptive guided filter technology to estimate the illuminance V(x) corresponding to the low-frequency part in the luminance information component V(x) of the image, and obtain the reflection component R(x) corresponding to the illuminance V(x) in the logarithmic domain, and normalize the reflection component R(x) to get L (x), and obtain the reflection component R(x) corresponding to the illuminance V(x) in the logarithmic domain, and normalize the reflection component R(x) to get

[0195] R(x) = log(V(x) + ε) - log(V L (x) + ε)

[0196]

[0197] Among them, ε is a small constant to prevent log0 when taking the logarithm and to prevent the denominator from being 0.

[0198] To prevent halos and blurs in the dehazed color image, use the enhanced adaptive guided filter algorithm to estimate the illuminance of the luminance information V component in the dehazed color image of the image, and convert the luminance information V component and its corresponding illuminance to the logarithmic domain (take the logarithm) respectively to obtain the logarithmic luminance information V component log(V(x)) and the logarithmic illuminance log(V L (x)); in the logarithmic domain, subtract the corresponding logarithmic illuminance from the logarithmic luminance information V component to determine the reflection component corresponding to the luminance information V component.

[0199] Step Twelve: Perform non-linear stretching on the normalized reflection component to obtain the non-linearly stretched reflection component to solve the problem of image detail loss.

[0200] For the V component of the luminance information and the illuminance V L corresponding normalized reflection component Perform non-linear stretching to obtain the non-linearly stretched reflection component R enh (x) to solve the problem of detail loss in mine photos.

[0201] Step thirteen: To prevent estimation errors from having a negative impact on the enhancement effect, it is necessary to combine the logarithmic illuminance log(V L (x)) and its corresponding non-linearly stretched reflection component R enh (x) together to create a new illumination component V new (x) (update the V component of the luminance information). To achieve the final enhancement effect, V new (x) is fused with the H(x) and S(X) components to obtain the HSV fused defogged color balance image HSV enh (x); then, it is converted back to the RGB space to obtain the defogged color balance image I enh (x).

[0202] V new (x) = exp(R enh (x) + log(V L (x) + ε))

[0203] HSV enh (x) = (H(x), S(x), V new (x))

[0204] According to the Retinex theory, the illuminance mainly consists of low-frequency data in the defogged color image, and these low-frequency data represent the object illumination intensity from the light source. Therefore, in order to reduce the illumination non-uniformity of the defogged color image, it is necessary to perform a balance change process on the defogged color image to obtain the defogged color balance image. Specifically, the method of performing a balance change process on the defogged color image to obtain the defogged color balance image includes: performing non-linear stretching on the reflection component corresponding to the V component of the luminance information to obtain the non-linearly stretched reflection component to solve the problem of detail loss in mine photos; further, to prevent estimation errors of the non-linearly stretched reflection component on the enhancement effect, it is necessary to combine the reflection component and its corresponding non-linearly stretched reflection component together to create a new V component of the luminance information (update the V component of the luminance information); to achieve the final enhancement effect of the defogged color image, the new V component of the luminance information is fused with the original H component and S component to obtain the HSV fused defogged color balance image HSV enh (x), and then the HSV fused defogged color balance image HSV enh (x) is converted back to the RGB space to obtain the defogged color balance image Ienh (x).

[0205] Step Fourteen: The enhanced haze-removed color-balanced image I enh (x) obtained above cannot be directly used for subsequent processing. It must be registered with the mine depth image to obtain the registered haze-removed color-balanced image I RGB (x).

[0206] Step Fifteen: Using the edge extraction algorithm corresponding to the Canny operator, extract the first edge information E RGB corresponding to the registered haze-removed color-balanced image I RGB and the second edge information E D corresponding to the mine depth image I D (x);

[0207] E D (x) = Canny(I D (x))

[0208] E RGB = Canny(I RGB (x))

[0209] Step Sixteen: Compare the first edge information and the second edge information of the registered haze-removed color-balanced color image and the mine depth image with the corresponding first edge binary image and second edge binary image pixel by pixel, and use the logical OR operation to obtain the fused edge information E(i, j).

[0210] E(i, j) = E D ∨E RGB

[0211] Step Seventeen: Introduce the judgment function H(E j ) corresponding to the fused edge information E when repairing the hole pixels in the mine depth image, and use the fused edge information E to judge which pixels are valid and which are missing. This can more accurately repair the hole area.

[0212]

[0213] In the joint bilateral filter, add the extracted fused edge information E as the judgment information. For the pixels to be repaired in the mine depth image, when the pixel i to be repaired is a missing pixel or the pixel j adjacent to the pixel i to be repaired is not the edge pixel (valid pixel) corresponding to the fused edge information E and is the edge pixel corresponding to the same area as the pixel i to be repaired in the fused edge information E, the pixel j participates in the joint bilateral filtering process.

[0214] Step Eighteen: By performing operations on the registered haze-removed color-balanced image I RGB(x)Perform weighted averaging on the pixels corresponding to the three channels to obtain the mine grayscale image I grey (x)Corresponding grayscale information.

[0215]

[0216] Step Nineteen: According to the mine grayscale image I grey (x)Calculate the corresponding pixel difference weight W g (I i ,I j ) and the Gaussian distribution distance weight w d (i,j).

[0217]

[0218] Where i is the coordinate position corresponding to the pixel point in the mine grayscale image I grey (x), j is the coordinate position of the pixel point surrounding i, and their relationship is j ∈ Ω(i), where the area size of Ω is 5×5. σ r Is the spatial distance standard deviation, σ d Is the pixel intensity standard deviation.

[0219] Step Twenty: In the joint bilateral filter, add the extracted edge information E as the judgment information. For the pixel i to be repaired, when the pixel i to be repaired is a missing pixel or the pixel j adjacent to the pixel i to be repaired is not the edge pixel (valid pixel) corresponding to the fused edge information E and is the edge pixel corresponding to the same area as the pixel i to be repaired in the fused edge information E, the pixel j participates in the joint bilateral filtering process. The specific formula for the above process is as follows:

[0220]

[0221] In the formula, Represents the mine depth image corresponding to the pixel i adjacent to the pixel i to be repaired; w d (i,j) Represents the distance weight of the Gaussian distribution of the mine grayscale image; Is the pixel intensity weight corresponding to the pixel i to be repaired in the Gaussian distribution of the mine grayscale image and the pixel i adjacent to the pixel i to be repaired; H(E j ) Is the judgment function for whether the pixel i participates in the joint bilateral filtering process based on the fused edge information E; Ω(i) represents the neighborhood centered on the pixel i to be repaired; K i Is the normalization constant used to ensure that the sum of the weights of the filter is 1.

[0222] The execution subject of the method for repairing a mine depth image can be a system or device for repairing a mine depth image. For example, the method for repairing a mine depth image can be executed by a terminal device, a server, or other processing devices. Among them, the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the method for repairing a mine depth image can be implemented by a processor calling computer-readable instructions stored in a memory.

[0223] Those skilled in the art can understand that in the above-mentioned method for repairing a mine depth image in the specific implementation manner, the writing order of each step does not mean a strict execution order and does not impose any limitation on the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic.

[0224] In some embodiments, the functions or modules included in the device provided in the embodiments of the present disclosure can be used to execute the method described in the embodiments of the above-mentioned method for repairing a mine depth image. Its specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0225] According to one aspect of the embodiments of the present disclosure, there is provided a device / system for repairing a mine depth image, including: an acquisition unit, configured to acquire a to-be-processed mine color image and a to-be-repaired mine depth image corresponding to the same mine environment object; a defogging and enhancement processing unit, configured to perform defogging and enhancement processing on the to-be-processed mine color image to obtain a defogged color balance image; an extraction unit, configured to respectively extract first edge information of the to-be-repaired mine depth image and second edge information of a registered defogged color balance image corresponding to the registered defogged color balance image thereof; a determination unit, configured to determine fused edge information according to the first edge information and the second edge information; and a repair unit, configured to repair to-be-repaired pixels in the to-be-repaired mine depth image by using a joint bilateral filter based on the fused edge information and a mine grayscale image corresponding to the registered defogged color balance image.

[0226] In an embodiment of the present disclosure, the haze removal and enhancement processing unit includes: a prior image extraction unit configured to extract a bright channel prior image and a dark channel prior image corresponding to the to-be-processed mine color image respectively; a transmittance image unit configured to determine a transmittance image according to the to-be-processed mine color image, the bright channel prior image, and the dark channel prior image; a haze removal color image processing unit configured to perform haze removal on the to-be-processed mine color image based on the to-be-processed mine color image, its corresponding transmittance image, and the atmospheric light value corresponding to each color channel to obtain a haze-removed color image; and an image enhancement processing unit configured to perform image enhancement on the haze-removed color image to obtain a haze-removed color balance image.

[0227] In an embodiment of the present disclosure, the transmittance image unit includes: an atmospheric light value determination unit configured to determine the atmospheric light value corresponding to each color channel according to the bright channel prior image and the prior image; wherein the atmospheric light value determination unit includes: a first determination unit, a second determination unit, and a third determination unit; the first determination unit is configured to multiply the dark channel atmospheric light value corresponding to each color channel by the corresponding first weight coefficient to obtain the first weighted dark channel atmospheric light value corresponding to each color channel; the second determination unit is configured to multiply the bright channel atmospheric light value corresponding to each color channel by the corresponding second weight coefficient to obtain the second weighted bright channel atmospheric light value corresponding to each color channel; and the third determination unit is configured to sum the first weighted dark channel atmospheric light value corresponding to each color channel and the second weighted bright channel atmospheric light value corresponding to each color channel respectively to determine the atmospheric light value corresponding to each color channel.

[0228] In an embodiment of the present disclosure, the image enhancement processing unit includes: a reflection component determination unit; the reflection component determination unit is configured to determine the reflection component corresponding to the luminance information V(x) component in the visible photon subset image based on the visible photon subset image corresponding to the haze-removed color image; wherein the reflection component determination unit includes: a luminance information V(x) component extraction unit configured to extract the luminance information V(x) component corresponding to the visible photon subset image; an illuminance estimation unit configured to estimate the illuminance corresponding to the luminance information V(x) component by using the guided filter technique; a logarithmic processing unit configured to convert the luminance information V(x) component and its corresponding illuminance into the logarithmic domain respectively to obtain the logarithmic luminance information V(x) component and the logarithmic illuminance; and a reflection component calculation unit configured to subtract the corresponding logarithmic illuminance from the logarithmic luminance information V(x) component to determine the reflection component corresponding to the luminance information V(x) component.

[0229] In an embodiment of the present disclosure, a dehazing color image processing unit includes: a set protection factor acquisition unit configured to acquire a set protection factor; a pixel difference calculation unit configured to calculate the difference between each color channel pixel in the to-be-processed mine color image and its corresponding atmospheric light value respectively, to obtain the pixel difference corresponding to each color channel; a maximum value determination unit configured to determine the maximum value corresponding to each color channel between each color channel pixel in the transmittance image and the set protection factor respectively; a ratio calculation unit configured to calculate the ratio corresponding to each color channel between the pixel difference of each color channel and the maximum value of the corresponding color channel respectively; and an addition unit configured to add the ratio corresponding to each color channel to the atmospheric light value of the corresponding color channel to obtain a dehazing color image.

[0230] According to one aspect of the embodiments of the present disclosure, a system for restoring a mine depth image is provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the above-mentioned method for restoring a mine depth image.

[0231] According to one aspect of the embodiments of the present disclosure, a system for restoring a mine depth image is provided, including: an electronic device; a processor configured on the electronic device; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above-mentioned method for restoring a mine depth image.

[0232] According to one aspect of the embodiments of the present disclosure, a system for restoring a mine depth image is provided, including: a computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions, when executed by a processor, implement the above-mentioned method for restoring a mine depth image.

[0233] According to one aspect of the embodiments of the present disclosure, a system for restoring a mine depth image is provided, including: a computer program product, the computer program product being provided with computer programs / instructions, and the computer programs / instructions, when executed by a processor, implement the above-mentioned method for restoring a mine depth image.

[0234] The electronic device may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components, and is used to execute the above-mentioned method.

[0235] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory including computer program instructions, and the computer program instructions can be executed by a processor of an electronic device to complete the above method.

[0236] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of an instruction, and the module, the segment of a program, or the part of an instruction contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0237] The embodiments of the present disclosure have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the disclosed embodiments.

Claims

1. A method for repairing mine depth images, characterized in that, Including: Obtain a to-be-processed mine color image and a to-be-repaired mine depth image corresponding to the same mine environment object; Perform defogging and enhancement processing on the to-be-processed mine color image to obtain a defogged color balance image; Extract the first edge information of the to-be-repaired mine depth image and the second edge information of the registered defogged color balance image registered with it, respectively; Determine the fused edge information according to the first edge information and the second edge information; Based on the fused edge information and the mine grayscale image corresponding to the registered defogged color balance image, use a joint bilateral filter to repair the to-be-repaired pixels in the to-be-repaired mine depth image.

2. The method for restoring the image of the mine depth according to claim 1, characterized in that The method for performing defogging and enhancement processing on the to-be-processed mine color image to obtain a defogged color balance image includes: Extract the bright channel prior image and the dark channel prior image corresponding to the to-be-processed mine color image, respectively; Determine the transmittance image according to the to-be-processed mine color image, the bright channel prior image and the dark channel prior image; Perform defogging based on the to-be-processed mine color image, its corresponding transmittance image, and the atmospheric light value corresponding to each color channel to obtain a defogged color image; Perform image enhancement on the defogged color image to obtain a defogged color balance image; and / or, The method for extracting the dark channel prior image corresponding to the to-be-processed mine color image includes: within a set first local area, calculate the multiple minimum values corresponding to each color channel in the to-be-processed mine color image respectively; configure the minimum value among the multiple minimum values corresponding to each color channel as the pixel corresponding to the dark channel prior image within the set first local area; repeat the above method, and traverse the to-be-processed mine color image with the set first local area to obtain the dark channel prior image corresponding to the to-be-processed mine color image; and / or, wherein, the set first local area is configured as 3×3; and / or, The method for extracting the bright channel prior image corresponding to the to-be-processed mine color image includes: within a set first local area, calculate the multiple maximum values corresponding to each color channel in the to-be-processed mine color image respectively; configure the maximum value among the multiple maximum values corresponding to each color channel as the pixel corresponding to the bright channel prior image within the set first local area; repeat the above method, and traverse the to-be-processed mine color image with the set first local area to obtain the bright channel prior image corresponding to the to-be-processed mine color image; and / or, wherein, the set first local area is configured as 3×3.

3. The method for repairing the mine depth image according to claim 2, characterized in that, The method for determining the transmittance image according to the to-be-processed mine color image, the bright channel prior image, and the dark channel prior image includes: determining the atmospheric light value corresponding to each color channel according to the bright channel prior image and the dark channel prior image; performing downsampling processing on the bright channel prior image to obtain a mine color downsampled image; performing transmission map estimation according to the mine color downsampled image and the atmospheric light value corresponding to each color channel; calculating the gradient corresponding to the transmission map estimation to obtain a corresponding gradient map; determining a gradient response weight map according to the gradient map; and determining the transmittance image based on the transmission map estimation by using a gradient domain weighted convolution filtering operator; and / or, The method for determining the gradient response weight map according to the gradient map includes: normalizing the gradient map, and determining the gradient response weight map based on the normalized gradient map; and / or, The method for determining the atmospheric light value corresponding to each color channel according to the bright channel prior image and the prior image includes: configuring the first set of bright pixel points for the first set percentage of pixels corresponding to the pixel values of each channel image in the dark channel prior image sorted from large to small; configuring the second set of bright pixel points for the first set percentage of pixels corresponding to the pixel values of each channel image in the bright channel prior image sorted from large to small; determining the atmospheric light value corresponding to each color channel based on the first set of bright pixel points and the second set of bright pixel points; and / or, wherein the first set percentage is configured to be 0.1%; and / or The method for determining the atmospheric light value corresponding to each color channel based on the first set of bright pixel points and the second set of bright pixel points includes: obtaining a first weight coefficient and a second weight coefficient; respectively summing and averaging the bright pixel points corresponding to each color channel in the first set of bright pixel points to obtain the atmospheric light value of each color channel of the dark channel corresponding to the first set of bright pixel points; respectively summing and averaging the bright pixel points corresponding to each color channel in the second set of bright pixel points to obtain the atmospheric light value of each color channel of the bright channel corresponding to the second set of bright pixel points; determining the atmospheric light value corresponding to each color channel based on the atmospheric light value of each color channel of the dark channel and its corresponding first weight coefficient, the atmospheric light value of each color channel of the bright channel and its corresponding second weight coefficient; and / or, the sum of the first weight coefficient and the second weight coefficient is configured to be 1; and / or, the first weight coefficient and the second weight coefficient are respectively configured to be 0.

5.

4. The method for repairing the mine depth image according to any one of claims 2 or 3, characterized in that The method for performing image enhancement on the defogged color image to obtain a defogged color balanced image includes: Converting the defogged color image in the RGB space to a visible photon subset image corresponding to the HSV space; Determining the reflection component corresponding to the luminance information V(x) component in the visible photon subset image based on the visible photon subset image; Performing non-linear stretching on the reflection component to obtain a non-linearly stretched reflection component; Update the V(x) component of the luminance information based on the corresponding logarithmic illuminance in the V(x) component of the luminance information and the non-linearly stretched reflection component to obtain an updated V(x) component of the luminance information; Fuse the updated V(x) component of the luminance information with the H(x) and S(x) components in the visible photon subset image to obtain an HSV-fused defogged color balance image; Convert the HSV-fused defogged color balance image corresponding to the HSV space to the RGB space to obtain a defogged color balance image; and / or, The method of non-linearly stretching the reflection component to obtain a non-linearly stretched reflection component includes: normalizing the reflection component to obtain a normalized reflection component; non-linearly stretching the normalized reflection component to obtain a non-linearly stretched reflection component.

5. The method for restoring the mine depth image according to any one of claims 1-4, characterized in that, The method of separately extracting the first edge information and the second edge information of the to-be-restored mine depth image and the registered defogged color balance image corresponding thereto includes: Using an edge extraction algorithm, respectively perform edge extraction on the to-be-restored mine depth image and the registered defogged color balance image to obtain the first edge information corresponding to the to-be-restored mine depth image and the second edge information corresponding to the registered defogged color balance image; and / or, The method of determining the fused edge information according to the first edge information and the second edge information includes: performing pixel-by-pixel comparison on the first edge binary image and the second edge binary image corresponding to the first edge information and the second edge information to obtain the fused edge information; and / or, The method of performing pixel-by-pixel comparison on the first edge binary image and the second edge binary image corresponding to the first edge information and the second edge information to obtain the fused edge information includes: performing a logical OR operation on the first edge binary image and the second edge binary image corresponding to the first edge information and the second edge information pixel by pixel to obtain the fused edge information.

6. The method for repairing the mine depth image according to any one of claims 1-5, characterized in that, The method of using a joint bilateral filter to repair the pixels to be repaired in the to-be-restored mine depth image based on the fused edge information and the mine gray image corresponding to the registered defogged color balance image includes: Obtain a judgment function corresponding to the fused edge information; Based on the judgment function and the fused edge information, judge whether the pixels adjacent to the pixels to be repaired in the to-be-restored mine depth image participate in the joint bilateral filtering; If the pixels adjacent to the pixels to be repaired participate in the joint bilateral filtering, then use the joint bilateral filter to repair the pixels to be repaired in the to-be-restored mine depth image based on the pixels adjacent to the pixels to be repaired and the mine gray image corresponding to the registered defogged color balance image; and / or, The method of judging whether the pixels adjacent to the pixels to be repaired in the to-be-restored mine depth image participate in the joint bilateral filtering based on the judgment function and the fused edge information includes: When the pixel to be repaired is a missing pixel, or the pixels adjacent to the pixel to be repaired are not the edge pixels corresponding to the fusion edge information and the pixel to be repaired is the edge pixel corresponding to the fusion edge information, the pixels adjacent to the pixel to be repaired participate in the joint bilateral filtering; Otherwise, the pixels adjacent to the pixel to be repaired do not participate in the joint bilateral filtering.

7. The method for repairing the mine depth image according to claim 6, wherein, The method for repairing the pixel to be repaired in the to-be-repaired mine depth image by using a joint bilateral filter based on the pixels adjacent to the pixel to be repaired and the mine gray-scale image corresponding to the registered haze-removed color-balanced image includes: Performing weighted averaging on the pixels corresponding to each color channel in the registered haze-removed color-balanced image to obtain the mine gray-scale image corresponding to the registered haze-removed color-balanced image; Based on the mine gray-scale image, respectively calculating the pixel difference weight and the Gaussian distribution distance weight value corresponding to the mine gray-scale image; Based on the pixel difference weight and the Gaussian distribution distance weight value corresponding to the mine gray-scale image and the pixels adjacent to the pixel to be repaired, using a joint bilateral filter to repair the pixel to be repaired in the to-be-repaired mine depth image.

8. A restoration system for mine depth images, characterized in that, Including: An acquisition unit, configured to acquire a to-be-processed mine color image and a to-be-repaired mine depth image corresponding to the same mine environment object; A haze-removing and enhancement processing unit, configured to perform haze-removing and enhancement processing on the to-be-processed mine color image to obtain a haze-removed color-balanced image; An extraction unit, configured to respectively extract the first edge information and the second edge information of the registered haze-removed color-balanced image corresponding to the to-be-repaired mine depth image and the registered haze-removed color-balanced image registered therewith; A determination unit, configured to determine the fusion edge information according to the first edge information and the second edge information; A repair unit, configured to repair the pixel to be repaired in the to-be-repaired mine depth image by using a joint bilateral filter based on the fusion edge information and the mine gray-scale image corresponding to the registered haze-removed color-balanced image.

9. The restoration system for the mine depth image according to claim 8, characterized in that, The haze-removing and enhancement processing unit includes: A prior image extraction unit, configured to respectively extract a bright-channel prior image and a dark-channel prior image corresponding to the to-be-processed mine color image; a transmittance image unit, configured to determine a transmittance image according to the to-be-processed mine color image, the bright-channel prior image, and the dark-channel prior image; a haze-removed color image processing unit, configured to perform haze-removing on the to-be-processed mine color image based on the to-be-processed mine color image, its corresponding transmittance image, and the atmospheric light value corresponding to each color channel to obtain a haze-removed color image; an image enhancement processing unit, configured to perform image enhancement on the haze-removed color image to obtain a haze-removed color-balanced image; and / or, The transmittance image unit includes: an atmospheric light value determination unit for determining the atmospheric light value corresponding to each color channel according to the bright channel prior image and the prior image; wherein, the atmospheric light value determination unit includes: a first determination unit, a second determination unit and a third determination unit; the first determination unit is configured to multiply the dark channel atmospheric light value corresponding to each color channel by the corresponding first weight coefficient to obtain the first weighted dark channel atmospheric light value corresponding to each color channel; the second determination unit is configured to multiply the bright channel atmospheric light value corresponding to each color channel by the corresponding second weight coefficient to obtain the second weighted bright channel atmospheric light value corresponding to each color channel; the third determination unit is configured to sum the first weighted dark channel atmospheric light value corresponding to each color channel and the second weighted bright channel atmospheric light value corresponding to each color channel respectively to determine the atmospheric light value corresponding to each color channel; and / or, The image enhancement processing unit includes: a reflection component determination unit; the reflection component determination unit is configured to determine the reflection component corresponding to the luminance information V(x) component in the visible photon subset image based on the dehazed color image corresponding visible photon subset image; wherein, the reflection component determination unit includes: a luminance information V(x) component extraction unit for extracting the luminance information V(x) component corresponding to the visible photon subset image; an illuminance estimation unit for estimating the corresponding illuminance in the luminance information V(x) component by using the guided filtering technique; a logarithmic processing unit for respectively converting the luminance information V(x) component and its corresponding illuminance into the logarithmic domain to obtain the logarithmic luminance information V(x) component and the logarithmic illuminance; a reflection component calculation unit for subtracting the corresponding logarithmic illuminance from the logarithmic luminance information V(x) component to determine the reflection component corresponding to the luminance information V(x) component; and / or, The dehazed color image processing unit includes: a set protection factor acquisition unit for acquiring a set protection factor; a pixel difference calculation unit for respectively calculating the difference between each color channel pixel in the to-be-processed mine color image and its corresponding atmospheric light value to obtain the pixel difference corresponding to each color channel; a maximum value determination unit for respectively determining the maximum value corresponding to each color channel between each color channel pixel in the transmittance image and the set protection factor; a ratio calculation unit for respectively calculating the ratio corresponding to each color channel between the pixel difference of each color channel and the maximum value of the corresponding color channel; an addition unit for adding the ratio corresponding to each color channel to the atmospheric light value of the corresponding color channel to obtain the dehazed color image.

10. A repair system for mine depth images, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor; wherein, the processor is configured to call the instructions stored in the memory to execute the method for repairing the mine depth image according to any one of claims 1 to 7; or, Comprising: an electronic device; a processor configured on the electronic device; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the method for repairing a mine depth image according to any one of claims 1 to 7; or, Comprising: a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions, when executed by a processor, implementing the method for repairing a mine depth image according to any one of claims 1 to 7; or, Comprising: a computer program product, the computer program product being provided with a computer program / instructions, the computer program / instructions, when executed by a processor, implementing the method for repairing a mine depth image according to any one of claims 1 to 7.

Images