Frequency domain illumination prior drive unsupervised low-light video enhancement method and system for shield slag hole monitoring, and storage medium

Abstract

The invention discloses a frequency domain illumination prior drive unsupervised low-light video enhancement method and system for shield slag hole monitoring and a storage medium. Composite noise suppression and basic restoration of a low-light image are realized through a noise suppression module; performing two-dimensional Haar wavelet decomposition on the basic restored image through a frequency domain illumination priori estimation module, and combining channel-space attention weighted high and low frequency sub-bands and fusing to generate frequency domain illumination priori features; accurate reconstruction of the reflection component is completed by means of a trans-attention converter unit of the reflection component reconstruction module; generating a global exponential enhancement factor by using an illumination enhancement coefficient module to obtain an enhanced image; network unsupervised end-to-end training is realized through gamma transformation branches and multi-dimensional loss constraints, the problems of overall darkness, detail submerging and serious noise of a shield slag hole monitoring picture can be effectively relieved on the premise of not increasing the onsite continuous illumination burden, and an enhanced video with balanced brightness, clear details and controlled overexposure is obtained.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a frequency domain illumination prior driven unsupervised low-light video enhancement method for shield slag outlet monitoring, a system and a storage medium. BACKGROUND

[0002] In a shield construction site, the slag outlet is usually located in a narrow and closed shield tail space, and long-term faces harsh working conditions such as low illumination, dust and water mist, mud splashing and mechanical vibration, which leads to the problems of insufficient brightness, detail submersion, serious noise and low contrast in the images collected by the monitoring camera, which not only affects the visual monitoring effect of construction, but also weakens the stability and reliability of subsequent slag quantity recognition, block stone jam detection, mechanical anomaly diagnosis and other high-level visual algorithms.

[0003] The existing low-light enhancement technology has obvious defects: 1. Traditional methods such as histogram equalization, Gamma correction and basic Retinex decomposition can only improve brightness in simple scenes, and are prone to problems such as noise amplification, uneven illumination, overexposure of bright parts and loss of dark details in complex shield working conditions; 2. Methods based on deep learning such as convolutional neural network (CNN) and general Transformer enhancement model perform well in general scenes, but generally rely on paired normal light / low light training samples, and strong illumination cannot be maintained for a long time in the shield site, and there is no condition to obtain a large amount of paired data; 3. Most of the above methods only model the illumination distribution in the spatial domain, ignoring the independent change rule of the frequency domain structure (low-frequency brightness and high-frequency texture), lacking a scene prior utilization mechanism for fixed angle and long-term sequence, and leading to unstable enhancement effect in engineering application; Therefore, we propose a frequency domain illumination prior driven unsupervised low-light video enhancement method for shield slag outlet monitoring, a system and a storage medium. SUMMARY

[0004] The purpose of the present application is to provide a frequency domain illumination prior driven unsupervised low-light video enhancement method for shield slag outlet monitoring, a system and a storage medium to solve the problems raised in the background art.

[0005] To achieve the above purpose, the present application provides the following technical scheme: a frequency domain illumination prior driven unsupervised low-light video enhancement method for shield slag outlet monitoring, comprising the following steps: Step 1, data acquisition and preprocessing: a fixed-view monitoring camera is arranged at the position of the slag outlet in the shield construction site to collect long-time low-light video sequences; during the short construction gap, the lighting is turned on to supplement a small amount of normally exposed images as prior samples; the collected video is frame-split to form a training dataset containing low-light shield images and normally exposed reference images; Step 2, noise suppression and basic restoration: given a low-light shield image, input it into the noise suppression module to suppress the complex noise introduced by dust, water mist and equipment vibration and retain the structural details to obtain a basic restoration image; Step 3, frequency domain illumination prior feature construction: input the basic restoration image into the frequency domain illumination prior estimation module, perform two-dimensional Haar wavelet decomposition on the basic restoration image to obtain a low-frequency subband and multiple high-frequency subbands, perform weighting processing on the low-frequency and high-frequency components through channel attention and spatial attention respectively, and after upsampling and splicing fusion, construct the frequency domain illumination prior feature; Step 4, illumination map estimation: input the basic restoration image into the illumination estimation module, obtain an intermediate illumination response map through the transformer block and convolution aggregation, and then perform residual connection and fusion with the local channel mean value map to obtain a single-channel illumination map; Step 5, reflectance component reconstruction: extract features from the basic restoration image, and splice the features obtained in step 3 in the channel dimension to form joint features, input the joint features into the reflectance component reconstruction module, calculate the attention weight in the cross-attention transformer unit with the frequency domain illumination prior feature as the query and the basic restoration image features as the key and value, and obtain the reflectance component map through residual reconstruction; Step 6, adaptive enhancement and image synthesis: input the single-channel illumination map into the illumination enhancement coefficient module, sequentially perform convolution, linear rectification activation and adaptive average pooling to obtain a 1x1 global description, then perform 1x1 convolution and linear rectification activation to obtain a non-negative global exponential enhancement factor, and according to the Retinex synthesis relationship, multiply the enhancement factor power of the illumination map with the reflectance component map element by element to obtain the enhanced shield slag outlet image; Step 7, unsupervised training and reference exposure guidance: train the network composed of steps 2-6 in an end-to-end manner with low-light frames as the main and a small amount of normally exposed frames as the auxiliary, construct a gamma transformation branch to perform nonlinear brightness transformation on the low-light images, and optimize the model through reflection invariance constraint, Retinex decomposition loss and illumination map constraint.

[0006] Preferably, in step 3, the specific steps of frequency domain illumination prior feature construction are as follows: 3.1, perform two-dimensional Haar wavelet decomposition on the basic restoration image to obtain a low-frequency subband, a vertical direction high-frequency subband, a horizontal direction high-frequency subband and a diagonal direction high-frequency subband; 3.2, input the low-frequency sub-band into the low-frequency enhancement branch, and obtain the low-frequency illumination prior feature by global average pooling, channel attention, and spatial attention weighting; 3.3, after splicing the multiple high-frequency sub-bands in the channel dimension, obtain the enhanced high-frequency feature by global average pooling, channel attention, and spatial attention weighting; 3.4, respectively perform transposed convolution upsampling on the low-frequency illumination prior feature and the enhanced high-frequency feature, splice them, and then obtain the frequency domain illumination prior feature through 1x1 convolution.

[0007] Preferably, in step 3.1, the calculation formula of the two-dimensional Haar wavelet decomposition is shown in formula (1): (1); Wherein, X represents the basic restoration image, DWT represents the two-dimensional Haar wavelet decomposition operation, LL represents the low-frequency sub-band, LH, HL, and HH represent the vertical, horizontal, and diagonal direction high-frequency sub-bands respectively; In step 3.2, the calculation formulas of the low-frequency sub-band channel description, the low-frequency channel attention weight, the low-frequency channel enhanced feature, the low-frequency spatial attention intermediate feature, the low-frequency spatial attention weight, and the low-frequency illumination prior feature are shown in formulas (2), (3), (4), (5), (6), and (7) respectively: (2); (3); (4); (5); (6); (7); Wherein, GAP represents a global average pooling operator; represents a channel description vector of the low-frequency sub-band; , represents a low-frequency channel weight matrix; represents a linear rectification activation operator; represents a Sigmoid activation operator; represents a low-frequency channel attention weight; represents an element-wise multiplication operator; represents a low-frequency channel enhanced feature; AvgPool represents an average pooling operator; MaxPool represents a maximum pooling operator; represents a low-frequency spatial attention intermediate feature; represents a 7x7 convolution operator; represents a low-frequency spatial attention weight map; This represents the a priori features of low-frequency illumination; In step 3.3, the calculation formulas for high-frequency subband splicing, high-frequency subband channel description, high-frequency channel attention weight, high-frequency channel enhancement feature, high-frequency spatial attention intermediate feature, high-frequency spatial attention weight, and enhanced high-frequency feature are shown in formulas (8), (9), (10), (11), (12), (13), and (14), respectively: (8); (9); (10); (11); (12); (13); (14); Wherein, Concat represents the channel splicing operator; H represents the high-frequency splicing feature; The channel description vector representing the high-frequency subband; , Represents the high-frequency channel weight matrix; Indicates the attention weights for high-frequency channels; Indicates high-frequency channel enhancement features; Represents intermediate features of high-frequency spatial attention; Represents a high-frequency spatial attention weight map; Indicates enhanced high-frequency characteristics; In step 3.4, the calculation formulas for high and low frequency feature upsampling, high and low frequency feature splicing, and frequency domain illumination prior feature generation are shown in formulas (15), (16), and (17), respectively: (15); (16); (17); Where Up represents the upsampling operation; This indicates the low-frequency characteristics of upsampling; Indicates high-frequency characteristics of upsampling; This represents a 1×1 convolution operator; represents the high- and low-frequency fusion characteristics; P represents the frequency domain illumination prior characteristics.

[0008] Preferably, in step 5, the specific steps for reconstructing the reflection components are as follows: 5.1 Extract features from the basic restored image and concatenate them with the frequency domain illumination prior features obtained in step 3 in the channel dimension to form joint features; 5.2 Input the joint features into the reflection component reconstruction module. In the cross-attention transformer unit, use the frequency domain illumination prior features as the query and the basic restored map features as the key and value to calculate the attention weights. Suppress areas that are greatly affected by illumination changes and enhance areas with significant material and texture to obtain the cross-attention feature map. 5.3. Perform residual connection and convolution reconstruction on the cross-attention output and input features to obtain the reflection component map.

[0009] Preferably, in step 5.1, the calculation formula for the joint features is as shown in formula (18): (18); The features extracted from the base reconstruction image are represented by F; joint features are represented by F. In step 5.2, the calculation formulas for cross-attention weights, cross-attention features, and attention feature shape transformation are shown in formulas (19), (20), and (21), respectively: (19); (20); (twenty one); Where A represents the attention weight matrix; Softmax represents the normalized exponential operator; Q represents the query vector; K represents the key vector; V represents the value vector; d represents the channel dimension of a single attention head; and T represents the matrix transpose operation. Indicates shape transformation operation; Represents cross-attention features; This represents the attentional features after shape transformation; The calculation formulas for the query vector, key vector, and value vector are shown in formulas (22), (23), and (24), respectively: (twenty two); (twenty three); (twenty four); in, , and Learnable weight matrices for generating query vectors, key vectors, and value vectors across attention units, respectively; This means flattening and rearranging the features into a token sequence; In step 5.3, the calculation formulas for residual connectivity enhancement and reflection component map generation are shown in formulas (25) and (26): (25); (26); in, Indicates residual enhancement characteristics; R represents the 3×3 convolution operator; R represents the reflection component map.

[0010] Preferably, in step 7, the specific steps of unsupervised training are as follows: 7.1 Constructing the Gamma Transform Branch: The gamma values ​​of the low-light image are randomly sampled in the interval [0.7, 1.3] and transformed. The formula for calculating the gamma value transformation is shown in formula (27): (27); in, Represents the original low-light image; This represents the gamma value of a random sample. Represents the image after gamma transformation; 7.2 Construct a loss function constraint model, including: Reflection invariance loss function: (28); Retinex reconstruction loss function: (29); Retinex consistency loss function: (30); Total variational regularization loss function: (31); Illumination diagram constraint loss: (32) Total loss function: (33); Where N represents the total number of pixels in the image; R represents the reflectance component map; This represents the reflection component map of the image after gamma transformation; Represents the squared L2 norm; L is the single-channel illumination diagram; This indicates the removal of zero small constants; , M represents the adjacent pixels in a single-channel illumination map; M represents the maximum value of the channel in the base reconstruction map. Indicates the weight of the total variation loss; , , and The weight parameters represent the individual losses in the total loss function.

[0011] Preferably, in step 6, adaptive average pooling is used to compress the features of the single-channel illumination map into a 1×1 global description, and 1×1 convolution with linear rectified activation is used to ensure that the global exponential enhancement factor is non-negative. Retinex assimilation relations satisfy ,in This represents the α-th power of the single-channel illumination map, where α represents the global exponential enhancement factor.

[0012] Preferably, the weight parameter in step 7.2 is set to the following values: , , , , ; During unsupervised training, the batch size is 8, the initial learning rate is 1e-4, cosine annealing decay is used, and the number of training iterations is 100 rounds.

[0013] A frequency-domain illumination prior-driven unsupervised low-light video enhancement system for monitoring the muck discharge opening of a tunnel boring machine (TBM) is provided. This system implements a frequency-domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the muck discharge opening of a TBM, comprising: Data acquisition module: used to acquire long-term low-light video sequences of the shield tunnel's slag discharge port through a fixed-view monitoring camera, as well as a small number of normally exposed images from the same viewpoint during construction breaks; Preprocessing module: used to split the acquired low-light video into frames and construct a training dataset containing low-light shield tunnel images and normal exposure reference images; Noise Suppression Module: Employs a lightweight convolutional neural network, combined with adaptive median filtering and residual learning, to receive low-light shield tunneling images, suppress complex noise introduced by dust, water mist, and equipment vibration while preserving structural details, and output a basic reconstruction image; Frequency domain illumination prior estimation module: It is used to perform two-dimensional Haar wavelet decomposition on the basic restored image to obtain low-frequency sub-bands and multiple high-frequency sub-bands. The low-frequency and high-frequency components are weighted by channel attention and spatial attention respectively. After upsampling and splicing fusion, the frequency domain illumination prior features are output. The illumination estimation module receives the basic restoration map, performs transform and convolution aggregation to obtain the intermediate illumination response map, and then connects and fuses it with the residual of the local channel mean map to output a single-channel illumination map. The reflection component reconstruction module is used to extract features from the base restored map and concatenate them with the frequency domain illumination prior features to form joint features. Attention weights are calculated through the attention transformer unit, and the reflection component map is output after residual reconstruction. Illumination enhancement factor module: This module receives the illumination map, processes it through convolution, linear rectified activation, adaptive average pooling, and 1×1 convolution, and outputs a non-negative global exponential enhancement factor. Reconstruction and Synthesis Module: This module is used to reconstruct the enhanced shield tunneling slag outlet image by multiplying the global exponential enhancement factor of the illumination image by the reflection component image element by element according to the Retinex synthesis relationship. Unsupervised training module: This module is used to train the network constructed by the above modules end-to-end using low-light frames as the main component and a small number of normal-exposure frames as the auxiliary component. The training is conducted through gamma transform branch, reflection invariance constraint, Retinex decomposition loss and illumination map constraint.

[0014] A computer-readable storage medium storing a computer program that, when executed by a processor, implements all steps of a frequency-domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention employs a noise suppression module, a frequency domain illumination prior estimation module, an illumination estimation module, a reflection component reconstruction module, and an illumination enhancement coefficient module to jointly construct an FRTnet network. This network collaboratively models and progressively restores the illumination distribution, material structure, and noise interference in low-light images of the shield tunneling slag outlet. During the enhancement process, this invention not only utilizes two-dimensional Haar wavelet decomposition and attention fusion to construct accurate frequency domain illumination priors, but also extracts multi-scale contextual information through cross-attention mechanism structures. This achieves effective separation of illumination and reflection components, significantly improving the accuracy of illumination estimation and the fidelity of reflection texture. 2. This invention incorporates frequency domain brightness priors into the reflection reconstruction process through a cross-attention mechanism, avoiding the noise amplification and structural misjudgment problems commonly found in traditional low-light enhancement methods. This results in more stable performance of the enhanced image in terms of brightness balance, detail restoration, and texture consistency. Furthermore, this invention achieves adaptive nonlinear adjustment of the illumination map through an exponential illumination enhancement strategy, effectively controlling the brightening of dark areas and the suppression of bright areas while maintaining structural information without distortion. This improves the enhancement robustness under complex conditions such as dust, water mist, and local reflections. 3. This invention constructs an unsupervised training mechanism that does not require corresponding normal light samples, through... Transformation invariance constraints, Retinex reconstruction constraints, illumination smoothness constraints, and multiple self-supervised losses such as exposure-structure-color guide the network to maintain a consistent reflection structure and natural enhancement effect under different lighting conditions, different brightness distributions, and different noise levels. Normal exposure frames collected during a small number of construction breaks are only used as reference priors for global brightness and color distribution, effectively reducing the difficulty of engineering deployment. 4. By using the reconstruction and synthesis module and the three-stage progressive enhancement structure to output the restored image, this invention can obtain low-light enhancement results with natural brightness improvement, clear texture details, and significant noise suppression, providing more reliable and stable input data for high-level vision tasks such as slag quantity estimation at the shield tunneling outlet, rock detection, and equipment anomaly identification. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of a low-light image frame obtained by video decomposition in an embodiment of the present invention; Figure 3 This is a schematic diagram of a normally exposed image frame obtained from video decomposition in an embodiment of the present invention; Figure 4 This is a schematic diagram of one frame of the target exposure image obtained by the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1 The present invention provides a frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine, comprising the following steps: Step 1: Data Acquisition and Preprocessing: Fixed-view monitoring cameras are set up at the slag outlet of the shield tunneling site to collect long-term low-light video sequences; lighting is turned on during short construction breaks, and a small number of normal-exposure images are collected from the same viewpoint as prior samples for the project; the collected videos are split into frames to form a training dataset containing low-light shield tunneling images and normal-exposure reference images. Step 2, Noise Suppression and Foundation Restoration: Given a low-light shield tunneling image, input it into the noise suppression module to suppress complex noises introduced by dust, water mist and equipment vibration while preserving structural details, and obtain the foundation restoration image; Step 3: Construction of frequency domain illumination prior features: Input the basic restoration image into the frequency domain illumination prior estimation module, perform two-dimensional Haar wavelet decomposition on the basic restoration image to obtain low-frequency sub-bands and multiple high-frequency sub-bands, and perform weighted processing on low-frequency and high-frequency components through channel attention and spatial attention respectively. After upsampling and splicing fusion, the frequency domain illumination prior features are constructed. In step 3, the specific steps for constructing the frequency domain illumination prior features are as follows: 3.1 Perform two-dimensional Haar wavelet decomposition on the basic restored image to obtain low-frequency sub-band, vertical high-frequency sub-band, horizontal high-frequency sub-band, and diagonal high-frequency sub-band; 3.2 Input the low-frequency subband into the low-frequency enhancement branch, and obtain the low-frequency illumination prior features through global average pooling, channel attention, and spatial attention weighting; 3.3 After splicing multiple high-frequency subbands along the channel dimension, the enhanced high-frequency features are obtained through global average pooling, channel attention, and spatial attention weighting. 3.4. The low-frequency illumination prior features and the enhanced high-frequency features are transposed and upsampled by convolution, and then concatenated and convolved by 1×1 to obtain the frequency domain illumination prior features. In step 3.1, the calculation formula for the two-dimensional Haar wavelet decomposition is shown in formula (1): (1); Where X represents the basic reconstruction image, DWT represents the two-dimensional Haar wavelet decomposition operation, LL represents the low-frequency subband, and LH, HL, and HH represent the high-frequency subbands in the vertical, horizontal, and diagonal directions, respectively. In step 3.2, the calculation formulas for low-frequency subband channel description, low-frequency channel attention weight, low-frequency channel enhancement feature, low-frequency spatial attention intermediate feature, low-frequency spatial attention weight, and low-frequency illumination prior feature are shown in formulas (2), (3), (4), (5), (6), and (7), respectively: (2); (3); (4); (5); (6); (7); Wherein, GAP represents the global average pooling operator; The channel description vector representing the low-frequency subband; , This represents the low-frequency channel weight matrix; This represents the linear rectification activation operator; Indicates the Sigmoid activation operator; This indicates the attention weight for low-frequency channels; This represents the element-wise multiplication operator; This indicates the low-frequency channel enhancement feature; AvgPool represents the average pooling operator; MaxPool represents the max pooling operator. This represents intermediate features of low-frequency spatial attention. This represents the 7×7 convolution operator; Represents a low-frequency spatial attention weight map; This represents the a priori features of low-frequency illumination; In step 3.3, the calculation formulas for high-frequency subband splicing, high-frequency subband channel description, high-frequency channel attention weight, high-frequency channel enhancement feature, high-frequency spatial attention intermediate feature, high-frequency spatial attention weight, and enhanced high-frequency feature are shown in formulas (8), (9), (10), (11), (12), (13), and (14), respectively: (8); (9); (10); (11); (12); (13); (14); Wherein, Concat represents the channel splicing operator; H represents the high-frequency splicing feature; The channel description vector representing the high-frequency subband; , Represents the high-frequency channel weight matrix; Indicates the attention weights for high-frequency channels; Indicates high-frequency channel enhancement features; Represents intermediate features of high-frequency spatial attention; Represents a high-frequency spatial attention weight map; Indicates enhanced high-frequency characteristics; In step 3.4, the calculation formulas for high and low frequency feature upsampling, high and low frequency feature splicing, and frequency domain illumination prior feature generation are shown in formulas (15), (16), and (17), respectively: (15); (16); (17); Where Up represents the upsampling operation; This indicates the low-frequency characteristics of upsampling; Indicates high-frequency characteristics of upsampling; This represents a 1×1 convolution operator; P represents the high- and low-frequency fusion characteristics; P represents the frequency domain illumination prior characteristics. Step 4, Illumination map estimation: Input the basic restoration map into the illumination estimation module, and obtain the intermediate illumination response map through the transformer block and convolution aggregation. Then, perform residual connection and fusion with the local channel mean map to obtain the single channel illumination map. Step 5, Reflection Component Reconstruction: Extract features from the base restoration image and concatenate them with the frequency domain illumination prior features obtained in Step 3 along the channel dimension to form joint features. Input the joint features into the reflection component reconstruction module. In the cross-attention transformer unit, use the frequency domain illumination prior features as the query and the base restoration image features as the key and value to calculate the attention weights. The reflection component map is obtained through residual reconstruction. In step 5, the specific steps for reconstructing the reflection components are as follows: 5.1 Extract features from the basic restored image and concatenate them with the frequency domain illumination prior features obtained in step 3 in the channel dimension to form joint features; 5.2 Input the joint features into the reflection component reconstruction module. In the cross-attention transformer unit, use the frequency domain illumination prior features as the query and the basic restored map features as the key and value to calculate the attention weights. Suppress areas that are greatly affected by illumination changes and enhance areas with significant material and texture to obtain the cross-attention feature map. 5.3. Perform residual connections and convolutional reconstructions on the cross-attention output and input features to obtain the reflection component map; In step 5.1, the formula for calculating the joint features is shown in formula (18): (18); The features extracted from the base reconstruction image are represented by F; joint features are represented by F. In step 5.2, the calculation formulas for cross-attention weights, cross-attention features, and attention feature shape transformation are shown in formulas (19), (20), and (21), respectively: (19); (20); (twenty one); Where A represents the attention weight matrix; Softmax represents the normalized exponential operator; Q represents the query vector; K represents the key vector; V represents the value vector; d represents the channel dimension of a single attention head; and T represents the matrix transpose operation. Indicates shape transformation operation; Represents cross-attention features; This represents the attentional features after shape transformation; The calculation formulas for the query vector, key vector, and value vector are shown in formulas (22), (23), and (24), respectively: (twenty two); (twenty three); (twenty four); in, , and Learnable weight matrices for generating query vectors, key vectors, and value vectors across attention units, respectively; This means flattening and rearranging the features into a token sequence; In step 5.3, the calculation formulas for residual connectivity enhancement and reflection component map generation are shown in formulas (25) and (26): (25); (26); in, Indicates residual enhancement characteristics; R represents the 3×3 convolution operator; R represents the reflection component map; Step 6, Adaptive Enhancement and Image Synthesis: Input the single-channel illumination map into the illumination enhancement coefficient module, and obtain a 1×1 global description by sequentially passing it through convolution, linear rectified activation and adaptive average pooling. Then, obtain a non-negative global exponential enhancement factor by passing it through 1×1 convolution and linear rectified activation. According to the Retinex synthesis relationship, multiply the enhancement factor of the illumination map by the element-wise of the reflection component map to obtain the enhanced shield slag outlet image. In step 6, adaptive average pooling is used to compress the features of the single-channel illumination map into a 1×1 global description, and 1×1 convolution with linear rectified activation is used to ensure that the global exponential enhancement factor is non-negative. Retinex assimilation relations satisfy ,in This represents the α power of the single-channel illumination map, where α represents the global exponential enhancement factor; Step 7, Unsupervised Training and Reference Exposure Guidance: The network constructed in Steps 2-6 is trained end-to-end using low-light frames as the main component and a small number of normal exposure frames as supplementary components. A gamma transform branch is constructed to perform nonlinear brightness transformation on the low-light image. The model is optimized through reflection invariance constraints, Retinex decomposition loss and illumination map constraints. In step 7, the specific steps of unsupervised training are as follows: 7.1 Constructing the Gamma Transform Branch: The gamma values ​​of the low-light image are randomly sampled in the interval [0.7, 1.3] and transformed. The formula for calculating the gamma value transformation is shown in formula (27): (27); in, Represents the original low-light image; This represents the gamma value of a random sample. Represents the image after gamma transformation; 7.2 Construct a loss function constraint model, including: Reflection invariance loss function: (28); Retinex reconstruction loss function: (29); Retinex consistency loss function: (30); Total variational regularization loss function: (31); Illumination diagram constraint loss: (32) Total loss function: (33); Where N represents the total number of pixels in the image; R represents the reflectance component map; This represents the reflection component map of the image after gamma transformation; Represents the squared L2 norm; L is the single-channel illumination diagram; This indicates the removal of zero small constants; , M represents the adjacent pixels in a single-channel illumination map; M represents the maximum value of the channel in the base reconstruction map. Indicates the weight of the total variation loss; , , and The weight parameters represent the individual losses in the total loss function; The values ​​for the weight parameters in step 7.2 are: , , , , ; During unsupervised training, the batch size is 8, the initial learning rate is 1e-4, cosine annealing decay is used, and the number of training iterations is 100 rounds.

[0019] The present invention provides a frequency domain illumination prior-driven unsupervised low-light video enhancement system for monitoring the muck outlet of a tunnel boring machine (TBM). This system is used to implement a frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the muck outlet of a TBM, comprising: Data acquisition module: used to acquire long-term low-light video sequences of the shield tunnel's slag discharge port through a fixed-view monitoring camera, as well as a small number of normally exposed images from the same viewpoint during construction breaks; Preprocessing module: used to split the acquired low-light video into frames and construct a training dataset containing low-light shield tunnel images and normal exposure reference images; Noise Suppression Module: Employs a lightweight convolutional neural network, combined with adaptive median filtering and residual learning, to receive low-light shield tunneling images, suppress complex noise introduced by dust, water mist, and equipment vibration while preserving structural details, and output a basic reconstruction image; Frequency domain illumination prior estimation module: It is used to perform two-dimensional Haar wavelet decomposition on the basic restored image to obtain low-frequency sub-bands and multiple high-frequency sub-bands. The low-frequency and high-frequency components are weighted by channel attention and spatial attention respectively. After upsampling and splicing fusion, the frequency domain illumination prior features are output. The illumination estimation module receives the basic restoration map, performs transform and convolution aggregation to obtain the intermediate illumination response map, and then connects and fuses it with the residual of the local channel mean map to output a single-channel illumination map. The reflection component reconstruction module is used to extract features from the base restored map and concatenate them with the frequency domain illumination prior features to form joint features. Attention weights are calculated through the attention transformer unit, and the reflection component map is output after residual reconstruction. Illumination enhancement factor module: This module receives the illumination map, processes it through convolution, linear rectified activation, adaptive average pooling, and 1×1 convolution, and outputs a non-negative global exponential enhancement factor. Reconstruction and Synthesis Module: This module is used to reconstruct the enhanced shield tunneling slag outlet image by multiplying the global exponential enhancement factor of the illumination image by the reflection component image element by element according to the Retinex synthesis relationship. Unsupervised training module: This module is used to train the network constructed by the above modules end-to-end using low-light frames as the main component and a small number of normal-exposure frames as the auxiliary component. The training is conducted through gamma transform branch, reflection invariance constraint, Retinex decomposition loss and illumination map constraint.

[0020] The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements all steps of a frequency-domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine.

[0021] This embodiment is a specific application of the present invention, such as... Figure 2 As shown, it is a low-light image in the monitoring scene of the shield tunnel's slag outlet. Under complex conditions such as insufficient brightness, obscured details, and interference from dust and noise, it is difficult to meet the actual construction monitoring needs. After processing the image using the method of this invention, an enhanced target exposure image is obtained, namely... Figure 4 As shown, it is clearly visible from the visual effect. Figure 4 Significant improvements were achieved in global brightness, regional contrast, and detail structure: dark areas were effectively restored, and conveyor belt edges, rock textures, and slag discharge cavity structures were clearly presented; at the same time, image noise was suppressed, and no overexposure or clipping occurred in bright areas. Compared with reference normal light images acquired under brief illumination conditions ( Figure 3Compared to the image shown, its exposure effect is almost identical to the actual component texture. This invention can obtain high-quality target exposure images without increasing the burden of continuous illumination, providing reliable support for subsequent high-level vision tasks such as slag quantity identification, stone detection, and abnormal working condition analysis.

[0022] This invention provides an unsupervised low-light enhancement method driven by frequency domain illumination prior for monitoring tunnel boring machine (TBM) slag outlets. By constructing an FRTnet network structure, it organically combines a noise suppression module, a frequency domain illumination prior estimation module, an illumination estimation module, a reflection component reconstruction module, and an illumination enhancement coefficient module to achieve fine modeling of the illumination structure and reflection texture. Based on the frequency domain illumination prior constructed using 2Vell wavelet decomposition and attention fusion, this invention can accurately separate brightness information from material structure. Combined with the global modeling capability across attention mechanisms, it effectively recovers scene details under low-light conditions. Furthermore, through an exponential illumination enhancement strategy and Retinex synthesis relationship, it achieves adaptive adjustment and natural enhancement of brightness, avoiding the noise amplification and over-enhancement problems common in traditional methods. Furthermore, the enhanced image output by the reconstruction and synthesis module of this invention exhibits outstanding performance in terms of brightness balance, detail fidelity, and noise suppression capabilities, with stable overall exposure recovery. It can be directly used as a high-quality input for shield tunnel slag discharge monitoring systems, video analysis systems, and various intelligent detection models, providing a more reliable image foundation for automated visual recognition on engineering sites.

[0023] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A frequency-domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag discharge port of a tunnel boring machine, characterized in that, Includes the following steps: Step 1: Data Acquisition and Preprocessing: Fixed-view monitoring cameras are set up at the slag outlet of the shield tunneling site to collect long-term low-light video sequences; lighting is turned on during short construction breaks, and a small number of normal-exposure images are collected from the same viewpoint as prior samples for the project; the collected videos are split into frames to form a training dataset containing low-light shield tunneling images and normal-exposure reference images. Step 2, Noise Suppression and Foundation Restoration: Given a low-light shield tunneling image, input it into the noise suppression module to suppress complex noises introduced by dust, water mist and equipment vibration while preserving structural details, and obtain the foundation restoration image; Step 3: Construction of frequency domain illumination prior features: Input the basic restoration image into the frequency domain illumination prior estimation module, perform two-dimensional Haar wavelet decomposition on the basic restoration image to obtain low-frequency sub-bands and multiple high-frequency sub-bands, and perform weighted processing on low-frequency and high-frequency components through channel attention and spatial attention respectively. After upsampling and splicing fusion, the frequency domain illumination prior features are constructed. Step 4, Illumination map estimation: Input the basic restoration map into the illumination estimation module, and obtain the intermediate illumination response map through the transformer block and convolution aggregation. Then, perform residual connection and fusion with the local channel mean map to obtain the single channel illumination map. Step 5, Reflection Component Reconstruction: Extract features from the base restoration image and concatenate them with the frequency domain illumination prior features obtained in Step 3 along the channel dimension to form joint features. Input the joint features into the reflection component reconstruction module. In the cross-attention transformer unit, use the frequency domain illumination prior features as the query and the base restoration image features as the key and value to calculate the attention weights. The reflection component map is obtained through residual reconstruction. Step 6, Adaptive Enhancement and Image Synthesis: Input the single-channel illumination map into the illumination enhancement coefficient module, and obtain a 1×1 global description by sequentially passing it through convolution, linear rectified activation and adaptive average pooling. Then, obtain a non-negative global exponential enhancement factor by passing it through 1×1 convolution and linear rectified activation. According to the Retinex synthesis relationship, multiply the enhancement factor of the illumination map by the element-wise of the reflection component map to obtain the enhanced shield slag outlet image. Step 7, Unsupervised Training and Reference Exposure Guidance: The network constructed in Steps 2-6 is trained end-to-end using low-light frames as the main component and a small number of normal exposure frames as supplementary components. A gamma transform branch is constructed to perform nonlinear brightness transformation on the low-light image. The model is optimized through reflection invariance constraints, Retinex decomposition loss, and illumination map constraints.

2. The frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine, as described in claim 1, is characterized in that: In step 3, the specific steps for constructing the frequency domain illumination prior features are as follows: 3.1 Perform two-dimensional Haar wavelet decomposition on the basic restored image to obtain low-frequency sub-band, vertical high-frequency sub-band, horizontal high-frequency sub-band, and diagonal high-frequency sub-band; 3.2 Input the low-frequency subband into the low-frequency enhancement branch, and obtain the low-frequency illumination prior features through global average pooling, channel attention, and spatial attention weighting; 3.3 After splicing multiple high-frequency subbands along the channel dimension, the enhanced high-frequency features are obtained through global average pooling, channel attention, and spatial attention weighting. 3.

4. The low-frequency illumination prior features and the enhanced high-frequency features are transposed and upsampled by convolution, and then concatenated and convolved by 1×1 to obtain the frequency domain illumination prior features.

3. The frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring shield tunnel muck outlets according to claim 2, characterized in that: In step 3.1, the calculation formula for the two-dimensional Haar wavelet decomposition is shown in formula (1): （1）； Where X represents the basic reconstruction image, DWT represents the two-dimensional Haar wavelet decomposition operation, LL represents the low-frequency subband, and LH, HL, and HH represent the high-frequency subbands in the vertical, horizontal, and diagonal directions, respectively. In step 3.2, the calculation formulas for low-frequency subband channel description, low-frequency channel attention weight, low-frequency channel enhancement feature, low-frequency spatial attention intermediate feature, low-frequency spatial attention weight, and low-frequency illumination prior feature are shown in formulas (2), (3), (4), (5), (6), and (7), respectively: （2）； （3）； （4）； （5）； （6）； （7）； Wherein, GAP represents the global average pooling operator; The channel description vector representing the low-frequency subband; , This represents the low-frequency channel weight matrix; This represents the linear rectification activation operator; Indicates the Sigmoid activation operator; This indicates the attention weight for low-frequency channels; This represents the element-wise multiplication operator; This indicates the low-frequency channel enhancement feature; AvgPool represents the average pooling operator; MaxPool represents the max pooling operator. This represents intermediate features of low-frequency spatial attention. This represents the 7×7 convolution operator; Represents a low-frequency spatial attention weight map; This represents the a priori features of low-frequency illumination; In step 3.3, the calculation formulas for high-frequency subband splicing, high-frequency subband channel description, high-frequency channel attention weight, high-frequency channel enhancement feature, high-frequency spatial attention intermediate feature, high-frequency spatial attention weight, and enhanced high-frequency feature are shown in formulas (8), (9), (10), (11), (12), (13), and (14), respectively: （8）； （9）； （10）； （11）； （12）； （13）； （14）； Wherein, Concat represents the channel splicing operator; H represents the high-frequency splicing feature; The channel description vector representing the high-frequency subband; , Represents the high-frequency channel weight matrix; Indicates the attention weights for high-frequency channels; Indicates high-frequency channel enhancement features; Represents intermediate features of high-frequency spatial attention; Represents a high-frequency spatial attention weight map; Indicates enhanced high-frequency characteristics; In step 3.4, the calculation formulas for high and low frequency feature upsampling, high and low frequency feature splicing, and frequency domain illumination prior feature generation are shown in formulas (15), (16), and (17), respectively: （15）； （16）； （17）； Where Up represents the upsampling operation; This indicates the low-frequency characteristics of upsampling; Indicates high-frequency characteristics of upsampling; This represents a 1×1 convolution operator; represents the high- and low-frequency fusion characteristics; P represents the frequency domain illumination prior characteristics.

4. The frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine, as described in claim 3, is characterized in that: In step 5, the specific steps for reconstructing the reflection components are as follows: 5.1 Extract features from the basic restored image and concatenate them with the frequency domain illumination prior features obtained in step 3 in the channel dimension to form joint features; 5.2 Input the joint features into the reflection component reconstruction module. In the cross-attention transformer unit, use the frequency domain illumination prior features as the query and the basic restored map features as the key and value to calculate the attention weights. Suppress areas that are greatly affected by illumination changes and enhance areas with significant material and texture to obtain the cross-attention feature map. 5.

3. Perform residual connection and convolution reconstruction on the cross-attention output and input features to obtain the reflection component map.

5. The frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring shield tunnel muck outlets according to claim 4, characterized in that: In step 5.1, the formula for calculating the joint features is shown in formula (18): （18）； This represents the features extracted from the base reconstruction image; F represents the joint feature; In step 5.2, the calculation formulas for cross-attention weights, cross-attention features, and attention feature shape transformation are shown in formulas (19), (20), and (21), respectively: （19）； （20）； （21）； Where A represents the attention weight matrix; Softmax represents the normalized exponential operator; Q represents the query vector; K represents the key vector; V represents the value vector; d represents the channel dimension of a single attention head; and T represents the matrix transpose operation. Indicates shape transformation operation; Represents cross-attention features; This represents the attentional features after shape transformation; The calculation formulas for the query vector, key vector, and value vector are shown in formulas (22), (23), and (24), respectively: （22）； （23）； （24）； in, , and Learnable weight matrices for generating query vectors, key vectors, and value vectors across attention units, respectively; This means flattening and rearranging the features into a token sequence; In step 5.3, the calculation formulas for residual connectivity enhancement and reflection component map generation are shown in formulas (25) and (26): （25）； （26）； in, Indicates residual enhancement characteristics; R represents the 3×3 convolution operator; R represents the reflection component map.

6. The frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine, as described in claim 5, is characterized in that: In step 7, the specific steps of unsupervised training are as follows: 7.1 Constructing the Gamma Transform Branch: The gamma values ​​of the low-light image are randomly sampled in the interval [0.7, 1.3] and transformed. The formula for calculating the gamma value transformation is shown in formula (27): （27）； in, Represents the original low-light image; This represents the gamma value of a random sample. Represents the image after gamma transformation; 7.2 Construct a loss function constraint model, including: Reflection invariance loss function: (28); Retinex reconstruction loss function: (29); Retinex consistency loss function: (30); Total variational regularization loss function: (31); Illumination diagram constraint loss: (32) Total loss function: (33); Where N represents the total number of pixels in the image; R represents the reflectance component map; This represents the reflection component map of the image after gamma transformation; Represents the squared L2 norm; L is the single-channel illumination diagram; This indicates the removal of zero small constants; , M represents the adjacent pixels in a single-channel illumination map; M represents the maximum value of the channel in the base reconstruction map. Indicates the weight of the total variation loss; , , and The weight parameters represent the individual losses in the total loss function.

7. The frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring shield tunnel muck outlets according to claim 6, characterized in that: In step 6, adaptive average pooling is used to compress the features of the single-channel illumination map into a 1×1 global description, and 1×1 convolution with linear rectified activation is used to ensure that the global exponential enhancement factor is non-negative. Retinex assimilation relations satisfy ,in This represents the α-th power of the single-channel illumination map, where α represents the global exponential enhancement factor.

8. The frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine, as described in claim 6, is characterized in that: The values ​​for the weight parameters in step 7.2 are: , , , , ; During unsupervised training, the batch size is 8, the initial learning rate is 1e-4, cosine annealing decay is used, and the number of training iterations is 100 rounds.

9. A frequency-domain illumination prior-driven unsupervised low-light video enhancement system for monitoring the muck outlet of a tunnel boring machine (TBM), used to implement the frequency-domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the muck outlet of a TBM as described in any one of claims 1-8, characterized in that, include: Data acquisition module: used to acquire long-term low-light video sequences of the shield tunnel's slag discharge port through a fixed-view monitoring camera, as well as a small number of normally exposed images from the same viewpoint during construction breaks; Preprocessing module: used to split the acquired low-light video into frames and construct a training dataset containing low-light shield tunnel images and normal exposure reference images; Noise Suppression Module: Employs a lightweight convolutional neural network, combined with adaptive median filtering and residual learning, to receive low-light shield tunneling images, suppress complex noise introduced by dust, water mist, and equipment vibration while preserving structural details, and output a basic reconstruction image; Frequency domain illumination prior estimation module: It is used to perform two-dimensional Haar wavelet decomposition on the basic restored image to obtain low-frequency sub-bands and multiple high-frequency sub-bands. The low-frequency and high-frequency components are weighted by channel attention and spatial attention respectively. After upsampling and splicing fusion, the frequency domain illumination prior features are output. The illumination estimation module receives the basic restoration map, performs transform and convolution aggregation to obtain the intermediate illumination response map, and then connects and fuses it with the residual of the local channel mean map to output a single-channel illumination map. The reflection component reconstruction module is used to extract features from the base restored map and concatenate them with the frequency domain illumination prior features to form joint features. Attention weights are calculated through the attention transformer unit, and the reflection component map is output after residual reconstruction. Illumination enhancement factor module: This module receives the illumination map, processes it through convolution, linear rectified activation, adaptive average pooling, and 1×1 convolution, and outputs a non-negative global exponential enhancement factor. Reconstruction and Synthesis Module: This module is used to reconstruct the enhanced shield tunneling slag outlet image by multiplying the global exponential enhancement factor of the illumination image by the reflection component image element by element according to the Retinex synthesis relationship. Unsupervised training module: This module is used to train the network constructed by the above modules end-to-end using low-light frames as the main component and a small number of normal-exposure frames as the auxiliary component. The training is conducted through gamma transform branch, reflection invariance constraint, Retinex decomposition loss and illumination map constraint.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements all the steps of the frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine as described in any one of claims 1-8.

Images