Frequency-domain illumination prior-driven unsupervised low-light video enhancement method, system, and storage medium for monitoring tunnel boring machine slag outlets.

Abstract

This invention discloses a frequency-domain illumination prior-driven unsupervised low-light video enhancement method, system, and storage medium for monitoring tunnel boring machine (TBM) slag outlets. The method utilizes a noise suppression module to achieve composite noise suppression and basic restoration of the low-light image. A frequency-domain illumination prior estimation module performs two-dimensional Haar wavelet decomposition on the restored image, combining channel-spatial attention-weighted high and low frequency sub-bands and fusing them to generate frequency-domain illumination prior features. The reflection component is accurately reconstructed using a cross-attention transformer unit in the reflection component reconstruction module. A global exponential enhancement factor is generated using an illumination enhancement coefficient module to obtain the enhanced image. Unsupervised end-to-end training of the network is achieved through gamma transform branching and multi-dimensional loss constraints. This method effectively alleviates the problems of overall darkness, detail obscuration, and severe noise in TBM slag outlet monitoring images without increasing the burden of continuous on-site lighting, resulting in enhanced videos with balanced brightness, clear details, and controlled overexposure.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a frequency domain illumination prior-driven unsupervised low-light video enhancement method, system, and storage medium for monitoring the slag discharge port of a tunnel boring machine. Background Technology

[0002] At the tunnel boring machine (TBM) construction site, the slag outlet is usually located in a narrow and enclosed space at the tail of the shield. It faces harsh conditions such as low light, dust and water mist, mud splashing and mechanical vibration. This results in the images captured by the monitoring cameras generally having problems such as insufficient brightness, obscured details, severe noise and low contrast. This not only affects the visual monitoring effect of the construction, but also weakens the stability and reliability of subsequent high-level visual algorithms such as slag volume identification, rock block jam detection and mechanical anomaly diagnosis.

[0003] Existing low-light enhancement technologies have significant drawbacks:

[0004] 1. Traditional methods such as histogram equalization, gamma correction, and basic Retinex decomposition can only improve brightness in simple scenes. In the complex working conditions of shield tunneling, they are prone to problems such as noise amplification, uneven lighting, overexposure of bright areas, and loss of details in dark areas.

[0005] 2. Deep learning-based methods such as convolutional neural networks (CNN) and general Transformer augmentation models perform well in general scenarios, but they generally rely on pairs of normal light / low light training samples. However, tunnel boring machines cannot maintain strong lighting for a long time and do not have the conditions to acquire large-scale paired data.

[0006] 3. Most of the above methods only model the illumination distribution in the spatial domain, ignoring the independent variation law of the frequency domain structure (low-frequency brightness and high-frequency texture), and lack the scene prior utilization mechanism for fixed viewpoints and long-term sequences, resulting in the enhancement effect still being unstable in engineering applications.

[0007] To address this, we propose a frequency-domain illumination prior-driven unsupervised low-light video enhancement method, system, and storage medium for monitoring the slag discharge port of a tunnel boring machine. Summary of the Invention

[0008] The purpose of this invention is to provide a frequency domain illumination prior-driven unsupervised low-light video enhancement method, system, and storage medium for monitoring the slag discharge port of a tunnel boring machine, so as to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag discharge port of a tunnel boring machine, comprising the following steps:

[0010] Step 1: Data Acquisition and Preprocessing: Fixed-view monitoring cameras are set up at the slag outlet of the shield tunneling site to collect long-term low-light video sequences; lighting is turned on during short construction breaks, and a small number of normal-exposure images are collected from the same viewpoint as prior samples for the project; the collected videos are split into frames to form a training dataset containing low-light shield tunneling images and normal-exposure reference images.

[0011] Step 2, Noise Suppression and Foundation Restoration: Given a low-light shield tunneling image, input it into the noise suppression module to suppress complex noises introduced by dust, water mist and equipment vibration while preserving structural details, and obtain the foundation restoration image;

[0012] Step 3: Construction of frequency domain illumination prior features: Input the basic restoration image into the frequency domain illumination prior estimation module, perform two-dimensional Haar wavelet decomposition on the basic restoration image to obtain low-frequency sub-bands and multiple high-frequency sub-bands, and perform weighted processing on low-frequency and high-frequency components through channel attention and spatial attention respectively. After upsampling and splicing fusion, the frequency domain illumination prior features are constructed.

[0013] Step 4, Illumination map estimation: Input the basic restoration map into the illumination estimation module, and obtain the intermediate illumination response map through the transformer block and convolution aggregation. Then, perform residual connection and fusion with the local channel mean map to obtain the single channel illumination map.

[0014] Step 5, Reflection Component Reconstruction: Extract features from the base restoration image and concatenate them with the frequency domain illumination prior features obtained in Step 3 along the channel dimension to form joint features. Input the joint features into the reflection component reconstruction module. In the cross-attention transformer unit, use the frequency domain illumination prior features as the query and the base restoration image features as the key and value to calculate the attention weights. The reflection component map is obtained through residual reconstruction.

[0015] Step 6, Adaptive Enhancement and Image Synthesis: Input the single-channel illumination map into the illumination enhancement coefficient module, and obtain a 1×1 global description by sequentially passing it through convolution, linear rectified activation and adaptive average pooling. Then, obtain a non-negative global exponential enhancement factor by passing it through 1×1 convolution and linear rectified activation. According to the Retinex synthesis relationship, multiply the enhancement factor of the illumination map by the element-wise of the reflection component map to obtain the enhanced shield slag outlet image.

[0016] Step 7, Unsupervised Training and Reference Exposure Guidance: The network constructed in Steps 2-6 is trained end-to-end using low-light frames as the main component and a small number of normal exposure frames as supplementary components. A gamma transform branch is constructed to perform nonlinear brightness transformation on the low-light image. The model is optimized through reflection invariance constraints, Retinex decomposition loss, and illumination map constraints.

[0017] Preferably, in step 3, the specific steps for constructing the frequency domain illumination prior features are as follows:

[0018] 3.1 Perform two-dimensional Haar wavelet decomposition on the basic restored image to obtain low-frequency sub-band, vertical high-frequency sub-band, horizontal high-frequency sub-band, and diagonal high-frequency sub-band;

[0019] 3.2 Input the low-frequency subband into the low-frequency enhancement branch, and obtain the low-frequency illumination prior features through global average pooling, channel attention, and spatial attention weighting;

[0020] 3.3 After splicing multiple high-frequency subbands along the channel dimension, the enhanced high-frequency features are obtained through global average pooling, channel attention, and spatial attention weighting.

[0021] 3.4. The low-frequency illumination prior features and the enhanced high-frequency features are transposed and upsampled by convolution, and then concatenated and convolved by 1×1 to obtain the frequency domain illumination prior features.

[0022] Preferably, in step 3.1, the calculation formula for the two-dimensional Haar wavelet decomposition is as shown in formula (1):

[0023] (1);

[0024] Where X represents the basic reconstruction image, DWT represents the two-dimensional Haar wavelet decomposition operation, LL represents the low-frequency subband, and LH, HL, and HH represent the high-frequency subbands in the vertical, horizontal, and diagonal directions, respectively.

[0025] In step 3.2, the calculation formulas for low-frequency subband channel description, low-frequency channel attention weight, low-frequency channel enhancement feature, low-frequency spatial attention intermediate feature, low-frequency spatial attention weight, and low-frequency illumination prior feature are shown in formulas (2), (3), (4), (5), (6), and (7), respectively:

[0026] (2);

[0027] (3);

[0028] (4);

[0029] (5);

[0030] (6);

[0031] (7);

[0032] Wherein, GAP represents the global average pooling operator; The channel description vector representing the low-frequency subband; , This represents the low-frequency channel weight matrix; This represents the linear rectification activation operator; Indicates the Sigmoid activation operator; This indicates the attention weight for low-frequency channels; This represents the element-wise multiplication operator; This indicates the low-frequency channel enhancement feature; AvgPool represents the average pooling operator; MaxPool represents the max pooling operator. This represents intermediate features of low-frequency spatial attention. This represents the 7×7 convolution operator; Represents a low-frequency spatial attention weight map; Indicates the prior features of low-frequency illumination;

[0033] In step 3.3, the calculation formulas for high-frequency subband splicing, high-frequency subband channel description, high-frequency channel attention weight, high-frequency channel enhancement feature, high-frequency spatial attention intermediate feature, high-frequency spatial attention weight, and enhanced high-frequency feature are shown in formulas (8), (9), (10), (11), (12), (13), and (14), respectively:

[0034] (8);

[0035] (9);

[0036] (10);

[0037] (11);

[0038] (12);

[0039] (13);

[0040] (14);

[0041] Wherein, Concat represents the channel splicing operator; H represents the high-frequency splicing feature; The channel description vector representing the high-frequency subband; , Represents the high-frequency channel weight matrix; Indicates the attention weights for high-frequency channels; Indicates high-frequency channel enhancement features; Represents intermediate features of high-frequency spatial attention; Represents a high-frequency spatial attention weight map; Indicates enhanced high-frequency characteristics;

[0042] In step 3.4, the calculation formulas for high and low frequency feature upsampling, high and low frequency feature splicing, and frequency domain illumination prior feature generation are shown in formulas (15), (16), and (17), respectively:

[0043] (15);

[0044] (16);

[0045] (17);

[0046] Where Up represents the upsampling operation; This indicates the low-frequency characteristics of upsampling; Indicates high-frequency characteristics of upsampling; This represents a 1×1 convolution operator; represents the high- and low-frequency fusion characteristics; P represents the frequency domain illumination prior characteristics.

[0047] Preferably, in step 5, the specific steps for reconstructing the reflection components are as follows:

[0048] 5.1 Extract features from the basic restored image and concatenate them with the frequency domain illumination prior features obtained in step 3 in the channel dimension to form joint features;

[0049] 5.2 Input the joint features into the reflection component reconstruction module. In the cross-attention transformer unit, use the frequency domain illumination prior features as the query and the basic restored map features as the key and value to calculate the attention weights. Suppress areas that are greatly affected by illumination changes and enhance areas with significant material and texture to obtain the cross-attention feature map.

[0050] 5.3. Perform residual connection and convolution reconstruction on the cross-attention output and input features to obtain the reflection component map.

[0051] Preferably, in step 5.1, the calculation formula for the joint features is as shown in formula (18):

[0052] (18);

[0053] The features extracted from the base reconstruction image are represented by F; joint features are represented by F.

[0054] In step 5.2, the calculation formulas for cross-attention weights, cross-attention features, and attention feature shape transformation are shown in formulas (19), (20), and (21), respectively:

[0055] (19);

[0056] (20);

[0057] (twenty one);

[0058] Where A represents the attention weight matrix; Softmax represents the normalized exponential operator; Q represents the query vector; K represents the key vector; V represents the value vector; d represents the channel dimension of a single attention head; and T represents the matrix transpose operation. Indicates shape transformation operation; Represents cross-attention features; This represents the attentional features after shape transformation;

[0059] The calculation formulas for the query vector, key vector, and value vector are shown in formulas (22), (23), and (24), respectively:

[0060] (twenty two);

[0061] (twenty three);

[0062] (twenty four);

[0063] in, , and Learnable weight matrices for generating query vectors, key vectors, and value vectors across attention units, respectively; This means flattening and rearranging the features into a token sequence;

[0064] In step 5.3, the calculation formulas for residual connectivity enhancement and reflection component map generation are shown in formulas (25) and (26):

[0065] (25);

[0066] (26);

[0067] in, Indicates residual enhancement characteristics; R represents the 3×3 convolution operator; R represents the reflection component map.

[0068] Preferably, in step 7, the specific steps of unsupervised training are as follows:

[0069] 7.1 Constructing the Gamma Transform Branch: The gamma values ​​of the low-light image are randomly sampled in the interval [0.7, 1.3] and transformed. The formula for calculating the gamma value transformation is shown in formula (27):

[0070] (27);

[0071] in, Represents the original low-light image; This represents the gamma value of a random sample. Represents the image after gamma transformation;

[0072] 7.2 Construct a loss function constraint model, including:

[0073] Reflection invariance loss function: (28);

[0074] Retinex reconstruction loss function: (29);

[0075] Retinex consistency loss function: (30);

[0076] Total variational regularization loss function: (31);

[0077] Illumination diagram constraint loss: (32)

[0078] Total loss function: (33);

[0079] Where N represents the total number of pixels in the image; R represents the reflectance component map; This represents the reflection component map of the image after gamma transformation. Represents the squared L2 norm; L is the single-channel illumination diagram; This indicates the elimination of zero small constants; , M represents the adjacent pixels in a single-channel illumination map; M represents the maximum value of the channel in the base reconstruction map. Indicates the weight of the total variation loss; , , and The weight parameters represent the individual losses in the total loss function.

[0080] Preferably, in step 6, adaptive average pooling is used to compress the features of the single-channel illumination map into a 1×1 global description, and 1×1 convolution with linear rectified activation is used to ensure that the global exponential enhancement factor is non-negative.

[0081] Retinex assimilation relations satisfy ,in This represents the α-th power of the single-channel illumination map, where α represents the global exponential enhancement factor.

[0082] Preferably, the weight parameter in step 7.2 is set to the following values: , , , , ;

[0083] During unsupervised training, the batch size is 8, the initial learning rate is 1e-4, cosine annealing decay is used, and the number of training iterations is 100 rounds.

[0084] A frequency-domain illumination prior-driven unsupervised low-light video enhancement system for monitoring the muck discharge opening of a tunnel boring machine (TBM) is provided. This system implements a frequency-domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the muck discharge opening of a TBM, comprising:

[0085] Data acquisition module: used to acquire long-term low-light video sequences of the shield tunnel's slag discharge port through a fixed-view monitoring camera, as well as a small number of normally exposed images from the same viewpoint during construction breaks;

[0086] Preprocessing module: used to split the acquired low-light video into frames and construct a training dataset containing low-light shield tunnel images and normal exposure reference images;

[0087] Noise Suppression Module: Employs a lightweight convolutional neural network, combined with adaptive median filtering and residual learning, to receive low-light shield tunneling images, suppress complex noise introduced by dust, water mist, and equipment vibration while preserving structural details, and output a basic reconstruction image;

[0088] Frequency domain illumination prior estimation module: It is used to perform two-dimensional Haar wavelet decomposition on the basic restored image to obtain low-frequency sub-bands and multiple high-frequency sub-bands. The low-frequency and high-frequency components are weighted by channel attention and spatial attention respectively. After upsampling and splicing fusion, the frequency domain illumination prior features are output.

[0089] The illumination estimation module receives the basic restoration map, performs transform and convolution aggregation to obtain the intermediate illumination response map, and then connects and fuses it with the residual of the local channel mean map to output a single-channel illumination map.

[0090] The reflection component reconstruction module is used to extract features from the base restored map and concatenate them with the frequency domain illumination prior features to form joint features. Attention weights are calculated through the attention transformer unit, and the reflection component map is output after residual reconstruction.

[0091] Illumination enhancement factor module: This module receives the illumination map, processes it through convolution, linear rectified activation, adaptive average pooling, and 1×1 convolution, and outputs a non-negative global exponential enhancement factor.

[0092] Reconstruction and Synthesis Module: This module is used to reconstruct the enhanced shield tunneling slag outlet image by multiplying the global exponential enhancement factor of the illumination image by the reflection component image element by element according to the Retinex synthesis relationship.

[0093] Unsupervised training module: This module is used to train the network constructed by the above modules end-to-end using low-light frames as the main component and a small number of normal-exposure frames as the auxiliary component. The training is conducted through gamma transform branch, reflection invariance constraint, Retinex decomposition loss and illumination map constraint.

[0094] A computer-readable storage medium storing a computer program that, when executed by a processor, implements all steps of a frequency-domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine.

[0095] Compared with the prior art, the beneficial effects of the present invention are:

[0096] 1. This invention employs a noise suppression module, a frequency domain illumination prior estimation module, an illumination estimation module, a reflection component reconstruction module, and an illumination enhancement coefficient module to jointly construct an FRTnet network. This network collaboratively models and progressively restores the illumination distribution, material structure, and noise interference in low-light images of the shield tunneling slag outlet. During the enhancement process, this invention not only utilizes two-dimensional Haar wavelet decomposition and attention fusion to construct accurate frequency domain illumination priors, but also extracts multi-scale contextual information through cross-attention mechanism structures. This achieves effective separation of illumination and reflection components, significantly improving the accuracy of illumination estimation and the fidelity of reflection texture.

[0097] 2. This invention incorporates frequency domain brightness priors into the reflection reconstruction process through a cross-attention mechanism, avoiding the noise amplification and structural misjudgment problems commonly found in traditional low-light enhancement methods. This results in more stable performance of the enhanced image in terms of brightness balance, detail restoration, and texture consistency. Furthermore, this invention achieves adaptive nonlinear adjustment of the illumination map through an exponential illumination enhancement strategy, effectively controlling the brightening of dark areas and the suppression of bright areas while maintaining structural information without distortion. This improves the enhancement robustness under complex conditions such as dust, water mist, and local reflections.

[0098] 3. This invention constructs an unsupervised training mechanism that does not require corresponding normal light samples, through... Transformation invariance constraints, Retinex reconstruction constraints, illumination smoothness constraints, and multiple self-supervised losses such as exposure-structure-color guide the network to maintain a consistent reflection structure and natural enhancement effect under different lighting conditions, different brightness distributions, and different noise levels. Normal exposure frames collected during a small number of construction breaks are only used as reference priors for global brightness and color distribution, effectively reducing the difficulty of engineering deployment.

[0099] 4. By using the reconstruction and synthesis module and the three-stage progressive enhancement structure to output the restored image, this invention can obtain low-light enhancement results with natural brightness improvement, clear texture details, and significant noise suppression, providing more reliable and stable input data for high-level vision tasks such as slag quantity estimation at the shield tunneling outlet, rock detection, and equipment anomaly identification. Attached Figure Description

[0100] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0101] Figure 2 This is a schematic diagram of a low-light image frame obtained by video decomposition in an embodiment of the present invention;

[0102] Figure 3 This is a schematic diagram of a normally exposed frame of image obtained by video decomposition in an embodiment of the present invention;

[0103] Figure 4 This is a schematic diagram of one frame of the target exposure image obtained by the present invention. Detailed Implementation

[0104] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0105] Please see Figure 1 The present invention provides a frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag discharge port of a tunnel boring machine, comprising the following steps:

[0106] Step 1: Data Acquisition and Preprocessing: Fixed-view monitoring cameras are set up at the slag outlet of the shield tunneling site to collect long-term low-light video sequences; lighting is turned on during short construction breaks, and a small number of normal-exposure images are collected from the same viewpoint as prior samples for the project; the collected videos are split into frames to form a training dataset containing low-light shield tunneling images and normal-exposure reference images.

[0107] Step 2, Noise Suppression and Foundation Restoration: Given a low-light shield tunneling image, input it into the noise suppression module to suppress complex noises introduced by dust, water mist and equipment vibration while preserving structural details, and obtain the foundation restoration image;

[0108] Step 3: Construction of frequency domain illumination prior features: Input the basic restoration image into the frequency domain illumination prior estimation module, perform two-dimensional Haar wavelet decomposition on the basic restoration image to obtain low-frequency sub-bands and multiple high-frequency sub-bands, and perform weighted processing on low-frequency and high-frequency components through channel attention and spatial attention respectively. After upsampling and splicing fusion, the frequency domain illumination prior features are constructed.

[0109] In step 3, the specific steps for constructing the frequency domain illumination prior features are as follows:

[0110] 3.1 Perform two-dimensional Haar wavelet decomposition on the basic restored image to obtain low-frequency sub-band, vertical high-frequency sub-band, horizontal high-frequency sub-band, and diagonal high-frequency sub-band;

[0111] 3.2 Input the low-frequency subband into the low-frequency enhancement branch, and obtain the low-frequency illumination prior features through global average pooling, channel attention, and spatial attention weighting;

[0112] 3.3 After splicing multiple high-frequency subbands along the channel dimension, the enhanced high-frequency features are obtained through global average pooling, channel attention, and spatial attention weighting.

[0113] 3.4. The low-frequency illumination prior features and the enhanced high-frequency features are transposed and upsampled by convolution, and then concatenated and convolved by 1×1 to obtain the frequency domain illumination prior features.

[0114] In step 3.1, the calculation formula for the two-dimensional Haar wavelet decomposition is shown in formula (1):

[0115] (1);

[0116] Where X represents the basic reconstruction image, DWT represents the two-dimensional Haar wavelet decomposition operation, LL represents the low-frequency subband, and LH, HL, and HH represent the high-frequency subbands in the vertical, horizontal, and diagonal directions, respectively.

[0117] In step 3.2, the calculation formulas for low-frequency subband channel description, low-frequency channel attention weight, low-frequency channel enhancement feature, low-frequency spatial attention intermediate feature, low-frequency spatial attention weight, and low-frequency illumination prior feature are shown in formulas (2), (3), (4), (5), (6), and (7), respectively:

[0118] (2);

[0119] (3);

[0120] (4);

[0121] (5);

[0122] (6);

[0123] (7);

[0124] Wherein, GAP represents the global average pooling operator; The channel description vector representing the low-frequency subband; , This represents the low-frequency channel weight matrix; This represents the linear rectification activation operator; Indicates the Sigmoid activation operator; This indicates the attention weight for low-frequency channels; This represents the element-wise multiplication operator; This indicates the low-frequency channel enhancement feature; AvgPool represents the average pooling operator; MaxPool represents the max pooling operator. This represents intermediate features of low-frequency spatial attention. This represents the 7×7 convolution operator; Represents a low-frequency spatial attention weight map; Indicates the prior features of low-frequency illumination;

[0125] In step 3.3, the calculation formulas for high-frequency subband splicing, high-frequency subband channel description, high-frequency channel attention weight, high-frequency channel enhancement feature, high-frequency spatial attention intermediate feature, high-frequency spatial attention weight, and enhanced high-frequency feature are shown in formulas (8), (9), (10), (11), (12), (13), and (14), respectively:

[0126] (8);

[0127] (9);

[0128] (10);

[0129] (11);

[0130] (12);

[0131] (13);

[0132] (14);

[0133] Wherein, Concat represents the channel splicing operator; H represents the high-frequency splicing feature; The channel description vector representing the high-frequency subband; , Represents the high-frequency channel weight matrix; Indicates the attention weights for high-frequency channels; Indicates high-frequency channel enhancement features; Represents intermediate features of high-frequency spatial attention; Represents a high-frequency spatial attention weight map; Indicates enhanced high-frequency characteristics;

[0134] In step 3.4, the calculation formulas for high and low frequency feature upsampling, high and low frequency feature splicing, and frequency domain illumination prior feature generation are shown in formulas (15), (16), and (17), respectively:

[0135] (15);

[0136] (16);

[0137] (17);

[0138] Where Up represents the upsampling operation; This indicates the low-frequency characteristics of upsampling; Indicates high-frequency characteristics of upsampling; This represents a 1×1 convolution operator; P represents the high- and low-frequency fusion characteristics; P represents the frequency domain illumination prior characteristics.

[0139] Step 4, Illumination map estimation: Input the basic restoration map into the illumination estimation module, and obtain the intermediate illumination response map through the transformer block and convolution aggregation. Then, perform residual connection and fusion with the local channel mean map to obtain the single channel illumination map.

[0140] Step 5, Reflection Component Reconstruction: Extract features from the base restoration image and concatenate them with the frequency domain illumination prior features obtained in Step 3 along the channel dimension to form joint features. Input the joint features into the reflection component reconstruction module. In the cross-attention transformer unit, use the frequency domain illumination prior features as the query and the base restoration image features as the key and value to calculate the attention weights. The reflection component map is obtained through residual reconstruction.

[0141] In step 5, the specific steps for reconstructing the reflection components are as follows:

[0142] 5.1 Extract features from the basic restored image and concatenate them with the frequency domain illumination prior features obtained in step 3 in the channel dimension to form joint features;

[0143] 5.2 Input the joint features into the reflection component reconstruction module. In the cross-attention transformer unit, use the frequency domain illumination prior features as the query and the basic restored map features as the key and value to calculate the attention weights. Suppress areas that are greatly affected by illumination changes and enhance areas with significant material and texture to obtain the cross-attention feature map.

[0144] 5.3. Perform residual connections and convolutional reconstructions on the cross-attention output and input features to obtain the reflection component map;

[0145] In step 5.1, the formula for calculating the joint features is shown in formula (18):

[0146] (18);

[0147] The features extracted from the base reconstruction image are represented by F; joint features are represented by F.

[0148] In step 5.2, the calculation formulas for cross-attention weights, cross-attention features, and attention feature shape transformation are shown in formulas (19), (20), and (21), respectively:

[0149] (19);

[0150] (20);

[0151] (twenty one);

[0152] Where A represents the attention weight matrix; Softmax represents the normalized exponential operator; Q represents the query vector; K represents the key vector; V represents the value vector; d represents the channel dimension of a single attention head; and T represents the matrix transpose operation. Indicates shape transformation operation; Represents cross-attention features; This represents the attentional features after shape transformation;

[0153] The calculation formulas for the query vector, key vector, and value vector are shown in formulas (22), (23), and (24), respectively:

[0154] (twenty two);

[0155] (twenty three);

[0156] (twenty four);

[0157] in, , and Learnable weight matrices for generating query vectors, key vectors, and value vectors across attention units, respectively; This means flattening and rearranging the features into a token sequence;

[0158] In step 5.3, the calculation formulas for residual connectivity enhancement and reflection component map generation are shown in formulas (25) and (26):

[0159] (25);

[0160] (26);

[0161] in, Indicates residual enhancement characteristics; R represents the 3×3 convolution operator; R represents the reflection component map;

[0162] Step 6, Adaptive Enhancement and Image Synthesis: Input the single-channel illumination map into the illumination enhancement coefficient module, and obtain a 1×1 global description by sequentially passing it through convolution, linear rectified activation and adaptive average pooling. Then, obtain a non-negative global exponential enhancement factor by passing it through 1×1 convolution and linear rectified activation. According to the Retinex synthesis relationship, multiply the enhancement factor of the illumination map by the element-wise of the reflection component map to obtain the enhanced shield slag outlet image.

[0163] In step 6, adaptive average pooling is used to compress the features of the single-channel illumination map into a 1×1 global description, and 1×1 convolution with linear rectified activation is used to ensure that the global exponential enhancement factor is non-negative.

[0164] Retinex assimilation relations satisfy ,in This represents the α power of the single-channel illumination map, where α represents the global exponential enhancement factor;

[0165] Step 7, Unsupervised Training and Reference Exposure Guidance: The network constructed in Steps 2-6 is trained end-to-end using low-light frames as the main component and a small number of normal exposure frames as supplementary components. A gamma transform branch is constructed to perform nonlinear brightness transformation on the low-light image. The model is optimized through reflection invariance constraints, Retinex decomposition loss and illumination map constraints.

[0166] In step 7, the specific steps of unsupervised training are as follows:

[0167] 7.1 Constructing the Gamma Transform Branch: The gamma values ​​of the low-light image are randomly sampled in the interval [0.7, 1.3] and transformed. The formula for calculating the gamma value transformation is shown in formula (27):

[0168] (27);

[0169] in, Represents the original low-light image; This represents the gamma value of a random sample. Represents the image after gamma transformation;

[0170] 7.2 Construct a loss function constraint model, including:

[0171] Reflection invariance loss function: (28);

[0172] Retinex reconstruction loss function: (29);

[0173] Retinex consistency loss function: (30);

[0174] Total variational regularization loss function: (31);

[0175] Illumination diagram constraint loss: (32)

[0176] Total loss function: (33);

[0177] Where N represents the total number of pixels in the image; R represents the reflectance component map; This represents the reflection component map of the image after gamma transformation. Represents the squared L2 norm; L is the single-channel illumination diagram; This indicates the elimination of zero small constants; , M represents the adjacent pixels in a single-channel illumination map; M represents the maximum value of the channel in the base reconstruction map. Indicates the weight of the total variation loss; , , and The weight parameters represent the individual losses in the total loss function;

[0178] The values ​​for the weight parameters in step 7.2 are: , , , , ;

[0179] During unsupervised training, the batch size is 8, the initial learning rate is 1e-4, cosine annealing decay is used, and the number of training iterations is 100 rounds.

[0180] The present invention provides a frequency domain illumination prior-driven unsupervised low-light video enhancement system for monitoring the muck outlet of a tunnel boring machine (TBM). This system is used to implement a frequency domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the muck outlet of a TBM, comprising:

[0181] Data acquisition module: used to acquire long-term low-light video sequences of the shield tunnel's slag discharge port through a fixed-view monitoring camera, as well as a small number of normally exposed images from the same viewpoint during construction breaks;

[0182] Preprocessing module: used to split the acquired low-light video into frames and construct a training dataset containing low-light shield tunnel images and normal exposure reference images;

[0183] Noise Suppression Module: Employs a lightweight convolutional neural network, combined with adaptive median filtering and residual learning, to receive low-light shield tunneling images, suppress complex noise introduced by dust, water mist, and equipment vibration while preserving structural details, and output a basic reconstruction image;

[0184] Frequency domain illumination prior estimation module: It is used to perform two-dimensional Haar wavelet decomposition on the basic restored image to obtain low-frequency sub-bands and multiple high-frequency sub-bands. The low-frequency and high-frequency components are weighted by channel attention and spatial attention respectively. After upsampling and splicing fusion, the frequency domain illumination prior features are output.

[0185] The illumination estimation module receives the basic restoration map, performs transform and convolution aggregation to obtain the intermediate illumination response map, and then connects and fuses it with the residual of the local channel mean map to output a single-channel illumination map.

[0186] The reflection component reconstruction module is used to extract features from the base restored map and concatenate them with the frequency domain illumination prior features to form joint features. Attention weights are calculated through the attention transformer unit, and the reflection component map is output after residual reconstruction.

[0187] Illumination enhancement factor module: This module receives the illumination map, processes it through convolution, linear rectified activation, adaptive average pooling, and 1×1 convolution, and outputs a non-negative global exponential enhancement factor.

[0188] Reconstruction and Synthesis Module: This module is used to reconstruct the enhanced shield tunneling slag outlet image by multiplying the global exponential enhancement factor of the illumination image by the reflection component image element by element according to the Retinex synthesis relationship.

[0189] Unsupervised training module: This module is used to train the network constructed by the above modules end-to-end using low-light frames as the main component and a small number of normal-exposure frames as the auxiliary component. The training is conducted through gamma transform branch, reflection invariance constraint, Retinex decomposition loss and illumination map constraint.

[0190] The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements all steps of a frequency-domain illumination prior-driven unsupervised low-light video enhancement method for monitoring the slag outlet of a tunnel boring machine.

[0191] This embodiment is a specific application of the present invention, such as... Figure 2As shown, it is a low-light image in the monitoring scene of the shield tunnel's slag outlet. Under complex conditions such as insufficient brightness, obscured details, and interference from dust and noise, it is difficult to meet the actual construction monitoring needs.

[0192] After processing the image using the method of this invention, an enhanced target exposure image is obtained, namely... Figure 4 As shown, it is clearly visible from the visual effect. Figure 4 Significant improvements were achieved in global brightness, regional contrast, and detail structure: dark areas were effectively restored, and conveyor belt edges, rock textures, and slag discharge cavity structures were clearly presented; at the same time, image noise was suppressed, and no overexposure or clipping occurred in bright areas.

[0193] Compared with reference normal light images acquired under brief illumination conditions ( Figure 3 Compared to the image shown, its exposure effect is almost identical to the actual component texture. This invention can obtain high-quality target exposure images without increasing the burden of continuous illumination, providing reliable support for subsequent high-level vision tasks such as slag quantity identification, stone detection, and abnormal working condition analysis.

[0194] This invention provides an unsupervised low-light enhancement method driven by frequency domain illumination prior for monitoring tunnel boring machine (TBM) slag outlets. By constructing an FRTnet network structure, it organically combines a noise suppression module, a frequency domain illumination prior estimation module, an illumination estimation module, a reflection component reconstruction module, and an illumination enhancement coefficient module to achieve fine modeling of the illumination structure and reflection texture. Based on the frequency domain illumination prior constructed using 2Vell wavelet decomposition and attention fusion, this invention can accurately separate brightness information from material structure. Combined with the global modeling capability across attention mechanisms, it effectively recovers scene details under low-light conditions. Furthermore, through an exponential illumination enhancement strategy and Retinex synthesis relationship, it achieves adaptive adjustment and natural enhancement of brightness, avoiding the noise amplification and over-enhancement problems common in traditional methods.

[0195] Furthermore, the enhanced image output by the reconstruction and synthesis module of this invention exhibits outstanding performance in terms of brightness balance, detail fidelity, and noise suppression capabilities, with stable overall exposure recovery. It can be directly used as a high-quality input for shield tunnel slag discharge monitoring systems, video analysis systems, and various intelligent detection models, providing a more reliable image foundation for automated visual recognition on engineering sites.

[0196] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A frequency domain illumination prior driven unsupervised low-light video enhancement method for monitoring the shield's discharge opening, characterized in that, The method comprises the following steps: Step 1, data acquisition and preprocessing: arranging a fixed-view monitoring camera at the position of the slag outlet of the shield construction site, collecting long-time low-light video sequences; turning on the illumination in the short construction gap allowed, supplementally collecting a small amount of normally exposed images under the same view as engineering prior samples; splitting the collected video by frames to form a training data set containing low-light shield images and normally exposed reference images; Step 2, noise suppression and basic restoration: inputting a low-light shield image into a noise suppression module, suppressing complex noise introduced by dust, water mist and equipment vibration and retaining structural details to obtain a basic restoration image; Step 3, frequency domain illumination prior feature construction: inputting the basic restoration image into a frequency domain illumination prior estimation module, performing two-dimensional Haar wavelet decomposition on the basic restoration image to obtain a low-frequency subband and multiple high-frequency subbands, performing weighting processing on the low-frequency and high-frequency components through channel attention and spatial attention respectively, and performing up-sampling and splicing fusion to construct a frequency domain illumination prior feature; Step 4, illumination map estimation: inputting the basic restoration image into an illumination estimation module, obtaining an intermediate illumination response map through a transformer block and convolution aggregation, and then performing residual connection and fusion with a local channel mean value map to obtain a single-channel illumination map; Step 5, reflection component reconstruction: extracting features from the basic restoration image, splicing the features with the frequency domain illumination prior features obtained in step 3 in the channel dimension to form joint features, inputting the joint features into a reflection component reconstruction module, calculating attention weights in a cross-attention transformer unit with the frequency domain illumination prior features as queries and the basic restoration image features as keys and values, and performing residual reconstruction to obtain a reflection component map; Step 6, adaptive enhancement and image synthesis: inputting the single-channel illumination map into an illumination enhancement coefficient module, sequentially performing convolution, linear rectification activation and adaptive average pooling to obtain a 1x1 global description, then performing 1x1 convolution and linear rectification activation to obtain a non-negative global exponential enhancement factor, and according to a Retinex synthesis relationship, multiplying the enhancement factor power of the illumination map with the reflection component map element by element to obtain an enhanced shield slag outlet image; Step 7, unsupervised training and reference exposure guidance: training the network formed by steps 2-6 in an end-to-end manner mainly with low-light frames and supplemented with a small amount of normally exposed frames, constructing a gamma transformation branch to perform nonlinear brightness transformation on the low-light images, and optimizing the model through reflection invariance constraint, Retinex decomposition loss and illumination map constraint.

2. The frequency domain illumination prior driven unsupervised low-light video enhancement method for monitoring the shield's discharge port according to claim 1, characterized in that: In step 3, the specific steps of frequency domain illumination prior feature construction are as follows: 3.1, performing two-dimensional Haar wavelet decomposition on the basic restoration image to obtain a low-frequency subband, a vertical direction high-frequency subband, a horizontal direction high-frequency subband and a diagonal direction high-frequency subband; 3.2, inputting the low-frequency subband into a low-frequency enhancement branch to obtain a low-frequency illumination prior feature through global average pooling, channel attention and spatial attention weighting; 3.3, splicing the multiple high-frequency subbands in the channel dimension, and then obtaining enhanced high-frequency features through global average pooling, channel attention and spatial attention weighting; 3.4, respectively, the low-frequency lighting prior feature and the enhanced high-frequency feature are transposed and up-sampled, and then concatenated and 1x1 convolution is performed to obtain the frequency domain lighting prior feature.

3. The frequency domain illumination prior driven unsupervised low-light video enhancement method for monitoring the shield's discharge port according to claim 2, characterized in that: In step 3.1, the calculation formula of two-dimensional Haar wavelet decomposition is shown in formula (1): （1）； Wherein, X represents the basic restoration image, DWT represents the two-dimensional Haar wavelet decomposition operation, LL represents the low-frequency sub-band, LH, HL and HH represent the vertical, horizontal and diagonal high-frequency sub-band respectively; In step 3.2, the calculation formulas of low-frequency sub-band channel description, low-frequency channel attention weight, low-frequency channel enhanced feature, low-frequency spatial attention intermediate feature, low-frequency spatial attention weight and low-frequency lighting prior feature are shown in formulas (2), (3), (4), (5), (6) and (7) respectively: （2）； （3）； （4）； （5）； （6）； （7）； wherein GAP denotes a global average pooling operator; denotes a channel description vector of the low-frequency subband; , denotes a low-frequency channel weight matrix; denotes a linear rectifier activation operator; denotes a Sigmoid activation operator; denotes a low-frequency channel attention weight; denotes an element-wise multiplication operator; denotes a low-frequency channel enhanced feature; AvgPool denotes an average pooling operator; MaxPool denotes a max pooling operator; denotes a low-frequency spatial attention intermediate feature; denotes a 7x7 convolution operator; denotes a low-frequency spatial attention weight map; denotes a low-frequency illumination prior feature; In step 3.3, the calculation formulas of high-frequency sub-band concatenation, high-frequency sub-band channel description, high-frequency channel attention weight, high-frequency channel enhanced feature, high-frequency spatial attention intermediate feature, high-frequency spatial attention weight and enhanced high-frequency feature are shown in formulas (8), (9), (10), (11), (12), (13) and (14) respectively: （8）； （9）； （10）； （11）； （12）； （13）； （14）； where Concat denotes a channel concatenation operator; H denotes a high-frequency concatenation feature; denotes a channel description vector of a high-frequency subband; , denotes a high-frequency channel weight matrix; denotes a high-frequency channel attention weight; denotes a high-frequency channel enhanced feature; denotes a high-frequency spatial attention intermediate feature; denotes a high-frequency spatial attention weight map; denotes an enhanced high-frequency feature; In step 3.4, the calculation formulas of high and low frequency feature up-sampling, high and low frequency feature concatenation and frequency domain lighting prior feature generation are shown in formulas (15), (16) and (17) respectively: （15）； （16）； （17）； where Up denotes an up-sampling operation; denotes up-sampled low-frequency features; denotes up-sampled high-frequency features; denotes a 1x1 convolution operator; denotes high-low frequency fused features; and P denotes frequency domain illumination prior features.

4. The frequency domain illumination prior driven unsupervised low-light video enhancement method for monitoring the shield's discharge port according to claim 3, characterized in that: In step 5, the specific steps of the reflection component reconstruction are as follows: 5.1, extracting features from the basic restoration image and concatenating the frequency domain lighting prior feature obtained in step 3 in the channel dimension to form a joint feature; 5.2, inputting the joint feature into the reflection component reconstruction module, calculating the attention weight in the cross-attention transformer unit with the frequency domain lighting prior feature as the query, the basic restoration image feature as the key and the value, suppressing the area affected by the lighting change and enhancing the area with significant material and texture, to obtain the cross-attention feature map; 5.3, residual connection and convolution reconstruction of the cross-attention output and the input feature to obtain the reflection component map.

5. The frequency domain illumination prior driven unsupervised low-light video enhancement method for monitoring the shield's discharge opening according to claim 4, characterized in that: In step 5.1, the calculation formula of the joint feature is shown in formula (18): （18）； representing features extracted from the base restoration map; F represents the joint feature; In step 5.2, the calculation formulas of cross-attention weight, cross-attention feature and attention feature shape conversion are shown in formulas (19), (20) and (21) respectively: （19）； （20）； （21）； wherein A denotes an attention weight matrix; Softmax denotes a normalization exponential operator; Q denotes a query vector; K denotes a key vector; V denotes a value vector; d denotes a single attention head channel dimension; T denotes a transpose operation of a matrix; denotes a shape conversion operation; denotes a cross-attention feature; denotes an attention feature after shape conversion; Wherein, the calculation formulas of query vector, key vector and value vector are shown in formulas (22), (23) and (24) respectively: （22）； （23）； （24）； wherein, , and are learnable weight matrices for the cross-attention unit to generate query vectors, key vectors, and value vectors, respectively; denotes flattening and rearranging the features into a sequence of tokens; In step 5.3, the calculation formulas of residual connection enhancement and reflection component map generation are shown in formulas (25) and (26): （25）； （26）； wherein denotes a residual enhancement feature; denotes a 3x3 convolution operator; R denotes a reflectance component map.

6. The frequency domain illumination prior driven unsupervised low-light video enhancement method for monitoring the shield's tailpiece according to claim 5, wherein: In step 7, the specific steps of unsupervised training are as follows: 7.1, constructing a gamma transform branch: randomly sampling gamma values in the interval [0.7, 1.3] for low-light image transformation, and the gamma value transformation calculation formula is shown in formula (27): （27）； wherein, represents the original low light image; represents a randomly sampled gamma value; represents the image after gamma transformation; 7.2, building a loss function constraint model, including: Reflection invariance loss function: (28); Retinex reconstruction loss function: (29); Retinex consistency loss function: (30); Total variation regularized loss function: (31); illumination map constraint loss: (32) Total loss function: (33); where N represents the total number of pixels of the image; R represents the reflection component image; represents the reflection component image after gamma transformation; represents the square of L2 norm; L is a single-channel illumination map; represents the prevention of zero small constants; , represents the adjacent pixel points of the single-channel illumination map; M represents the maximum value of the basic restoration map channel; represents the total variation loss weight; , , and represents the weight parameters of each item loss in the total loss function.

7. The frequency domain illumination prior driven unsupervised low-light video enhancement method for monitoring the shield's tailpiece according to claim 6, wherein: In step 6, adaptive average pooling is used to compress the single-channel lighting map feature into a 1x1 global description, and 1x1 convolution and linear rectified activation are used to ensure that the global exponential enhancement factor is non-negative; The Retinex synthesis relation satisfies wherein denotes the alpha power of the single-channel illumination map, and alpha denotes a global exponent enhancement factor.

8. The frequency domain illumination prior driven unsupervised low-light video enhancement method for monitoring the shield's discharge opening according to claim 6, characterized in that: The weight parameter in step 7.2 is set as: , , , , ; The batch size is 8, the initial learning rate is 1e-4, the cosine annealing decay is adopted, and the number of training iterations is 100 rounds in the unsupervised training process.

9. A frequency domain illumination prior driven unsupervised low-light video enhancement system for shield slag port monitoring, for implementing the frequency domain illumination prior driven unsupervised low-light video enhancement method for shield slag port monitoring according to any one of claims 1-8, characterized in that, Comprise: A data acquisition module for acquiring long-time low-light video sequences of the shield slag outlet through a fixed-view monitoring camera and a small number of normally exposed images under the same view of the construction gap; A preprocessing module for splitting the acquired low-light video by frames, and constructing a training data set containing low-light shield images and normally exposed reference images; A noise suppression module for receiving low-light shield images, suppressing complex noise introduced by dust, water mist and equipment vibration and retaining structural details, and outputting a basic restoration image, using a lightweight convolutional neural network combined with adaptive median filtering and residual learning; A frequency domain illumination prior estimation module for performing two-dimensional Haar wavelet decomposition on the basic restoration image to obtain a low-frequency subband and multiple high-frequency subbands, and performing weighted processing on the low-frequency and high-frequency components through channel attention and spatial attention, respectively, and outputting a frequency domain illumination prior feature after upsampling and splicing fusion; An illumination estimation module for receiving a basic restoration image, obtaining an intermediate illumination response map through a transformer block and convolution aggregation, and then connecting and fusing with a local channel mean value map residual error to output a single-channel illumination map; A reflection component reconstruction module for extracting features of the basic restoration image and splicing the frequency domain illumination prior feature to form a joint feature, calculating attention weights through a cross-attention transformer unit, and outputting a reflection component map through residual reconstruction; An illumination enhancement coefficient module for receiving an illumination map, performing convolution, linear rectification activation, adaptive average pooling and 1x1 convolution processing, and outputting a non-negative global exponential enhancement factor; A reconstruction and synthesis module for reconstructing an enhanced shield slag outlet image by element-wise multiplication of the global exponential enhancement factor of the illumination map and the reflection component map according to the Retinex synthesis relationship; An unsupervised training module for training the network composed of the above modules end-to-end through a gamma transform branch, reflection invariance constraint, Retinex decomposition loss and illumination map constraint, mainly using low-light frames and supplemented by a small number of normally exposed frames.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which is executed by the processor to implement all steps of the frequency domain illumination prior driven unsupervised low-light video enhancement method for shield slag outlet monitoring according to any one of claims 1-8.

Images