Four-legged body intelligent inspection robot dual-domain adaptive night glare removal method

Abstract

The application discloses a four-legged body intelligent inspection robot dual-domain self-adaptive night glare removal method, specifically comprising the following steps: acquiring a night glare image collected by an on-board camera and performing pretreatment; subsequently, a glare removal network with an encoder-decoder structure is constructed, paired samples containing a glare image and a corresponding non-glare image are used to train the glare removal network, an end-to-end optimization is performed on a total loss function which is a weighted sum of a reconstruction loss, a structural similarity loss and a perception loss; and the pretreated night glare image is input into the trained glare removal network, and a glare-removed image is output. The application solves the problems in the prior art that the spectral structural disturbance caused by glare lacks dynamic adaptive modeling and the spatial domain and frequency domain information are insufficiently coordinated, resulting in poor image restoration quality and low visual clarity.

Description

Technical Field

[0001] This invention belongs to the field of image processing and computer vision technology, specifically relating to a dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot. Background Technology

[0002] When quadrupedal intelligent inspection robots operate in environments such as underground utility tunnels, substations, tunnels, and nighttime parks, they mainly rely on onboard cameras to complete visual tasks such as equipment recognition, obstacle detection, and path planning. However, the imaging conditions of low light and local strong light sources, coupled with the continuous changes in the viewing angle and the incident angle of the light source during the robot's movement, cause strong light to be scattered or reflected multiple times inside the lens, forming lens glare with complex spatial distribution. This results in the loss of texture details and the degradation of structural information, thereby affecting the visual perception of the inspected target.

[0003] Existing glare suppression technologies mainly include optical hardware methods, traditional computational methods, and deep learning methods. Optical hardware methods are represented by anti-reflective coatings and lens hoods. The former's suppression effect is limited by the incident wavelength and angle, while the latter has limited effect on glare caused by reflections inside the lens, and both are constrained by the size of the airborne equipment. Among traditional computational methods, the deconvolution method is based on the assumption of spatial invariance, making it difficult to adapt to complex glare with non-uniform distribution, and obtaining the point spread function is also difficult; the region segmentation and restoration method has poor performance in processing large-scale structural glare and is prone to misidentifying high-brightness light sources as glare, resulting in loss of detail.

[0004] Deep learning-based methods have made significant progress in recent years. Convolutional neural network methods often employ an encoder-decoder architecture, fusing masks or edge details to preserve scene structure, but they mainly rely on spatial domain features and do not adequately utilize the frequency domain characteristics of glare. Transformer-based methods improve restoration performance by leveraging global modeling capabilities. Some works have introduced strategies such as fast Fourier convolution, frequency band decomposition, or dual-domain feature enhancement to explicitly model frequency domain information, demonstrating the important value of frequency domain features for glare removal.

[0005] However, glare exhibits significantly different response characteristics in the frequency domain compared to glare-free images. For example... Figure 1As shown, the spectral distribution of glare-free images is relatively compact and directionally balanced, while glare images exhibit a significant directional stripe structure accompanied by enhanced low-frequency response. Furthermore, under different glare morphologies and illumination conditions, this type of spectral response variation shows significant differences in both distribution pattern and intensity. This indicates that glare interference is not simply additive noise, but a complex frequency perturbation with structural and directional bias. Existing frequency domain methods mostly rely on fixed frequency decomposition or static convolution kernels, where the frequency response is determined by predefined rules and is independent of the input content. They also rarely explicitly model the local correlations within the spectrum, making it difficult to adaptively adjust the frequency response for diverse glare degradation. Simultaneously, spatial feature extraction and frequency domain modulation are often relatively decoupled, resulting in insufficient characterization of the spatial distribution characteristics of glare. For continuously moving quadrupedal intelligent inspection robots, the incident angle and relative position of the light source change rapidly with gait. These limitations make it difficult for the model to adaptively adjust the spatial and frequency responses according to the current input, limiting the recovery effect and generalization ability.

[0006] Therefore, there is an urgent need for a nighttime glare removal method that closely coordinates spatial and frequency domains and has content-adaptive frequency adjustment capabilities, in order to meet the comprehensive requirements of quadrupedal intelligent inspection robots for glare removal effect, environmental adaptability, and onboard computing power. Summary of the Invention

[0007] The purpose of this invention is to provide a dual-domain adaptive nighttime glare removal method for quadrupedal intelligent inspection robots, which solves the problems of poor image restoration quality and low visual clarity in the prior art due to the lack of dynamic adaptive modeling of spectral structural disturbances caused by glare and insufficient coordination of spatial and frequency domain information.

[0008] The technical solution adopted in this invention is a dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot, comprising the following steps: S1, acquire nighttime glare images captured by the airborne camera and perform preprocessing; S2, construct a glare removal network with an encoder-decoder structure. Each stage of the encoder and decoder is composed of stacked hybrid window blocks. Each hybrid window block contains a local window self-attention block and a spatial-frequency domain co-module. S3 uses paired samples containing glare images and corresponding non-glare images to train the glare removal network, and performs end-to-end optimization using the weighted sum of reconstruction loss, structural similarity loss and perceptual loss as the total loss function; S4. Input the preprocessed nighttime glare image into the trained glare removal network and output the glare-removed image.

[0009] The invention is further characterized in that, The preprocessing in step S1 includes: adjusting the nighttime glare images to a preset resolution to obtain input images of consistent size; and performing pixel value normalization on the input images to map the pixel values ​​to a preset value range. Data augmentation was performed during the training phase using random scaling, translation, and shearing geometric transformations.

[0010] In step S2, the glare removal network is an encoder-decoder architecture with one convolutional layer at each end: a 3×3 convolutional layer is set at the beginning to project the input image to a high-dimensional feature space and extract shallow features to obtain a shallow feature map. The encoder consists of three stages, each consisting of several stacked blending window blocks. At the end of each stage, downsampling is used to reduce the spatial resolution of the feature map to half of its original value and double the number of channels. A bottleneck layer is set after the encoder to aggregate global context information. The decoder and encoder have a symmetrical structure and contain three stages. Each stage consists of an upsampling layer and several mixing window blocks. The feature maps of each stage of the encoder are concatenated with the feature maps of the corresponding stages of the decoder through skip connections. A 3×3 convolutional layer is set at the end to reconstruct the decoded features and output them.

[0011] Each hybrid window block consists of a local window self-attention block and a spatial-frequency domain co-processing module connected in series. The input feature map is first processed by the local window self-attention block and then sent to the spatial-frequency domain co-processing module. The local window self-attention block divides the feature map into several non-overlapping local windows. Within each window, a query vector, key vector, and value vector are generated through linear transformation. The query vector and key vector are multiplied by a dot product and normalized by a scaling factor. Then, the attention weights at each position are obtained through Softmax activation. The attention weights are weighted and summed with the value vector to obtain the self-attention output within the window. The self-attention output is transformed by a feedforward network and connected to the input feature residual. Layer normalization is used to stabilize the feature distribution. The spatial-frequency domain collaborative module sequentially sets up a multi-scale spatial feature extraction unit, a channel attention module, and an adaptive frequency modulation unit along the feature flow direction. Instance normalization is used to enhance robustness to changes in glare intensity and local contrast. The input feature map is first fed into the multi-scale spatial feature extraction unit, where multi-scale spatial features are extracted through convolutional branches with different receptive fields and then concatenated with the original input features in the channel dimension. The concatenated features are then fed into the channel attention module, where channel statistics and adaptive weighting are used to obtain a feature map enhanced with key features. This feature map is then convolved to compress the channel dimension and fed into the adaptive frequency modulation unit, where dynamic frequency modulation is performed in the frequency domain and then mapped back to the spatial domain. Finally, the frequency domain modulation result is concatenated with the residual of the original input features to obtain the output features of the spatial-frequency domain collaborative module, as shown in the following equation: ; In the formula, Y The output characteristics of the space-frequency domain co-operation module; X These are the input features for the space-frequency domain collaborative module; X c Features enhanced by a multi-scale spatial feature extraction unit and a channel attention module; Conv 1×1 (·) indicates the operation of channel dimension compression using 1×1 convolution; AFM(·) indicates the adaptive frequency modulation operation.

[0012] The multi-scale spatial feature extraction unit contains two parallel convolutional branches with kernel sizes of 3×3 and 5×5, respectively. Each branch consists of a convolutional layer, an instance normalization layer, and an activation function in sequence. The input feature map is fed into two convolutional branches for feature extraction. The features extracted by the two branches are then concatenated with the original input features along the channel dimension to obtain a multi-scale fused feature, as shown in the following expression: ; ; ; In the formula, Conv_k×k(·) represents a convolutional layer with a kernel size of k×k. For instance normalization, This involves concatenating features along the channel dimension; ReLU(·) represents the modified linear unit activation function. F 3 indicates the spatial features extracted by the 3×3 convolution branch; F 5 represents the spatial features extracted by the 5×5 convolution branch; X ms This represents the multi-scale fusion feature.

[0013] The channel attention module processes the concatenated multi-scale fused features: It compresses the spatial dimension of the feature map into channel statistical vectors using average pooling and max pooling respectively. The channel statistical vectors obtained from average pooling and max pooling are then summed element-wise. After modeling the inter-channel dependencies using a 1×1 convolutional layer, an attention weight for each channel is generated using the Sigmoid activation function. This attention weight is then used to weight the input features channel-wise, resulting in the channel-attention-enhanced output features, as shown in the following expression: ; In the formula, · represents element-wise multiplication; X c This represents the output feature after channel attention enhancement; AvgPool(·) represents the average pooling operation; MaxPool(·) represents the max pooling operation; σ(·) represents the Sigmoid activation function. The output features enhanced by channel attention are then compressed into channel dimensions by a 1×1 convolutional layer. The compressed features are then fed into an adaptive frequency modulation unit for frequency domain processing.

[0014] The adaptive frequency modulation unit performs frequency domain processing on the convolutionally compressed features as follows: First, the input spatial features are transformed to the frequency domain using a two-dimensional fast Fourier transform to obtain a complex-valued spectral representation containing real and imaginary parts, as shown in the following expression: ; In the formula, F c This represents the frequency domain features obtained after the input features have undergone a two-dimensional fast Fourier transform. F in The spatial domain characteristics of the input adaptive frequency modulation unit are represented; FFT(·) represents the two-dimensional fast Fourier transform operation; Simultaneously, a frequency attention kernel is dynamically generated based on the input features. Specifically, the spatial features of the input are aggregated using average pooling to gather global context information, and then convolutional operations are performed to extract features related to frequency modulation, generating a content-adaptive frequency attention kernel. This frequency attention kernel can adaptively adjust the response intensity of different frequency components according to the shape and intensity of glare in the input image, as shown in the following expression: ; In the formula, K represents the frequency attention kernel dynamically generated based on the input features; LeakyReLU(·) represents the linear unit activation function with leakage correction; Subsequently, this frequency attention kernel is used to perform depthwise convolution operations on the real and imaginary parts of the spectrum, with the same frequency attention kernel shared when performing depthwise convolution on the real and imaginary parts, as shown in the following expression: ; In the formula, Y f Represents the frequency domain characteristics after frequency modulation; Conv freq (·) denotes a frequency domain depthwise convolution operation; Finally, the real and imaginary parts after convolution are recombine to form a complex-valued spectrum, which is then mapped back to the spatial domain via a two-dimensional inverse fast Fourier transform to obtain the frequency-domain modulated output features, as shown in the following expression: ; In the formula, F out It represents the output characteristics mapped back to the spatial domain after frequency domain modulation; IFFT(·) represents the two-dimensional inverse fast Fourier transform operation.

[0015] The training process in step S3 includes: first, constructing paired training samples, pairing glare images with their corresponding non-glare images, then inputting the glare images into the glare removal network, and having the network simultaneously output the reconstructed non-glare image and the predicted glare-only image, and finally optimizing the network end-to-end with the total loss function. The total loss function is a weighted combination of reconstruction loss, structural similarity loss, and perceptual loss. The reconstruction loss constrains the pixel-level consistency between the glare-free image and the glare-only image and the real image, respectively, and also constrains the consistency between the reconstruction of the two images combined and the original input image. The expression is shown in the following formula: ; In the formula, Indicates the losses incurred during reconstruction; This represents the glare-free image output by the network. This represents a true, glare-free image; This represents the glare image predicted by the network. Represents a true glare image; Indicates the input glare image; This represents a pixel-by-pixel addition operation on the image; ||·| |1 Let L1 represent the L1 norm; L1(·) represents the L1 distance loss, which measures the pixel-level difference between the predicted image and the real image by calculating the average of the absolute differences between corresponding pixel values ​​of two images. The calculation formula is shown below: ; In the formula, A and B represent the two images involved in the loss calculation; and Represent the first two images respectively. The pixel value at each corresponding position; N represents the total number of pixel values ​​involved in the calculation, including the pixel values ​​in each channel; The structural similarity loss constraint restores the structural information and local contrast consistency of the image, as shown in the following expression: ; In the formula, The structural similarity loss is represented by SSIM(·), which is used to measure the similarity between two images in terms of brightness, contrast, and structural information. Its calculation formula is shown in the following formula: ; In the formula, A and B represent two images used in the structural similarity evaluation; μ A and μ B σ represents the mean pixel value within a local window of each of the two images; A ² and σ B ² represents the pixel variance within a local window of each of the two images; σ represents the variance within a local window of each image.AB C1 and C2 represent the pixel covariance within the corresponding local window of the two images; C1 and C2 are stability constants set to avoid the denominator being zero. The perceptual loss constraint restores the consistency of high-level semantic features between the image and the real image, as shown in the following expression: ; In the formula, Indicates perceived loss. (·) represents the sensing distance calculated based on the high-level features of the VGG network; The total loss function is shown in the following formula: ; In the formula, λ1, λ2 and λ3 represent the weights of reconstruction loss, structural similarity loss and perception loss, respectively.

[0016] In step S4, the preprocessed nighttime glare image is input into the trained glare removal network. Shallow features are extracted by the first 3×3 convolutional layer, and then the encoder performs multi-scale feature learning, bottleneck layer aggregation, and decoder reconstruction. The last 3×3 convolutional layer outputs the reconstructed glare-free image and the predicted glare-only image, respectively. The reconstructed glare-free image is used as the output image after glare removal.

[0017] The beneficial effects of this invention are: (1) This invention integrates multi-scale spatial feature extraction, channel attention enhancement and adaptive frequency modulation into a unified module by embedding a spatial-frequency domain collaborative module in the encoder-decoder architecture. This enables the spatial domain structure representation and frequency domain response modeling to be tightly coupled and collaboratively optimized, solving the problem that existing methods are relatively decoupled between spatial modeling and frequency modulation and are difficult to adapt to the dynamic changes in the position, shape and intensity of glare during the movement of a quadrupedal intelligent inspection robot.

[0018] Among them, adaptive frequency modulation dynamically generates frequency attention kernels based on input spatial features, applies deep convolution with shared kernels to the real and imaginary parts of the spectrum respectively, and explicitly models the local correlation within the spectrum. This replaces the traditional fixed frequency decomposition and static convolution kernel processing method, enabling the model to adaptively adjust the response of different frequency components according to the glare morphology and intensity formed by the quadrupedal intelligent inspection robot under different inspection positions, motion angles and local lighting conditions. This effectively addresses structural frequency disturbances such as directional stripes, local overexposure and low-frequency enhancement caused by glare during inspection, and improves the modeling and suppression capabilities for glare degradation in complex nighttime inspection scenarios.

[0019] (2) This invention addresses the problem that equipment, instruments, personnel, and obstacles inspected by a quadrupedal intelligent inspection robot are easily affected by local glare and interference during nighttime movement. It employs parallel convolutional branches with different receptive fields to extract multi-scale spatial features. Small kernel branches focus on capturing local texture and edge details, while large kernel branches model the large-scale contextual structure information related to glare artifacts. This complementary approach enables the network to simultaneously possess fine-grained detail perception and global structure representation capabilities, solving the problem of insufficient characterization of the spatial distribution features of glare artifacts in existing methods. Based on this, the channel attention module combines average pooling and max pooling statistical information to generate channel weights, adaptively strengthening key feature responses related to the recovery of inspection targets such as equipment, instruments, personnel, and obstacles, suppressing glare interference and redundant features, and providing more discriminative feature representations for subsequent frequency domain modeling. Through the above processing, while effectively removing glare artifacts, it reduces the erroneous removal of edges, textures, and local structures of inspection targets, avoiding excessive smoothing and loss of detail, and providing structurally complete visual information for target recognition, equipment status judgment, and environmental perception of the quadrupedal intelligent inspection robot.

[0020] (3) This invention addresses the unstable glare morphology of quadrupedal intelligent inspection robots due to the continuous changes in the relative position of the light source, the incident angle, and the viewing angle during movement. Through data-driven adaptive frequency modulation and spatial-frequency domain collaborative feature learning, the model can dynamically adjust its spatial and frequency responses according to the input content, coping with complex and varied nighttime glare interference without relying on predefined rules or fixed transformations. Simultaneously, the model employs collaborative optimization of reconstruction loss, structural similarity loss, and perception loss, achieving a balance between pixel accuracy, structural preservation, and high-level semantic consistency. Furthermore, the overall parameter and computational load is low, making it suitable for airborne deployment. Images processed by this method exhibit less glare residue and retain complete structural and texture information, effectively improving the recognition accuracy and confidence of downstream visual tasks such as target detection and semantic segmentation. This provides a stable and reliable visual perception foundation for autonomous inspection of robots in complex lighting environments such as underground utility tunnels, substations, tunnels, and nighttime parks. Attached Figure Description

[0021] Figure 1 This is a comparison chart of glare-free and glare-free images and their corresponding spectra. Figure 2 This is a network framework diagram of the dual-domain adaptive nighttime glare removal method for the quadrupedal intelligent inspection robot of the present invention; Figure 3 This is a schematic diagram of the channel attention module in this invention; Figure 4 This is a schematic diagram of the adaptive frequency modulation module in this invention; Figure 5This is a comparison of the visual effects of different glare removal methods of the present invention on the Flare7K++ real test set; Figure 6 This is a comparison chart of the visual effects of different glare removal methods of this invention on the Flare7K++ synthesis test set. Figure 7 This is a comparison of the visual effects of different general image restoration models of the present invention on the Flare7K++ real test set; Figure 8 This is a comparison of the visual effects of different general image restoration models of the present invention on the Flare7K++ synthetic test set; Figure 9 This is a comparison chart of the glare removal effects of different module ablation settings in this invention; Figure 10 A comparison chart showing the target detection performance of images processed by different glare removal methods. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Example 1 This embodiment provides a dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot, applicable to glare removal processing of images captured by airborne cameras in complex lighting environments such as nighttime parks, underground utility tunnels, substations, and tunnels. The method includes the following steps: S1: Acquire nighttime glare images captured by the airborne camera and perform preprocessing.

[0024] During nighttime inspections, quadrupedal intelligent inspection robots often encounter lens glare artifacts in the raw images captured by their onboard cameras, caused by strong light sources. These artifacts damage image texture details and structural information, thus affecting the accuracy of downstream visual perception tasks. The first step is to acquire the nighttime glare images to be processed and adjust them to a format suitable for network input. Preprocessing includes uniformly adjusting the images to a preset resolution and performing pixel value normalization.

[0025] S2, constructing a glare removal network with an encoder-decoder structure. Each stage of the encoder and decoder is composed of stacked hybrid window blocks, each of which contains a local window self-attention block and a spatial-frequency domain co-operation module.

[0026] The glare removal network employs an encoder-decoder architecture. A 3×3 convolutional layer at the beginning projects the input image from a three-channel RGB space to a high-dimensional feature space, extracting shallow features. The encoder consists of multiple stages, each composed of stacked blending window blocks. At the end of each stage, downsampling gradually reduces the feature map spatial resolution and increases the number of channels to expand the receptive field and extract multi-scale contextual information. A bottleneck layer follows the encoder to further aggregate global contextual information. The decoder is symmetrical to the encoder, gradually restoring spatial resolution through upsampling and using skip connections to concatenate features from each stage of the encoder with corresponding features from the decoder, preserving shallow texture details and structural information. A 3×3 convolutional layer at the end reconstructs the decoded features.

[0027] S3 uses paired samples containing glare images and corresponding non-glare images to train the glare removal network, and performs end-to-end optimization using the weighted sum of reconstruction loss, structural similarity loss and perceptual loss as the total loss function.

[0028] During the training phase, paired nighttime glare and non-glare images are used as supervision signals. Training samples can be constructed from public datasets or generated synthetically. Glare images are input into the network, which simultaneously outputs reconstructed non-glare images and predicted glare-only images. The total loss function consists of three parts: reconstruction loss, structural similarity loss, and perceptual loss, which respectively constrain pixel-level consistency, structural consistency, and high-level semantic feature consistency. The network parameters are optimized through end-to-end backpropagation until the model converges.

[0029] S4. Input the preprocessed nighttime glare image into the trained glare removal network and output the glare-removed image.

[0030] During the inference phase, the nighttime glare image to be processed is input into the trained glare removal network. After shallow features are extracted by the first 3×3 convolutional layer, the features are sequentially processed by the encoder for multi-scale feature learning, the bottleneck layer for global context aggregation, and the decoder for reconstruction. The last 3×3 convolutional layer outputs the reconstructed glare-free image and the predicted glare-only image, respectively. The reconstructed glare-free image is used as the glare-removed output image and can be directly used for subsequent visual tasks such as object detection and scene understanding.

[0031] Example 2 Based on Example 1, this embodiment further defines the preprocessing operation in step S1 and the overall structure of the glare removal network in step S2.

[0032] In step S1, preprocessing includes: uniformly adjusting the nighttime glare images to a preset resolution to obtain input images of consistent size. In this embodiment, the input image resolution can be set to 512×512. Then, pixel value normalization is performed on the input images, mapping the pixel values ​​to a preset numerical range, such as [0,1]. During the training phase, data augmentation can also be performed through geometric transformations such as random scaling, translation, and shearing to improve the robustness of the model to different imaging conditions and geometric changes.

[0033] like Figure 2 As shown, the glare removal network in step S2 is an encoder-decoder architecture with one convolutional layer at each end. Given an input nighttime glare image... Where H×W represents the spatial resolution, a 3×3 convolutional layer is set at the beginning of the network to project the input image onto a high-dimensional feature space and extract shallow features to obtain a shallow feature map. Where C represents the number of channels. In this embodiment, the number of channels in the shallow feature map is set to C=32.

[0034] The encoder consists of three stages. Each stage comprises several stacked hybrid window blocks. At the end of each stage, downsampling reduces the spatial resolution of the feature map to half its original value while doubling the number of channels, thereby gradually expanding the receptive field and enhancing the ability to represent high-level semantic information. In this embodiment, each encoder stage contains two repeatedly stacked hybrid window blocks. A bottleneck layer is then set after the encoder to further aggregate global contextual information and learn deeper feature representations.

[0035] The decoder and encoder have a symmetrical structure and consist of three stages. Each decoding stage comprises an upsampling layer and several blending window blocks. The upsampling layer restores the spatial resolution of the feature maps to twice their original value while reducing the number of channels to half. The feature maps from each stage of the encoder are concatenated with the corresponding feature maps from the decoder in the channel dimension via skip connections to preserve shallow texture details and structural information. Each stage of the decoder also contains two blending window blocks.

[0036] A 3×3 convolutional layer is placed at the end of the network to reconstruct the decoded features and output a reconstructed glare-free image. And predicted glare-only images Among them, the reconstructed glare-free image The output image after glare removal.

[0037] Example 3 Based on Example 2, this embodiment further defines the specific structure of the hybrid window block and the overall composition of the space-frequency domain collaborative module.

[0038] Each hybrid window block consists of a local window self-attention block and a spatial-frequency collaborative block (SFCB) connected in series. The input feature map is first processed by the local window self-attention block and then fed into the spatial-frequency collaborative block. The local window self-attention block can be implemented using a local window self-attention mechanism based on LeWin Transformer.

[0039] Specifically, the local window self-attention block divides the feature map into several non-overlapping local windows. Within each local window, a query vector, key vector, and value vector are generated through a linear transformation. The query vector and key vector are then multiplied by a dot product, normalized by a scaling factor, and activated using Softmax to obtain attention weights at each location. The attention weights are then weighted and summed with the value vector to obtain the self-attention output within the window. The self-attention output is transformed by a feedforward network and then residually connected to the input features. The local window self-attention block uses layer normalization to stabilize the feature distribution and promote network optimization. Because self-attention is computed independently within non-overlapping windows, the high computational complexity of global self-attention is avoided, while effectively modeling local contextual information.

[0040] The spatial-frequency domain co-processing module sequentially includes a multi-scale spatial feature extraction unit, a channel attention block (CAB), and an adaptive frequency modulation unit (AFM) along the feature flow direction. Unlike the local window self-attention block, the spatial-frequency domain co-processing module employs instance normalization to enhance the model's robustness to changes in glare intensity and local contrast.

[0041] The specific processing flow of the spatial-frequency domain collaborative module is as follows: The input feature map is first fed into the multi-scale spatial feature extraction unit. After multi-scale spatial features are extracted by convolutional branches with different receptive fields, they are concatenated with the original input features in the channel dimension. The concatenated features are then fed into the channel attention module. After channel statistics and adaptive weighting, the feature map after key feature enhancement is obtained. After the channel dimension of this feature map is compressed by 1×1 convolution, it is fed into the adaptive frequency modulation unit. After dynamic frequency modulation is completed in the frequency domain, it is mapped back to the spatial domain. Finally, the frequency domain modulation result is residually connected with the original input features to obtain the output features of the spatial-frequency domain collaborative module. As shown in the following equation (1): (1); In the formula, Y The output characteristics of the space-frequency domain co-operation module; X These are the input features for the space-frequency domain collaborative module; X cFeatures enhanced by a multi-scale spatial feature extraction unit and a channel attention module; Conv 1×1 (·) indicates the operation of channel dimension compression using 1×1 convolution; AFM(·) indicates the adaptive frequency modulation operation.

[0042] By connecting a spatial-frequency domain collaborative module after the self-attention block in a local window, the network can first use the self-attention mechanism to model local spatial dependencies, and then further extract multi-scale structural features and perform frequency domain adaptive modulation through the spatial-frequency domain collaborative module, thereby achieving effective collaboration between spatial and frequency domain information and improving the perception and recovery capabilities for complex glare degradation.

[0043] Example 4 This embodiment, based on embodiment 3, further defines the specific implementation of the multi-scale spatial feature extraction unit and the channel attention module.

[0044] The multi-scale spatial feature extraction unit comprises two parallel convolutional branches with kernel sizes of 3×3 and 5×5, respectively. The 3×3 convolutional branch focuses on capturing local texture and edge details, while the 5×5 branch models contextual structure information related to glare artifacts through a larger receptive field. Each branch consists of a convolutional layer, an instance normalization layer, and an activation function in sequence. In this embodiment, ReLU is used as the activation function.

[0045] Let the input features of the space-frequency domain co-operation module be... The input features are fed into two convolutional branches for feature extraction. The output features F3 of the 3×3 convolutional branch and F5 of the 5×5 convolutional branch are represented by the following equations (2) and (3): (2); (3); In the formula, Conv k×k (·) represents a convolutional layer with a kernel size of k×k. For instance normalization, ReLU(·) denotes the modified linear unit activation function. F 3 represents the spatial features extracted by the 3×3 convolution branch. F 5 represents the spatial features extracted by the 5×5 convolution branch.

[0046] The features extracted from the two branches are concatenated with the original input features along the channel dimension to obtain multi-scale fused features. As shown in equation (4): (4); In the formula, This indicates feature concatenation along the channel dimension. Xms This represents the multi-scale fusion feature.

[0047] Since the information contained in different channels varies significantly, with some channels focusing more on glare artifacts and others on scene structure and texture details, a channel attention module is set up to process the multi-scale fusion features after stitching in order to enhance key features and suppress redundant responses.

[0048] like Figure 3 As shown, the channel attention module consists of an average pooling branch, a max pooling branch, a 1×1 convolutional layer, a sigmoid activation layer, and a channel-wise weighted operation. The average pooling and max pooling branches extract channel description information of the input features under different statistical methods, the 1×1 convolutional layer establishes the correlation between features of different channels, and the sigmoid activation unit generates the attention weights for each channel. The channel-wise weighted operation adaptively adjusts the input features according to the attention weights, outputting the channel-enhanced features.

[0049] The channel attention module processes the spliced ​​multi-scale fusion features: it compresses the spatial dimension of the feature map into channel statistical vectors through average pooling and max pooling, respectively. Average pooling aggregates responses across the entire spatial range, reflecting the overall activation level of each channel; max pooling retains the most significant local responses, helping to highlight discriminative features related to glare artifacts. The channel statistical vectors obtained from average pooling and max pooling are added element-wise. After modeling the channel dependencies through a 1×1 convolutional layer, the attention weights for each channel are generated using the Sigmoid activation function. These attention weights are then used to weight the input features channel-wise, resulting in the channel-attention-enhanced output features, as shown in equation (5). (5); In the formula, · denotes element-wise multiplication. X c This represents the output feature after channel attention enhancement. AvgPool(·) represents the average pooling operation, MaxPool(·) represents the max pooling operation, and σ(·) represents the Sigmoid activation function.

[0050] The output features enhanced by channel attention are then compressed into channel dimensions by a 1×1 convolutional layer to reduce computational redundancy and promote cross-channel interaction. The compressed features are then fed into an adaptive frequency modulation unit for frequency domain processing.

[0051] By combining complementary statistical information from average pooling and max pooling, the channel attention module can simultaneously focus on the overall feature distribution and local salient responses, thereby more accurately assessing the importance of each channel. Features enhanced by channel attention not only strengthen effective information related to glare removal but also suppress irrelevant or redundant features, providing a more compact and discriminative feature representation for subsequent frequency domain modeling.

[0052] Example 5 This embodiment, based on embodiment 4, further defines the specific implementation of the adaptive frequency modulation unit.

[0053] Glare degradation not only disrupts the spatial structure of images but also produces significant changes in the spectral response in the frequency domain. For quadrupedal intelligent inspection robots, during autonomous inspection, the morphology, intensity, and spectral distribution of glare vary significantly across different scenarios due to changes in robot gait, viewing angle, and the relative position of the light source. To dynamically adjust the frequency response based on the input content, an adaptive frequency modulation unit performs frequency domain processing on the convolutionally compressed features.

[0054] like Figure 4 As shown, the adaptive frequency modulation unit has the following structure: it consists of a frequency domain transformation operation, a frequency attention kernel generation, a frequency domain feature modulation, and a frequency domain inverse transformation operation. Specifically, the frequency domain transformation operation converts the input spatial domain features into frequency domain features using a two-dimensional fast Fourier transform; the frequency attention kernel generation performs global information aggregation on the input features and generates a dynamic frequency attention kernel that matches the input content through convolution operations; the frequency domain feature modulation uses the frequency attention kernel to adaptively modulate the frequency domain features, achieving dynamic adjustment of the response intensity of different frequency components; and the frequency domain inverse transformation operation maps the modulated frequency domain features back to the spatial domain using a two-dimensional inverse fast Fourier transform, obtaining the frequency-enhanced output features.

[0055] The input spatial features are first transformed to the frequency domain by a two-dimensional fast Fourier transform to obtain a complex-valued spectral representation containing both real and imaginary parts. Let the input characteristics of the adaptive frequency modulation unit be... Then the frequency domain transformation can be expressed as shown in equation (6): (6); In the formula, F c This represents the frequency domain features obtained after the input features have undergone a two-dimensional Fast Fourier Transform. F in The input adaptive frequency modulation unit represents the spatial domain characteristics, and FFT(·) represents the two-dimensional fast Fourier transform operation.

[0056] Because the Fast Fourier Transform (FFT) possesses Hermitian symmetry, its output complex spectrum F c The real and imaginary parts can be stored as two tensors of the same shape.

[0057] Meanwhile, to adaptively adjust the frequency response according to the input content, the adaptive frequency modulation unit dynamically generates a frequency attention kernel from the spatial domain features. Specifically, the spatial features of the input are averaged and pooled to aggregate global context information, and then convolutional operations are performed to extract features related to frequency modulation, generating a content-adaptive frequency attention kernel. This frequency attention kernel can adaptively adjust the response intensity of different frequency components according to the shape and intensity of glare in the input image. Its generation process can be expressed as shown in equation (7): (7); In the formula, K represents the frequency attention kernel dynamically generated based on the input features, and LeakyReLU(·) represents the linear unit activation function with leakage correction.

[0058] Subsequently, the frequency attention kernel is used to perform deep convolution operations on the real and imaginary parts of the spectrum, and the same frequency attention kernel is shared when performing deep convolution on the real and imaginary parts. In this embodiment, a 5×5 convolution kernel is used for the deep convolution performed on the real and imaginary parts of the spectrum. By applying deep convolution to the real and imaginary parts in the frequency domain, local spectral correlation can be modeled while maintaining the independence of frequency channels, thereby more effectively capturing the frequency response changes caused by glare degradation, as shown in Equation (8): (8); In the formula, Y f Represents the frequency domain characteristics after frequency modulation, Conv freq (·) denotes a frequency domain depthwise convolution operation, used to adaptively modulate frequency domain features using a frequency attention kernel.

[0059] Finally, the real and imaginary parts after convolution are recombined into a complex-valued spectrum, which is then mapped back to the spatial domain via a two-dimensional inverse fast Fourier transform to obtain the frequency-domain modulated output features, as shown in equation (9): (9); In the formula, F out The output features are mapped back to the spatial domain after frequency domain modulation. IFFT(·) represents the two-dimensional inverse fast Fourier transform operation, which is used to restore the frequency domain features to the spatial domain.

[0060] Unlike fixed-frequency filters, adaptive frequency modulation units (CFDs) directly generate dynamic modulation parameters from spatial features, achieving tight coupling between frequency-domain filtering and spatial representation learning, enabling both to be optimized collaboratively within a unified framework. This design can dynamically adjust the response of different frequency components according to the morphology and intensity of glare in the input image, enhancing the adaptive modeling capability for complex frequency domain changes, thereby improving the recovery effect for diverse glare degradation.

[0061] Example 6 This embodiment further defines the training process in step S3 based on embodiment 5.

[0062] Step S3 training process includes the following three stages: First, paired training samples are constructed, matching glare images with their corresponding glare-free images. Training samples can be constructed using publicly available datasets, such as the Flare7K++ dataset. This dataset consists of 7000 synthetic glare images and 962 real-world images. The synthetic glare images include 5000 scattered glare images and 2000 reflected glare images, covering 25 types of scattered glare and 10 types of reflected glare. The real-world images were captured in low-light conditions using a mobile phone's rear camera, and different contaminants were applied to the lens to simulate real glare patterns. This dataset comprehensively covers the various types of glare degradation characteristics commonly encountered during nighttime patrols.

[0063] To generate paired training samples, a dynamic synthesis strategy can be used. The background image is taken from the 24K Flickr dataset, while the glare image and its corresponding light source are selected from the synthetic glare image and the real-world image with equal probability.

[0064] First, the background and glare images are converted to the linear luminance domain using inverse gamma correction, where the gamma value γ changes from a uniform distribution. U Random sampling was performed in (1.8, 2.2). To increase the diversity of glare, a series of random transformations were applied to the glare and its source, including... U Rotation within the range (0, 2π), U ( Translation within the range of 300, 300 U ( Shearing within the range of π / 9, π / 9) U Scale scaling of samples in (0.8, 1.5), based on U Gaussian blur, image flip, and (0.1,3) sampling parameters U ( Global color perturbation within the range of 0.02, 0.02.

[0065] For the background image, additional random photometric enhancement can be introduced to simulate different imaging conditions, such as multiplying the three RGB channels by a factor from... U The random factor sampled from (0.5, 1.2) is injected, and the variance satisfies σ. 2 ~0.01χ 2 Gaussian noise is used to enhance the diversity of the background scene.

[0066] After the above enhancements are performed, the light source image is superimposed on the background image, and the pixel values ​​are cropped to the range of [0,1] to generate the corresponding glare-free target image. Simultaneously, a glare-only image is obtained by removing the light source component from the glare image. Finally, the glare-only image is fused with the processed background image to generate a glare image for network training.

[0067] Secondly, the glare image is input into the glare removal network, which simultaneously outputs the reconstructed glare-free image. And predicted glare-only images .

[0068] Finally, the network is optimized end-to-end using the total loss function. The total loss function is a weighted combination of reconstruction loss, structural similarity loss, and perceptual loss.

[0069] Reconstruction losses L rec This constraint is used to constrain the pixel-level consistency between the predicted result and the real image. In addition to supervising the reconstruction of the glare-free image and the glare-only image respectively, a reconstruction consistency constraint is introduced to ensure that both can reconstruct the original input image. The reconstruction loss expression is shown in Equation (10): (10); In the formula, Indicates the losses incurred during reconstruction. This represents the glare-free image output by the network. This represents a true, glare-free image. This represents the glare image predicted by the network. Represents a true glare image. This indicates the input glare image. This represents a pixel-by-pixel addition operation on the image, ||·| |1 Let L1 represent the L1 norm, and L1(·) represent the L1 distance loss, which is used to constrain the pixel difference between the predicted result and the true result. The pixel-level difference between the predicted image and the true image is measured by calculating the average of the absolute differences between the corresponding pixel values ​​of the two images. The calculation formula is shown in the following formula (11): (11); In the formula, A and B represent the two images involved in the loss calculation; and Represent the first two images respectively. The pixel value at each corresponding position; N represents the total number of pixel values ​​involved in the calculation, including the pixel values ​​in each channel; Structural similarity loss L SSIM The expression used to constrain the structural information and local contrast consistency of the recovered image is shown in Equation (12): (12); In the formula, SSIM(·) represents the structural similarity loss and is used to measure the similarity between two images in terms of brightness, contrast, and structural information. Its calculation formula is shown in the following formula (13): (13); In the formula, A and B represent two images used in the structural similarity evaluation; μ A and μ B σ represents the mean pixel value within a local window of each of the two images; A ² and σ B ² represents the pixel variance within a local window of each of the two images; σ represents the variance within a local window of each image. AB C1 and C2 represent the pixel covariance within the corresponding local window of the two images; C1 and C2 are stability constants set to avoid the denominator being zero.

[0070] Perceived loss L per The expression used to constrain the consistency of high-level semantic features between the restored image and the real image, thereby improving the visual quality of the restored result, is shown in Equation (14): (14); In the formula, Indicates perceived loss. (·) represents the sensing distance calculated based on the high-level features of the VGG network.

[0071] The total loss function is the weighted sum of the three losses mentioned above, and its expression is shown in equation (15): (15); In the formula, λ1, λ2, and λ3 represent the weights of the reconstruction loss, structural similarity loss, and perceptual loss, respectively. In this embodiment, λ1=0.5, λ2=0.2, and λ3=0.8 are set.

[0072] During training, all input images were uniformly adjusted to a resolution of 512×512, and data augmentation was performed using geometric transformations such as random scaling, translation, and cropping. The model was trained using the Adam optimizer with an initial learning rate of 1×10⁻⁶. 4 β1 and β2 are set to 0.9 and 0.99, respectively. A MultiStepLR learning rate scheduling strategy is also introduced to gradually decay the learning rate, thereby stabilizing the training process and improving convergence performance.

[0073] Through the above training process, the network can learn the mapping relationship from glare images to glare-free images and achieve a good balance between pixel accuracy, structure preservation and visual perception quality, thereby obtaining a better nighttime glare removal effect.

[0074] Example 7 To comprehensively evaluate the nighttime glare removal performance of the dual-domain transformer with data-adaptive frequency attention for nighttimeflare removal (DDFAFormer) method for quadrupedal intelligent inspection robots of this invention, this embodiment compares it with several representative image restoration models and glare removal methods. HINet and MPRNet were retrained on the Flare7K++ dataset. Uformer, Restormer MPRNet is compared with representative image restoration models such as MambaIRv2, and with six glare removal methods proposed by Zhou et al., Kotp and Torki, Jiang et al., Huang et al., Lu et al., and Xue et al. and Restormer Both methods employ a multi-stage architecture, requiring consistent channel counts for both input and output. To accommodate these two models, the implementation example concatenates a three-channel all-zero image with a glare image to form a six-channel input tensor, ensuring network structure compatibility. All methods utilize consistent experimental settings, and training configurations and data augmentation operations remain unchanged between models.

[0075] Tables 1 and 2 present the quantitative results of different methods on real and synthetic nighttime glare datasets, respectively. As shown in Table 1, the proposed method achieves the best results in LPIPS, SSIM, and S-PSNR, indicating that DDFAFormer has significant advantages in structure preservation, perceptual quality, and stripe glare region recovery. These results validate the robustness and generalization ability of the proposed method in complex real-world scenes. Although PSNR and G-PSNR are slightly lower than the optimal results, the differences are small. Due to the greater complexity and uncertainty of glare degradation in real-world scenes, pixel-level error metrics cannot fully reflect the visual quality of images. In contrast, DDFAFormer performs more stably in maintaining scene structural integrity and visual naturalness, thus achieving better results in metrics such as SSIM and LPIPS, which better reflect perceptual quality.

[0076] Table 1 shows the performance comparison on the Flare7K++ real test set.

[0077] Table 2 shows the performance comparison on the Flare7K++ synthetic test set.

[0078] Table 2 shows that the proposed method achieves the best results in SSIM, LPIPS, and S-PSNR, and ranks second in PSNR and G-PSNR. Although there are small gaps compared to the best results in some pixel-level metrics, DDFAFormer performs better in terms of structure preservation and perceptual quality. This is mainly due to the synergistic use of spatial and frequency domain information. Through adaptive frequency modulation and spatial-frequency feature interaction, the model can more effectively model the complex spectral changes caused by glare degradation while preserving scene structure and texture details. Therefore, compared to methods that focus more on pixel-level reconstruction optimization, DDFAFormer can obtain more natural and structurally consistent restoration results. In addition, the best performance in S-PSNR indicates that the proposed method has stronger reconstruction capabilities in stripe-like glare artifact regions. At the same time, the higher SSIM and lower LPIPS further verify the model's advantages in maintaining structural integrity and visual perceptual quality. Overall, DDFAFormer achieves a good balance between glare suppression, structure preservation, and detail restoration, demonstrating strong comprehensive performance.

[0079] Figure 5 and Figure 6 The visual comparison results of DDFAFormer and other glare removal methods on the Flare7K++ real and synthetic test sets are presented. Figure 5As can be seen, in real-world scenes, DDFAFormer effectively removes large-scale glare artifacts around the light source while preserving the original structure and texture information of the scene relatively well. In contrast, some contrasting methods are prone to problems such as loss of detail, blurred edges, or local structural deformation during glare removal.

[0080] from Figure 6 As can be seen, DDFAFormer can effectively handle long-stripe glare and complex glare artifacts, accurately recovering areas affected by degradation, while maintaining good brightness distribution and color consistency in the light source area. Compared with other methods, DDFAFormer produces fewer residual artifacts and can effectively avoid over-smoothing, thus preserving rich structural details and texture information while suppressing glare interference.

[0081] Figure 7 and Figure 8 The visual comparison results of DDFAFormer and a representative general image restoration model on the Flare7K++ real and synthetic test sets are presented respectively. Figure 7 As shown, in real-world scenes, the glare morphology and intensity vary significantly across different regions, placing high demands on the model's generalization ability. For example, the second row of images contains two independent glare regions simultaneously. Most contrast methods struggle to remove both artifacts at the same time, typically only suppressing one while leaving a noticeable residue in the other. In contrast, DDFAFormer effectively removes both glare artifacts while preserving the surrounding scene content relatively well. In the third row of images, the surveillance camera is located near the glare region. HINet and MPRNet During the glare removal process, the camera structure was mistakenly removed along with the glare, and Restormer... While Uformer and others preserved the target structure, they failed to accurately restore its original brightness. Furthermore, the remaining methods still exhibited varying degrees of residual artifacts or detail distortion near the light source. In contrast, DDFAFormer achieved a cleaner restoration and better preserved scene structure and texture details.

[0082] Figure 8 Similar phenomena are observed on the synthetic nighttime glare dataset shown. Traditional image restoration models have limitations when faced with glare artifacts of different shapes and intensities, easily exhibiting problems such as artifact persistence, loss of detail, or inaccurate illumination restoration. In contrast, DDFAFormer can more accurately restore glare-affected areas while maintaining consistency between global illumination distribution and local structural information, thus achieving a more natural and realistic visual restoration effect.

[0083] Table 3 Comparison of model complexity for different glare removal methods

[0084] Table 3 presents the comparison results of DDFAFormer and other methods in terms of model parameter count and computational complexity. It can be seen that DDFAFormer has the fewest parameters among all the compared methods, while maintaining the lowest floating-point operation cost. This result demonstrates that DDFAFormer can achieve effective glare feature modeling with low computational cost, obtaining strong feature representation capabilities without relying on a large number of parameters. Therefore, DDFAFormer balances glare removal effectiveness with lightweight model and computational efficiency, providing a more practical solution for nighttime glare removal in resource-constrained scenarios.

[0085] Example 8 This embodiment conducts an ablation experiment on the dual-domain adaptive nighttime glare removal method for the quadrupedal intelligent inspection robot of the present invention. To verify the effectiveness of each key component in DDFAFormer, this embodiment removes the multi-scale 5×5 convolution branches in AFM, SFCB, CAB, and SFCB respectively. To ensure the consistency and reliability of the evaluation, all ablation experiments are conducted on the real test set and synthetic test set of Flare7K++.

[0086] Table 4 presents the ablation experiment results for each module. It can be seen that removing any key component leads to a decrease in model performance, indicating that each module plays a crucial role in nighttime glare removal.

[0087] Removing the AFM (Adaptive Frequency Modulation) significantly degraded all evaluation metrics, indicating that frequency domain modeling plays a crucial role in characterizing the complex spectral features corresponding to glare degradation. By adaptively adjusting the frequency response, the AFM enhances the model's ability to represent glare-related frequency components, thereby improving recovery quality. Removing the SFCB (Spatial-Frequency Convolutional Branch) also significantly reduced model performance. This demonstrates that collaborative modeling in the spatial and frequency domains is essential for nighttime glare removal. The SFCB, by fusing multi-scale spatial features, channel attention, and adaptive frequency modulation, enables the model to simultaneously utilize local structural information and global spectral information, resulting in more effective feature representations. Furthermore, removing the CAB (Channel-Area-Body-Body) further reduced model performance, indicating that the channel attention mechanism effectively models inter-channel dependencies and highlights important features related to glare suppression, thus improving feature utilization efficiency. Finally, directly removing the multi-scale 5×5 convolutional branch from the SFCB resulted in a decrease in model performance. This suggests that a larger receptive field helps capture long-range spatial dependencies and contextual information related to glare, while the multi-scale convolutional design further enhances the model's ability to represent glare patterns at different scales.

[0088] Table 4 shows the ablation experiment results on real and synthetic test sets.

[0089] Figure 9 The impact of different ablation settings on glare removal results was further demonstrated from a visual perspective. It can be seen that removing AFM or SFCB results in more pronounced glare residue in the restored image, and some degraded areas are difficult to fully recover, indicating that frequency domain modeling and spatial-frequency co-learning of features play a crucial role in glare suppression. Removing CAB leads to some degradation of local details in the restored image, while residual artifacts increase, indicating that the channel attention mechanism effectively enhances discriminative features related to glare removal. In contrast, removing the multi-scale 5×5 convolutional branch in SFCB limits the model's ability to model a large spatial context, resulting in incomplete recovery of some structural details. Overall, each module contributes positively to the final restoration performance, and their synergistic effect allows DDFAFormer to achieve a good balance between glare suppression, structure preservation, and detail recovery.

[0090] To analyze the impact of kernel size on AFM performance, this paper further replaced the 5×5 convolution in the module with 3×3 and 7×7 convolutions, respectively, and compared the results on real and synthetic nighttime glare test sets. The results are shown in Table 5.

[0091] Table 5 Ablation results of AFM kernel size on real and synthetic test sets.

[0092] As shown in Table 5, the 5×5 convolution exhibits the best overall performance on both test sets. On the real test set, it achieves the highest PSNR and SSIM, and the lowest LPIPS, indicating that this configuration can achieve better visual perception quality while maintaining structural consistency. Although the 7×7 convolution has a slight advantage in G-PSNR, all other metrics are lower than the 5×5 convolution, indicating that further expanding the receptive field does not bring about an overall performance improvement.

[0093] Similarly, on the synthetic test set, the 5×5 convolution achieved the best results in all five metrics: PSNR, SSIM, LPIPS, G-PSNR, and S-PSNR, further validating the effectiveness of this kernel size. In contrast, the 3×3 convolution, limited by its smaller receptive field, struggles to fully capture contextual information related to glare degradation; while the 7×7 convolution, although achieving a larger receptive field, tends to introduce redundant information, thus weakening its ability to represent local details. Overall, the 5×5 convolution achieves a more reasonable balance between local detail modeling and global context awareness, providing more effective feature representations for subsequent adaptive frequency modulation, and thus obtaining the best overall recovery performance.

[0094] To analyze the contribution of different loss terms, this paper sequentially removes each component from the total loss function and evaluates the impact of each loss term on the Flare7K++ test set using metrics such as PSNR, SSIM, and LPIPS. Table 6 shows that removing structural similarity loss or perceptual loss weakens the model's performance to varying degrees.

[0095] Table 6 Ablation analysis on real and synthetic test sets with different loss function settings.

[0096] Specifically, after removing the perceptual loss, the PSNR on the real dataset decreased from 27.970 to 27.269, and the PSNR on the synthetic dataset decreased from 30.410 to 29.690. Simultaneously, LPIPS increased significantly, indicating that pixel-level supervision alone is insufficient to adequately constrain the consistency of high-level semantic features, leading to a decline in the perceptual quality of the restored results. When the structural similarity loss was removed, SSIM and other related metrics showed varying degrees of degradation, indicating that this loss term effectively constrains the structural information and local contrast of the image, helping to maintain structural consistency between the restored results and the real image. In summary, the reconstruction loss provides pixel-level supervision, the perceptual loss enhances the consistency of high-level features, and the structural similarity loss further constrains the image's structural information. The synergistic effect of these three factors allows the model to achieve a better balance between pixel accuracy, structural preservation, and visual perceptual quality.

[0097] Figure 10 This paper presents a visual comparison of DDFAFormer with existing glare removal methods and representative image restoration models in object detection tasks. To evaluate the impact of glare degradation on downstream visual tasks, this paper uses a YOLOv11 object detector pre-trained on the MS COCO dataset to detect all test images, and presents the detection results through bounding boxes and confidence scores. Figure 10It can be seen that severe glare degradation significantly reduces the visibility of targets, leading to problems such as missed detections, false detections, and decreased detection confidence. Although the detection results have been improved after processing with other glare removal methods, problems such as residual glare artifacts, structural distortion, and over-smoothing still interfere with target recognition.

[0098] In contrast, DDFAFormer can more effectively recover the structural and textural information of glare-affected areas, thereby improving the discernibility of target areas and obtaining more accurate target localization results and higher detection confidence. Further quantitative comparisons show that for successfully detected targets, the confidence score obtained by DDFAFormer is closer to the detection results on real glare-free images, indicating that it can better preserve semantic information related to target recognition. These results demonstrate that DDFAFormer can not only improve image visual quality but also effectively improve the performance of downstream target detection tasks. This characteristic is particularly important for mobile platforms such as quadrupedal intelligent inspection robots. Under complex conditions such as continuous motion, lighting changes, and viewpoint switching, front-end image restoration not only needs to effectively suppress glare interference but also needs to retain key visual cues such as instrument scales, equipment outlines, personnel targets, and obstacle edges, thereby providing more reliable visual input for subsequent environmental perception and decision-making.

[0099] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0100] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot, characterized in that, Includes the following steps: S1, acquire nighttime glare images captured by the airborne camera and perform preprocessing; S2, construct a glare removal network with an encoder-decoder structure. Each stage of the encoder and decoder is composed of stacked hybrid window blocks. Each hybrid window block contains a local window self-attention block and a spatial-frequency domain co-module. S3 uses paired samples containing glare images and corresponding non-glare images to train the glare removal network, and performs end-to-end optimization using the weighted sum of reconstruction loss, structural similarity loss and perceptual loss as the total loss function; S4: Input the preprocessed nighttime glare image into the trained glare removal network and output the glare-removed image.

2. The dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot according to claim 1, characterized in that, The preprocessing in step S1 includes: adjusting the nighttime glare images to a preset resolution to obtain input images of consistent size; and performing pixel value normalization processing on the input images to map the pixel values ​​to a preset value range. Data augmentation was performed during the training phase using random scaling, translation, and shearing geometric transformations.

3. The dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot according to claim 1, characterized in that, In step S2, the glare removal network is an encoder-decoder architecture with one convolutional layer at the beginning and one at the end: a 3×3 convolutional layer is set at the beginning to project the input image to a high-dimensional feature space and extract shallow features to obtain a shallow feature map. The encoder consists of three stages, each consisting of several stacked blending window blocks. At the end of each stage, downsampling is used to reduce the spatial resolution of the feature map to half of its original value and double the number of channels. A bottleneck layer is set after the encoder to aggregate global context information. The decoder and encoder have a symmetrical structure and contain three stages. Each stage consists of an upsampling layer and several mixing window blocks. The feature maps of each stage of the encoder are concatenated with the feature maps of the corresponding stages of the decoder through skip connections. A 3×3 convolutional layer is set at the end to reconstruct the decoded features and output them.

4. The dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot according to claim 3, characterized in that, Each of the hybrid window blocks consists of a local window self-attention block and a spatial-frequency domain collaborative module connected in series. The input feature map is first processed by the local window self-attention block and then sent to the spatial-frequency domain collaborative module. The local window self-attention block divides the feature map into several non-overlapping local windows. Within each window, a query vector, key vector, and value vector are generated through linear transformation. The query vector and key vector are multiplied by a dot product and normalized by a scaling factor. Then, the attention weights at each position are obtained through Softmax activation. The attention weights are weighted and summed with the value vector to obtain the self-attention output within the window. The self-attention output is transformed by a feedforward network and connected to the input feature residual. Layer normalization is used to stabilize the feature distribution. The spatial-frequency domain collaborative module is set up with a multi-scale spatial feature extraction unit, a channel attention module and an adaptive frequency modulation unit along the feature flow direction. Instance normalization is used to enhance the robustness to changes in glare intensity and local contrast: the input feature map is first fed into the multi-scale spatial feature extraction unit, and after multi-scale spatial features are extracted by convolution branches of different receptive fields, they are concatenated with the original input features in the channel dimension. The concatenated features are then fed into the channel attention module, where channel statistics and adaptive weighting are applied to obtain a feature map enhanced with key features. This feature map is then convolved to compress the channel dimensions before being fed into the adaptive frequency modulation unit. After dynamic frequency modulation in the frequency domain, it is mapped back to the spatial domain. Finally, the frequency domain modulation result is concatenated with the residual of the original input features to obtain the output features of the spatial-frequency domain collaborative module, as shown in the following equation: ； In the formula, Y The output characteristics of the space-frequency domain co-operation module; X These are the input features for the space-frequency domain collaborative module; X c Features enhanced by a multi-scale spatial feature extraction unit and a channel attention module; Conv 1×1 (·) indicates the operation of channel dimension compression using 1×1 convolution; AFM(·) indicates the adaptive frequency modulation operation.

5. The dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot according to claim 4, characterized in that, The multi-scale spatial feature extraction unit contains two parallel convolutional branches with kernel sizes of 3×3 and 5×5, respectively. Each branch consists of a convolutional layer, an instance normalization layer, and an activation function in sequence. The input feature map is fed into two convolutional branches for feature extraction. The features extracted by the two branches are then concatenated with the original input features along the channel dimension to obtain a multi-scale fused feature, as shown in the following expression: ； ； ； In the formula, Conv_k×k(·) represents a convolutional layer with a kernel size of k×k. For instance normalization, This involves concatenating features along the channel dimension; ReLU(·) represents the modified linear unit activation function. F 3 indicates the spatial features extracted by the 3×3 convolution branch; F 5 represents the spatial features extracted by the 5×5 convolution branch; X ms This represents the multi-scale fusion feature.

6. The dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot according to claim 5, characterized in that, The channel attention module processes the concatenated multi-scale fused features: It compresses the spatial dimension of the feature map into channel statistical vectors using average pooling and max pooling respectively. The channel statistical vectors obtained from average pooling and max pooling are then summed element-wise. After modeling the inter-channel dependencies using a 1×1 convolutional layer, an attention weight for each channel is generated using the Sigmoid activation function. This attention weight is then used to weight the input features channel-wise, resulting in the channel-attention-enhanced output features, as shown in the following expression: ； In the formula, · represents element-wise multiplication; X c This represents the output feature after channel attention enhancement; AvgPool(·) represents the average pooling operation; MaxPool(·) represents the max pooling operation; σ(·) represents the Sigmoid activation function. The output features enhanced by channel attention are then compressed into channel dimensions by a 1×1 convolutional layer. The compressed features are then fed into an adaptive frequency modulation unit for frequency domain processing.

7. The dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot according to claim 6, characterized in that, The adaptive frequency modulation unit performs frequency domain processing on the convolutionally compressed features as follows: First, the input spatial features are transformed to the frequency domain using a two-dimensional fast Fourier transform to obtain a complex-valued spectral representation containing real and imaginary parts, as shown in the following expression: ； In the formula, F c This represents the frequency domain features obtained after the input features have undergone a two-dimensional fast Fourier transform. F in This represents the spatial domain characteristics of the input adaptive frequency modulation unit; FFT(·) represents the two-dimensional Fast Fourier Transform operation; Simultaneously, a frequency attention kernel is dynamically generated based on the input features. Specifically, the spatial features of the input are aggregated using average pooling to gather global context information, and then convolutional operations are performed to extract features related to frequency modulation, generating a content-adaptive frequency attention kernel. This frequency attention kernel can adaptively adjust the response intensity of different frequency components according to the shape and intensity of glare in the input image, as shown in the following expression: ； In the formula, K represents the frequency attention kernel dynamically generated based on the input features; LeakyReLU(·) represents the linear unit activation function with leakage correction; Subsequently, this frequency attention kernel is used to perform depthwise convolution operations on the real and imaginary parts of the spectrum, with the same frequency attention kernel shared when performing depthwise convolution on the real and imaginary parts, as shown in the following expression: ； In the formula, Y f Represents the frequency domain characteristics after frequency modulation; Conv freq (·) denotes a frequency domain depthwise convolution operation; Finally, the real and imaginary parts after convolution are recombine to form a complex-valued spectrum, which is then mapped back to the spatial domain via a two-dimensional inverse fast Fourier transform to obtain the frequency-domain modulated output features, as shown in the following expression: ； In the formula, F out It represents the output characteristics mapped back to the spatial domain after frequency domain modulation; IFFT(·) represents the two-dimensional inverse fast Fourier transform operation.

8. The dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot according to claim 1, characterized in that, The training process in step S3 includes: first, constructing paired training samples, pairing glare images with their corresponding non-glare images, then inputting the glare images into the glare removal network, and having the network simultaneously output the reconstructed non-glare image and the predicted glare-only image, and finally performing end-to-end optimization of the network using the total loss function. The total loss function is a weighted combination of reconstruction loss, structural similarity loss, and perceptual loss. The reconstruction loss constrains the pixel-level consistency between the glare-free image and the glare-only image and the real image, respectively, and also constrains the consistency between the reconstruction of the two images combined and the original input image. The expression is shown in the following formula: ； In the formula, Indicates the losses incurred during reconstruction; This represents the glare-free image output by the network. This represents a true, glare-free image; This represents the glare image predicted by the network. Represents a true glare image; Indicates the input glare image; This represents a pixel-by-pixel addition operation on the image; ||·| |1 Let L1 represent the L1 norm; L1(·) represents the L1 distance loss, which measures the pixel-level difference between the predicted image and the real image by calculating the average of the absolute differences between corresponding pixel values ​​of two images. The calculation formula is shown below: ； In the formula, A and B represent the two images involved in the loss calculation; and Represent the first two images respectively. The pixel value at each corresponding position; N represents the total number of pixel values ​​involved in the calculation, including the pixel values ​​in each channel; The structural similarity loss constraint restores the structural information and local contrast consistency of the image, as shown in the following expression: ； In the formula, The structural similarity loss is represented by SSIM(·), which is used to measure the similarity between two images in terms of brightness, contrast, and structural information. Its calculation formula is shown in the following formula: ； In the formula, A and B represent two images used in the structural similarity evaluation; μ A and μ B σ represents the mean pixel value within a local window of each of the two images; A ² and σ B ² represents the pixel variance within a local window of each of the two images; σ represents the variance within a local window of each image. AB C1 and C2 represent the pixel covariance within the corresponding local window of the two images; C1 and C2 are stability constants set to avoid the denominator being zero. The perceptual loss constraint restores the consistency of high-level semantic features between the image and the real image, as shown in the following expression: ； In the formula, Indicates perceived loss. (·) represents the sensing distance calculated based on the high-level features of the VGG network; The total loss function is shown in the following formula: ； In the formula, λ1, λ2 and λ3 represent the weights of reconstruction loss, structural similarity loss and perception loss, respectively.

9. The dual-domain adaptive nighttime glare removal method for a quadrupedal intelligent inspection robot according to claim 3, characterized in that, In step S4, the preprocessed nighttime glare image is input into the trained glare removal network. Shallow features are extracted by the first 3×3 convolutional layer, and then the encoder performs multi-scale feature learning, bottleneck layer aggregation, and decoder reconstruction. The last 3×3 convolutional layer outputs the reconstructed glare-free image and the predicted glare-only image, respectively. The reconstructed glare-free image is used as the output image after glare removal.

Images