Tunnel inspection image enhancement method and system based on conditional generative adversarial network

Abstract

The present disclosure provides a tunnel inspection image enhancement method and system based on a conditional generative adversarial network, which relates to the technical field of tunnel image processing. The method and system include obtaining a tunnel image to be enhanced; inputting the tunnel image to be enhanced into a trained image enhancement model, and outputting an enhanced optimized image; the specific training process of the image enhancement model is: obtaining a high-quality tunnel image under ideal conditions, pre-processing the high-quality image to obtain a low-quality tunnel image; inputting the low-quality tunnel image into an image enhancement model based on a conditional generative adversarial network, and generating a false high-quality image through a generator; constructing a real image pair and a false image pair, wherein the real image pair is formed by splicing a low-quality tunnel image and a high-quality tunnel image; and the false image pair is formed by splicing a low-quality tunnel image pair and a false high-quality image; and inputting the false image pair and the real image pair into a discriminator respectively for iterative discrimination until a discrimination result is obtained.

Description

Technical Field

[0001] The present disclosure relates to the technical field of tunnel image processing, and in particular to a tunnel inspection image enhancement method and system based on a conditional generative adversarial network. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] As a key component of transportation infrastructure, the safe and stable operation of tunnels is directly linked to the stability of transportation and the safety of life and property. To ensure the safe operation of tunnels, regular inspections are essential. Among the many inspection methods, image acquisition-based inspections are widely used in tunnel inspections. Regular inspections capture tunnel images that visually reveal surface defects such as cracks and water leaks in the tunnel lining, providing a key basis for tunnel maintenance decisions and playing a crucial role in the safe operation of tunnel structures. However, the unique environmental conditions inside tunnels present numerous challenges for image acquisition.

[0004] Due to the extremely complex and harsh lighting conditions in tunnels, the tunnel entrance area is directly exposed to external light and has high light intensity. The captured images are prone to being too bright or even overexposed, resulting in the loss of key details. Deep in the tunnel, the light is seriously insufficient, and the images are often dim, making it difficult to clearly display many disease features. This uneven lighting greatly increases the difficulty of fully obtaining effective information. Secondly, the humid air, diffuse dust, and electromagnetic interference generated by the operation of various equipment in the tunnel inevitably cause noise to be mixed into the captured images, seriously reducing the image quality and creating many obstacles for subsequent disease identification and analysis. In addition, existing methods can obtain high-quality images by strengthening the acquisition equipment, but the acquisition cost is high, the subsequent maintenance cost is also high, and the detection efficiency is low. Therefore, a method for efficiently enhancing the quality of tunnel inspection images is needed.

[0005] Currently, traditional image enhancement methods have significant limitations when it comes to optimizing tunnel inspection image quality. For example, while histogram equalization can improve overall image contrast, it can easily lead to over-enhancement or loss of local details when dealing with tunnel images with large brightness differences. It can also amplify image noise, further degrading image quality. While homomorphic filtering can separate the illumination and reflection components of an image, it requires extremely high filter parameter settings, making it difficult to adapt to the complex and changing environmental conditions within tunnels. Furthermore, the algorithm's high computational complexity makes it unable to meet the stringent real-time requirements of inspection work. Summary of the Invention

[0006] In order to solve the above problems, the present disclosure proposes a tunnel inspection image enhancement method and system based on conditional generative adversarial network, adopts the pix2pix model as the core architecture of the conditional generative adversarial network, and takes the tunnel inspection image as the conditional constraint to ensure that the quality enhanced image output by the generator is completely aligned with the tunnel inspection image in spatial position. At the same time, a spatial attention mechanism is introduced to adaptively focus on the key areas in the image that are most relevant to the enhancement task, better capture the local structure and texture information of the image, ignore irrelevant information and noise in the image, and improve the robustness of the model.

[0007] According to some embodiments, the present disclosure adopts the following technical solutions:

[0008] The tunnel inspection image enhancement method based on conditional generative adversarial network includes:

[0009] Acquire a tunnel image to be enhanced; input the tunnel image to be enhanced into a trained image enhancement model, and output an enhanced optimized image;

[0010] The image enhancement model is built based on a conditional generative adversarial network, and the specific training process is as follows:

[0011] Obtain high-quality tunnel images under ideal conditions, and pre-process the high-quality images to obtain low-quality tunnel images;

[0012] The generator of the conditional generative adversarial network extracts feature maps of different channel numbers and sizes from the low-quality tunnel image, and then outputs the corresponding quality-enhanced tunnel inspection image after weighted processing through the spatial attention mechanism, which is recorded as a false high-quality image;

[0013] A low-quality tunnel image and a high-quality tunnel image are spliced together to form a real image pair, and a low-quality tunnel image pair and a fake high-quality image are spliced together to form a fake image pair. The fake image pair and the real image pair are respectively input into the discriminator of the conditional generative adversarial network to obtain a discrimination result; the training is repeated until the requirements are met.

[0014] According to some embodiments, the present disclosure adopts the following technical solutions:

[0015] The tunnel inspection image enhancement system based on conditional generative adversarial networks includes:

[0016] An image acquisition module, used for acquiring the tunnel image to be enhanced;

[0017] An image enhancement module is used to input the tunnel image to be enhanced into the trained image enhancement model and output the enhanced optimized image;

[0018] The image enhancement model is built based on a conditional generative adversarial network, and the specific training process is as follows:

[0019] Obtain high-quality tunnel images under ideal conditions, and pre-process the high-quality images to obtain low-quality tunnel images;

[0020] The generator of the conditional generative adversarial network extracts feature maps of different channel numbers and sizes from the low-quality tunnel image, and then outputs the corresponding quality-enhanced tunnel inspection image after weighted processing through the spatial attention mechanism, which is recorded as a false high-quality image;

[0021] A low-quality tunnel image and a high-quality tunnel image are spliced together to form a real image pair, and a low-quality tunnel image pair and a fake high-quality image are spliced together to form a fake image pair. The fake image pair and the real image pair are respectively input into the discriminator of the conditional generative adversarial network to obtain a discrimination result; the training is repeated until the requirements are met.

[0022] According to some embodiments, the present disclosure adopts the following technical solutions:

[0023] A computer program product includes a computer program, which, when executed by a processor, implements the tunnel inspection image enhancement method based on a conditional generative adversarial network.

[0024] According to some embodiments, the present disclosure adopts the following technical solutions:

[0025] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the tunnel inspection image enhancement method based on the conditional generative adversarial network is implemented.

[0026] According to some embodiments, the present disclosure adopts the following technical solutions:

[0027] An electronic device includes: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the tunnel inspection image enhancement method based on the conditional generative adversarial network.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The tunnel inspection image enhancement method based on conditional generative adversarial network disclosed in the present invention designs an image enhancement model based on conditional generative adversarial network, adopts the pix2pix model as the core architecture of the network, and uses the tunnel inspection image as a conditional constraint to ensure that the quality enhanced image output by the generator is completely aligned with the tunnel inspection image in spatial position.

[0030] The tunnel inspection image enhancement method based on conditional generative adversarial network disclosed in this paper introduces a spatial attention module to further improve the feature extraction capability of the model, so that the pix2pix model can adaptively focus on the key areas in the image that are most relevant to the enhancement task, better capture the local structure and texture information of the image, ignore irrelevant information and noise in the image, and improve the robustness of the model.

[0031] The tunnel inspection image enhancement method based on conditional generative adversarial network disclosed in the present invention effectively makes up for the shortcomings of the above-mentioned traditional image enhancement methods, significantly improves the quality of tunnel inspection images, and provides more reliable and efficient data and technical support for tunnel safety operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which constitute a part of the present disclosure, are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation to the present disclosure.

[0033] Figure 1 This is a schematic diagram of the generator network structure of an embodiment of the present disclosure;

[0034] Figure 2 Schematic diagram of the discriminator network structure of an embodiment of the present disclosure;

[0035] Figure 3 Schematic diagram of the image enhancement model training process based on the conditional generative adversarial network in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0037] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs.

[0038] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0039] Explanation of terms

[0040] pix2pix model: As an image-derived model based on the GAN architecture, it can learn the mapping relationship between the input image and the target image, and performs well in tasks such as image restoration and enhancement.

[0041] GAN adversarial loss: It is a binary cross entropy loss function. The input of the adversarial loss is the image generated by the generator, and the output is the degree to which the generated image is discriminated as a real image by the discriminator. The purpose is to make the image generated by the generator able to deceive the discriminator.

[0042] L1 loss: It is to ensure the similarity between the generated image and the real image at the pixel level. The input of L1 loss is the image generated by the generator, and the output is a scalar value indicating the degree of pixel-level difference between the generated image and the real image. The smaller the output value, the closer the generated image and the real image are at the pixel level.

[0043] Spatial attention-guided reconstruction loss function: It aims to ensure that the image generated by the generator using spatial attention is more similar to the target image in key areas. By combining the spatial attention map with the reconstruction loss, the generator is encouraged to pay more attention to the image reconstruction quality of important spatial areas.

[0044] tf.image.resize function: A function in TensorFlow used to adjust the image size, supporting multiple interpolation methods and configuration options.

[0045] High-quality tunnel images: Tunnel images acquired under ideal conditions, specifically using an industrial camera with a resolution greater than 300 DPI. Under specified brightness, temperature, humidity, and non-reflective conditions, the grayscale values of the acquired images should be within a reasonable range of 50 to 255, with a resolution greater than 1920×1080 and noise (measured by standard deviation) less than 5%.

[0046] Low-quality tunnel images: images obtained by adding noise, reducing brightness, and blurring high-quality tunnel images. More specifically, low-quality tunnel images have a grayscale value lower than 50, a resolution lower than 1920×1080, and a noise level (measured by standard deviation) higher than 5%.

[0047] Example 1

[0048] In one embodiment of the present disclosure, a method for enhancing tunnel inspection images based on a conditional generative adversarial network is provided, comprising the following steps:

[0049] Step 1: Acquire the tunnel image to be enhanced;

[0050] Step 2: Input the tunnel image to be enhanced into the trained image enhancement model, and output the enhanced optimized image;

[0051] The image enhancement model is built based on a conditional generative adversarial network, and the specific training process is as follows:

[0052] Obtain high-quality tunnel images under ideal conditions, and pre-process the high-quality images to obtain low-quality tunnel images;

[0053] The generator of the conditional generative adversarial network extracts feature maps of different channel numbers and sizes from the low-quality tunnel image, and then outputs the corresponding quality-enhanced tunnel inspection image after weighted processing through the spatial attention mechanism, which is recorded as a false high-quality image;

[0054] A low-quality tunnel image and a high-quality tunnel image are spliced together to form a real image pair, and a low-quality tunnel image pair and a fake high-quality image are spliced together to form a fake image pair. The fake image pair and the real image pair are respectively input into the discriminator of the conditional generative adversarial network to obtain a discrimination result; the training is repeated until the requirements are met.

[0055] As an embodiment, the present disclosure discloses a tunnel inspection image enhancement method based on a conditional generative adversarial network. This method utilizes deep learning methods, particularly generative adversarial networks (GANs) and their derivative models, to demonstrate powerful image generation and processing capabilities. The pix2pix model, as an image derivative model based on the GAN architecture, can learn the mapping relationship between the input image and the target image, and performs well in tasks such as image restoration and enhancement. The present disclosure proposes an image enhancement model based on a conditional generative adversarial network, using the pix2pix model as the core architecture of the learning network and using the tunnel inspection image as a conditional constraint to ensure that the quality-enhanced image output by the generator is completely aligned with the tunnel inspection image in spatial position. To further enhance the model's feature extraction capabilities, a spatial attention module is introduced, enabling the pix2pix model to adaptively focus on the key areas in the image that are most relevant to the enhancement task, better capture the local structure and texture information of the image, ignore irrelevant information and noise in the image, and improve the robustness of the model. The specific training implementation process of the image enhancement model based on a conditional generative adversarial network disclosed in the present disclosure is as follows:

[0056] Step 1: Image data acquisition: Tunnel inspection images are collected and pre-processed through various methods.

[0057] Specifically, high-quality tunnel images under ideal conditions are acquired through a variety of methods. The process of acquiring high-quality tunnel images under ideal conditions includes: deploying a high-resolution, low-noise industrial camera with an optical lens that can automatically adjust parameters; placing the camera in a special waterproof, dustproof, and anti-reflective cover to isolate it from interference such as moisture, dust, and light reflections in the tunnel; and installing an LED matrix fill light, which is adjusted in real time according to the tunnel lighting to obtain high-quality tunnel images with high brightness, high resolution, and low noise.

[0058] The high-quality tunnel images are processed by adding noise, reducing brightness, and blurring to obtain low-quality tunnel images. Low-quality tunnel images are the result of adding noise, reducing brightness, and blurring to the high-quality tunnel images, resulting in low brightness, low resolution, and high noise.

[0059] In practical applications, due to the dim light in the tunnel, the actually acquired image is a low-quality tunnel image to be enhanced.

[0060] Furthermore, the high-quality tunnel images obtained under ideal conditions serve as the target reference for the generator's expected output during the subsequent image enhancement model training process.

[0061] Step 2: Data preprocessing: resize and normalize the acquired low-quality and high-quality tunnel images in turn to provide standardized data samples for the subsequent model training dataset construction.

[0062] 1) Resizing: Use bilinear interpolation to scale the tunnel inspection image. By calling the tf.image.resize function in TensorFlow, the image can be resized to a specified height and width, such as 256×256.

[0063] 2) Normalization: Normalize the pixel values of the tunnel inspection image to the range [-1, 1]. This can be done using the following formula:

[0064]

[0065] in, is the original pixel value, x is the normalized pixel value.

[0066] Step 3: Dataset Partitioning: Split the preprocessed dataset into training, validation, and test sets, typically in a 70% / 15% / 15% ratio. The training set is used to learn the model's training parameters, the validation set is used to evaluate the model's performance during training and adjust hyperparameters, and the test set is used to evaluate the model's generalization capabilities.

[0067] Step 4: Using the tunnel inspection image as a conditional constraint, we construct an image enhancement model based on a conditional generative adversarial network. This model uses the Pix2Pix architecture, including a generator network and a discriminator network. The specific structure is as follows:

[0068] 1) The generator network is a NetG network, which includes a downsampling module, an upsampling module, and a skip connection module. A spatial attention mechanism is added to the downsampling module and the upsampling module, and the generator loss function includes a GAN adversarial loss function, an L1 loss function, and a spatial attention-guided reconstruction loss function.

[0069] The downsampling module consists of one convolutional layer, one activation function layer (LeakyReLU function), and one normalization layer. The convolutional layers gradually reduce the spatial dimensions of the input image while increasing the number of feature channels to extract high-level features of the tunnel inspection images, helping the generator network capture semantic and contextual information in the images.

[0070] The upsampling module consists of a deconvolution layer, an activation function layer (ReLU function), and a normalization layer. A dropout layer is added to improve the robustness of the generator network, reduce reliance on specific neurons, and avoid overfitting. The upsampling module uses transposed convolution to gradually restore the spatial dimensions of the feature map to the original image size while reducing the number of feature channels.

[0071] During the downsampling process, the skip connection module, after undergoing multiple convolutional and normalization layers, generates feature maps with varying numbers of channels and sizes. When upsampling begins, the upsampled feature maps are concatenated with the feature maps of the same spatial size from the downsampling process. This implementation of skip connections helps preserve feature information across different channel counts, preventing information loss. This allows the generator to leverage both low-level and high-level feature information to produce more accurate tunnel inspection images.

[0072] Among them, in order to solve the problems of insufficient feature representation ability of the model and insufficient utilization of spatial information by the model, a spatial attention mechanism (SAM) is added to the downsampling module and upsampling module of the generator network to improve the generalization ability of the model, reduce the risk of overfitting, and enhance the interpretability of the model.

[0073] As an example, the generator extracts feature maps of different channel numbers and sizes from the conditional input image (a low-quality tunnel image), performs weighted processing through a spatial attention mechanism, and then outputs the corresponding quality-enhanced tunnel inspection image, which is denoted as a fake high-quality image. The processing of the different modules in the generator is as follows:

[0074] ① Downsampling module

[0075] The input is the pre-processed low-quality tunnel inspection image, which is used as the input feature map of the downsampling module;

[0076] Perform convolution on the input feature map (with a kernel size of 4×4 and a stride of 2). After each convolution, the feature map size is halved and the number of channels is doubled, thereby extracting higher-level and more abstract features from the image.

[0077] Perform maximum pooling and average pooling operations on the convolutional feature map, output two single-channel feature maps, and filter out important information in the input feature map;

[0078] The two single-channel feature maps output by maximum pooling and average pooling are spliced together to output a spliced feature map with more comprehensive information;

[0079] Perform a convolution operation on the spliced feature map (the convolution kernel size is 3×3). The convolution kernel will learn the importance of the spatial position of each element in the spliced feature map and then output a spatial attention map.

[0080] To enable the model to pay more attention to key feature areas such as abnormal lighting, blur and noise interference, the generated spatial attention map and the original input feature map are weighted element by element according to different spatial positions, and different weights are assigned to different areas in the feature map to obtain a weighted feature map;

[0081] The final weighted feature map is reprocessed as a new input feature map, and the downsampling module repeats the above operation several times (until the number of channels of the feature map is 512 and the size is 4×4). Each operation will reduce the size of the feature map and increase the number of channels to extract more advanced feature information.

[0082] ②Upsampling module

[0083] Except for the initial upsampling input, which is the final feature map generated by the downsampling module (with 512 channels and a size of 4×4), the input of the remaining upsampling modules is the feature map output by the previous upsampling layer;

[0084] The up-sampled input is the fused feature map after the jump connection module is spliced. The fused feature map is subjected to maximum pooling and average pooling operations respectively to obtain two single-channel feature maps to filter out important information.

[0085] The two single-channel feature maps output by maximum pooling and average pooling are spliced to form a spliced feature map;

[0086] Perform a convolution operation (convolution kernel size 3×3) on the concatenated feature map to generate a spatial attention map.

[0087] The generated spatial attention map and the original input feature map are weighted element by element according to different spatial positions, and the weights of different regions of the feature map are adjusted to obtain a weighted feature map;

[0088] The weighted feature map is upsampled using the deconvolution method (the convolution kernel size is 4×4 and the step size is 2). After each deconvolution, the size of the feature map is doubled and the number of channels is halved, thereby gradually restoring the resolution of the original image.

[0089] The upsampling operation will expand the size of the feature map and reduce the number of channels through deconvolution, gradually restoring the resolution of the image. At the same time, the upsampling process will perform jump connections with the corresponding feature maps in the downsampling process (see the jump connection module for details).

[0090] ③ Skip connection module

[0091] The input feature map of the skip connection module is the feature map of the downsampled output. The feature map of the downsampled layer is directly concatenated with the feature map of the upsampled layer in the channel dimension. That is, the downsampled feature map and the upsampled feature map of the same spatial size are stacked in the channel direction. The resulting fused feature map contains both high-level semantic information and low-level detail information, resulting in a more comprehensive and rich feature representation.

[0092] The fused feature map, as the output of the skip connection module, will be passed to the subsequent upsampling module to continue participating in the model's calculation and feature extraction process;

[0093] The function of the skip connection module is to fuse the low-level feature information extracted in the downsampling module with the feature information with restored resolution in the upsampling module, so that the generated image can retain the detail information and have a higher resolution.

[0094] 2) The discriminator network is a NetD network, which uses the discriminator of PatchGAN. The discriminator divides the input image into multiple small blocks, discriminates each small block, and finally takes the average of the sum of all the small block judgments as the final judgment result. The discriminator network consists of 3 convolutional layers, 1 normalization layer, 1 fully connected layer, and 1 spatial attention module. Through convolution and normalization, it extracts image features and converts the input image into a single-channel feature map. The spatial attention module then focuses on the spatial region of the image that has key discriminant information. The specific structure is as follows:

[0095] The discriminator uses the discriminator of PatchGAN. PatchGAN does not judge the entire tunnel inspection image, but divides the tunnel inspection image into multiple small blocks, judges each small block, and finally takes the average of the sum of the judgments of all small blocks as the final judgment result.

[0096] The discriminator network consists of three convolutional layers (with a kernel size of 4×4), one normalization layer, one fully connected layer, and one spatial attention module. Convolution and normalization are used to extract features from tunnel inspection images, converting the input image into a single-channel feature map. The spatial attention module then enables the discriminator to more accurately focus on spatial regions within the image that contain key discriminative information, thereby improving its ability to distinguish between generated and real images. A fully connected layer maps the feature map into a low-dimensional vector space, and finally, a sigmoid activation function is used to output a probability value. This discriminator network structure allows for arbitrary adjustment of the number of convolutional layers and the optional setting of a bias term (vector).

[0097] As an example, unlike the discriminator in a traditional adversarial generative network, the discriminator here no longer takes a single input, but rather a pair of images. Low-quality tunnel images, high-quality tunnel images, and fake high-quality images are paired in different ways to obtain real image pairs and fake image pairs. A real image pair is a concatenation of a low-quality tunnel image and a high-quality tunnel image; a fake image pair is a concatenation of a low-quality tunnel image pair and a fake high-quality image output by the generator. The details are as follows:

[0098] The first type of real image pair (low-quality tunnel image, high-quality tunnel image), where the low-quality image is the input image of the generator (pre-processed low-brightness / blurred / noise image), and the high-quality image is the image corresponding to the low-quality image (pre-processed normal brightness / clear image). The spliced real image pair is input into the discriminator to determine whether the pair of images is a real matching pair.

[0099] The second type of fake image pair (low-quality tunnel image, fake high-quality image), where the low-quality tunnel image is the input image of the generator (pre-processed low-brightness / blurred / noise image), and the fake high-quality image is the image generated by the generator based on the low-quality image. The spliced fake image pair is input into the discriminator to determine whether the pair of images is a real match.

[0100] As an embodiment, a spatial attention module is also introduced into the discriminator. The spatial attention module is added after the second convolutional layer in the discriminator network. Because the feature map has undergone feature extraction and has semantic information, the discriminator can better focus on key areas before making a final judgment, thereby enhancing the feature expression capability.

[0101] Specifically, the processing flow in the discriminator includes:

[0102] 1) Input image pair stitching

[0103] There are two types of input image pairs for the discriminator:

[0104] The first type of real image pair is a real image pair that is spliced together from a low-quality tunnel image (a pre-processed low-brightness / blurred / noise image) and a high-quality tunnel image (a pre-processed normal-brightness / clear image).

[0105] The second type of false image pair is a false image pair that is a combination of a low-quality tunnel image (a pre-processed low-brightness / blurred / noise image) and a false high-quality image (an image generated by the generator based on the low-quality tunnel image).

[0106] 2) Two-layer convolution extracts initial features

[0107] The concatenated image pairs are convolved using a convolutional layer with a kernel size of 4 × 4. Through two convolution operations, preliminary feature information is extracted from the image pairs, while the image resolution is gradually reduced and the number of feature channels is increased, converting the image pairs into feature map representations.

[0108] After two layers of convolution processing, the feature map contains more semantic information. At this time, adding the spatial attention module allows the discriminator to better focus on key areas before making the final judgment, thereby enhancing the feature expression capability.

[0109] 3) Spatial Attention Mechanism:

[0110] The feature map output by the second convolutional layer is subjected to maximum pooling and average pooling, respectively, to generate two single-channel feature maps. Maximum pooling highlights significant features in the feature map, while average pooling focuses on the overall feature distribution, thereby filtering out important information in the feature map.

[0111] The two single-channel feature maps output by maximum pooling and average pooling are spliced to form a spliced feature map containing more comprehensive information.

[0112] A convolution operation is performed on the concatenated feature map (with a convolution kernel size of 3×3). The convolution kernel learns the spatial importance of each element in the new feature map and generates a spatial attention map, which represents the importance of different spatial positions in the original feature map.

[0113] The generated spatial attention map is then weighted element-by-element with the feature map output by the second convolutional layer according to their spatial locations to produce a weighted feature map. This weighting allows different regions in the feature map to be assigned different weights, enabling the discriminator to more specifically focus on key regions and features within the image pair during subsequent processing. For example, areas within the image pair containing important information, such as brightness differences and structural consistency, are then considered.

[0114] 4) The third layer of convolution extracts features

[0115] The weighted feature map processed by the spatial attention module is convolved using a 4×4 convolution kernel to further extract and refine features, thereby improving the abstraction level and discriminative ability of the features.

[0116] 5) Normalization

[0117] Normalize the feature maps output by the third convolutional layer. Normalization can stabilize the network training process, accelerate convergence, and help alleviate the problem of vanishing or exploding gradients, making the data distribution of the feature maps more stable and reasonable.

[0118] 6) Output judgment results

[0119] After the above series of processing, the discriminator makes a final judgment on the feature map output by the normalization layer. A fully connected layer maps the feature map to a low-dimensional vector space, and then uses a sigmoid activation function to output a scalar value. This scalar value represents the discriminator's judgment on whether the input image pair is a true match. Values closer to 1 indicate a higher probability that the discriminator considers the image pair a true match, while values closer to 0 indicate a higher probability of a false match.

[0120] After adding the spatial attention module to the discriminator, the discriminator can automatically focus on those spatial areas in the image that have a greater impact on the discrimination results, such as key feature areas such as lighting anomalies, blurriness, and noise interference that may exist in tunnel inspection images. This helps the discriminator more accurately identify the differences between generated images and real images, thereby prompting the generator to continuously improve the quality of generated images and enhance the performance of the entire generative adversarial network.

[0121] Step 5: Construct the loss function of the image enhancement model based on the conditional generative adversarial network. The loss function of the image enhancement model based on the conditional generative adversarial network includes the generator loss function and the discriminator loss function. The generator loss function includes the GAN adversarial loss function (Generative Adversarial Network Adversarial Loss), the L1 loss function (Mean Absolute Error Loss) and the spatial attention-guided reconstruction loss function (Spatial Attention-Guided Reconstruction Loss).

[0122] (1) The generator loss function is as follows:

[0123] ①GAN adversarial loss function

[0124] The GAN adversarial loss is a binary cross-entropy loss function. The input of the adversarial loss is the image generated by the generator, and the output is the degree to which the generated image is identified as a real image by the discriminator. The goal is to make the images generated by the generator deceive the discriminator. The adversarial loss is used to measure the difference between the images generated by the generator and the real images in the discriminator, encouraging the images generated by the generator to be identified as real images by the discriminator.

[0125]

[0126] in: x is the input image; G(x) Is a generator G Based on the input image x The generated image; D(G(x)) is the discriminator D to generate the image G(x) The discriminant result is a probability value, which indicates the probability that the discriminator believes that the image is a real image. E x Indicates about variables x The mathematical expectation of the distribution (i.e. x On the entire distribution of (D(G(x))-1)² Perform average calculation to measure the average of the discriminator’s judgment results on the samples generated by the generator).

[0127] ②L1 loss function

[0128] The L1 loss is designed to ensure pixel-level similarity between the generated image and the real image. The input of the L1 loss is the image generated by the generator, and the output is a scalar value representing the degree of pixel-level difference between the generated image and the real image. The smaller the output value, the closer the generated image and the real image are at the pixel level. It directly calculates the average of the absolute pixel-level differences between the generated image and the real image, ensuring that the generated image is closer to the real image in terms of detail and content.

[0129]

[0130] in: x is the input image; y is the real image corresponding to the input image x; G(x) Is a generator G Based on the input image x The generated image. E xy Indicates about variables x and y The mathematical expectation of the joint distribution (i.e. x and y On the joint distribution of yG (x 1 )The average calculation is used to measure the average difference between the generator output and the true target).

[0131] ③ Spatial attention-guided reconstruction loss function

[0132] The spatial attention-guided reconstruction loss function aims to ensure that the image generated by the generator using spatial attention is more similar to the target image in key areas. By combining the spatial attention map with the reconstruction loss, the generator is encouraged to pay more attention to the image reconstruction quality of important spatial areas.

[0133] Let the input image of the generator be x, and the generated image be G(x) , the target image is y , the spatial attention map is A G First, the attention map A G Normalize the pixels so that their element values are in the range [0, 1], then calculate the pixel-by-pixel mean square error (MSE) loss and weight it according to the attention map:

[0134]

[0135] in: H, W, C are the height, width and number of channels of the image, respectively. A ij is the position in the attention map ( i,j ), G(x) ijk and y ijk are the generated image and the target image at positions ( i, j, k ) is the pixel value at .

[0136] Will GAN Fighting losses, L1 The loss and the reconstruction loss function guided by spatial attention are combined to take into account the realism of the generated image (adversarial loss), detail accuracy ( L1 loss) and key area quality (spatial attention guided reconstruction loss function). By introducing hyperparameters To adjust the adversarial loss and L1 The weight of the loss increases the flexibility and adaptability of the model.

[0137]

[0138] in: is a hyperparameter used to adjust GAN Fighting losses, L1 loss and the weights of the spatial attention-guided reconstruction loss.

[0139] (2) The discriminator loss function is as follows:

[0140] The discriminator loss function uses the least squares loss function (Least Squares GAN, LSGAN) and the attention region feature difference loss function (Attention Region Feature Difference Loss).

[0141] ① Least squares loss function

[0142] When the discriminator input image is a false image pair, the output of the discriminator D is expected to generate an image pair. D (Low quality tunnel image, G (low-quality tunnel images)) is close to 0. By minimizing this part of the loss, the discriminator is trained to identify the generated image pairs as fake, that is, output a lower probability value (close to 0).

[0143]

[0144] When the discriminator input image is a real image pair (low-quality tunnel image, high-quality tunnel image), the expected output of the discriminator D for the real image pair is D (Low-quality tunnel image, high-quality tunnel image) is close to 1. By minimizing this part of the loss, the discriminator is trained to identify real image pairs as real, that is, output a higher probability value (close to 1).

[0145]

[0146] ②Attention region feature difference loss function

[0147] This loss function focuses on the feature differences in the area of spatial attention, prompting the discriminator to more keenly capture the feature differences between the generated image and the real image in the key areas, thereby enhancing the ability to discriminate the generated image.

[0148] Assume the real image is y , the generated image is G(x) , the spatial attention map of the discriminator is A D First, the real image and the generated image are input into the discriminator to obtain the corresponding feature map F y and F G(x) . For attention map A D Perform binarization to obtain the mask M D , only retain the attention area, and calculate the mean square error (MSE) of the elements at the corresponding position of the feature map in the attention area:

[0149]

[0150] in: N M It is a mask M D The total number of elements in , F yi and F G(x)i are the feature element values of the real image and the generated image at the corresponding positions in the mask area.

[0151] Introducing hyperparameters To adjust the weights of the least squares loss and the attention region feature difference loss, the flexibility and adaptability of the model are increased. The total loss of the discriminator is as follows:

[0152]

[0153] Step 6: Use Adam Optimizer, used to update the parameters of the generator and discriminator to minimize their loss function, set the hyperparameter learning rate ( learning rate ) is 0.0002, betas is (0.5, 0.999).

[0154] Step 7: Training cycle. The training cycle process includes:

[0155] (1) Forward propagation:

[0156] In each training iteration, the input image and the true target image are fed into the discriminator as a true image pair, and the discriminator outputs the true image pair. At the same time, the input image is passed through the generator to obtain the generated image, and the input image and the generated image are fed into the discriminator as a generated image pair, and the discriminator outputs the generated image pair.

[0157] (2) Calculate the loss:

[0158] For the discriminator, the discriminator's loss function is used to calculate the discriminator's loss based on its output for the real image pair and the generated image pair. For the generator, the generator's loss function is used to calculate the generator's loss based on the generated image and the real target image.

[0159] (3) Back propagation:

[0160] Calculate the gradient of the discriminator loss with respect to the discriminator parameters, as well as the gradient of the generator loss with respect to the generator parameters. Update the model parameters through the backpropagation algorithm and continuously adjust the model weights so that the generated image gradually approaches the real high-quality enhanced image.

[0161] (4) Update parameters:

[0162] The discriminator's optimizer is used to update the discriminator's parameters according to the calculated gradient. The generator's optimizer is used to update the generator's parameters according to the calculated gradient.

[0163] Step 8: Learning rate adjustment. Adjust the learning rate according to a predefined strategy. After a specific number of training steps or training rounds (such as every 100 rounds), multiply the learning rate by a decay factor (such as 0.1) to achieve learning rate adjustment.

[0164] Step 9: Save model checkpoints periodically during training. Set the save frequency rounds. After each round, save the model parameters of the generator and discriminator for subsequent training resumption, model performance evaluation, or model inference.

[0165] Step 10: Model application. Set the total number of epochs for training. When the training has completed the predetermined number of epochs and reached the set number, stop training, test and evaluate the quality of the generated images, and apply them to specific applications if they are qualified.

[0166] Example 2

[0167] In one embodiment of the present disclosure, a tunnel inspection image enhancement system based on a conditional generative adversarial network is provided, comprising:

[0168] An image acquisition module, used for acquiring the tunnel image to be enhanced;

[0169] An image enhancement module is used to input the tunnel image to be enhanced into the trained image enhancement model and output the enhanced optimized image;

[0170] The image enhancement model is built based on a conditional generative adversarial network, and the specific training process is as follows:

[0171] Obtain high-quality tunnel images under ideal conditions, and pre-process the high-quality images to obtain low-quality tunnel images;

[0172] The generator of the conditional generative adversarial network extracts feature maps of different channel numbers and sizes from the low-quality tunnel image, and then outputs the corresponding quality-enhanced tunnel inspection image after weighted processing through the spatial attention mechanism, which is recorded as a false high-quality image;

[0173] A low-quality tunnel image and a high-quality tunnel image are spliced together to form a real image pair, and a low-quality tunnel image pair and a fake high-quality image are spliced together to form a fake image pair. The fake image pair and the real image pair are respectively input into the discriminator of the conditional generative adversarial network to obtain a discrimination result; the training is repeated until the requirements are met.

[0174] Example 3

[0175] In one embodiment of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method for enhancing tunnel inspection images based on a conditional generative adversarial network is implemented.

[0176] Example 4

[0177] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, which is used to store computer instructions. When the computer instructions are executed by a processor, the tunnel inspection image enhancement method based on the conditional generative adversarial network is implemented.

[0178] Example 5

[0179] In one embodiment of the present disclosure, an electronic device is provided, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the tunnel inspection image enhancement method based on the conditional generative adversarial network.

[0180] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0182] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Those skilled in the art should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A tunnel inspection image enhancement method based on conditional generative adversarial network, characterized by: include: Acquiring a tunnel image to be enhanced; The tunnel image to be enhanced is input into the trained image enhancement model, and the enhanced optimized image is output; The image enhancement model is built based on a conditional generative adversarial network, and the specific training process is as follows: Obtain high-quality tunnel images under ideal conditions, and pre-process the high-quality images to obtain low-quality tunnel images; The generator of the conditional generative adversarial network extracts feature maps of different channel numbers and sizes from the low-quality tunnel image, and then outputs the corresponding quality-enhanced tunnel inspection image after weighted processing through the spatial attention mechanism, which is recorded as a false high-quality image; A low-quality tunnel image and a high-quality tunnel image are spliced together to form a real image pair, and a low-quality tunnel image pair and a fake high-quality image are spliced together to form a fake image pair. The fake image pair and the real image pair are respectively input into the discriminator of the conditional generative adversarial network to obtain a discrimination result; the training is repeated until the requirements are met.

2. The tunnel inspection image enhancement method based on conditional generative adversarial network according to claim 1, characterized in that: The image enhancement model based on conditional generative adversarial network includes a generator network and a discriminator network. The generator network is a NetG network, including a downsampling module, an upsampling module and a jump connection module, and a spatial attention mechanism is added to the downsampling module and the upsampling module, and the generator loss function is a reconstruction loss function including a GAN adversarial loss function, an L1 loss function and a spatial attention guided reconstruction loss function.

3. The tunnel inspection image enhancement method based on conditional generative adversarial network according to claim 1, characterized in that: The discriminator network is the NetD network, which uses the discriminator of PatchGAN. The discriminator divides the input image into multiple small blocks, discriminates each small block, and finally takes the average of the sum of the judgments of all small blocks as the final judgment result. The discriminator network consists of 3 convolutional layers, 1 normalization layer and 1 spatial attention module. It extracts image features through convolution and normalization processing, converts the input image into a single-channel feature map, and then focuses on the spatial area of the image with key discriminant information through the spatial attention module.

4. The tunnel inspection image enhancement method based on conditional generative adversarial network according to claim 1, characterized in that: The downsampling module of the generator network consists of 1 convolutional layer, 1 activation function layer and 1 normalization layer. The convolutional layer gradually reduces the spatial dimension of the input low-quality tunnel image while increasing the number of feature channels to extract high-level features of the image and capture the semantic and contextual information in the image. The upsampling module consists of 1 deconvolution layer, 1 activation function layer, 1 normalization layer and random dropout layer. The upsampling module gradually restores the spatial dimension of the feature map to the original image size through transposed convolution while reducing the number of feature channels.

5. The tunnel inspection image enhancement method based on conditional generative adversarial network according to claim 4, characterized in that: The input of the downsampling module is a low-quality tunnel image, and the input of the upsampling module is the feature map output by the previous upsampling layer; the input of the skip connection module is the spliced feature map output by the downsampling module, and the two feature maps output by maximum pooling and average pooling are spliced again to form a new feature map; the new feature map is convolved to generate a spatial attention map; the generated spatial attention map and the original input are weighted element by element according to different spatial positions, different weights are assigned to different areas, and feature fusion is performed. The fused feature map is used to output a quality-enhanced tunnel inspection image, that is, a false high-quality image.

6. The tunnel inspection image enhancement method based on conditional generative adversarial network according to claim 1, characterized in that: In the discriminator network, after the spatial attention module is added to the second convolutional layer in the discriminator network, it focuses on the spatial areas in the image that have a great influence on the discrimination results, including key feature areas with abnormal lighting, blur and noise interference; the discriminator loss function uses the least squares loss function and the attention area feature difference loss function.

7. A tunnel inspection image enhancement system based on conditional generative adversarial networks, characterized by: include: An image acquisition module, used for acquiring the tunnel image to be enhanced; An image enhancement module is used to input the tunnel image to be enhanced into the trained image enhancement model and output the enhanced optimized image; The image enhancement model is built based on a conditional generative adversarial network, and the specific training process is as follows: Obtain high-quality tunnel images under ideal conditions, and pre-process the high-quality images to obtain low-quality tunnel images; The generator of the conditional generative adversarial network extracts feature maps of different channel numbers and sizes from the low-quality tunnel image, and then outputs the corresponding quality-enhanced tunnel inspection image after weighted processing through the spatial attention mechanism, which is recorded as a false high-quality image; A low-quality tunnel image and a high-quality tunnel image are spliced together to form a real image pair, and a low-quality tunnel image pair and a fake high-quality image are spliced together to form a fake image pair. The fake image pair and the real image pair are respectively input into the discriminator of the conditional generative adversarial network to obtain a discrimination result; the training is repeated until the requirements are met.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the tunnel inspection image enhancement method based on conditional generative adversarial network according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the tunnel inspection image enhancement method based on the conditional generative adversarial network as described in any one of claims 1 to 6 is implemented.

10. An electronic device, characterized in that: include: A processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the tunnel inspection image enhancement method based on the conditional generative adversarial network as described in any one of claims 1 to 6.

Images