Method for enhancing image in tunnel

Through the image enhancement method of convolutional neural network in the tunnel, the problem of unstable image enhancement effect and low computational efficiency in the prior art is solved, and efficient and stable image processing is achieved, which is suitable for autonomous driving, monitoring and intelligent transportation and other fields. .

CN119090787AInactive Publication Date: 2024-12-06SUZHOU BOXINCHUANG TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410866641.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-01
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When the prior art deals with complex light and mist interference in the tunnel, the image enhancement effect is unstable, the computing efficiency is low, and it is difficult to meet the real-time requirements of autonomous driving and monitoring systems, and the system design is complex and difficult to maintain.

Method used

Using image enhancement method based on convolutional neural network (CNN), image denoising, defog and contrast enhancement are achieved through end-to-end adaptive processing, multi-task learning and parallel computing, and using deep learning and computer vision technology to adapt to multiple Changeable tunnel environment.

Benefits of technology

It realizes efficient and stable image enhancement, improves the real-time and robustness of autonomous driving, safety monitoring and intelligent transportation systems, reduces manual intervention, and is suitable for multiple practical application fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090787A_ABST
    Figure CN119090787A_ABST
Patent Text Reader

Abstract

The invention discloses an image enhancement method in a tunnel, and the method comprises the following steps: obtaining original image data in the tunnel, and obtaining images in the original image data in the tunnel under different illumination conditions, fog and noise interferences; s2, inputting the obtained original image data in the tunnel into a pre-trained convolutional neural network; s3, in the convolutional neural network, using convolution operation and pooling operation to extract local features of the image and reduce the size of a feature map; s4, performing image denoising, defogging and contrast enhancement processing under a multi-task learning framework of sharing network parameters and feature representation; s5, realizing real-time image processing through parallel computing and hardware acceleration by using the optimized convolutional neural network model; s6, processing different types of noise and distortion based on a convolutional neural network model trained on the image data of various environments in the tunnel; and S7, outputting the enhanced high-quality image in the tunnel. According to the in-tunnel image enhancement method based on the convolutional neural network, computer vision and deep learning technologies are fully utilized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of images in tunnels, and in particular to a method for enhancing images in tunnels. Background Art

[0002] Image processing in tunnels is one of the key technologies in the fields of autonomous driving, safety monitoring, intelligent transportation and robotics. In tunnel environments, due to the complex and changeable lighting conditions, fog and various noise interferences, traditional image processing methods often cannot provide sufficient image quality. This not only affects the environmental perception ability of the autonomous driving system, but also reduces the abnormal detection efficiency of the safety monitoring system, and also has an adverse impact on the performance of intelligent transportation and robot navigation.

[0003] Existing image enhancement technologies mainly rely on traditional image processing algorithms, which usually require manual design of specific image processing steps, such as denoising, dehazing, and contrast enhancement. However, this approach has the following core difficulties:

[0004] 1. Complex manual design: Traditional methods require designing different processing steps to deal with different image quality issues, which makes the system design complex and difficult to maintain.

[0005] 2. Poor adaptability: Traditional algorithms are difficult to adapt to the changing environmental conditions in tunnels, especially lighting changes and fog interference, resulting in unstable enhancement effects.

[0006] 3. Lack of real-time performance: The computational efficiency of traditional image processing algorithms is low, and it is difficult to meet the real-time requirements of autonomous driving and monitoring systems.

[0007] 4. Limited robustness: Faced with various complex tunnel environments, traditional algorithms perform poorly in dealing with different types of noise and distortion, resulting in inconsistent output image quality.

[0008] Therefore, how to provide a tunnel image enhancement method based on convolutional neural network (CNN) to achieve efficient and stable image enhancement, meet real-time requirements, and adapt to the changing tunnel environment is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0009] One object of the present invention is to propose a method for image enhancement in a tunnel. The present invention is based on a convolutional neural network (CNN) method for image enhancement in a tunnel, makes full use of computer vision and deep learning technology, and describes in detail the implementation methods of end-to-end adaptive processing, multi-task learning, real-time processing, high robustness and wide applicability.

[0010] A method for enhancing an image in a tunnel according to an embodiment of the present invention comprises the following steps:

[0011] S1. Acquire original image data in the tunnel, wherein the original image data in the tunnel includes images under different lighting conditions, fog and noise interference;

[0012] S2, inputting the original image data in the tunnel into the pre-trained convolutional neural network;

[0013] S3. In convolutional neural networks, convolution operations and pooling operations are used to extract local features of the image and reduce the size of the feature map;

[0014] S4, image denoising, dehazing and contrast enhancement are performed under a multi-task learning framework with shared network parameters and feature representation;

[0015] S5. Use the optimized convolutional neural network model to achieve real-time image processing through parallel computing and hardware acceleration;

[0016] S6, based on a convolutional neural network model trained on image data of various tunnel environments, to handle different types of noise and distortion;

[0017] S7, outputting enhanced high-quality tunnel images.

[0018] Optionally, the S1 specifically includes:

[0019] S11, arranging a plurality of image sensors in the tunnel, wherein the image sensors capture images at different positions in the tunnel;

[0020] S12, collecting raw image data in the tunnel in real time through an image sensor, where the raw image data includes images under different lighting conditions, fog, and noise interference;

[0021] S13, preprocessing the collected image data, the preprocessing steps include image denoising and grayscale processing, denoising method:

[0022]

[0023] in, represents the pixel value of the denoised image, X i+m,j+n Represents the pixel value of the original image, M and N represent the height and width of the denoising window respectively;

[0024] S14. Convert the preprocessed image data into a standard format for input into a convolutional neural network:

[0025] I pre =[I1,I2,I3,…,I k ];

[0026] Among them, Ipre Represents the preprocessed image data set, I k represents the kth preprocessed image;

[0027] S15. The format-converted image data is stored in a data storage unit, waiting to be input into a convolutional neural network for further processing.

[0028] Optionally, the S2 specifically includes:

[0029] S21, read the pre-processed image data from the data storage unit I pre , image data I pre For a three-dimensional tensor:

[0030] I pre =[I1,I2,I3,…,I k ];

[0031] Among them, I pre Represents the preprocessed image data set, I k represents the kth preprocessed image;

[0032] S22, for each image data I k Normalization is performed so that the image pixel values ​​are distributed within the standard range. Normalization:

[0033]

[0034] Among them, I′ i,j,k Represents the normalized image pixel value, I i,j,k represents the original image pixel value, μ represents the mean of the pixel value, and σ represents the standard deviation of the pixel value;

[0035] S23, the normalized image data I' pre Input into the pre-trained convolutional neural network, which includes multiple convolutional layers, pooling layers, fully connected layers and activation functions:

[0036] CNN(I′ pre )=F fc (F relu (F pool (F conv (I′ pre ))));

[0037] Among them, F conv represents the convolution operation, F pool represents the pooling operation, F relu Represents the RELU activation function, F relu Represents the fully connected layer operation;

[0038] S24. In each layer of the convolutional neural network, the convolution operation is used to extract the local features of the image. The convolution operation:

[0039]

[0040] Among them, Y i,j,k Represents the pixel value of the output feature map, I′ i+m,j+n,k represents the pixel value of the input image, K m,n,k Represents the weight of the convolution kernel, M and N represent the height and width of the convolution kernel respectively;

[0041] S25. In the pooling layer of the convolutional neural network, the pooling operation is used to reduce the size of the feature map. The pooling operation:

[0042]

[0043] Among them, P i,j,k Represents the pixel value of the feature map after pooling, and max represents the maximum pooling operation;

[0044] S26. Input the image features after convolution and pooling operations into the fully connected layer for processing to obtain enhanced image feature representation.

[0045] Optionally, the S3 specifically includes:

[0046] S31. By sharing network parameters, a multi-task learning model is constructed, where each task of denoising, dehazing and contrast enhancement corresponds to a specific sub-network;

[0047] S32, input the feature map output by the convolutional neural network into the denoising sub-network, and the denoising sub-network performs denoising on the feature map. Denoising:

[0048] D i,j,k =F conv_denoise (P i,j,k );

[0049] Among them, D i,j,k Represents the pixel value of the feature map after denoising, F conv_denoise represents the convolution operation of the denoising sub-network, P i,j,k Represents the pixel value of the feature map output by the pooling layer of the convolutional neural network;

[0050] S33, the feature map output by the convolutional neural network is input into the defogging network, and the defogging network performs defogging processing on the feature map. Defogging processing:

[0051] H i,jk =F conv_deh aze (P i,j,k );

[0052] Among them, Hi,j,k Represents the pixel value of the feature map after dehazing, F conv_deh aze Represents the convolution operation of the dehazing network;

[0053] S34, inputting the feature map output by the convolutional neural network into the contrast enhancement subnetwork, and the contrast enhancement subnetwork performs contrast enhancement processing on the feature map, and the contrast enhancement processing:

[0054] C i,j,k =F conv_contrast (P i,,j,k );

[0055] Among them, C i,j,k represents the pixel value of the feature map after contrast enhancement, F conv_contrast represents the convolution operation of the contrast enhancement sub-network;

[0056] S35, the denoised feature map D i,j,k , feature map H after dehazing i,j,k And the contrast-enhanced feature map C i,j,k Perform fusion and fusion operations:

[0057] E i,j,k =αD i,j,k +βH i,j,k +γC i,j,k ;

[0058] Among them, E i,j,k represents the pixel value of the fused enhanced feature map, α, β and γ represent the weight coefficients of denoising, dehazing and contrast enhancement features respectively;

[0059] S36. Input the fused enhanced feature map into the final output layer to obtain an enhanced high-quality image.

[0060] Optionally, the S4 specifically includes:

[0061] S41, input the image feature representation output by the fully connected layer into the multi-task learning framework;

[0062] S42. Use the shared convolution layer and pooling layer parameters to denoise the image features and extract the noise features through convolution operation:

[0063]

[0064] Among them, Y noise represents the noise feature map, F fc (i+m,j+n) represents the image features output by the fully connected layer, K noise (m,n) represents the weight of the denoising convolution kernel. The denoising activation function is applied to the noise feature map to obtain the denoised image features:

[0065] A noise =max(0,Y noise );

[0066] S43, using the shared convolution layer and pooling layer parameters, defogging the image features, and extracting fog features through convolution operation:

[0067]

[0068] Among them, Y deh aze represents the dehazing feature map, F fc (i+m,j+n) represents the image features output by the fully connected layer, K deh aze (m,n) represents the weight of the defogging convolution kernel. The defogging activation function is applied to the defogging feature map to obtain the defogging image features:

[0069] A deh aze =max(0,Y deh aze );

[0070] S44. Use the shared convolution layer and pooling layer parameters to perform contrast enhancement on the image features and extract contrast features through convolution operation:

[0071]

[0072] Among them, Y contrast represents the contrast feature map, F fc (i+m,j+n) represents the image features output by the fully connected layer, K contrast (m,n) represents the weight of the contrast convolution kernel. The contrast enhancement activation function is applied to the contrast feature map to obtain the image features after contrast enhancement. The mathematical expression of the activation function is:

[0073] A contrast =max(0,Y contrast )

[0074] S45, combining the image features after denoising, defogging and contrast enhancement to form a final enhanced image feature representation:

[0075] F enh anced =A noise +A deh aze +A contrast ;

[0076] Among them, F enh anced Represents enhanced image feature representation.

[0077] Optionally, the S5 specifically includes:

[0078] S51, the enhanced image feature representation F after processing by the multi-task learning framework enh anced Input into the optimized convolutional neural network model;

[0079] S52. In the convolutional neural network, parallel computing technology is used to accelerate the execution of convolution and pooling operations. The input image feature representation is divided into multiple sub-blocks, and each sub-block performs convolution operations in parallel. The mathematical expression of the convolution operation is:

[0080]

[0081] Among them, Y′ i,j,k Represents the pixel value of the output feature map after parallel calculation, F enh anced (i+m,j+n,k) represents the input image features, K′(m,n,k) represents the weight of the parallel convolution kernel;

[0082] S53, apply activation function to the feature map after parallel calculation:

[0083] A′ i,j,k =max(0,Y′ i,j,k );

[0084] S54, input the activated feature map into the parallel pooling operation, pooling operation:

[0085]

[0086] Among them, P′ i,j,k Represents the pixel value of the feature map after parallel pooling;

[0087] S55. Using parallel computing and hardware acceleration, the hardware acceleration device performs the above parallel computing steps, and significantly improves the real-time processing capability of the convolutional neural network through efficient parallel processing and hardware acceleration.

[0088] Optionally, the S6 specifically includes:

[0089] S61. Represent the enhanced image features after multi-task learning framework F enhanced Input into the optimized convolutional neural network model;

[0090] S62. In the convolutional neural network, parallel computing technology is used to accelerate the execution of convolution and pooling operations. The input image feature representation is divided into multiple sub-blocks, and each sub-block performs convolution operations in parallel. The convolution operation:

[0091]

[0092] Among them, Y′ i,j,k Represents the pixel value of the output feature map after parallel calculation, Fenhanced (i+m,j+n,k) represents the input image features, K′(m,n,k) represents the weight of the parallel convolution kernel;

[0093] S63, applying an activation function to the feature map after parallel calculation, the activation function:

[0094] A′ i,j,k =max(0,Y′ i,j,k );

[0095] S64, input the activated feature map into the parallel pooling operation, pooling operation:

[0096]

[0097] Among them, P′ i,j,k Represents the pixel value of the feature map after parallel pooling;

[0098] S65, outputting the image features after parallel calculation and hardware acceleration to the final output layer to obtain an enhanced image processed in real time.

[0099] Optionally, the S7 specifically includes:

[0100] S71, the image feature F processed by the optimized convolutional neural network enh anced Input to the final output layer;

[0101] S72. In the output layer, the image features are mapped to an enhanced image using a fully connected operation:

[0102] O i,j,k =W·F enh anced (i,j,k)+b;

[0103] Among them, O i,j,k represents the pixel value of the output enhanced image, W represents the weight matrix of the fully connected layer, and b represents the bias term;

[0104] S73, performing post-processing steps on the output enhanced image, including denormalization and format conversion, denormalization processing:

[0105] I final (i,j,k)=O i,j,k σ+μ;

[0106] Among them, I final (i, j, k) represents the enhanced image pixel value after denormalization, σ represents the standard deviation, and μ represents the mean;

[0107] S74, converting the denormalized enhanced image into a standard format suitable for different application fields.

[0108] The beneficial effects of the present invention are:

[0109] The tunnel image enhancement method based on convolutional neural network (CNN) of the present invention makes full use of computer vision and deep learning technology, describes in detail the implementation method of end-to-end adaptive processing, multi-task learning, real-time processing, high robustness and wide applicability, and has the following advantages:

[0110] 1. Adaptive processing: The CNN image enhancement method of the present invention realizes end-to-end adaptive processing from input image to output enhanced image, without the need to manually design a specific image processing algorithm, greatly reducing manual intervention and intermediate steps, and improving the degree of automation of the system.

[0111] 2. Multi-task learning: Through the multi-task learning framework, the present invention can simultaneously process multiple image enhancement tasks such as denoising, dehazing and contrast enhancement, improve the comprehensive processing effect of the model on different tasks, and enable the system to efficiently solve multiple problems within one framework.

[0112] 3. Real-time processing capability: The present invention greatly improves the speed of image processing through parallel computing and hardware acceleration (such as GPU and TPU), and can achieve efficient real-time image enhancement in complex environments, meeting the strict real-time requirements of autonomous driving and monitoring systems.

[0113] 4. High robustness and adaptability: The CNN model is trained on image data of various tunnel environments, and has high robustness and adaptability to complex lighting and environmental conditions. It can handle different types of noise and distortion, ensuring the output of high-quality images under various tunnel conditions.

[0114] 5. Wide applicability: The image enhancement method of the present invention is applicable to multiple practical application fields such as autonomous driving, safety monitoring, intelligent transportation and robotics technology. By improving the quality of images in tunnels, the environmental perception ability of the autonomous driving system is improved, the recognition accuracy of the monitoring system and the efficiency of the intelligent transportation system are enhanced, and the operating performance of the robot in complex environments is improved.

[0115] In summary, the present invention demonstrates remarkable innovation and superiority in terms of adaptive processing, multi-task learning, real-time processing, high robustness and wide applicability, and provides solid technical support for image processing in tunnels. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0117] Figure 1This is an overall flow chart of a tunnel image enhancement method based on a convolutional neural network proposed by the present invention;

[0118] Figure 2 This is a flow chart of image acquisition and preprocessing in the tunnel in the present invention. DETAILED DESCRIPTION

[0119] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0120] refer to Figure 1 , a method for enhancing an image in a tunnel, comprising the following steps:

[0121] S1. Acquire original image data in the tunnel, wherein the original image data in the tunnel includes images under different lighting conditions, fog and noise interference;

[0122] In this implementation, S1 specifically includes:

[0123] S11, arranging a plurality of image sensors in the tunnel, wherein the image sensors capture images at different positions in the tunnel;

[0124] S12, collecting raw image data in the tunnel in real time through an image sensor, where the raw image data includes images under different lighting conditions, fog, and noise interference;

[0125] S13, preprocessing the collected image data, the preprocessing steps include image denoising and grayscale processing, and the denoising method adopts the following formula:

[0126]

[0127] in, represents the pixel value of the denoised image, X i+m,j+n Represents the pixel value of the original image, M and N represent the height and width of the denoising window respectively;

[0128] S14. Convert the preprocessed image data into a standard format for input into a convolutional neural network:

[0129] I pre =[I1,I2,I3,…,I k ];

[0130] Among them, I pre Represents the preprocessed image data set, I k represents the kth preprocessed image;

[0131] S15. The format-converted image data is stored in a data storage unit, waiting to be input into a convolutional neural network for further processing.

[0132] S2, inputting the original image data in the tunnel into the pre-trained convolutional neural network;

[0133] In this implementation, S2 specifically includes:

[0134] S21, read the pre-processed image data from the data storage unit I pre , image data I pre For a three-dimensional tensor:

[0135] I pre =[I1,I2,I3,…,I k ];

[0136] Among them, I pre Represents the preprocessed image data set, I k represents the kth preprocessed image;

[0137] S22, for each image data I k Normalization is performed so that the image pixel values ​​are distributed within the standard range. Normalization:

[0138]

[0139] Among them, I′ i,j,k Represents the normalized image pixel value, I i,j,k represents the original image pixel value, μ represents the mean of the pixel value, and σ represents the standard deviation of the pixel value;

[0140] S23, the normalized image data I' pre Input into the pre-trained convolutional neural network, which includes multiple convolutional layers, pooling layers, fully connected layers and activation functions:

[0141] CNN(I′ pre )=F fc (F relu (F pool (F conv (I′ pre ))));

[0142] Among them, F conv represents the convolution operation, F pool represents the pooling operation, F relu Represents the RELU activation function, F relu Represents the fully connected layer operation;

[0143] S24. In each layer of the convolutional neural network, the convolution operation is used to extract the local features of the image. The convolution operation:

[0144]

[0145] Among them, Y i,j,k Represents the pixel value of the output feature map, I′ i+m,j+n,k represents the pixel value of the input image, K m,n,k Represents the weight of the convolution kernel, M and N represent the height and width of the convolution kernel respectively;

[0146] S25. In the pooling layer of the convolutional neural network, the pooling operation is used to reduce the size of the feature map. The pooling operation:

[0147]

[0148] Among them, P i , j ,k represents the pixel value of the feature map after pooling, and max represents the maximum pooling operation;

[0149] S26. Input the image features after convolution and pooling operations into the fully connected layer for processing to obtain enhanced image feature representation.

[0150] S3. In convolutional neural networks, convolution operations and pooling operations are used to extract local features of the image and reduce the size of the feature map;

[0151] In this implementation, S3 specifically includes:

[0152] S31. By sharing network parameters, a multi-task learning model is constructed, where each task of denoising, dehazing and contrast enhancement corresponds to a specific sub-network;

[0153] S32, input the feature map output by the convolutional neural network into the denoising sub-network, and the denoising sub-network performs denoising on the feature map. Denoising:

[0154] D i,j,k =F conv_denoise (P i,j,k );

[0155] Among them, D i,j,k Represents the pixel value of the feature map after denoising, F conv_decoise represents the convolution operation of the denoising sub-network, P i,j,k Represents the pixel value of the feature map output by the pooling layer of the convolutional neural network;

[0156] S33, the feature map output by the convolutional neural network is input into the defogging sub-network, and the defogging sub-network performs defogging processing on the feature map, defogging processing:

[0157] H i,j,k =F conv_deh aze (P i,j,k );

[0158] Among them, H i,j,k Represents the pixel value of the feature map after dehazing, F conv_dehaze Represents the convolution operation of the dehazing network;

[0159] S34, inputting the feature map output by the convolutional neural network into the contrast enhancement subnetwork, and the contrast enhancement subnetwork performs contrast enhancement processing on the feature map, and the contrast enhancement processing:

[0160] C i,j,k =F conv_contrast (P i,j,k );

[0161] Among them, C i,j,k represents the pixel value of the feature map after contrast enhancement, F conv_contrast represents the convolution operation of the contrast enhancement sub-network;

[0162] S35, the denoised feature map D i,j,k , feature map H after dehazing i,j,k And the contrast-enhanced feature map C i,j,k Perform fusion and fusion operations:

[0163] E i,j,k =αD i,j,k +βH i,j,k +γC i,j,k ;

[0164] Among them, E i,j,k represents the pixel value of the fused enhanced feature map, α, β and γ represent the weight coefficients of denoising, dehazing and contrast enhancement features respectively;

[0165] S36. Input the fused enhanced feature map into the final output layer to obtain an enhanced high-quality image.

[0166] S4, image denoising, dehazing and contrast enhancement are performed under a multi-task learning framework with shared network parameters and feature representation;

[0167] In this implementation, S4 specifically includes:

[0168] S41, input the image feature representation output by the fully connected layer into the multi-task learning framework;

[0169] S42. Use the shared convolution layer and pooling layer parameters to denoise the image features and extract the noise features through convolution operation:

[0170]

[0171] Among them, Y noise represents the noise feature map, F fc (i+m,j+n) represents the image features output by the fully connected layer, K noise (m,n) represents the weight of the denoising convolution kernel. The denoising activation function is applied to the noise feature map to obtain the denoised image features:

[0172] A noise =max(0,Y noise );

[0173] S43, using the shared convolution layer and pooling layer parameters, defogging the image features, and extracting fog features through convolution operation:

[0174]

[0175] Among them, Y deh aze represents the dehazing feature map, F fc (i+m,j+n) represents the image features output by the fully connected layer, K deh aze (m,n) represents the weight of the defogging convolution kernel. The defogging activation function is applied to the defogging feature map to obtain the defogging image features:

[0176] A deh aze =max(0,Y deh aze );

[0177] S44. Use the shared convolution layer and pooling layer parameters to perform contrast enhancement on the image features and extract contrast features through convolution operation:

[0178]

[0179] Among them, Y contrast represents the contrast feature map, F fc (i+m,j+n) represents the image features output by the fully connected layer, K contrast (m,n) represents the weight of the contrast convolution kernel. The contrast enhancement activation function is applied to the contrast feature map to obtain the image features after contrast enhancement. The mathematical expression of the activation function is:

[0180] A contrast =max(0,Y contrast )

[0181] S45, combining the image features after denoising, defogging and contrast enhancement to form a final enhanced image feature representation:

[0182] F enh anced =A noise +A deh aze +Acontrast ;

[0183] Among them, F enh anced Represents enhanced image feature representation.

[0184] S5. Use the optimized convolutional neural network model to achieve real-time image processing through parallel computing and hardware acceleration;

[0185] In this implementation, S5 specifically includes:

[0186] S51, the enhanced image feature representation F after processing by the multi-task learning framework enh anced Input into the optimized convolutional neural network model;

[0187] S52. In the convolutional neural network, parallel computing technology is used to accelerate the execution of convolution and pooling operations. The input image feature representation is divided into multiple sub-blocks, and each sub-block performs convolution operations in parallel. The mathematical expression of the convolution operation is:

[0188]

[0189] Among them, Y′ i,j,k Represents the pixel value of the output feature map after parallel calculation, F enh anced (i+m,j+n,k) represents the input image features, K′(m,n,k) represents the weight of the parallel convolution kernel;

[0190] S53, apply activation function to the feature map after parallel calculation:

[0191] A′ i,j,k =max(0,Y′ i,j,k );

[0192] S54, input the activated feature map into the parallel pooling operation, pooling operation:

[0193]

[0194] Among them, P′ i,j,k Represents the pixel value of the feature map after parallel pooling;

[0195] S55. Using parallel computing and hardware acceleration, the hardware acceleration device performs the above parallel computing steps, and significantly improves the real-time processing capability of the convolutional neural network through efficient parallel processing and hardware acceleration.

[0196] S6, based on a convolutional neural network model trained on image data of various tunnel environments, to handle different types of noise and distortion;

[0197] In this implementation, S6 specifically includes:

[0198] S61. Represent the enhanced image features after multi-task learning framework F enhanced Input into the optimized convolutional neural network model;

[0199] S62. In the convolutional neural network, parallel computing technology is used to accelerate the execution of convolution and pooling operations. The input image feature representation is divided into multiple sub-blocks, and each sub-block performs convolution operations in parallel. The convolution operation:

[0200]

[0201] Among them, Y′ i,j,k Represents the pixel value of the output feature map after parallel calculation, F enhanced (i+m,j+n,k) represents the input image features, K′(m,n,k) represents the weight of the parallel convolution kernel;

[0202] S63, applying an activation function to the feature map after parallel calculation, the activation function:

[0203] A′ i,j,k =max(0,Y′ i,j,k );

[0204] S64, input the activated feature map into the parallel pooling operation, pooling operation:

[0205]

[0206] Among them, P′ i,j,k Represents the pixel value of the feature map after parallel pooling;

[0207] S65, outputting the image features after parallel calculation and hardware acceleration to the final output layer to obtain an enhanced image processed in real time.

[0208] S7, outputting enhanced high-quality tunnel images.

[0209] In this implementation, S7 specifically includes:

[0210] S71, the image feature F processed by the optimized convolutional neural network enh anced Input to the final output layer;

[0211] S72. In the output layer, the image features are mapped to an enhanced image using a fully connected operation:

[0212] j ,j,k =W·F enh anced (i,j,k)+b;

[0213] Among them, O i,j,krepresents the pixel value of the output enhanced image, W represents the weight matrix of the fully connected layer, and b represents the bias term;

[0214] S73, performing post-processing steps on the output enhanced image, including denormalization and format conversion, denormalization processing:

[0215] I final (i,j,k)=O i,j,k σ+μ;

[0216] Among them, I final (i, j, k) represents the enhanced image pixel value after denormalization, σ represents the standard deviation, and μ represents the mean;

[0217] S74, converting the denormalized enhanced image into a standard format suitable for different application fields.

[0218] Embodiment 1:

[0219] From March to June 2023, we conducted a field test of the in-tunnel image enhancement method based on convolutional neural network (CNN) in an important traffic tunnel in the city center. The tunnel is 3 kilometers long and has four monitoring points, each of which is equipped with multiple high-resolution image sensors to capture images in different locations and conditions. These sensors collect image data 24 hours a day, covering a variety of environmental conditions such as day, night, fog and dust.

[0220] During the experiment, we first collected a large amount of raw image data from each monitoring point, totaling 10,000 images, including images under different lighting, fog, and noise interference. The collected image data was preprocessed, including denoising and grayscale processing, and then input into the pre-trained convolutional neural network model. The model outputs enhanced high-quality images through end-to-end adaptive processing, multi-task learning, and real-time processing.

[0221] The following are the specific experimental data and results under various environmental conditions:

[0222] 1. During the day:

[0223] Average PSNR of original image: 21.2dB

[0224] Average SSIM of original image: 0.70

[0225] Enhanced image average PSNR: 33.1dB

[0226] Enhanced image average SSIM: 0.92

[0227] Example: At 12:00 noon on April 15, 2023, the image collected by monitoring point 1 shows that the details deep in the tunnel are blurred due to strong light. After enhancement processing, the details deep in the tunnel become clearly visible.

[0228] 2. Under night conditions:

[0229] Average PSNR of original image: 18.5dB

[0230] Average SSIM of original images: 0.63

[0231] Enhanced image average PSNR: 30.2dB

[0232] Enhanced image average SSIM: 0.89

[0233] Example: At 10 pm on May 3, 2023, the image collected by monitoring point 2 shows that due to insufficient light, the overall brightness of the image is low and there is a lot of noise. After enhancement processing, the image brightness is significantly improved, the noise is significantly reduced, and the outline of the object is clearly visible.

[0234] 3. In foggy conditions:

[0235] Average PSNR of original image: 17.0dB

[0236] Average SSIM of original images: 0.61

[0237] Enhanced image average PSNR: 29.0dB

[0238] Enhanced image average SSIM: 0.86

[0239] Example: At 9:00 am on March 25, 2023, the image collected by monitoring point 3 shows that the image is blurred due to fog interference. After enhancement processing, the fog dissipates and the image becomes clear.

[0240] 4. Under dusty conditions:

[0241] Average PSNR of original image: 19.5dB

[0242] Average SSIM of original images: 0.66

[0243] Enhanced image average PSNR: 31.0dB

[0244] Enhanced image average SSIM: 0.90

[0245] Example: At 2:00 pm on June 10, 2023, the image collected by monitoring point 4 showed that due to the construction in the tunnel, there was a lot of dust interference and the image quality was poor. After enhancement processing, the dust impact was reduced and the image clarity was significantly improved.

[0246] In terms of real-time processing, the optimized convolutional neural network model has an average processing time of 18 milliseconds per frame under GPU acceleration, which fully meets the real-time requirements of autonomous driving and monitoring systems.

[0247] Table 1 Quality indicators of original images and enhanced images under different conditions

[0248]

[0249] Table 2. Test results of autonomous driving system

[0250] project Before enhancement After enhancement Obstacle detection accuracy (%) 85 95 Path planning error rate (%) 12 4

[0251] Table 3 Test results of security monitoring system

[0252] project Before enhancement After enhancement Person identification accuracy (%) 80 93

[0253] Table 4 Intelligent transportation system test results

[0254] project Before enhancement After enhancement Flow monitoring efficiency improvement (%) - 15 Event detection efficiency improvement (%) - 18

[0255] Table 5 Robotics technology test results

[0256] project Before enhancement After enhancement Task completion rate improvement (%) - 25

[0257] Table 6 Processing time

[0258]

[0259]

[0260] In addition, we have also verified the wide applicability of the present invention in practical application scenarios:

[0261] 1. Autonomous driving system: In an autonomous driving test in a downtown tunnel, the enhanced images significantly improved the vehicle's environmental perception capabilities. The accuracy of obstacle detection increased from 85% to 95%, and the error rate of path planning decreased from 12% to 4%.

[0262] 2. Security monitoring system: In the monitoring system inside the tunnel, the enhanced images improve the accuracy of abnormal event detection. In a test on April 20, 2023, the accuracy of personnel identification increased from 80% to 93%.

[0263] 3. Intelligent transportation system: In traffic flow monitoring and traffic incident detection, enhanced images improve system efficiency. In a traffic monitoring in May 2023, the monitoring efficiency increased by 15% and the incident detection efficiency increased by 18%.

[0264] 4. Robotics: In tests of robot navigation and operation in tunnels, enhanced images significantly improved the robot's performance in complex environments. In a test in May 2023, the task completion rate increased by 25%.

[0265] In summary, the tunnel image enhancement method based on convolutional neural network of the present invention has shown significant advantages in adaptive processing, multi-task learning, real-time processing, high robustness and wide applicability, effectively solving the shortcomings of the existing technology in processing complex environments in tunnels and providing a high-quality image enhancement solution.

[0266] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for enhancing an image in a tunnel, characterized in that: The steps include: S1. Acquire original image data in the tunnel, wherein the original image data in the tunnel includes images under different lighting conditions, fog and noise interference; S2, inputting the original image data in the tunnel into the pre-trained convolutional neural network; S3. In convolutional neural networks, convolution operations and pooling operations are used to extract local features of the image and reduce the size of the feature map; S4, image denoising, dehazing and contrast enhancement are performed under a multi-task learning framework with shared network parameters and feature representation; S5. Use the optimized convolutional neural network model to achieve real-time image processing through parallel computing and hardware acceleration; S6, based on a convolutional neural network model trained on image data of various tunnel environments, to handle different types of noise and distortion; S7, outputting enhanced high-quality tunnel images.

2. The method for enhancing images in a tunnel according to claim 1, characterized in that: The S1 specifically includes: S11, arranging a plurality of image sensors in the tunnel, wherein the image sensors capture images at different positions in the tunnel; S12, collecting raw image data in the tunnel in real time through an image sensor, where the raw image data includes images under different lighting conditions, fog, and noise interference; S13, preprocessing the collected image data, the preprocessing steps include image denoising and grayscale processing, denoising method: in, represents the pixel value of the denoised image, X i+m,j+n Represents the pixel value of the original image, M and N represent the height and width of the denoising window respectively; S14. Convert the preprocessed image data into a standard format for input into a convolutional neural network: I pre =[I1,I2,I3,…,I k ]; Among them, I pre Represents the preprocessed image data set, I k represents the kth preprocessed image; S15. The format-converted image data is stored in a data storage unit, waiting to be input into a convolutional neural network for further processing.

3. The method for enhancing images in a tunnel according to claim 1, characterized in that: The S2 specifically includes: S21, read the pre-processed image data from the data storage unit I pre , image data I pre For a three-dimensional tensor: I pre =[I1,I2,I3,…,I k ]; Among them, I pre Represents the preprocessed image data set, I k represents the kth preprocessed image; S22, for each image data I k Normalization is performed so that the image pixel values ​​are distributed within the standard range. Normalization: Among them, I′ i,j,k Represents the normalized image pixel value, I i,j,k represents the original image pixel value, μ represents the mean of the pixel value, and σ represents the standard deviation of the pixel value; S23, the normalized image data I' pre Input into the pre-trained convolutional neural network, which includes multiple convolutional layers, pooling layers, fully connected layers and activation functions: CNN(I′ pre )=F fc (F relu (F pool (F conv (I′ pre )))); Among them, F conv represents the convolution operation, F pool represents the pooling operation, F relu Represents the RELU activation function, F relu Represents the fully connected layer operation; S24. In each layer of the convolutional neural network, the convolution operation is used to extract the local features of the image. The convolution operation: Among them, Y i,j,k Represents the pixel value of the output feature map, I ′ i+m,j+n,k represents the pixel value of the input image, K m,n,k Represents the weight of the convolution kernel, M and N represent the height and width of the convolution kernel respectively; S25. In the pooling layer of the convolutional neural network, the pooling operation is used to reduce the size of the feature map. The pooling operation: Among them, Pi ,j,k Represents the pixel value of the feature map after pooling, and max represents the maximum pooling operation; S26. Input the image features after convolution and pooling operations into the fully connected layer for processing to obtain enhanced image feature representation.

4. The method for enhancing images in a tunnel according to claim 1, characterized in that: The S3 specifically includes: S31. By sharing network parameters, a multi-task learning model is constructed, where each task of denoising, dehazing and contrast enhancement corresponds to a specific sub-network; S32, input the feature map output by the convolutional neural network into the denoising sub-network, and the denoising sub-network performs denoising on the feature map. Denoising: From ,j,k =Fconv _denoise (Pi ,j,k ); Among them, Di ,j,k Indicates the pixel value of the feature map after denoising, Fconv _denoise represents the convolution operation of the denoising subnetwork, Pi ,j,k Represents the pixel value of the feature map output by the pooling layer of the convolutional neural network; S33, the feature map output by the convolutional neural network is input into the defogging network, and the defogging network performs defogging processing on the feature map. Defogging processing: Hello ,j,k =Fconv _deh aze (Pi ,j,k ); Among them, Hi ,j,k Indicates the pixel value of the feature map after defogging, Fconv _deh aze Represents the convolution operation of the dehazing network; S34, inputting the feature map output by the convolutional neural network into the contrast enhancement subnetwork, and the contrast enhancement subnetwork performs contrast enhancement processing on the feature map, and the contrast enhancement processing: You ,j,k =Fconv _contrast (Pi ,j,k ); Among them, Ci ,j,k Indicates the pixel value of the feature map after contrast enhancement, Fconv _contrast represents the convolution operation of the contrast enhancement sub-network; S35, the denoised feature map Di ,j,k , Feature map after defogging Hi ,j,k And the contrast-enhanced feature map Ci ,j,k Perform fusion and fusion operations: No ,j,k =αDi ,j,k +βHi ,j,k +γCi ,j,k ; Among them, Ei ,j,k represents the pixel value of the fused enhanced feature map, α, β and γ represent the weight coefficients of denoising, dehazing and contrast enhancement features respectively; S36. Input the fused enhanced feature map into the final output layer to obtain an enhanced high-quality image.

5. The method for enhancing images in a tunnel according to claim 1, characterized in that: The S4 specifically includes: S41, input the image feature representation output by the fully connected layer into the multi-task learning framework; S42. Use the shared convolution layer and pooling layer parameters to denoise the image features and extract the noise features through convolution operation: Among them, Ynoise represents the noise feature map, Ffc(i+m,j+n) represents the image features output by the fully connected layer, Knoise(m,n) represents the weight of the denoising convolution kernel, and the denoising activation function is applied to the noise feature map to obtain the denoised image features: Anoise = max(0, Ynoise); S43, using the shared convolution layer and pooling layer parameters, defogging the image features, and extracting fog features through convolution operation: Among them, Ydehaze represents the dehazing feature map, Ffc(i+m,j+n) represents the image features output by the fully connected layer, Kdehaze(m,n) represents the weight of the dehazing convolution kernel, and the dehazing activation function is applied to the dehazing feature map to obtain the dehazed image features: Adehaze=max(0,Ydehaze); S44. Use the shared convolution layer and pooling layer parameters to perform contrast enhancement on the image features and extract contrast features through convolution operation: Among them, Ycontrast represents the contrast feature map, Ffc(i+m,j+n) represents the image features output by the fully connected layer, Kcontrast(m,n) represents the weight of the contrast convolution kernel, and the contrast enhancement activation function is applied to the contrast feature map to obtain the image features after contrast enhancement. The mathematical expression of the activation function is: Acontrast=max(0,Ycontrast) S45, combining the image features after denoising, defogging and contrast enhancement to form a final enhanced image feature representation: Fenh anced=Anoise+Adeh aze+Acontrast; Among them, Fenh amced means enhanced image feature representation.

6. The method for enhancing images in a tunnel according to claim 1, characterized in that: The S5 specifically includes: S51, inputting the enhanced image feature representation Fenh anced processed by the multi-task learning framework into the optimized convolutional neural network model; S52. In the convolutional neural network, parallel computing technology is used to accelerate the execution of convolution and pooling operations. The input image feature representation is divided into multiple sub-blocks, and each sub-block performs convolution operations in parallel. The mathematical expression of the convolution operation is: Among them, Y ′ i,j,k Represents the pixel value of the output feature map after parallel calculation, F enh anced (i+m,j+n,k) represents the input image features, K′(m,n,k) represents the weight of the parallel convolution kernel; S53, apply activation function to the feature map after parallel calculation: TO ′ i,j,k =max(0,Y ′ i,j,k ); S54, input the activated feature map into the parallel pooling operation, pooling operation: Among them, P ′ i,j,k Represents the pixel value of the feature map after parallel pooling; S55. Using parallel computing and hardware acceleration, the hardware acceleration device performs the above parallel computing steps, and significantly improves the real-time processing capability of the convolutional neural network through efficient parallel processing and hardware acceleration.

7. The method for enhancing images in a tunnel according to claim 1, characterized in that: The S6 specifically includes: S61. Represent the enhanced image features after multi-task learning framework F enhanced Input into the optimized convolutional neural network model; S62. In the convolutional neural network, parallel computing technology is used to accelerate the execution of convolution and pooling operations. The input image feature representation is divided into multiple sub-blocks, and each sub-block performs convolution operations in parallel. The convolution operation: Among them, Y ′ i,j,k Represents the pixel value of the output feature map after parallel calculation, F enhanced (i+m,j+n,k) represents the input image features, K ′ (m,n,k) represents the weight of the parallel convolution kernel; S63, applying an activation function to the feature map after parallel calculation, the activation function: TO ′ i,j,k =max(0,Y ′ i,j,k ); S64, input the activated feature map into the parallel pooling operation, pooling operation: Among them, P ′ i,j,k Represents the pixel value of the feature map after parallel pooling; S65, outputting the image features after parallel calculation and hardware acceleration to the final output layer to obtain an enhanced image processed in real time.

8. The method for enhancing images in a tunnel according to claim 1, characterized in that: The S7 specifically includes: S71, inputting the image feature Fenh anced processed by the optimized convolutional neural network into the final output layer; S72. In the output layer, the image features are mapped to an enhanced image using a fully connected operation: Oi ,j,k =W·Fenh anced(i,j,k)+b; Among them, Oi ,j,k represents the pixel value of the output enhanced image, W represents the weight matrix of the fully connected layer, and b represents the bias term; S73, performing post-processing steps on the output enhanced image, including denormalization and format conversion, denormalization processing: Iffinal(i,j,k)=Oi , j,k·s+μ; Where Ifinal(i,j,k) represents the enhanced image pixel value after denormalization, σ represents the standard deviation, and μ represents the mean; S74, converting the denormalized enhanced image into a standard format suitable for different application fields.