Intelligent defogging processing method and system for pipeline CCTV detection image

Abstract

The invention discloses an intelligent defogging processing method and system for a pipeline CCTV detection image, and aims to solve the problem that the pipeline CCTV detection image is atomized and blurred due to an internal complex and humid environment so as to improve the accuracy of pipeline defect recognition. The invention provides a defogging algorithm based on a deep convolutional neural network. The algorithm comprises a haze measurement and calculation network and a generation module. The haze measurement network adopts a lightweight and efficient convolutional neural network structure, the calculation amount is reduced by using depth separable convolution, a position normalization module is introduced to enhance the model expression ability and realize pixel-level haze measurement, and a generation module combines a haze measurement result with a foggy image based on an improved atmospheric scattering model to realize pixel-level haze measurement. According to the method, fog fuzzy interference in the pipeline CCTV detection image can be removed in real time, the image definition is remarkably improved, background personnel can accurately recognize defects such as cracks and corrosion in the pipeline, and the efficiency and accuracy of pipeline detection and maintenance are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pipeline detection and maintenance, and in particular to an intelligent defogging processing method and system for pipeline CCTV detection images. Background Art

[0002] In the field of pipeline inspection and maintenance, CCTV (Closed Circuit Television) inspection technology is widely used due to its intuitive and efficient features. CCTV inspection technology uses a camera to shoot the inside of the pipeline and transmits the captured images to the background for inspection personnel to observe and analyze. However, in actual applications, pipeline CCTV inspection images are often interfered by various factors, resulting in poor image quality, among which fogging and blurring is a particularly prominent problem.

[0003] The main cause of fogging and blurring is the complex environmental conditions inside the pipeline. Since different pipelines may have a certain temperature difference, and the interior of the pipeline is mostly in a humid environment with a lot of pollutants such as water vapor and oil, the lens surface is easily blocked when the camera moves quickly inside the pipeline, making the captured image blurry. In addition, the lighting conditions inside the pipeline may also affect the image quality. For example, insufficient light or uneven light distribution will further aggravate the degree of fogging and blurring of the image.

[0004] In order to solve the fogging and blurring problem of pipeline CCTV inspection images, some image defogging methods have appeared in the prior art. For example, CN115482165A discloses an image defogging method based on dark channel prior, which belongs to the field of image processing technology, including: obtaining a dark channel image and a bright channel image of the haze weather image according to a haze weather image; selecting pixel values ​​in the bright channel image to obtain an atmospheric light value A; obtaining a transmittance image according to the bright channel image and the atmospheric light value A; obtaining a defogging restored image according to the transmittance image and the atmospheric light value A; using the bright channel pixel value to distinguish whether the pixel point meets the dark channel prior, and respectively obtaining the transmittance of the image area that meets the dark channel prior and does not meet the dark channel prior, thereby avoiding the poor image defogging effect caused by inaccurate transmittance estimation, and making the image defogging containing highlight areas and sky areas more realistic and clear; the method estimates the atmospheric light by calculating the dark channel map of the image, and then uses the atmospheric scattering model to defog the image, however, the method has some problems when processing pipeline CCTV detection images; on the one hand, the internal environment of the pipeline is complex, and the applicability of the dark channel prior is limited; on the other hand, the method has a large amount of calculation and is difficult to meet the real-time requirements.

[0005] In addition, some image defogging methods based on deep learning have been proposed. These methods learn the mapping relationship between foggy images and defogging images by training neural network models, and defog the images based on the learning results. However, these methods usually require a large amount of training data and a relatively complex model structure, resulting in high computational costs and poor real-time performance. When processing pipeline CCTV detection images, it is often difficult to achieve ideal defogging effects due to the particularity and complexity of the images.

[0006] In summary, the existing technology has many shortcomings in dealing with the fogging and blurring problem of pipeline CCTV inspection images, such as limited applicability, large amount of calculation, poor real-time performance, etc. Therefore, it is necessary to propose a new pipeline CCTV inspection image defogging and blurring method to solve the problems existing in the existing technology and improve the efficiency of pipeline inspection and maintenance. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide an intelligent defogging processing method and system for pipeline CCTV inspection images, so as to solve the problem that pipeline CCTV inspection images are difficult to identify defects due to fogging and blurring, and overcome the limitations of insufficient image clarity and low defect recognition accuracy in the prior art. Specifically, during pipeline inspection and maintenance, CCTV technology is widely used to observe the internal conditions of pipelines. However, due to the complex internal environment of pipelines, there are often interference factors such as water vapor and oil pollution, which causes the inspection image to be fogged and blurred, seriously affecting the back-end personnel's accurate identification of defects such as cracks and corrosion in the pipeline.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is: a pipeline CCTV detection image intelligent defogging processing method, comprising the following steps: Step 1: Input the pipeline CCTV detection image with fog blur; Step 2: Use the haze measurement network to process the input foggy image to measure the degree of image haze; Step 3: According to the measured image fogging level, the defogging image is generated through the generation module; Step 4: Output the processed dehazed image.

[0009] In a preferred solution, the haze measurement network in Step 2 includes at least two depth-separable convolutional layers for extracting shallow features of the image, and a position normalization module is introduced to adjust the feature distribution.

[0010] In a preferred solution, the depth-separable convolution layer in the haze measurement network includes two parts: depth-wise convolution and point-wise convolution, wherein the depth-wise convolution is used to perform convolution operations on each input channel independently, and the point-by-point convolution is used to transform the dimension of the output features of the depth-wise convolution and integrate cross-channel information.

[0011] In a preferred solution, the position normalization module normalizes the features in the channel dimension, normalizes the feature values ​​to a finer and more compact data range, and introduces learnable scaling factors and bias terms to dynamically adjust the feature distribution and enhance the expressiveness of the model.

[0012] In a preferred solution, the position normalization module subtracts the mean from the input of the activation function and divides it by the standard deviation, so that the mean of the feature map in each channel is 0 and the standard deviation is 1. The position normalization is expressed as: (1) In the formula, For input features at position The value at For location The mean value in the channel dimension, For location The variance in the channel dimension, To prevent decimals with zero denominators, is the scaling factor, is the bias term; The activation function adopts the ReLU activation function, and its expression is: (2) In the formula, is the input value, is the output value.

[0013] In a preferred solution, the generation module in Step 3 associates the image fog level with the foggy image based on an improved atmospheric scattering model, and establishes a mapping relationship between the fog level input and the defogged image at the pixel level.

[0014] In a preferred solution, the generation module generates a defogged image by the following formula, which is improved from the atmospheric scattering model, with the standard atmospheric scattering model as a reference, and is expressed as: (3) In the formula, Indicates the pixel position The observed image at Indicates the pixel position Clear images of Indicates pixel position The transmittance at , which describes the fraction of light that remains after passing through the atmosphere, represents the global atmospheric light, is the proportion of light scattered; Taking the input fogged image as input and the defogged image as output, the atmospheric scattering model is transformed and expressed as: (4) set up , Indicates a bias with a set value of 1.

[0015] The preferred solution also includes using a mean square error loss function during the training process of the haze measurement network and the generation module. To guide the training optimization of the model, the loss function It is expressed as: (5) In the formula, Indicates the number of samples. For the image dehazing task, it indicates the number of all pixels. For multi-channel color images, Value is ; is the real image pixel value; The pixel value predicted by the model.

[0016] In a preferred solution, the method further comprises a step of post-processing the defogged image, wherein the post-processing step comprises color correction and detail enhancement.

[0017] In a preferred solution, the output feature map of the haze measurement network is consistent with the foggy image in size and number of channels, so that the output haze measurement value can accurately reflect the degree of haze of the image at the pixel level.

[0018] A system for implementing any one of the above pipeline CCTV detection image intelligent defogging processing methods comprises: An image input module for receiving pipeline CCTV inspection images with fog blur; The haze measurement network module is used to process the haze-blurred image input by the image input module to measure the image haze degree level; A generation module, used for generating a defogged image according to the image fogging degree level calculated by the fogging measurement network module; The image output module is used to output the dehazed image processed by the generation module.

[0019] In a preferred embodiment, the feature extraction module of the haze measurement network module includes two depth-separable convolutional layers and a position normalization module.

[0020] In a preferred solution, the generation module is constructed based on an improved atmospheric scattering model, and includes an image reconstruction module, which is used to generate a defogged image using an adaptive fusion strategy based on the haze measurement result and the foggy image.

[0021] The pipeline CCTV detection image intelligent defogging processing method and system provided by the present invention have the following beneficial effects: 1. The present invention aims at the problem of image fogging and blurring caused by the complex and humid environment inside the pipeline in CCTV pipeline technology. This problem seriously affects the backstage personnel to accurately identify key information such as cracks, corrosion, sediments, deformation and root invasion in the pipeline. Although the traditional image defogging method has a certain effect, it is insufficient in restoring details, and has poor robustness and generalization. Although the method based on deep learning has advantages in theory, it is often difficult to meet the actual task requirements in terms of real-time defogging effect. The intelligent defogging processing method of the present invention effectively solves these problems; 2. The present invention proposes a lightweight and efficient convolutional neural network for measuring the degree of fogging of foggy images. The network uses a depthwise separable convolution to reduce the amount of model calculation. At the same time, a position normalization module is introduced into the network to enhance the expression ability of the model so as to more accurately identify and process foggy images. 3. The generation module of the present invention is constructed based on the improved atmospheric scattering model. During training, the module dynamically adjusts the fogging degree parameter according to the mapping relationship between the fog output feature map generated by the established fog measurement network and the defogging image at the pixel level; then, when predicting the output, the foggy image is fused according to the predicted value of the fog measurement network to generate the defogging output image; the introduction of the atmospheric scattering model enables the generation module to more accurately simulate and restore the real scene of the image, thereby effectively removing the fog blur in the image; 4. In the model training process, the present invention adopts the mean square error loss function ( ) to guide the optimization of the model, It is a commonly used loss function that measures the difference between the model output and the true value. value, the model can gradually learn how to remove fog blur in the image more accurately; 5. The technical solution of the present invention can remove fog and blur interference in pipeline CCTV detection images in real time, significantly improving the clarity of the image. This improvement enables back-end personnel to more accurately identify various problems in the pipeline, providing strong support for pipeline maintenance and management; 6. By adopting the depth-separable convolution and position normalization modules, the network model of the present invention has been effectively controlled in terms of computational complexity. This improvement not only improves the real-time performance of the algorithm, but also enables the model to complete the defogging task in a shorter time, thereby meeting the real-time requirements in practical applications; 7. The present invention introduces a position normalization module and a ReLU activation function, which further enhances the model's expressiveness and defogging effect. The ReLU activation function can increase the model's nonlinear characteristics and improve the model's generalization ability; while the position normalization module can more accurately capture local features in the image, thereby enhancing the model's ability to process fogged images; 8. The processing method of the present invention can process pipeline CCTV detection images in real time and improve image clarity, thereby helping backstage personnel to accurately identify defects such as cracks and corrosion in the pipeline. This improvement not only improves detection efficiency, but also provides more accurate data support for pipeline maintenance and care; 9. The defogging algorithm of the present invention has the advantages of being lightweight, automated and highly real-time, and is suitable for real-time application scenarios of pipeline CCTV detection; 10. The present invention has significant advantages in solving the image fogging and blurring problem in CCTV pipeline technology. It builds the overall architecture of the defogging algorithm by designing a lightweight and efficient convolutional neural network and introducing an improved atmospheric scattering model. At the same time, the mean square error loss function is used to guide the optimization of parameters in the network model, which is relatively friendly to the practical application of the algorithm. Moreover, the method has a significant effect on improving the performance of pipeline CCTV detection tasks in complex downstream environments. The method shows a certain robustness. This invention has broad application prospects and important practical value in the field of pipeline detection and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described below in conjunction with the accompanying drawings and implementation examples: Figure 1 It is a schematic diagram of the overall process of the present invention; Figure 2 It is a schematic diagram of the haze measurement network of the present invention; Figure 3 Schematic diagram of depth-wise separable convolution of the present invention; Figure 4 It is a schematic diagram of position normalization of the present invention; Figure 5 Visualize the defogging effect of the present invention Figure 1 ; Figure 6 Visualize the defogging effect of the present invention Figure 2 ; Figure 7 Visualize the defogging effect of the present invention Figure 3 ; Figure 8 Visualize the defogging effect of the present invention Figure 4 . DETAILED DESCRIPTION

[0023] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments: Example 1 like Figures 1 to 8 As shown, the pipeline CCTV detection image intelligent defogging processing method includes the following steps: Step 1: Input the pipeline CCTV detection image with fog blur; Step 2: Use the haze measurement network to process the input foggy image to measure the degree of image haze; Step 3: According to the measured image fogging level, the defogging image is generated through the generation module; Step 4: Output the processed dehazed image.

[0024] In this embodiment, the haze measurement network in Step 2 includes at least two depth-separable convolutional layers for extracting shallow features of the image, and a position normalization module is introduced to adjust feature distribution.

[0025] Furthermore, the depth-separable convolution layer in the haze measurement network includes two parts: depth-wise convolution and point-wise convolution. The depth-wise convolution is used to perform convolution operations on each input at the channel level independently, and the point-by-point convolution is used to transform the dimension of the output features of the depth-wise convolution and integrate cross-channel information.

[0026] Furthermore, the position normalization module normalizes the features in the channel dimension, normalizes the feature values ​​to a finer and more compact data range, and introduces learnable scaling factors and bias terms to dynamically adjust the feature distribution and enhance the expressiveness of the model.

[0027] Furthermore, the position normalization module subtracts the mean from the input of the activation function and divides it by the standard deviation, so that the mean of the feature map in each channel is 0 and the standard deviation is 1. The position normalization is expressed as: (1) In the formula, For input features at position The value at For location The mean value in the channel dimension, For location The variance in the channel dimension, To prevent decimals with zero denominators, it is usually set to , is a scaling factor that allows the model to scale the normalized features, thereby changing their dynamic range. It is a bias term that allows the model to add an offset to the normalized feature, thereby changing its center position; The activation function uses the ReLU activation function, which has a small amount of calculation and can better alleviate the problem of gradient disappearance. Its expression is: (2) In the formula, is the input value, is the output value.

[0028] Furthermore, the generation module in Step 3 associates the image fogging level with the foggy image based on the improved atmospheric scattering model, and establishes a mapping relationship between the fogging level input and the defogged image at the pixel level.

[0029] Furthermore, the generation module generates the defogged image by the following formula, which is improved from the atmospheric scattering model, with the standard atmospheric scattering model as a reference, and is expressed as: (3) In the formula, Indicates the pixel position The observed image at the location can be understood as a foggy image in this implementation. Indicates the pixel position The clear image at the location can be understood as the defogging image in this implementation. Indicates pixel position The transmittance at , which describes the fraction of light that remains after passing through the atmosphere, Represents the global atmospheric light, that is, the color and intensity of the background light, is the proportion of light scattered; Taking the input fogged image as input and the defogged image as output, the atmospheric scattering model is transformed and expressed as: (4) set up , Indicates a bias with a set value of 1.

[0030] Furthermore, the mean square error loss function is used in the training process of the haze measurement network and the generation module. (Mean Squared Error) is used to guide the training optimization of the model, where the loss function It is expressed as: (5) In the formula, Indicates the number of samples. For the image dehazing task, it indicates the number of all pixels. For multi-channel color images, Value is ; is the real image pixel value (the pixel value in the annotated image or the dehazed image); The pixel value predicted by the model (the pixel value of the dehazed reconstructed output image).

[0031] Furthermore, the method further includes a step of post-processing the defogging image, wherein the post-processing step includes color correction and detail enhancement.

[0032] Furthermore, the output feature map of the haze measurement network is consistent with the foggy image in size and number of channels, so that the output haze measurement value can accurately reflect the degree of haze of the image at the pixel level.

[0033] Example 2 In another preferred embodiment, based on the above embodiment 1, a system for implementing any one of the above pipeline CCTV detection image intelligent defogging processing methods includes: An image input module for receiving pipeline CCTV inspection images with fog blur; The haze measurement network module is used to process the haze-blurred image input by the image input module to measure the image haze degree level; A generation module, used for generating a defogged image according to the image fogging degree level calculated by the fogging measurement network module; The image output module is used to output the dehazed image processed by the generation module.

[0034] In this embodiment, the feature extraction module of the haze measurement network module includes two depth-separable convolutional layers and a position normalization module.

[0035] Furthermore, the generation module is constructed based on an improved atmospheric scattering model, and includes an image reconstruction module, which is used to generate a defogged image by adopting an adaptive fusion strategy according to the haze measurement result and the foggy image.

[0036] Example 3 In another preferred embodiment, based on the above-mentioned embodiments 1 and 2, this embodiment elaborates on the specific implementation of the present invention: The present invention provides a defogging algorithm based on a deep convolutional neural network to remove fog and blur interference in pipeline CCTV detection images. The overall algorithm diagram is shown in the attached Figure 1 As shown in the figure, the left side is the input foggy pipe image, and the right side is the output image processed by the dehazing algorithm. The algorithm consists of two parts, a haze measurement network and a generation module. The haze measurement network learns the image haze level from the haze-blurred image, while the generation module associates the image haze level with the dehazed image, and directly establishes a mapping between the haze level input and the dehazed image at the pixel level.

[0037] The defogging algorithm of the present invention mainly includes three parts: haze measurement network, generation module and loss function. The specific contents are as follows: 1. Fog measurement network: This paper proposes a lightweight and efficient convolutional neural network for measuring the degree of fog in foggy images. The network model is shown in the attached Figure 2 As shown in the figure, the input is first a foggy image, which is then subjected to two depthwise separable convolutions to extract shallow features. Compared with conventional convolutions, deep convolutions can retain the representational capabilities of the network model to a large extent while reducing the amount of computation of the network model, thereby speeding up the real-time performance of the network model. Specifically, it is divided into two steps: a deep convolution and a point-by-point convolution.

[0038] For conventional convolution, the amount of computation required for one convolution is expressed as: (6) In the formula, 、 are the length and width of the feature map respectively, is the number of channels of the input feature, is the number of channels of the output feature, is the size of the convolution kernel.

[0039] When using depth-wise separable convolution, since it is divided into two parts, the amount of computation for depth-wise convolution is expressed as: (7); For point-by-point convolution, its computational cost is expressed as: (8); Therefore, after summing up, it is found that the computational cost of depth-separable convolution is twice that of conventional convolution. times (much less than 1).

[0040] In addition, the first two convolution operations in the haze measurement network proposed in this embodiment both use depthwise separable convolutions, so the overall amount of computation is conventional convolution. times.

[0041] In order to improve the expression ability of the model and enhance the defogging effect of the model at the pixel level, inspired by the idea of ​​position normalization, this embodiment introduces a position normalization module in the haze measurement network. Specifically, the features are normalized in the channel dimension. By not changing the overall distribution of the features, the values ​​are normalized to a more refined and compact data range, that is, the input of the activation function is subtracted from the mean and divided by the standard deviation, so that the mean of the feature map in each channel is 0 and the standard deviation is 1. Position normalization can be expressed as: (1).

[0042] Adding learnable parameters can dynamically adjust the feature distribution according to task requirements, thereby enhancing the expressive power of the model. Among them, all activation functions in the present invention use ReLU activation function, which has a small amount of calculation and can better alleviate the problem of gradient disappearance. Its expression is: (2).

[0043] 2. Generation module: The generation module is improved from the atmospheric scattering model. Specifically, it takes the standard atmospheric scattering model as a reference: (3).

[0044] Taking the input fogged image as input and the defogged image as output, the atmospheric scattering model is transformed and expressed as: (4).

[0045] 3. Loss function: The loss function uses the mean square error loss function To guide the training optimization of the model: (5).

[0046] Example 4 In another preferred embodiment, based on the above embodiments 1, 2, and 3, this embodiment further illustrates the specific implementation of the present invention: For CCTV inspection tasks in pipelines, in order to alleviate the misjudgment and missed judgment of pipeline defects caused by fog and blur interference of the inspection image, the present invention provides a lightweight automatic real-time defogging algorithm based on deep learning. The algorithm includes a haze measurement module composed of a convolutional neural network for obtaining the haze level at the pixel level and a generation module improved by an atmospheric scattering model for generating a defogged image. The specific implementation steps are as follows: First, the input image is passed through two depth-wise separable convolutions (see Appendix Figure 3 ) extracts shallow features of the image, mainly including the color, texture, brightness, edge and other features of the image; it is divided into two steps: first, the image uses 3 convolution kernels to extract features, and then the extracted features are convolved point by point. The point-by-point convolution uses a 1×1 convolution with the same number of channels as the input; specifically: assuming the input image size is First, a deep convolution is performed on the image with a convolution kernel of size 3×3 and number 1. The feature map obtained by the convolution is then passed through a convolution kernel of size 1×1×3 (3 represents the number of channels of the convolution kernel) and number 3. The convolution step size of both convolutions is 1, and there is no padding. The output size after a deep separable convolution is ; The calculation method of the output feature map size after deep convolution is expressed as: (9) (10) In the formula, and are the output feature length and width of the depth convolution respectively, Indicates the number of filling times, represents the step size of the convolution, is the size of the convolution kernel, and the number of output channels of the depthwise convolution is equal to the number of input channels.

[0047] The feature map size of the point-by-point convolution output is ,in It is determined by the number of point-by-point convolution kernels. Considering the computational overhead, the number of point-by-point convolution kernels is always 3 in this embodiment.

[0048] After the first round of depth-wise separable convolution, the first round of feature maps is obtained, followed by the second round of depth-wise separable convolution. Unlike the first round of depth-wise separable convolution, the second round of depth-wise separable convolution performs zero padding around the feature maps to ensure that the feature maps output by the two convolutions remain consistent in size in the subsequent feature map merging. In addition, the idea of ​​residual connection is borrowed to ensure the diversity of features, and the outputs of the two previous and subsequent depth-wise separable convolutions are merged as the input of the next round of convolution feature extraction operations. Therefore, the number of feature channels after merging becomes 6; the feature outputs after two depth-wise separable convolutions will use position normalization to adjust the distribution of features, and the mean and variance of the features will be calculated in the channel dimension.

[0049] After that, a regular convolution is used to readjust the number of channels to 3 to reduce the subsequent computational overhead. The convolution kernel size of this regular convolution is 5×5, the number of convolution kernels is 3, the step size is 1, and two zero paddings are used. The purpose of two zero paddings is also to ensure that the feature maps are consistent in size when the subsequent features are merged.

[0050] After a 5×5 regular convolution, the output features are merged again with the feature map of the second depthwise separable convolution. Then a 7×7 regular convolution is used to extract deep features. The number of convolution kernels remains at 3, the step size is 1, and three zero paddings are performed. The output feature map is again merged with the outputs of the first two depthwise separable convolutions and the output features of the 5×5 regular convolution to form a 12-channel feature layer. Finally, a standard convolution kernel of 3×3×12 is used to directly obtain the haze prediction value at the pixel level. The number of convolution kernels is 3, the step size is 1, and one zero padding is performed. The final output feature map is consistent with the foggy image in size and number of channels. Before the first and second feature merging of the haze measurement network, the position parameters learned in the aforementioned position normalization step will be used. , As well as the mean and variance to dynamically adjust the distribution of the feature map.

[0051] The feature map output by the haze measurement network obtains a haze measurement value at the pixel level, which is then combined with the improved atmospheric scattering model to guide the generation of dehazed images.

[0052] Model training and results: The model was trained on the public defogging dataset RESIDE. The hardware and software configuration used the Ubuntu operating system, Pytorch version 1.12.0 and Python 3.7. The model training and testing were completed with the help of NVIDIA GeForce RTX4090 graphics card. The actual defogging effect is shown in the attached figure. Figure 5 , Attachment Figure 6 , Attachment Figure 7 and attached Figure 8 After the pipeline CCTV detection image is processed by the defogging algorithm of the present invention, the fogging and blurring phenomenon is effectively removed, and the image clarity is significantly improved; the accuracy rate of backstage technicians in identifying problems such as cracks, corrosion, sediments, deformation, root invasion, etc. inside the pipeline is improved, and the risk of misjudgment and missed judgment is reduced.

[0053] Example 5 In another preferred embodiment, based on the above embodiments 1, 2, 3, and 4, this embodiment further describes the specific implementation of the present invention: 1. Algorithm composition and workflow The defogging algorithm provided by the present invention mainly includes three parts: haze measurement network, generation module and loss function. The overall algorithm workflow is as follows: 1. Input image preprocessing: The pipeline CCTV detection foggy image is used as input, and the image is fogged and blurred due to the complex and humid environment inside the pipeline.

[0054] 2. Fog measurement network: The input image first passes through two depth-wise separable convolutional layers to extract shallow features of the image, such as color, texture, brightness, edges, etc. Depthwise separable convolution consists of depthwise convolution and pointwise convolution. Depthwise convolution performs convolution operation on each input channel independently, while pointwise convolution fuses the output features of depthwise convolution between channels through 1×1 convolution kernel. In the haze measurement network, the first two convolutional layers use depthwise separable convolution to reduce the amount of calculation and improve real-time performance; Introduce a position normalization module to normalize the feature map output by the convolution to enhance the expressiveness of the model; After a series of convolution and normalization operations, the haze measurement network outputs a haze prediction value map that is consistent with the input image size and number of channels, which reflects the degree of haze of the image at the pixel level.

[0055] 3. Generate module: Taking the input fogged image and the output of the haze measurement network as input, the generation module generates a dehazed image based on an improved atmospheric scattering model; The improved atmospheric scattering model converts the foggy image into a clear image by adjusting parameters such as transmittance and global atmospheric light; In the generation module, the output of the haze measurement network is used as a reference for transmittance, and combined with the global atmospheric light value to generate a dehazed image.

[0056] 4. Loss function and optimization: Using mean square error loss function Guide the training optimization of the model; by calculating the mean square error between the generated dehazed image and the real clear image, the model parameters are continuously adjusted to minimize the loss function value.

[0057] 5. Model training and testing The model of this embodiment is trained on a public dehazing dataset. In terms of software and hardware configuration: Windows is selected as the operating system, the Pytorch1.12.0 deep learning framework is used to build the development environment, the programming language is Python3.7, and the NVIDIA GeForce RTX 3060 graphics card is used to complete the training and testing of the model.

[0058] Through training, the model can learn the mapping relationship between haze images and clear images, and achieve good dehazing effect on the test set.

[0059] Example 6 In another preferred embodiment, based on the above-mentioned embodiments 1, 2, 3, 4, and 5, the specific implementation steps of the pipeline CCTV detection image intelligent defogging processing method of the present invention are: first, the pipeline CCTV detection image is obtained. The image may be atomized and blurred due to the complex and humid environment inside the pipeline, affecting the accuracy of subsequent defect identification.

[0060] Next, a haze measurement network based on a deep convolutional neural network is used to measure the haze of the pipeline CCTV detection image; the haze measurement network includes a lightweight and efficient convolutional neural network structure, which uses deep separable convolution to reduce the amount of calculation, and introduces a position normalization module to enhance the model's expression ability; specifically, the haze measurement network first extracts image features through a feature extraction module, then uses a global context module to capture the global context information of the image, and finally outputs pixel-level haze measurement results.

[0061] A generation module is constructed based on the improved atmospheric scattering model, and the haze measurement results are combined with the foggy image to generate a defogged image; the generation module includes an image reconstruction module, which adopts an adaptive fusion strategy to generate a defogged image according to the haze measurement results and the foggy image; the adaptive fusion strategy can dynamically adjust the fusion weight according to the different degrees of haze to obtain the best defogging effect.

[0062] In order to further improve the image quality, the dehazed image can also be post-processed; the post-processing steps include color correction and detail enhancement, which can further improve the visual effect of the image and make it easier to identify defects such as cracks and corrosion in the pipeline.

[0063] Through the above method, pipeline CCTV inspection images can be processed in real time to improve image clarity, thereby helping back-end personnel to accurately identify defects in the pipeline and improve the efficiency and accuracy of pipeline inspection and maintenance.

[0064] Example 7 In another preferred embodiment, as a further improvement of Example 6, this embodiment adds an attention mechanism module in the haze measurement network; the attention mechanism module can further focus on important areas in the image and improve the accuracy of haze measurement; at the same time, in the generation module, a more complex adaptive fusion strategy is adopted to better handle image dehazing problems under different haze levels.

[0065] In addition, this embodiment also optimizes the post-processing step and adopts a more advanced color correction algorithm and detail enhancement technology to further improve the image quality.

[0066] Example 8 In another preferred embodiment, this embodiment provides a pipeline CCTV detection image intelligent defogging processing method based on GPU (Graphics Processing Unit) acceleration; compared with Example 6 and Example 7, this embodiment optimizes the algorithm implementation, fully utilizes the parallel computing capability of the GPU, and improves the running speed of the algorithm.

[0067] Specifically, this embodiment transplants the implementation of the haze measurement network and generation module to the GPU and accelerates them through the CUDA (Compute Unified Device Architecture) programming model; at the same time, the matrix operations and convolution operations in the algorithm are optimized to further improve the computing efficiency.

[0068] Example 9 In order to demonstrate the innovation and feasibility of the pipeline CCTV detection image intelligent defogging processing method and system of the present invention, we have carried out a series of comparative experiments. In the experiment, we mainly compared the two key indicators for measuring the image defogging effect: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM).

[0069] 1. Definition of key indicators 1. Peak Signal-to-Noise Ratio (PSNR): Peak signal-to-noise ratio is a quality assessment indicator widely used in image compression, image enhancement, image reconstruction and other fields.

[0070] For super-resolution images, PSNR calculates the error between the foggy image and the defogged image at the pixel level. Its definition formula is as follows: (11) In the formula, It represents the average value of the sum of squares of all pixel position differences between two images, which has the same definition as the mean square error loss (see formula 5). Indicates the theoretical maximum pixel value of an image, for example, for an 8-bit binary image, The value of is 255; in addition, for a color RGB image with a channel bit depth of 8 bits, the maximum possible value of each channel is also 255; when calculating the PSNR value of a color RGB image, the mean square error of the three RGB channels is usually calculated first, and then divided by 3 to obtain their average value.

[0071] 2. Structural Similarity Index of Images (SSIM): The structural similarity index is a metric used to measure the structural similarity between two images.

[0072] It comprehensively considers the brightness, contrast and structural information of the image.

[0073] The SSIM value ranges from -1 to 1. When the SSIM value is closer to 1, it means that the similarity between the two images is higher, that is, the image defogging effect is better. Its definition formula is as follows: (12) In the formula, and They are the pixel matrix representations of the two images respectively; and Respectively represent images and The pixel mean, which represents the brightness level of the image. When the average value of all pixels in an image is larger, the brightness of the overall image is higher. and Respectively represent images and The pixel variance of the image, the contrast of the image is a description of the distribution of image pixels in the entire dynamic range; the wider the distribution of image pixels in the dynamic range, the better the contrast; conversely, the more compact the distribution of image pixels, the worse the contrast of the image; Representing images and The covariance between pixels of and is a constant used to stabilize the algorithm, usually set to and ,in is the dynamic range of the pixel, generally expressed as the maximum pixel value minus the minimum pixel value, and and is a parameter used to control the contrast and structure effects.

[0074] 2. Comparative Experimental Results Table 1 shows the experimental results comparison between the proposed method and two other common dehazing methods (atmospheric scattering model ASM and dark channel prior DCP).

[0075]

[0076] From Table 1 we can see that: The peak signal-to-noise ratio and structural similarity index of the method of the present invention are higher than those of the dark channel prior defogging algorithm and the traditional defogging method based on the atmospheric scattering model, indicating that the method of the present invention has obvious advantages in defogging effect.

[0077] In terms of the processing time of a single image, the method of the present invention only takes 2.89 seconds, which is much lower than the 20.20 seconds of the dark channel prior defogging method and the 53.89 seconds of the traditional atmospheric scattering model method, showing that the method of the present invention is significantly ahead in real-time performance; in terms of image processing quality, the peak signal-to-noise ratio of the present invention is 11.76 decibels, which is also higher than the 10.75 decibels of the dark channel prior defogging method and the 9.55 decibels of the traditional atmospheric scattering model method, showing the technical advantage of the method of the present invention in defogging effect.

[0078] 3. Method advantages and application prospects Through the description and analysis of the above embodiments and comparative experiments, it can be seen that the pipeline CCTV detection image intelligent defogging processing method of the present invention has multiple implementation methods, and can be selected and optimized according to actual needs; at the same time, the method has the following significant advantages: High real-time performance: fast processing speed, able to meet the real-time needs of pipeline inspection and maintenance.

[0079] Good dehazing effect: significantly improves image quality by increasing the peak signal-to-noise ratio and structural similarity index.

[0080] Easy to implement: The method is relatively simple and easy to deploy and implement in practical applications.

[0081] Therefore, the method of the present invention can be widely used in the field of pipeline detection and maintenance, providing strong support for the intelligent and efficient development of related industries.

[0082] In the preferred scheme, the haze measurement network in Step 2 includes at least two deeply separable convolutional layers, which are used to extract shallow features of the image and adjust the feature distribution through a position normalization module; the above settings effectively improve the efficiency and accuracy of feature extraction, and the position normalization module ensures the consistency of features at different positions, providing a more reliable input for subsequent haze calculations.

[0083] In the preferred solution, the depth-separable convolution layer in the haze measurement network includes two parts: depth convolution and point-by-point convolution. The depth convolution is used to perform convolution operations on each input channel independently, and the point-by-point convolution is used to linearly combine the output channels of the depth convolution. The above settings not only significantly reduce the computational complexity, but also improve the model's ability to learn haze features. In addition, by introducing batch normalization and ReLU activation functions, the network's nonlinear expression ability and generalization performance are further enhanced.

[0084] In the preferred solution, the position normalization module normalizes the features in the channel dimension, normalizes the feature values ​​to a finer and more compact data range, and introduces learnable scaling factors and bias terms to dynamically adjust the feature distribution and enhance the expressiveness of the model. The above settings enable the model to maintain stable and efficient performance under different inputs. At the same time, the parameters of the scaling factors and bias terms are optimized through the gradient return mechanism, further improving the generalization ability and adaptability of the model.

[0085] In the preferred solution, the generation module in Step 3 associates the image fog level with the foggy image based on the improved atmospheric scattering model, and establishes a mapping relationship between the fog level input and the defogged image at the pixel level; the above settings not only improve the accuracy of image defogging, but also ensure the processing efficiency; by finely controlling the fog level, the system can automatically adapt to different scenes, generate clear and natural defogged images, and significantly improve the visual experience.

[0086] In the preferred scheme, the generation module generates the defogged image through the following formula. The generation module is improved from the atmospheric scattering model, takes the input fogged image as input, and outputs the defogged image. The above setting effectively utilizes the depth information and atmospheric light information of the image, gradually approaches the real scene through the iterative optimization algorithm, and finally outputs a clear, natural, and detail-rich defogged image, which significantly improves the image quality.

[0087] The preferred solution also includes using a mean square error loss function during the training process of the haze measurement network and the generation module. To guide the training optimization of the model; the above settings can make the generated image closer to the real image in terms of haze, thereby improving the model's ability to learn haze features and ensuring higher processing accuracy and efficiency in subsequent image dehazing tasks.

[0088] The preferred scheme also includes a step of post-processing the dehazed image, and the post-processing step includes color correction and detail enhancement; the above settings are intended to further enhance the visual effect of the image, making the image more realistic and natural, and the details clearer and richer, so as to meet the needs of more application scenarios and enhance the user experience.

[0089] In the preferred scheme, the output feature map of the haze measurement network is consistent with the foggy image in size and number of channels, so that the output haze measurement value can accurately reflect the degree of haze of the image at the pixel level; the above setting ensures that the haze measurement result is highly aligned with the foggy image, which facilitates the subsequent dehazing algorithm to perform refined processing on each pixel point, effectively improving the accuracy and efficiency of image dehazing.

[0090] In a preferred scheme, the haze measurement network module includes at least two deeply separable convolutional layers and a position normalization module, and also includes a feature extraction module and a global context module. The feature extraction module is used to extract image features, and the global context module is used to capture the global context information of the image. The above settings can effectively improve the accuracy and efficiency of haze measurement, among which the deeply separable convolutional layer reduces the computational complexity, the position normalization module enhances the feature position information, and the global context module provides rich contextual basis for haze prediction.

[0091] In a preferred scheme, the generation module is constructed based on an improved atmospheric scattering model, including an image reconstruction module, which is used to generate a defogged image based on the haze measurement result and the foggy image using an adaptive fusion strategy; the above settings can significantly improve the accuracy and efficiency of image defogging, reduce image blur and detail loss problems caused by foggy days, make the defogged image clearer and more realistic, and provide strong support for subsequent image processing and analysis.

[0092] In summary, the intelligent defogging processing method and system for pipeline CCTV detection images provided by the present invention effectively overcome the problem of image fogging and blurring caused by the complex internal humid environment in the existing CCTV pipeline technology; the defogging algorithm based on deep convolutional neural network carefully designed by the present invention is specially customized for pipeline CCTV detection images, and achieves unprecedented defogging effect; the algorithm cleverly integrates the haze measurement network and the generation module. The former uses a lightweight and efficient convolutional neural network structure, combined with deep separable convolution technology and position normalization module, which significantly improves the real-time processing capability and expression capability of the model, ensuring that the defogging processing is both efficient and accurate; the latter makes innovative improvements to the atmospheric scattering model, and closely cooperates with the haze measurement network, and successfully constructs an accurate mapping from foggy blurred images to clear images through pixel-level deep learning, opening up a new path for the defogging algorithm; in the model training stage, the present invention adopts the mean square error loss function as an optimization guide to ensure the stability and reliability of the defogging effect. At the same time, by using a public defogging data set for training, The generalization ability and practicality of the model are further enhanced. It is worth mentioning that the present invention applies deep convolutional neural network to the defogging of pipeline CCTV detection images for the first time, which completely subverts the traditional method based on physical model or image restoration. The image fogging degree level is learned through the haze measurement network, and the mapping relationship between the haze level input and the defogging image is established at the pixel level through the generation module. This innovative combination method is unique in the field of pipeline CCTV detection image defogging; in addition, the haze measurement network proposed in the present invention adopts deep separable convolution technology, which not only greatly reduces the computational amount of the network model, but also maintains a high representation ability, and realizes the effect of real-time defogging. At the same time, the position normalization module is introduced into the haze measurement network, and the expression ability of the model is further enhanced by dynamically adjusting the feature distribution, which significantly improves the defogging effect; the present invention has made a major breakthrough in the field of pipeline CCTV detection image defogging, provides strong technical support for pipeline detection and maintenance work, and has broad application prospects and important practical value in the field of pipeline detection.

Claims

1. Intelligent defogging processing method for pipeline CCTV detection images, characterized in that: The following steps are involved: Step 1: Input the pipeline CCTV detection image with fog blur; Step 2: Use the haze measurement network to process the input foggy image to measure the degree of image haze; Step 3: According to the measured image fogging level, the defogging image is generated through the generation module; Step 4: The image output module outputs the processed dehazed image.

2. The pipeline CCTV detection image intelligent defogging processing method according to claim 1 is characterized by: The haze measurement network in Step 2 includes at least two depth-separable convolutional layers for extracting shallow features of the image, and a position normalization module is introduced to adjust feature distribution.

3. The intelligent defogging method for pipeline CCTV inspection images according to claim 2 is characterized by: The depthwise separable convolution layer in the haze measurement network includes two parts: depthwise convolution and pointwise convolution. The depthwise convolution is used to perform convolution operations on each input at the channel level independently, and the pointwise convolution is used to transform the dimension of the output features of the depthwise convolution and integrate cross-channel information.

4. The intelligent defogging processing method for pipeline CCTV detection images according to claim 3 is characterized by: The position normalization module normalizes the features in the channel dimension, normalizes the feature values ​​to a data range, and introduces learnable scaling factors and bias terms to dynamically adjust the feature distribution and enhance the expressiveness of the model.

5. The intelligent defogging method for pipeline CCTV inspection images according to claim 4 is characterized in that: The position normalization module subtracts the mean from the input of the activation function and divides it by the standard deviation, so that the mean of the feature map on each channel is 0 and the standard deviation is 1. The position normalization is expressed as: （1）； In the formula, is the input feature at position The value at Location The mean value in the channel dimension, It's location The variance in the channel dimension, is to prevent the denominator from being zero. is the scaling factor, is the bias term; The activation function adopts the ReLU activation function, and its expression is: （2）； In the formula, is the input value, is the output value.

6. The pipeline CCTV detection image intelligent defogging processing method according to claim 5 is characterized by: The generation module in Step 3 associates the image fogging level with the foggy image based on the improved atmospheric scattering model, and establishes a mapping relationship between the fogging level input and the defogged image at the pixel level.

7. The pipeline CCTV detection image intelligent defogging processing method according to claim 6 is characterized in that: The generation module generates the defogged image by the following formula. The generation module is improved from the atmospheric scattering model and takes the standard atmospheric scattering model as a reference: （3）； In the formula, Indicates the pixel position The observed image at Indicates the pixel position Clear images of Indicates pixel position The transmittance at , which describes the fraction of light that remains after passing through the atmosphere, represents the global atmospheric light, is the proportion of light scattered; Taking the input fogged image as input and the defogged image as output, the atmospheric scattering model is transformed and expressed as: （4）； set up , Indicates a bias with a set value of 1.

8. The pipeline CCTV detection image intelligent defogging processing method according to claim 7 is characterized in that it also includes using a mean square error loss function in the model training process of the generation module. To guide the training optimization of the model, the mean square error represents the error between the predicted output and the actual real image at the pixel level, where the loss function It is expressed as: （5）； In the formula, Indicates the number of samples, multi-channel color images Value is ; is the real image pixel value; The pixel value predicted by the model.

9. The intelligent defogging processing method for pipeline CCTV detection images according to claim 8 is characterized by: The output feature map of the haze measurement network is consistent with the foggy image in size and number of channels, so that the output haze measurement value can accurately reflect the degree of haze of the image at the pixel level.

10. A system for implementing the pipeline CCTV detection image intelligent defogging processing method according to any one of claims 1 to 9, characterized in that: include: An image input module for receiving pipeline CCTV inspection images with fog blur; The haze measurement network module is used to process the haze-blurred image input by the image input module to measure the image haze degree level; A generation module, used for generating a defogged image according to the image fogging degree level calculated by the fogging measurement network module; The image output module is used to output the dehazed image processed by the generation module.

11. The system of the pipeline CCTV detection image intelligent defogging processing method according to claim 10 is characterized in that: The feature extraction module of the haze measurement network module includes two depth-separable convolutional layers and a position normalization module.

12. The system of the pipeline CCTV detection image intelligent defogging processing method according to claim 10 is characterized in that: The generation module is constructed based on an improved atmospheric scattering model, and uses an adaptive fusion strategy to generate a defogged image according to a haze measurement result and a foggy image.

Images