Pipeline disease identification method, device, electronic equipment and computer storage medium

Abstract

The present application discloses a pipeline disease identification method, device, electronic device and computer storage medium, which belongs to the field of image processing technology. Among them, the method includes: obtaining a pipeline image collected by a pipeline drifting robot; defogging the pipeline image based on a first image enhancement network model, and outputting a fog-free image; demotion blurring the pipeline image based on a second image enhancement network model, and outputting a motion-free blur image; performing pipeline disease identification on the fog-free image and the motion-free blur image, respectively, and outputting a disease identification result. The present application uses an image enhancement network to perform defogging and demotion blurring on the pipeline image collected by the pipeline drifting robot, and then performs pipeline disease identification, which solves the problems of water mist blur and motion blur of the pipeline image collected by the pipeline drifting robot, enhances the quality of image data, and thereby improves the accuracy of pipeline disease identification.

Description

Technical Field

[0001] The present application relates to the technical field of image processing and defect recognition, and in particular to a pipeline defect recognition method, device, electronic equipment and computer storage medium. Background Art

[0002] A healthy and complete urban underground drainage system can quickly divert rainwater and urban sewage to designated treatment points, which can not only avoid waterlogging caused by excessive accumulation of water, but also maintain the urban environment and public health, and have a positive impact on the sustainability, environmental health and socio-economic development of the city. Some common internal diseases of drainage pipes, such as ruptures, cracks, deformations, etc., may act alone or in combination to negatively affect the performance and reliability of the underground drainage system. Therefore, regular inspections and maintenance are important steps to ensure the normal operation of the drainage system.

[0003] However, with the continuous acceleration of urbanization, the maintenance of underground drainage systems has become increasingly difficult. At present, urban drainage pipelines are large in scale and pipeline disease formation cycles are short. Common inspection methods are often limited by multiple factors and have low detection efficiency. For example, sonar is limited by turbid water quality; QV is limited by visual range; closed-circuit television (CCTV) is limited by the waterless environment during operation.

[0004] As a new type of inspection tool, the pipeline drifting robot is driven by water flow and records the internal conditions of the pipeline during the drifting process. It is easy to operate and has high inspection efficiency, but it faces the challenges of fog and motion blur during data collection. First, during operation, the pipeline drifting robot does not need to meet the waterless environment and is in direct contact with water, which can easily cause water fog and blur on the lens. Second, the pipeline drifting robot relies on water flow to drive during operation, moves at a fast speed, and interacts with drones, which makes the motion blur prominent. These problems result in poor quality of collected data, affecting the accurate identification of subsequent pipeline diseases.

[0005] With respect to the above-mentioned problems existing in the related technologies, no effective solutions have been found so far. Summary of the invention

[0006] The present application provides a pipeline disease identification method, device, electronic equipment and computer storage medium to solve the above-mentioned technical problems existing in the related technology.

[0007] According to an embodiment of the present application, a pipeline disease identification method is provided, including: acquiring a pipeline image collected by a pipeline drifting robot; performing defogging processing on the pipeline image based on a first image enhancement network model, and outputting a fog-free image; performing demotion blur processing on the pipeline image based on a second image enhancement network model, and outputting a motion-free blur image; performing pipeline disease identification on the fog-free image and the motion-free blur image, respectively, and outputting a disease identification result.

[0008] According to another embodiment of the present application, a pipeline disease identification device is provided, including: an acquisition module, used to acquire a pipeline image acquired by a pipeline drifting robot; an enhancement module, used to perform a defogging process on the pipeline image based on a first image enhancement network model, and output a fog-free image; perform a demotion blurring process on the pipeline image based on a second image enhancement network model, and output a motion-free blur image; and an identification module, used to perform pipeline disease identification on the fog-free image and the motion-free blur image, respectively, and output a disease identification result.

[0009] Optionally, the enhancement module includes a first input unit, used to input the foggy image into a first image enhancement network model; a feature extraction unit, used to separate the foggy image into images of three channels of R, G, and B through the first branch of the first image enhancement network model, and obtain the first feature map corresponding to each channel after the image of each channel passes through the convolution block respectively; a weight acquisition unit, used to obtain the weight of each channel of the foggy image through the second branch of the first image enhancement network model using the SE-Unet network; a weighting unit, used to weight the weight of each channel with the first feature map corresponding to each channel, perform feature modulation adaptation on the weighted feature results, and output a fog-free image.

[0010] A squeezing and excitation module is added to the upsampling module and the downsampling module of the SE-Unet network, the squeezing module includes global average pooling, the excitation module is composed of two layers of fully connected networks, and the weight acquisition unit includes: an input subunit, used to input the foggy image into the convolution layer to obtain the second feature map of each channel of the foggy image; a compression subunit, used to compress the second feature map of each channel through global average pooling to form a channel descriptor of each channel; a dimensionality reduction subunit, used to reduce the number of channels in the channel descriptor to 1 / r times the original number of channels through the first fully connected layer, where r is greater than 1; a recovery subunit, used to restore the number of channels to the original number of channels by using the second fully connected layer after the ReLU activation function; an output subunit, used to limit the output between 0 and 1 through the sigmoid activation function to obtain the weight of each channel.

[0011] The pipeline image is a motion blurred image, and the second image enhancement network model uses a generative adversarial network, wherein the generative network consists of a 7×7 convolution block, two 3×3 convolution blocks, six residual blocks and two upsampling convolution blocks. The enhancement module also includes: a second input unit, used to input the motion blurred image into the second image enhancement network model; a generation unit, used to generate a non-motion blur image through the generation network; and a calculation unit, used to calculate the probability that the non-motion blur image is a real image through the adversarial network.

[0012] The pipeline disease identification device further includes: a first synthesis module, which is used to synthesize the pipeline image into a foggy image by using the following formula to form a defogging training data set: ;in, For foggy images, is the initial haze-free image, is the atmospheric transmittance, is the fog concentration coefficient, is the scene depth, is the atmospheric light value at infinity, is the position of the pixel in the image; and a second synthesis module is used to synthesize the pipeline image into a motion blurred image by using the following formula to form a de-motion blur training data set: ;in, and represent blurred image and clear image respectively, represents the motion blur kernel, represents Gaussian noise, Represents a convolution operation; a training module, used to input the defogging training data set and the motion blur training data set into the initial defogging network and the initial demotion blur network for multiple update iterations, respectively, to obtain the first image enhancement network model and the second image enhancement network model.

[0013] According to another embodiment of the present application, a computer storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned device embodiments when running.

[0014] According to another embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; the processor is used to execute the steps in the above method by running the program stored in the memory.

[0015] According to yet another embodiment of the present application, a computer program product comprising instructions is also provided, which enables the computer to execute the steps in the above method when the computer program product is executed on the computer.

[0016] Through the embodiments of the present application, the pipeline images collected by the pipeline drifting robot are defogged and de-motion blurred using an image enhancement network before pipeline disease identification is performed, thereby solving the problems of water mist blur and motion blur in the pipeline images collected by the pipeline drifting robot, enhancing the quality of the image data, and thereby improving the accuracy of pipeline disease identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a hardware structure block diagram of a computer according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of a pipeline disease identification method according to an embodiment of the present application;

[0020] Figure 3 This is a schematic diagram of the overall framework flow of pipeline disease identification in an embodiment of the present application;

[0021] Figure 4 This is a defogging network structure framework diagram of an embodiment of the present application;

[0022] Figure 5 This is a SE-Unet network architecture diagram of an embodiment of the present application;

[0023] Figure 6 1 is a diagram of the architecture of a squeeze and excitation convolution module according to an embodiment of the present application;

[0024] Figure 7 is a diagram of the architecture of a feature modulation adaptation module in an embodiment of the present application;

[0025] Figure 8 This is a framework diagram of the motion blur removal network structure of an embodiment of the present application;

[0026] Fig. 9 It is a structural block diagram of a pipeline disease identification device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Example 1

[0030] The method embodiment provided in the first embodiment of the present application can be executed in a pipeline robot, a computer, a tablet or a similar computing device. Taking running on a computer as an example, Figure 1 is a hardware structure block diagram of a computer in an embodiment of the present application. Figure 1 As shown, the computer may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the computer may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0031] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a pipeline disease identification method in an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0032] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] Pipeline drifting robots (such as capsule robots) are driven by water flow and record the internal conditions of the pipeline during the drifting process, but they face the challenges of fog blur and motion blur during data collection. These problems lead to poor quality of collected data and affect the subsequent accurate identification of pipeline defects. In view of the above problems, a pipeline defect identification method is provided in this embodiment. Figure 2 is a flow chart of a pipeline disease identification method according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:

[0034] Step S10, obtaining a pipeline image collected by the pipeline drifting robot;

[0035] Step S20, performing defogging processing on the pipeline image based on the first image enhancement network model to output a fog-free image; performing demotion blurring processing on the pipeline image based on the second image enhancement network model to output a motion-free blur image;

[0036] Step S30, performing pipeline defect recognition on the fog-free image and the image without motion blur, respectively, and outputting a defect recognition result.

[0037] First, the pipeline image taken by the pipeline drifting robot is obtained, and then the pipeline image is defogged and de-motion blurred using the pre-built image enhancement network model to output a fog-free image and a non-motion blurred image. Among them, the first image enhancement network model is a defogging network model, and the second image enhancement network model is a de-motion blurred network model. In the pipeline disease recognition stage, the trained YOLOv5 model can be used to perform disease recognition on the capsule robot image data after defogging and de-motion blurring to obtain the disease recognition results.

[0038] Through the above steps, the pipeline images collected by the pipeline drifting robot are defogged and de-motion blurred using the image enhancement network, and then pipeline disease identification is performed, which solves the problems of water mist blur and motion blur in the pipeline images collected by the pipeline drifting robot, enhances the quality of image data, and thus improves the accuracy of pipeline disease identification.

[0039] In one implementation of this embodiment, the pipeline image is a foggy image, and defogging the pipeline image is performed based on the first image enhancement network model to output a fog-free image, including:

[0040] A21, inputting the foggy image into a first image enhancement network model;

[0041] A22, separating the foggy image into images of three channels, namely, R, G, and B, through the first branch of the first image enhancement network model, and obtaining a first feature map corresponding to each channel after the image of each channel passes through a convolution block respectively;

[0042] A23, using the SE-Unet network through the second branch of the first image enhancement network model, to obtain the weight of each channel of the foggy image;

[0043] A24, weighting the weight of each channel with the first feature map corresponding to each channel, performing feature modulation adaptation on the weighted feature results, and outputting a haze-free image.

[0044] In this implementation, the first image enhancement network model (defogging network) is first constructed. Figure 4 The dehazing network structure framework diagram shown in the figure shows that the dehazing network consists of two branches. The first branch includes a channel separation module, three convolution blocks, a weight allocation module and a feature modulation adaptation module. Each convolution block contains a convolution kernel 3×3 convolution (3×3 Conv) and a ReLU activation function. The second branch consists of a SE-Unet network, an adaptive average pooling and a matrix reshaping module.

[0045] In the first branch, the channel separation module is used to perform channel separation operation, and the input foggy image is divided into three color channels: (R), green (G), and blue (B). The image of each channel passes through three convolution blocks respectively. This process extracts the primary features of each channel image and obtains three feature maps. Since fog has different effects on different color channels, channel separation is performed to process each color channel independently. In the second branch, the SE-Unet network is used to extract features from the input image and reshape it to obtain a guide to achieve feature channel-by-channel weighting.

[0046] The first feature map obtained by the first branch is re-weighted using the R, G, and B channel weights obtained by the second branch, and the weighted feature results are input into Figure 7 The feature modulation adaptation module shown obtains high-quality features that enhance channel information and outputs a haze-free image.

[0047] In this embodiment, the loss function of the defogging network is: , where D is the number of training images, I is the haze-free image after network processing, and J is the corresponding real image.

[0048] This implementation method decomposes the input image into three independent channels of R, G, and B for processing, so that the defogging network can realize channel-by-channel processing of the input image and channel-by-channel feature weighting, and can independently process the information of each channel to obtain high-quality features that strengthen the channel information and better capture the subtle differences between different color channels. In addition, after the SE-UNet network is processed, the weights of the three channels of R, G, and B will be redistributed. Since different color channels may be affected by fog to different degrees, the information of certain channels can be strengthened or weakened by redistributing the weights to improve the defogging effect.

[0049] In this embodiment, a squeezing and excitation module is added to the upsampling module and the downsampling module of the SE-Unet network, and the SE-Unet network is used to obtain the weight of each channel of the foggy image, including:

[0050] a231, input the foggy image into a convolutional layer to obtain a second feature map of each channel of the foggy image;

[0051] a232, compresses the second feature map of each channel through global average pooling to form a channel descriptor of each channel;

[0052] a233, reducing the number of channels in the channel descriptor to 1 / r times the original number of channels through a first fully connected layer, where r is greater than 1;

[0053] a234, after the ReLU activation function, the second fully connected layer is used to restore the number of channels to the original number of channels;

[0054] a235, the output is limited between 0 and 1 through the sigmoid activation function to obtain the weight of each channel.

[0055] Refer to Figure 5 The architecture diagram of the SE-Unet network shown in FIG. 1 is a diagram of the SE-Unet network. In this embodiment, a squeeze and excitation (SE) convolution block is added to the up-sampling module (Up block) and the down-sampling module (Down block) of the SE-Unet network, and a squeeze and excitation module is added between the convolution layer and the scaling alignment. Among them, 3, 64, 128, 256, and 512 represent the number of channels of each layer of the feature map in the SE-Unet structure. The squeeze and excitation convolution module is shown in FIG. Figure 6 As shown. The image is input into a convolutional layer (3×3 Conv), and then a squeeze operation and an excitation operation are performed. The squeeze operation compresses each channel feature map into a single value through a global average pooling operation to form a channel descriptor, which captures channel-level features in the global space range. The excitation operation consists of two layers of fully connected networks FC. The first fully connected layer reduces the number of input channels to 1 / r times the original, and uses the ReLU activation function to introduce nonlinearity and learn more complex features. The second fully connected layer converts it back to the original number of channels, and then limits the output between 0 and 1 through the Sigmoid function to obtain the weight of each channel. Finally, the scaling operation performs channel multiplication operation on the obtained weight and the original feature map, and weights the feature map according to the importance of each channel. In this embodiment, for the downsampling process, the foggy image is subjected to maximum pooling before being input into the convolution layer, and then a squeeze operation and an excitation operation are performed. Maximum pooling can reduce the dimensions (such as width and height) of the feature map while retaining important features. In the dehazing network, by inserting the squeeze and excitation (SE) module into the feature extraction module and the feature modulation adaptation module, the network can realize channel-by-channel processing of the input image and channel-by-channel feature weighting.

[0056] In this embodiment, in addition to inserting the squeeze and excitation blocks into the constructed dehazing network SE-Unet module, a channel attention mechanism (ECA) and a hybrid mechanism of channel attention and spatial attention (CBAM) can also be inserted. The dual attention mechanism can simultaneously enhance the network's ability to capture channel and spatial features, thereby improving the dehazing effect.

[0057] In another implementation of this embodiment, the pipeline image is a motion blurred image, and de-motion blurring the pipeline image based on the second image enhancement network model to output a motion-free blur image includes:

[0058] B21, inputting the motion blurred image into a second image enhancement network model, where the second image enhancement network model adopts a generative adversarial network;

[0059] B22, generating motion blur-free images through the generative network;

[0060] B23, calculating the probability that the image without motion blur is a real image through an adversarial network.

[0061] Reference Figure 8 The de-motion blur network structure framework diagram shown in the figure uses a generative adversarial network, in which the generative network consists of a 7×7 convolution block and two 3×3 convolution blocks (the 3×3 convolution block consists of Figure 8 3×3Conv, BN, ReLU), six residual blocks and two upsampling convolution blocks (the upsampling convolution blocks are composed of Figure 8 Nearest interpolation, 3×3Conv, BN, ReLU). Each residual block consists of two convolutional layers (3×3Conv), two instance normalization layers (BN), and a ReLU activation function. The adversarial network consists of five 4×4 convolutional blocks. The first four convolutional blocks are connected to an instance normalization layer (BN) and a LeakyReLU activation function after convolution, and the last convolutional block is connected to a Sigmoid function after convolution. The de-motion blur training set is input into the network, the generator generates de-motion blur images, and the discriminator determines whether the image output by the generator is real.

[0062] The loss function of the generated network consists of adversarial loss and perceptual loss, and the formula is as follows:

[0063] ;

[0064] ；

[0065] ；

[0066] in, To generate the total network loss, To combat losses, is the perceived loss, is the weight of perceptual loss, To blur the image, is the generator network, The input blurred image The target image generated later is is the discriminator network, is the judgment result of the discriminator on the image generated by the generator, is the number of training batch samples, For clear images, In the deep convolutional neural network (VGG19 network), The first layer before the max pooling layer The feature map obtained after convolution is For clear images Through the convolutional network (VGG19 network) Tier The channel is located after the feature map ( ) at the position, To generate images Through the convolutional network (VGG19 network) Tier The channel is located after the feature map ( ) eigenvalue at the position, For a specific layer The width of the For a specific layer The height on.

[0067] The loss function of the discriminant network is:

[0068] ;

[0069] in, To blur the image, The input blurred image The target image generated later is For the discriminator For the generator Generated Image The discrimination score of For the discriminator For blurred images The discrimination score of To generate the image, To generate the gradient of the image, is the number of training batch samples, The weight used to control the gradient penalty term, the default value is 10.

[0070] In the de-motion blur network, this embodiment uses the nearest neighbor interpolation upsampling operation to replace the deconvolution operation, which solves the checkerboard effect problem generated by the capsule robot data in the de-motion blur process, and uses a smaller number of six residual blocks to replace the original nine residual blocks to reduce the number of model parameters and the amount of calculation.

[0071] In order to solve the problem of poor disease recognition caused by water mist blur and motion blur faced by pipeline drifting robots, this embodiment proposes a drainage pipeline disease recognition method for pipeline drifting robots based on image enhancement network, which includes three parts: data set preparation, image enhancement network construction and verification, and disease recognition model training and testing. The overall framework process is as follows: Figure 3 As shown, the method includes: obtaining drainage pipe images taken by a capsule robot to construct a data set; constructing a defogging network and a de-motion blurring network respectively; inputting data into the defogging network and the de-motion blurring network for training respectively, continuously updating and iterating, obtaining an optimal model, and outputting a fog-free image and a motion blur-free image; according to the fog-free image and the motion blur-free image, using a trained YOLOv5 model to identify drainage pipe diseases and obtain an identification result.

[0072] First, prepare the dataset: obtain the video file shot by the pipeline drifting robot inside the drainage pipe, extract the frames of the video file, and obtain the drainage pipe image data. Through different processing, we can obtain the defogging dataset, de-motion blur dataset, and disease recognition dataset.

[0073] The construction of the defogging dataset includes: synthesizing the drainage pipe images obtained by the pipeline drifting robot into foggy images, and the formula is as follows:

[0074] ;

[0075] in, is the observed foggy image, is the initial haze-free image, , is the atmospheric transmittance, is the fog concentration coefficient, is the scene depth, is the atmospheric light value at infinity, is the position of the pixel in the image. Each drainage pipe image is paired with the corresponding foggy image to form a defogging dataset.

[0076] The construction of the de-motion blur dataset includes: synthesizing the drainage pipe image obtained by the pipeline drifting robot into a motion blurred image, the formula is as follows:

[0077] ;

[0078] in, and represent blurred image and clear image respectively, represents the motion blur kernel, represents Gaussian noise, Represents the convolution operation. Each drainage pipe image is paired with the corresponding motion blurred image to form a de-motion blurred dataset.

[0079] The construction of the disease recognition dataset includes: using the Labelme annotation tool to annotate the pipeline drifting robot drainage pipeline images in a format that conforms to YOLOv5. Then, the annotated images are divided into training sets and test sets in proportion.

[0080] Verify the image enhancement network: First, input the defogging and demotion blurring training data sets into the constructed defogging network and demotion blurring network respectively. The two networks extract the feature information of the training data sets respectively. After multiple updates and iterations, the optimal parameters are saved to obtain the optimal defogging model and the optimal demotion blurring model respectively. Then, input the defogging and demotion blurring test data sets into the above two optimal models respectively to obtain the final defogging and demotion blurring results. Based on the obtained defogging and demotion blurring images, the trained YOLOv5 model is used to identify drainage pipe diseases and obtain the disease recognition results.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0082] Example 2

[0083] In this embodiment, a pipeline disease identification device is also provided, which is used to implement the above embodiments and preferred implementations, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0084] Fig. 9 is a structural block diagram of a pipeline disease identification device according to an embodiment of the present application, such as Fig. 9 As shown, the device comprises:

[0085] The acquisition module 91 is used to acquire the pipeline image collected by the pipeline drifting robot;

[0086] The enhancement module 92 is used to perform a defogging process on the pipeline image based on the first image enhancement network model to output a defogging image; perform a demotion blurring process on the pipeline image based on the second image enhancement network model to output a demotion blurring image;

[0087] The recognition module 93 is used to perform pipeline disease recognition on the fog-free image and the non-motion blur image respectively, and output the disease recognition result.

[0088] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0089] Example 3

[0090] An embodiment of the present application further provides a computer storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0091] Optionally, in this embodiment, the computer storage medium may be configured to store a computer program for performing the following steps:

[0092] S1, obtaining the pipeline image collected by the pipeline drifting robot;

[0093] S2, performing defogging processing on the pipeline image based on the first image enhancement network model, and outputting a fog-free image; performing demotion blurring processing on the pipeline image based on the second image enhancement network model, and outputting a motion blur-free image;

[0094] S3, performing pipeline defect recognition on the fog-free image and the image without motion blur, respectively, and outputting defect recognition results.

[0095] Optionally, in this embodiment, the above-mentioned computer storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0096] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0097] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0098] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0099] S1, obtaining the pipeline image collected by the pipeline drifting robot;

[0100] S2, performing defogging processing on the pipeline image based on the first image enhancement network model, and outputting a fog-free image; performing demotion blurring processing on the pipeline image based on the second image enhancement network model, and outputting a motion blur-free image;

[0101] S3, performing pipeline defect recognition on the fog-free image and the image without motion blur, respectively, and outputting defect recognition results.

[0102] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0103] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0104] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0106] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0108] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a computer storage medium, including several instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned computer storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0109] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A pipeline disease identification method, characterized in that: The method comprises: Acquire pipeline images collected by the pipeline drifting robot; Based on the first image enhancement network model, the pipeline image is defogged and a non-fog image is output; based on the second image enhancement network model, the pipeline image is demotion blurred and a non-motion blurred image is output; Respectively performing pipeline defect recognition on the fog-free image and the image without motion blur, and outputting defect recognition results; Wherein, when the pipeline image is a foggy image, the pipeline image is defogged based on the first image enhancement network model, and the output of the fog-free image includes: inputting the foggy image into the first image enhancement network model; through the first branch of the first image enhancement network model, the foggy image is separated into images of three channels of R, G, and B, and the images of each channel are respectively passed through the convolution block to obtain the first feature map corresponding to each channel; through the second branch of the first image enhancement network model, the SE-Unet network is used to obtain the weight of each channel of the foggy image; the weight of each channel is weighted with the first feature map corresponding to each channel, and the weighted feature result is input into the feature modulation adaptation module for feature modulation adaptation, and the fog-free image is output, wherein the feature modulation adaptation module is composed of a feature splicing layer, a 3×3 convolution layer, a squeeze and excitation module, and a scaling alignment module; A squeezing and excitation module is added to the upsampling module and the downsampling module of the SE-Unet network, the squeezing module includes global average pooling, and the excitation module is composed of two layers of fully connected networks. The SE-Unet network is used to obtain the weight of each channel of the foggy image, including: inputting the foggy image into a convolutional layer to obtain a second feature map of each channel of the foggy image; compressing the second feature map of each channel by global average pooling to form a channel descriptor of each channel; reducing the number of channels in the channel descriptor to 1 / r times the original number of channels by a first fully connected layer, where r is greater than 1; after a ReLU activation function, using a second fully connected layer to restore the number of channels to the original number of channels; limiting the output between 0 and 1 by a sigmoid activation function to obtain the weight of each channel.

2. The method according to claim 1, characterized in that The pipeline image is a motion blurred image, and the second image enhancement network model uses a generative adversarial network, wherein the generative network consists of a 7×7 convolution block, two 3×3 convolution blocks, six residual blocks, and two upsampling convolution blocks. The pipeline image is de-motion blurred based on the second image enhancement network model, and outputting a motion-free blur image includes: Inputting the motion blurred image into a second image enhancement network model; Generate a motion blur-free image through the generation network; The probability that the image without motion blur is a real image is calculated through an adversarial network.

3. The method according to claim 2, characterized in that The generator network uses the following loss function: ； ； ； in, To generate the total network loss, To combat losses, is the perceived loss, is the weight of perceptual loss, To blur the image, is the generator network, The input blurred image The target image generated later is is the discriminator network, is the judgment result of the discriminator on the image generated by the generator, is the number of training batch samples, For clear images, In deep convolutional neural networks, The first layer before the max pooling layer The feature map obtained after convolution is For clear images Through the convolutional network Tier The channel is located after the feature map ( ) at the position, To generate images Through the convolutional network Tier The channel is located after the feature map ( ) at the position, and is the dimension of the feature map.

4. The method according to claim 2, characterized in that: The generative adversarial network includes a discriminative network, which uses the following loss function: ； in, To blur the image, The input blurred image The target image generated later is For the discriminator For the generator Generated Image The discrimination score of For the discriminator For blurred images The discrimination score of To generate the image, To generate the gradient of the image, is the weight used to control the gradient penalty term, is the number of samples in the training batch.

5. The method according to claim 1, characterized in that Before performing defogging and de-motion blurring processing on the pipeline image based on the first image enhancement network model and the second image enhancement network model, the method further includes: The pipeline image is synthesized into a foggy image using the following formula to form a defogging training dataset: ;in, For foggy images, is the initial haze-free image, is the atmospheric transmittance, is the fog concentration coefficient, is the scene depth, is the atmospheric light value at infinity, and a is the position of the pixel in the image; The pipeline image is synthesized into a motion blurred image using the following formula to form a de-motion blur training dataset: 1; among them, and represent blurred image and clear image respectively, represents the motion blur kernel, r1 represents Gaussian noise, Represents the convolution operation; The dehazing training data set and the motion blur training data set are respectively input into an initial dehazing network and an initial de-motion blurring network for multiple update iterations to obtain a first image enhancement network model and a second image enhancement network model.

6. A pipeline disease identification device, characterized in that: include: An acquisition module, used to obtain pipeline images collected by the pipeline drifting robot; An enhancement module, configured to perform defogging processing on the pipeline image based on a first image enhancement network model, and output a haze-free image; Performing a motion blur removal process on the pipeline image based on a second image enhancement network model, and outputting a motion blur-free image; An identification module, used to identify pipeline defects on the fog-free image and the image without motion blur, and output a defect identification result; The enhancement module includes a first input unit for inputting the foggy image into a first image enhancement network model; A feature extraction unit, configured to separate the foggy image into images of three channels, namely, R, G, and B, through a first branch of the first image enhancement network model, and obtain a first feature map corresponding to each channel after the image of each channel passes through a convolution block respectively; A weight acquisition unit, configured to acquire a weight of each channel of the foggy image by using a SE-Unet network through the second branch of the first image enhancement network model; A weighting unit, used for weighting the weight of each channel with the first feature map corresponding to each channel, inputting the weighted feature result into the feature modulation adaptation module for feature modulation adaptation, and outputting a haze-free image, wherein the feature modulation adaptation module is composed of a feature splicing layer, a 3×3 convolution layer, a squeezing and excitation module, and a scaling alignment module; A squeezing and excitation module is added to the upsampling module and the downsampling module of the SE-Unet network, the squeezing module includes global average pooling, the excitation module is composed of two layers of fully connected networks, and the weight acquisition unit includes: an input subunit, used to input the foggy image into the convolution layer to obtain the second feature map of each channel of the foggy image; a compression subunit, used to compress the second feature map of each channel through global average pooling to form a channel descriptor of each channel; a dimensionality reduction subunit, used to reduce the number of channels in the channel descriptor to 1 / r times the original number of channels through the first fully connected layer, where r is greater than 1; a recovery subunit, used to restore the number of channels to the original number of channels by using the second fully connected layer after the ReLU activation function; an output subunit, used to limit the output between 0 and 1 through the sigmoid activation function to obtain the weight of each channel.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the pipeline disease identification method according to any one of claims 1 to 5 by running a program stored in a memory.

8. A computer storage medium, characterized in that: The computer storage medium includes a stored program, wherein the pipeline disease identification method according to any one of claims 1 to 5 is executed when the program is run.

Images