Visual analysis method and device for multiple internal corrosion failure types of pipeline

Abstract

The invention relates to a visual analysis method and device for multiple internal corrosion failure types of a pipeline. The method comprises the following steps: inputting internal corrosion failure picture data to be recognized into an internal corrosion failure type recognition model to obtain a predicted internal corrosion failure type and a feature map of the last convolutional layer; analyzing the predicted internal corrosion failure type and the feature map of the last convolutional layer by using Grad-CAM to obtain a feature region thermodynamic diagram; based on the predicted internal corrosion failure type and the characteristic region thermodynamic diagram, obtaining the contribution degree of the characteristic of each characteristic region to an identification result; wherein the internal corrosion failure type identification model is obtained by the following steps: obtaining a pipeline internal corrosion failure data set; inputting the internal corrosion failure data set into an initial internal corrosion failure type identification model for training to obtain a trained internal corrosion failure type identification model; wherein the initial internal corrosion failure type identification model is constructed by using a Resnet18 convolutional neural network.

Description

Technical Field

[0001] The invention relates to the technical field of image recognition and classification, and in particular to a method and device for visually analyzing various internal corrosion failure types of pipelines. Background Art

[0002] Oil and gas field gathering and transportation pipelines are one of the key areas of safe production in the oil and gas industry. These pipelines are the key link to ensure the smooth development and production of oil and gas fields. Ensuring the safe operation of oil and gas field gathering and transportation pipelines and effectively reducing risks are huge challenges faced by oil and gas field companies. In order to adapt to the requirements of quality, efficiency, safety, environmental protection and other aspects under the new situation, improving the inherent safety of oil and gas gathering and transportation pipelines and promoting the high-quality and rapid development of oil and gas fields have become the key tasks of oil and gas field companies to promote in an all-round way.

[0003] Most of my country's oil and gas fields have entered the secondary and tertiary development stages. The components of the produced materials are complex, and the pipeline network is seriously corroded and aged. According to statistics, more than 60% of the failures of oil and gas field pipelines in my country are caused by internal corrosion. The medium transported by oil and gas field pipelines is complex, containing corrosive gases such as H2S, CO2, O2, bacteria such as SRB, and Cl- plasma. There are many factors affecting internal corrosion, and there are many types of internal corrosion failures. Through preliminary research, a classification method for oil and gas field pipeline failures was developed, and internal corrosion failures were divided into 6 categories. A "three-level identification" strategy was proposed and promoted and applied on site in oil and gas fields. Existing technologies mainly rely on the experience of production managers or technicians for identification or send to third-party laboratories for testing and analysis. Summary of the invention

[0004] In order to obtain more effective visualization analysis results of various internal corrosion failure types of pipelines, an embodiment of the present invention provides a visualization analysis method and device for various internal corrosion failure types of pipelines.

[0005] In a first aspect, an embodiment of the present application provides a method for visually analyzing multiple internal corrosion failure types of pipelines, the method comprising:

[0006] Input the internal corrosion failure image data to be identified into the internal corrosion failure type recognition model to obtain the predicted internal corrosion failure type and the feature map of the last convolution layer;

[0007] Using Grad-CAM to analyze the predicted internal corrosion failure type and the feature map of the last convolution layer to obtain a feature area thermal map;

[0008] Based on the predicted internal corrosion failure type and the characteristic area thermal map, the contribution degree of the characteristics of each characteristic area to the recognition result is obtained;

[0009] The internal corrosion failure type identification model is obtained in the following way:

[0010] Obtain pipeline corrosion failure data set;

[0011] The internal corrosion failure data set is input into the initial internal corrosion failure type recognition model for training to obtain a trained internal corrosion failure type recognition model; wherein the initial internal corrosion failure type recognition model is constructed using a Resnet18 convolutional neural network.

[0012] In one or some optional implementations of the embodiment of the present application, the use of Grad-CAM to analyze the predicted internal corrosion failure type and the feature map of the last convolution layer to obtain a feature area thermal map includes:

[0013] Back propagation is performed according to the predicted internal corrosion failure type and the feature map of the last convolution layer to calculate and obtain the gradient information of the last convolution layer in the internal corrosion failure type recognition model;

[0014] Performing global average pooling on the gradient information of the last convolutional layer to obtain the weight information of the last convolutional layer;

[0015] According to the weight information of the last convolutional layer and the feature map of the last convolutional layer, the feature area heat map is obtained by calculating according to the following formula 1 through ReLU operation:

[0016]

[0017] In the formula, is the weight information, A k is the feature map of the last convolutional layer, k is the number of channels, and c is the predicted internal corrosion failure type.

[0018] In one or some optional implementations of the embodiment of the present application, the inputting the internal corrosion failure data set into the initial internal corrosion failure type identification model for training to obtain a trained internal corrosion failure type identification model includes:

[0019] The internal corrosion failure data set is input into the initial internal corrosion failure type identification model using a five-fold cross validation method to obtain prediction data;

[0020] The parameters of the initial internal corrosion failure type identification model are adjusted according to the predicted data by using a gradient descent method until a preset condition is reached, thereby obtaining a trained internal corrosion failure type identification model.

[0021] In one or some optional implementations of the embodiment of the present application, the step of obtaining a pipeline internal corrosion failure data set includes:

[0022] Acquire corrosion failure images inside pipelines;

[0023] Obtaining annotated internal corrosion failure data obtained by classifying and annotating the pipeline internal corrosion failure image using an image annotation tool;

[0024] The annotated internal corrosion failure data is preprocessed to obtain the pipeline internal corrosion failure data set.

[0025] In one or some optional implementations of the embodiment of the present application, the preprocessing of the annotated internal corrosion failure data to obtain the internal corrosion failure data set includes:

[0026] The annotated internal corrosion failure data is trimmed to remove redundant information irrelevant to the internal corrosion failure characteristics, and the image ratio is unified to 1:1;

[0027] Performing size transformation on the annotated internal corrosion failure data to unify the image size;

[0028] Data augmentation is performed on the annotated internal corrosion failure data to obtain the internal corrosion failure data set.

[0029] In a second aspect, an embodiment of the present application provides a device for visually analyzing various internal corrosion failure types of a pipeline, the device comprising:

[0030] The first prediction module is used to input the internal corrosion failure image data to be identified into the internal corrosion failure type recognition model to obtain the predicted internal corrosion failure type and the feature map of the last convolution layer;

[0031] A first analysis module is used to use Grad-CAM to analyze the predicted internal corrosion failure type and the feature map of the last convolution layer to obtain a feature area thermal map;

[0032] The second analysis module is used to obtain the contribution of the characteristics of each characteristic area to the identification result based on the predicted internal corrosion failure type and the characteristic area thermal map; wherein the internal corrosion failure type identification model is obtained by the following method:

[0033] The first acquisition module is used to acquire a pipeline corrosion failure data set;

[0034] The first training module is used to input the internal corrosion failure data set into the initial internal corrosion failure type recognition model for training to obtain a trained internal corrosion failure type recognition model; wherein the initial internal corrosion failure type recognition model is constructed using a Resnet18 convolutional neural network.

[0035] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for visually analyzing various internal corrosion failure types of pipelines.

[0036] In a fourth aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for visually analyzing multiple internal corrosion failure types of pipelines as described above is implemented.

[0037] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer device, enables the computer device to execute the above-mentioned method for visualizing and analyzing various internal corrosion failure types of pipelines.

[0038] In a sixth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run a computer program or instruction to implement the above-mentioned visualization analysis method for multiple internal corrosion failure types of pipelines.

[0039] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0040] The visualization analysis method of various internal corrosion failure types of pipelines provided in the embodiment of the present application inputs the internal corrosion failure image data to be identified into the internal corrosion failure type identification model to obtain the predicted internal corrosion failure type; the internal corrosion failure type identification model is constructed by the Resnet18 convolutional neural network and trained using the pipeline internal corrosion failure data set. The classification model based on the convolutional neural network can extract all the features in the pipeline internal corrosion failure image and learn the mapping relationship between all failure features and internal corrosion categories, so as to realize the "photographic" intelligent identification of internal corrosion failure types through the pipeline internal corrosion failure image, breaking the bottleneck of traditional internal corrosion analysis relying on the experience of business personnel; Grad-CAM is used to analyze and predict the internal corrosion failure type to obtain a characteristic area heat map, and the characteristic area heat map is analyzed and interpreted to obtain the identification basis. The heat map generated by Grad-CAM will mark the abnormal area based on which the internal corrosion type is judged, and the contribution of the features in each area to the classification result is displayed in the form of a heat map on the original image, so as to help on-site personnel to realize accurate and efficient internal corrosion type classification in practical applications.

[0041] The visualization analysis method for various internal corrosion failure types of pipelines provided in the embodiment of the present application does not need to be sent to a third-party laboratory for testing and analysis, which reduces capital investment and can greatly improve the recognition coverage rate of internal corrosion failure types of on-site gathering and transportation pipelines in oil and gas fields.

[0042] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of a method for visually analyzing various internal corrosion failure types of pipelines provided in an embodiment of the present application;

[0044] Figure 2 It is a schematic diagram of the network structure of the internal corrosion failure type identification model provided in the embodiment of the present application;

[0045] Figure 3 It is a schematic diagram of the structure of the Basicblock module and the Bottleneck module in the internal corrosion failure type identification model provided in the embodiment of the present application;

[0046] Figure 4 It is a schematic diagram of a visualization process for identifying internal corrosion failure types provided in an embodiment of the present application;

[0047] Figure 5 A schematic diagram of the structure of a device for visually analyzing various internal corrosion failure types of pipelines provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0049] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or clusters thereof.

[0050] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0051] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0052] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0054] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0055] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0056] The inventors found that field practical experience shows that due to the large number of corrosion failure types in pipelines and the very complex failure mechanisms, the quality requirements for identification personnel are very high. Relying on experience-based judgment, the efficiency and accuracy of corrosion failure identification are low; relying on third-party laboratories, the workload of failure analysis is large, the funding investment is high, and the "three-level identification" rate is low, which cannot meet the requirements of pipeline failure management, risk management and corrosion prevention. More scientific and convenient failure identification tools and methods are needed to improve the accuracy and efficiency of on-site corrosion failure identification.

[0057] The development of big data technologies such as machine learning and artificial intelligence has brought new opportunities to solve problems such as large workload, low failure recognition efficiency, insufficient recognition accuracy, and limited funds. Computer vision technology and artificial intelligence algorithms can be used to realize "photographic" recognition, which is of great significance for improving the accuracy and work efficiency of failure recognition on site in oil and gas fields, improving the "three-level recognition" rate, and improving the accuracy of corrosion risk assessment and the pertinence and effectiveness of repair and maintenance measures.

[0058] However, due to the many types of internal corrosion failures in pipelines, different types of internal corrosion morphologies have both similarities and differences. Even for the same type of internal corrosion failure, the corrosion morphologies under different conditions are different. For example, CO2 corrosion has both terrace-like corrosion morphologies and various corrosion morphologies such as honeycomb, teardrop, and horseshoe, which makes the extraction and annotation of corrosion image features extremely difficult. At the same time, on-site internal corrosion failure is often the result of multi-factor coupling. The interaction of multiple corrosion factors leads to a very large difference between the internal corrosion morphology and the typical single-type internal corrosion morphology. If some features are blindly extracted for matching, it is likely to affect the image recognition results due to the loss of some key features. This makes it difficult to apply traditional "face recognition" and "flower recognition" image classification methods based on "feature extraction-feature annotation-matching and recognition" to internal corrosion failure classification. It is urgent to construct a new image recognition method to meet the complex and changeable internal corrosion failure cause identification needs. Based on this, the inventors have made the present invention after further research, providing a method and device for visual analysis of multiple internal corrosion failure types of pipelines.

[0059] Embodiment 1

[0060] See also Figure 1 , is a visual analysis method for multiple internal corrosion failure types of pipelines provided in Example 1 of the present application, referring to Figure 1 As shown, the method includes:

[0061] S101: Input the internal corrosion failure image data to be identified into the internal corrosion failure type recognition model to obtain the predicted internal corrosion failure type and the feature map of the last convolution layer.

[0062] The internal corrosion failure type identification model is obtained in the following way:

[0063] Obtain pipeline corrosion failure data set;

[0064] The internal corrosion failure data set is input into the initial internal corrosion failure type recognition model for training to obtain a trained internal corrosion failure type recognition model; wherein the initial internal corrosion failure type recognition model is constructed using a Resnet18 convolutional neural network.

[0065] In an embodiment of the present application, the above-mentioned acquisition of the pipeline internal corrosion failure data set includes: acquiring pipeline internal corrosion failure images; obtaining the labeled internal corrosion failure data by classifying and labeling the pipeline internal corrosion failure images using an image labeling tool; and preprocessing the labeled internal corrosion failure data to obtain the pipeline internal corrosion failure data set.

[0066] Among them, the collection of corrosion failure images in pipelines requires standardization of specific collection. In a specific embodiment, the image collection specifications include: shooting tool pixel requirements, pixel requirements ≥ 12 million; before image collection, the failure position should be processed, and the sediment on the surface of the sample should be properly removed, including cutting waste, loose corrosion products, sediment inside and outside the pipe, sedimentary oil, etc., to reveal the failure characteristics; collect images in a dry environment with sufficient light during the day. If the light is poor, use a fill light to assist in collection; priority is given to collecting images in an indoor environment, and the shooting angle is a positive bird's-eye view angle, shooting vertically from top to bottom.

[0067] Obtain the annotated internal corrosion failure data obtained by classifying and annotating the pipeline internal corrosion failure image using an image annotation tool. The pipeline internal corrosion failure data includes: pipeline failure image data and corresponding internal corrosion types. In a specific embodiment, the collected internal corrosion failures have 6 types of internal corrosion, namely erosion corrosion, CO2-H2S corrosion, bacterial corrosion, under-scale corrosion, weld corrosion and oxygen corrosion. The collected failure data are classified and marked according to the corrosion morphology characteristics and environmental information of each corrosion type.

[0068] The annotated internal corrosion failure data is preprocessed to obtain the pipeline internal corrosion failure data set. The preprocessing operations include: image clipping: clipping the internal corrosion failure image data, including manual image clipping and center clipping, removing redundant information irrelevant to the internal corrosion failure characteristics, and unifying the image ratio; image size transformation: performing size transformation and image scaling on the internal corrosion failure image data to unify the image size; image augmentation: performing data augmentation on the internal corrosion failure image data, rotating the image by 90°, rotating by 180°, and denoising in turn, and finally obtaining the internal corrosion failure data set.

[0069] In an embodiment of the present application, the internal corrosion failure data set is input into the initial internal corrosion failure type recognition model for training to obtain a trained internal corrosion failure type recognition model. The internal corrosion image data set supports batch access, indexing, and retrieval of corrosion image data, thereby supporting the training of the model. The data output format is failure image data + failure category corresponding to the image. Among them, the image data shape is N×C×H×W, N is the number of samples, C is the number of color channels of the image, and H and W are the height and width of the image respectively.

[0070] In the embodiment of the present application, the initial internal corrosion failure type identification model is constructed by a Resnet18 convolutional neural network, and its network structure is as follows: Figure 2 In the shallow layer of the network, a convolution with a kernel size of 7*7 and a step size of 2 and a maximum pooling operation with a kernel size of 3*3 and a step size of 2 are performed. The convolution process is shown in Formula 2:

[0071] O(i,j)=(K*I)(i,j)=∑ m ∑ n I(i+m,j+n)K(m,n) Formula 2;

[0072] Here, I(i+m,j+n)K(m,n) represents the multiplication of an element I(i+m,j+n) of the input matrix and an element K(m,n) of the kernel matrix. ∑m and ∑n are accumulated horizontally and vertically in the matrix respectively.

[0073] After a shallow network, Figure 2 In the first two layers of the convolution operation, the image size is halved twice. The size of the feature map generated after convolution is calculated by formula 3:

[0074]

[0075] In the formula, since the inner erosion image size is unified to 1:1, W = H, p represents the number of padding pixels, k represents the kernel size, and s represents the stride convolution step.

[0076] The size of the feature map generated by the maximum pooling is calculated by Formula 4:

[0077]

[0078] Then the internal corrosion failure type identification model connects two Basicblock modules and three combination modules (Bottleneck module + Basicblock module). The structure of the Bottleneck module and the Basicblock module is as follows: Figure 3 As shown, the structure of the Basicblock block is as follows Figure 3 As shown in (a), for a given input, the input is sequentially subjected to convolution with a kernel size of 3 and a stride of 1, BatchNorm regularization, ReLU activation mapping, convolution with a kernel size of 3 and a stride of 1, and BatchNorm regularization. The result is then concatenated with the input X, and the concatenated result is subjected to ReLU activation mapping.

[0079] Among them, the BatchNorm2d function and ReLU are shown in Formula 5 and Formula 6:

[0080]

[0081]

[0082] Among them, mean(X) and Var(X) are the mean and variance of the current feature map, x is the value of each element in the feature map, eps is a constant, generally set to 1e-5 for the stability of calculation, gamma and beta are learnable parameters. The function of the ReLU function is to make the value of the element in the feature map non-negative.

[0083] The structure of the Bottleneck block is as follows Figure 3 As shown in (b), for a given input, the input is sequentially subjected to convolution with a kernel size of 3 and a stride of 2, BatchNorm regularization, ReLU activation mapping, convolution with a kernel size of 3 and a stride of 1, and BatchNorm regularization. After processing, the size of the input becomes half of the original. In order to be connected with the input later, the input is subjected to convolution with a kernel size of 1 and a stride of 2, and BatchNorm regularization. The results of the two parts are then concatenated, and the concatenated results are subjected to ReLU activation mapping.

[0084] Therefore, after the above convolution module structure, the size of the feature map changes from 56*56 to 56*56, 56*56, 28*28, 14*14, and 7*7. Then, an average pooling process with an output of 1*1 is performed to change the size of the feature map to 1*1. Then, the feature map is converted into a one-dimensional vector and a fully connected layer process with an output of 6 is performed, as shown in Formula 7:

[0085] y=f(ω1x1+ω2x2+...+ω n x n +b) Formula 7;

[0086] Among them, x1, x2...x n is the value of the previous layer of neurons, ω1, ω2...ω n , b is the weight coefficient and deviation between the ith neuron in the next layer and all neurons in the previous layer. At the end of the network is the softmax layer, as shown in Formula 8. The vector of length 6 is converted into the probabilities corresponding to six types (0- erosion corrosion, 1- CO2 + H2S corrosion, 2- bacterial corrosion, 3- under-scale corrosion, 4- weld corrosion, 5- oxygen corrosion), among which the type with the highest probability is the final prediction result of the model.

[0087]

[0088] Among them, x i is the value of the i-th element of the output layer, and C is the number of categories.

[0089] In the embodiment of the present application, after the construction of the initial internal corrosion failure type identification model is completed, the internal corrosion failure data set is input into the initial internal corrosion failure type identification model for training. The training process is a process of continuous adjustment and optimization of parameters in the model, which is iteratively optimized through gradient descent and back propagation algorithms. The gradient descent algorithm is shown in Formula 9:

[0090]

[0091] First, the final error term (function to be optimized) is calculated, where y i represents the target value, represents the actual value calculated, and then the mean square error is used to represent the error term, as shown in Formula 10:

[0092]

[0093] Next, take the derivative of E, as shown in Formula 11, and substitute the derivative into Formula 9 to get the updated value of the weight

[0094]

[0095] During the training process, a five-fold cross validation training method is used, so for each training, 80% of the data is used for training and 20% of the data is used for testing. In a specific embodiment, the learning rate setting, the number of iterations, the batch training size, the optimization function, the loss function, the training strategy and the evaluation index are shown in Table 1, and the hardware model and software version used are shown in Table 2.

[0096] Table 1 Parameter settings and evaluation indicators of CNN classification model

[0097]

[0098] Table 2 Hardware model, software version

[0099] GPU Model Python Version Pytorch Version NVIDIA GeForce RTX3090 Python 3.9 Torch1.9.1+cu111

[0100] In the embodiment of the present application, the process of training the internal corrosion failure type recognition model also includes testing the model performance. The model performance test is based on the parameters updated and optimized in the above model training process, inputting new test samples, and predicting the internal corrosion category corresponding to each failure image. The preset condition for stopping the training can be to limit the number of iterations or whether the model performance test index reaches a threshold. The prediction accuracy (Accuracy), precision (Precision), recall (Recall) and F1 score (F1 score) of the model are calculated according to formulas 12 to 15:

[0101]

[0102]

[0103]

[0104]

[0105] Among them, TP is a true positive example, TN is a true negative example, FP is a false positive example, and FN is a false negative example. The preset condition for stopping training can be to limit the number of iterations or whether the model performance test index reaches a threshold. The training of the internal corrosion failure type recognition model is completed. The trained internal corrosion failure type recognition model is obtained.

[0106] In an embodiment of the present application, after the trained internal corrosion failure type recognition model is obtained through the above steps, the internal corrosion failure image data to be identified is input into the trained internal corrosion failure type recognition model to obtain the predicted internal corrosion failure type of the internal corrosion failure image to be identified, as well as the feature map of the last convolutional layer, to facilitate subsequent Grad-CAM calculations.

[0107] S102: Use Grad-CAM to analyze the predicted internal corrosion failure type and the feature map of the last convolution layer to obtain a feature area thermal map.

[0108] In the embodiment of the present application, Grad-CAM is used to analyze and predict the internal corrosion failure type and the feature map of the last convolution layer to obtain a feature area thermal map, including:

[0109] Back propagation is performed according to the predicted internal corrosion failure type and the feature map of the last convolution layer to calculate and obtain the gradient information of the last convolution layer in the internal corrosion failure type recognition model;

[0110] Performing global average pooling on the gradient information of the last convolutional layer to obtain the weight information of the last convolutional layer;

[0111] According to the weight information of the last convolutional layer and the feature map of the last convolutional layer, the feature area heat map is obtained by calculating according to the following formula 1 through ReLU operation:

[0112]

[0113] In the formula, is the weight information, A k is the feature map of the last convolutional layer, k is the number of channels, and c is the predicted internal corrosion failure type.

[0114] In the embodiment of the present application, Grad-CAM represents Gradient-weighted Class Activation Mapping, i.e., gradient-weighted class activation mapping. When predicting an image, Grad-CAM uses the predicted internal corrosion failure type obtained by back propagation to the parameter layer (such as the last convolutional layer) to be visualized, obtains a gradient matrix of the same size as the output feature layer of the parameter layer, performs global average pooling of the gradient matrix in the spatial dimension, obtains weight information (the length of the vector is the number of channels of the feature layer), and uses the weight information to weight each channel of the feature map to obtain a heat map. The steps specifically include: selecting the target internal corrosion type for which the heat map explanation is desired; obtaining the predicted internal corrosion failure type obtained in step S101 and the weight information of the last convolutional layer; obtaining the predicted score of the target category, i.e., the value of the target category of the softmax layer, indicating the confidence of the internal corrosion failure type recognition model for the category; calculating the gradient of the target category score to the output of the last convolutional layer to measure the degree of influence of each channel on the target type; multiplying the gradient by the feature map of the last convolutional layer to obtain the weight information of each channel, indicating the importance of the channel, calculated according to the following formula 16:

[0115]

[0116] In the formula, is the weight information, A k is the feature map of the last convolutional layer, k is the number of channels, c is the predicted internal corrosion failure type, i and j are the width and height of the feature map, and Z is equal to the width × height of the feature map;

[0117] The weights of each channel are summed to obtain the total importance of the channel; the ReLU operation removes the weights of negative values ​​and retains the part that contributes to the explanation of the basis for target category recognition; the weights after the ReLU operation are scaled to ensure that the range is appropriate; the scaled weights are multiplied by the output of the last convolutional layer to obtain a heat map, which is superimposed on the input image to show the area that the model focuses on. The heat map calculation formula is shown in Formula 1.

[0118] S103: Based on the predicted internal corrosion failure type and the characteristic region thermal map, the contribution degree of the features of each characteristic region to the recognition result is obtained.

[0119] In the embodiment of the present application, the internal corrosion failure image to be identified is subjected to data collection, preprocessing, and input into the internal corrosion failure type identification model to obtain the internal corrosion failure type identification visualization flow diagram of the thermal map as shown in FIG. Figure 4 As shown in the figure, the class activation map (CAM) is the thermal map corresponding to the predicted internal corrosion failure type. Figure 4The selected heatmap interpretation target is to predict the internal corrosion failure type, which is Class 1 erosion corrosion with the highest probability obtained by softmax. The heatmap shows a visualization effect that highlights the areas that the model focuses on in the prediction. Figure 4 In the figure, the dark area in the center of the picture represents the basis for the model to predict the internal corrosion failure type, showing the contribution of the features of each feature area to the identification result.

[0120] Domain experts or field workers can view the heat map and understand the basis for the model's predictions by analyzing the areas that the model focuses on. For example, if the heat map shows that the model has a stronger focus on the corrosion areas, then this can be considered as identification evidence because the model believes that these areas are associated with specific types of internal corrosion failures. Using heat maps to explain the prediction results of the convolutional neural network model in the task of internal corrosion failure type identification can help understand the model's decision-making process and provide a visual way to determine the identification basis, thereby enhancing the interpretability and credibility of the model. This is very valuable in the decision-making and diagnosis process in the practical application of internal corrosion failure type identification in oil and gas field pipelines.

[0121] As a specific example of the embodiment of the present application, the internal corrosion failure image to be identified is subjected to data collection, preprocessing, and input into the internal corrosion failure type identification model to obtain a visualization flow chart of the internal corrosion failure type identification of the thermal map. Figure 4 shown.

[0122] In a specific embodiment, the internal corrosion failure data of the pipeline is obtained: a total of 6 types of internal corrosion failure data of the pipeline are collected, namely, erosion corrosion, CO2-H2S corrosion, bacterial corrosion, under-scale corrosion, weld corrosion and oxygen corrosion. Among them, 10 failure samples are collected for each corrosion type, and 2 to 6 macroscopic corrosion morphology images (taken by mobile phone) are collected on each failure sample, and a total of 240 macroscopic corrosion images are collected. Classification of internal corrosion failure images: A total of 240 pieces of internal corrosion failure data of 6 types are collected, among which 40 pieces are each for erosion corrosion, CO2-H2S corrosion, bacterial corrosion, under-scale corrosion, weld corrosion and oxygen corrosion. The collected failure data are classified and marked according to the corrosion morphological characteristics of each corrosion type and its service environment information. The marking method is: category-sample number-image number, such as the first failure image on the first failure sample of erosion corrosion is recorded as 0-0-0, and so on.

[0123] Internal corrosion failure image preprocessing: The preprocessing operations of the internal corrosion failure image data specifically include: manual image cropping, center image cropping, image size transformation, and image augmentation. Manual image cropping Image cropping: Remove redundant information that is not related to the pipeline failure characteristics, such as background that is not related to the pipeline, cutting waste, loose corrosion products, sediment deposited inside and outside the pipe, deposited oil, etc. Center cropping of the image: Center crop the manually cropped image to obtain an image of size n*n. Image size transformation: Perform the same image scaling on all images to obtain an image of size 224*224. Image augmentation: For an original failure image, the pipeline failure image is rotated 90°, rotated 180°, and denoised in turn to obtain 4 corrosion failure images. The augmented images have the same label.

[0124] In a specific embodiment, the present application uses the pytorch deep learning framework to build a network. First, an internal corrosion failure image dataset is established, and the torch.utils.data.Dataset function is combined with the torch.utils.data.DataLoader function to obtain a data iterator. During each training, this iterator is used to output each batch of data, and the data can be preprocessed or enhanced accordingly during output. The data output format is failure image data + the failure category corresponding to the image. Among them, the image data shape is (32,3,224,224), 32 is the number of samples, 3 is the number of color channels of the image, and 224 and 224 are the height and width of the image, respectively.

[0125] A ResNet18 convolutional neural network is constructed as an internal corrosion failure type recognition model; the input shape is (32, 3, 224, 224), 32 is the number of batch training samples, 3 is the number of color channels, that is, color images, and 224*224 is the size of the feature map. In the shallow layer of the network, a convolution with a kernel size of 7*7 and a step size of 2 is performed; BatchNorm2d(64) regularization processing, 64 is the number of feature maps; a maximum pooling operation with a kernel size of 3*3 and a step size of 2. After the shallow network, the size of the image is calculated by formula 3, and the size is 56*56. This is followed by two Basicblock modules and three (Bottleneck module + Basicblock module) combined models. The size of the feature map remains unchanged after being processed by the Basicblock module; the size of the feature map becomes half of the original after being processed by the Bottleneck module. Therefore, after two Basicblock modules and three (Bottleneck module + Basicblock module) combination modules, the size of the feature map changes from 56*56 to 56*56, 56*56, 28*28, 14*14, and 7*7. Then, an average pooling process with an output of 1*1 is performed to change the size of the feature map to 1*1. Then, the feature map is converted into a one-dimensional vector and a fully connected layer process with an output of 6 is performed, as shown in Formula 17:

[0126] y=f(ω1x1+ω2x2+...+ω 512 x 512 +b) Formula 17;

[0127] Among them, x1, x2...x 512 is the value of the previous layer of neurons; 512 is the length of the one-dimensional vector after the feature map is transformed; ω1, ω2...ω 512 , b is the weight coefficient and deviation between the ith neuron in the next layer and all neurons in the previous layer. Then the 6 values ​​of the output layer are calculated 6 times by formula 17. At the end of the network is the softmax layer, which converts the vector of length 6 into the probabilities corresponding to the six categories (0-erosion corrosion, 1-CO2+H2S corrosion, 2-bacterial corrosion, 3-under-scale corrosion, 4-weld corrosion, 5-oxygen corrosion). The final prediction result of the model is the type with the highest probability. If the 6 values ​​of the output layer are -1, 2, 0.3, -1.2, 10, 5, then they are calculated by formula 8, and the calculation process is shown in formulas 18 to 23:

[0128]

[0129]

[0130]

[0131]

[0132]

[0133]

[0134] The results are 0- erosion corrosion, 1- CO2 + H2S corrosion, 2- bacterial corrosion, 3- under-scale corrosion, 4- weld corrosion, 5- oxygen corrosion, and the probabilities are 0, 0.0003, 0.0001, 0, 0.9929, and 0.0067 respectively. The prediction result of the model is weld corrosion.

[0135] In the training process of the internal corrosion failure type recognition model, the optim.SGD optimization function, nn.CrossEntropyLoss loss function and the five-fold cross validation method are used. Batch training is adopted, and the size is 32. The learning rate is set in a step-by-step decreasing manner, with an initial value of 0.02 and a decrease of 50% every 10 training times. After a total of 100 training times, the loss eventually dropped to about 0.02, and the model began to gradually converge after about 50 iterations of training. After the training is completed, the model parameters are saved to obtain the trained internal corrosion failure type recognition model to test the performance of the model on the test set.

[0136] The parameters saved in the model training were used to evaluate the performance of the model on the test set using accuracy, precision, recall, and F1 score. The calculated accuracy was 0.92, and the precision, recall, and F1 scores of erosion corrosion were 0.89, 0.86, and 0.87, respectively; the precision, recall, and F1 scores of CO2+H2S corrosion were 0.89, 0.89, and 0.89, respectively; the precision, recall, and F1 scores of bacterial corrosion were 0.94, 0.88, and 0.91, respectively; the precision, recall, and F1 scores of under-deposit corrosion were 0.84, 0.96, and 0.90, respectively; the precision, recall, and F1 scores of weld corrosion were 0.93, 0.90, and 0.91, respectively; and the precision, recall, and F1 scores of oxygen corrosion were 1.00, 1.00, and 1.00, respectively.

[0137] Interpretability analysis of the internal corrosion failure type recognition model. That is, the Grad-CAM method is used to visualize the weight parameters in the last convolution layer of ResNet18 (because the convolution layer retains spatial information and captures high-level visual structures in deeper layers) to display the important areas in the failure image that determine the corresponding internal corrosion type. This method can be used as a basis for distinguishing multiple types of internal corrosion. And the contribution of the features in each area to the recognition result is displayed in the form of a heat map on the original internal corrosion failure image. The darker the color, the greater the contribution of the features in the area to the recognition result.

[0138] Embodiment 2

[0139] Based on the same inventive concept, the present application also provides a device for visually analyzing various internal corrosion failure types of pipelines, referring to Figure 5 As shown, the device comprises:

[0140] The first prediction module 101 is used to input the internal corrosion failure image data to be identified into the internal corrosion failure type recognition model to obtain the predicted internal corrosion failure type and the feature map of the last convolution layer;

[0141] A first analysis module 102 is used to use Grad-CAM to analyze the predicted internal corrosion failure type and the feature map of the last convolution layer to obtain a feature area thermal map;

[0142] The second analysis module 103 is used to obtain the contribution of the features of each characteristic area to the identification result based on the predicted internal corrosion failure type and the characteristic area thermal map; wherein the internal corrosion failure type identification model is obtained by:

[0143] The first acquisition module 104 is used to acquire a pipeline corrosion failure data set;

[0144] The first training module 105 is used to input the internal corrosion failure data set into the initial internal corrosion failure type recognition model for training to obtain a trained internal corrosion failure type recognition model; wherein the initial internal corrosion failure type recognition model is constructed using a Resnet18 convolutional neural network.

[0145] Embodiment 3

[0146] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for visualizing and analyzing various internal corrosion failure types of pipelines as described in the first embodiment above is implemented.

[0147] Embodiment 4

[0148] Based on the same inventive concept, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for visualizing and analyzing multiple internal corrosion failure types of pipelines as described in the first embodiment above is implemented.

[0149] Embodiment 5

[0150] Based on the same inventive concept, an embodiment of the present application also provides a computer program product comprising instructions. When the computer program product is run on a computer device, the computer device executes the method for visualizing and analyzing multiple internal corrosion failure types of pipelines as described in the first embodiment above.

[0151] Embodiment 6

[0152] Based on the same inventive concept, an embodiment of the present application further provides a chip, the chip comprising a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run a computer program or instruction to implement the visualization analysis method of multiple internal corrosion failure types of pipelines as described in the above embodiment 1.

[0153] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiment when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0154] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing the computer program product.

[0155] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0156] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0157] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A visual analysis method for multiple internal corrosion failure types of pipelines, characterized in that: include: Input the internal corrosion failure image data to be identified into the internal corrosion failure type recognition model to obtain the predicted internal corrosion failure type and the feature map of the last convolution layer; Using Grad-CAM to analyze the predicted internal corrosion failure type and the feature map of the last convolution layer to obtain a feature area thermal map; Based on the predicted internal corrosion failure type and the characteristic area thermal map, the contribution degree of the characteristics of each characteristic area to the recognition result is obtained; The internal corrosion failure type identification model is obtained in the following way: Obtaining pipeline corrosion failure data set; The internal corrosion failure data set is input into the initial internal corrosion failure type recognition model for training to obtain a trained internal corrosion failure type recognition model; wherein the initial internal corrosion failure type recognition model is constructed using a Resnet18 convolutional neural network.

2. The method according to claim 1, characterized in that The Grad-CAM is used to analyze the predicted internal corrosion failure type and the feature map of the last convolution layer to obtain a feature area thermal map, including: Back propagation is performed according to the predicted internal corrosion failure type and the feature map of the last convolution layer to calculate and obtain the gradient information of the last convolution layer in the internal corrosion failure type recognition model; Performing global average pooling on the gradient information of the last convolutional layer to obtain the weight information of the last convolutional layer; According to the weight information of the last convolutional layer and the feature map of the last convolutional layer, the feature area heat map is obtained by calculating according to the following formula 1 through ReLU operation: In the formula, is the weight information, A k is the feature map of the last convolutional layer, k is the number of channels, and c is the predicted internal corrosion failure type.

3. The method according to claim 1, characterized in that The inputting the internal corrosion failure data set into the initial internal corrosion failure type identification model for training to obtain a trained internal corrosion failure type identification model comprises: The internal corrosion failure data set is input into the initial internal corrosion failure type identification model using a five-fold cross validation method to obtain prediction data; The parameters of the initial internal corrosion failure type identification model are adjusted according to the predicted data by using a gradient descent method until a preset condition is reached, thereby obtaining a trained internal corrosion failure type identification model.

4. The method according to claim 1, characterized in that The step of obtaining a pipeline corrosion failure data set includes: Acquire corrosion failure images inside pipelines; Obtaining annotated internal corrosion failure data obtained by classifying and annotating the pipeline internal corrosion failure image using an image annotation tool; The annotated internal corrosion failure data is preprocessed to obtain the pipeline internal corrosion failure data set.

5. The method according to claim 4, characterized in that The preprocessing of the annotated internal corrosion failure data to obtain the internal corrosion failure data set includes: The annotated internal corrosion failure data is trimmed to remove redundant information irrelevant to the internal corrosion failure characteristics, and the image ratio is unified to 1:1; Performing size transformation on the annotated internal corrosion failure data to unify the image size; Data augmentation is performed on the annotated internal corrosion failure data to obtain the internal corrosion failure data set.

6. A visual analysis device for various internal corrosion failure types of pipelines, characterized in that: include: The first prediction module is used to input the internal corrosion failure image data to be identified into the internal corrosion failure type recognition model to obtain the predicted internal corrosion failure type and the feature map of the last convolution layer; A first analysis module is used to use Grad-CAM to analyze the predicted internal corrosion failure type and the feature map of the last convolution layer to obtain a feature area thermal map; The second analysis module is used to obtain the contribution of the characteristics of each characteristic area to the identification result based on the predicted internal corrosion failure type and the characteristic area thermal map; wherein the internal corrosion failure type identification model is obtained by the following method: The first acquisition module is used to acquire a pipeline corrosion failure data set; The first training module is used to input the internal corrosion failure data set into the initial internal corrosion failure type recognition model for training to obtain a trained internal corrosion failure type recognition model; wherein the initial internal corrosion failure type recognition model is constructed using a Resnet18 convolutional neural network.

7. A computer-readable storage medium storing instructions, which, when executed on a terminal, enable the terminal to execute the method for visually analyzing multiple internal corrosion failure types of pipelines as claimed in any one of claims 1 to 5.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for visually analyzing multiple internal corrosion failure types of pipelines as claimed in any one of claims 1 to 5 is implemented.

9. A computer program product comprising instructions, which, when executed on a computer device, enables the computer device to execute the method for visually analyzing multiple internal corrosion failure types of a pipeline as claimed in any one of claims 1 to 5.

10. A chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a computer program or instruction to implement the method for visually analyzing multiple internal corrosion failure types of a pipeline as claimed in any one of claims 1 to 5.

Images