Pipeline defect assessment method and system based on multi-source data fusion

Abstract

The invention provides a pipeline defect assessment method and system based on multi-source data fusion, and the method comprises the following steps: collecting the data of the operation state of a pipeline body through a related collection device, and marking the state and fault type of the pipeline, so as to construct a training set and a test set; and preprocessing the static data of the pipeline, wherein the preprocessing comprises data cleaning, standardization and feature extraction. And optimizing the extreme learning machine by using a bird flock algorithm, analyzing static data and evaluating pipeline defects. Preprocessing the image data of the pipeline body, wherein the preprocessing comprises image conversion, enhancement and denoising; and evaluating the circumferential weld defect based on the pre-trained VGG16 model. And performing decision-level fusion on a defect evaluation conclusion obtained by static data analysis and circumferential weld image analysis by using a D-S evidence reasoning algorithm to obtain a final evaluation result. The method effectively improves the accuracy and reliability of pipeline defect evaluation, and has wide application prospects and important practical value.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-source data fusion, and relates to a pipeline defect assessment method and system based on multi-source data fusion. Background Art

[0002] As the core infrastructure for the transportation of natural gas and oil, the complexity and importance of pipelines are self-evident. They not only span a vast area, but also often face the test of extreme natural conditions, such as high temperature, low temperature, strong wind, heavy rain, earthquake, etc. These natural factors, coupled with the long-term effects of high pressure and corrosive media inside the pipeline, make the pipeline body very prone to various types of defects and failures, including but not limited to corrosion, cracks, wear, deformation, and weld defects. If these potential safety hazards are not discovered and properly handled in a timely manner, they will not only cause material leakage, environmental pollution and waste of resources, but may also directly lead to major safety accidents such as fire and explosion, seriously threatening the safety of people's lives and property and social stability.

[0003] Therefore, it is particularly important to carry out efficient and accurate defect diagnosis of the pipeline body. This requires the use of advanced detection technologies and methods, such as ultrasonic testing, magnetic particle testing, eddy current testing, radiographic testing, as well as the rapid development of drone inspection, intelligent robot inspection, and fiber optic sensor monitoring technology in recent years. These technologies can achieve a comprehensive scan of the inner and outer walls of the pipeline, accurately identify and locate the defect location, assess the severity of the defect, and provide a scientific basis for subsequent maintenance decisions. It is also essential to establish a complete pipeline management system. The system should integrate functions such as data collection, analysis and early warning, and emergency response. By real-time monitoring of the pipeline operation status, analyzing operation data, and predicting potential risks, early detection, early reporting, and early handling of faults can be achieved. In addition, strengthening personnel training and improving the professional skills and emergency handling capabilities of inspection personnel are also key links to ensure the safe operation of pipelines.

[0004] With the rapid development of technologies such as the Internet of Things, big data, and artificial intelligence, pipeline defect diagnosis is gradually moving towards intelligence and automation. By building a digital twin model of the pipeline, simulating the operating status of the pipeline, and combining historical fault data, it is possible to more accurately predict and prevent faults and achieve refined management of pipeline operation and maintenance. At present, traditional pipeline defect diagnosis methods mainly rely on manual inspection and experience judgment, lacking automation and intelligence. These methods are not only time-consuming and labor-intensive, but also easily affected by human factors, resulting in inaccurate or untimely diagnostic results. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a pipeline defect assessment and analysis method and system based on multi-source data fusion. The method is applied to the daily operation and maintenance of smart pipeline networks. Its high efficiency and intelligence will greatly improve the operation and maintenance efficiency and safety of smart pipeline networks, thereby effectively reducing maintenance costs and risks.

[0006] The present invention is achieved through the following technical solutions:

[0007] A pipeline defect assessment method based on multi-source data fusion includes the following steps:

[0008] Obtain data on the operation status of the pipeline itself, including static data and image data;

[0009] Preprocess static data and image data;

[0010] Based on the optimization of the extreme learning machine based on the bird flock algorithm, the pre-processed static data is analyzed and evaluated to obtain the pipeline defect assessment results of the static data;

[0011] The preprocessed image data is analyzed and evaluated based on the VGG16 model to obtain the pipeline girth weld defect evaluation results of the image data;

[0012] Based on the DS evidence reasoning algorithm, the pipeline defect assessment results of static data and the pipeline girth weld defect assessment results of image data are fused at the decision level to obtain the final pipeline defect assessment results.

[0013] Preferably, each data in the data of the pipeline main body operation status is marked with the pipeline status and the fault type;

[0014] The pipeline status includes normal, slight defect and serious defect; the fault type includes corrosion, crack and deformation.

[0015] Preferably, the static data preprocessing process is:

[0016] The welding process data and characteristic weld data in the static data are standardized using Z-score, and the welding process data and characteristic weld data are converted into a standard normal distribution with zero mean and unit variance, and then normalized using Min-Max.

[0017] Preferably, the process of preprocessing the image data is:

[0018] Preprocessing the girth weld image in the pipeline body, including data enhancement, denoising and image edge enhancement.

[0019] The data enhancement method of generative adversarial network is used to enhance the data of girth weld images.

[0020] The data enhancement process of the girth weld image is to convert the girth weld image into RGB channels, resize it to 224×224 pixels, scale the pixel intensity to the range of 0-1, and perform data enhancement through random geometric transformation;

[0021] The denoising process of the girth weld image is to use a Gaussian filter to remove the noise of the girth weld image;

[0022] The edge processing of the enhanced image of the ring weld is to use the Sobel operator for edge detection, and the defective edge of the ring weld is highlighted by calculating the difference between the gray value of each pixel in the image and the gray value of its surrounding pixels.

[0023] Preferably, the specific process of optimizing the extreme learning machine based on the bird flock algorithm to analyze the static data and evaluate the pipeline defects is as follows:

[0024] The initial weights and biases of the extreme learning machine are optimized by the bird flock algorithm. The mean square error of the extreme learning machine on the training set is defined as the fitness function. The fitness function value is minimized by iteratively optimizing the position of individual birds to achieve the optimization of the extreme learning machine.

[0025] The preprocessed static data is used as the feature input of the extreme learning machine, and the defect assessment result of the pipeline is used as the label of the extreme learning machine. The input weight, bias range, number of hidden layer neurons, number of bird flocks, and maximum number of iterations of the extreme learning machine are set; the number of input neurons is the actual number of numerical categories of welding process data and characteristic weld data; the number of output neurons is 4, which represents the category of pipeline defects after fusion. The static data is trained, and after the training is completed, the static feature data of the pipeline body is input into the extreme learning machine to obtain the assessment result of pipeline defects;

[0026] Among them, pipeline defect categories include serious defects, relatively serious defects, minor defects and no defects.

[0027] Preferably, the specific process of optimizing the extreme learning machine is:

[0028] The input weights and biases of the extreme learning machine are used to initialize each bird, and then the regression error fitness value corresponding to the extreme learning machine is calculated based on the parameters of each bird, and the optimal fitness value and the corresponding weights and biases are recorded. After executing the flight behavior, the fitness value is calculated and updated. After the update iteration, the optimal fitness value is assigned to the extreme learning machine.

[0029] Preferably, a regularization parameter is introduced into the extreme learning machine to balance the error matrix and the output matrix, and a sample weighting matrix is ​​introduced into the objective function. The sample weighting matrix is ​​constructed using a Gaussian kernel function, and weighted processing is performed according to the similarity between the training sample and the sample to be predicted.

[0030] Preferably, the pipeline girth weld defects in the image data are evaluated based on the VGG16 model, specifically:

[0031] The features of girth weld film data and girth weld defect image data in the image data are extracted through the VGG16 network. The number of features is reduced by the PCA dimensionality reduction technology. The girth weld defect features and the girth weld film features are fused with the attention feature fusion method to form a unified fusion feature set. The fused features will be output through a two-layer fully connected network to obtain the pipeline girth weld defect assessment results.

[0032] Preferably, the defect assessment conclusions obtained from static data and image data are fused at the decision level based on the DS evidence reasoning algorithm, specifically:

[0033] Based on the DS evidence reasoning algorithm, the recognition framework Θ is defined to represent all possible states. According to the pipeline defect assessment conclusions of static data analysis and girth weld image analysis, basic probabilities are assigned to each state, that is, the result of static data analysis is m1, and the result of girth weld image analysis is m2. The conflict coefficient K represents the degree of inconsistency between the two evidences. The pipeline defect assessment conclusions of static data analysis and girth weld image analysis are combined by calculating the conflict coefficient K and Dempster's combination rule to obtain the fused assessment result.

[0034] Among them, Θ includes no defects, slight defects, more serious defects and serious defects.

[0035] A pipeline defect assessment system based on multi-source data fusion, comprising:

[0036] Data acquisition module, preprocessing module, static data analysis and evaluation module, image data analysis and evaluation module, and decision-level fusion evaluation module;

[0037] The data acquisition module is used to acquire the data of the pipeline body operation status, including static data and image data;

[0038] The preprocessing module is used to preprocess static data and image data;

[0039] The static data analysis and evaluation module is used to optimize the extreme learning machine based on the bird flock algorithm, analyze and evaluate the pre-processed static data, and obtain the pipeline defect evaluation results of the static data;

[0040] The image data analysis and evaluation module is used to analyze and evaluate the preprocessed image data based on the VGG16 model to obtain the pipeline girth weld defect evaluation results of the image data;

[0041] The decision-level fusion evaluation module is used to perform decision-level fusion on the pipeline defect evaluation results of static data and the pipeline girth weld defect evaluation results of image data based on the DS evidence reasoning algorithm to obtain the final pipeline defect evaluation results.

[0042] Compared with the prior art, the present invention has the following beneficial technical effects:

[0043] The present invention utilizes extreme learning machine (ELM) and bird swarm algorithm (BSA) to process and optimize the multi-sensor data fusion process in pipeline defect assessment. Based on the bird swarm algorithm to optimize the extreme learning machine, the static data is analyzed and the pipeline defects are evaluated; based on the VGG16 model, the pipeline girth weld defects in the image data are evaluated; based on the DS evidence reasoning algorithm, the pipeline defect assessment conclusions obtained from the static data and image data are fused at the decision level to obtain the final pipeline defect assessment result. The extreme learning machine (ELM) is used for model training, and the ELM has the advantages of fast training speed and good generalization performance. This method is applied to the daily operation and maintenance of smart pipe networks. Its high efficiency and intelligence will greatly improve the operation and maintenance efficiency and safety of smart pipe networks, thereby effectively reducing maintenance costs and risks.

[0044] Furthermore, in order to improve the generalization ability of the extreme learning machine (ELM), a regularization parameter is introduced to balance the error matrix and the output matrix, and a sample weighting matrix is ​​introduced into the objective function. The Gaussian kernel function is used to construct the sample weighting matrix, and weighted processing is performed according to the similarity between the training samples and the samples to be predicted to reduce the weight of the outliers.

[0045] Furthermore, the bird flock algorithm defines the mean square error (MSE) of ELM on the training set as the fitness function, and minimizes the value of the fitness function by iteratively optimizing the position of individual birds, thereby improving the accuracy and efficiency of pipeline defect assessment.

[0046] Furthermore, the Sobel operator is used for edge detection. By calculating the difference between the grayscale value of each pixel in the image and the grayscale values ​​of its surrounding pixels, the edge of the weld defect is highlighted, thereby improving the image contrast and the visibility of the defect.

[0047] Furthermore, the present invention adopts a data enhancement method based on a generative adversarial network (GAN), measures the distance between the generated sample distribution and the real sample distribution by the Wasserstein distance, optimizes the performance of the generator and the discriminator, and generates new data that is closer to the real sample, thereby improving the training effect and generalization ability of the model.

[0048] Furthermore, the present invention uses a pre-trained VGG16 network to extract features of girth weld film data and girth weld defect image data, and reduces the number of features through PCA dimensionality reduction technology to reduce computational costs and avoid overfitting problems. The attention feature fusion method (AFF) is used for feature-level fusion, and a multi-scale channel attention module is used to extract channel attention weights of global and local features to better fuse semantically and scale-inconsistent features. The fused features are passed through a two-layer fully connected network to output the final pipeline girth weld defect assessment results. This design enables the model to improve the accuracy of girth weld defect assessment through feature fusion while retaining the original information of the two images. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The pipeline defect assessment flow chart based on multi-source data fusion of the present invention;

[0050] Figure 2 It is a network structure diagram of the VGG16 model of the present invention;

[0051] Figure 3 This is a structural diagram of the multi-scale channel attention module of the present invention;

[0052] Figure 4 It is a network structure diagram of the feature-level fusion model of the present invention;

[0053] Figure 5 It is a schematic diagram of the decision-level fusion solution of the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described in detail below in conjunction with specific embodiments, which are intended to explain the present invention rather than to limit it.

[0055] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0056] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0057] Step 1: Data acquisition and storage

[0058] For data collection, high-precision testing equipment such as ultrasonic detectors, magnetic particle detectors, eddy current detectors, etc. are used to provide detailed information on the inside and outside of the pipeline, including but not limited to changes in wall thickness, corrosion degree, cracks, etc., to ensure that the collected data is widely representative, covering various possible pipeline states (normal, minor defects, serious defects, etc.) and different types of faults (such as corrosion, cracks, deformation, etc.), so that the model can learn the characteristics under different conditions. At the same time, record the environmental conditions during collection, such as temperature, humidity, pressure, etc.

[0059] Data annotation: experienced professionals annotate the collected data to ensure the accuracy and reliability of the annotations. A set of clear annotation rules and standards are established to ensure that all personnel involved in the annotation work follow the same set of standards and improve the consistency of the annotations. The quality of the annotations is checked regularly to promptly identify and correct errors or inconsistencies.

[0060] Data storage: various forms of collected data (such as pipeline image data, pipeline static data, etc.) are stored using databases (such as MySQL, NoSQL) or distributed storage systems to ensure data security and scalability. An efficient data management system is established to facilitate data retrieval and use, as well as data version management and updates.

[0061] Step 2: Preprocess the static data of the pipeline

[0062] The welding process data and characteristic weld data are standardized using Z-score, and the data are converted into a standard normal distribution with zero mean and unit variance. The Z-score standardization formula is as follows:

[0063]

[0064] In the formula, μ is the mean and σ is the standard deviation.

[0065] Then use Min-Max to normalize and scale the data to the [0,1] interval. The Min-Max normalization formula is as follows:

[0066]

[0067] In the formula, min(x) represents the minimum value and max(x) represents the maximum value.

[0068] Step 3: Use the bird flock algorithm to optimize the extreme learning machine to analyze the static data of the pipeline and evaluate the pipeline defects

[0069] Extreme Learning Machine (ELM) is a new learning algorithm for training a single hidden layer feedforward neural network. Its core idea is to randomly initialize the weights and biases from the input layer to the hidden layer, and then train the weights from the hidden layer to the output layer by the least squares method. Extreme learning machines are simpler than traditional neural networks, have faster training speeds, and have better generalization performance. However, the ELM model is prone to overfitting in practical applications. In order to improve the generalization ability of the extreme learning machine, the present invention introduces a regularization parameter to balance the error matrix and the output matrix. At the same time, in order to consider the impact of different samples on the output of the extreme learning machine, a sample weighting matrix is ​​introduced into its objective function, and a Gaussian kernel function is used to construct the sample weighting matrix. According to the similarity between the training sample and the sample to be predicted, the predicted sample is also weighted. This method can reduce the weight of outliers, thereby improving the learning ability and generalization performance of the extreme learning machine.

[0070] The objective function L of the extreme learning machine ELM for

[0071]

[0072] Among them, T, H, β are the expected output matrix, hidden layer matrix and output layer weight respectively, λ is the regularization parameter, is the square of the L2 norm of the error matrix Hβ-T; V is the weighting matrix, which is composed of Gaussian kernel functions:

[0073]

[0074] Among them, x is the sample to be predicted, x i is the training set sample; δ is the Gaussian kernel function parameter.

[0075] Therefore, the optimal solution of β is

[0076]

[0077] Among them, N is the number of samples, L is the number of neurons, and I is the unit matrix.

[0078] When using ELM as a multi-sensor data fusion model, since the weights and biases between the input layer and the hidden layer of the model are randomly generated, this may lead to the multi-sensor data fusion model having poor generalization ability and insufficient stability. Therefore, in order to improve the generalization ability and stability of the fusion model, the present invention continues to introduce the Bird Swarm Algorithm (BSA) in this article to optimize the initial values ​​of the weights and bias parameters of the model.

[0079] The Bird Swarm Algorithm is a new type of bio-inspired algorithm proposed in 2015. It solves the optimization problem by imitating the social behavior of birds and their foraging behavior, vigilance behavior and flight behavior during interaction. When designing the Bird Swarm Algorithm, the present invention defines the mean square error (MSE) of ELM on the training set as the fitness function, which is used to evaluate the position of each individual bird (i.e., each set of weights and biases). According to the update rule of the Bird Swarm Algorithm, the position of the individual bird is iteratively optimized to minimize the value of the fitness function, and the optimal weights and biases finally found are used to train the final ELM model.

[0080] Assume the size of the flock is N, and the dimension of the flight space is D. represents the position of the bird, FQ represents the flight interval, and P is the foraging probability. If the random number between (0,1) is less than P, the bird will choose to forage, otherwise, the bird will continue to be vigilant.

[0081] (1) Foraging behavior:

[0082] Each bird searches for food through its own experience and group experience, which can be expressed mathematically as follows:

[0083]

[0084] Where j∈[1,2,…,N], rand(0,1) represents the number of independent uniform distributions in (0,1), C and S are the acceleration coefficients of self-cognition and social cognition, P i,j is the best previous position of the ith bird, and g j is the best previous position shared by the population.

[0085] (2) Vigilant behavior:

[0086] Birds try to move to the center of the flock, and they will compete with each other, which can be expressed mathematically as follows:

[0087]

[0088] Among them, k (k≠i) is a random integer between 1 and N, a1 and a2 are two constants, and pFit i represents the best fitness of the i-th bird, sumFit represents the sum of the best fitness of the population, mean j The jth element represents the average position of the entire group. ε is used to avoid zero-score error and is the minimum constant.

[0089] (3) Flight behavior:

[0090] When birds arrive at a new location and perform flight behavior, they are divided into producers and beggars, which can be expressed mathematically as follows:

[0091]

[0092] Among them, randn(0,1) represents a Gaussian distributed random number with mean 0 and standard deviation 1, k∈[1,2,…,N], k≠i, and FL(FL∈[0,2]) represents the following probability.

[0093] In the present invention, we use the non-image data of the pipeline, that is, static numerical data, as the feature input of the extreme learning machine, and use the defect assessment result of the pipeline as the label of the extreme learning machine. When constructing the relevant model, the present invention uses the input weight and bias of the extreme learning machine to initialize each bird. Then, the regression error fitness value corresponding to the extreme learning machine is calculated according to the parameters of each bird, and the optimal fitness value and the corresponding weight and bias should be carefully recorded. After executing the flight behavior, the updated fitness value is then calculated. If this updated fitness value is better than the current optimal fitness value, then it is used to replace the original optimal fitness value and the corresponding parameters; if there is no better, then proceed to the next step. Finally, the number of evolutions of the flock and the maximum number of iterations are compared. If the number of evolutions of the flock is less than the maximum number of iterations, then the loop operation continues; otherwise, the corresponding parameters are assigned to the extreme learning machine, so that the model establishment work is completed.

[0094] The input weight and bias range of the extreme learning machine are set to [0, 1], the number of hidden layer neurons is 15, the number of bird groups is 200, the maximum number of population iterations is 50, the number of input neurons is the actual number of numerical categories of welding process data and characteristic weld data, and the number of output neurons is set to 1, which represents the fused pipeline defect category, namely serious defect, relatively serious defect, slight defect and no defect. During the training process, the data set is divided into a training set and a test set. After the training is completed, the characteristic data of the test set is input into the extreme learning machine, and the result is the predicted defect situation.

[0095] Step 4: Preprocess the image data of the pipeline body

[0096] In the present invention, the girth weld image is first preprocessed, converted into RGB channels, and resized to 224*224 pixels. The pixel intensity is scaled to a range of 0-1, and the weld image data is enhanced by applying random geometric transformations, such as scaling, moving, flipping, and other different combinations, to generate new data from the available training data, thereby expanding the available training data with new credible samples, which ensures that the existing model has better generalization ability and avoids overfitting. The present invention divides the data set into a training set and a test set, wherein the training set accounts for 80% and the test set accounts for 20%. Subsequently, the present invention denoises the weld image data so that weld defects can be more clearly identified in subsequent training.

[0097] Gaussian filtering is a widely used image processing technique that smoothes images based on Gaussian functions. Gaussian filters are very effective in removing noise and details from images, especially for dealing with Gaussian noise. The core idea is to convolve the image with a Gaussian kernel to achieve a smoothing effect. The core of the Gaussian filter is the Gaussian function, and its mathematical expression is:

[0098]

[0099] Where (x, y) is the pixel coordinate and σ is the standard deviation of the Gaussian kernel, which determines the smoothness of the filter.

[0100] The convolution kernel of the Gaussian filter is a two-dimensional matrix whose values ​​are calculated by the Gaussian function. In image processing, this convolution kernel is applied to each pixel of the image, and the new pixel value is obtained by weighted summation.

[0101] Some pipeline weld defect images are small in size, low in image contrast, and have blurred edges, which is not conducive to artificial or deep neural networks capturing effective defect information. Therefore, it is very necessary to perform edge enhancement on defect images. Edge enhancement technology can highlight the edges of pipeline weld defects, improve image contrast, and make defects more obvious, showing defect morphology more clearly, and helping with defect classification.

[0102] The use of Gaussian filtering and Sobel operators can effectively suppress noise, locate edges with high precision, and at a relatively fast speed. The Sobel operator detects edge information in an image by calculating the difference between the grayscale value of each pixel in the image and the grayscale value of the pixels around it. The Sobel operator combines Gaussian smoothing and differential derivation. It is also called a first-order differential operator and contains two sets of 3X3 filters, which are sensitive to edges in the horizontal and vertical directions respectively. The two directional templates are convolved with the image along the X-axis and Y-axis respectively, from top to bottom and from left to right. The center of the template is overlapped with a pixel on the image, and the points around the pixel are multiplied by the coefficients on the template.

[0103] Furthermore, since traditional defect assessment methods are usually based on feature selection and shallow machine learning algorithms, they have limitations such as reliance on expert experience and difficulty in capturing deep features. In contrast, intelligent defect diagnosis methods based on deep learning can make up for these shortcomings and have stronger feature extraction capabilities and higher automation. However, in practical applications, on-site sample collection is time-consuming and laborious, and the amount of data collected in a short period of time is often insufficient to meet the needs of deep model training. For example, the ImageNet dataset commonly used for training deep models contains more than 14 million images, covering more than 20,000 categories. In contrast, the dataset used for defect diagnosis is much smaller in both data volume and number of categories. In the case of insufficient training data, the diagnostic performance of the established model is poor, which may lead to diagnosis failure. In addition, there is a common sample imbalance problem in actual scenarios, that is, the number of defect samples detected is far less than the number of normal samples. Under different detection environments, the data distribution of samples will also be different, resulting in more scarce defect samples in specific detection environments. These problems are collectively referred to as the small sample problem in defect diagnosis.

[0104] In order to solve the problem of small samples, the present invention chooses to use a data enhancement method based on GAN. Generative Adversarial Network (GAN) is a generative deep neural network model proposed by Goodfellow et al. in 2014. Traditional GAN ​​has problems such as mode collapse, gradient disappearance and gradient explosion. GAN variants solve the above problems by redesigning the network structure or redefining new loss functions, such as Deep Convolutional GAN ​​(DCGAN), Wasserstein GAN (WGAN) and WGAN-GP. GAN and its variants achieve the alignment of the source domain and the target domain in the sample space or feature space by generating new samples, so they can be used to solve the data dilemma of missing training samples. This project uses the WGAN method for data enhancement. This method learns the feature information of the target domain samples by adversarial training, so that the generated samples are closer to the real samples. GAN consists of two neural networks that are trained against each other, one is a sample generator (Generator, G) that captures the data distribution, and the other is a discriminator (Discriminator, D) that estimates the probability of the sample coming from the training data. The training process of the network is a game process between G and D, and the goal of both parties is to maximize their own interests. The present invention uses Wasserstein distance to measure the distance between the generated distribution and the true distribution. Therefore, the optimization function is as follows:

[0105]

[0106] Where D∈Lip means that D obeys Lipschitz continuity, and Pg is the generated sample distribution.

[0107] In the present invention, we input the existing girth weld defect image data set into the data enhancement network model based on WGAN. Specifically, the parameters of the generator and the discriminator are first initialized, including setting the appropriate convolution kernel size, number of layers, number of neurons and other network structure related parameters. The specific parameter information of the generator network is the first convolution layer with a convolution kernel size of 7×7, a step size of 1, and a number of filters of 64, followed by two convolution layers with a convolution kernel size of 3×3, a step size of 2, and a number of filters of 128 and 256, respectively, to complete the downsampling process. Entering the residual network part, it will pass through 18 convolution layers with a convolution kernel size of 3×3, a step size of 1, and a number of filters of 256. Finally, the upsampling process is performed, after two convolution layers with a convolution kernel size of 3×3, a step size of 2, and a number of filters of 128 and 64, respectively, a convolution layer with a convolution kernel size of 7×7, a step size of 2, and a number of filters of 3, and after the Tanh activation function, the image reconstructed by the generator is output. The specific parameter information of the discriminator network is three convolution layers with a convolution kernel size of 4×4, a step size of 2, and the number of filters is 64, 128, and 256 respectively, followed by two convolution layers with a convolution kernel size of 4×4, a step size of 1, and the number of filters is 512 and 1 respectively, resulting in a tensor of size 30×30×1. Finally, the average value is obtained through the mean pooling layer to obtain a tensor of size 1×1×1, and then the dimension of the tensor is adjusted to obtain the final probability value p of judging that the input sample is a real sample. In the training stage, the real image is input into the discriminator, and the "fake" image generated by the generator is also input into the discriminator. The discriminator judges and feeds back the result, and uses the back propagation algorithm to update the weights of the generator and the discriminator according to the feedback. After multiple iterations, the generator can generate sufficiently realistic images and the discriminator has difficulty distinguishing between real and fake images. At this point, we have generated more girth weld image data. Based on these generated data, we use manual experience to annotate the defect labels of the pipelines and add the generated images to the original dataset, which eventually becomes a dataset that can be used for model training.

[0108] Step 5: Evaluate the girth weld defects of the pipeline body based on the VGG16 model

[0109] In this paper, we use Keras, an open source deep learning library, to develop models. The library contains a series of pre-trained deep learning classifiers, including VGG16, DenseNet121, InceptionV3, and ResNet50. Each classifier model has a different depth value. In this research work, due to the relatively small amount of available data and the need for faster inference speed, a shallow model is the best choice. We chose VGG16 as the training model.

[0110] See also Figure 2 As shown in the figure, the VGG16 network can effectively extract detailed features in the image through the stacking of multi-layer deep structures and small convolution kernels (3x3). It is particularly suitable for pipeline weld defect recognition tasks that require high-resolution features. Through the combination of multiple convolutional layers and pooling layers, the input image can be processed at different scales, which is particularly important when detecting weld defects of different types and sizes. Since the VGG network is pre-trained on large-scale datasets such as ImageNet, out-of-the-box feature extraction can accelerate the training of weld defect detection models, especially when the amount of data is insufficient. By visualizing the feature maps of the convolutional layers, we can better understand the features that the model focuses on during the weld defect recognition process or other pipeline defect recognition tasks, which helps to further optimize and adjust the model.

[0111] In the present invention, we use the pre-trained VGG16 network to extract the features of the girth weld film data and the girth weld defect image data, merge the weld defect features with the girth weld film features to form a unified feature set, and use PCA dimensionality reduction technology to reduce the number of features in order to reduce the computational cost and avoid overfitting problems.

[0112] PCA is the most commonly used linear dimensionality reduction method. Its goal is to map high-dimensional data to a low-dimensional space through a linear projection, and to maximize the variance of the data in the projected dimension, so as to use fewer data dimensions while retaining more characteristics of the original data points. PCA has also been proven to be a linear dimensionality reduction method that loses the least information about the original data. It has a wide range of applications in data compression, redundancy elimination, and data noise elimination.

[0113] The feature-level fusion model uses the attention feature fusion method AFF, which is applicable to most common feature-level fusion scenarios, including short jump and long jump connections and feature fusion caused in the Inception layer. In order to better fuse features with inconsistent semantics and scales, the present invention also uses a multi-scale channel attention module, which solves the problem of fusing features given at different scales. Figure 3As shown in Figure 1, this is a multi-scale channel attention module that extracts channel attention weights by using two branches of different scales. One branch uses Global AvgPooling to extract the attention of global features, and the other branch directly uses point-wise convolution to extract the channel attention of local features.

[0114] See also Figure 4 As shown in the figure, the girth weld defect feature map extracted by VGG16 and the corresponding girth weld film feature map are processed by the multi-scale channel attention module respectively, and finally the channel attention weight is obtained. Based on the multi-scale channel attention module, the attention feature fusion can be expressed as:

[0115]

[0116] Finally, the fused features will pass through a two-layer fully connected network, which outputs the final pipeline girth weld defect assessment results. This design enables the model to retain the original information of the two images while improving the accuracy of girth weld defect assessment through feature fusion.

[0117] Step 6: Use the DS evidence reasoning algorithm to perform decision-level fusion on the defect assessment conclusions obtained from two different data sources and different methods

[0118] join Figure 5 As shown in the figure, the decision-level fusion method performs correlation analysis on the target decisions of different sensors and integrates them to obtain the final results of target identification, situation estimation, and threat assessment. The data processing, feature extraction, and decision-making of each sensor are completed locally. The primary decision information is then analyzed through the decision-level fusion algorithm to derive the joint inference result. Common decision-level fusion methods are usually based on improvements on traditional probability theory, such as DS evidence theory, entropy theory, and correlation analysis.

[0119] The DS evidence reasoning algorithm is an extended application theory of Bayesian reasoning and belongs to the data fusion technology of the decision-making layer. Its core elements include basic credibility distribution function, trust function and likelihood function. The reasoning process of the DS method is divided into three levels. The first level is target synthesis, which combines observations from different data sources into a summarized output result; the second level is inference, which converts the observations of the data source into a target report by processing them. This reasoning is based on the fact that the information of a specific data source can logically produce some certain reasonable target information; the third level is update. When conducting data analysis and decision-making, multiple groups of data should be considered comprehensively, and comparison and statistical analysis should be carried out to minimize the impact of random errors on decision-making. Therefore, before drawing conclusions and integrating data from multiple sources, it is necessary to first combine and update the observation data of each data source.

[0120] DS evidence theory contains the following key functions:

[0121] (1) Basic credibility distribution function:

[0122] Let the set of all elements of variable X be Θ. All subsets of Θ can also be called propositions, including the empty set and Θ itself, and are recorded as identification frames. The number of possible sets in Θ is represented by the power set 2Θ. When there are N elements in the set Θ, its space size is 2N. Then

[0123]

[0124] If A is a subset of the set Θ (proposition), let the function m: 2Θ→[0,1], if it satisfies:

[0125]

[0126] (2) Trust function:

[0127] For a set Θ, for any subset A, we have Bel:2Θ→[0,1], and we define:

[0128]

[0129] If the function Bel satisfies the following conditions, then Bel is called the trust function of the set Θ:

[0130]

[0131] Bel(θ)=1

[0132] (3) Likelihood function:

[0133] For set Θ, for any subset A, there is Pl:2Θ→[0,1], then the likelihood function of set A is:

[0134]

[0135] Pl(A) represents the probability of supporting the validity of proposition A, which is the sum of the basic credibility distribution of all sets that have intersection with A and are not empty.

[0136] (4) Credibility space:

[0137] In the set Θ, Let the confidence interval be [Bel(A), Pl(A)], and the uncertainty of proposition A is expressed by this interval. The upper limit of the uncertainty set is Bel(A), and the lower limit of the uncertainty set is Pl(A).

[0138] (5) Dempster synthesis rule:

[0139] After obtaining the discretized and simplified decision table, m1(Ai) and m2(Bj) are two basic credibility distribution values, and the conflict factor is defined as:

[0140]

[0141] The synthesis rules are:

[0142]

[0143] like And A i ≠A1, then:

[0144]

[0145] If a decision A1 satisfies the above formula, then A1 can be used as the final decision result. ε1 and ε2 are the thresholds set in the experiment.

[0146] In the present invention, the recognition framework Θ is first defined to represent all possible states:

[0147] Θ={no defect, slight defect, relatively serious defect, serious defect}

[0148] Then, according to the results of static data analysis and girth weld image analysis, basic probabilities are assigned to each state, that is, the result of static data analysis is m1, and the result of girth weld image analysis is m2:

[0149] Static data analysis results m1:

[0150] m1(no defect)=p11,m1(slight defect)=p12

[0151] m1 (more serious defect) = p13, m1 (serious defect) = p14

[0152] Girth weld image analysis results m2:

[0153] m2(no defect)=p21,m2(slight defect)=p22

[0154] m2 (more serious defect) = p23, m2 (serious defect) = p24

[0155] The conflict coefficient K represents the degree of inconsistency between two pieces of evidence, and the calculation formula is:

[0156]

[0157] For the four states, the conflict coefficients are calculated:

[0158]

[0159] Specifically:

[0160] K = m1 (no defect) · (m2 (minor defect) + m2 (more serious defect) + m2 (severe defect)) + m1 (minor defect) · (m2 (no defect) + m2 (more serious defect) + m2 (severe defect)) + m1 (more serious defect) · (m2 (no defect) + m2 (minor defect) + m2 (severe defect)) + m1 (severe defect) · (m2 (no defect) + m2 (minor defect) + m2 (severe defect)) + m1 (severe defect) · (m2 (no defect) + m2 (minor defect) + m2 (more serious defect))

[0161] Use Dempster's combination rule to combine the two BPAs to get a new BPA m:

[0162]

[0163] Specifically calculate the new BPA for each state:

[0164]

[0165]

[0166] Finally, according to the maximum probability principle, the state with the maximum probability is selected as the final evaluation result:

[0167] If m(no defect) is the largest, it is judged to be no defect;

[0168] If m (minor defect) is the largest, it is judged as a minor defect;

[0169] If m (more serious defect) is the largest, it is judged as a more serious defect;

[0170] If m (serious defect) is the largest, it is judged as a serious defect.

[0171] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0172] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any ordinary technician in the industry can smoothly implement the present invention as shown in the drawings and described above. However, any equivalent changes, modifications and evolutions made by technicians familiar with the profession without departing from the scope of the technical solution of the present invention using the technical content disclosed above are all equivalent embodiments of the present invention. At the same time, any equivalent changes, modifications and evolutions made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. A pipeline defect assessment method based on multi-source data fusion, characterized in that: The following steps are involved: Obtain data on the operation status of the pipeline itself, including static data and image data; Preprocess static data and image data; Based on the optimization of the extreme learning machine based on the bird flock algorithm, the pre-processed static data is analyzed and evaluated to obtain the pipeline defect assessment results of the static data; The preprocessed image data is analyzed and evaluated based on the VGG16 model to obtain the pipeline girth weld defect evaluation results of the image data; Based on the DS evidence reasoning algorithm, the pipeline defect assessment results of static data and the pipeline girth weld defect assessment results of image data are fused at the decision level to obtain the final pipeline defect assessment results.

2. According to claim 1, a pipeline defect assessment method based on multi-source data fusion is characterized in that: Annotate each data item in the pipeline operation status data with the pipeline status and the fault type; The pipeline status includes normal, slight defect and serious defect; the fault type includes corrosion, crack and deformation.

3. The pipeline defect assessment method based on multi-source data fusion according to claim 1 is characterized in that: The static data preprocessing process is as follows: The welding process data and characteristic weld data in the static data are standardized using Z-score, and the welding process data and characteristic weld data are converted into a standard normal distribution with zero mean and unit variance, and then normalized using Min-Max.

4. The pipeline defect assessment method based on multi-source data fusion according to claim 1 is characterized in that: The process of preprocessing image data is as follows: Preprocessing the girth weld image in the pipeline body, including data enhancement, denoising and image edge enhancement. The data enhancement method of generative adversarial network is used to enhance the data of girth weld images. The data enhancement process of the girth weld image is to convert the girth weld image into RGB channels, resize it to 224×224 pixels, scale the pixel intensity to the range of 0-1, and perform data enhancement through random geometric transformation; The denoising process of the girth weld image is to use a Gaussian filter to remove the noise of the girth weld image; The edge processing of the enhanced image of the ring weld is to use the Sobel operator for edge detection, and the defective edge of the ring weld is highlighted by calculating the difference between the gray value of each pixel in the image and the gray value of its surrounding pixels.

5. The pipeline defect assessment method based on multi-source data fusion according to claim 1 is characterized in that: Based on the optimization of the extreme learning machine based on the bird flock algorithm, the pre-processed static data is analyzed and evaluated. The specific process is as follows: The initial weights and biases of the extreme learning machine are optimized by the bird flock algorithm. The mean square error of the extreme learning machine on the training set is defined as the fitness function. The fitness function value is minimized by iteratively optimizing the position of individual birds to achieve the optimization of the extreme learning machine. The preprocessed static data is used as the feature input of the extreme learning machine, and the defect assessment result of the pipeline is used as the label of the extreme learning machine. The input weight, bias range, number of hidden layer neurons, number of bird flocks, and maximum number of iterations of the extreme learning machine are set; the number of input neurons is the actual number of numerical categories of welding process data and characteristic weld data; the number of output neurons is 4, which represents the category of pipeline defects after fusion. The static data is trained, and after the training is completed, the static feature data of the pipeline body is input into the extreme learning machine to obtain the assessment result of pipeline defects; Among them, pipeline defect categories include serious defects, relatively serious defects, minor defects and no defects.

6. A pipeline defect assessment method based on multi-source data fusion according to claim 5, characterized in that: The specific process of optimizing the extreme learning machine is as follows: The input weights and biases of the extreme learning machine are used to initialize each bird, and then the regression error fitness value corresponding to the extreme learning machine is calculated based on the parameters of each bird, and the optimal fitness value and the corresponding weights and biases are recorded. After executing the flight behavior, the fitness value is calculated and updated. After the update iteration, the optimal fitness value is assigned to the extreme learning machine.

7. The pipeline defect assessment method based on multi-source data fusion according to claim 5 is characterized in that: A regularization parameter is introduced into the extreme learning machine to balance the error matrix and the output matrix, and a sample weighting matrix is ​​introduced into the objective function. The Gaussian kernel function is used to construct the sample weighting matrix, and weighted processing is performed according to the similarity between the training sample and the sample to be predicted.

8. The pipeline defect assessment method based on multi-source data fusion according to claim 1 is characterized in that: The preprocessed image data is analyzed and evaluated based on the VGG16 model, specifically: The features of girth weld film data and girth weld defect image data in the image data are extracted through the VGG16 network. The number of features is reduced by the PCA dimensionality reduction technology. The girth weld defect features and the girth weld film features are fused with the attention feature fusion method to form a unified fusion feature set. The fused features will be output through a two-layer fully connected network to obtain the pipeline girth weld defect assessment results.

9. The pipeline defect assessment method based on multi-source data fusion according to claim 1 is characterized in that: Based on the DS evidence reasoning algorithm, the defect assessment conclusions obtained from static data and image data are fused at the decision level, specifically: Based on the DS evidence reasoning algorithm, the recognition framework Θ is defined to represent all possible states. According to the pipeline defect assessment conclusions of static data analysis and girth weld image analysis, basic probabilities are assigned to each state, that is, the result of static data analysis is m1, and the result of girth weld image analysis is m2. The conflict coefficient K represents the degree of inconsistency between the two evidences. The pipeline defect assessment conclusions of static data analysis and girth weld image analysis are combined by calculating the conflict coefficient K and Dempster's combination rule to obtain the fused assessment result. Among them, Θ includes no defects, slight defects, more serious defects and serious defects.

10. A pipeline defect assessment system based on multi-source data fusion, based on a pipeline defect assessment method based on multi-source data fusion according to any one of claims 1 to 9, characterized in that: include Data acquisition module, preprocessing module, static data analysis and evaluation module, image data analysis and evaluation module, and decision-level fusion evaluation module; The data acquisition module is used to acquire the data of the pipeline body operation status, including static data and image data; The preprocessing module is used to preprocess static data and image data; The static data analysis and evaluation module is used to optimize the extreme learning machine based on the bird flock algorithm, analyze and evaluate the pre-processed static data, and obtain the pipeline defect evaluation results of the static data; The image data analysis and evaluation module is used to analyze and evaluate the preprocessed image data based on the VGG16 model to obtain the pipeline girth weld defect evaluation results of the image data; The decision-level fusion evaluation module is used to perform decision-level fusion on the pipeline defect evaluation results of static data and the pipeline girth weld defect evaluation results of image data based on the DS evidence reasoning algorithm to obtain the final pipeline defect evaluation results.

Images