Pipeline defect evaluation method and system based on multi-source data fusion

Abstract

The application provides a pipeline defect evaluation method and system based on multi-source data fusion, including the following steps: collecting data of the pipeline body operation state by using relevant acquisition devices, and marking the pipeline state and fault type, for constructing a training set and a test set. The pipeline static data is preprocessed, including data cleaning, standardization and feature extraction. The bird swarm algorithm is used to optimize the extreme learning machine, the static data is analyzed and the pipeline defect is evaluated. The image data of the pipeline body is preprocessed, including image conversion, enhancement and denoising. The VGG16 model is pre-trained based on the evaluation of the girth weld defects. The D-S evidence reasoning algorithm is used to fuse the defect evaluation conclusions obtained by the static data analysis and the girth weld image analysis at the decision level, and the final evaluation result is obtained. This method effectively improves the accuracy and reliability of the pipeline defect evaluation, and has wide application prospect and important practical value.

Description

Technical Field

[0001] This invention belongs to the field of multi-source data fusion technology, and relates to a pipeline defect assessment method and system based on multi-source data fusion. Background Technology

[0002] As a core infrastructure for natural gas and oil transportation, pipelines are undeniably complex and crucial. They not only traverse vast geographical areas but also frequently face extreme natural conditions such as high temperatures, low temperatures, strong winds, torrential rains, and earthquakes. These natural factors, coupled with the long-term effects of high pressure and corrosive media within the pipelines, make them highly susceptible to various types of defects and failures, including but not limited to corrosion, cracks, wear, deformation, and weld defects.

[0003] Therefore, efficient and accurate defect diagnosis of pipelines is crucial. This requires the use of advanced testing technologies and methods, such as ultrasonic testing, magnetic particle testing, eddy current testing, radiographic testing, and the rapidly developing technologies of drone inspection, intelligent robot inspection, and fiber optic sensing monitoring. These technologies can achieve comprehensive scanning of the inner and outer walls of pipelines, accurately identify and locate defects, assess the severity of defects, and provide a scientific basis for subsequent maintenance decisions. Establishing a comprehensive pipeline management system is also essential. This system should integrate data acquisition, analysis and early warning, and emergency response functions, enabling early detection, reporting, and handling of faults by monitoring pipeline operating status in real time, analyzing operating data, and predicting potential risks. Furthermore, strengthening personnel training and improving the professional skills and emergency response capabilities of inspection personnel are also key aspects of ensuring the safe operation of pipelines.

[0004] With the rapid development of technologies such as the Internet of Things, big data, and artificial intelligence, pipeline defect diagnosis is gradually moving towards intelligence and automation. By constructing digital twin models of pipelines to simulate their operating conditions and combining this with historical fault data, it is possible to more accurately predict and prevent faults, achieving refined management of pipeline operation and maintenance. Currently, traditional pipeline defect diagnosis methods mainly rely on manual inspection and experience-based judgment, lacking automation and intelligence. These methods are not only time-consuming and labor-intensive but also easily affected by human factors, leading to inaccurate or untimely diagnostic results. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention aims to provide a pipeline defect assessment and analysis method and system based on multi-source data fusion. This method can be applied to the daily operation and maintenance of smart pipeline networks. Its high efficiency and intelligence will significantly improve the operation and maintenance efficiency and safety of smart pipeline networks, thereby effectively reducing maintenance costs and risks.

[0006] This invention is achieved through the following technical solution:

[0007] A pipeline defect assessment method based on multi-source data fusion includes the following steps:

[0008] Acquire data on the operational status of the pipeline body, including static data and image data;

[0009] Preprocess static and image data;

[0010] The bird flocking algorithm is used to optimize the extreme learning machine, and the preprocessed static data is analyzed and evaluated to obtain the pipeline defect evaluation results of the static data.

[0011] The preprocessed image data was analyzed and evaluated based on the VGG16 model to obtain the evaluation results of pipeline circumferential weld defects in the image data.

[0012] The pipeline defect assessment results based on static data and the pipeline circumferential weld defect assessment results based on image data are fused at the decision level to obtain the final pipeline defect assessment result.

[0013] Preferably, each data point in the pipeline's operational status data is labeled with its pipeline status and fault type;

[0014] The pipeline condition includes normal, minor defects, and severe defects; the fault types include corrosion, cracks, and deformation.

[0015] Preferably, the static data preprocessing process is as follows:

[0016] Welding process data and characteristic weld joint data in static data are standardized using Z-score to convert them into a standard normal distribution with zero mean and unit variance, and then normalized using Min-Max.

[0017] Preferably, the image data preprocessing process is as follows:

[0018] Preprocessing of the circumferential weld images in the pipeline body includes data augmentation, noise reduction, and edge enhancement of the circumferential weld images;

[0019] A generative adversarial network (GAN) data augmentation method was used to perform data augmentation processing on circumferential weld seam images.

[0020] The data enhancement processing of the circumferential weld image involves converting the circumferential weld image into an RGB channel, resizing it to 224×224 pixels, scaling the pixel intensity to the range of 0-1, and performing data enhancement through random geometric transformation.

[0021] The noise reduction process for the circumferential weld image is to remove noise from the circumferential weld image using a Gaussian filter;

[0022] Edge processing for enhancing images of circumferential welds involves using the Sobel operator for edge detection. By calculating the difference between the grayscale value of each pixel in the image and the grayscale values ​​of its surrounding pixels, the defect edges of the circumferential weld are highlighted.

[0023] Preferably, the specific process of using the bird flocking algorithm to optimize the extreme learning machine, analyze static data, and evaluate pipeline defects is as follows:

[0024] The initial weights and biases of the Extreme Learning Machine are optimized by using the bird flocking algorithm. The mean squared error of the Extreme Learning Machine on the training set is defined as the fitness function. By iteratively optimizing the position of individual birds, the fitness function value is minimized, thereby optimizing the Extreme Learning Machine.

[0025] The preprocessed static data is used as the feature input of the Extreme Learning Machine (ELM), and the pipeline defect evaluation results are used as the label of the ELM. The input weights, bias range, number of hidden layer neurons, flock size, and maximum number of iterations of the population are set. The number of input neurons is the actual number of numerical categories of welding process data and feature weld data. The number of output neurons is 4, representing the fused pipeline defect categories. The static data is used for training. After training, the static feature data of the pipeline body is input into the ELM to obtain the pipeline defect evaluation results.

[0026] Pipeline defects are categorized into serious defects, more serious defects, minor defects, and no defects.

[0027] The preferred optimization process for the Extreme Learning Machine is as follows:

[0028] The input weights and biases of the Extreme Learning Machine are used to initialize each bird. Then, the regression error fitness value of the Extreme Learning Machine is calculated based on the parameters of each bird. The optimal fitness value and its corresponding weights and biases are recorded. After the flight behavior is executed, the fitness value is calculated and updated. After the update iteration, the optimal fitness value is assigned to the Extreme Learning Machine.

[0029] Preferably, a regularization parameter is introduced into the extreme learning machine to balance the error matrix and the output matrix, and a sample weighting matrix is ​​introduced into the objective function. The sample weighting matrix is ​​constructed using a Gaussian kernel function, and weighting is performed based on the similarity between the training samples and the samples to be predicted.

[0030] Preferably, the evaluation of pipe circumferential weld defects in image data is based on the VGG16 model, specifically as follows:

[0031] Features of circumferential weld film data and circumferential weld defect image data are extracted from the image data using a VGG16 network. The number of features is reduced by PCA dimensionality reduction technology. Then, the circumferential weld defect features and circumferential weld film features are fused using an attention feature fusion method to form a unified fused feature set. The fused features are output through a two-layer fully connected network to obtain the pipeline circumferential weld defect assessment result.

[0032] Preferably, the defect assessment conclusions obtained from static data and image data based on the DS evidence reasoning algorithm are fused at the decision level, specifically as follows:

[0033] Based on the DS evidence reasoning algorithm, an identification framework Θ is defined to represent all possible states. According to the pipeline defect assessment conclusions from static data analysis and circumferential weld image analysis, a basic probability is assigned to each state, i.e., the result of static data analysis is m1, and the result of circumferential weld image analysis is m2. The conflict coefficient K represents the degree of inconsistency between the two pieces of evidence. By calculating the conflict coefficient K and Dempster's combination rule, the pipeline defect assessment conclusions from static data analysis and circumferential weld image analysis are combined to obtain the fused assessment result.

[0034] Θ includes no defects, minor defects, more serious defects, and serious defects.

[0035] A pipeline defect assessment system based on multi-source data fusion includes:

[0036] The system includes a data acquisition module, a preprocessing module, a static data analysis and evaluation module, an image data analysis and evaluation module, and a decision-level fusion evaluation module.

[0037] The data acquisition module is used to acquire data on the operating status of the pipeline body, including static data and image data;

[0038] The preprocessing module is used to preprocess static data and image data;

[0039] The static data analysis and evaluation module is used to optimize the extreme learning machine based on the bird flocking algorithm, analyze and evaluate the preprocessed static data, and obtain the pipeline defect evaluation results of the static data.

[0040] The image data analysis and evaluation module is used to analyze and evaluate preprocessed image data based on the VGG16 model to obtain the evaluation results of pipeline circumferential weld defects in the image data.

[0041] The decision-level fusion evaluation module is used to perform decision-level fusion of pipeline defect evaluation results from static data and pipeline circumferential weld defect evaluation results from image data based on the DS evidence reasoning algorithm, so as to obtain the final pipeline defect evaluation result.

[0042] Compared with the prior art, the present invention has the following beneficial technical effects:

[0043] This invention utilizes Extreme Learning Machine (ELM) and Bird Swarm Optimization (BSA) to process and optimize the multi-sensor data fusion process in pipeline defect assessment. The ELM is optimized based on the Bird Swarm Optimization algorithm to analyze and assess pipeline defects from static data; the VGG16 model is used to assess pipeline circumferential weld defects from image data; and the DS evidence reasoning algorithm is used to perform decision-level fusion of the pipeline defect assessment conclusions obtained from static and image data to obtain the final pipeline defect assessment result. The ELM model is used for training, offering advantages such as fast training speed and good generalization performance. This method, applied to the daily operation and maintenance of smart pipeline networks, significantly improves the efficiency and safety of smart pipeline network operation and maintenance due to its high efficiency and intelligence, thereby effectively reducing maintenance costs and risks.

[0044] Furthermore, to improve the generalization ability of Extreme Learning Machine (ELM), a regularization parameter is introduced to balance the error matrix and the output matrix, and a sample weighting matrix is ​​introduced into the objective function. The sample weighting matrix is ​​constructed using a Gaussian kernel function, and weighting is performed based on the similarity between the training samples and the samples to be predicted, thereby reducing the weight of outliers.

[0045] Furthermore, the bird flocking algorithm defines the mean squared error (MSE) of ELM on the training set as a fitness function. By iteratively optimizing the position of individual birds, the fitness function value is minimized, thereby improving the accuracy and efficiency of pipeline defect assessment.

[0046] Furthermore, edge detection is performed using the Sobel operator. By calculating the difference between the gray value of each pixel in the image and the gray value of its surrounding pixels, the edges of weld defects are highlighted, improving image contrast and the visibility of defects.

[0047] Furthermore, this invention employs a data augmentation method based on Generative Adversarial Networks (GANs). By measuring the distance between the generated sample distribution and the real sample distribution using the Wasserstein distance, the performance of the generator and discriminator is optimized, generating new data that is closer to the real samples, thereby improving the training effect and generalization ability of the model.

[0048] Furthermore, this invention uses a pre-trained VGG16 network to extract features from circumferential weld film data and circumferential weld defect image data, and reduces the number of features using PCA dimensionality reduction technology to reduce computational cost and avoid overfitting. An attention feature fusion (AFF) method is employed for feature-level fusion, using a multi-scale channel attention module to extract channel attention weights for global and local features, better fusing features with inconsistent semantics and scale. The fused features are passed through a two-layer fully connected network to output the final pipeline circumferential weld defect assessment result. This design allows the model to improve the accuracy of circumferential weld defect assessment by preserving the original information of the two images through feature fusion. Attached Figure Description

[0049] Figure 1 This is a flowchart of the pipeline defect assessment based on multi-source data fusion according to the present invention;

[0050] Figure 2 This is a network structure diagram of the VGG16 model of the present invention;

[0051] Figure 3 This is a structural diagram of the multi-scale channel attention module of the present invention;

[0052] Figure 4 This is a network structure diagram of the feature-level fusion model of the present invention;

[0053] Figure 5 This is a schematic diagram of the decision-level fusion scheme of the present invention. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0055] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0056] The present invention will now be described in further detail with reference to the accompanying drawings:

[0057] Step 1: Data Acquisition and Storage

[0058] Data acquisition utilizes high-precision testing equipment, such as ultrasonic testing instruments, magnetic particle testing instruments, and eddy current testing instruments, to provide detailed information on the internal and external aspects of the pipeline, including but not limited to wall thickness variations, corrosion levels, and cracks. This ensures the acquired data is broadly representative, covering various possible pipeline conditions (normal, minor defects, severe defects, etc.) and different fault types (such as corrosion, cracks, deformation, etc.) so that the model can learn characteristics under different conditions. Simultaneously, environmental conditions during data acquisition, such as temperature, humidity, and pressure, are recorded.

[0059] Data annotation is performed by experienced professionals to ensure the accuracy and reliability of the collected data. A clear set of annotation rules and standards is established to ensure that all personnel involved in the annotation work follow the same standards, improving annotation consistency. The quality of the annotations is checked regularly to promptly identify and correct errors or inconsistencies.

[0060] Data storage involves storing the collected data in various formats (such as pipeline image data and pipeline static data) using databases (such as MySQL or NoSQL) or distributed storage systems to ensure data security and scalability. An efficient data management system is established to facilitate data retrieval and use, as well as version management and updates.

[0061] Step 2: Preprocess the static data of the pipeline body.

[0062] Welding process data and characteristic weld data are standardized using Z-score, transforming the data into a standard normal distribution with zero mean and unit variance. The Z-score standardization formula is as follows:

[0063]

[0064] In the formula, The mean, The standard deviation is denoted as .

[0065] Then, use Min-Max normalization to scale the data to the [0, 1] interval. The Min-Max normalization formula is as follows:

[0066]

[0067] In the formula, This represents the minimum value. This represents the maximum value.

[0068] Step 3: Use the bird flocking algorithm to optimize the extreme learning machine, analyze the static data of the pipeline, and evaluate pipeline defects.

[0069] Extreme Learning Machine (ELM) is a novel learning algorithm for training single-hidden-layer feedforward neural networks. Its core idea is to randomly initialize the weights and biases from the input layer to the hidden layer, and then train the weights from the hidden layer to the output layer using the least squares method. ELM is simpler, trains faster, and exhibits better generalization performance than traditional neural networks. However, ELM models are prone to overfitting in practical applications. To improve the generalization ability of ELM, this invention introduces a regularization parameter to balance the error matrix and the output matrix. Simultaneously, to consider the influence of different samples on the ELM output, a sample weighting matrix is ​​introduced into its objective function. A Gaussian kernel function is used to construct the sample weighting matrix, and the samples to be predicted are also weighted according to the similarity between the training samples and the samples to be predicted. This method can reduce the weight of outliers, thereby improving the learning ability and generalization performance of ELM.

[0070] The objective function of Extreme Learning Machine for

[0071]

[0072] in, , , These are the expected output matrix, the hidden layer matrix, and the output layer weights, respectively. For regularization parameters, Error matrix The square of the L2 norm; The weighted matrix is ​​composed of Gaussian kernel functions:

[0073]

[0074] in, For the sample to be predicted, These are training set samples; These are the parameters of the Gaussian kernel function.

[0075] therefore, The optimal solution is

[0076]

[0077] in, For the sample size, The number of neurons, It is an identity matrix.

[0078] When using ELM as a multi-sensor data fusion model, the weights and biases between the input and hidden layers are randomly generated, which may lead to poor generalization ability and insufficient stability. Therefore, to improve the generalization ability and stability of the fusion model, this paper introduces the Bird Swarm Algorithm (BSA) to optimize the initial values ​​of the model's weights and biases.

[0079] The Bird Swarm Algorithm, proposed in 2015, is a novel bio-inspired algorithm that solves optimization problems by mimicking the social behaviors, foraging behaviors, alertness behaviors, and flight behaviors of birds. In designing the Bird Swarm Algorithm, this invention defines the mean squared error (MSE) of the ELM on the training set as the fitness function, used to evaluate the position of each individual bird (i.e., each set of weights and biases). According to the update rules of the Bird Swarm Algorithm, the positions of individual birds are iteratively optimized to minimize the fitness function value. The optimal weights and biases found are then used to train the final ELM model.

[0080] Let the size of the flock be The dimensions of flight space are , ( ∈[1, 2,…,N]) represents the bird's position. Indicates flight intervals, Let (the probability of foraging) be the random number between (0,1) less than (the probability of finding food). If the bird chooses to forage, it will do so; otherwise, it will remain vigilant.

[0081] (1) Foraging behavior:

[0082] Each bird finds food based on its own experience and the experience of the group, which can be expressed mathematically as follows:

[0083]

[0084] Where j ∈ [1,2,…,N], express The number of independent uniformly distributed numbers in the data. and Acceleration coefficient for self-awareness and social awareness It is the first The best previous position for the bird, and It is the best prior location for population sharing.

[0085] (2) Vigilant behavior:

[0086] Birds attempt to move to the center of the population, and they will compete with each other, which can be expressed mathematically as follows:

[0087]

[0088]

[0089] in, It is a random integer between 1 and N, where a1 and a2 are two constants. Representing the The optimal adaptation of a single bird This represents the sum of the optimal fitness of the population. This represents the j-th element representing the average position of the entire group. It is used to avoid zero-point error and is the smallest constant.

[0090] (3) Flight behavior:

[0091] When birds arrive at a new location and engage in flight behavior, they are divided into producers and beggars, which can be represented by the following mathematical formula:

[0092]

[0093] in, This represents a Gaussian distributed random number with a mean of 0 and a standard deviation of 1. ∈ [1,2,…,N],k ≠ i, ( ∈[0,2]) represents the probability of following.

[0094] In this invention, we use the non-image data of the pipeline, i.e., static numerical data, as the feature input of the Extreme Learning Machine (ELM), and the defect evaluation results of the pipeline as the label of the ELM. When constructing the relevant model, this invention uses the input weights and biases of the ELM to initialize each bird. Then, based on the parameters possessed by each bird, the regression error fitness value corresponding to the ELM is calculated, and the optimal fitness value and its corresponding weights and biases are carefully recorded. After executing flight behavior, the updated fitness value is calculated. If this updated fitness value is better than the current best fitness value, then it replaces the original best fitness value and its corresponding parameters; otherwise, the next step is performed. Finally, the number of bird evolutions is compared with the maximum number of iterations. If the number of bird evolutions is less than the maximum number of iterations, the cycle continues; otherwise, the corresponding parameters are assigned to the ELM, thus completing the model construction.

[0095] The input weights and biases of the Extreme Learning Machine (ELM) are set to [0, 1], the number of hidden layer neurons is 15, the flock size is 200, and the maximum number of iterations is 50. The number of input neurons represents the actual number of numerical categories of the welding process data and feature weld data. The number of output neurons is set to 4, representing the fused pipeline defect categories: severe defect, moderately severe defect, minor defect, and no defect. During training, the dataset is divided into a training set and a test set. After training, the feature data of the test set is input into the ELM, and the result is the predicted defect situation.

[0096] Step 4: Preprocess the image data of the pipeline body.

[0097] This invention first preprocesses the circumferential weld image, converting it to RGB channels and resizing it to 224*224 pixels. Pixel intensity is scaled to the range of 0-1, and the weld image data is enhanced by applying random geometric transformations, such as different combinations of scaling, shifting, and flipping. New data is generated from the available training data, thus expanding the available training data with new, reliable samples. This ensures that the existing model has better generalization ability and avoids overfitting. The dataset is divided into a training set and a test set, with the training set accounting for 80% and the test set accounting for 20%. Subsequently, the weld image data is denoised to more clearly identify weld defects in subsequent training.

[0098] Gaussian filtering is a widely used image processing technique that smooths images based on a Gaussian function. Gaussian filters are very effective at removing noise and details from images, especially Gaussian noise. The core idea is to convolve the image with a Gaussian kernel to achieve a smoothing effect. The core of a Gaussian filter is the Gaussian function, whose mathematical expression is:

[0099]

[0100] Where (x, y) are pixel coordinates, and σ is the standard deviation of the Gaussian kernel, which determines the smoothness of the filter.

[0101] The convolution kernel of a Gaussian filter is a two-dimensional matrix whose value is calculated from a Gaussian function. In image processing, this convolution kernel is applied to each pixel of the image, and a new pixel value is obtained through weighted summation.

[0102] Some images of pipe weld defects are small in size, have low image contrast, and blurry edges, making it difficult for manual or deep neural networks to capture effective defect information. Therefore, edge enhancement of defect images is very necessary. Edge enhancement technology can highlight the edges of pipe weld defects, improve image contrast, and make the defects more obvious and clearly show their shape, thus helping with defect classification.

[0103] Using Gaussian filtering and the Sobel operator can effectively suppress noise, achieve high-precision edge localization, and is relatively fast. The Sobel operator detects edge information in an image by calculating the difference between the gray value of each pixel and the gray values ​​of its surrounding pixels. The Sobel operator combines Gaussian smoothing and differentiation. It is also known as the first-order differential operator, containing two sets of 3x3 filters, sensitive to edges in the horizontal and vertical directions respectively. Two directional templates are convolved with the image along the X and Y axes, respectively, from top to bottom and from left to right. The center of the template is aligned with a pixel in the image, and the coefficients of the points surrounding that pixel are multiplied by the coefficients on the template.

[0104] Furthermore, traditional defect assessment methods, typically based on feature selection and shallow machine learning algorithms, suffer from limitations such as reliance on expert experience and difficulty in capturing deep features. In contrast, deep learning-based intelligent defect diagnosis methods can overcome these shortcomings, offering stronger feature extraction capabilities and higher automation. However, in practical applications, on-site sample collection is time-consuming and labor-intensive, and the amount of data collected in a short time is often insufficient to meet the needs of deep model training. For example, the ImageNet dataset, commonly used for training deep models, contains over 14 million images covering more than 20,000 categories. In contrast, datasets used for defect diagnosis are much smaller in both quantity and number of categories. Insufficient training data leads to poor diagnostic performance in the established model, potentially resulting in diagnostic failure. Moreover, imbalanced sample distribution is a common problem in real-world scenarios, meaning the number of detected defect samples is far less than the number of normal samples. The data distribution of samples also varies under different detection environments, making defect samples even scarcer in specific detection environments. These problems are collectively referred to as the small sample problem in defect diagnosis.

[0105] To address the small sample size problem, this invention employs a GAN-based data augmentation method. Generative Adversarial Networks (GANs) are generative deep neural network models proposed by Goodfellow et al. in 2014. Traditional GANs suffer from problems such as mode collapse, vanishing gradients, and exploding gradients. Variants of GANs address these issues by redesigning the network structure or redefining the loss function, such as Deep Convolutional GAN ​​(DCGAN), Wasserstein GAN (WGAN), and WGAN-GP. GANs and their variants align the source and target domains in the sample or feature space by generating new samples, thus solving the data dilemma of missing training samples. This project uses the WGAN method for data augmentation. This method learns the feature information of target domain samples through adversarial training, making the generated samples closer to real samples. A GAN consists of two adversarially trained neural networks: a generator (G) that captures the data distribution, and a discriminator (D) that estimates the probability that a sample comes from the training data. The training process of a network is a game between G and D, where both sides aim to maximize their own interests. This invention uses the Wasserstein distance to measure the distance between the generated distribution and the true distribution. Therefore, the optimization function is as follows:

[0106]

[0107] In the formula, Let D be continuous according to Lipschitz, and Pg be the distribution of the generated samples.

[0108] In this invention, we input an existing dataset of circumferential weld defect images into a WGAN-based data augmentation network model. Specifically, we first initialize the parameters of the generator and discriminator, including setting appropriate convolutional kernel size, number of layers, number of neurons, and other network structure-related parameters. The specific parameter information for the generator network is that the convolutional kernel size is... The first convolutional layer has a stride of 1 and 64 filters, followed by a convolutional kernel size of... Two convolutional layers with a stride of 2 and the number of filters of 128 and 256 respectively are used to complete the downsampling process. The residual network then passes through 18 convolutional kernels of size [missing information]. The convolutional layer has a stride of 1 and 256 filters. Finally, there is an upsampling process, passing through two convolutional kernels of size [missing information]. The convolutional layers have a stride of 2 and contain 128 and 64 filters respectively. The kernel size is [missing value]. The discriminator network consists of convolutional layers with a stride of 2 and 3 filters, and an image reconstructed by the generator after passing through the Tanh activation function. Specific parameters of the discriminator network include a convolutional kernel size of... The convolutional layers consist of three layers with a stride of 2 and the number of filters being 64, 128, and 256 respectively, followed by a convolutional kernel size of... Two convolutional layers with a stride of 1 and filter counts of 512 and 1 respectively result in a size of [size missing]. The tensor is finally averaged through an average pooling layer to obtain a value of . The probability value for determining whether an input sample is a real sample is obtained by adjusting the dimensions of the tensor. During the training phase, real images are input into the discriminator, while "fake" images generated by the generator are also input into the discriminator. The discriminator judges and provides feedback. Based on the feedback, the weights of the generator and discriminator are updated using the backpropagation algorithm. This process is repeated multiple times until the generator can generate sufficiently realistic images, and the discriminator can no longer distinguish between real and fake images. At this point, we have generated a larger number of circumferential weld seam image datasets. Based on this generated data, defect labels for the pipes are added using human experience. The generated images are then added to the original dataset, ultimately creating a dataset that can be used for model training.

[0109] Step 5: Evaluate the defects of the circumferential weld of the pipeline body based on the VGG16 model.

[0110] In this invention, we utilize the open-source deep learning library Keras to develop our model. This library contains a series of pre-trained deep learning classifiers, including VGG16, DenseNet121, InceptionV3, and ResNet50. Each classifier model has a different depth. In this study, due to the relatively limited available data and the need for fast inference speed, a shallow model is the optimal choice, and we selected VGG16 as the training model.

[0111] See Figure 2As shown, the VGG16 network, through its multi-layered deep structure and stacked small convolutional kernels (3x3), can effectively extract detailed features from images, making it particularly suitable for pipeline weld defect recognition tasks that require high-resolution features. The combination of multiple convolutional and pooling layers allows for processing of input images at different scales, which is especially important when detecting weld defects of different types and sizes. Since the VGG network is pre-trained on large-scale datasets such as ImageNet, its out-of-the-box feature extraction can accelerate the training of weld defect detection models, especially when data is limited. Visualizing the feature maps of the convolutional layers provides a better understanding of the features the model focuses on during weld defect recognition or other pipeline defect recognition tasks, facilitating further model optimization and tuning.

[0112] In this invention, we use a pre-trained VGG16 network to extract features from circumferential weld film data and circumferential weld defect image data. We fuse the weld defect features with the circumferential weld film features to form a unified feature set. To reduce computational costs and avoid overfitting, we use PCA dimensionality reduction technology to reduce the number of features.

[0113] PCA is the most commonly used linear dimensionality reduction method. Its goal is to map high-dimensional data into a low-dimensional space through some linear projection, aiming to maximize the variance of the data in the projected dimension. This allows for the use of fewer data dimensions while preserving more of the original data points. PCA has also been proven to be the linear dimensionality reduction method that loses the least amount of original data information. It has wide applications in data compression to eliminate redundancy and data noise reduction.

[0114] The feature-level fusion model uses the attention-based feature fusion method (AFF), which is applicable to most common feature-level fusion scenarios, including short-hop and long-hop connections, as well as feature fusion induced in the Inception layer. To better fuse features with inconsistent semantics and scales, this invention also uses a multi-scale channel attention module, which addresses the problems encountered when fusing features from different scales. See also Figure 3 As shown, this is a multi-scale channel attention module that extracts channel attention weights by using two branches with different scales. One branch uses Global AvgPooling to extract attention for global features, while the other branch directly uses point-wise convolution to extract channel attention for local features.

[0115] See Figure 4 As shown, the circumferential weld defect feature map extracted using VGG16 and the corresponding circumferential weld film feature map are processed by a multi-scale channel attention module to obtain the channel attention weights. Based on the multi-scale channel attention module, the attention feature fusion can be represented as:

[0116]

[0117] Finally, the fused features are passed through a two-layer fully connected network, which outputs the final assessment result of the pipe circumferential weld defect. This design allows the model to improve the accuracy of circumferential weld defect assessment by fusion of features while retaining the original information of the two images.

[0118] Step 6: Use the DS evidence reasoning algorithm to perform decision-level fusion of defect assessment conclusions obtained from two different data sources and methods.

[0119] See Figure 5 As shown, the decision-level fusion method performs correlation analysis on target decisions from different sensors and integrates them to obtain the final results such as target identification, situational estimation, and threat assessment. Data processing, feature extraction, and decision-making for each sensor are all completed locally. Subsequent primary decision information is then analyzed by the decision-level fusion algorithm to derive the joint inference result. Common decision-level fusion methods are usually based on improvements to traditional probability theory, such as DS evidence theory, entropy theory, and correlation analysis.

[0120] The Data Synthesis (DS) evidence reasoning algorithm is an extension of Bayesian inference theory and belongs to the data fusion technology of the decision-making level. Its core elements include a basic credibility assignment function, a trust function, and a likelihood function. The reasoning process of the DS method is divided into three levels. The first level is target synthesis, which combines observations from different data sources into a summarized output. The second level is inference, which processes the observations from the data sources and transforms them into a target report. This reasoning is based on the fact that information from a specific data source can logically produce some definite and reasonable target information. The third level is updating. When conducting data analysis and decision-making, multiple sets of data should be considered comprehensively, and comparisons and statistical analyses should be performed to minimize the impact of random errors on the decision. Therefore, before drawing conclusions and integrating data from multiple sources, it is necessary to first combine and update the observation data from each data source.

[0121] The DS evidence theory includes the following key functions:

[0122] (1) Basic credibility assignment function:

[0123] Let the set of all elements of variable X be . , All subsets can also be called propositions, including the empty set and It is itself denoted as the recognition frame. The number of possible sets in the set is expressed as a power of set 2. To represent. When a set When there are N elements, its space size is 2N. Then...

[0124]

[0125] If A is a set A subset (proposition), let function m: If the following conditions are met:

[0126]

[0127]

[0128] (2) Trust function:

[0129] For sets For any subset A, we have : ,definition:

[0130]

[0131] If function If the following conditions are met, then Called a set Trust function:

[0132]

[0133]

[0134] (3) Likelihood function:

[0135] For sets For any subset A, we have : Then the likelihood function of set A is:

[0136]

[0137] Let represent the probability that proposition A is true, and let represent the sum of the basic confidence assignments of all sets that intersect with A and are not empty.

[0138] (4) Reliability space:

[0139] In the set middle, Let the confidence interval be... The uncertainty of proposition A is represented by this interval. The upper limit of the uncertainty set is... The lower bound of the uncertain set is .

[0140] (5) Dempster synthesis rules:

[0141] After obtaining the discretized and reduced decision table, m1(Ai) and m2(Bj) are two basic confidence assignment values. The conflict factor is then defined as follows:

[0142]

[0143] The synthesis rules are as follows:

[0144]

[0145] like ,but:

[0146]

[0147] If a decision A1 satisfies the above formula, then A1 can be used as the final decision result. The threshold set for the experiment.

[0148] In this invention, the recognition framework is first defined. , representing all possible states:

[0149] ={No defects, Minor defects, Serious defects, Major defects}

[0150] Subsequently, based on the results of static data analysis and circumferential weld image analysis, a basic probability was assigned to each state, namely, m1 for the static data analysis and m2 for the circumferential weld image analysis:

[0151] Static data analysis result m1:

[0152] m1 (no defects) = p11, m1 (minor defects) = p12

[0153] m1 (more serious defect) = p13, m1 (serious defect) = p14

[0154] Image analysis result m2 of circumferential weld:

[0155] m2 (no defects) = p21, m2 (minor defects) = p22

[0156] m2 (more serious defect) = p23, m2 (serious defect) = p24

[0157] The conflict coefficient K represents the degree of inconsistency between two pieces of evidence, and is calculated using the following formula:

[0158]

[0159] For the four states, calculate the conflict coefficient:

[0160]

[0161] Specifically:

[0162] K = m1(No Defect) • (m2(Minor Defect) + m2(Slight Defect) + m2(Severe Defect)) + m1(Minor Defect) • (m2(No Defect) + m2(Slight Defect) + m2(Severe Defect)) + m1(Slight Defect) • (m2(No Defect) + m2(Minor Defect) + m2(Severe Defect)) + m1(Severe Defect) • (m2(No Defect) + m2(Minor Defect) + m2(Slight Defect))

[0163] Using Dempster's combination rules, the two BPAs are combined to obtain a new BPA m:

[0164]

[0165] Specifically, calculate the new BPA for each state:

[0166]

[0167]

[0168]

[0169]

[0170] Finally, based on the principle of maximum probability, the state with the highest probability is selected as the final evaluation result:

[0171] If m (no defects) is the largest, then it is determined to be no defects;

[0172] If m (minor defect) is the largest, it is judged as a minor defect;

[0173] If m (more severe defect) is the largest, it is judged as a more severe defect;

[0174] If m (critical defect) is the largest, it is judged as a critical defect.

[0175] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0176] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Those skilled in the art can readily implement the present invention based on the accompanying drawings and the above description. However, any modifications, alterations, or variations made by those skilled in the art without departing from the scope of the present invention, utilizing the disclosed technical content, are equivalent embodiments of the present invention. Furthermore, any modifications, alterations, or variations made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the present invention.

Claims

1. A pipeline defect evaluation method based on multi-source data fusion, characterized in that, The method comprises the following steps: acquiring data of the pipeline body operating state, including static data and image data; preprocessing the static data and the image data; analyzing and evaluating the preprocessed static data based on a bird swarm algorithm optimized extreme learning machine, to obtain a pipeline defect evaluation result of the static data; analyzing and evaluating the preprocessed image data based on a VGG16 model, to obtain a pipeline girth weld defect evaluation result of the image data; performing decision-level fusion on the pipeline defect evaluation result of the static data and the pipeline girth weld defect evaluation result of the image data based on a D-S evidence reasoning algorithm, to obtain a final pipeline defect evaluation result; analyzing and evaluating the preprocessed static data based on a bird swarm algorithm optimized extreme learning machine, and the specific process is as follows: optimizing the initial weight and bias of the extreme learning machine through the bird swarm algorithm, defining the mean square error of the extreme learning machine on the training set as a fitness function, and minimizing the fitness function value by iteratively optimizing the position of the bird individual, to realize optimization of the extreme learning machine; inputting the preprocessed static data as the feature of the extreme learning machine, inputting the defect evaluation result of the pipeline as the label of the extreme learning machine, setting the input weight, bias range, number of hidden layer neurons, bird swarm quantity and maximum iteration number of the extreme learning machine; wherein the number of input neurons is the actual value category number of the welding process data and the feature weld data; the number of output neurons is 4, representing the pipeline defect category after fusion, the static data is trained, and after the training, the static feature data of the pipeline body is input into the extreme learning machine, to obtain the evaluation result of the pipeline defect; wherein the pipeline defect category includes severe defect, relatively severe defect, slight defect and no defect; the specific process of optimizing the extreme learning machine is as follows: the input weight and bias of the extreme learning machine are used to initialize each bird, then the corresponding regression error fitness value of the extreme learning machine is calculated according to the parameters possessed by each bird, and the optimal fitness value and the corresponding weight and bias are recorded, after the flight behavior is executed, the updated fitness value is calculated, and after the iteration is updated, the optimal fitness value is assigned to the extreme learning machine; a regularization parameter balancing error matrix and an output matrix are introduced into the extreme learning machine, a sample weighting matrix is introduced into the objective function, and the sample weighting matrix is constructed using a Gaussian kernel function, and the weighting processing is performed according to the similarity between the training sample and the to-be-predicted sample; analyzing and evaluating the preprocessed image data based on the VGG16 model, and the specific process is as follows: extracting the features of the girth weld negative data and the girth weld defect image data in the image data through the VGG16 network, reducing the number of features through the PCA dimension reduction technology, and then adopting an attention feature fusion method to fuse the girth weld defect features and the girth weld negative features to form a unified fusion feature set, the fusion features are output through a two-layer fully connected network, and the pipeline girth weld defect evaluation result is obtained.

2. The method of claim 1, wherein, labeling the pipeline state and fault type for each data in the data of the pipeline body operating state; The pipeline state includes normal, slight defect and serious defect; and the fault type includes corrosion, crack and deformation.

3. The method of claim 1, wherein, The static data preprocessing process is: The welding process data and the characteristic weld data in the static data are standardized by using Z-score, converted into a standard normal distribution with zero mean and unit variance, and then normalized by using Min-Max.

4. The method of claim 1, wherein, The image data preprocessing process is: The girth weld image in the pipeline body is preprocessed, including data enhancement processing, denoising processing and edge enhancement processing of the girth weld image; The data enhancement processing of the girth weld image is performed by using a generative adversarial network method; The data enhancement processing of the girth weld image is converting the girth weld image into an RGB channel, adjusting the size to 224x224 pixels, scaling the pixel intensity to the range of 0-1, and performing data enhancement through random geometric transformation; The denoising processing of the girth weld image is removing the noise of the girth weld image by using a Gaussian filter; The edge processing of the enhanced image of the girth weld image is edge detection by using a Sobel operator, highlighting the defect edges of the girth weld by calculating the difference between the gray value of each pixel point in the image and the gray value of the surrounding pixel points.

5. The method of claim 1, wherein, The decision-level fusion of the defect evaluation conclusions obtained based on the D-S evidence reasoning algorithm for the static data and the image data is specifically: Based on D-S evidence reasoning algorithm, define recognition framework , represent all possible states; according to the static data analysis and girth weld image analysis of pipeline defect evaluation conclusion, respectively for each state is given the basic probability, namely the result of static data analysis m1, girth weld image analysis result m2, conflict coefficient K represents the degree of inconsistency between the two evidence, through the calculation of conflict coefficient K and Dempster combination rule of static data analysis and girth weld image analysis of pipeline defect evaluation conclusion combined, draw the fusion evaluation result; Wherein Θ includes no defect, slight defect, relatively serious defect and serious defect.

6. A pipeline defect assessment system based on multi-source data fusion, based on the pipeline defect assessment method based on multi-source data fusion of any one of claims 1-5, characterized in that, It includes The data acquisition module, the preprocessing module, the static data analysis and evaluation module, the image data analysis and evaluation module and the decision-level fusion evaluation module; The data acquisition module is used for acquiring the data of the pipeline body running state, including static data and image data; The preprocessing module is used for preprocessing the static data and the image data; The static data analysis and evaluation module is used for analyzing and evaluating the preprocessed static data based on the bird swarm algorithm optimized extreme learning machine, to obtain the pipeline defect evaluation result of the static data; The image data analysis and evaluation module is used for analyzing and evaluating the preprocessed image data based on the VGG16 model, to obtain the pipeline girth weld defect evaluation result of the image data; The decision-level fusion evaluation module is used for performing decision-level fusion on the pipeline defect evaluation result of the static data and the pipeline girth weld defect evaluation result of the image data based on the D-S evidence reasoning algorithm, to obtain the final pipeline defect evaluation result.

Images