Pipeline circumferential weld defect detection method and system based on deep learning

Abstract

The invention relates to pipeline circumferential weld detection, and discloses a pipeline circumferential weld defect detection method and system based on deep learning, multiple sensors are arranged on the surface layer of a pipeline to collect multi-modal signals, an inertial measurement device is combined to carry out pose correction, and a multi-modal time sequence data matrix is constructed; performing preprocessing and segmentation, calculating to obtain a dynamic reference band, and outputting upper and lower bounds; performing broken line fitting on each segment to extract a phase change factor triple, and comparing with a reference band to screen event candidate segments; constructing a multi-segment time domain coupling relation graph based on the candidate segments, and generating a dominant response mode candidate set; inputting a multi-branch time sequence coding network to extract a three-mode feature and generate a time sequence fusion feature; and performing similarity matching with a defect prototype library to obtain a defect category probability sequence, performing single-mode ablation and time sequence playback verification to obtain a high-confidence index, mapping the high-confidence index to a spatial position, and outputting a defect category and severity in combination with a feature index. According to the method, high-precision defect detection and reliable classification under the trenchless condition can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to pipeline girth weld detection, in particular to a pipeline girth weld defect detection method and system based on deep learning. BACKGROUND

[0002] As an important infrastructure for energy transportation, the quality of the weld of the oil and gas pipeline is directly related to the operation safety and service life of the pipeline. Due to the influence of factors such as welding stress concentration, welding defects and external load, the girth weld is prone to defects such as incomplete penetration, porosity, slag inclusion and cracks. The existing pipeline girth weld detection methods mainly include excavation detection, internal detection and weak magnetic external detection, etc. Among them, although the excavation detection can directly observe the weld state, it has long construction period, high cost and is destructive; the internal detection depends on the ball passability of the pipeline and specific detection devices, and has poor applicability to complex working conditions and some small diameter pipelines; although the weak magnetic external detection can realize non-excavation detection, the existing technology mainly depends on the amplitude characteristics or kurtosis characteristics of a single magnetic signal for peak positioning, which is easily disturbed by factors such as surrounding metal structures, changes in geological conditions and changes in sensor posture, resulting in insufficient detection stability and accuracy, and it is difficult to accurately classify and evaluate the severity of different types of defects.

[0003] With the development of multi-modal sensing technology and deep learning, the fusion analysis of multiple sensing signals such as magnetic flux response, ground micro-vibration and electric field disturbance, combined with time sequence feature extraction and pattern recognition, can improve the accuracy and robustness of girth weld defect detection under non-excavation conditions. However, the existing methods still have deficiencies in multi-modal time sequence data alignment, feature fusion, candidate event screening and result verification, and there is an urgent need for a technical solution that can fully utilize multi-source information, reduce false detection rate and output high-confidence defect results. SUMMARY

[0004] In view of this, the present application provides a pipeline girth weld defect detection method and system based on deep learning to solve the above problems.

[0005] In one aspect, the present application provides a pipeline girth weld defect detection method based on deep learning, comprising: disposing multiple types of response sensors at the surface layer of the pipeline, collecting original multi-modal signal sequences and recording the pose changes of the multiple types of response sensors relative to the ground through an inertial measurement device to correct the original multi-modal signal sequences, and obtaining a multi-modal time sequence data matrix; preprocessing and segmenting the multi-modal time sequence data matrix to obtain a plurality of time segment matrices, calculating the median absolute deviation of the background noise of the plurality of time segment matrices to form a dynamic reference band, and outputting the upper and lower bounds of the reference band of each time segment matrix; The broken line fitting is performed on the multiple time segment matrices to obtain phase change factor triplets of the multiple time segment matrices, the phase change factor triplets of the multiple time segment matrices are compared with the corresponding reference band upper and lower bounds to obtain event candidate segments and record event candidate segment start and end indexes; A multi-segment time domain coupling relationship graph is constructed based on the event candidate segments, and a dominant response mode candidate set is generated through the multi-segment time domain coupling relationship graph; The dominant response mode candidate set is input into a multi-branch time sequence coding network to obtain three output modal feature vector sequences; The feature vectors of the three modalities are input into a graph attention fusion network constructed based on a priori topology, the correlation of different modalities in the same time segment and adjacent time segments is weighted and fused to obtain a time sequence fusion feature sequence; The time sequence fusion feature sequence is subjected to similarity matching with a pre-established defect prototype feature library, and a defect category probability sequence and a corresponding time index sequence are output according to a matching result; The defect category probability sequence is subjected to multi-modal feature ablation and time sequence segment playback verification to obtain a high-confidence candidate index; The high-confidence candidate index is mapped to a pipeline space position, and a defect category and severity are output in combination with a category probability and a feature index.

[0006] Further, the multiple types of response sensors include a magnetic flux response sensor, a ground piezoelectric vibration sensor, and an electric field induction sensor. The original multi-modal signal sequence includes a magnetic flux response signal sequence, a ground micro-vibration interference signal sequence, and a low-frequency electric field disturbance signal sequence.

[0007] Further, the obtaining of the multi-modal time sequence data matrix specifically includes: synchronizing the original multi-modal signal sequence according to a unified time reference, correcting the positions and postures of the multiple types of response sensors in the collection process by using data in an inertial measurement device, and arranging and combining the corrected magnetic flux response signal sequence, the ground micro-vibration interference signal sequence, and the low-frequency electric field disturbance signal sequence in time sequence to form a multi-modal time sequence data matrix with consistent column vectors. The correcting of the positions and postures of the multiple types of response sensors in the collection process specifically includes: using fusion solving to correct the time indexes and spatial displacements of the original multi-modal signal sequence by using three-axis acceleration and three-axis angular velocity in an inertial measurement device and synchronously collected displacement encoding information.

[0008] Further, the obtaining of the multiple time segment matrices specifically includes: performing de-drifting and band-pass noise suppression processing on the multi-modal time sequence data matrix, and cutting the multi-modal time sequence data matrix into continuous and non-overlapping multiple time segment matrices with the length of multiple periods as a fixed window. The specific content of the output of the upper and lower boundaries of the reference band of each time segment matrix is that a set of sample points not containing significant response peaks in each time segment matrix is taken as background samples, the median absolute deviation of the set of sample points is calculated and multiplied by a preset proportion coefficient to obtain a dynamic reference band, the center of the dynamic reference band is set as the mean value of the background samples, and the upper and lower boundaries of the reference band of each time segment matrix are determined by adding and subtracting the bandwidth to the center of the dynamic reference band.

[0009] Further, the specific content of obtaining the phase change factor triplets is that the broken line fitting is performed on each time segment sequence segment, the broken line slope mutation points and inversion points are extracted, and the mutation points of the multi-modal time sequence data matrix in the same time segment sequence segment are time-aligned, the turning synchronization degree, the main frequency consistency degree and the energy distribution offset are calculated, and the phase change factor triplets of the time segment sequence are formed. The specific content of obtaining the event candidate segment and recording the start and end indexes of the event candidate segment is that for each time segment matrix, it is judged whether any two items in the corresponding phase change factor triplet exceed the upper boundary of the reference band of the corresponding time segment matrix at the same time, if the condition is met, the time segment matrix is marked as an event candidate segment, and the start sampling index and the end sampling index of the time segment matrix in the original time sequence are recorded as the start and end indexes of the event candidate segment.

[0010] Further, the specific content of constructing the multi-segment time domain coupling relationship graph is that the event candidate segment is taken as a node of the graph, the comprehensive similarity of the phase change factor triplet of the event candidate segment is taken as an edge weight, the nodes satisfying the edge weight greater than a set threshold edge weight are connected, and the multi-segment time domain coupling relationship graph of the association between different time segments is formed. The specific content of generating the dominant response mode candidate set is that in the multi-segment time domain coupling relationship graph, a connected subgraph composed of multiple nodes and having edge weights all higher than a set threshold edge weight is selected, and the event candidate segments corresponding to the nodes in each connected subgraph are combined to obtain the dominant response mode candidate set.

[0011] Further, the specific content of obtaining the three output modal feature vector sequences is that the dominant response mode candidate set is input into a multi-branch time sequence encoding network, a one-dimensional hollow convolution stacking and residual connection time sequence convolution network is used to encode the multi-scale slope change of the magnetic response for the magnetic flux response branch in the dominant response mode candidate set, a one-dimensional convolution-gated recurrent unit hybrid structure is used to encode the transient frequency band energy cluster for the surface micro-vibration branch by inputting the amplitude and phase, and a one-dimensional convolution and causal hollow convolution cascaded time sequence network is used to encode the low-frequency phase drift for the low-frequency electric field branch to obtain the three output modal feature vector sequences.

[0012] Further, the obtained time sequence fusion feature sequence specifically includes: taking a multi-segment time domain coupling relationship graph as a prior topology, constructing a graph attention fusion layer, applying directional attention weights between the same time period and adjacent time periods of a multi-modal time sequence data matrix, performing message passing on the multi-segment time domain coupling relationship graph on three output modal feature vector sequences and outputting a fusion feature sequence, and parallelly feeding the fusion feature sequence into a double-path time sequence reasoning unit, wherein a first path is a gated recurrent unit stack, and a second path is a time sequence convolutional network, and the double paths output are combined in residual to obtain a time sequence fusion feature.

[0013] Further, the high-confidence candidate index is obtained by: performing single-modal ablation playback and time sequence segment playback verification on the time index corresponding to the peak value of each defect category probability sequence, respectively, if the peak value decreases after single-modal ablation and the attention weight at the time index is inconsistent, the corresponding defect category probability sequence is marked as an inconsistent point, if the peak value does not appear in a phase stable interval after time sequence segment playback, the corresponding defect category probability sequence is marked as an unstable point, and when there is no inconsistent point and unstable point at the same time index, the corresponding defect category probability sequence is marked as a high-confidence candidate index.

[0014] Compared with the prior art, the present application has the following advantages: By arranging various sensors such as magnetic flux response, ground piezoelectric vibration and electric field induction on the surface of the pipeline, multi-modal signals are collected and aligned under a unified time reference, after attitude and position correction, a multi-modal time sequence data matrix consistent in time and space is constructed, avoiding the defect that a single signal is easily disturbed by environmental noise; the median absolute deviation of the background noise of the segmented multi-modal time sequence signal is calculated to form a dynamic reference band, and the event candidate segment is extracted with reference to the dynamic reference band, effectively filtering false signals caused by environmental interference, and improving the authenticity of the candidate event; the event candidate segment is constructed as a multi-segment time domain coupling relationship graph, and the dominant response mode candidate set is screened through the phase change factor comprehensive similarity between nodes, which can capture related features across time segments and reduce missed detection; a multi-branch time sequence coding network optimized for different modalities is used to extract multi-scale slope change features of magnetic flux response, transient frequency band energy features of ground micro-vibration and phase drift features of low-frequency electric field, ensuring the pertinence and effectiveness of feature extraction of each modality; the similarity of the fusion features is matched with a pre-established defect prototype feature library, which can distinguish different defect types such as incomplete penetration, undercut, porosity, slag inclusion and crack, and output probability results; two verification methods of single-modal feature ablation and time sequence segment playback are used to eliminate inconsistent points and unstable points, and only high-confidence candidate indexes are retained, which greatly reduces the false detection rate.

[0015] In another aspect, the present application provides a pipeline girth weld defect detection system based on deep learning, which comprises: a data acquisition module configured to acquire an original multi-modal signal sequence and record pose changes of a plurality of response sensors relative to the ground through an inertial measurement device and correct the original multi-modal signal sequence to obtain a multi-modal time sequence data matrix; a data preprocessing module configured to preprocess and segment the multi-modal time sequence data matrix to obtain a plurality of time segment matrices, calculate a median absolute deviation of background noise of the plurality of time segment matrices to form a dynamic reference band, and output upper and lower bounds of the reference band of each time segment matrix; an event candidate extraction module configured to perform polyline fitting on the plurality of time segment matrices to obtain phase change factor triplets of the plurality of time segment matrices, compare the phase change factor triplets of the plurality of time segment matrices with corresponding upper and lower bounds of the reference band to obtain event candidate segments and record start and end indexes of the event candidate segments; an alignment and graph structure construction module configured to construct a multi-segment time domain coupling relationship graph based on the event candidate segments and generate a dominant response mode candidate set through the multi-segment time domain coupling relationship graph; a deep learning module configured to input the dominant response mode candidate set into a multi-branch time sequence encoding network to obtain three output modal feature vector sequences; a cross-modal graph attention fusion module configured to input the feature vectors of the three modalities into a graph attention fusion network constructed based on prior topology, weight and fuse correlations of different modalities in the same time segment and adjacent time segments to obtain a time sequence fusion feature sequence; a comparison and determination module configured to perform similarity matching between the time sequence fusion feature sequence and a pre-established defect prototype feature library, output a defect category probability sequence and a corresponding time index sequence according to a matching result; a verification module configured to perform multi-modal feature ablation and time sequence segment playback verification on the defect category probability sequence to obtain a high-confidence candidate index; an output module configured to map the high-confidence candidate index to a pipe space position, output a defect category and a severity in combination with a category probability and a feature index.

[0016] It should be noted that the pipe girth weld defect detection method based on deep learning has the same beneficial effects as the pipe girth weld defect detection system, and thus will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments and are not meant to limit the present application. Moreover, the same reference numerals in the attached drawings indicate the same or similar components. In the drawings: Figure 1 A flow chart of a pipeline girth weld defect detection method based on deep learning provided by an embodiment of the present application.

[0018] Figure 2 A functional block diagram of a pipeline girth weld defect detection system based on deep learning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0019] Exemplary embodiments of the present application will be described in detail with reference to the drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood, and the scope of the present disclosure can be accurately conveyed to those skilled in the art. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0020] Referring to Figure 1 As shown in the drawings, the embodiment of the present application provides a pipeline girth weld defect detection method based on deep learning, comprising: S1: arranging multiple types of response sensors at the surface layer of the pipeline, collecting original multi-modal signal sequences and recording the pose changes of the multiple types of response sensors relative to the ground through an inertial measurement device and correcting the original multi-modal signal sequences to obtain a multi-modal time series data matrix; S2: pre-processing and segmenting the multi-modal time series data matrix to obtain a plurality of time segment matrices, calculating the median absolute deviation of the background noise of the plurality of time segment matrices to form a dynamic reference band, and outputting the upper and lower bounds of the reference band of each time segment matrix; S3: performing polyline fitting on the plurality of time segment matrices to obtain a phase change factor triplet of the plurality of time segment matrices, comparing the phase change factor triplet of the plurality of time segment matrices with the corresponding reference band upper and lower bounds to obtain event candidate segments and record the start and end indexes of the event candidate segments; S4: constructing a multi-segment time domain coupling relationship graph based on the event candidate segments, and generating a dominant response mode candidate set through the multi-segment time domain coupling relationship graph; S5: inputting the dominant response mode candidate set into a multi-branch time sequence encoding network to obtain three output modal feature vector sequences; S6: inputting the feature vectors of the three modalities into a graph attention fusion network based on a priori topology, weighting and fusing the correlation of different modalities in the same time segment and adjacent time segments to obtain a time sequence fusion feature sequence; S7: similarity matching between the time sequence fusion feature sequence and the pre-established defect prototype feature library, outputting a defect category probability sequence and a corresponding time index sequence according to a matching result; S8: multi-modal feature ablation and time sequence segment playback verification on the defect category probability sequence, to obtain a high-confidence candidate index; S9: mapping the high-confidence candidate index to a pipeline space position, and outputting a defect category and severity in combination with a category probability and a feature index.

[0021] In some embodiments of the present application, the multiple types of response sensors include a magnetic flux response sensor, a ground piezoelectric vibration sensor, and an electric field induction sensor. The original multi-modal signal sequence includes a magnetic flux response signal sequence, a ground micro-vibration interference signal sequence, and a low-frequency electric field disturbance signal sequence.

[0022] Specifically, the magnetic flux response signal sequence reflects the trend of the magnetic field strength along the detection path position changing over time; the ground micro-vibration interference signal sequence describes the vibration amplitude and frequency spectrum distribution of the ground at different positions during the detection process; and the low-frequency electric field disturbance signal sequence reflects the variation characteristics of the low-frequency electric field intensity around the buried conductor. The response mechanisms of the three types of signals to the weld defects are different. The magnetic flux response signal is sensitive to the magnetic permeability and stress concentration of the weld position, the vibration signal is sensitive to the local stiffness change of the pipeline structure, and the electric field signal is sensitive to the charge distribution change of the conductor. By simultaneously acquiring and analyzing the above signal sequences, the state characteristics of the pipeline girth weld can be described from different physical dimensions, and more accurate determination of the defect position and type can be achieved.

[0023] In some embodiments of the present application, the multi-modal time sequence data matrix specifically includes: synchronizing the original multi-modal signal sequence according to a unified time reference, correcting the position and attitude of the multiple types of response sensors during the acquisition process by using the data in the inertial measurement device, and arranging and combining the corrected magnetic flux response signal sequence, the ground micro-vibration interference signal sequence, and the low-frequency electric field disturbance signal sequence in time sequence to form a multi-modal time sequence data matrix with consistent column vectors. The specific content of correcting the position and attitude of the multiple types of response sensors during the acquisition process includes: using fusion solving to correct the time index and spatial displacement of the original multi-modal signal sequence by using the three-axis acceleration and three-axis angular velocity in the inertial measurement device together with the synchronously collected displacement encoding information.

[0024] Specifically, the magnetic flux response sensor, the ground piezoelectric vibration sensor and the electric field induction sensor respectively collect corresponding physical signals in real time through the data acquisition module. In order to ensure the time consistency of the original multi-modal signal sequence, all sensor acquisition channels are controlled by the same clock source, and time stamp calibration is performed at the beginning of each acquisition, so as to avoid sampling delay or time drift between different modal signals. In order to eliminate the measurement error caused by the change of the sensor position and attitude during the acquisition process, an inertial measurement device (IMU) is introduced for pose monitoring and correction in the embodiment. The inertial measurement device includes a three-axis accelerometer and a three-axis gyroscope, which are used to measure the acceleration information and angular velocity information of the sensor in space, respectively. At the same time, displacement data output by a displacement encoder is also synchronously collected during the sensor movement detection process, which is used to accurately depict the spatial position change of the sensor relative to the pipeline and the ground. The three-axis acceleration data and the three-axis angular velocity data output by the inertial measurement device are combined with the displacement information of the displacement encoder, and the attitude sequence and the position trajectory of the sensor in the entire detection process are obtained through a fusion solving algorithm (such as a Kalman filter-based attitude solving method). The attitude sequence is used to correct the error related to the change of signal amplitude and sensor pointing angle, and the position trajectory is used to correct the signal in space indexing, so as to realize the alignment of the original multi-modal signal sequence in the unified space reference system. After the above correction processing, the magnetic flux response signal sequence, the ground micro-vibration interference signal sequence and the low-frequency electric field disturbance signal sequence are arranged in the order of acquisition time, and it is ensured that at each time sampling point, the three types of signals constitute the corresponding components in the column vector, thereby forming a multi-modal time sequence data matrix with consistent column vector, time alignment and unified space reference system.

[0025] In some embodiments of the present application, the obtaining of the plurality of time segment matrices specifically includes: performing de-drifting and band-pass noise suppression processing on the multi-modal time sequence data matrix, and cutting the multi-modal time sequence data matrix into a plurality of continuous and non-overlapping time segment matrices with the length of a plurality of cycles as a fixed window. The specific content of the output of the upper and lower boundaries of the reference band of each time segment matrix includes: taking a sample point set not containing a significant response peak in each time segment matrix as a background sample, calculating the median absolute deviation of the sample point set and multiplying it by a preset proportion coefficient to obtain a dynamic reference band, taking the center of the dynamic reference band as the mean value of the background sample, and determining the upper and lower boundaries of the reference band of each time segment matrix by adding and subtracting the bandwidth from the center of the dynamic reference band, respectively.

[0026] Specifically, the obtained multi-modal time series data matrix is first subjected to de-drifting and band-pass noise suppression processing. The de-drifting processing is used to eliminate the low-frequency baseline drift phenomenon caused by factors such as temperature change and electronic drift of the sensor during long-time operation. The slow trend of the signal can be calculated by using methods such as robust local regression (Robust LOESS) or sliding mean subtraction, and the trend component is deducted from the original signal, so that the mean of the signal tends to zero, facilitating subsequent analysis. The purpose of the band-pass noise suppression processing is to retain the frequency component of the target defect response and suppress low-frequency environmental interference and high-frequency random noise. For different modal signals, different band-pass filter parameters can be selected. For example, the magnetic flux response signal can use a finite impulse response (FIR) band-pass filter covering 1-30 Hz, the ground micro-vibration signal can use an adaptive band-pass filter around the dominant vibration frequency band, and the low-frequency electric field signal can use a band-pass filter with a lower cutoff frequency to retain the slow-changing electric field disturbance component. After de-drifting and band-pass noise suppression processing, the multi-modal time series data matrix is cut into fixed-length time windows. The length of each window is set to a time range containing not less than several complete periods of the dominant frequency of the target defect response, which can ensure that each time segment matrix contains sufficient feature information and that adjacent segments do not overlap in time, avoiding data redundancy and feature leakage. The sample point set in the time segment matrix that does not contain significant response peaks is taken as the background sample. Significant response peaks can be removed by amplitude threshold or energy mutation detection method. The median absolute deviation (MAD) of the background sample set is calculated as a robust statistic of background noise, and the value is multiplied by a preset proportion coefficient to obtain a dynamic reference bandwidth. The center position of the reference band is set to the mean of the background samples, and the upper and lower boundaries of the reference band are determined by adding or subtracting the reference bandwidth from the center value, which can effectively adapt to the changes of background noise of different detection environments and different modal signals, and significantly improve the sensitivity and anti-interference ability of defect detection.

[0027] In some embodiments of the present application, the phase change factor triplet is obtained by: performing polyline fitting on each time segment sequence, extracting the polyline slope mutation points and inversion points, and performing time alignment on the mutation points of the multi-modal time series data matrix in the same time segment sequence, calculating the turning synchronization degree, the main frequency consistency degree and the energy distribution offset, and forming the phase change factor triplet of the time segment sequence. The event candidate segment is obtained and the start and end indexes of the event candidate segment are recorded by: for each time segment matrix, judging whether any two items in the corresponding phase change factor triplet exceed the upper boundary of the reference band of the corresponding time segment matrix at the same time, if the condition is met, marking the time segment matrix as an event candidate segment, and recording the start sampling index and the end sampling index of the time segment matrix in the original time sequence as the start and end indexes of the event candidate segment.

[0028] Specifically, first, a broken line fitting process is performed on each time segment sequence segment, the broken line fitting adopts a piecewise least square method to model the time sequence signal, the continuous sampling points are approximately represented by a series of line segments with fixed slope, by calculating the local maximum points of the fitting residual, the positions where the signal change trend changes significantly can be effectively located, these positions are the slope mutation points of the broken line; when the fitting slope sign changes, mark the position as a reversal point; in the same time segment sequence segment, for the multi-modal time sequence data matrix, the mutation points and reversal points are extracted respectively, and a time alignment operation is performed, after completing the time alignment, three indexes for describing the correlation degree between modes are calculated: turning synchronous degree: measure the consistency of the time position of the mutation points of different modes in the segment; dominant frequency consistency: extract the dominant frequency of each mode by fast Fourier transform, calculate the normalized reciprocal of the frequency difference; energy distribution offset: calculate the energy distribution vector of the time segment in the low frequency, medium frequency and high frequency band, and measure the distance with the corresponding vector of the adjacent segment; combine the above three indexes in a predetermined order to form the phase change factor triplet of the time segment sequence, which is used to represent the comprehensive characteristics of the multi-modal signal in time synchronization, frequency characteristics and energy distribution change in the time period; after obtaining the phase change factor triplet of all time segments, event candidate judgment is performed on each time segment matrix, the judgment method is: compare each index in the phase change factor triplet of the segment with the upper bound of the dynamic reference band of the segment, when any two indexes in the triplet exceed the upper bound of the corresponding reference band at the same time, it is determined that the segment is an event candidate segment, indicating that the segment may contain defect-related response characteristics; for the time segment matrix determined as an event candidate segment, record its starting sampling index and ending sampling index in the original multi-modal time sequence as the time range information of the event candidate segment.

[0029] In some embodiments of the present application, the multi-segment time domain coupling relationship graph is constructed by taking the event candidate segment as the node of the graph, taking the comprehensive similarity of the phase change factor triplet of the event candidate segment as the edge weight, connecting the nodes that satisfy the edge weight greater than the set threshold edge weight, and forming the multi-segment time domain coupling relationship graph of the correlation between different time periods. The dominant response mode candidate set is generated by selecting a connected subgraph composed of multiple nodes and having edge weights higher than the set threshold edge weight in the multi-segment time domain coupling relationship graph, and combining the event candidate segments corresponding to the nodes in each connected subgraph as the dominant response mode candidate set.

[0030] Specifically, first, the event candidate segment is taken as the node of the graph, each node contains not only the time range information (start sample index and end sample index) of the event candidate segment, but also the phase change factor triplet value of the segment, that is, the three indicators of turning point synchronization degree, main frequency consistency and energy distribution offset; in order to establish the association between nodes, the comprehensive similarity is defined as the edge weight of the graph, and the calculation method of the comprehensive similarity can be: the phase change factor triplets of the two event candidate segments are calculated according to the elements respectively (for example, cosine similarity or Euclidean distance normalization value), and then the similarity of the three indicators is weighted and summed according to the preset weight, so as to obtain the edge weight reflecting the overall similarity of the two time segments in the multi-modal response mode; when the comprehensive similarity between two nodes is greater than a set threshold, the two nodes are connected in the graph, and the edge weight is assigned to the connected edge, and the multi-segment time domain coupling relationship graph constructed in this way can directly represent the mutual association of different time segments in the response mode, wherein the higher the edge weight, the more similar the two response modes; in the constructed multi-segment time domain coupling relationship graph, a connected subgraph composed of multiple nodes and all edge weights higher than the set threshold is selected, the connected subgraph represents that each node (event candidate segment) in it has high similarity with other nodes in the mode characteristics; the event candidate segments corresponding to the nodes in each connected subgraph are combined to form a dominant response mode candidate set; the dominant response mode candidate set represents a group of response events highly consistent in time domain and characteristic domain, corresponding to the same defect type or defect signal of the same physical region.

[0031] In some embodiments of the present application, the specific content of obtaining three output modal feature vector sequences is: inputting the dominant response mode candidate set into a multi-branch time sequence encoding network, using a one-dimensional dilated convolution stacking and residual connection time sequence convolution network to encode the multi-scale slope change of the magnetic response in the magnetic flux response branch of the dominant response mode candidate set; using a one-dimensional convolution-gated recurrent unit hybrid structure to encode the transient frequency band energy cluster in the surface micro-vibration branch; using a one-dimensional convolution and causal dilated convolution cascaded time sequence network to encode the low-frequency phase drift in the low-frequency electric field branch to obtain three output modal feature vector sequences.

[0032] Specifically, the one-dimensional dilated convolution stacking and residual connection combined time sequence convolution network structure is used in the magnetic flux response branch, the one-dimensional dilated convolution can expand the receptive field without increasing the calculation amount, so as to obtain short-term and long-term dependent features at the same time, and the residual connection can avoid the problem of gradient disappearance when the network is deepened, and enhance the continuity of feature transmission. The output of this branch is a feature vector sequence encoding the multi-scale slope change characteristics of the magnetic communication signal; In the surface micro-vibration branch, first, the short-time Fourier transform (STFT) is performed on the surface micro-vibration interference signal sequence, the time-domain signal is converted into a time-frequency domain representation, and the magnitude spectrum thereof is extracted as the network input. Subsequently, a one-dimensional convolution layer is used to extract the local mode of the frequency distribution, and then a gated recurrent unit (GRU) is used to capture the time-dependent characteristics of the frequency energy variation. The convolution-GRU hybrid structure has both the ability to capture local patterns and the ability to model time dynamics, and finally outputs a feature vector sequence that encodes the transient frequency band energy cluster distribution characteristics. In the low-frequency electric field branch, a time series network structure of one-dimensional convolution and causal dilated convolution is adopted. The one-dimensional convolution layer is used to preliminarily extract local variation characteristics, and the causal dilated convolution can expand the receptive field while ensuring the time causality, thereby effectively capturing the phase drift trend over a long time span. The output of this branch is a feature vector sequence that encodes the low-frequency phase variation mode.

[0033] In some embodiments of the present application, the specific content of the time series fusion feature sequence is as follows: a graph attention fusion layer is constructed with a multi-segment time domain coupling relationship graph as the prior topology, a directional attention weight is applied between the same time period and the adjacent time period of the multi-modal time series data matrix, the three output modal feature vector sequences are subjected to message passing on the multi-segment time domain coupling relationship graph and output a fusion feature sequence, and the fusion feature sequence is sent into a double-path time series reasoning unit in parallel. The first path is a gated recurrent unit stack, and the second path is a time series convolution network. The outputs of the double paths are combined in the same time period to obtain a time series fusion feature.

[0034] Specifically, the multi-segment time domain coupling relationship graph is input into the graph attention fusion layer as a prior topological structure, wherein the nodes in the multi-segment time domain coupling relationship graph represent each time segment in the dominant response mode candidate set, the edges represent the mode similarity between the time segments, and the edge weights reflect the similarity degree. In the graph attention mechanism (GAT), the feature vector of each node is combined with the feature vectors of its adjacent nodes in a weighted manner, and the weight coefficient is determined by the feature similarity between the nodes and the graph prior edge weight. In this embodiment, a directional constraint is introduced when calculating the attention weight, and the feature vectors of the three modalities are subjected to cross-modal message passing in the graph attention fusion layer. The process of message passing can be understood as follows: the fusion features of each time segment are composed of not only the multi-modal features of the segment itself, but also the feature information of other time segments connected to it in the graph, thereby introducing global mode relationships in the feature space. After multiple rounds of graph attention calculation, the output fusion feature sequence can contain both the local detail information within the time sequence and the mode consistency information across time segments; the obtained fusion feature sequence is input into the double-path time sequence reasoning unit in parallel, and the double-path structure includes: the first path is a GRU stack, which is suitable for capturing the dependence relationship and rapidly changing mode features in a short time scale. The GRU stack controls the flow of information through a gating mechanism and can efficiently model the nonlinear dependence relationship between time segments; the second path is a TCN, which obtains the feature mode in a long time span through a multi-layer one-dimensional convolution and dilated convolution structure, and is suitable for capturing slowly changing or global trend features across multiple time segments; after the double-path parallel calculation is completed, the output feature vectors of the first path and the second path at the same time segment index are subjected to residual fusion, the residual fusion is performed by adding the output vectors of the two paths and processing them through a nonlinear activation function, thereby enhancing the feature complementarity while maintaining the independence of the features of each path. The finally obtained time sequence fusion feature vector sequence contains both short-time dynamic information and long-time trend features, thereby providing a rich and robust input representation for subsequent defect judgment.

[0035] In some embodiments of the present application, obtaining the high-confidence candidate index specifically includes: performing single-modality ablation playback and time-segment reverse playback verification on the time index corresponding to the peak value of each defect category probability sequence, respectively; if the peak value decreases after single-modality ablation and the decrease is inconsistent with the attention weight at the time index, marking the corresponding defect category probability sequence as an inconsistent point; if the peak value does not appear in a phase stable interval after time-segment reverse playback, marking the corresponding defect category probability sequence as an unstable point; and when there is neither an inconsistent point nor an unstable point at the same time index, marking the corresponding defect category probability sequence as a high-confidence candidate index.

[0036] In some embodiments of the present application, outputting the defect category probability sequence and the corresponding time index sequence specifically includes: obtaining a defect prototype dictionary based on offline training of the excavation verified sample, wherein a prototype vector represents a time response center of incomplete penetration, undercut, porosity, slag inclusion, and crack; projecting the time fusion feature into a prototype subspace, calculating a similarity vector with each prototype vector, and outputting the defect category probability sequence and the corresponding time index sequence.

[0037] In some embodiments of the present application, outputting the defect category and the severity specifically includes: mapping the high-confidence candidate index back to the multi-category response sensor track coordinates, combining the pose correction result to obtain the spatial position of the defect; and outputting the defect category and the severity level according to a rule combination of three indicators including the defect category probability, the maximum component of the similarity vector, and the energy concentration degree of the fusion feature.

[0038] Referring to Figure 2 The embodiment of the present application provides a pipeline girth weld defect detection system based on deep learning, which comprises: A data acquisition module is configured to acquire an original multi-modal signal sequence and record the pose change of the multi-category response sensor relative to the ground through an inertial measurement device to correct the original multi-modal signal sequence and obtain a multi-modal time sequence data matrix; A data preprocessing module is configured to preprocess and segment the multi-modal time sequence data matrix to obtain a plurality of time segment matrices, calculate the median absolute deviation of background noise to form a dynamic reference band, and output the upper and lower bounds of the reference band of each time segment matrix; An event candidate extraction module is configured to perform polyline fitting on the plurality of time segment matrices to obtain a plurality of phase change factor triplets of the plurality of time segment matrices, compare the plurality of phase change factor triplets with the corresponding upper and lower bounds of the reference band, obtain event candidate segments, and record the start and end indexes of the event candidate segments; An alignment and graph structure construction module is configured to construct a multi-segment time domain coupling relationship graph based on the event candidate segments, and generate a dominant response mode candidate set through the multi-segment time domain coupling relationship graph. a deep learning module configured to input the leading response mode candidate set into a multi-branch time sequence coding network to obtain three output modal feature vector sequences; a cross-modal graph attention fusion module configured to input the feature vectors of the three modalities into a graph attention fusion network based on a prior topology to weight and fuse the correlation of different modalities in the same time period and adjacent time periods to obtain a time sequence fusion feature sequence; a comparison determination module configured to perform similarity matching between the time sequence fusion feature sequence and a pre-established defect prototype feature library, and output a defect category probability sequence and a corresponding time index sequence according to the matching result; a verification module configured to perform multi-modal feature ablation and time sequence segment playback verification on the defect category probability sequence to obtain a high-confidence candidate index; an output module configured to map the high-confidence candidate index to a pipeline space position, and output a defect category and severity in combination with the category probability and feature index.

[0039] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it, although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that: the specific embodiments of the present application can still be modified or replaced by the equivalent, without departing from the spirit and scope of the present application, any modification or equivalent replacement should be covered within the protection scope of the claims of the present application.

Claims

1. A method for detecting defects in pipe circumferential welds based on deep learning, characterized in that, include: Multiple types of response sensors are deployed on the surface of the pipeline to collect the original multimodal signal sequence. The pose changes of the multiple types of response sensors relative to the ground are recorded by an inertial measurement device, and the original multimodal signal sequence is corrected to obtain a multimodal time series data matrix. The multimodal time series data matrix is ​​preprocessed and segmented to obtain multiple time segment matrices. The median absolute deviation of the background noise of the multiple time segment matrices is calculated to form a dynamic reference band, and the upper and lower bounds of the reference band of each time segment matrix are output. Pieceline fitting is performed on multiple time segment matrices to obtain phase transition factor triplets of multiple time segment matrices. The phase transition factor triplets of multiple time segment matrices are compared with the upper and lower bounds of the corresponding reference band to obtain event candidate segments and record the start and end indices of the event candidate segments. Based on event candidate segments, a multi-segment temporal coupling relationship graph is constructed, and a candidate set of dominant response modes is generated through the multi-segment temporal coupling relationship graph. The dominant response mode candidate set is input into a multi-branch temporal coding network to obtain three output mode feature vector sequences; The feature vectors of the three modalities are input into a graph attention fusion network constructed based on prior topology. The correlations of different modalities in the same time period and adjacent time periods are weighted and fused to obtain a temporal fusion feature sequence. The temporal fusion feature sequence is matched with a pre-established defect prototype feature library for similarity, and the defect category probability sequence and the corresponding time index sequence are output based on the matching results. Multimodal feature ablation and time segment replay verification are performed on the defect category probability sequence to obtain high-confidence candidate indices; The high-confidence candidate index is mapped to the pipeline spatial location, and the defect category and severity are output by combining the category probability and feature indicators.

2. The method for detecting defects in pipe circumferential welds based on deep learning according to claim 1, characterized in that, The various types of response sensors include magnetic flux response sensors, ground piezoelectric vibration sensors, and electric field induction sensors; The original multimodal signal sequence includes a magnetic flux response signal sequence, a surface micro-vibration interference signal sequence, and a low-frequency electric field disturbance signal sequence.

3. The method for detecting defects in pipe circumferential welds based on deep learning according to claim 2, characterized in that, The specific content of the obtained multimodal time series data matrix is ​​as follows: the original multimodal signal sequence is synchronized according to a unified time reference, the position and attitude of various types of response sensors are corrected during the acquisition process using data from the inertial measurement device, and the corrected magnetic flux response signal sequence, surface micro-vibration interference signal sequence and low-frequency electric field disturbance signal sequence are arranged and combined in chronological order to form a multimodal time series data matrix with consistent column vectors. The specific content of correcting the position and attitude of various types of response sensors during the acquisition process is as follows: the displacement encoding information acquired synchronously by combining the triaxial acceleration and triaxial angular velocity in the inertial measurement device is fused and calculated to correct the time index and spatial displacement of the original multimodal signal sequence.

4. The method for detecting defects in pipe circumferential welds based on deep learning according to claim 3, characterized in that, The specific content of obtaining multiple time segment matrices is as follows: the multimodal time series data matrix is ​​subjected to drift removal and bandpass noise reduction processing, and the multimodal time series data matrix is ​​divided into multiple continuous and non-overlapping time segment matrices using the length of multiple periods as a fixed window. The specific content of the upper and lower bounds of the reference band for each time segment matrix is ​​as follows: the set of sample points in each time segment matrix that do not contain significant response peaks is taken as background samples, the absolute deviation of the median of the sample point set is calculated and multiplied by a preset scaling factor to obtain the dynamic reference band, the center of the dynamic reference band is set as the mean of the background samples, and the upper and lower bounds of the reference band for each time segment matrix are determined by adding or subtracting the band width from the center of the dynamic reference band.

5. The method for detecting defects in pipe circumferential welds based on deep learning according to claim 4, characterized in that, The specific content of obtaining the phase transition factor triplet is as follows: perform piecewise linear fitting on each time segment sequence, extract the abrupt change points and reversal points of the piecewise linear slope, and perform time alignment on the abrupt change points of the multimodal time series data matrix within the same time segment sequence, calculate the transition synchronization degree, the main frequency consistency degree and the energy distribution offset, and form the phase transition factor triplet of the time segment sequence. The specific content of obtaining the event candidate segment and recording the start and end index of the event candidate segment is as follows: For each time segment matrix, determine whether any two items in the corresponding phase transition factor triplet simultaneously exceed the upper bound of the reference band of the corresponding time segment matrix. If the condition is met, mark the time segment matrix as an event candidate segment, and record the start sampling index and end sampling index of the time segment matrix in the original time series as the start and end index of the event candidate segment.

6. The method for detecting defects in pipe circumferential welds based on deep learning according to claim 5, characterized in that, The specific content of constructing a multi-segment temporal coupling relationship graph is as follows: taking event candidate segments as nodes of the graph, using the comprehensive similarity of the phase transition factor triplet of the event candidate segments as the edge weight, connecting nodes that satisfy the edge weight being greater than a set threshold edge weight, and forming a multi-segment temporal coupling relationship graph of correlation between different time periods; The specific content of generating the candidate set of dominant response modes is as follows: In the multi-segment temporal coupling relationship graph, select a connected subgraph composed of multiple nodes and whose edge weights are all higher than the set threshold edge weights, and merge the event candidate segments corresponding to the nodes in each connected subgraph as the candidate set of dominant response modes.

7. The method for detecting defects in pipe circumferential welds based on deep learning according to claim 6, characterized in that, The specific content of the obtained three output mode feature vector sequences is as follows: the dominant response mode candidate set is input into a multi-branch temporal coding network, and a one-dimensional dilated convolutional network with residual connection is used to encode the multi-scale slope change of the magnetic response in the magnetic flux response branch of the dominant response mode candidate set; the surface micro-vibration branch is encoded by a one-dimensional convolution-gated recurrent unit hybrid structure with short-time Fourier transform amplitude relative to the input to encode transient frequency band energy clusters. The low-frequency electric field branch uses a time-series network that concatenates one-dimensional convolution and causal dilated convolution to encode the low-frequency phase drift, resulting in three output mode feature vector sequences.

8. The method for detecting defects in pipe circumferential welds based on deep learning according to claim 7, characterized in that, The specific content of the obtained temporal fusion feature sequence is as follows: using a multi-segment temporal coupling relationship graph as a prior topology, a graph attention fusion layer is constructed. Directional attention weights are applied to the multimodal time series data matrix between the same time period and adjacent time periods. The three output modal feature vector sequences are message-passed on the multi-segment temporal coupling relationship graph and output as a fusion feature sequence. The fusion feature sequence is fed into a dual-path temporal inference unit in parallel. The first path is a gated recurrent unit stack, and the second path is a temporal convolutional network. The outputs of the two paths are merged at the same time period to obtain the temporal fusion feature.

9. The method for detecting defects in pipe circumferential welds based on deep learning according to claim 8, characterized in that, The specific content of obtaining the high-confidence candidate index is as follows: For the time index corresponding to the peak value of each defect category probability sequence, perform two verifications: single-modal ablation playback and time-series segment inversion playback. If the peak value decrease after single-modal ablation is inconsistent with the attention weight at the time index, the corresponding defect category probability sequence is marked as an inconsistency point. If the peak value does not show a phase stable interval after time-series segment inversion playback, the corresponding defect category probability sequence is marked as an unstable point. When there are no inconsistencies and unstable points at the same time index, the corresponding defect category probability sequence is recorded as a high-confidence candidate index.

10. A deep learning-based pipeline circumferential weld defect detection system, implemented in the deep learning-based pipeline circumferential weld defect detection method according to any one of claims 1-9, characterized in that, The system includes: The data acquisition module is configured to acquire the original multimodal signal sequence and record the pose changes of various types of response sensors relative to the ground through the inertial measurement device, and correct the original multimodal signal sequence to obtain a multimodal time series data matrix; The data preprocessing module is configured to preprocess and segment the multimodal time series data matrix to obtain multiple time segment matrices, calculate the median absolute deviation of the background noise of the multiple time segment matrices to form a dynamic reference band, and output the upper and lower bounds of the reference band for each time segment matrix. The event candidate extraction module is configured to perform piecewise linear fitting on multiple time segment matrices to obtain phase transition factor triplets of multiple time segment matrices, compare the phase transition factor triplets of multiple time segment matrices with the corresponding upper and lower bounds of the reference band to obtain event candidate segments and record the start and end indices of the event candidate segments. The alignment and graph structure construction module is configured to construct a multi-segment temporal coupling relationship graph based on event candidate segments, and generate a candidate set of dominant response modes through the multi-segment temporal coupling relationship graph. The deep learning module is configured to input the candidate set of dominant response modes into a multi-branch temporal coding network to obtain three output modality feature vector sequences. The cross-modal graph attention fusion module is configured to input the feature vectors of the three modalities into a graph attention fusion network constructed based on prior topology, and to perform weighted fusion of the correlations of different modalities in the same time period and adjacent time periods to obtain a temporal fusion feature sequence. The comparison and judgment module is configured to perform similarity matching between the time-series fusion feature sequence and the pre-established defect prototype feature library, and output the defect category probability sequence and the corresponding time index sequence based on the matching result; The verification module is configured to perform multimodal feature ablation and time segment replay verification on the probability sequence of defect categories to obtain high-confidence candidate indices; The output module is configured to map high-confidence candidate indices to pipeline spatial locations and output the defect category and severity by combining class probabilities and feature metrics.

Images