Pipeline defect identification method and device, storage medium and equipment

Abstract

The invention discloses a pipeline defect identification method and device, a storage medium and equipment, and belongs to the technical field of pipeline defect detection. The method comprises the following steps: S1, acquiring a time sequence sound signal acquired by an acoustic sensor arranged on the inner wall and / or the outer wall of a pipeline; s2, performing spectrogram extraction on the time sequence sound signal to obtain a Mel spectrogram and a LOFAR spectrogram; and S3, performing feature extraction on the Mel spectrogram, the LOFAR spectrogram and the time sequence sound signal through a deep learning model of multi-modal signal fusion to obtain a feature vector containing spectrogram features and time sequence features, and performing classification according to the feature vector to obtain a classification result of pipeline defects. According to the method, the acoustic signals are converted into different types of spectrograms, the original time sequence signals are combined, feature extraction, fusion and classification are carried out through a deep learning model, complementary information of the spectrograms and the time sequence signals is fully utilized, the limitation of a single recognition mode is overcome, and the accuracy and robustness of pipeline defect detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pipeline defect detection, and in particular to a pipeline defect recognition method, device, storage medium and equipment. Background Art

[0002] With the rapid development of urban water supply networks, pipeline aging, corrosion and defects have gradually intensified, resulting in a high leakage rate in water supply pipelines, posing severe challenges to water resource management and urban operations. According to statistics, there is a high leakage rate in water supply networks around the world, and the leakage in some old pipeline areas is particularly serious, resulting in a large amount of water resource waste. In addition, water supply pipeline leakage is also prone to cause secondary disasters, such as road collapse, foundation damage and groundwater pollution, which seriously affect the safe operation of cities. Therefore, how to efficiently and accurately detect defects and leakage in water supply networks has become an important issue that needs to be solved urgently.

[0003] At present, the technologies for water supply pipeline defect detection mainly include acoustic signal detection and optical image detection, but these technologies have obvious limitations in practical applications. Acoustic recognition methods are mainly based on vibration and noise signals generated by pipeline leakage. Commonly used methods include listening leak detection and sensor-based acoustic signal analysis. However, these methods are easily interfered by environmental noise, resulting in unstable recognition results. In addition, existing acoustic signal processing methods mostly rely on manual feature extraction. Traditional methods such as wavelet transform and Fourier transform have limited noise suppression capabilities and are difficult to adapt to high-precision detection requirements under complex working conditions. On the other hand, although optical image detection methods can intuitively identify pipeline surface defects, under low light conditions or complex internal structures of the pipeline, the image quality is limited, and it is difficult to capture deep-level defect information. In addition, optical detection has high requirements on equipment cost and construction conditions, which limits large-scale promotion and application. Summary of the invention

[0004] In order to solve the defects of the prior art, the present invention provides a pipeline defect identification method, device, storage medium and equipment, which fully utilize the complementary information of the spectrogram and the time series signal to overcome the limitations of a single identification mode and improve the accuracy and robustness of pipeline defect detection.

[0005] The present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a pipeline defect identification method, the method comprising:

[0007] S1: Acquiring a time-series sound signal collected by an acoustic sensor arranged on the inner wall and / or outer wall of a pipeline;

[0008] S2: extracting a spectrogram of the time-series sound signal to obtain a Mel spectrogram and a LOFAR spectrogram;

[0009] S3: The Mel spectrogram, LOFAR spectrogram and time series sound signal are subjected to feature extraction through the constructed multimodal signal fusion deep learning model to obtain a feature vector including spectrogram features and time series features, and classification is performed according to the feature vector to obtain a classification result of pipeline defects.

[0010] Further, the S3 includes:

[0011] S31: extracting features of the Mel spectrum through two layers of convolution to obtain Mel spectrum features;

[0012] S32: Enhance the features of the LOFAR spectrogram by using a frequency domain attention mechanism, and then extract features by convolution to obtain LOFAR spectrogram features;

[0013] S33: splicing the Mel spectrum feature and the LOFAR spectrum feature in the channel dimension to obtain a spliced ​​feature;

[0014] S34: extracting the splicing features through a residual network to obtain fused spectrogram features;

[0015] S35: extracting time series features from the time series sound signal through a long short-term memory network to obtain time series features;

[0016] S36: splicing the fused spectrogram features and the time series features in the channel dimension to obtain the feature vector;

[0017] S37: Input the feature vector into an artificial neural network model for processing to obtain a classification result of pipeline defects.

[0018] Furthermore, the acoustic sensor adopts an acoustic sensor with wide-frequency response characteristics, the sampling rate is set to 8 kHz or above, the layout interval of the acoustic sensor is 1 meter to 5 meters, and the original acoustic signal captured by the acoustic sensor is recorded by a real-time acquisition device to generate a continuous time-series sound signal.

[0019] Further, the S2 includes:

[0020] S21: dividing the time series sound signal into signal frames of fixed length, with a set overlap ratio between two adjacent signal frames, and performing windowing processing on each signal frame using a Hamming window;

[0021] S22: Performing fast Fourier transform on the windowed signal frame to convert the time domain signal into a frequency domain signal;

[0022] S23: using the Mel filter bank to process the frequency domain signal, calculating the output energy of each filter, and obtaining a Mel spectrogram;

[0023] S24: Perform statistical analysis on the power spectral density of the low-frequency signal of the frequency domain signal, extract the low-frequency features, and obtain a LOFAR spectrogram.

[0024] Furthermore, the deep learning model of multimodal signal fusion is trained by the following steps:

[0025] The data set composed of the collected time-series sound signals is divided into training set and validation set in a ratio of 8:2;

[0026] Using the training set to train the deep learning model of the multimodal signal fusion;

[0027] The Adam optimizer was used for training, the initial learning rate was 0.001, the momentum parameters were set to β1=0.9, β2=0.999, and the cross entropy loss function was used as the objective function. The parameters of the deep learning model for multimodal signal fusion were optimized through multiple rounds of iterations.

[0028] The validation set is used to evaluate the performance of the trained deep learning model for multimodal signal fusion.

[0029] In a second aspect, the present invention provides a pipeline defect identification device, the device comprising:

[0030] A sound signal acquisition module, used to acquire time-series sound signals collected by acoustic sensors arranged on the inner wall and / or outer wall of the pipeline;

[0031] A spectrogram extraction module, used for performing spectrogram extraction on the time-series sound signal to obtain a Mel spectrogram and a LOFAR spectrogram;

[0032] The pipeline defect classification module is used to extract features from the Mel spectrogram, LOFAR spectrogram and time-series sound signal through a constructed multimodal signal fusion deep learning model to obtain a feature vector containing spectrogram features and time-series features, and to perform classification according to the feature vector to obtain a classification result of the pipeline defect.

[0033] Furthermore, the pipeline defect classification module includes:

[0034] A Mel spectrum feature extraction unit, used for extracting features from the Mel spectrum through two layers of convolution to obtain Mel spectrum features;

[0035] A LOFAR spectrogram feature extraction unit, used to perform feature enhancement on the LOFAR spectrogram through a frequency domain attention mechanism, and then perform feature extraction through convolution to obtain LOFAR spectrogram features;

[0036] A spectral feature fusion unit, used for splicing the Mel spectral feature and the LOFAR spectral feature in the channel dimension to obtain a spliced ​​feature;

[0037] A residual network processing unit, used for extracting the splicing features through a residual network to obtain fused spectrogram features;

[0038] A time series feature extraction unit, used to extract time series features from the time series sound signal through a long short-term memory network to obtain time series features;

[0039] A fusion unit is used to splice the fused spectrogram features and time series features in the channel dimension to obtain the feature vector;

[0040] The pipeline defect classification unit is used to input the feature vector into the artificial neural network model for processing to obtain the classification result of the pipeline defect.

[0041] Furthermore, the acoustic sensor adopts an acoustic sensor with wide-frequency response characteristics, the sampling rate is set to 8 kHz or above, the layout interval of the acoustic sensor is 1 meter to 5 meters, and the original acoustic signal captured by the acoustic sensor is recorded by a real-time acquisition device to generate a continuous time-series sound signal.

[0042] Furthermore, the spectrogram extraction module includes:

[0043] A framing and windowing unit, used for dividing the time series sound signal into signal frames of fixed length, with a set overlap ratio between two adjacent signal frames, and performing windowing processing on each signal frame using a Hamming window;

[0044] A Fourier transform unit, used for performing a fast Fourier transform on the windowed signal frame to convert the time domain signal into a frequency domain signal;

[0045] A Mel spectrogram extraction unit is used to process the frequency domain signal using a Mel filter bank, calculate the output energy of each filter, and obtain a Mel spectrogram;

[0046] The LOFAR spectrogram extraction unit is used to perform statistical analysis on the power spectrum density of the low-frequency signal of the frequency domain signal, extract the low-frequency features, and obtain the LOFAR spectrogram.

[0047] Furthermore, the deep learning model of multimodal signal fusion is trained through the following process:

[0048] The data set composed of the collected time-series sound signals is divided into training set and validation set in a ratio of 8:2;

[0049] Using the training set to train the deep learning model of the multimodal signal fusion;

[0050] The Adam optimizer was used for training, the initial learning rate was 0.001, the momentum parameters were set to β1=0.9, β2=0.999, and the cross entropy loss function was used as the objective function. The parameters of the deep learning model for multimodal signal fusion were optimized through multiple rounds of iterations.

[0051] The validation set is used to evaluate the performance of the trained deep learning model for multimodal signal fusion.

[0052] In a third aspect, the present invention provides a computer-readable storage medium for pipeline defect identification, comprising a memory for storing processor-executable instructions, wherein the instructions, when executed by the processor, implement the steps of the pipeline defect identification method described in the first aspect.

[0053] In a fourth aspect, the present invention provides a device for pipeline defect identification, characterized in that it includes at least one processor and a memory storing computer executable instructions, and when the processor executes the instructions, the steps of the pipeline defect identification method described in the first aspect are implemented.

[0054] The present invention has the following beneficial effects:

[0055] The present invention extracts the acoustic signal as a feature spectrogram, extracts features through convolution and attention mechanism and performs splicing; the original time domain signal is input into the timing network to extract dynamic features; and finally, the efficient recognition and positioning of pipeline defects are achieved through feature fusion. The present invention converts the acoustic signal into different types of spectrograms, and combines the original time series signal to perform feature extraction, fusion and classification through a deep learning model, solving the problems of low recognition accuracy and poor robustness in the prior art. The present invention is an intelligent recognition method that integrates multiple information and combines deep learning technology, making full use of the complementary information of the spectrogram and the time series signal to overcome the limitations of a single recognition modality, improves the accuracy and robustness of pipeline defect detection, and adapts to the needs of pipeline defect detection under complex working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of the pipeline defect identification method of the present invention;

[0057] Figure 2 A schematic diagram of a deep learning model for multimodal signal fusion used in the present invention;

[0058] Figure 3 It is a schematic diagram of the pipeline defect identification device of the present invention. DETAILED DESCRIPTION

[0059] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0060] The embodiment of the present invention provides a pipeline defect recognition method for recognizing defects such as cracks and leakages in pipelines. Figure 1 As shown, the method includes:

[0061] S1: Acquire a time-series sound signal collected by an acoustic sensor arranged on the inner wall and / or outer wall of a pipeline.

[0062] In pipeline defect detection, signal acquisition is the starting point of the entire method. The present invention uses acoustic sensors arranged on the inner wall and / or outer wall of the pipeline to collect abnormal vibration and noise caused by pipeline defects (such as cracks and leakage). The details are as follows:

[0063] Sensor type and sampling rate: Use an acoustic sensor with wide-frequency response characteristics, and set the sampling rate to above 8kHz to ensure that high-frequency and low-frequency acoustic features caused by defects such as pipeline leakage and cracks can be effectively captured.

[0064] Sensor arrangement: The sensor spacing can be adjusted according to the pipe diameter and material, usually 1 to 5 meters apart to form evenly distributed acoustic sampling points.

[0065] Signal recording: The original acoustic signal captured by the sensor is recorded through real-time acquisition equipment to generate continuous time domain waveform data, that is, time-series sound signals. The data storage format is a standard digital signal format (such as WAV or RAW).

[0066] S2: Extract the spectrogram of the time series sound signal to obtain the Mel spectrogram and LOFAR spectrogram.

[0067] This step is used to extract effective spectrogram features from the collected original time-series sound signal. As an example, the implementation method of this step includes the following steps:

[0068] S21: Divide the time series sound signal into signal frames of fixed length (for example, each frame length L=20ms), and there is a set ratio (for example, 50%) overlap between two adjacent signal frames. Use a Hamming window to perform windowing processing on each signal frame to reduce spectrum leakage and improve the accuracy of frequency domain analysis.

[0069] The window function formula of the Hamming window is:

[0070]

[0071] Where N is the number of sampling points in each signal frame, and n represents the nth sampling point.

[0072] S22: Perform a fast Fourier transform (FFT) on the windowed signal frame to convert the time domain signal into a frequency domain signal to obtain a power spectrum of the signal.

[0073] The calculation formula of FFT is:

[0074]

[0075] Where X(f) is the complex value of the frequency domain signal, n is the sampling point, and f is the frequency.

[0076] S23: Use the Mel filter bank to process the frequency domain signal, map the frequency to the Mel scale based on the human hearing characteristics, calculate the output energy of each filter, and obtain the Mel spectrogram.

[0077] The calculation formula is as follows:

[0078]

[0079] Where Mel(i) is the output energy of the i-th Mel filter; X(k) is the amplitude of the signal in the frequency domain (the power spectrum obtained by fast Fourier transform); H Mel (k, i) is the response function of the i-th Mel filter; f Mel（i-1） and f Mel（i） Corresponding to the frequency range of the i-1th and i-th Mel filters.

[0080] S24: Perform statistical analysis on the power spectral density of the low-frequency signal of the frequency domain signal, extract the low-frequency features, obtain the LOFAR spectrogram, and focus on capturing the characteristics of the low-frequency signal.

[0081] The formula is:

[0082]

[0083] Where L(f) is the LOFAR spectrum value at frequency f; X(f, n) represents the power of the nth frame at frequency f; N represents the total number of frames; fmin and f cutoff They refer to the start frequency and cutoff frequency of the low frequency range respectively.

[0084] S3: The Mel spectrogram, LOFAR spectrogram and time series sound signal are extracted by constructing a deep learning model of multimodal signal fusion to obtain a feature vector containing spectrogram features and time series features. Classification is performed based on the feature vector to obtain the classification result of pipeline defects.

[0085] As an improvement of an embodiment of the present invention, the deep learning model of multimodal signal fusion is a medium-fusion deep learning model, including feature fusion of spectrogram features, medium-feature fusion of convolutional neural network processing (ResNet) and LSTM processing of time domain signals.

[0086] Specifically, the deep learning model of multimodal signal fusion is as follows: Figure 2 As shown, accordingly, the implementation of step S3 includes:

[0087] S31: Mel spectrogram F is processed through two layers of convolution Conv2d Mel Perform feature extraction to obtain the Mel spectrum feature F Mel，1 .

[0088]

[0089] S32: LOFAR spectrogram F through frequency domain attention mechanism Freq-Attention lofar Perform feature enhancement, and then extract features through convolution Conv2d to obtain the LOFAR spectrum feature F lofar,1 .

[0090]

[0091] S33: Mel spectrum feature F Mel，1 and LOFAR spectral feature F lofar,1 Perform concatenation on the channel dimension to obtain the concatenated feature F fusion .

[0092]

[0093] S34: Extract the splicing features through the residual network ResNet50 to obtain the fused spectrogram feature F ResNet .

[0094] The core idea of ​​ResNet is to introduce the residual block to solve the gradient vanishing problem in deep networks through skip connections. The process is as follows:

[0095] Standard convolution calculation: After performing a convolution operation on the input x, we get the feature map F(x):

[0096]

[0097] Where W and b are the weight and bias of the convolution kernel; Conv2D() represents a two-dimensional convolution operation; ReLU is the activation function, defined as ReLU(z)=max(0,z).

[0098] Residual module: The core formula of ResNet is to add the input x to the convolution feature F(x) to form a residual connection:

[0099]

[0100] Where F(x) is the feature map obtained by two layers of convolution and activation function; x is the input feature of the skip connection; y is the output feature map.

[0101] Global Average Pooling: The output layer of ResNet usually uses Global Average Pooling (GAP) instead of the fully connected layer. The formula is:

[0102]

[0103] Where H and W are the height and width of the feature map respectively.

[0104] S35: Extract the time series features of the time series sound signal through the long short-term memory network LSTM to obtain the time series feature F LSTM .

[0105] The deep learning model of multimodal signal fusion of the present invention adopts ResNet50 and LSTM as backbone, and ResNet50 is as mentioned above.

[0106] The core of LSTM is to introduce a gating mechanism to control the storage, update and output of information. The following are the main formulas:

[0107] Forget gate: The forget gate controls which past memories need to be forgotten:

[0108]

[0109] where x t is the input of the current time step, h t-1 is the hidden layer input of the previous time step, W f 、b f is the weight and bias of the forget gate, and σ is the Sigmoid activation function.

[0110] Input gate: The input gate determines what new information needs to be saved at the current time step:

[0111]

[0112] Combining candidate memories :

[0113]

[0114] Memory Update:

[0115]

[0116] Output gate: The output gate determines the output of the current time step:

[0117]

[0118] Hidden layer state update:

[0119]

[0120] Final output:

[0121] h t : Hidden layer output at the current time step;

[0122] C t : The memory cell state at the current time step.

[0123] S36: Fusing the spectral features F ResNet and the time series feature F LSTM Concatenate on the channel dimension to get the feature vector F fusion，1 .

[0124]

[0125] The fusion feature in the present invention refers to the spectral feature F extracted by ResNet ResNet And the time series features F extracted by LSTM LSTM Integration in progress, as described above.

[0126] S37: The feature vector F fusion，1 The data is input into the artificial neural network MLP model for processing to obtain the classification results of pipeline defects.

[0127] In order to solve the problems of low recognition accuracy, poor environmental adaptability and insufficient automation in pipeline defect detection in the prior art, the present invention extracts the acoustic signal into a feature spectrogram, extracts features through convolution and attention mechanism and performs splicing; the original time domain signal is input into the timing network to extract dynamic features; and finally, the efficient recognition and positioning of pipeline defects are achieved through feature fusion. The present invention converts the acoustic signal into different types of spectrograms, and combines the original time series signal to extract, fuse and classify features through a deep learning model, solving the problems of low recognition accuracy and poor robustness in the prior art. The present invention is an intelligent recognition method that integrates multiple information and combines deep learning technology, making full use of the complementary information of the spectrogram and the time series signal to overcome the limitations of a single recognition modality, improve the accuracy and robustness of pipeline defect detection, and adapt to the needs of pipeline defect detection under complex working conditions.

[0128] The deep learning model of multimodal signal fusion of the present invention is trained by the following steps:

[0129] 1. Divide the data set composed of the collected time-series sound signals into training sets and validation sets in a ratio of 8:2. The training data is used for model parameter optimization, and the validation data is used for performance evaluation.

[0130] 2. Use the training set to train the deep learning model for multimodal signal fusion.

[0131] The Adam optimizer is used in training, with an initial learning rate of 0.001, momentum parameters set to β1=0.9, β2=0.999, and the cross entropy loss function as the objective function. The parameters of the deep learning model for multimodal signal fusion are optimized through multiple rounds of iterations to ensure model convergence and achieve optimal performance.

[0132] The formula for the cross entropy loss function is:

[0133]

[0134] 3. Use the validation set to comprehensively evaluate the performance of the trained multimodal signal fusion deep learning model.

[0135] The main evaluation indicators include accuracy, F1 score and AUC curve. The recognition accuracy of accuracy can reach 0.951, and the F1 score and AUC curve can reach 0.946 and 0.948 respectively.

[0136] Ultimately, the trained model can efficiently and accurately identify pipeline defects and output specific defect types and their location information, providing a reliable basis for pipeline maintenance and management.

[0137] The trained multimodal signal fusion deep learning model is deployed in the actual pipeline inspection system to achieve real-time defect detection and positioning. After deployment, the model can analyze the input acoustic signal, accurately identify the defect type of the pipeline (such as cracks, leakage, etc.), and provide corresponding positioning information to help quickly identify the problem area. It is suitable for real-time monitoring needs under complex working conditions such as water supply networks, oil and gas pipelines, and provides reliable technical support for daily maintenance and safety management of pipelines.

[0138] The present invention also provides a pipeline defect identification device. Figure 3 As shown, the device comprises:

[0139] The sound signal acquisition module 1 is used to acquire time-series sound signals collected by acoustic sensors arranged on the inner wall and / or outer wall of the pipeline.

[0140] The spectrogram extraction module 2 is used to extract the spectrogram of the time series sound signal to obtain the Mel spectrogram and the LOFAR spectrogram.

[0141] The pipeline defect classification module 3 is used to extract features of the Mel spectrogram, LOFAR spectrogram and time series sound signal through the constructed multimodal signal fusion deep learning model, obtain a feature vector containing spectrogram features and time series features, and classify according to the feature vector to obtain the classification result of the pipeline defect.

[0142] As an improvement of the embodiment of the present invention, the aforementioned pipeline defect classification module includes:

[0143] The Mel spectrum feature extraction unit is used to extract features from the Mel spectrum through two layers of convolution to obtain Mel spectrum features.

[0144] The LOFAR spectrogram feature extraction unit is used to enhance the features of the LOFAR spectrogram through the frequency domain attention mechanism, and then extract features through convolution to obtain the LOFAR spectrogram features.

[0145] The spectral feature fusion unit is used to splice the Mel spectral features and the LOFAR spectral features in the channel dimension to obtain the spliced ​​features.

[0146] The residual network processing unit is used to extract the splicing features through the residual network to obtain the fused spectrum features.

[0147] The time series feature extraction unit is used to extract the time series features of the time series sound signal through the long short-term memory network to obtain the time series features.

[0148] The fusion unit is used to concatenate the fused spectrogram features and time series features in the channel dimension to obtain a feature vector.

[0149] The pipeline defect classification unit is used to input the feature vector into the artificial neural network model for processing to obtain the classification result of the pipeline defect.

[0150] Among them, the acoustic sensor uses an acoustic sensor with wide-frequency response characteristics, the sampling rate is set to 8kHz and above, the layout interval of the acoustic sensor is 1 meter to 5 meters, and the original acoustic signal captured by the acoustic sensor is recorded by the real-time acquisition equipment to generate a continuous time-series sound signal.

[0151] As an example, the aforementioned spectrogram extraction module includes:

[0152] The framing and windowing unit is used to divide the time series sound signal into signal frames of fixed length, with a set overlap ratio between two adjacent signal frames, and use a Hamming window to perform windowing processing on each signal frame.

[0153] The Fourier transform unit is used to perform fast Fourier transform on the windowed signal frame to convert the time domain signal into a frequency domain signal.

[0154] The Mel spectrogram extraction unit is used to process the frequency domain signal using the Mel filter bank, calculate the output energy of each filter, and obtain the Mel spectrogram.

[0155] The LOFAR spectrogram extraction unit is used to perform statistical analysis on the power spectrum density of the low-frequency signal of the frequency domain signal, extract the low-frequency features, and obtain the LOFAR spectrogram.

[0156] The deep learning model of multimodal signal fusion of the present invention is trained through the following process:

[0157] The data set consisting of the collected time-series sound signals is divided into training set and validation set in a ratio of 8:2.

[0158] Use the training set to train the deep learning model for multimodal signal fusion.

[0159] Among them, the Adam optimizer is used in training, the initial learning rate is 0.001, the momentum parameters are set to β1=0.9, β2=0.999, and the cross entropy loss function is used as the objective function. The parameters of the deep learning model of multimodal signal fusion are optimized through multiple rounds of iterations.

[0160] The performance of the trained deep learning model for multimodal signal fusion is evaluated using the validation set.

[0161] The present invention extracts the acoustic signal as a feature spectrogram, extracts features through convolution and attention mechanism and performs splicing; the original time domain signal is input into the timing network to extract dynamic features; and finally, the efficient recognition and positioning of pipeline defects are achieved through feature fusion. The present invention converts the acoustic signal into different types of spectrograms, and combines the original time series signal to perform feature extraction, fusion and classification through a deep learning model, solving the problems of low recognition accuracy and poor robustness in the prior art. The present invention is an intelligent recognition method that integrates multiple information and combines deep learning technology, making full use of the complementary information of the spectrogram and the time series signal to overcome the limitations of a single recognition modality, improves the accuracy and robustness of pipeline defect detection, and adapts to the needs of pipeline defect detection under complex working conditions.

[0162] The device provided in the above embodiment, its implementation principle and the technical effect produced correspond to the embodiment of the above method one by one. For the sake of brief description, the part not mentioned in the embodiment of the device can refer to the corresponding content in the embodiment of the above method. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the modules and units described in the device can refer to the corresponding process in the embodiment of the above method, and will not be repeated here.

[0163] The pipeline defect identification method described in the above embodiment provided by the present invention can implement business logic through a computer program and record it on a storage medium, and the storage medium can be read and executed by a computer to achieve the effect of the solution described in the method embodiment of this specification. Therefore, the embodiment of the present invention also provides a computer-readable storage medium for pipeline defect identification, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the pipeline defect identification method including the above embodiment are implemented.

[0164] The storage medium may include a physical device for storing information, which is usually a medium that digitizes the information and then stores it in an electrical, magnetic or optical manner. The storage medium may include: a device that stores information in an electrical energy manner, such as various memories, such as RAM, ROM, etc.; a device that stores information in a magnetic energy manner, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information in an optical manner, such as a CD or a DVD. Of course, there are other readable storage media, such as quantum memory, graphene memory, etc.

[0165] The storage medium described above may also include other implementation methods according to the description of the method embodiment. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment. For details, please refer to the description of the relevant method embodiment, and no further description will be given here.

[0166] The embodiment of the present invention also provides a device for pipeline defect identification, which may be a separate computer, or may include an actual operating device using one or more of the methods or one or more of the embodiments of this specification. The pipeline defect identification device may include at least one processor and a memory storing computer executable instructions, and when the processor executes the instructions, any one or more of the steps of the pipeline defect identification method described above is implemented.

[0167] The device described above may also include other implementation methods according to the description of the method embodiment. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment. For details, please refer to the description of the relevant method embodiment, and no further description will be given here.

[0168] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A pipeline defect identification method, characterized in that: The method comprises: S1: Acquiring a time-series sound signal collected by an acoustic sensor arranged on the inner wall and / or outer wall of a pipeline; S2: extracting a spectrogram from the time-series sound signal to obtain a Mel spectrogram and a LOFAR spectrogram; S3: The Mel spectrogram, LOFAR spectrogram and time series sound signal are subjected to feature extraction through the constructed multimodal signal fusion deep learning model to obtain a feature vector including spectrogram features and time series features, and classification is performed according to the feature vector to obtain a classification result of pipeline defects.

2. The pipeline defect identification method according to claim 1, characterized in that: The S3 includes: S31: extracting features of the Mel spectrum through two layers of convolution to obtain Mel spectrum features; S32: Enhance the features of the LOFAR spectrogram by using a frequency domain attention mechanism, and then extract features by convolution to obtain LOFAR spectrogram features; S33: splicing the Mel spectrum feature and the LOFAR spectrum feature in the channel dimension to obtain a spliced ​​feature; S34: extracting the splicing features through a residual network to obtain fused spectrogram features; S35: extracting time series features from the time series sound signal through a long short-term memory network to obtain time series features; S36: splicing the fused spectrogram features and the time series features in the channel dimension to obtain the feature vector; S37: Input the feature vector into an artificial neural network model for processing to obtain a classification result of pipeline defects.

3. The pipeline defect identification method according to claim 2, characterized in that: The S2 includes: S21: dividing the time series sound signal into signal frames of fixed length, with a set overlap ratio between two adjacent signal frames, and performing windowing processing on each signal frame using a Hamming window; S22: Performing fast Fourier transform on the windowed signal frame to convert the time domain signal into a frequency domain signal; S23: using the Mel filter bank to process the frequency domain signal, calculating the output energy of each filter, and obtaining a Mel spectrogram; S24: Perform statistical analysis on the power spectral density of the low-frequency signal of the frequency domain signal, extract the low-frequency features, and obtain a LOFAR spectrogram.

4. The pipeline defect identification method according to claim 3, characterized in that: The acoustic sensor adopts an acoustic sensor with wide-frequency response characteristics, the sampling rate is set to 8kHz or above, the layout interval of the acoustic sensor is 1 meter to 5 meters, and the original acoustic signal captured by the acoustic sensor is recorded by a real-time acquisition device to generate a continuous time-series sound signal.

5. The pipeline defect identification method according to claim 4, characterized in that: The deep learning model of multimodal signal fusion is trained by the following steps: The data set composed of the collected time-series sound signals is divided into training set and validation set in a ratio of 8:2; Using the training set to train the deep learning model of the multimodal signal fusion; The Adam optimizer was used for training, the initial learning rate was 0.001, the momentum parameters were set to β1=0.9, β2=0.999, and the cross entropy loss function was used as the objective function. The parameters of the deep learning model for multimodal signal fusion were optimized through multiple rounds of iterations. The validation set is used to evaluate the performance of the trained deep learning model for multimodal signal fusion.

6. A pipeline defect identification device, characterized in that: The device comprises: A sound signal acquisition module, used to acquire time-series sound signals collected by acoustic sensors arranged on the inner wall and / or outer wall of the pipeline; A spectrogram extraction module, used for performing spectrogram extraction on the time-series sound signal to obtain a Mel spectrogram and a LOFAR spectrogram; The pipeline defect classification module is used to extract features from the Mel spectrogram, LOFAR spectrogram and time-series sound signal through a constructed multimodal signal fusion deep learning model to obtain a feature vector containing spectrogram features and time-series features, and to perform classification according to the feature vector to obtain a classification result of the pipeline defect.

7. The pipeline defect identification device according to claim 6, characterized in that: The pipeline defect classification module includes: A Mel spectrum feature extraction unit, used for extracting features from the Mel spectrum through two layers of convolution to obtain Mel spectrum features; A LOFAR spectrogram feature extraction unit, used to perform feature enhancement on the LOFAR spectrogram through a frequency domain attention mechanism, and then perform feature extraction through convolution to obtain LOFAR spectrogram features; A spectral feature fusion unit, used for splicing the Mel spectral feature and the LOFAR spectral feature in the channel dimension to obtain a spliced ​​feature; A residual network processing unit, used for extracting the splicing features through a residual network to obtain fused spectrogram features; A time series feature extraction unit, used to extract time series features from the time series sound signal through a long short-term memory network to obtain time series features; A fusion unit is used to splice the fused spectrogram features and time series features in the channel dimension to obtain the feature vector; The pipeline defect classification unit is used to input the feature vector into the artificial neural network model for processing to obtain the classification result of the pipeline defect.

8. The pipeline defect identification device according to claim 7, characterized in that: The spectrogram extraction module comprises: A framing and windowing unit, used for dividing the time series sound signal into signal frames of fixed length, with a set overlap ratio between two adjacent signal frames, and performing windowing processing on each signal frame using a Hamming window; A Fourier transform unit, used for performing a fast Fourier transform on the windowed signal frame to convert the time domain signal into a frequency domain signal; A Mel spectrogram extraction unit is used to process the frequency domain signal using a Mel filter bank, calculate the output energy of each filter, and obtain a Mel spectrogram; The LOFAR spectrogram extraction unit is used to perform statistical analysis on the power spectrum density of the low-frequency signal of the frequency domain signal, extract the low-frequency features, and obtain the LOFAR spectrogram.

9. A computer-readable storage medium for pipeline defect identification, characterized in that: It comprises a memory for storing processor executable instructions, and when the instructions are executed by the processor, the steps of the pipeline defect identification method according to any one of claims 1 to 5 are implemented.

10. A device for pipeline defect identification, characterized in that: The method comprises at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the pipeline defect identification method according to any one of claims 1 to 5 are implemented.

Images