Pipe joint welding seam defect detection method and system based on multi-modal fusion

Abstract

The application provides a small connecting pipe welding seam defect detection method and system based on multi-modal fusion, relates to the welding seam defect detection technical field, and obviously has higher application flexibility and higher practicability compared to the single and rigid multi-modal fusion application mode of the existing small connecting pipe welding seam detection mode. The existing modal data is extracted, corresponding application modes, that is, application scenes, are divided according to the corresponding functions, the required modal data is spliced and fused according to the different application scenes, the detection mode required in a specific scene is formed, the purpose of making effective targeted measures for the specific scene is achieved, and the multi-modal fusion application mode is more flexible and ingenious, and the use efficiency can be embodied.

Description

Technical Field

[0001] This invention relates to the field of weld defect detection technology, and in particular to a method and system for detecting weld defects in small connecting pipes based on multimodal fusion. Background Technology

[0002] The weld seam of the small connecting pipe is an important component of the pipeline system. During operation, the small connecting pipe is in the same environment as the main pipeline. However, due to its special structure and usage conditions, it will face some specific challenges and hidden dangers. Therefore, special attention needs to be paid to the maintenance and management of the weld seam of the small connecting pipe. Regular inspection, timely repair and replacement of damaged weld seams are the key to ensuring the safe and stable operation of the pipeline system. At the same time, the use of advanced detection technologies and methods, such as X-ray detection, ultrasonic detection, magnetic particle detection, penetrant detection, infrared thermal imaging detection, optical detection and laser holographic detection, can more effectively detect and deal with potential hidden dangers.

[0003] Based on the above, current defect detection methods for small-diameter pipe welds generally employ only one detection method, which can be understood as a single-modal detection approach. While this method offers numerous advantages in practical applications—such as simplified procedures, reduced skill requirements for operators, and greater focus and efficiency for specific weld defect types—its disadvantages are equally apparent. Single-modal detection cannot cover all types of weld defects, is ineffective for complex or concealed defects, and carries a high risk of false positives and false negatives due to reliance on a single method, affecting the accuracy of the results. Furthermore, single-modal detection is often susceptible to environmental factors, leading to unstable results. Therefore, the choice between using a single-modal approach for small-diameter pipe weld defect detection requires comprehensive consideration of the specific application scenario, defect type, and detection requirements. In certain situations, it is necessary to combine this method with other detection techniques to compensate for the shortcomings of single-modal detection and improve accuracy and reliability. Based on this, multimodal detection methods have emerged.

[0004] Currently, when using multimodal detection methods to detect defects in the weld seams of small-diameter pipes, the process first involves acquiring various modal data of the weld seams, including radiation imaging modal data, acoustic detection modal data, magnetic detection modal data, surface detection modal data, thermal imaging modal data, electromagnetic detection modal data, visual detection modal data, and three-dimensional detection modal data. This modal data can also be categorized into image data, infrared thermographic data, ultrasonic data, and electrical signal data. Next, the acquired modal data undergoes preprocessing, including noise removal, data smoothing, normalization, and data augmentation. Subsequently, feature extraction is performed separately for each modality. For image data, deep learning techniques are used to extract feature vectors from the images. For infrared thermographic data... Thermal imaging data is analyzed using information entropy, which extracts feature vectors by calculating the average information content after eliminating redundancy. For ultrasonic data, wavelet transform feature extraction is used. Based on the ultrasonic signal, wavelet basis functions and decomposition levels are selected to decompose the signal into multiple scales, obtaining wavelet coefficients at different scales. Then, feature vectors reflecting the time-frequency characteristics of the signal are extracted from the wavelet coefficients. Next, the feature vectors extracted from each mode are concatenated and fused. Specifically, the feature vectors of each mode are directly concatenated to form feature information after the fusion of different modal data, resulting in a more comprehensive and richer feature representation. Finally, based on the fused feature information, a deep learning model is used to identify and locate the weld.

[0005] While the existing technologies employed can comprehensively utilize data from different modes to improve the accuracy and reliability of detection, effectively compensating for the shortcomings of single-modal detection and achieving comprehensive and accurate detection of small pipe welds, their application scenarios are not adequately considered, and the data volume is significantly large. This results in poor flexibility when applying this method of fusing all modes to multiple scenarios. Specifically, in scenarios with a large number of welds to be inspected and tight project timelines requiring rapid and precise inspection, using this method can easily delay project progress by consuming a significant amount of time. Furthermore, in scenarios where the internal or external quality of the welds can already be guaranteed, using this method can easily waste project time by providing some useless work. Therefore, the existing multimodal fusion method is poorly considered, lacks flexibility, and cannot effectively provide targeted solutions for specific scenarios.

[0006] Therefore, we propose a method and system for detecting defects in weld seams of small connecting pipes based on multimodal fusion, in order to effectively solve the above problems. Summary of the Invention

[0007] To address the aforementioned technical issues, a multimodal fusion-based method for detecting weld defects in small-diameter pipes includes the following steps:

[0008] Step S1, Data Acquisition: Obtain modal data according to the detection method of the small pipe weld;

[0009] The inspection methods for the weld seams of the small connecting pipe include: X-ray inspection, ultrasonic inspection, magnetic particle inspection, penetrant inspection, infrared thermal imaging inspection, eddy current inspection, optical inspection, and laser holographic inspection;

[0010] Modal data are obtained according to the detection methods of the small pipe weld, including: radiation imaging mode, acoustic detection mode, magnetic detection mode, surface detection mode, thermal imaging mode, electromagnetic detection mode, visual detection mode, and three-dimensional detection mode;

[0011] The modal data are classified into the following categories based on data type: image data, ultrasonic data, infrared thermal imaging data, and electrical signal data.

[0012] Step S2, Function Extraction: Extract the functions of the modal data;

[0013] The function of extracting the modal data includes:

[0014] The functions of extracting the radiation imaging modes include non-destructive testing of welds, internal testing, small weld testing, and large weld testing.

[0015] The functions of extracting the acoustic detection modes include non-destructive testing of welds, internal testing, rapid testing, small weld testing, and large weld testing.

[0016] The functions of extracting the magnetic detection modes include non-destructive testing of welds, surface inspection, and large weld inspection.

[0017] The functions for extracting the surface inspection modes include non-destructive testing of welds, surface inspection, small weld inspection, and large weld inspection.

[0018] The functions of extracting the thermal imaging modes include non-destructive testing of welds, surface inspection, rapid inspection, small weld inspection, and large weld inspection.

[0019] The functions of extracting the electromagnetic detection modes include non-destructive testing of welds, surface inspection, small weld inspection, and large weld inspection.

[0020] The functions of extracting the visual inspection modal include non-destructive testing of welds and surface inspection.

[0021] The functions for extracting the three-dimensional detection modes include non-destructive testing of welds, internal testing, and surface testing.

[0022] The distinction between the large weld and the small weld is based on the following criteria:

[0023] When the diameter of the small connecting pipe is less than or equal to 50% of the diameter of the main pipe, the weld of the small connecting pipe is determined to be a small weld; conversely, when the diameter of the small connecting pipe is greater than 50% of the diameter of the main pipe, the weld of the small connecting pipe is determined to be a large weld.

[0024] Step S3, Mode Classification: Based on the function of the modal data extracted in step S2, the application modes of the small pipe weld are classified.

[0025] The application modes are divided into non-destructive rapid inspection mode, non-destructive surface inspection mode, non-destructive internal inspection mode, non-destructive high-precision inspection mode, non-destructive small weld inspection mode, and non-destructive large weld inspection mode.

[0026] Step S4, Modal data combination: The application mode in step S3 is combined with the corresponding modal data, and each application mode contains several modal data.

[0027] Combining the application mode with the modal data, specifically:

[0028] The non-destructive rapid detection mode consists of an acoustic detection mode and a thermal imaging mode;

[0029] The non-destructive surface inspection mode consists of a magnetic inspection mode, a surface inspection mode, a thermal imaging mode, an electromagnetic inspection mode, a visual inspection mode, and a three-dimensional inspection mode;

[0030] The non-destructive internal inspection mode consists of a radiation imaging mode, an acoustic inspection mode, and a three-dimensional inspection mode;

[0031] The non-destructive high-precision testing mode consists of radiation imaging mode, acoustic testing mode, magnetic testing mode, surface testing mode, thermal imaging mode, electromagnetic testing mode, visual testing mode, and three-dimensional testing mode;

[0032] The non-destructive small weld inspection mode consists of radiation imaging mode, surface inspection mode, thermal imaging mode and electromagnetic inspection mode;

[0033] The non-destructive large weld inspection mode consists of radiation imaging mode, acoustic inspection mode, magnetic inspection mode, surface inspection mode, thermal imaging mode and electromagnetic inspection mode.

[0034] Step S5, Data preprocessing: Preprocess the modal data under the application mode respectively;

[0035] The preprocessing includes: preprocessing the image data, ultrasonic data, infrared thermal image data, and electrical signal data respectively;

[0036] The image data preprocessing steps include: image loading, noise removal, contrast enhancement, edge detection, binarization, morphological operations, and image saving;

[0037] The ultrasonic data preprocessing steps include: data loading, DC offset removal, bandpass filtering, threshold processing, envelope detection, ultrasonic signal plotting, and data saving.

[0038] The infrared thermal image data preprocessing steps include: loading the infrared thermal image, removing background noise, enhancing contrast, normalizing data, and saving the infrared thermal image.

[0039] The electrical signal data preprocessing steps include: loading electrical signal data, removing DC offset, applying a low-pass filter, data standardization, and saving electrical signal data.

[0040] Step S6, fusion weight calculation: Based on the function of the modal data extracted in step S2, the fusion weight is calculated for the preprocessed modal data;

[0041] The fusion weights are calculated for the preprocessed modal data, specifically as follows:

[0042] Step S61: Statistically analyze all the functions of the modal data, namely: non-destructive testing function, surface testing function, internal testing function, rapid testing function, small weld seam testing function, and large weld seam testing function;

[0043] Step S62: Calculate the percentage of each modal data function among all functions, as follows:

[0044] The radiation imaging mode accounts for 4 / 6 of all the functions described.

[0045] The acoustic detection mode accounts for 5 / 6 of all the functions described.

[0046] The magnetic detection mode accounts for 3 / 6 of all the functions described.

[0047] The surface detection mode accounts for 4 / 6 of all the functions described.

[0048] The thermal imaging modality accounts for 5 / 6 of all the functions described.

[0049] The electromagnetic detection mode accounts for 4 / 6 of all the functions described.

[0050] The visual detection modality accounts for 2 / 6 of all the functions described.

[0051] The three-dimensional detection modality accounts for 3 / 6 of all the functions described.

[0052] Step S63: Calculate the fusion weights for the modal data to be fused based on their proportions, as follows:

[0053] The fusion weight of the acoustic detection mode and the thermal imaging mode is 5 / 6÷5 / 6, that is, a fusion weight of 1:1.

[0054] The fusion weights of the magnetic detection mode, surface detection mode, thermal imaging mode, electromagnetic detection mode, visual detection mode and three-dimensional detection mode are 3 / 6÷4 / 6÷5 / 6÷4 / 6÷2 / 6÷3 / 6, that is, 3:4:5:4:2:3.

[0055] The fusion weights of the radiation imaging mode, acoustic detection mode, and three-dimensional detection mode are 4 / 6÷5 / 6÷3 / 6, i.e., 4:5:3.

[0056] The fusion weights of the radiation imaging mode, acoustic detection mode, magnetic detection mode, surface detection mode, thermal imaging mode, electromagnetic detection mode, visual detection mode, and three-dimensional detection mode are calculated according to the ratio of 4 / 6÷5 / 6÷3 / 6÷4 / 6÷5 / 6÷4 / 6÷2 / 6÷3 / 6, which is 4:5:3:4:5:4:2:3.

[0057] The fusion weights of the radiation imaging mode, surface detection mode, thermal imaging mode and electromagnetic detection mode are 4 / 6÷4 / 6÷5 / 6÷4 / 6, that is, 4:4:5:4.

[0058] The fusion weights of the radiation imaging mode, acoustic detection mode, magnetic detection mode, surface detection mode, thermal imaging mode, and electromagnetic detection mode are calculated according to a fusion weight of 4 / 6÷5 / 6÷3 / 6÷4 / 6÷5 / 6÷4 / 6, which is 4:5:3:4:5:4.

[0059] Step S7, Feature Extraction and Fusion: Feature vectors are extracted from the modal data after the preprocessing in Step S5, and the extracted feature vectors are spliced ​​and fused according to the combination processing result of the application mode and the modal data in Step S4 and the fusion weight provided in Step S6 to form fused feature information under six application modes.

[0060] Step S8, Defect Identification and Localization: The weld seam is identified and located using six deep learning models based on the fused feature information from the six application modes.

[0061] The small-diameter pipe weld defect detection system based on multimodal fusion, using the small-diameter pipe weld defect detection method based on multimodal fusion as described in any of the preceding claims, includes the following components:

[0062] Data acquisition module: used to acquire modal data based on the detection method of the small pipe weld;

[0063] Function extraction module: connected to the data acquisition module, used to extract the function of the modal data;

[0064] Pattern segmentation module: connected to the function extraction module, used to segment the application modes of the small pipe weld according to the functions of the modal data extracted by the function extraction module;

[0065] Modal data combination module: connected to the mode division module, used to combine the corresponding modal data according to the application mode in the mode division module;

[0066] Data preprocessing module: connected to the modal data combining module, used to preprocess the modal data in the modal data combining module respectively;

[0067] Fusion weight calculation module: connected to the data preprocessing module, used to calculate the fusion weight of the preprocessed modal data according to the functions of the modal data extracted by the function extraction module;

[0068] Feature extraction and fusion module: connected to the fusion weight calculation module, used to extract feature vectors from the modal data after preprocessing by the data preprocessing module, and to splice and fuse the extracted feature vectors according to the combination processing result of the application mode and the modal data in the modal data combination module and the fusion weight provided by the fusion weight calculation module, to form fused feature information under six application modes;

[0069] Defect identification and localization module: connected to the feature extraction and fusion module, used to identify and locate weld seams by using six deep learning models to extract and fuse the feature information from the six application modes.

[0070] The embodiments of the present invention have the following technical effects:

[0071] Compared to the existing single and rigid multimodal fusion application method for small pipe weld inspection, this invention is obviously more flexible and practical. By extracting the functions of existing modal data and dividing them into corresponding application modes, i.e. application scenarios, according to the corresponding functions, the required modal data is spliced ​​and fused according to the different application scenarios to form the inspection method required in a specific scenario. This achieves the goal of making effective and targeted measures for specific scenarios, making the application method of multimodal fusion more flexible, ingenious, and efficient.

[0072] This invention calculates the fusion weight of all modal data. This allows subsequent modal data to be fused based on this fusion weight, resulting in a higher or lower proportion of the corresponding modal data to be fused. This makes the specific functions of the fused modal data more apparent, making it more targeted in application, without affecting the detection of different weld defect types, thus improving the efficiency of application. Attached Figure Description

[0073] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0074] Figure 1 This is a flowchart of the small pipe weld defect detection method based on multimodal fusion provided in the embodiments of the present invention;

[0075] Figure 2 This is a statistical chart showing the proportion of each modal data function among all functions in the small pipe weld defect detection method based on multimodal fusion provided in the embodiments of the present invention;

[0076] Figure 3 This is a framework diagram of the ultrasonic imaging simulation system for small pipe fillet welds based on finite element method provided in this embodiment of the invention. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0078] Example 1: As Figure 1 As shown, the present invention provides a method for detecting defects in weld seams of small connecting pipes based on multimodal fusion, comprising the following steps:

[0079] Step S1, Data Acquisition: Obtain modal data according to the detection method of the small pipe weld;

[0080] The inspection methods for the weld seams of the small connecting pipe include: X-ray inspection, ultrasonic inspection, magnetic particle inspection, penetrant inspection, infrared thermal imaging inspection, eddy current inspection, optical inspection, and laser holographic inspection;

[0081] Modal data are obtained according to the detection methods of the small pipe weld, including: radiation imaging mode, acoustic detection mode, magnetic detection mode, surface detection mode, thermal imaging mode, electromagnetic detection mode, visual detection mode, and three-dimensional detection mode;

[0082] The modal data is classified into the following categories based on data type: image data, ultrasonic data, infrared thermal imaging data, and electrical signal data.

[0083] The radiation imaging mode is obtained by X-ray detection, where X-rays penetrate the weld to form an image of the inside of the weld, which belongs to the image data.

[0084] The acoustic detection mode is obtained through ultrasonic detection. By collecting and analyzing the reflected wave signals based on the propagation and reflection of ultrasonic waves in the weld, it belongs to the ultrasonic data.

[0085] The magnetic detection mode is obtained through magnetic particle detection. The magnetic field is applied to the weld, and the magnetic traces formed by the magnetic particles at the defect site show the location and shape of the defect. This is the image data that can be directly observed.

[0086] The surface detection mode is obtained through penetrant detection. The penetrant penetrates the weld surface defects and the developer forms visible defect displays, which are also image data that can be directly observed.

[0087] The thermal imaging mode is obtained by infrared thermal imaging detection, which captures the infrared radiation on the weld surface to form a temperature distribution image, and belongs to the infrared thermal image data.

[0088] The electromagnetic detection mode is obtained through eddy current detection, which uses the generation and changes of eddy currents in the weld to detect defects and belongs to the electrical signal data.

[0089] The visual inspection modality is obtained through optical inspection, which involves observing the weld surface with optical instruments to directly acquire the appearance information of the weld, and belongs to the image data.

[0090] The three-dimensional detection mode is obtained through laser holographic detection, which combines optical and holographic imaging technologies. It uses laser interference to form a three-dimensional holographic image of the weld, providing three-dimensional morphological data of the weld, and belongs to the image data.

[0091] Step S2, Function Extraction: Extract the functions of the modal data;

[0092] The function of extracting the modal data includes:

[0093] The radiation imaging mode can visually display the internal structure and defects of the weld by observing the absorption and scattering of X-rays. At the same time, the X-ray detection process does not damage the weld. It can detect internal defects of the weld by using the different absorption of the transmitted rays by the inspected workpiece. Therefore, X-ray detection is very effective for the internal quality inspection of small welds, and it is equally applicable to the inspection of large welds.

[0094] The functions of extracting the radiation imaging modes include non-destructive testing of welds, internal testing, small weld testing, and large weld testing.

[0095] The advantages of the acoustic detection mode are its fast detection speed and high accuracy. It uses the propagation and reflection of ultrasonic waves in the weld to detect internal defects, so it will not damage the weld during the detection process. It is also very suitable for the detection of small welds. At the same time, because ultrasonic detection has strong penetration and good directionality, it is also suitable for the detection of large welds.

[0096] The functions of extracting the acoustic detection modes include non-destructive testing of welds, internal testing, rapid testing, small weld testing, and large weld testing.

[0097] The magnetic detection mode involves coating the surface of the weld with magnetic powder and observing the deposition of the magnetic powder. Therefore, it can detect defects on the surface of the weld without damaging the weld. For large welds, magnetic particle detection can be an effective detection method.

[0098] The functions of extracting the magnetic detection modes include non-destructive testing of welds, surface inspection, and large weld inspection.

[0099] The surface inspection modality is a non-destructive testing technique for detecting surface and near-surface opening defects in welds. It uses polishing and brushing methods to apply a penetrant to the surface of the weld to be inspected. As long as there are opening defects on the surface of the weld to be inspected, regardless of size, they can be visualized. Therefore, penetrant testing can be used as an effective means of inspecting welds of both large and small sizes.

[0100] The functions for extracting the surface inspection modes include non-destructive testing of welds, surface inspection, small weld inspection, and large weld inspection.

[0101] The aforementioned thermal imaging mode detects defects by capturing temperature changes on the weld surface, and is particularly suitable for online monitoring and rapid screening. However, it has limited ability to detect internal structures or defects in welds. For issues such as porosity and cracks inside welds, other non-destructive testing technologies are needed for more in-depth detection. Therefore, infrared thermal imaging detection is mainly used for the detection of weld surfaces. At the same time, infrared thermal imaging detection is also a non-destructive testing method for welds and is suitable for the detection of both large and small welds.

[0102] The functions of extracting the thermal imaging modes include non-destructive testing of welds, surface inspection, rapid inspection, small weld inspection, and large weld inspection.

[0103] The electromagnetic detection mode detects defects on and near the surface of the weld through the principle of eddy current induction. Therefore, it can effectively detect the weld surface without damaging the weld. For large welds, eddy current detection can obviously be used as an effective detection method. It can also be used to detect wall thickness corrosion and thinning of ferromagnetic metal materials such as carbon steel and low alloy steel, as well as non-ferromagnetic metal materials such as stainless steel and aluminum alloy. Therefore, it is also suitable for the detection of small welds.

[0104] The functions of extracting the electromagnetic detection modes include non-destructive testing of welds, surface inspection, small weld inspection, and large weld inspection.

[0105] The visual inspection modality uses an optical microscope to inspect the surface and near-surface of the weld. There is no contact with the weld being inspected during the process, so it will not damage the weld and is one of the non-destructive testing methods.

[0106] The functions of extracting the visual inspection modal include non-destructive testing of welds and surface inspection.

[0107] The three-dimensional detection mode uses holographic interference technology to form a three-dimensional holographic image of the weld. By comparing the changes in the two sets of light waves and interference fringes, the defects on the surface and inside of the weld can be determined. This is one of the methods of non-destructive testing.

[0108] The functions for extracting the three-dimensional detection modes include non-destructive testing of welds, internal testing, and surface testing.

[0109] The distinction between the large weld and the small weld is based on the following criteria:

[0110] When the diameter of the small connecting pipe is less than or equal to 50% of the diameter of the main pipe, the weld of the small connecting pipe is determined to be a small weld; conversely, when the diameter of the small connecting pipe is greater than 50% of the diameter of the main pipe, the weld of the small connecting pipe is determined to be a large weld.

[0111] For example, suppose there is a main pipe with a diameter of D = 1000 mm. Now, a small connecting pipe with a diameter of d = 100 mm needs to be connected to this main pipe. In this case, since the diameter of the small connecting pipe is much smaller than the diameter of the main pipe, only 10% of the diameter of the main pipe, i.e., d / D = 50 / 500 = 0.1 = 10%, the weld between the small connecting pipe and the main pipe is considered a small weld relative to the main pipe.

[0112] Step S3, Mode Classification: Based on the function of the modal data extracted in step S2, the application modes of the small pipe weld are classified.

[0113] The application modes are divided into non-destructive rapid inspection mode, non-destructive surface inspection mode, non-destructive internal inspection mode, non-destructive high-precision inspection mode, non-destructive small weld inspection mode, and non-destructive large weld inspection mode.

[0114] Step S4, Modal data combination: Step S4, Modal data combination: The application mode in step S3 is combined with the corresponding modal data, and each application mode contains several modal data.

[0115] Combining the application mode with the modal data, specifically:

[0116] The non-destructive rapid detection mode consists of an acoustic detection mode and a thermal imaging mode;

[0117] The non-destructive surface inspection mode consists of a magnetic inspection mode, a surface inspection mode, a thermal imaging mode, an electromagnetic inspection mode, a visual inspection mode, and a three-dimensional inspection mode;

[0118] The non-destructive internal inspection mode consists of a radiation imaging mode, an acoustic inspection mode, and a three-dimensional inspection mode;

[0119] The non-destructive high-precision testing mode consists of radiation imaging mode, acoustic testing mode, magnetic testing mode, surface testing mode, thermal imaging mode, electromagnetic testing mode, visual testing mode, and three-dimensional testing mode;

[0120] The non-destructive small weld inspection mode consists of radiation imaging mode, surface inspection mode, thermal imaging mode and electromagnetic inspection mode;

[0121] The non-destructive large weld inspection mode consists of radiation imaging mode, acoustic inspection mode, magnetic inspection mode, surface inspection mode, thermal imaging mode and electromagnetic inspection mode.

[0122] Step S5, Data preprocessing: Preprocess the modal data under the application mode respectively;

[0123] The preprocessing includes: preprocessing the image data, ultrasonic data, infrared thermal image data, and electrical signal data respectively;

[0124] The image data preprocessing steps include: image loading, noise removal, contrast enhancement, edge detection, binarization, morphological operations, and image saving;

[0125] For example:

[0126] Image loading: Use Python to load the images from the image data, load the images, and check the image size, data type, and pixel value range to ensure they meet the conditions for subsequent processing;

[0127] Noise Removal: Applying Gaussian filtering algorithms to remove noise from images helps to smooth the image and reduce interference;

[0128] Contrast Enhancement: The contrast in the image is adjusted using a contrast stretching method to make the contrast between the weld and the background more obvious;

[0129] Edge detection: The Canny edge detection algorithm is applied to highlight the edges of welds in the image, which helps with subsequent feature extraction and analysis;

[0130] Binarization: Converting an image into a binary image, containing only black and white, helps to further simplify the image and highlight key features;

[0131] Morphological operations: Use opening operations to further clean up noise and details in the image;

[0132] Image saving: Save the preprocessed image;

[0133] For example, processing can be done using Python:

[0134] import cv2

[0135] import numpy as np

[0136] #Image loading

[0137] image=cv2.imread('weld_image.jpg',cv2.IMREAD_GRAYSCALE)

[0138] #Noise Removal

[0139] blurred=cv2.GaussianBlur(image)

[0140] #Contrast Enhancement

[0141] equ=cv2.equalizeHist(blurred)

[0142] #Edge Detection

[0143] edges = cv2.Canny(equ, 40, 140)

[0144] #Binarization

[0145] _,binary=cv2.threshold(edges,117,245,cv2.THRESH_BINARY)

[0146] #Morphological Operations

[0147] kernel=np.ones((4,5),np.uint8)

[0148] binary=cv2.morphologyEx(binary,cv2.MORPH_OPEN,kernel)

[0149] #Image saving

[0150] cv2.imwrite('preprocessed_image.jpg',binary).

[0151] The ultrasonic data preprocessing steps include: ultrasonic data loading, DC offset removal, bandpass filtering, threshold processing, envelope detection, ultrasonic signal plotting, and ultrasonic data saving.

[0152] For example:

[0153] Ultrasonic data loading: Use Python to load the ultrasonic data, and check the format, sampling rate, temporal resolution and signal strength of the ultrasonic data to ensure data integrity;

[0154] DC offset removal: The ultrasonic signal in the ultrasonic data may contain DC offset, that is, the average value of the ultrasonic signal is not zero. DC offset is removed by calculating the average value of the ultrasonic signal and subtracting this value.

[0155] Bandpass filtering: A bandpass filter is applied to remove low-frequency and high-frequency noise from the ultrasonic signal in the ultrasonic data;

[0156] Threshold processing: Set the threshold to 10% of the maximum value of the ultrasonic signal in the ultrasonic data to further remove noise and background signals;

[0157] Envelope detection: By calculating the envelope of the ultrasonic signal in the ultrasonic data, the characteristics of the weld echo can be highlighted, which helps in subsequent weld identification and location.

[0158] Ultrasonic signal plotting: Based on the ultrasonic signal in the ultrasonic data, plot the echo arrival time, peak amplitude, and echo width of the ultrasonic signal to facilitate subsequent weld quality assessment and defect detection.

[0159] Ultrasonic data storage: The preprocessed ultrasonic data is stored.

[0160] For example, processing can be done using Python:

[0161]

[0162]

[0163] The infrared thermal image data preprocessing steps include: loading the infrared thermal image, removing background noise, enhancing contrast, normalizing data, and saving the infrared thermal image.

[0164] For example:

[0165] Infrared thermal image loading: Use Python to load the infrared thermal images from the infrared thermal image data, load the infrared thermal images, and check the size, resolution, and temperature range of the infrared thermal images to ensure data integrity;

[0166] Background noise removal: Gaussian filtering algorithm is applied to remove noise from infrared thermal images;

[0167] Contrast Enhancement: Histogram equalization is used to adjust the contrast in infrared thermal images, making temperature changes in the weld area more noticeable.

[0168] Data normalization: Map the pixel values ​​of infrared thermal images to the range of 0-255, and then normalize the temperature displayed in the infrared thermal images to a uniform temperature range for subsequent processing and analysis.

[0169] Infrared thermal image saving: The preprocessed infrared thermal image is saved;

[0170] For example, processing can be done using Python:

[0171] import cv2

[0172] import numpy as np

[0173] #Loading Infrared Thermal Image

[0174] image=cv2.imread('infrared_image.png',cv2.IMREAD_GRAYSCALE)

[0175] #Remove background noise

[0176] _,thresholded=cv2.threshold(image,100,255,

[0177] cv2.THRESH_BINARY_INV)

[0178] #Contrast Enhancement

[0179] equalized=cv2.equalizeHist(cleaned)

[0180] #Data Normalization

[0181] normalized=cv2.normalize(equalized,None,0,255,cv2.NORM_MINMAX)

[0182] #Saving Infrared Thermal Images

[0183] cv2.imwrite('preprocessed_infrared_image.png',normalized).

[0184] The electrical signal data preprocessing steps include: electrical signal data loading, DC offset removal, application of low-pass filter, data standardization, and electrical signal data saving;

[0185] For example:

[0186] Electrical signal data loading: Use Python to load the electrical signal data, and check the format, sampling rate, time resolution, and signal range of the electrical signal data to ensure data integrity;

[0187] Remove DC offset: Calculate the average value of the electrical signal data and subtract the average value from each data point to remove the DC offset;

[0188] Applying a low-pass filter: Applying a low-pass filter removes high-frequency noise and low-frequency interference from electrical signal data;

[0189] Data standardization: Standardizing or normalizing electrical signal data to a uniform range for subsequent processing and analysis. This is achieved by calculating the mean and standard deviation of the data and then converting each data point into a standard score.

[0190] Save the preprocessed electrical signal data;

[0191] For example, Python can be used for processing:

[0192]

[0193]

[0194] Step S6, fusion weight calculation: Based on the function of the modal data extracted in step S2, the fusion weight is calculated for the preprocessed modal data;

[0195] The preprocessed modal data is then subjected to a fusion weight calculation, specifically as follows:

[0196] Step S61: Statistically analyze all the functions of the modal data, namely: non-destructive testing function, surface testing function, internal testing function, rapid testing function, small weld seam testing function, and large weld seam testing function;

[0197] Step S62: Calculate the proportion of each modal data function among all functions, such as... Figure 2 As shown, they are respectively:

[0198] The radiation imaging mode accounts for 4 / 6 of all the functions described.

[0199] The acoustic detection mode accounts for 5 / 6 of all the functions described.

[0200] The magnetic detection mode accounts for 3 / 6 of all the functions described.

[0201] The surface detection mode accounts for 4 / 6 of all the functions described.

[0202] The thermal imaging modality accounts for 5 / 6 of all the functions described.

[0203] The electromagnetic detection mode accounts for 4 / 6 of all the functions described.

[0204] The visual detection modality accounts for 2 / 6 of all the functions described.

[0205] The three-dimensional detection modality accounts for 3 / 6 of all the functions described.

[0206] Step S63: Calculate the fusion weights for the modal data to be fused based on their proportions, as follows:

[0207] The fusion weight of the acoustic detection mode and the thermal imaging mode is 5 / 6÷5 / 6, that is, a fusion weight of 1:1.

[0208] The fusion weights of the magnetic detection mode, surface detection mode, thermal imaging mode, electromagnetic detection mode, visual detection mode and three-dimensional detection mode are 3 / 6÷4 / 6÷5 / 6÷4 / 6÷2 / 6÷3 / 6, that is, 3:4:5:4:2:3.

[0209] The fusion weights of the radiation imaging mode, acoustic detection mode, and three-dimensional detection mode are 4 / 6÷5 / 6÷3 / 6, i.e., 4:5:3.

[0210] The fusion weights of the radiation imaging mode, acoustic detection mode, magnetic detection mode, surface detection mode, thermal imaging mode, electromagnetic detection mode, visual detection mode, and three-dimensional detection mode are calculated according to the ratio of 4 / 6÷5 / 6÷3 / 6÷4 / 6÷5 / 6÷4 / 6÷2 / 6÷3 / 6, which is 4:5:3:4:5:4:2:3.

[0211] The fusion weights of the radiation imaging mode, surface detection mode, thermal imaging mode and electromagnetic detection mode are 4 / 6÷4 / 6÷5 / 6÷4 / 6, that is, 4:4:5:4.

[0212] The fusion weights of the radiation imaging mode, acoustic detection mode, magnetic detection mode, surface detection mode, thermal imaging mode, and electromagnetic detection mode are calculated according to a fusion weight of 4 / 6÷5 / 6÷3 / 6÷4 / 6÷5 / 6÷4 / 6, which is 4:5:3:4:5:4.

[0213] Step S7, Feature Extraction and Fusion: Feature vectors are extracted from the modal data after the preprocessing in Step S5, and the extracted feature vectors are spliced ​​and fused according to the combination processing result of the application mode and the modal data in Step S4 and the fusion weight provided in Step S6 to form fused feature information under six application modes.

[0214] Step S8, Defect Identification and Localization: The weld seam is identified and located using six deep learning models based on the fused feature information from the six application modes.

[0215] Example 2: Figure 3 As shown, this invention also proposes a small-diameter pipe weld defect detection system based on multimodal fusion, which performs the above-described small-diameter pipe weld defect detection method based on multimodal fusion, and includes the following modules:

[0216] Data acquisition module: used to acquire modal data based on the detection method of the small pipe weld;

[0217] Function extraction module: connected to the data acquisition module, used to extract the function of the modal data;

[0218] Pattern segmentation module: connected to the function extraction module, used to segment the application modes of the small pipe weld according to the functions of the modal data extracted by the function extraction module;

[0219] Modal data combination module: connected to the mode division module, used to combine the corresponding modal data according to the application mode in the mode division module;

[0220] Data preprocessing module: connected to the modal data combining module, used to preprocess the modal data in the modal data combining module respectively;

[0221] Fusion weight calculation module: connected to the data preprocessing module, used to calculate the fusion weight of the preprocessed modal data according to the functions of the modal data extracted by the function extraction module;

[0222] Feature extraction and fusion module: connected to the fusion weight calculation module, used to extract feature vectors from the modal data after preprocessing by the data preprocessing module, and to splice and fuse the extracted feature vectors according to the combination processing result of the application mode and the modal data in the modal data combination module and the fusion weight provided by the fusion weight calculation module, to form fused feature information under six application modes;

[0223] Defect identification and localization module: connected to the feature extraction and fusion module, used to identify and locate weld seams by using six deep learning models to extract and fuse the feature information from the six application modes.

[0224] Example 3:

[0225] An electronic device, the electronic device comprising:

[0226] Processor and memory;

[0227] The processor executes the steps of the small pipe weld defect detection method based on multimodal fusion as described in any of Embodiment 1 by calling the program or instructions stored in the memory.

[0228] Example 4:

[0229] A computer-readable storage medium comprising computer program instructions that cause a computer to perform the steps of the small pipe weld defect detection method based on multimodal fusion as described in any of Embodiment 1.

[0230] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0231] It should be noted that the terminology used in this invention is for describing specific embodiments only and is not intended to limit the scope of this application. As shown in this specification, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may include the plural. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element.

[0232] It should also be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Unless otherwise expressly specified and limited, the terms "installed," "connected," "linked," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components. For those skilled in the art, the specific meaning of the above terms in the present invention can be understood according to the specific circumstances.

[0233] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting weld defects in small-diameter pipes based on multimodal fusion, characterized in that, Includes the following steps: Step S1, Data Acquisition: Obtain modal data according to the detection method of the small pipe weld; Step S2, Function Extraction: Extract the functions of the modal data; Step S3, Mode Classification: Based on the function of the modal data extracted in step S2, the application modes of the small pipe weld are classified. Step S4, Modal data combination: The application mode in step S3 is combined with the corresponding modal data, and each application mode contains several modal data. Step S5, Data preprocessing: Preprocess the modal data under the application mode respectively; Step S6, fusion weight calculation: Based on the function of the modal data extracted in step S2, the fusion weight is calculated for the preprocessed modal data; Step S7, Feature Extraction and Fusion: Feature vectors are extracted from the modal data after the preprocessing in Step S5. The modal data are then spliced ​​and fused according to the combination processing result of the application mode and the modal data in Step S4 and the fusion weight provided in Step S6 to form fused feature information under the six application modes. Step S8, Defect Identification and Localization: The weld seam is identified and located using six deep learning models based on the fused feature information from the six application modes. In step S1, the detection methods for the weld of the small pipe include: X-ray detection, ultrasonic detection, magnetic particle detection, penetrant detection, infrared thermal imaging detection, eddy current detection, optical detection, and laser holographic detection. Modal data are obtained according to the detection methods of the small pipe weld, including: radiation imaging mode, acoustic detection mode, magnetic detection mode, surface detection mode, thermal imaging mode, electromagnetic detection mode, visual detection mode, and three-dimensional detection mode; In step S2, the function of extracting the modal data includes: The functions of extracting the radiation imaging modes include non-destructive testing of welds, internal testing, small weld testing, and large weld testing. The functions of extracting the acoustic detection modes include non-destructive testing of welds, internal testing, rapid testing, small weld testing, and large weld testing. The functions of extracting the magnetic detection modes include non-destructive testing of welds, surface inspection, and large weld inspection. The functions for extracting the surface inspection modes include non-destructive testing of welds, surface inspection, small weld inspection, and large weld inspection. The functions of extracting the thermal imaging modes include non-destructive testing of welds, surface inspection, rapid inspection, small weld inspection, and large weld inspection. The functions of extracting the electromagnetic detection modes include non-destructive testing of welds, surface inspection, small weld inspection, and large weld inspection. The functions of extracting the visual inspection modal include non-destructive testing of welds and surface inspection. The functions for extracting the three-dimensional detection modes include non-destructive testing of welds, internal testing, and surface testing. In step S3, the application modes are divided into non-destructive rapid inspection mode, non-destructive surface inspection mode, non-destructive internal inspection mode, non-destructive high-precision inspection mode, non-destructive small weld inspection mode and non-destructive large weld inspection mode. In step S4, combining the application mode with the modal data specifically involves: The non-destructive rapid detection mode consists of an acoustic detection mode and a thermal imaging mode; The non-destructive surface inspection mode consists of a magnetic inspection mode, a surface inspection mode, a thermal imaging mode, an electromagnetic inspection mode, a visual inspection mode, and a three-dimensional inspection mode; The non-destructive internal inspection mode consists of a radiation imaging mode, an acoustic inspection mode, and a three-dimensional inspection mode; The non-destructive high-precision testing mode consists of radiation imaging mode, acoustic testing mode, magnetic testing mode, surface testing mode, thermal imaging mode, electromagnetic testing mode, visual testing mode, and three-dimensional testing mode; The non-destructive small weld inspection mode consists of radiation imaging mode, surface inspection mode, thermal imaging mode and electromagnetic inspection mode; The non-destructive large weld inspection mode consists of radiation imaging mode, acoustic inspection mode, magnetic inspection mode, surface inspection mode, thermal imaging mode and electromagnetic inspection mode; In step S5, the preprocessing includes: preprocessing the image data, ultrasonic data, infrared thermal image data, and electrical signal data respectively; The image data preprocessing steps include: image loading, noise removal, contrast enhancement, edge detection, binarization, morphological operations, and image saving; The ultrasonic data preprocessing steps include: ultrasonic data loading, DC offset removal, bandpass filtering, threshold processing, envelope detection, ultrasonic signal plotting, and ultrasonic data saving. The infrared thermal image data preprocessing steps include: loading the infrared thermal image, removing background noise, enhancing contrast, normalizing data, and saving the infrared thermal image. The electrical signal data preprocessing steps include: electrical signal data loading, DC offset removal, application of low-pass filter, data standardization, and electrical signal data saving; In step S6, the fusion weights are calculated for the preprocessed modal data, specifically as follows: Step S61: Statistically analyze all the functions of the modal data, namely: non-destructive testing function, surface testing function, internal testing function, rapid testing function, small weld seam testing function, and large weld seam testing function; Step S62: Calculate the percentage of each modal data function among all functions, as follows: The radiation imaging mode accounts for 4 / 6 of all the functions described. The acoustic detection mode accounts for 5 / 6 of all the functions described. The magnetic detection mode accounts for 3 / 6 of all the functions described. The surface detection mode accounts for 4 / 6 of all the functions described. The thermal imaging modality accounts for 5 / 6 of all the functions described. The electromagnetic detection mode accounts for 4 / 6 of all the functions described. The visual detection modality accounts for 2 / 6 of all the functions described. The three-dimensional detection modality accounts for 3 / 6 of all the functions described. Step S63: Calculate the fusion weights for the modal data to be fused based on their proportions, as follows: The fusion weight of the acoustic detection mode and the thermal imaging mode is 5 / 6÷5 / 6, that is, a fusion weight of 1:

1. The fusion weights of the magnetic detection mode, surface detection mode, thermal imaging mode, electromagnetic detection mode, visual detection mode and three-dimensional detection mode are 3 / 6÷4 / 6÷5 / 6÷4 / 6÷2 / 6÷3 / 6, that is, 3:4:5:4:2:

3. The fusion weights of the radiation imaging mode, acoustic detection mode, and three-dimensional detection mode are 4 / 6÷5 / 6÷3 / 6, i.e., 4:5:

3. The fusion weights of the radiation imaging mode, acoustic detection mode, magnetic detection mode, surface detection mode, thermal imaging mode, electromagnetic detection mode, visual detection mode, and three-dimensional detection mode are calculated according to the ratio of 4 / 6÷5 / 6÷3 / 6÷4 / 6÷5 / 6÷4 / 6÷2 / 6÷3 / 6, which is 4:5:3:4:5:4:2:

3. The fusion weights of the radiation imaging mode, surface detection mode, thermal imaging mode and electromagnetic detection mode are 4 / 6÷4 / 6÷5 / 6÷4 / 6, that is, 4:4:5:

4. The fusion weights of the radiation imaging mode, acoustic detection mode, magnetic detection mode, surface detection mode, thermal imaging mode, and electromagnetic detection mode are calculated according to a fusion weight of 4 / 6÷5 / 6÷3 / 6÷4 / 6÷5 / 6÷4 / 6, which is 4:5:3:4:5:

4.

2. The method for detecting weld defects in small-diameter pipes based on multimodal fusion according to claim 1, characterized in that, The modal data are classified into the following categories based on data type: image data, ultrasonic data, infrared thermal imaging data, and electrical signal data.

3. The method for detecting weld defects in small-diameter pipes based on multimodal fusion according to claim 1, characterized in that, The distinction between the large weld and the small weld is based on the following criteria: When the diameter of the small connecting pipe is less than or equal to 50% of the diameter of the main pipe, the weld of the small connecting pipe is determined to be a small weld; conversely, when the diameter of the small connecting pipe is greater than 50% of the diameter of the main pipe, the weld of the small connecting pipe is determined to be a large weld.

4. A small-diameter pipe weld defect detection system based on multimodal fusion, using the small-diameter pipe weld defect detection method based on multimodal fusion as described in any one of claims 1-3, characterized in that, Includes the following parts: Data acquisition module: used to acquire modal data based on the detection method of the small pipe weld; Function extraction module: connected to the data acquisition module, used to extract the function of the modal data; Pattern segmentation module: connected to the function extraction module, used to segment the application modes of the small pipe weld according to the functions of the modal data extracted by the function extraction module; Modal data combination module: connected to the mode division module, used to combine the corresponding modal data according to the application mode in the mode division module; Data preprocessing module: connected to the modal data combining module, used to preprocess the modal data in the modal data combining module respectively; Fusion weight calculation module: connected to the data preprocessing module, used to calculate the fusion weight of the preprocessed modal data according to the functions of the modal data extracted by the function extraction module; Feature extraction and fusion module: connected to the fusion weight calculation module, used to extract feature vectors from the modal data after preprocessing by the data preprocessing module, and to splice and fuse the extracted feature vectors according to the combination processing result of the application mode and the modal data in the modal data combination module and the fusion weight provided by the fusion weight calculation module, to form fused feature information under six application modes; Defect identification and localization module: connected to the feature extraction and fusion module, used to identify and locate weld seams by using six deep learning models to extract and fuse the feature information from the six application modes.

Images