High-precision stainless steel weld defect recognition device based on multi-round cross-modal attention mechanism

Abstract

The invention provides a high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism. The high-precision stainless steel weld defect recognition device comprises an ultrasonic signal acquisition module, a data preprocessing module, a cross-modal defect recognition module and an online result display module, the ultrasonic signal acquisition module is used for synchronously acquiring an A scanning original waveform and workpiece position information corresponding to the A scanning original waveform; the data preprocessing module is used for preprocessing the A scanning original waveform, mapping a time sequence signal into a Gramb angle field two-dimensional image, and generating complementary time sequence image bimodal data; the cross-modal defect identification module sends the bimodal data into a deep learning network, extracts time sequence and spatial frequency features through a feature extraction neural network, reinforces correlation between the time sequence and spatial frequency features by means of multi-round cross-modal multi-head attention iteration interaction, and combines channel attention and focus loss strategies at the same time; and outputting a stable defect classification result under the condition that the sample categories are unbalanced.

Description

Technical Field

[0001] The present invention relates to the technical field of stainless steel weld defect recognition, and in particular to a high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism. Background Art

[0002] Austenitic stainless steel, due to its corrosion resistance and high strength, is widely used in nuclear power pipelines, marine engineering, and chemical transportation systems. The quality of its welds directly determines the safety of the entire system. Due to factors such as coarse grains, ultrasonic waves experience strong attenuation and high scattering noise in such materials, causing the effective signal in the A-scan signal to be drowned out by noise and making individual echo peaks difficult to separate and analyze. Industrial sites still rely primarily on manual analysis of signal amplitude and phase to identify internal defects such as cracks, slag inclusions, and unfused components. This approach is inefficient, subjective, and has poor repeatability, making it difficult to meet the real-time, high-precision testing requirements of modern production lines. While multi-device joint inspection technologies (such as X-ray, magnetic particle, and penetrant) can address certain blind spots, they carry the cost burden of X-ray safety, surface preparation, and cycle delays.

[0003] With the acceleration of digital manufacturing, signal processing and machine learning are being introduced into ultrasonic testing to reduce reliance on manual labor. Early methods relied on manual feature extraction combined with traditional classifiers to achieve detection. However, problems still exist, such as feature sensitivity to noise and limited model generalization. Although deep learning has improved recognition rates in experimental environments, it still struggles to maintain stable performance under field conditions with strong scattering noise, unbalanced samples, and limited computing resources. This is especially true for a few types of defects, such as microcracks and small pores, whose detection rates are insufficient. Therefore, there is an urgent need to develop a stainless steel weld defect recognition method that can simultaneously meet the requirements of real-time, reliability, and high precision. Summary of the Invention

[0004] The present invention relates to a high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism, which aims to realize real-time automatic identification of defects such as cracks, slag inclusions, lack of fusion, incomplete penetration, and porosity in welds under industrial environments with strong scattering noise and sample imbalance.

[0005] The specific technical solution is: a high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism, including: an ultrasonic signal acquisition module, a data preprocessing module, a cross-modal defect recognition module, and an online result display module;

[0006] The ultrasonic signal acquisition module is used to synchronously obtain the A-scan original waveform and its corresponding workpiece position information on the production line;

[0007] The data preprocessing module preprocesses the A-scan raw waveform and maps the time series signal into a Gram angle field two-dimensional image to generate complementary time series image dual-modal data;

[0008] The cross-modal defect recognition module feeds bimodal data into a deep learning network, extracting temporal and spatial frequency features through a feature extraction neural network. It then uses multiple rounds of cross-modal multi-head attention to iteratively strengthen the correlation between the two. Combining channel attention and focal loss strategies, it outputs stable defect classification results even under conditions of imbalanced sample categories.

[0009] The online result display module receives the recognition output, completes defect location and alarm, and transmits the results back to the manufacturing execution system to achieve closed-loop quality control.

[0010] The present invention has the following beneficial effects:

[0011] The advantages of the present invention include accuracy, speed and robustness: first, multimodal deep fusion significantly improves the ability to identify tiny defects and multiple scattering interference; second, the overall network architecture is integrated through lightweight design to meet the real-time operation of online detection; third, the end-to-end automatic judgment process reduces dependence on manual experience and can be seamlessly integrated with existing manufacturing execution systems, providing an economical, efficient and scalable new approach to non-destructive testing of welds for the field of high-end equipment manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic diagram of the connection of various modules of the high-precision stainless steel weld defect recognition method based on the multi-round cross-modal attention mechanism provided by the present invention;

[0013] Figure 2 This is the specific workflow of the core module of cross-modal defect recognition. DETAILED DESCRIPTION

[0014] To make the technical problems solved, the technical solutions adopted, and the technical effects achieved by the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, rather than all of the contents.

[0015] like Figure 1 As shown, the present invention provides a high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism, and a schematic diagram of the connection of its modules. The specific implementation is as follows:

[0016] The entire inspection chain is divided into four functional modules: ultrasonic signal acquisition, data preprocessing, cross-modal defect recognition, and online results display. These modules are interconnected via standardized data interfaces and form a closed-loop control loop with the Manufacturing Execution System (MES) via a return bus.

[0017] First, the ultrasonic signal acquisition module is responsible for synchronously acquiring ultrasonic A-scan signals and their corresponding workpiece position information on the production line, providing consistent calibration for subsequent processing; second, the data preprocessing module performs noise suppression and amplitude normalization on the A-scan signals, and maps the time series signals into a two-dimensional image of the Gram angle field to generate complementary time series-image bimodal data; third, the cross-modal defect recognition module feeds the bimodal data into a deep learning network, extracts the time series and spatial frequency features through a feature extraction neural network, and then uses multiple rounds of cross-modal multi-head attention iterative interaction to strengthen the association between the two. At the same time, combined with channel attention and focal loss strategies, it outputs stable defect classification results under conditions of imbalanced sample categories; fourth, the online result display module receives the recognition output, completes defect location and alarm, and transmits the results back to the manufacturing execution system to achieve closed-loop quality control.

[0018] When a workpiece with a weld defect enters the inspection station, the ultrasonic signal acquisition module begins operation. This module consists of an array ultrasonic probe, an automatic scanning slide, and an incremental encoder mounted coaxially with the probe's displacement. The probe is positioned in a coupling water tank at a fixed angle close to the weld surface, while the slide drives the probe along the weld's axis at a constant speed. The encoder records the probe's displacement in real time, and the acquisition card packages and outputs the raw A-scan waveform with timestamps and position information, providing unified temporal and spatial calibration for subsequent digital processing.

[0019] The original waveform of the A-scan signal is sent to the data preprocessing module via the bus. The module first performs Gaussian smoothing on all channels to suppress random electronic noise and coarse grain scattering burrs; the filtered output ,in, For the moment The original waveform amplitude is sampled at represents the Gaussian kernel standard deviation (reflecting the smoothing intensity), is the half-width of the symmetric window (determines the range of sampling points involved in convolution), is the current sampling index, is the output amplitude after filtering. Then the signal amplitude is mapped to the [0,1] interval using maximum normalization. , in, is the normalized signal, and are the minimum and maximum values ​​of the waveform of this frame respectively; this step is used to reduce the dynamic range fluctuation caused by the difference in coupling conditions. Finally, the Gram Angle Field (GAF) conversion unit is called to convert the normalized time series data (N is the number of sampling points in a single frame, For the Normalized amplitudes) are mapped point by point to polar coordinates and the cosine or sine of the angular difference is calculated, resulting in a 2D polar coordinate image of consistent size. Thus, each sampling point has two complementary modal representations: a denoised A-scan timing signal and a GAF polar coordinate image, maintaining a one-to-one correspondence with the weld coordinates.

[0020] The cross-modal defect recognition module plays a core role in the classification system. This module is further divided into three sub-units: bimodal feature extraction unit, multi-round cross-modal attention fusion unit, and defect classification decision unit. The data transmission, iterative interaction and training mechanism between each sub-unit are shown in Figure 2 Here, we will only describe its overall functionality: the bimodal feature extraction unit extracts temporal features and spatial frequency texture features from the A-scan time series signal and GAF ​​image, respectively; the multi-round cross-modal attention fusion unit uses multi-head attention to perform a deep interactive fusion of the two features; and the defect classification decision unit assigns defect categories based on the fused features and maps weld coordinates. The module's output is transmitted to the display layer via a standard JSON report and simultaneously written back to the MES for generating rework instructions or quality reports.

[0021] The bimodal feature extraction unit uses a long short-term memory (LSTM) network to extract the temporal dependency information of the denoised A-scan signal, and a convolutional neural network to extract the spatial-frequency texture of the GAF two-dimensional image, providing a foundation for the subsequent fusion of complementary features. The cross-modal attention fusion unit uses multiple rounds of multi-head attention iteration to adaptively capture the correspondence between key A-scan signal echoes and images, strengthening effective features and suppressing scattered noise in each round. It also uses channel attention to dynamically weight the spectrum-time dimension to highlight signal patterns highly correlated with defects. The defect classification decision unit splices the fused features and inputs them into a fully connected structure. During the training phase, focal loss is introduced to improve the detection sensitivity of minority defects, ensuring that the model maintains a high recall rate even in data imbalance scenarios. This invention can achieve recognition accuracy rates exceeding 98% on public datasets of similar welds and can be directly embedded in digital production lines, enabling online, automatic, and low-labor dependency weld quality inspection.

[0022] The online result display module visualizes the identification results for the operator. The system updates the defect category label and 2D weld coordinates in real time on the interface and simultaneously pushes the complete data package to the MES, achieving a digital closed-loop control of weld quality. If the MES determines that a defect exceeds the process tolerance threshold, the production line automatically triggers a repair or alarm process, ensuring the safety of welding high-end pressure-bearing structures.

[0023] Through these four modules, the present invention achieves online, high-precision, automatic identification of multiple types of defects in austenitic stainless steel welds without changing production cycles. The cross-modal defect recognition module significantly improves the detection rate of small cracks and weak scattering defects through bimodal deep feature learning and multiple rounds of attention iteration. The front-end processing of Gaussian denoising, amplitude normalization, and GAF ​​image conversion provides robust and consistent input for the deep model.

[0024] like Figure 2 As shown in the figure, the specific workflow and implementation details of each sub-unit within the cross-modal defect recognition module will be further explained.

[0025] After entering the module, the signal first passes through the bimodal feature extraction unit. In this stage, weld data is input in two parallel forms: a denoised and normalized A-scan time series vector, and a two-dimensional GAF image obtained by applying the Gram Angular Field Transform to this vector. The time series branch encodes the dynamic dependencies of the waveform using a long short-term memory network. This is then compressed into a one-dimensional time series feature vector through dropout regularization. The image branch uses a convolution-batch normalization-relinear activation (ReLU) stacking structure to extract multi-scale spatial frequency texture features from the GAF image, generating a spatial feature vector. Each convolution output then undergoes a squeeze-and-excitation (SE) channel recalibration. The network automatically amplifies the channel response most relevant to the defect scattering texture, resulting in a spatial feature vector of the same dimension after global average pooling. The two feature vectors are now aligned in both time and space, extracting complementary information before entering the next unit.

[0026] The multi-round cross-modal attention fusion unit is responsible for deeply interacting the two feature vectors. The system first linearly maps the temporal feature vector and the spatial feature vector to generate a query / key / value pair. Then, through "cross-modal multi-head attention," the two vectors are matched back and forth and iterated four times, achieving bidirectional alignment from "time series to image" and "image to time series." Within each round, attention is calculated on Q / K / V and added to the input residual. LayerNorm is then applied to preserve the original information while gradually reducing scattered noise and artifacts. After four rounds, the output vector enters a fully connected layer and is mapped to 512 dimensions, serving as the most discriminative fused feature for the weld in this frame in both domains.

[0027] After receiving the fused features, the defect classification decision unit first compresses the 512-dimensional features to 256 dimensions through a fully connected mapping, applying ReLU activation and Dropout regularization. Then, through a final linear mapping, it outputs the logits for the five defect categories, which are then normalized using Softmax to obtain the class probability distribution. During the inference phase, the category with the highest probability is selected and its coordinates recorded. During the training phase, a focal loss is introduced after Softmax to automatically enhance learning for minority class defects through the focusing factor γ and class weight α. This approach maintains high recall and overall accuracy despite sample imbalance. While maintaining an overall accuracy of over 98%, the model maintains a recall of approximately 98% for minority class defects. The entire inference process takes an average of 15 ms on an industrial GPU, enabling online, high-precision automatic identification of austenitic stainless steel weld defects.

Claims

1. A high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism, characterized in that: include: Ultrasonic signal acquisition module, data preprocessing module, cross-modal defect recognition module and online result display module; The ultrasonic signal acquisition module is used to synchronously obtain the A-scan original waveform and its corresponding workpiece position information on the production line; The data preprocessing module preprocesses the A-scan raw waveform and maps the time series signal into a Gram angle field two-dimensional image to generate complementary time series image dual-modal data; The cross-modal defect recognition module feeds bimodal data into a deep learning network, extracting temporal and spatial frequency features through a feature extraction neural network. It then uses multiple rounds of cross-modal multi-head attention to iteratively strengthen the correlation between the two. Combining channel attention and focal loss strategies, it outputs stable defect classification results even under conditions of imbalanced sample categories. The online result display module receives the recognition output, completes defect location and alarm, and transmits the results back to the manufacturing execution system to achieve closed-loop quality control.

2. A high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism according to claim 1, characterized in that: When a workpiece with weld defects enters the inspection station, the ultrasonic signal acquisition module starts working first. The module consists of an array ultrasonic probe, an automatic scanning slide, and an incremental encoder installed coaxially with the probe displacement. The probe is placed close to the weld surface at a fixed depression angle in the coupling water tank. The slide drives the probe to move axially along the weld at a constant speed. The encoder records the probe displacement in real time. The acquisition card packages and outputs the original A-scan waveform with timestamp and position information, providing unified time and space calibration for subsequent digital processing.

3. The high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism according to claim 1 is characterized in that: The original waveform of the A-scan signal is sent to the data preprocessing module via the bus, and the module outputs , is the output amplitude after filtering; then the signal amplitude is mapped to the [0,1] interval using the maximum value normalization, and finally the Gram angle field conversion unit is called to convert the normalized A-scan timing signal data Map point by point to polar coordinates and calculate the cosine or sine value of the angle difference, and transcribe it into a two-dimensional GAF polar coordinate image with the same size. N is the number of sampling points in a single frame. For the Normalized amplitude, so that each sampling point has two complementary modal representations: denoised A-scan timing signal and GAF ​​polar coordinate image.

4. The high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism according to claim 1 is characterized in that: The cross-modal defect recognition module includes: a bimodal feature extraction unit, a multi-round cross-modal attention fusion unit and a defect classification decision unit. The bimodal feature extraction unit extracts bimodal time series image data from the A-scan time series signal and the GAF polar coordinate image respectively. The bimodal data includes: time series feature vector and spatial feature vector; the multi-round cross-modal attention fusion unit uses multi-head attention to perform deep interactive fusion of time series feature vector and spatial feature vector; the defect classification decision unit gives the defect category based on the fused features and maps the weld coordinates.

5. The high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism according to claim 1 is characterized in that: The online result display module is used to refresh the defect category labels and two-dimensional weld coordinates in real time on the interface, and at the same time push the complete data package to the MES, realizing a digital closed loop for weld quality control. If the MES determines that the defect exceeds the process tolerance threshold, the production line automatically triggers the rework or alarm process.

6. The high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism according to claim 4 is characterized in that: In the dual-modal feature extraction unit, weld data is input in parallel in two forms, including: A-scan timing signals and two-dimensional GAF images. The timing branch of the dual-modal feature extraction unit encodes the dynamic dependency of the waveform through a long short-term memory network; it is then compressed into a one-dimensional timing feature vector under the regularization of the Dropout layer. The image branch uses a convolution-batch normalization-linear activation ReLU superposition structure to extract multi-scale spatial frequency texture of the GAF image. The output of each level of convolution is then recalibrated through the compression-excitation SE channel. The network automatically amplifies the channel response most relevant to the defect scattering texture, and obtains a spatial feature vector of the same dimension after global average pooling. At this point, the two feature vectors are aligned in time and space coordinates, extracting complementary information and entering the next unit.

7. The high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism according to claim 6 is characterized in that: In the multi-round cross-modal attention fusion unit, a linear mapping is first performed on the temporal and spatial feature vectors to generate the query, key and value required for the multi-head attention. Then, N rounds of multi-head attention iterations are performed in a bidirectional manner: in the first round, the temporal vector is used to retrieve the image texture, the roles are swapped in the second round, and after the third and fourth rounds, the matching relationship formed in the first two rounds is reinforced. The output of each round is superimposed on the input through the residual connection layer. Within each round, the query, key and value are calculated for attention and added to the input residual, and then passed through the LayerNorm layer. After the four rounds, the output vector enters the fully connected layer and is mapped to 512 dimensions, which serves as the discriminative fusion feature of the weld in this frame in the dual domain.

8. The high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism according to claim 7 is characterized in that: After receiving the above fusion features, the defect classification decision unit completes the judgment through two-level fully connected layer mapping: the first level first uses the ReLU activation function to compress the dimension and uses Dropout regularization to prevent overfitting. The second level linearly outputs the classification probability of five types of defects. During inference, the confidence of different types of defects is obtained through the Softmax layer, and the category corresponding to the highest probability is output together with the probe displacement coordinates.

9. The high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism according to claim 3 is characterized in that: The pre-processing module first performs Gaussian smoothing on all channels to suppress random electronic noise and coarse-grained scattering burrs; the filtered output ,in, For the moment The original waveform amplitude is sampled at represents the Gaussian kernel standard deviation, is the half-width of the symmetric window, is the current sampling index, is the output amplitude after filtering.

10. The high-precision stainless steel weld defect recognition device based on a multi-round cross-modal attention mechanism according to claim 3 is characterized in that: The formula for mapping the signal amplitude to the interval [0,1] using maximum value normalization is: ,in, is the normalized signal, and They are the minimum and maximum values ​​of the waveform of this frame respectively.

Images