An image recognition-based defect detection analysis system

Abstract

The application discloses a kind of based on image recognition defect detection analysis system, it is related to welding detection technical field, including the data acquisition unit of the multiple ultrasonic images data of welding piece in each acquisition moment being obtained using ultrasonic phased array equipment, the data preprocessing unit of each frame of the ultrasonic image obtained is preprocessed, the spatial feature and time sequence feature of weld defect are extracted as initial feature, form the extraction unit and fusion unit of a comprehensive feature tensor including time and space information, and the classification unit of the classification output and position output of weld defect and the welding parameter optimization unit of welding parameter adjustment according to defect analysis result.The application is based on the dynamic analysis and positioning result of weld defect to guide the adjustment of welding process parameter, realizes from welding defect detection to cause analysis, again to closed-loop control of process parameter optimization, adjusts welding parameter to avoid the reoccurrence of defect, improves welding efficiency and quality.

Description

Technical Field

[0001] This invention relates to the field of welding inspection technology, specifically an image recognition-based defect detection and analysis system. Background Technology

[0002] In welding quality assessment, post-cooling inspection is a traditional and intuitive method. By analyzing ultrasonic images of the weld after cooling, the type of welding defect can be identified. However, this method only provides information about the fact that the defect has occurred, such as the presence of porosity, cracks, or lack of fusion in the weld. It cannot reveal the process and causes of these defects during welding, which are often accompanied by dynamic changes in the molten state of the weld area. For example, during welding, porosity may form due to insufficient gas removal from the molten pool or an inappropriate cooling rate, while cracks may occur due to stress concentration caused by excessive local temperature differences within the molten pool. This dynamic process information cannot be obtained through single-frame inspection after cooling, limiting a deeper understanding of the causes of defects.

[0003] The inadequacy of this defect analysis is particularly evident in the automated welding process of new energy vehicle components. Welding of new energy vehicle components demands extremely high quality and consistency, while purely result-oriented analysis is insufficient for effectively optimizing the welding process. For example, when cooling inspection detects porosity defects, while the existence of the defect is known, its dynamic causes during the welding process cannot be ascertained, thus failing to provide a basis for optimizing welding parameters. Summary of the Invention

[0004] 1) Technical problems to be solved

[0005] This invention provides an image recognition-based defect detection and analysis system that can detect and classify welding defects by extracting dynamic features in spatial and temporal dimensions, and optimize subsequent welding processes by combining dynamic feature information.

[0006] (ii) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution: an image recognition-based defect detection and analysis system, applicable to the identification and classification of weld defects in welded parts, comprising:

[0008] The data acquisition unit uses an ultrasonic phased array device to continuously acquire multiple frames of ultrasonic images of the weld area. Each frame of the ultrasonic image contains the ultrasonic reflection characteristics of the weld area and the time dimension information of the acquisition time.

[0009] The data preprocessing unit constructs a unified reference coordinate system based on the initial calibration parameters of the ultrasonic phased array device. After preprocessing each frame of the acquired ultrasonic image, the weld area is unified into the reference coordinate system.

[0010] The feature extraction unit extracts spatial and temporal features of the weld area from multiple frames of ultrasonic images. The spatial features include the edge morphology and ultrasonic reflection intensity distribution of the weld area. The temporal features are extracted by sequentially extracting the changes in ultrasonic reflection signals in each frame of the ultrasonic image. A three-dimensional image of the weld area is constructed using image stitching technology, and the extracted spatial and temporal features of the weld area are embedded into the three-dimensional image.

[0011] The fusion unit receives the three-dimensional image and uses a neural network to first decode the spatial and temporal features embedded in the three-dimensional image into feature tensors of corresponding scales, and then integrates all the feature tensors for convolution processing to generate a comprehensive feature tensor of a unified scale.

[0012] The classification unit constructs a classification model based on a multi-task learning architecture, takes the comprehensive feature tensor as the input of the classification model, and outputs the type and confidence level of the defect and its position in the reference coordinate system.

[0013] The welding parameter optimization unit combines the spatial and temporal characteristics of the defect, uses causal reasoning to analyze the cause of the defect, uses reinforcement learning algorithm to establish a correlation between historical welding process parameters and corresponding historical defect causes, and after setting the optimization objective, the reinforcement learning algorithm iteratively outputs the adjusted welding process parameters.

[0014] Furthermore, the ultrasonic phased array device in the data acquisition unit consists of multiple independently controlled ultrasonic transducers. Each transducer can emit ultrasonic waves, and a controllable beam direction and focus are formed by adjusting the timing of the transducer excitation to receive ultrasonic reflected signals.

[0015] By continuously acquiring multiple frames of the ultrasonic images, the ultrasonic reflection characteristics of the welded parts at different time points and in different ultrasonic emission spaces are captured, forming an ultrasonic image sequence with a time dimension.

[0016] Furthermore, after receiving multiple frames of ultrasound images at each acquisition time transmitted by the data acquisition unit, the data preprocessing unit reads the timestamp information of each frame of ultrasound image to ensure the temporal consistency of the ultrasound image sequence.

[0017] The weld seam area was initially delineated using a grayscale threshold segmentation algorithm, and noise was removed.

[0018] After adjusting the image contrast, the coordinate information of the weld area is extracted, and the marked coordinate information is bound to the ultrasonic image data to form a standardized data format.

[0019] Furthermore, the feature extraction unit, based on the multi-frame ultrasound images output by the data preprocessing unit and their labeled coordinate information, analyzes the weld area in each frame of the ultrasound image to obtain the reflection intensity distribution and edge contour information as the spatial features.

[0020] Furthermore, the feature extraction unit performs time series analysis based on the pixel value changes in the weld area in the multi-frame ultrasonic images. By calculating the rate of change of the ultrasonic reflection signal and the change of the ultrasonic signal amplitude, the dynamic change of the ultrasonic reflection signal over time is quantified. Based on a temporal convolutional network, the change law of the ultrasonic reflection signal over time is captured, thereby extracting dynamic change features as the temporal features.

[0021] Furthermore, the feature extraction unit integrates the spatial features and temporal features through convolutional concatenation to form the feature matrix containing multi-scale information. According to the time sequence of the acquisition time, the feature matrices corresponding to multiple frames of ultrasound images are concatenated. For the concatenated feature matrix, the feature values ​​located at the same pixel position are accumulated to generate the global feature.

[0022] Furthermore, the fusion unit receives the target 3D image and global features transmitted by the feature fusion unit, extracts the deep features of the weld area using a convolutional neural network, and concatenates the tensors of the deep features and global features of the target 3D image in the channel dimension to output a comprehensive feature tensor containing spatial and temporal information.

[0023] Furthermore, the classification unit, through the comprehensive feature tensor input by the fusion unit, utilizes a pre-built multi-task learning architecture to complete the classification and localization of weld defects; specifically:

[0024] The spatial features, temporal features, and their corresponding fusion information in the comprehensive feature tensor are input into the shared feature extraction layer in the multi-task learning architecture to extract deep patterns.

[0025] By analyzing the probability distribution of the output defect type after analyzing the deep pattern using the fully connected layer and activation function in the multi-task learning architecture, the category corresponding to the highest probability is selected as the classification result, and the highest probability is used as the confidence level.

[0026] The location coordinates of the defect are output using a regression approach within a multi-task learning architecture.

[0027] The classification unit outputs the type of weld defect, its confidence level, and its location in the weld area.

[0028] Furthermore, the welding parameter optimization unit receives the defect type and location output by the classification unit, and combines the constructed reference coordinate system of the weld area with the change of ultrasonic reflection intensity of the weld area to construct a dynamic change feature map of the defect formation process, and receives real-time monitoring data of the current welding process parameters during the welding process, including heat input, welding speed and shielding gas flow rate.

[0029] Combining the formation mechanisms of different defect types, causal reasoning is used to analyze the cause of the defect based on the dynamic change characteristic spectrum and the current welding process parameters, that is, to analyze and deduce the causal relationship between the defect and the current welding process parameters;

[0030] A parameter optimization model is constructed using a reinforcement learning algorithm. This model establishes the correlation between historical welding process parameters and the corresponding historical defect causes. The current welding process parameters and defect causes are used as inputs to the parameter optimization model, with defect minimization as the optimization objective. The adjusted welding process parameters are then iteratively output using the reinforcement learning algorithm.

[0031] (iii) Beneficial effects:

[0032] Compared with the prior art, this invention has the following beneficial effects:

[0033] This invention utilizes the time-series information of multiple ultrasonic images and three-dimensional reconstruction technology of the weld area to comprehensively analyze the spatial and temporal characteristics of weld defects, thereby improving the recognition accuracy of complex defect patterns and reducing misjudgments and omissions caused by insufficient information in a single frame.

[0034] By utilizing feature extraction and fusion units, the reflection intensity distribution, morphological edge features, and temporal dynamic variation patterns of the weld area are analyzed and fused at multiple scales to generate comprehensive features containing global information. This enhances the system's adaptability and robustness to diverse defects. Furthermore, a multi-task learning architecture is introduced to integrate the classification and localization tasks of weld defects, enabling simultaneous detection of defect types and locations. This dynamic process information guides the adjustment of welding parameters, thereby preventing the continuous generation of defects in subsequent welding processes on automated welding lines and improving welding quality and production efficiency. Attached Figure Description

[0035] Figure 1 This is a block diagram illustrating the principle of an image recognition-based defect detection and analysis system provided in an embodiment of the present invention.

[0036] Figure 2 This is a flowchart of a method for a classification unit in an image recognition defect detection and analysis system provided by an embodiment of the present invention;

[0037] Figure 3This is a schematic diagram illustrating the principle of welding defect detection provided in this embodiment of the invention, from acquiring multiple frames of ultrasonic images to using a comprehensive feature tensor for classification.

[0038] Figure 4 This is a schematic diagram of the method flow from acquiring multiple ultrasound images to generating a comprehensive feature tensor for classification, provided in an embodiment of the present invention.

[0039] In the picture:

[0040] 10. Data acquisition unit; 20. Data preprocessing unit; 30. Feature extraction unit; 40. Fusion unit; 50. Classification unit; 60. Welding parameter optimization unit. Detailed Implementation

[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0042] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0043] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0044] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.

[0045] In recent years, ultrasonic testing technology has become an important tool for weld defect detection due to its high-resolution acquisition capabilities. In the process of defect detection in workpiece welds using ultrasonic testing technology, the inventors discovered that the evaluation of weld quality typically utilizes ultrasonic testing after the weld has cooled; that is, the type of welding defect can be identified by analyzing the ultrasonic images of the cooled weld. However, this time-phase detection method only provides information about the result after the defect has occurred, such as the presence of cracks, porosity, or lack of fusion in the weld, but it cannot reveal the process and causes of these defects during the welding process.

[0046] Considering that welding defects often involve dynamic changes in the molten state of the weld area—for example, porosity may form due to insufficient gas removal from the molten pool or an inappropriate cooling rate, while cracks may occur due to stress concentration caused by excessive local temperature differences within the molten pool—this dynamic process information cannot be obtained through single-frame image detection after cooling, limiting a deeper understanding of the causes of defects.

[0047] In the automated welding process of new energy vehicle components mentioned in some embodiments of this invention, the shortcomings of this defect analysis are particularly prominent. The welding of new energy vehicle components has extremely high requirements for quality and consistency, while simple result-oriented analysis is difficult to effectively optimize the welding process. For example, when cooling detection discovers porosity defects, although the existence of the defects can be known, the dynamic causes of the defects during the welding process cannot be known, thus failing to provide a basis for optimizing welding parameters.

[0048] In recent years, welding dynamic inspection methods based on ultrasonic phased array technology and artificial intelligence have gradually emerged. These methods capture multiple frames of ultrasonic images of the weld area in real time during the welding process, extracting dynamic features in both spatial and temporal dimensions, thus providing possibilities for analyzing the causes of welding defects. For example, dynamic inspection can reveal how certain welding parameters, such as heat input, welding speed, or shielding gas flow rate, can trigger porosity or crack defects. This dynamic process information can guide the real-time adjustment of welding parameters, thereby avoiding defects and improving welding quality and production efficiency.

[0049] In summary, the above problems can be understood as follows: traditional methods mostly focus only on welding characteristics at a single moment or in a single dimension, such as reflection intensity or edge shape, failing to combine the dynamic changes of weld defects in the time dimension or the global dynamic changes in three-dimensional space. The singular characteristics limit the ability to identify complex defect patterns. When welding defects occur without considering their causes and the inherent relationship with the current welding process parameters, if the current welding process parameters are maintained, such defects may persist on automated processing lines.

[0050] To solve the above problems, combined with Figures 1 to 4 As shown in the embodiments of the present invention, an image recognition-based defect detection and analysis system is proposed. By combining time series analysis and three-dimensional reconstruction technology of multi-frame ultrasonic images with a deep learning model, the spatial and temporal characteristics of weld defects are jointly modeled. The subsequent welding process is optimized by combining the welding melting dynamic process information. It is suitable for welding quality control and parameter adjustment of new energy vehicle components.

[0051] Specifically, first refer to Figure 1The data acquisition unit 10 in the middle is composed of a core component consisting of multiple independently controlled ultrasonic transducer arrays, which can independently transmit and receive ultrasonic signals. By adjusting the excitation timing of the transducer transmission, also known as phase delay, an ultrasonic beam with a certain directionality and focusing capability can be formed. This beam controllability allows the ultrasonic phased array equipment to flexibly scan the internal structure of the welded part and generate high-resolution cross-sectional images.

[0052] When an ultrasonic beam passes through a welded part, it will be reflected and scattered when it encounters discontinuities in the material, such as cracks, pores, or inclusions. The received reflected signals are digitally processed to generate ultrasonic images that reflect the internal structure of the welded part. These images not only contain geometric information of the weld, but also reveal the characteristics of weld defects, such as reflection intensity, shape, location, and size.

[0053] Therefore, it can be understood that the data acquisition unit 10, by dynamically controlling the focus and direction of the beam, can achieve high-resolution imaging at different depths and regions of the weldment. Furthermore, since each transducer in the array can be independently controlled, different scanning modes, such as linear scanning, sector scanning, or focused scanning, can be implemented through programming. Finally, by continuously acquiring multiple frames of ultrasonic images, the dynamic changes inside the weldment are captured, thus providing a more comprehensive reflection of the spatiotemporal characteristics of weld defects.

[0054] In some embodiments of the present invention, an ultrasonic phased array probe is installed. It is important to ensure good contact between the probe and the surface of the weldment, and to use an appropriate coupling agent, such as water or oil, to optimize the transmission of the ultrasonic waves. The frequency, beam direction, and depth of focus of the ultrasonic waves are set according to the material, thickness, and target detection area of ​​the weldment.

[0055] After setting up the ultrasonic phased array equipment, the array transducer is activated, and the beam direction and focus are gradually adjusted according to the set scanning mode, such as linear scanning. It is important to note that at each scanning position, the transducer emits ultrasonic waves, records the returned reflected signals, and converts the signals into digital images. The target area of ​​the weldment is continuously scanned, and multiple frames of ultrasonic images are acquired at different time points. Each frame corresponds to the reflection characteristics of the same cross-section of the weldment. Furthermore, each frame includes temporal information to record the temporal changes in the reflection characteristics, forming a time-series image.

[0056] In some embodiments of the present invention, the target welded part is set as a 20 mm thick welded steel plate for a vehicle frame. For potential weld defects in its weld area, an ultrasonic phased array device is used for detection. For example, the transducer's center frequency is set to 5 MHz, the scanning mode to linear scanning, the step size to 1 mm, and the focusing depth to 10 mm. The device continuously acquires 10 frames of ultrasonic images, with each frame capturing the reflection characteristics of the same cross-section of the welded part.

[0057] Finally, the data acquisition unit 10 saves the acquired multi-frame ultrasound images as sequence data and binds them with timestamps for subsequent feature extraction and analysis.

[0058] In summary, the data acquisition unit 10 provides a high-resolution, multi-dimensional data foundation for welding defect detection by acquiring multiple frames of ultrasonic images from the ultrasonic phased array device, which helps subsequent units to accurately complete the defect analysis task.

[0059] After the data acquisition unit 10 acquires multiple frames of ultrasonic images, all images are transmitted to the data preprocessing unit 20. The data preprocessing unit 20 extracts the core information of the weld area from the original multi-frame ultrasonic images and accurately labels the weld area to provide standardized, high-quality input data for the subsequent feature extraction unit 30.

[0060] It is understandable that, since ultrasonic images typically contain a large amount of complex information, including the distribution of reflection intensity, noise interference, and background areas, the data preprocessing unit 20 performs technical processing on them, including timestamp parsing, image segmentation, contrast adjustment, and region annotation, to ensure that the finally extracted weld area is clear and accurate, while retaining key features reflecting weld defects.

[0061] Specifically, the data acquisition unit 10 transmits a multi-frame ultrasound image sequence, with each frame accompanied by a timestamp. The data preprocessing unit 20 reads the timestamp of each frame to verify its temporal consistency, ensuring that the temporal dimension of the image sequence is not affected by device latency or data transmission errors. Furthermore, if discontinuous timestamps or missing frames are found, interpolation methods are used to fill in the missing time points, ensuring the temporal continuity of the image sequence. This ensures that subsequent temporal features are based on correct time information.

[0062] Subsequently, the data preprocessing unit 20 sets an initial grayscale threshold based on the characteristic that the weld seam appears as a high-reflection area in the ultrasonic image; for example, the pixel grayscale value is greater than a certain set value. It then iterates through the pixels of each frame, marking pixels that meet the threshold condition as weld seam areas and the remaining pixels as background areas, thereby generating a preliminary weld seam area mask that highlights the weld seam location.

[0063] After initial segmentation, morphological operations, such as opening and closing operations, are used to remove isolated small noise points or fill gaps in the weld area. Furthermore, in some embodiments, connected component analysis is also considered, which involves removing smaller isolated regions and retaining only larger connected regions as weld candidate regions.

[0064] Finally, the coordinates of the weld area are labeled on the image. Before the coordinate labeling, a unified reference coordinate system is constructed. This reference coordinate system is a three-dimensional Cartesian coordinate system used to describe the spatial position of the weld area within the welded workpiece. In some embodiments of the present invention, the X-axis is along the length direction of the welded workpiece, i.e., the welding trajectory direction; the Y-axis is along the width direction of the welded workpiece, i.e., perpendicular to the welding trajectory direction; and the Z-axis is perpendicular to the surface of the welded workpiece, representing the depth direction.

[0065] Regarding the construction of the reference coordinate system, the initial calibration parameters of the ultrasonic phased array device, such as the position and angle of the probe and the scanning start point, can be used to determine the relative spatial relationship between the ultrasonic acquisition device and the welding workpiece. It is important to note that the welding workpiece should be placed in a standard tooling fixture to ensure that the actual position of the weld area is consistent with the reference position of the device.

[0066] Specifically, starting with the first acquired ultrasonic image, the starting point of the weld area is first determined, and then mapped to the origin of the reference coordinate system according to the calibration parameters. Next, based on the device's scanning path (assuming a straight-line scanning path), the acquisition position of each image frame is calculated and mapped to its position coordinates in the reference coordinate system. It is important to note that because the multiple acquired images are from different perspectives, subsequent rotation and scaling operations are needed to align the weld area of ​​each image with the direction of the reference coordinate system.

[0067] The data preprocessing unit 20 extracts the boundary information of the weld region from the segmented and enhanced image, records the coordinates of the weld region in the reference coordinate system (e.g., pixel location), and binds the extracted coordinate information with the original image data to form a standardized data format. It is important to note that this standardized data includes the weld region location and timestamp for each frame of the image. Furthermore, considering that if multiple weld regions exist in the image, each region can be further labeled with a number to distinguish the characteristics of different welds.

[0068] In summary, it is understandable that the data preprocessing unit 20 transforms complex ultrasonic image data into high-quality, standardized weld feature data, laying the foundation for the computational operation of the entire system.

[0069] The task of feature extraction unit 30 is to extract spatial and temporal features of the weld region based on preprocessed multi-frame ultrasonic images and annotation information, and generate a global feature through multi-scale analysis. This global feature not only describes the spatial distribution of weld defects, but also captures the dynamic characteristics of the defect region over time, laying the foundation for subsequent defect classification and location identification.

[0070] Spatial feature extraction focuses on the weld region of a single-frame image, describing the geometry and physical properties of the defect by analyzing the reflection intensity distribution and edge contours. Temporal feature extraction is based on multi-frame ultrasonic images, capturing the regularity of dynamic changes in the weld region through time series analysis and temporal convolutional networks. These two types of features are integrated based on convolutional stitching and multi-scale analysis to ultimately generate global features containing both temporal and spatial information.

[0071] More specifically, the multi-frame ultrasonic images and their weld area annotation information output by the data preprocessing unit 20 are fed to the feature extraction unit 30 to extract the pixel grayscale values ​​within the weld area, thereby analyzing the pattern of reflection intensity distribution. Statistical methods, such as grayscale histograms, are then used to calculate the average intensity, peak intensity, and distribution range of the weld area, quantifying the reflection characteristics.

[0072] In some embodiments, edge detection algorithms, such as the Canny algorithm, can be used to obtain the edge morphology of the weld area. The length, width, curvature, and smoothness of the edge contour are derived and recorded based on the algorithm, further describing the geometric characteristics of the weld defect.

[0073] Subsequently, the feature extraction unit 30 extracts temporal features based on the weld region and its pixel value changes in multiple frames of ultrasonic images. Specifically, for the pixel value changes in the weld region, the rate of change of pixel grayscale values ​​over time can be calculated to reflect the dynamic change speed of the weld region's reflection intensity. The stability of the reflected signal is quantified by the amplitude of grayscale value fluctuations. Finally, the trajectory of the peak reflection intensity point over time is recorded to describe the dynamic changes in the weld defect morphology.

[0074] Based on this, in some embodiments of the present invention, by designing a temporal convolutional network (TCN), the convolutional kernel is used to perform sliding analysis on the features in the time dimension, thereby extracting the regularity of the weld area morphology changing over time. This can capture the dynamic pattern of defects within a specific time window, such as crack propagation or abrupt changes in reflection signals.

[0075] After extracting both spatial and temporal features, they are spliced ​​and integrated. (See reference [link to reference] for details.) Figure 3 The left side of the image shows multiple consecutively acquired ultrasonic images. These images record the ultrasonic reflection signals in the weld area, containing information on changes in both time and space. Each frame corresponds to an ultrasonic signal image at a different time point during the welding process, and each ultrasonic image shows the morphology and internal structure of the weld. Now, let's look at... Figure 3Taking three ultrasound images from a multi-frame ultrasound image dataset as an example, these ultrasound image data provide a time series, and each frame contains local spatial features. Among them, image one captures the beginning part of the weld, image two captures the middle part of the weld, which may contain defect areas, and image three captures the end part of the weld, where the overall shape of the weld gradually shrinks.

[0076] Regarding the spatial features in the image, as mentioned above, image processing techniques, such as Canny edge detection, are used to extract the morphological features of the weld edge, such as weld length, width, and boundary smoothness. The distribution of ultrasonic reflection intensity inside the weld is calculated, such as high-intensity areas, which indicate higher weld density, and low-intensity areas, which may be voids or cracks.

[0077] Regarding the temporal features in the figure, the trend of intensity change in the weld area in multiple frames of images is analyzed. As mentioned above, by comparing the change of reflection intensity at the same ultrasonic capture location over time, for example, abrupt changes in intensity can reflect abnormal fluctuations in weld quality. Then, statistical analysis is used, such as calculating its rate of change, or time series modeling methods such as RNN or TCN are used to extract dynamic features.

[0078] After extracting spatial and temporal features, these two types of features are embedded into the aforementioned 3D image. This feature embedding integrates the spatial and temporal features of the weld area into the 3D image, forming a multi-dimensional data representation. Specifically, the 3D contour information of the weld edge in the 3D image is extracted, and the edge morphology features are directly associated with the surface mesh of the 3D image. Then, the ultrasonic reflection intensity information of each frame is mapped to the corresponding points in the 3D point cloud and stored as an additional attribute of the point cloud. For example, different colors or textures can be used to visually display these additional attributes.

[0079] Embedding temporal features can also be understood as embedding the changing characteristics of a signal into a 3D image. By analyzing the temporal changes in ultrasonic reflection intensity, dynamic attributes are added to a 3D point cloud or its 3D surface mesh, such as the rate of intensity change at a specific point or the magnitude of that change. Through data structures, such as the feature tensor mentioned later, these spatial and temporal features are encoded into a unified 3D representation. This allows for direct input of the embedded multidimensional features as input when processing 3D images using convolutional neural networks, facilitating subsequent classification or analysis. This process preserves crucial spatial and temporal information.

[0080] Suppose that the weld area of ​​a welded component appears as a high-reflection area in ultrasonic images. Multiple frames of ultrasonic images show that the reflected signal changes with time in a certain pattern. In a single frame, the average reflection intensity of the weld area is approximately 150 grayscale values, the edge contour length is approximately 30 pixels, and the closure is 85%. Time series analysis shows that the trajectory of the peak reflection intensity point shifts 5 pixels to the right, with a rate of change of 2 grayscale values ​​per frame and an amplitude of 20 grayscale values. This allows a temporal convolutional network to capture the consistency between signal fluctuations and crack propagation patterns. In other words, in the 3D image after stitching and embedding features, the dynamic characteristics of the defect expanding from left to right can be displayed in the time dimension, while highlighting the integrity of the defect morphology in the spatial dimension.

[0081] In summary, it is understandable that the feature extraction unit 30, through advanced feature analysis and integration technology, provides high-quality, multi-dimensional data input for weld defect detection, ensuring the accuracy and robustness of the entire system.

[0082] Next is the system's fusion unit 40, which still refers to... Figure 3 The fusion unit 40 receives the target 3D image generated by the feature extraction unit 30 and uses a neural network to perform deep fusion of these features. The purpose is to integrate spatial features, temporal features, and their interaction information into a comprehensive feature tensor, providing input for the multi-task learning architecture of the classification unit 50.

[0083] The fusion unit 40 separates global features by channel; for example, spatial features and temporal features correspond to different feature channels. It's important to note that the features of each channel are first normalized to ensure a consistent numerical distribution, which is beneficial for subsequent fusion. Here, "channel" refers to each channel of the ultrasound matrix device that acquires ultrasound images.

[0084] A fusion neural network is used to deeply fuse features from different channels. The structure of the fusion network includes:

[0085] Convolutional layers: extract the interaction relationships between features;

[0086] Attention mechanism: Reinforce key features and suppress noise or redundant information;

[0087] Feature fusion layer: compresses multi-channel features into a unified comprehensive feature tensor.

[0088] The output comprehensive feature tensor contains spatial dimension, temporal dimension and inter-channel correlation information, forming a complete description of the weld area, and the comprehensive feature tensor is used as the input of classification unit 50.

[0089] Finally, there is classification unit 50, which can be found here. Figure 2The classification unit 50, through a multi-task learning architecture, completes the classification and localization of weld defects based on the comprehensive feature tensor output by the fusion unit 40.

[0090] Classification unit 50 first performs S501: inputting the spatial features, temporal features, and their corresponding fusion information from the comprehensive feature tensor into the shared feature extraction layer in the multi-task learning architecture to extract deep patterns. This can also be understood as mapping spatial features and temporal features to different feature channels. It's important to note that the features of each channel need to be normalized to ensure a consistent numerical distribution, which is beneficial for subsequent fusion. Here, "channel" refers to the various channels of the ultrasound matrix device that acquire ultrasound images.

[0091] Since the comprehensive feature tensor output by the fusion unit 40 contains deep spatial features, deep temporal features, and interactive information between spatial and temporal features, we still use a convolutional neural network to extract the deep patterns in the comprehensive feature tensor.

[0092] Then, S502 is performed: by analyzing the probability distribution of the defect type after analyzing the deep pattern through the fully connected layer and activation function in the multi-task learning architecture, the category corresponding to the highest probability is selected as the classification result.

[0093] Specifically, for the multidimensional structure of the comprehensive feature tensor, local patterns are extracted through multi-layer convolution operations, pooling layers are used to reduce feature dimensions while retaining key information, and finally, the output of the shared extraction layer is a deep pattern tensor, which contains the spatial and temporal characteristics of weld defects, as well as shared information for classification and localization tasks.

[0094] Based on this, the features complete the classification and localization tasks through the task branch layer. The classification task outputs the probability distribution of defect types through a fully connected layer and an activation function. Finally, the category with the highest probability is selected as the classification result.

[0095] Then, in step S503, the precise location coordinates of the defect are output using a regression method within the multi-task learning architecture. And in step S504, the classification unit 50 outputs the type, confidence level, and location of the weld defect within the weld area.

[0096] In some embodiments, the localization task outputs the precise location coordinates or region mask of the defect via regression, and optimizes the localization accuracy using mean squared error or IoU loss. Additionally, in some embodiments, non-maximum suppression is introduced to remove redundant overlapping boxes to ensure the uniqueness of the localization.

[0097] Finally, the classification unit 50 outputs the type, confidence level, and precise location of the weld defect in the weld area, completing the integrated detection of classification and location.

[0098] Here, we assume that the comprehensive feature tensor of a certain weld area contains the following information: its spatial features show that the average reflection intensity of the weld area is 140; its edge features show that there are cracks distributed; its temporal features show that the dynamic changes over time show that the crack edges are expanding; and the feature point trajectory offset rate is 3 pixels per second.

[0099] After processing by fusion unit 40, classification unit 50 outputs the defect category as "crack" with a confidence level of 88% based on the classification task. The location task outputs the defect location.

[0100] Finally, based on the dynamic analysis and location results of weld defects obtained from the aforementioned unit, the welding parameter optimization unit 60 is used to analyze the dynamic process of defect formation and optimize welding parameters to improve welding quality.

[0101] Specifically, the welding parameter optimization unit 60 receives defect information output by the classification unit 50, including defect type, location, and confidence level. It also receives the comprehensive feature tensor constructed by the fusion unit 40 to extract the dynamic change trajectory of the temporal features before and after defect formation. Furthermore, it receives real-time monitoring data of the current welding process parameters during the welding process, including heat input, welding speed, and shielding gas flow rate.

[0102] More specifically, the temporal features in the comprehensive feature tensor are first analyzed for their origin. Combined with the weld reference coordinate system, the temporal and spatial features are superimposed to construct a dynamic feature map of the defect formation process. Using a causal analysis algorithm and combining formation mechanism models for different defect types, the causal relationship between the current defect and the currently set welding process parameters, such as heat input, welding speed, and shielding gas flow rate, is derived. This can be understood as deducing the causes of defects based on the physical mechanisms of the welding process, such as heat conduction, material melting, and shielding gas flow corresponding to various welding process parameters. For example, excessively high welding speed in a certain section may lead to porosity, while insufficient heat input may result in incomplete penetration.

[0103] Then, a parameter optimization model is constructed using reinforcement learning algorithms. The training samples of this model are historical welding process parameters and corresponding historical defect causes. This model is trained to establish the correlation between historical welding process parameters and corresponding historical defect causes. In this way, after setting optimization objectives, such as reducing the probability of defect occurrence and improving welding quality, the real-time detection results of defects, that is, the current welding process parameters and the defect causes of the defects that have been caused, are used to iteratively calculate and generate welding parameter adjustment suggestions, including heat input, welding speed and shielding gas flow rate.

[0104] Finally, the optimized welding parameters are transmitted to the welding equipment via a feedback mechanism to update the parameter settings during the welding process. For example, the heat input is adjusted based on the real-time detected causes of defects to avoid similar porosity or crack defects, thereby achieving real-time adjustment of the welding process. The real-time effects of the adjusted welding parameters are then monitored, and verification and optimization are performed again by re-acquiring ultrasonic images to ensure that the welding quality meets expectations.

[0105] Integrating the above, based on the dynamic analysis and location results of weld defects from the aforementioned multiple units, and with the addition of the welding parameter optimization unit 60, a closed-loop control is achieved from weld defect detection and cause analysis to process parameter optimization. Especially in the automated welding scenarios of new energy vehicle components, this system can effectively capture the dynamic causes of welding defects, adjust welding parameters in real time, prevent the recurrence of defects, improve welding efficiency and quality, and significantly reduce production costs and rework rates.

[0106] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. The scope of patent protection of the present invention shall be determined by the claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention shall also be included within the scope of protection of the present invention.

Claims

1. An image recognition-based defect detection and analysis system, applicable to the identification and classification of weld defects in welded parts, characterized in that, include: The data acquisition unit uses an ultrasonic phased array device to continuously acquire multiple frames of ultrasonic images of the weld area. Each frame of the ultrasonic image contains the ultrasonic reflection characteristics of the weld area and the time dimension information of the acquisition time. The data preprocessing unit constructs a unified reference coordinate system based on the initial calibration parameters of the ultrasonic phased array device. After preprocessing each frame of the acquired ultrasonic image, the weld area is unified into the reference coordinate system. The feature extraction unit extracts spatial and temporal features of the weld area from multiple frames of ultrasonic images. The spatial features include the edge morphology and ultrasonic reflection intensity distribution of the weld area. The temporal features are extracted by sequentially extracting the changes in ultrasonic reflection signals in each frame of the ultrasonic image. A three-dimensional image of the weld area is constructed using image stitching technology, and the extracted spatial and temporal features of the weld area are embedded into the three-dimensional image. The fusion unit receives the three-dimensional image and uses a neural network to first decode the spatial and temporal features embedded in the three-dimensional image into feature tensors of corresponding scales, and then integrates all the feature tensors for convolution processing to generate a comprehensive feature tensor of a unified scale. The classification unit constructs a classification model based on a multi-task learning architecture, takes the comprehensive feature tensor as the input of the classification model, and outputs the type and confidence level of the defect and its position in the reference coordinate system. The welding parameter optimization unit combines the spatial and temporal characteristics of the defect, uses causal reasoning to analyze the cause of the defect, uses reinforcement learning algorithm to establish a correlation between historical welding process parameters and corresponding historical defect causes, and after setting the optimization objective, the reinforcement learning algorithm iteratively outputs the adjusted welding process parameters.

2. The image recognition-based defect detection and analysis system according to claim 1, characterized in that, The ultrasonic phased array device in the data acquisition unit consists of multiple independently controlled ultrasonic transducers. Each transducer can emit ultrasonic waves. By adjusting the timing of the transducer excitation, a controllable beam direction and focus are formed to receive ultrasonic reflected signals. By continuously acquiring multiple frames of the ultrasonic images, the ultrasonic reflection characteristics of the welded parts at different time points and in different ultrasonic emission spaces are captured, forming an ultrasonic image sequence with a time dimension.

3. The image recognition-based defect detection and analysis system according to claim 1, characterized in that, After receiving multiple frames of ultrasound images at each acquisition time transmitted by the data acquisition unit, the data preprocessing unit reads the timestamp information of each frame of ultrasound image to ensure the temporal consistency of the ultrasound image sequence. The weld seam area was initially delineated using a grayscale threshold segmentation algorithm, and noise was removed. After adjusting the image contrast, the coordinate information of the weld area is extracted, and the marked coordinate information is bound to the ultrasonic image data to form a standardized data format.

4. The image recognition-based defect detection and analysis system according to claim 3, characterized in that, The feature extraction unit, based on the multi-frame ultrasound images output by the data preprocessing unit and their labeled coordinate information, analyzes the weld area in each frame of the ultrasound image to obtain the reflection intensity distribution and edge contour information as the spatial features.

5. The image recognition-based defect detection and analysis system according to claim 4, characterized in that, The feature extraction unit performs time series analysis based on the pixel value changes in the weld area in the multi-frame ultrasonic images. By calculating the rate of change and amplitude change of the ultrasonic reflection signal, it quantifies the dynamic change of the ultrasonic reflection signal over time. Based on a temporal convolutional network, it captures the change law of the ultrasonic reflection signal over time, thereby extracting dynamic change features as the temporal features.

6. The image recognition-based defect detection and analysis system according to claim 1, characterized in that, The feature extraction unit integrates the spatial and temporal features through convolutional concatenation to form a feature matrix containing multi-scale information. Following the temporal sequence of the acquisition time, the feature matrices corresponding to multiple frames of ultrasound images are concatenated. For the concatenated feature matrix, the feature values ​​located at the same pixel position are accumulated to generate global features.

7. The image recognition-based defect detection and analysis system according to claim 6, characterized in that, The fusion unit receives the 3D image and global features transmitted by the feature extraction unit, extracts the deep features of the weld area using a convolutional neural network, and concatenates the tensors of the deep features and global features of the 3D image in the channel dimension to output a comprehensive feature tensor containing spatial and temporal information.

8. The image recognition-based defect detection and analysis system according to claim 7, characterized in that, The classification unit uses the comprehensive feature tensor input from the fusion unit to complete the classification and localization of weld defects using a pre-built multi-task learning architecture; specifically: The spatial features, temporal features, and their corresponding fusion information in the comprehensive feature tensor are input into the shared feature extraction layer in the multi-task learning architecture to extract deep patterns. By analyzing the probability distribution of the output defect type after analyzing the deep pattern using the fully connected layer and activation function in the multi-task learning architecture, the category corresponding to the highest probability is selected as the classification result, and the highest probability is used as the confidence level. The location coordinates of the defect are output using a regression approach within a multi-task learning architecture. The classification unit outputs the type of weld defect, its confidence level, and its location in the weld area.

9. A defect detection and analysis system based on image recognition according to any one of claims 1 to 8, characterized in that, The welding parameter optimization unit receives the defect type and location output by the classification unit, and combines the constructed reference coordinate system of the weld area with the changes in the ultrasonic reflection intensity of the weld area to construct a dynamic change feature map of the defect formation process. It also receives real-time monitoring data of the current welding process parameters during the welding process, including heat input, welding speed and shielding gas flow rate. Combining the formation mechanisms of different defect types, causal reasoning is used to analyze the cause of the defect based on the dynamic change characteristic spectrum and the current welding process parameters, that is, to analyze and deduce the causal relationship between the defect and the current welding process parameters; A parameter optimization model is constructed using a reinforcement learning algorithm. This model establishes the correlation between historical welding process parameters and the corresponding historical defect causes. The current welding process parameters and defect causes are used as inputs to the parameter optimization model, with defect minimization as the optimization objective. The adjusted welding process parameters are then iteratively output using the reinforcement learning algorithm.

Images