Welding defect identification method and system based on molten pool image

Abstract

The invention relates to the technical field of welding defect identification, and discloses a welding defect identification method and system based on a molten pool image. According to the method, a multispectral high-speed camera is used for collecting a molten pool dynamic image sequence in the welding process, after multi-scale morphological filtering preprocessing is conducted, a fusion feature vector is extracted through a depth separable convolutional network, and then the fusion feature vector is input into a defect classification model adopting a heterogeneous graph neural network architecture to obtain a defect probability distribution matrix. Then, constructing a dynamic sparse optimization model to position defects, generating a defect space coordinate set, and finally, outputting welding defect types and position information through hierarchical verification framework processing. According to the method, the problems of welding image noise interference, complex defect characteristics and the like are effectively solved, the accuracy and reliability of welding defect identification are improved, and powerful technical support is provided for welding quality control.

Description

Technical Field

[0001] The present invention relates to the technical field of welding defect recognition, and specifically to a welding defect recognition method and system based on molten pool images. Background Art

[0002] In modern industrial production, welding is an indispensable metal connection technology, which is widely used in many key fields such as aerospace, automobile manufacturing, shipbuilding, and machining. The quality of welding directly determines the safety, reliability, and service life of products, and the existence of welding defects is the core factor affecting welding quality.

[0003] Traditional welding defect detection methods have many limitations. Taking radiographic testing in non-destructive testing as an example, although it can detect internal defects relatively accurately, the equipment cost is high, the detection environment requirements are harsh, and the detection process involves radioactive substances, there are certain safety risks. At the same time, the professional skills and protective measures of the detection personnel are required to be extremely high. Although ultrasonic testing is relatively convenient, the qualitative and quantitative analysis of defects depends on the experience judgment of the detection personnel, and the accuracy is difficult to guarantee. Moreover, for defects with complex shapes and small sizes, it is easy to miss detection. And destructive testing, such as mechanical property tests, etc., although it can directly obtain the performance data of the welded joint, it will cause permanent damage to the welded part, is not suitable for finished product testing, and has low detection efficiency, and cannot meet the quality monitoring requirements of large-scale and continuous production.

[0004] With the development of computer vision and artificial intelligence technologies, image-based welding defect recognition methods have gradually emerged. However, in practical applications, these methods face numerous challenges. The welding process is a complex physical and chemical process, accompanied by strong arc light, spatter, smoke and other interference factors, which makes a large amount of noise exist in the collected molten pool images, seriously affecting the image quality and the extraction accuracy of defect features. In addition, there are many types of welding defects, such as pores, cracks, incomplete penetration, slag inclusions, etc. The manifestation forms of each defect in the image are complex and diverse, and under different welding processes, materials, and working conditions, the defect features vary significantly. This makes it difficult for a single feature extraction and classification algorithm to accurately identify various defects. Moreover, the existing technologies perform poorly in terms of the accuracy and real-time performance of defect location, cannot timely feedback the quality problems in the welding process, and are difficult to realize the real-time adjustment and optimization of the welding process.

[0005] Early image acquisition devices had low resolution and frame rate, unable to clearly capture the dynamic change details of the molten pool, which restricted the research on the formation process of welding defects. When traditional image processing algorithms were used to process molten pool images with complex backgrounds, the effect of noise suppression was limited, and it was easy to lose the key edge information of defects, resulting in deviations in subsequent feature extraction and analysis. In the field of machine learning, early classification models had low computational efficiency and insufficient generalization ability when facing large-scale and high-dimensional welding image data, and were difficult to adapt to the welding defect recognition tasks in different production environments. It was urgent to develop an efficient, accurate and highly adaptable welding defect recognition method and system, which had important practical significance for improving the welding quality of industrial products, ensuring production safety and reducing production costs. Summary of the Invention

[0006] The purpose of the present invention is to provide a welding defect recognition method and system based on molten pool images to solve the problems proposed in the above background technology.

[0007] To achieve the above purpose, the present invention provides the following technical solutions: A welding defect recognition method based on molten pool images, the method includes: Collecting a sequence of dynamic molten pool images during the welding process through a multi-spectral high-speed camera; Based on multi-scale morphological filtering, suppressing noise and enhancing edges for the sequence of dynamic molten pool images to generate preprocessed molten pool images; Constructing a depthwise separable convolutional network to perform multi-level feature extraction on the preprocessed molten pool images to generate a fusion feature vector containing texture, temperature gradient and geometric deformation; Inputting the fusion feature vector into a pre-trained defect classification model to obtain a defect probability distribution matrix; the defect classification model adopts a heterogeneous graph neural network architecture, and distinguishes the features of the molten pool area, heat affected zone and pore defects through a node type perception mechanism; Constructing a dynamic sparse optimization model, the dynamic sparse optimization model aims to minimize the feature redundancy and maximize the classification confidence, and uses an adaptive threshold screening algorithm to locate the key areas of the defect probability distribution matrix; generating a set of defect space coordinates based on the dynamic sparse optimization model; Constructing a hierarchical verification framework according to the set of defect space coordinates, the hierarchical verification framework includes a candidate layer, a refinement layer and a decision layer, where the candidate layer aggregates adjacent defect points through a region growing algorithm, the refinement layer uses a variational autoencoder to perform topological reconstruction on the defect morphology, and the decision layer fuses the multi-frame temporal consistency test based on a probability graph model, and finally outputs the welding defect type and location information.

[0008] Preferably, the multi-scale morphological filtering includes: Define a set of multi-structural elements, including circular, cross-shaped, and diamond-shaped kernels, where the kernel size is inversely proportional to the resolution of the molten pool image; Use a cascaded combination of opening and closing operations to iteratively filter the molten pool image, where the opening operation suppresses high-frequency noise and the closing operation fills in microscopic pore artifacts; Construct a morphological gradient map, extract the oscillation characteristics of the molten pool boundary through the difference operation between the original image and the filtered result, and superimpose it on the preprocessed molten pool image.

[0009] Preferably, the depthwise separable convolutional network adopts a dual-branch feature interaction structure, including: The first branch uses an atrous convolutional layer to extract the distribution characteristics of the large-scale temperature field of the molten pool, and the dilation rate increases with the network depth; The second branch uses a channel attention mechanism to enhance local texture detail features, and dynamically adjusts the feature channel weights through a squeeze-and-excitation module; Set up a cross-branch feature fusion layer, splice the output of the atrous convolution and the channel-weighted features tensorially, and generate the fused feature vector through 1×1 convolution for dimensionality reduction.

[0010] Preferably, the heterogeneous graph neural network architecture includes: Construct a molten pool-defect heterogeneous graph, where the nodes in the graph are divided into molten pool main nodes, heat conduction nodes, and defect candidate nodes, and the node attributes include grayscale value, gradient magnitude, and spectral energy; Adopt a type-aware graph attention mechanism to assign independent attention calculation paths for different node types, and introduce a two-way message passing channel between the molten pool nodes and the defect nodes; Perform hierarchical aggregation of the node features through graph pooling operations, use a soft mask mechanism to retain the high-confidence defect regions, and generate a defect probability distribution matrix.

[0011] Preferably, the adaptive threshold screening algorithm includes: Map the defect probability distribution matrix to a three-dimensional space probability field, where the dimensions include the horizontal coordinate, the vertical coordinate, and the probability density; Construct a dynamic energy function, which includes a regional connectivity constraint and a probability smoothing term, and use the simulated annealing algorithm to solve the minimum value of the energy function; Adaptively adjust the probability threshold according to the annealing temperature curve, and screen the set of defect coordinates whose connected region volume is greater than the preset threshold.

[0012] Preferably, the variational autoencoder performs topological reconstruction on the defect morphology as follows: The encoder uses a three-dimensional convolutional network to extract the voxelized representation of the defect, and the decoder uses transposed convolution and a spatial transformation network to generate a continuous surface mesh; Design a reconstruction loss function, which includes a voxel cross-entropy loss and a surface curvature regularization term; Introduce an adversarial training strategy to distinguish the real defect morphology from the reconstruction result through a discriminator network, and improve topological consistency.

[0013] Preferably, the calculation of the morphological gradient map further includes: Perform non-local mean filtering on the difference result to construct an adaptive weight matrix to suppress the pseudo-gradient response; Adopt Hessian matrix eigenvalue analysis to enhance the curvature feature of the molten pool boundary, and encode the maximum curvature direction into the channel dimension of the gradient map.

[0014] Preferably, the channel attention mechanism is implemented by a dynamic gating unit, including: Perform global average pooling and max pooling on the input feature map to generate a two-channel global description vector; Learn the temporal dependence relationship of the pooling result through a gated recurrent unit and output the channel weight coefficient; Perform element-wise multiplication of the weight coefficient and the original feature map to generate enhanced local texture features.

[0015] Preferably, the type-aware graph attention mechanism includes: Construct independent query-key-value pair generation networks for molten pool nodes and defect nodes respectively; During the message passing process, the molten pool nodes only receive the feature updates of the same type of nodes, and the defect nodes adopt cross-type attention to fuse the heat conduction path features; Stabilize the propagation process of the heterogeneous graph features through residual graph convolutional layers.

[0016] Preferably, the probability graph model fusion multi-frame temporal consistency check includes: Construct a hidden Markov model, use the single-frame defect detection result as the observation state, and the real existence of the defect as the hidden state; Decode the optimal state sequence through the Viterbi algorithm, and calculate the state transition probability by combining the overlap degree of the defect regions in adjacent frames; Adopt the expectation maximization algorithm to iteratively optimize the model parameters until the temporal consistency likelihood function converges.

[0017] Compared with the prior art, the beneficial effects of the present invention are: In the image acquisition stage, a multi-spectral high-speed camera is used to collect a dynamic image sequence of the molten pool. The multi-spectral characteristics enable the device to obtain molten pool information from different spectral bands, which contains richer details than a single-spectral image. For example, under a specific spectrum, pore defects may exhibit significantly different reflection or absorption characteristics from the surrounding area, making them easier to capture. The high-speed shooting function can record the instantaneous changes of the molten pool during the welding process, which are often closely related to the generation and development of defects. By obtaining these dynamic images, a comprehensive and accurate data basis is provided for subsequent analysis.

[0018] In the image preprocessing stage, multi-scale morphological filtering is adopted. It iteratively filters the image by defining a set of multi-structuring elements including circular, cross-shaped, and diamond-shaped kernels and combining the cascaded combination of opening operation and closing operation. The opening operation can effectively suppress high-frequency noise, which may come from arc light flickering or electronic device interference during the welding process, avoiding the interference of noise on subsequent analysis. The closing operation can fill the micro-pore artifacts, preventing misjudging the artifacts as real defects and improving the authenticity of the image. In addition, constructing a morphological gradient map and superimposing it on the preprocessed molten pool image further enhances the oscillation characteristics of the molten pool boundary, making the edges and details of the molten pool clearer and creating good conditions for subsequent feature extraction.

[0019] The dual-branch feature interaction structure of the depthwise separable convolutional network is ingeniously designed. The dilated convolutional layer in the first branch increases the dilation rate with the increase of network depth, which can gradually expand the receptive field and effectively extract the temperature field distribution characteristics of the large range of the molten pool. During the welding process, the change of the temperature field is closely related to the formation of defects. For example, uneven temperature distribution may lead to the generation of cracks. By accurately obtaining the temperature field characteristics, the potential risk of defects can be better judged. The second branch uses the channel attention mechanism to dynamically adjust the feature channel weights through the squeeze-and-excitation module, enhancing the local texture detail features. Welding defects often have unique manifestations in texture. For example, the texture at the crack will show discontinuous or abnormal trends. This mechanism can highlight these details and improve the recognition rate of defects. The cross-branch feature fusion layer fuses the two-way features to generate a fusion feature vector containing rich information such as texture, temperature gradient, and geometric deformation, providing more comprehensive and representative features for subsequent defect classification.

[0020] The defect classification model adopting the heterogeneous graph neural network architecture constructs a molten pool-defect heterogeneous graph and uses the node type perception mechanism to distinguish the molten pool area, heat-affected zone, and pore defect features. This model can fully consider the relationships and feature differences between different regions and has stronger adaptability and accuracy than traditional neural network models when dealing with complex welding images. It can more accurately judge the type and location of defects, reduce the situations of misjudgment and missed judgment, and provide a reliable basis for welding quality assessment.

[0021] The dynamic sparse optimization model aims to minimize feature redundancy and maximize classification confidence. It uses an adaptive threshold screening algorithm to locate key regions in the defect probability distribution matrix and generate a set of defect space coordinates. This algorithm can dynamically adjust the threshold according to the specific features of the image, avoiding the limitations of fixed thresholds under different welding conditions and improving the accuracy of defect location. Whether it is a tiny porosity defect or a large crack defect, it can be accurately located, providing precise position information for subsequent repair and quality improvement.

[0022] The hierarchical verification framework constructed based on the set of defect space coordinates includes a candidate layer, a refinement layer, and a decision layer. The candidate layer aggregates adjacent defect points through a region growing algorithm, which can connect scattered defect points into a complete defect region, avoiding fragmented understanding of defects. The refinement layer uses a variational autoencoder to perform topological reconstruction on the defect morphology. It can not only restore the true shape of the defect but also improve the topological consistency between the reconstruction result and the true defect morphology by designing a reconstruction loss function that includes a voxel cross-entropy loss and a surface curvature regularization term, as well as introducing an adversarial training strategy. This helps technicians more intuitively understand the actual morphology of the defect and judge its harm level. The decision layer fuses multi-frame temporal consistency verification based on a probability graph model. By constructing a hidden Markov model, taking the single-frame defect detection result as the observation state and the true existence of the defect as the hidden state, combining with the Viterbi algorithm to decode the optimal state sequence, and using the expectation-maximization algorithm to iteratively optimize the model parameters. This method makes full use of the temporal information of multi-frame images, overcomes the uncertainty of single-frame detection, improves the accuracy and stability of defect recognition, and can more reliably judge the true situation of the defect. Description of the Drawings

[0023] Figure 1 It is the working principle diagram of the welding defect recognition method based on the molten pool image described in the present invention; Figure 2 It is the flow chart of multi-scale morphological filtering processing; Figure 3 It is the flow chart of feature extraction by a depthwise separable convolutional network; Figure 4 It is the flow chart of defect classification by a heterogeneous graph neural network architecture. Detailed Embodiments

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0025] Please refer to Figures 1 - 4 , the present invention provides a welding defect recognition method based on molten pool images, and the specific steps are as follows: Collect a sequence of dynamic images of the molten pool during the welding process through a multi-spectral high-speed camera. In an actual welding scenario, the multi-spectral high-speed camera is installed at a suitable position to ensure that the dynamic changes of the welding molten pool can be clearly captured. Its parameters such as shooting frame rate and spectral range are reasonably set according to the welding process and defect recognition requirements to ensure that the obtained sequence of dynamic molten pool images can accurately reflect the real-time state of the molten pool and provide sufficient data for subsequent analysis.

[0026] Perform noise suppression and edge enhancement on the sequence of dynamic molten pool images based on multi-scale morphological filtering to generate a preprocessed molten pool image. Use multi-scale morphological filtering to process the collected sequence of dynamic molten pool images. This filtering process can effectively suppress the noise in the image and at the same time enhance the edge information of the molten pool, making the contour of the molten pool clearer and providing better image data for subsequent feature extraction and defect recognition.

[0027] Construct a depthwise separable convolutional network to perform multi-level feature extraction on the preprocessed molten pool image to generate a fusion feature vector containing texture, temperature gradient, and geometric deformation. The depthwise separable convolutional network is one of the key technologies of the present invention. Through its unique network structure, it can extract rich feature information from the preprocessed molten pool image. These feature information cover multiple aspects such as the texture, temperature gradient, and geometric deformation of the molten pool, laying a solid foundation for accurately identifying welding defects in the future.

[0028] Input the fusion feature vector into a pre-trained defect classification model to obtain a defect probability distribution matrix; the defect classification model adopts a heterogeneous graph neural network architecture and distinguishes the features of the molten pool area, heat affected zone, and pore defects through a node type perception mechanism. The pre-trained defect classification model is constructed based on a heterogeneous graph neural network architecture. This model can accurately distinguish the features of the molten pool area, heat affected zone, and pore defects through a node type perception mechanism. When the fusion feature vector is input into this model, the model will output a defect probability distribution matrix, which reflects the likelihood of different regions in the image having defects.

[0029] Construct a dynamic sparse optimization model. The dynamic sparse optimization model aims to minimize feature redundancy and maximize classification confidence. It uses an adaptive threshold screening algorithm to locate key regions in the defect probability distribution matrix; generate a set of defect space coordinates based on the dynamic sparse optimization model. The dynamic sparse optimization model aims to minimize feature redundancy and maximize classification confidence, and uses an adaptive threshold screening algorithm to process the defect probability distribution matrix, thereby achieving accurate positioning of the key regions of the defect and generating a set of defect space coordinates. This set clarifies the specific position information of the defect in the image in space, providing an important basis for further analyzing the type and severity of the defect in the follow-up.

[0030] Construct a hierarchical verification framework according to the set of defect space coordinates. The hierarchical verification framework includes a candidate layer, a refinement layer, and a determination layer. Among them, the candidate layer aggregates adjacent defect points through a region growing algorithm, the refinement layer uses a variational autoencoder to perform topological reconstruction on the defect morphology, and the determination layer fuses multi-frame temporal consistency tests based on a probability graph model, and finally outputs the type and position information of the welding defect. Construct a hierarchical verification framework according to the generated set of defect space coordinates. This framework includes a candidate layer, a refinement layer, and a determination layer. Through the collaborative work of these three layers, the type and position information of the defect can be accurately judged and output, greatly improving the accuracy and reliability of welding defect recognition.

[0031] The present invention will be further described below in conjunction with Embodiments 1 to 6:

[0032] Embodiment 1: In the image preprocessing stage, multi-scale morphological filtering first defines a multi-structuring element set, which includes circular, cross-shaped, and diamond-shaped kernels. The sizes of these kernels are inversely proportional to the resolution of the molten pool image. This is because for an image with a higher resolution, its detailed information is rich, and smaller-sized kernels are required for fine processing; while for an image with a lower resolution, larger-sized kernels can better capture the overall features. For example, if the resolution of the molten pool image is 1920×1080, the radius of the circular kernel may be set to 3 pixels, the cross length of the cross-shaped kernel is 5 pixels, and the side length of the diamond-shaped kernel is 4 pixels.

[0033] Use a cascaded combination of opening and closing operations to perform iterative filtering on the molten pool image. The opening operation first performs an erosion operation on the image and then a dilation operation, which can effectively suppress high-frequency noise. Suppose there are some isolated bright spot noises in the image generated by spatter during the welding process. After the opening operation, these small bright spot noises will be removed. The closing operation is to dilate first and then erode, and is used to fill micro-porosity artifacts. During the welding process, due to reasons such as gas escape, some tiny porosity artifacts may be formed in the image. The closing operation can fill these small holes and make the image smoother.

[0034] When constructing the morphological gradient map, the oscillation characteristics of the molten pool boundary are first extracted through the differential operation between the original image and the filtering result. To further optimize the quality of the morphological gradient map, non-local means filtering is performed on the differential result. Non-local means filtering suppresses the pseudo-gradient response by constructing an adaptive weight matrix. Its principle is to find pixel blocks similar to the current pixel in the image, assign weights to these pixel blocks according to the degree of similarity, and then use these weights to filter the current pixel. Let the current pixel be and in the neighborhood centered at , the weight of pixel is calculated by the formula:

[0035] where is the normalization constant to ensure ; and are the pixel values of the original image and the filtering result at position respectively; is the filtering parameter used to control the filtering intensity, and the larger is, the smoother the filtering effect.

[0036] Then, Hessian matrix eigenvalue analysis is used to enhance the curvature characteristics of the molten pool boundary. The Hessian matrix can describe the second-order derivative information of the image at a certain point. By calculating the eigenvalues of the Hessian matrix, the curvature information of the image in different directions at this point can be obtained. The maximum curvature direction is encoded into the channel dimension of the gradient map, which can highlight the bending degree of the molten pool boundary and provide more accurate edge information for subsequent defect recognition. For example, in the area where the molten pool boundary shows obvious bending, the characteristics of this area can be enhanced through Hessian matrix analysis, making it easier to identify potential defects in subsequent processing. After the above processing, the morphological gradient map is superimposed on the preprocessed molten pool image to complete the image preprocessing work.

[0037] Example 2: During the feature extraction process of the depthwise separable convolutional network, a dual-branch feature interaction structure is adopted.

[0038] The first branch uses an atrous convolutional layer to extract the distribution characteristics of the large-scale temperature field of the molten pool. The atrous convolutional layer expands the receptive field of the convolution by introducing holes in the convolutional kernel. The atrous rate increases with the network depth. For example, in the shallow layer of the network, the atrous rate is set to 1, and as the number of network layers increases, the atrous rates are set to 2, 3, etc. in sequence. In this way, at different depths of the network, temperature field distribution characteristics of different scales can be obtained. Suppose in a certain layer of atrous convolution, the convolutional kernel size is 3×3 and the atrous rate is 2, then the actual receptive field size is 7×7, which can capture a larger range of temperature information.

[0039] The second branch uses a channel attention mechanism to enhance local texture detail features and dynamically adjusts the feature channel weights through a squeeze-and-excitation module. The channel attention mechanism is implemented using a dynamic gating unit, and the specific steps are as follows: First, global average pooling and max pooling are performed on the input feature map to obtain an average pooling feature map and a max pooling feature map respectively. These two feature maps are then subjected to global average pooling and max pooling operations respectively to generate a two-channel global description vector. Assuming the size of the input feature map is , after global average pooling, an average pooling feature map of size is obtained; after max pooling, a max pooling feature map of the same size is obtained. These two feature maps are concatenated together to obtain a two-channel global description vector.

[0040] Then, a gated recurrent unit is used to learn the temporal dependence relationship of the pooling results and output the channel weight coefficients. The gated recurrent unit can adaptively update the memory state according to the input information, thereby learning the dependence relationship between different channels. Finally, the weight coefficients are multiplied with the original feature map channel by channel to generate enhanced local texture features. In this way, the channel attention mechanism can weight the feature map according to the importance of different channels, highlighting local texture detail features.

[0041] A cross-branch feature fusion layer is set up to perform tensor concatenation on the output of the dilated convolution and the channel-weighted features, and generate a fused feature vector through 1×1 convolution for dimensionality reduction. Tensor concatenation combines the features of the two branches in the channel dimension. Assuming the size of the output feature map of the dilated convolution is , and the size of the channel-weighted feature map is , the size of the concatenated feature map is . Then, through 1×1 convolution, the dimensionality of the concatenated feature map is reduced to generate the final fused feature vector, which contains the temperature field distribution features and local texture detail features of the molten pool, providing rich information for subsequent defect classification.

[0042] Example 3: The heterogeneous graph neural network architecture adopted by the defect classification model includes the following key steps: Construct a molten pool-defect heterogeneous graph, where the nodes in the graph are divided into molten pool main nodes, heat conduction nodes, and defect candidate nodes. Each node has rich attributes, including gray value, gradient magnitude, and spectral energy. The gray value reflects the brightness information of the area where the node is located, the gradient magnitude can represent the edge strength of the area, and the spectral energy reflects the energy distribution of the image at different frequencies. For example, in the edge area of the molten pool, the gradient magnitude is relatively large, indicating that there are obvious edges in this area; while in the uniform area inside the molten pool, the gray value changes little and the spectral energy is relatively low.

[0043] The type-aware graph attention mechanism is adopted to assign independent attention calculation paths to different node types. Specifically, independent query-key-value pair generation networks are constructed for the molten pool nodes and defect nodes respectively. During the message passing process, the molten pool nodes only receive the feature updates of the same type of nodes, which can maintain the consistency and stability of the features in the molten pool area. For example, the nodes inside the molten pool pass information to each other, which can better reflect the overall features of the molten pool. The defect nodes adopt a cross-type attention fusion heat conduction path feature, because the generation of defects is often closely related to the heat conduction process. By fusing the heat conduction path features, defects can be more accurately identified.

[0044] During the heterogeneous graph feature propagation process, the residual graph convolutional layer is used to stabilize the propagation process of the heterogeneous graph features. The residual graph convolutional layer can effectively avoid the problem of gradient disappearance, enabling the network to better learn the deep features of the heterogeneous graph. Let the input feature be , after the transformation of the residual graph convolutional layer, the output feature is calculated by the formula:

[0045] where, is the adjacency matrix of the heterogeneous graph, representing the connection relationship between nodes; is the weight matrix, used to learn the transformation of features; is the activation function, such as the ReLU function. In this way, the heterogeneous graph neural network can effectively learn the features of the molten pool area, heat affected zone and pore defects.

[0046] Finally, the node features are hierarchically aggregated through the graph pooling operation, and the soft mask mechanism is adopted to retain the high-confidence defect areas and generate the defect probability distribution matrix. The graph pooling operation can reduce the scale of the graph while retaining the important feature information. The soft mask mechanism screens the defect areas according to the confidence of the nodes, retains the areas with higher confidence, and the generated defect probability distribution matrix can intuitively reflect the possibility of defects in different areas of the image.

[0047] Example 4:

[0048] During the key area positioning and defect space coordinate generation process, an adaptive threshold screening algorithm is adopted. First, the defect probability distribution matrix is mapped into a three-dimensional space probability field, and its dimensions include the horizontal coordinate, vertical coordinate and probability density. In this way, the two-dimensional probability distribution matrix can be extended to the three-dimensional space to more intuitively represent the position and probability information of the defects in the image.

[0049] Construct a dynamic energy function, which includes regional connectivity constraints and a probability smoothing term. The regional connectivity constraints ensure that the selected defect regions are continuous, avoiding isolated noise points being misjudged as defects. The probability smoothing term makes the probability distribution smoother and reduces fluctuations. Let the dynamic energy function be , and its calculation formula is:

[0050] where and are weight coefficients used to balance the importance of regional connectivity constraints and the probability smoothing term; is the set of all pixel points, is the pixel point 's neighborhood; and are the probability values of pixel points and respectively; is the set of all connected regions, is the perimeter of the connected region . The larger the perimeter, the more irregular the region. By penalizing regions with larger perimeters, the selected defect regions can be made more compact.

[0051] The simulated annealing algorithm is used to solve the minimum value of the energy function. The simulated annealing algorithm is a heuristic optimization algorithm that simulates the physical annealing process and gradually reduces the temperature during the search process to avoid falling into local optimal solutions. During the solution process, the probability threshold is adaptively adjusted according to the annealing temperature curve. As the temperature decreases, the probability threshold gradually decreases, which can search for possible defect regions more widely in the initial stage of the search and gradually focus on high-confidence defect regions in the later stage of the search. When the algorithm converges, filter the set of defect coordinates whose connected region volume is greater than the preset threshold, and this preset threshold can be adjusted according to the actual welding process and defect detection requirements.

[0052] Example 5: In the refinement layer of the hierarchical verification framework, a variational autoencoder is used to perform topological reconstruction on the defect morphology.

[0053] The encoder uses a three-dimensional convolutional network to extract the voxelized representation of the defect. The three-dimensional convolutional network can perform convolutional operations on the defect data in three-dimensional space to extract the spatial features of the defect.

[0054] The decoder uses transposed convolution and a spatial transformation network to generate a continuous surface mesh. The transposed convolution operation can upsample the feature map extracted by the encoder to restore it to the size of the original data. The spatial transformation network can perform spatial transformation on the result after transposed convolution to make it more conform to the morphology of the real defect.

[0055] Design a reconstruction loss function that includes voxel cross-entropy loss and surface curvature regularization term. The voxel cross-entropy loss is used to measure the difference between the reconstruction result and the true defect voxels, and its calculation formula is:

[0056] where, is the total number of voxels, is the label (0 or 1) of the true defect voxel at position , is the predicted probability of the reconstruction result at position . The surface curvature regularization term is used to make the reconstructed defect surface smoother and conform to real physical laws. Let the curvature of the surface be , and the calculation formula of the surface curvature regularization term is:

[0057] where, represents the set of all points on the reconstructed surface, is the weight of point , which is assigned according to the position and importance of the point on the surface.

[0058] Introduce an adversarial training strategy. The discriminator network is used to distinguish the true defect morphology from the reconstruction result to improve topological consistency. The discriminator network conducts adversarial training with the generator (i.e., the encoder and decoder) of the variational autoencoder. The generator attempts to generate a reconstruction result closer to the true defect morphology, while the discriminator attempts to distinguish between the true defect and the reconstruction result. During the training process, the parameters of the generator and the discriminator are continuously adjusted until the two reach a balance. At this time, the generated reconstruction result is more similar to the true defect in topological structure and can more accurately reflect the morphology of the defect.

[0059] Example 6: In the decision layer of the hierarchical verification framework, fuse multi-frame temporal consistency checks based on a probabilistic graphical model.

[0060] Construct a hidden Markov model, taking the single-frame defect detection result as the observation state and the true existence of the defect as the hidden state. The hidden Markov model is a statistical model that assumes that the state of the system is not directly observable (hidden state), but the hidden state can be inferred through the observed signal (observation state). In welding defect detection, the defect detection result of each frame of image is the observation state, while whether there is actually a defect is the hidden state.

[0061] Decode the optimal state sequence through the Viterbi algorithm, and calculate the state transition probability by combining the overlap degree of the defect regions in adjacent frames. The Viterbi algorithm is a dynamic programming algorithm used to find the optimal state sequence in a hidden Markov model. Let the state set of the hidden Markov model be , the observation sequence be , the state transition probability matrix be , the observation probability matrix be , and the initial state probability vector be . The Viterbi algorithm finds the optimal state sequence by calculating the maximum probability path for each state at each time step. When calculating the state transition probability, the overlap degree of the defect regions in adjacent frames is combined. If the overlap degree of the defect regions in adjacent frames is high, it indicates that the defect is temporally continuous and the state transition probability is large; conversely, if the overlap degree is low, the state transition probability is small.

[0062] Use the expectation-maximization algorithm to iteratively optimize the model parameters until the temporal consistency likelihood function converges. The expectation-maximization algorithm is an iterative algorithm used to estimate the parameters of a model containing hidden variables. In a hidden Markov model, the hidden state is the hidden variable. By continuously iterating and calculating the expectation step (E-step) and the maximization step (M-step), the parameters of the model are updated, making the temporal consistency likelihood function increase continuously until convergence. In the expectation step, the posterior probability of the hidden state is calculated according to the current model parameters; in the maximization step, the model parameters are updated by maximizing the likelihood function based on the posterior probability obtained in the expectation step. After multiple iterations, the hidden Markov model can more accurately fuse multi-frame temporal information, improve the accuracy of welding defect recognition, and finally output more reliable welding defect type and location information.

[0063] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0064] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A welding defect recognition method based on molten pool images, characterized in that Including: Collecting a sequence of dynamic images of the molten pool during the welding process through a multi-spectral high-speed camera; Performing noise suppression and edge enhancement on the sequence of dynamic images of the molten pool based on multi-scale morphological filtering to generate a preprocessed molten pool image; Constructing a depthwise separable convolutional network to perform multi-level feature extraction on the preprocessed molten pool image to generate a fused feature vector including texture, temperature gradient, and geometric deformation; Inputting the fused feature vector into a pre-trained defect classification model to obtain a defect probability distribution matrix; the defect classification model adopts a heterogeneous graph neural network architecture and distinguishes the features of the molten pool area, heat affected zone, and pore defects through a node type perception mechanism; Constructing a dynamic sparse optimization model, which aims to minimize feature redundancy and maximize classification confidence, and uses an adaptive threshold screening algorithm to locate key regions in the defect probability distribution matrix; generating a set of defect space coordinates based on the dynamic sparse optimization model; Constructing a hierarchical verification framework according to the set of defect space coordinates, the hierarchical verification framework includes a candidate layer, a refinement layer, and a decision layer. Among them, the candidate layer aggregates adjacent defect points through a region growing algorithm, the refinement layer reconstructs the defect morphology topologically using a variational autoencoder, and the decision layer fuses the multi-frame temporal consistency test based on a probability graph model, and finally outputs the type and location information of the welding defect.

2. The welding defect identification method according to claim 1, characterized in that The multi-scale morphological filtering includes: Defining a set of multi-structuring elements, including circular, cross-shaped, and diamond-shaped kernels, and the kernel size is inversely proportional to the resolution of the molten pool image; Performing iterative filtering on the molten pool image using a cascaded combination of opening operation and closing operation, where the opening operation suppresses high-frequency noise and the closing operation fills the micro-pore artifacts; Constructing a morphological gradient map, extracting the oscillation characteristics of the molten pool boundary through the differential operation between the original image and the filtering result, and superimposing it on the preprocessed molten pool image.

3. The welding defect identification method according to claim 1, wherein, The depthwise separable convolutional network adopts a dual-branch feature interaction structure, including: The first branch uses an atrous convolutional layer to extract the distribution characteristics of the large-scale temperature field of the molten pool, and the atrous rate increases with the network depth; The second branch uses a channel attention mechanism to enhance local texture detail features and dynamically adjusts the feature channel weights through a squeeze-and-excitation module; Setting a cross-branch feature fusion layer, splicing the atrous convolutional output and the channel-weighted features tensorially, and reducing the dimension through a 1×1 convolution to generate the fused feature vector.

4. The welding defect identification method according to claim 1, characterized in that The heterogeneous graph neural network architecture includes: Constructing a molten pool-defect heterogeneous graph, where the nodes in the graph are divided into molten pool main nodes, heat conduction nodes, and defect candidate nodes, and the node attributes include gray value, gradient magnitude, and spectral energy; Adopting a type-aware graph attention mechanism to assign independent attention calculation paths to different node types, and introducing a bidirectional message passing channel between the molten pool node and the defect node; Performing hierarchical aggregation on the node features through graph pooling operations, using a soft mask mechanism to retain high-confidence defect regions, and generating a defect probability distribution matrix.

5. The welding defect identification method according to claim 1, wherein The adaptive threshold screening algorithm includes: Mapping the defect probability distribution matrix into a three-dimensional space probability field, and the dimensions include horizontal coordinate, vertical coordinate, and probability density; Construct a dynamic energy function, which includes regional connectivity constraints and a probability smoothing term, and use the simulated annealing algorithm to solve the minimum value of the energy function; Adaptively adjust the probability threshold according to the annealing temperature curve, and screen the set of defect coordinates whose connected region volume is greater than the preset threshold.

6. The welding defect identification method according to claim 1, characterized in that The variational autoencoder performs topological reconstruction on the defect morphology, including: The encoder uses a three-dimensional convolutional network to extract the voxelized representation of the defect, and the decoder uses a transposed convolution and a spatial transformation network to generate a continuous surface mesh; Design a reconstruction loss function, which includes voxel cross-entropy loss and surface curvature regularization terms; Introduce an adversarial training strategy, and use a discriminator network to distinguish the true defect morphology from the reconstruction result to improve topological consistency.

7. The welding defect identification method according to claim 2, wherein The calculation of the morphological gradient map further includes: Perform non-local mean filtering on the difference result to construct an adaptive weight matrix to suppress the pseudo-gradient response; Use Hessian matrix eigenvalue analysis to enhance the curvature features of the molten pool boundary, and encode the maximum curvature direction into the channel dimension of the gradient map.

8. The welding defect recognition method according to claim 3, characterized in that The channel attention mechanism is implemented using a dynamic gating unit, including: Perform global average pooling and max pooling on the input feature map to generate a two-channel global description vector; Learn the temporal dependence relationship of the pooling results through a gated recurrent unit and output the channel weight coefficients; Multiply the weight coefficients with the original feature map channel by channel to generate enhanced local texture features.

9. The welding defect identification method according to claim 4, characterized in that The type-aware graph attention mechanism includes: Construct independent query-key-value pair generation networks for the molten pool nodes and the defect nodes respectively; During the message passing process, the molten pool nodes only receive the feature updates of the same type of nodes, and the defect nodes use cross-type attention to fuse the heat conduction path features; Stabilize the propagation process of the heterogeneous graph features through residual graph convolutional layers.

10. The welding defect identification method according to claim 1, characterized in that, The probability graph model fusion multi-frame temporal consistency check includes: Construct a hidden Markov model, use the single-frame defect detection result as the observation state, and the true existence of the defect as the hidden state; Decode the optimal state sequence through the Viterbi algorithm, and calculate the state transition probability by combining the overlap degree of the defect regions in adjacent frames; Use the expectation maximization algorithm to iteratively optimize the model parameters until the temporal consistency likelihood function converges.

Images