Welding Defect Recognition Method and System Based on Molten Pool Image

Abstract

The present invention relates to the technical field of welding defect recognition, and discloses a welding defect recognition method and system based on molten pool images. The method first uses a multi-spectral high-speed camera to collect a dynamic image sequence of the molten pool during the welding process. After preprocessing by multi-scale morphological filtering, it extracts a fused feature vector through a depthwise separable convolutional network, and then inputs it into a defect classification model adopting a heterogeneous graph neural network architecture to obtain a defect probability distribution matrix. Then, a dynamic sparse optimization model is constructed to locate the defects and generate a set of defect spatial coordinates. Finally, after being processed by a hierarchical verification framework, the welding defect type and position information are output. The present invention effectively overcomes problems such as welding image noise interference and complex defect features, improves the accuracy and reliability of welding defect recognition, and provides strong technical support for welding quality control.

Description

Technical Field

[0001] The present invention relates to the technical field of welding defect recognition, and particularly to a welding defect recognition method and system based on molten pool images. Background Art

[0002] In modern industrial production, welding is an indispensable metal connection technology, which is widely used in many key fields such as aerospace, automotive manufacturing, shipbuilding, and machining. The quality of welding directly determines the safety, reliability, and service life of products, and the existence of welding defects is the core factor affecting welding quality.

[0003] Traditional welding defect detection means have many limitations. Taking radiographic testing in non-destructive testing as an example, although it can accurately detect internal defects, the equipment cost is high, the detection environment requirements are harsh, the detection process involves radioactive substances, there are certain safety risks, and the professional skills and protective measures of the detection personnel are required to be extremely high. Ultrasonic testing is relatively convenient, but the qualitative and quantitative analysis of defects depends on the experience judgment of the detection personnel, the accuracy is difficult to guarantee, and it is easy to miss small and complex-shaped defects. And destructive testing, such as mechanical property tests, etc., although it can directly obtain the performance data of the welded joint, it will cause permanent damage to the welded part, is not suitable for finished product testing, and has low detection efficiency and cannot meet the quality monitoring requirements of large-scale and continuous production.

[0004] With the development of computer vision and artificial intelligence technologies, image-based welding defect recognition methods have gradually emerged. However, in practical applications, these methods face numerous challenges. The welding process is a complex physical and chemical process, accompanied by strong arc light, spatter, smoke and other interference factors, which makes a large amount of noise exist in the collected molten pool images, seriously affecting the image quality and the extraction accuracy of defect features. In addition, there are many types of welding defects, such as pores, cracks, lack of penetration, slag inclusions, etc. The manifestation forms of each defect in the image are complex and diverse, and the defect features vary significantly under different welding processes, materials, and working conditions, which makes it difficult for a single feature extraction and classification algorithm to accurately identify various defects. Moreover, the existing technologies perform poorly in terms of the accuracy and real-time performance of defect location, cannot timely feedback the quality problems in the welding process, and are difficult to realize the real-time adjustment and optimization of the welding process.

[0005] Early image acquisition devices had low resolution and frame rate, unable to clearly capture the dynamic change details of the molten pool, which restricted the research on the formation process of welding defects. When traditional image processing algorithms were used to process molten pool images under complex backgrounds, the effect of noise suppression was limited, and the key edge information of defects was easily lost, resulting in deviations in subsequent feature extraction and analysis. In the field of machine learning, early classification models had low computational efficiency and insufficient generalization ability when facing large-scale and high-dimensional welding image data, and were difficult to adapt to the welding defect recognition tasks under different production environments. It was urgent to develop an efficient, accurate and highly adaptable welding defect recognition method and system, which had important practical significance for improving the welding quality of industrial products, ensuring production safety and reducing production costs. Summary of the Invention

[0006] The purpose of the present invention is to provide a welding defect recognition method and system based on molten pool images to solve the problems mentioned in the above background technology.

[0007] To achieve the above purpose, the present invention provides the following technical solution: A welding defect recognition method based on molten pool images, the method includes:

[0008] Collect a dynamic image sequence of the molten pool during the welding process through a multi-spectral high-speed camera;

[0009] Based on multi-scale morphological filtering, perform noise suppression and edge enhancement on the dynamic image sequence of the molten pool to generate a preprocessed molten pool image;

[0010] Construct a depthwise separable convolutional network to perform multi-level feature extraction on the preprocessed molten pool image to generate a fusion feature vector including texture, temperature gradient and geometric deformation;

[0011] Input the fusion feature vector into a pre-trained defect classification model to obtain a defect probability distribution matrix; the defect classification model adopts a heterogeneous graph neural network architecture, and distinguishes the features of the molten pool area, heat affected zone and pore defects through a node type perception mechanism;

[0012] Construct a dynamic sparse optimization model, the dynamic sparse optimization model aims to minimize the feature redundancy and maximize the classification confidence, and uses an adaptive threshold screening algorithm to locate the key areas of the defect probability distribution matrix; generate a set of defect space coordinates based on the dynamic sparse optimization model;

[0013] Construct a hierarchical verification framework according to the set of defect space coordinates, the hierarchical verification framework includes a candidate layer, a refinement layer and a decision layer, where the candidate layer aggregates adjacent defect points through a region growing algorithm, the refinement layer uses a variational autoencoder to perform topological reconstruction on the defect morphology, and the decision layer fuses the multi-frame temporal consistency test based on a probability graph model, and finally outputs the type and location information of the welding defect.

[0014] Preferably, the multi-scale morphological filtering includes:

[0015] Define a multi-structuring element set, including circular, cross-shaped, and diamond-shaped kernels, and the kernel size is inversely proportional to the resolution of the molten pool image;

[0016] Use a cascaded combination of opening and closing operations to iteratively filter the molten pool image, where the opening operation suppresses high-frequency noise and the closing operation fills the microscopic pore artifacts;

[0017] Construct a morphological gradient map, extract the molten pool boundary oscillation characteristics through the difference operation between the original image and the filtering result, and superimpose it on the preprocessed molten pool image.

[0018] Preferably, the depthwise separable convolutional network adopts a dual-branch feature interaction structure, including:

[0019] The first branch uses an atrous convolutional layer to extract the characteristics of the large-scale temperature field distribution of the molten pool, and the dilation rate increases with the network depth;

[0020] The second branch uses a channel attention mechanism to enhance the local texture detail features, and dynamically adjusts the feature channel weights through a squeeze-and-excitation module;

[0021] Set a cross-branch feature fusion layer, splice the atrous convolution output and the channel-weighted features tensorially, and generate the fusion feature vector through 1×1 convolution for dimensionality reduction.

[0022] Preferably, the heterogeneous graph neural network architecture includes:

[0023] Construct a molten pool-defect heterogeneous graph, where the nodes in the graph are divided into molten pool main nodes, heat conduction nodes, and defect candidate nodes, and the node attributes include grayscale value, gradient magnitude, and spectral energy;

[0024] Adopt a type-aware graph attention mechanism to assign independent attention calculation paths for different node types, and introduce a bidirectional message passing channel between the molten pool nodes and the defect nodes;

[0025] Hierarchically aggregate the node features through graph pooling operations, use a soft mask mechanism to retain the high-confidence defect regions, and generate a defect probability distribution matrix.

[0026] Preferably, the adaptive threshold screening algorithm includes:

[0027] Map the defect probability distribution matrix to a three-dimensional space probability field, and the dimensions include horizontal coordinates, vertical coordinates, and probability density;

[0028] Construct a dynamic energy function, which includes regional connectivity constraints and probability smoothing terms, and use the simulated annealing algorithm to solve the minimum value of the energy function;

[0029] Adaptive adjustment of the probability threshold according to the annealing temperature curve to screen the set of defect coordinates where the volume of the connected region is greater than the preset threshold.

[0030] Preferably, the variational autoencoder for topologically reconstructing the defect morphology includes:

[0031] The encoder uses a three-dimensional convolutional network to extract the voxelized representation of the defect, and the decoder uses transposed convolution and a spatial transformation network to generate a continuous surface mesh;

[0032] Design a reconstruction loss function, including voxel cross-entropy loss and surface curvature regularization term;

[0033] Introduce an adversarial training strategy to distinguish the real defect morphology from the reconstruction result through a discriminator network, and improve the topological consistency.

[0034] Preferably, the calculation of the morphological gradient map further includes:

[0035] Perform non-local mean filtering on the difference result to construct an adaptive weight matrix to suppress the pseudo-gradient response;

[0036] Use Hessian matrix eigenvalue analysis to enhance the curvature feature of the molten pool boundary, and encode the maximum curvature direction into the channel dimension of the gradient map.

[0037] Preferably, the channel attention mechanism is implemented using a dynamic gating unit, including:

[0038] Perform global average pooling and max pooling on the input feature map to generate a two-channel global description vector;

[0039] Learn the temporal dependence relationship of the pooling results through a gated recurrent unit and output the channel weight coefficients;

[0040] Multiply the weight coefficients and the original feature map channel by channel to generate enhanced local texture features.

[0041] Preferably, the type-aware graph attention mechanism includes:

[0042] Construct independent query-key-value pair generation networks for molten pool nodes and defect nodes respectively;

[0043] During the message passing process, the molten pool nodes only receive the feature updates of nodes of the same type, and the defect nodes use cross-type attention to fuse the heat conduction path features;

[0044] Stabilize the propagation process of heterogeneous graph features through residual graph convolutional layers.

[0045] Preferably, the probability graph model fusion multi-frame temporal consistency check includes:

[0046] Build a Hidden Markov Model, taking the single-frame defect detection result as the observation state and the true existence of the defect as the hidden state;

[0047] Decode the optimal state sequence through the Viterbi algorithm, and calculate the state transition probability by combining the overlap degree of the defect regions in adjacent frames;

[0048] Adopt the Expectation-Maximization algorithm to iteratively optimize the model parameters until the temporal consistency likelihood function converges.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] In the image acquisition stage, a multi-spectral high-speed camera is used to collect a dynamic image sequence of the molten pool. The multi-spectral characteristics enable the device to obtain molten pool information from different spectral bands, which contains richer details than a single-spectral image. For example, under a specific spectrum, pore defects may exhibit significantly different reflection or absorption characteristics from the surrounding area, making them easier to capture. The high-speed shooting function can record the instantaneous changes of the molten pool during the welding process, which are often closely related to the generation and development of defects. By obtaining these dynamic images, a comprehensive and accurate data basis is provided for subsequent analysis.

[0051] In the image preprocessing stage, multi-scale morphological filtering is adopted. It iteratively filters the image by defining a multi-structural element set containing circular, cross-shaped, and diamond-shaped kernels and combining the cascaded combination of opening operation and closing operation. The opening operation can effectively suppress high-frequency noise, which may come from arc light flickering or electronic device interference during the welding process, avoiding the interference of noise on subsequent analysis. The closing operation can fill the microscopic pore artifacts, preventing misjudging the artifacts as real defects and improving the authenticity of the image. In addition, constructing a morphological gradient map and superimposing it on the preprocessed molten pool image further enhances the oscillation characteristics of the molten pool boundary, making the edge and details of the molten pool clearer and creating good conditions for subsequent feature extraction.

[0052] The dual-branch feature interaction structure of the depthwise separable convolutional network is ingeniously designed. The dilated convolutional layer in the first branch increases the dilation rate as the network depth increases, which can gradually expand the receptive field and effectively extract the temperature field distribution characteristics in a large range of the molten pool. During the welding process, the change of the temperature field is closely related to the formation of defects. For example, uneven temperature distribution may lead to the generation of cracks. By accurately obtaining the temperature field characteristics, the potential risk of defects can be better judged. The second branch uses the channel attention mechanism to dynamically adjust the feature channel weights through the squeeze-and-excitation module, enhancing the local texture detail features. Welding defects often have unique manifestations in texture. For example, the texture at the crack will show discontinuous or abnormal trends. This mechanism can highlight these details and improve the recognition rate of defects. The cross-branch feature fusion layer fuses the features of the two branches to generate a fused feature vector containing rich information such as texture, temperature gradient, and geometric deformation, providing more comprehensive and representative features for subsequent defect classification.

[0053] The defect classification model using the heterogeneous graph neural network architecture constructs a molten pool-defect heterogeneous graph and uses the node type perception mechanism to distinguish the features of the molten pool area, heat affected zone, and pore defects. This model can fully consider the relationships and feature differences between different regions. Compared with traditional neural network models, it has stronger adaptability and accuracy in processing complex welding images. It can more accurately judge the type and location of defects, reduce misjudgment and missed judgment situations, and provide a reliable basis for welding quality assessment.

[0054] The dynamic sparse optimization model aims to minimize the feature redundancy and maximize the classification confidence. It uses an adaptive threshold screening algorithm to locate the key areas in the defect probability distribution matrix and generate a set of defect spatial coordinates. This algorithm can dynamically adjust the threshold according to the specific features of the image, avoiding the limitations of fixed thresholds under different welding conditions and improving the accuracy of defect location. Whether it is a tiny pore defect or a large crack defect, it can be accurately located, providing precise position information for subsequent repair and quality improvement.

[0055] A hierarchical verification framework constructed based on a set of defect space coordinates includes a candidate layer, a refinement layer, and a determination layer. The candidate layer aggregates adjacent defect points through a region growing algorithm, which can connect scattered defect points into a complete defect region, avoiding fragmented understanding of defects. The refinement layer uses a variational autoencoder to perform topological reconstruction on the defect morphology. It can not only restore the true shape of the defect but also improve the topological consistency between the reconstruction result and the true defect morphology by designing a reconstruction loss function that includes a voxel cross-entropy loss and a surface curvature regularization term, as well as introducing an adversarial training strategy. This helps technicians more intuitively understand the actual morphology of the defect and judge its harm degree. The determination layer fuses multi-frame temporal consistency tests based on a probabilistic graphical model. By constructing a hidden Markov model, taking the single-frame defect detection result as the observation state and the true existence of the defect as the hidden state, combining with the Viterbi algorithm to decode the optimal state sequence, and using the expectation-maximization algorithm to iteratively optimize the model parameters. This method makes full use of the temporal information of multi-frame images, overcomes the uncertainty of single-frame detection, improves the accuracy and stability of defect recognition, and can more reliably judge the true situation of the defect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is the working principle diagram of the welding defect recognition method based on the molten pool image according to the present invention;

[0057] Figure 2 is the flowchart of multi-scale morphological filtering processing;

[0058] Figure 3 is the flowchart of feature extraction by a depthwise separable convolutional network;

[0059] Figure 4 is the flowchart of defect classification by a heterogeneous graph neural network architecture. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0061] Please refer to Figures 1 - 4 , the present invention provides a welding defect recognition method based on the molten pool image, and the specific steps are as follows:

[0062] Collect the dynamic image sequence of the molten pool during the welding process through a multi-spectral high-speed camera. In the actual welding scenario, the multi-spectral high-speed camera is installed at a suitable position to ensure that the dynamic changes of the welding molten pool can be clearly captured. Its parameters such as the shooting frame rate and spectral range are reasonably set according to the welding process and defect recognition requirements to ensure that the obtained dynamic image sequence of the molten pool can accurately reflect the real-time state of the molten pool and provide sufficient data for subsequent analysis.

[0063] Perform noise suppression and edge enhancement on the dynamic image sequence of the molten pool based on multi-scale morphological filtering to generate a preprocessed molten pool image. Use multi-scale morphological filtering to process the collected dynamic image sequence of the molten pool. This filtering process can effectively suppress the noise in the image and at the same time enhance the edge information of the molten pool, making the contour of the molten pool clearer and providing better image data for subsequent feature extraction and defect recognition.

[0064] Construct a depthwise separable convolutional network to perform multi-level feature extraction on the preprocessed molten pool image to generate a fusion feature vector containing texture, temperature gradient, and geometric deformation. The depthwise separable convolutional network is one of the key technologies of the present invention. Through its unique network structure, it can extract rich feature information from the preprocessed molten pool image. These feature information cover multiple aspects such as the texture, temperature gradient, and geometric deformation of the molten pool, laying a solid foundation for accurately identifying welding defects subsequently.

[0065] Input the fusion feature vector into a pre-trained defect classification model to obtain a defect probability distribution matrix; the defect classification model adopts a heterogeneous graph neural network architecture and distinguishes the features of the molten pool area, heat affected zone, and pore defects through a node type perception mechanism. The pre-trained defect classification model is constructed based on a heterogeneous graph neural network architecture. This model can accurately distinguish the features of the molten pool area, heat affected zone, and pore defects through a node type perception mechanism. When the fusion feature vector is input into this model, the model will output a defect probability distribution matrix, which reflects the likelihood of defects existing in different regions of the image.

[0066] Construct a dynamic sparse optimization model. The dynamic sparse optimization model aims to minimize the feature redundancy and maximize the classification confidence, and uses an adaptive threshold screening algorithm to locate the key areas of the defect probability distribution matrix; generate a set of defect space coordinates based on the dynamic sparse optimization model. The dynamic sparse optimization model aims to minimize the feature redundancy and maximize the classification confidence, and uses an adaptive threshold screening algorithm to process the defect probability distribution matrix, thereby achieving accurate positioning of the key areas of the defect and generating a set of defect space coordinates. This set clearly defines the specific position information of the defects in the image in space, providing an important basis for further analyzing the type and severity of the defects subsequently.

[0067] Construct a hierarchical verification framework according to the set of defect space coordinates. The hierarchical verification framework includes a candidate layer, a refinement layer, and a determination layer. Among them, the candidate layer aggregates adjacent defect points through a region growing algorithm, the refinement layer uses a variational autoencoder to perform topological reconstruction on the defect morphology, and the determination layer fuses multi-frame temporal consistency checks based on a probabilistic graphical model, and finally outputs the type and location information of the welding defect. Construct a hierarchical verification framework according to the generated set of defect space coordinates. This framework includes a candidate layer, a refinement layer, and a determination layer. Through the collaborative work of these three layers, the type and location information of the defect can be accurately judged and output, greatly improving the accuracy and reliability of welding defect recognition.

[0068] The present invention will be further described below in conjunction with Embodiments 1 to 6:

[0069] Embodiment 1:

[0070] In the image preprocessing stage, multi-scale morphological filtering first defines a multi-structuring element set, which includes circular, cross-shaped, and diamond-shaped kernels. The sizes of these kernels are inversely proportional to the resolution of the molten pool image. This is because for an image with a higher resolution, its detailed information is rich and smaller-sized kernels are needed for fine processing; while for an image with a lower resolution, larger-sized kernels can better capture the overall features. For example, if the resolution of the molten pool image is 1920×1080, the radius of the circular kernel may be set to 3 pixels, the cross length of the cross-shaped kernel is 5 pixels, and the side length of the diamond-shaped kernel is 4 pixels.

[0071] Use a cascaded combination of opening and closing operations to iteratively filter the molten pool image. The opening operation first erodes the image and then dilates it, which can effectively suppress high-frequency noise. Suppose there are some isolated bright spot noises in the image generated by spatter during the welding process. After the opening operation, these small bright spot noises will be removed. The closing operation dilates first and then erodes, and is used to fill microvoid artifacts. During the welding process, due to reasons such as gas escape, some tiny void artifacts may be formed in the image. The closing operation can fill these small holes and make the image smoother.

[0072] When constructing the morphological gradient map, first extract the molten pool boundary oscillation characteristics through the difference operation between the original image and the filtering result. To further optimize the quality of the morphological gradient map, perform non-local means filtering on the difference result. Non-local means filtering suppresses the pseudo-gradient response by constructing an adaptive weight matrix. Its principle is to find pixel blocks similar to the current pixel in the image, assign weights to these pixel blocks according to the similarity degree, and then use these weights to filter the current pixel. Let the current pixel be , in the neighborhood centered on , the weight of pixel The calculation formula is as follows:

[0073]

[0074] where is the normalization constant to ensure ; and are the pixel values of the original image and the filtered result at position respectively; is the filtering parameter used to control the filtering intensity, the larger it is, the smoother the filtering effect.

[0075] Next, the Hessian matrix eigenvalue analysis is adopted to enhance the curvature feature of the molten pool boundary. The Hessian matrix can describe the second-order derivative information of the image at a certain point. By calculating the eigenvalues of the Hessian matrix, the curvature information of the image in different directions at this point can be obtained. The maximum curvature direction is encoded as the channel dimension of the gradient map, so that the bending degree of the molten pool boundary can be highlighted, providing more accurate edge information for subsequent defect recognition. For example, in the area where the molten pool boundary shows obvious bending, the feature of this area can be enhanced through the Hessian matrix analysis, making it easier to identify potential defects in subsequent processing. After the above processing, the morphological gradient map is superimposed on the preprocessed molten pool image to complete the image preprocessing work.

[0076] Example 2:

[0077] During the feature extraction process of the depthwise separable convolutional network, a dual-branch feature interaction structure is adopted.

[0078] The first branch uses an atrous convolutional layer to extract the distribution features of the large-scale temperature field of the molten pool. The atrous convolutional layer expands the receptive field of the convolution by introducing holes in the convolutional kernel. The dilation rate increases with the network depth. For example, in the shallow layer of the network, the dilation rate is set to 1, and as the number of network layers increases, the dilation rates are sequentially set to 2, 3, etc. In this way, at different depths of the network, temperature field distribution features of different scales can be obtained. Suppose in a certain layer of atrous convolution, the convolutional kernel size is 3×3 and the dilation rate is 2, then the actual receptive field size is 7×7, which can capture a larger range of temperature information.

[0079] The second branch uses a channel attention mechanism to enhance the local texture detail features, and dynamically adjusts the feature channel weights through a squeeze-and-excitation module. The channel attention mechanism is implemented using a dynamic gating unit, and the specific steps are as follows: First, global average pooling and max pooling are performed on the input feature map to obtain the average pooling feature map and the max pooling feature map respectively. These two feature maps are then respectively subjected to global average pooling and max pooling operations to generate a two-channel global description vector. Suppose the size of the input feature map is , after global average pooling, an average pooling feature map with a size of is obtained; after max pooling, a max pooling feature map with the same size of is obtained. These two feature maps are concatenated together to obtain a two-channel global description vector.

[0080] Then, the gated recurrent unit is used to learn the temporal dependence relationship of the pooling results and output the channel weight coefficients. The gated recurrent unit can adaptively update the memory state according to the input information, thereby learning the dependence relationship between different channels. Finally, the weight coefficients are multiplied with the original feature map channel by channel to generate enhanced local texture features. In this way, the channel attention mechanism can weight the feature map according to the importance of different channels, highlighting the local texture detail features.

[0081] A cross-branch feature fusion layer is set up to tensor-concatenate the output of the dilated convolution and the channel-weighted features, and generate a fused feature vector through 1×1 convolution for dimensionality reduction. Tensor concatenation combines the features of the two branches in the channel dimension. Suppose the size of the output feature map of the dilated convolution is , the size of the channel-weighted feature map is , and the size of the concatenated feature map is . Then, through 1×1 convolution, the dimensionality of the concatenated feature map is reduced to generate the final fused feature vector, which contains the temperature field distribution features and local texture detail features of the molten pool, providing rich information for subsequent defect classification.

[0082] Example 3:

[0083] The heterogeneous graph neural network architecture adopted by the defect classification model includes the following key steps:

[0084] Construct a molten pool-defect heterogeneous graph, in which the nodes in the graph are divided into molten pool main nodes, heat conduction nodes, and defect candidate nodes. Each node has rich attributes, including grayscale value, gradient magnitude, and spectral energy. The grayscale value reflects the brightness information of the area where the node is located, the gradient magnitude can represent the edge strength of the area, and the spectral energy reflects the energy distribution of the image at different frequencies. For example, in the edge area of the molten pool, the gradient magnitude is relatively large, indicating that there are obvious edges in this area; while in the uniform area inside the molten pool, the change of the grayscale value is small and the spectral energy is relatively low.

[0085] The type-aware graph attention mechanism is adopted to assign independent attention calculation paths to different node types. Specifically, independent query-key-value pair generation networks are constructed for the molten pool nodes and defect nodes respectively. During the message passing process, the molten pool nodes only receive the feature updates of the same type of nodes, which can maintain the consistency and stability of the features in the molten pool area. For example, the nodes inside the molten pool transmit information to each other, which can better reflect the overall features of the molten pool. The defect nodes adopt the cross-type attention fusion heat conduction path features because the generation of defects is often closely related to the heat conduction process. By fusing the heat conduction path features, defects can be identified more accurately.

[0086] During the heterogeneous graph feature propagation process, the residual graph convolutional layer stabilizes the propagation process of the heterogeneous graph features. The residual graph convolutional layer can effectively avoid the problem of gradient disappearance, enabling the network to better learn the deep features of the heterogeneous graph. Let the input feature be , after the transformation of the residual graph convolutional layer, the output feature is calculated by the formula:

[0087]

[0088] where, is the adjacency matrix of the heterogeneous graph, representing the connection relationship between nodes; is the weight matrix, used to learn the transformation of features; is the activation function, such as the ReLU function. In this way, the heterogeneous graph neural network can effectively learn the features of the molten pool area, heat affected zone and pore defects.

[0089] Finally, through the graph pooling operation, the node features are hierarchically aggregated, and the soft mask mechanism is adopted to retain the high-confidence defect areas and generate the defect probability distribution matrix. The graph pooling operation can reduce the scale of the graph while retaining important feature information. The soft mask mechanism screens the defect areas according to the confidence of the nodes, retains the areas with higher confidence, and the generated defect probability distribution matrix can intuitively reflect the possibility of defects in different areas of the image.

[0090] Example 4:

[0091] During the key area positioning and defect space coordinate generation process, the adaptive threshold screening algorithm is adopted. First, the defect probability distribution matrix is mapped into a three-dimensional space probability field, and its dimensions include the horizontal coordinate, vertical coordinate and probability density. In this way, the two-dimensional probability distribution matrix can be extended to the three-dimensional space to more intuitively represent the position and probability information of the defects in the image.

[0092] Construct a dynamic energy function, which includes regional connectivity constraints and a probability smoothing term. The regional connectivity constraints ensure that the selected defect regions are continuous, avoiding isolated noise points being misjudged as defects. The probability smoothing term makes the probability distribution smoother and reduces fluctuations. Let the dynamic energy function be , and its calculation formula is:

[0093]

[0094] where and are weight coefficients used to balance the importance of regional connectivity constraints and the probability smoothing term; is the set of all pixel points, is the neighborhood of pixel point ; and are the probability values of pixel points and respectively; is the set of all connected regions, is the perimeter of connected region . The larger the perimeter, the more irregular the region. By penalizing regions with larger perimeters, the selected defect regions can be made more compact.

[0095] Use the simulated annealing algorithm to solve the minimum value of the energy function. The simulated annealing algorithm is a heuristic optimization algorithm that simulates the physical annealing process and gradually reduces the temperature during the search process to avoid falling into local optimal solutions. During the solution process, the probability threshold is adaptively adjusted according to the annealing temperature curve. As the temperature decreases, the probability threshold gradually decreases, which can search for possible defect regions more widely in the initial stage of the search and gradually focus on high-confidence defect regions in the later stage of the search. When the algorithm converges, screen the set of defect coordinates whose connected region volume is greater than the preset threshold, and this preset threshold can be adjusted according to the actual welding process and defect detection requirements.

[0096] Example 5:

[0097] In the refinement layer of the hierarchical verification framework, use a variational autoencoder to perform topological reconstruction on the defect morphology.

[0098] The encoder uses a three-dimensional convolutional network to extract the voxelized representation of the defect. The three-dimensional convolutional network can perform convolutional operations on the defect data in three-dimensional space to extract the spatial features of the defect.

[0099] The decoder uses transposed convolution and a spatial transformation network to generate a continuous surface mesh. The transposed convolution operation can upsample the feature map extracted by the encoder to restore it to the size of the original data. The spatial transformation network can perform spatial transformation on the result after transposed convolution to make it more conform to the morphology of the real defect.

[0100] Design a reconstruction loss function, which includes voxel cross-entropy loss and surface curvature regularization term. The voxel cross-entropy loss is used to measure the difference between the reconstruction result and the true defect voxels, and its calculation formula is:

[0101]

[0102] where, is the total number of voxels, is the label (0 or 1) of the true defect voxel at position , is the predicted probability of the reconstruction result at position . The surface curvature regularization term is used to make the reconstructed defect surface smoother and conform to the true physical laws. Let the curvature of the surface be , and the calculation formula of the surface curvature regularization term is:

[0103]

[0104] where, represents the set of all points on the reconstructed surface, is the weight of point , which is assigned according to the position and importance of the point on the surface.

[0105] Introduce an adversarial training strategy. The discriminator network is used to distinguish the true defect morphology and the reconstruction result, so as to improve the topological consistency. The discriminator network and the generator (i.e., the encoder and decoder) of the variational autoencoder are trained adversarially. The generator tries to generate a reconstruction result closer to the true defect morphology, while the discriminator tries to distinguish the true defect and the reconstruction result. During the training process, the parameters of the generator and the discriminator are continuously adjusted until the two reach a balance. At this time, the generated reconstruction result is more similar to the true defect in topological structure and can more accurately reflect the morphology of the defect.

[0106] Example 6:

[0107] In the decision layer of the hierarchical verification framework, multi-frame temporal consistency checking is fused based on the probabilistic graphical model.

[0108] Construct a hidden Markov model, taking the single-frame defect detection result as the observation state and the true existence of the defect as the hidden state. The hidden Markov model is a statistical model that assumes that the state of the system is not directly observable (hidden state), but can be inferred from the observed signal (observation state). In welding defect detection, the defect detection result of each frame of image is the observation state, while whether there is a defect actually is the hidden state.

[0109] Decode the optimal state sequence through the Viterbi algorithm, and calculate the state transition probability by combining the overlap degree of the defect regions in adjacent frames. The Viterbi algorithm is a dynamic programming algorithm used to find the optimal state sequence in a hidden Markov model. Let the state set of the hidden Markov model be , the observation sequence be , the state transition probability matrix be , the observation probability matrix be , and the initial state probability vector be . The Viterbi algorithm finds the optimal state sequence by calculating the maximum probability path for each state at each time step. When calculating the state transition probability, the overlap degree of the defect regions in adjacent frames is combined. If the overlap degree of the defect regions in adjacent frames is high, it indicates that the defect is temporally continuous, and the state transition probability is large; conversely, if the overlap degree is low, the state transition probability is small.

[0110] Adopt the expectation maximization algorithm to iteratively optimize the model parameters until the temporal consistency likelihood function converges. The expectation maximization algorithm is an iterative algorithm used to estimate the model parameters containing hidden variables. In a hidden Markov model, the hidden state is the hidden variable. By continuously iteratively calculating the expectation step (E-step) and the maximization step (M-step), the model parameters are updated, making the temporal consistency likelihood function increase continuously until convergence. In the expectation step, the posterior probability of the hidden state is calculated according to the current model parameters; in the maximization step, the likelihood function is maximized to update the model parameters according to the posterior probability obtained in the expectation step. After multiple iterations, the hidden Markov model can more accurately fuse multi-frame temporal information, improve the accuracy of welding defect recognition, and finally output more reliable welding defect type and location information.

[0111] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0112] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made in these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A welding defect recognition method based on molten pool images, characterized in that, Including: Collecting a sequence of dynamic images of the molten pool during the welding process through a multi-spectral high-speed camera; Performing noise suppression and edge enhancement on the sequence of dynamic images of the molten pool based on multi-scale morphological filtering to generate a preprocessed molten pool image; Constructing a depthwise separable convolutional network to extract multi-level features from the preprocessed molten pool image, generating a fusion feature vector containing texture, temperature gradient, and geometric deformation; Inputting the fusion feature vector into a pre-trained defect classification model to obtain a defect probability distribution matrix; the defect classification model adopts a heterogeneous graph neural network architecture and distinguishes the features of the molten pool area, heat-affected zone, and pore defects through a node type perception mechanism; Constructing a dynamic sparse optimization model, which aims to minimize feature redundancy and maximize classification confidence, and uses an adaptive threshold screening algorithm to locate key regions in the defect probability distribution matrix; generating a set of defect spatial coordinates based on the dynamic sparse optimization model; Constructing a hierarchical verification framework according to the set of defect spatial coordinates, the hierarchical verification framework includes a candidate layer, a refinement layer, and a decision layer. Among them, the candidate layer aggregates adjacent defect points through a region growing algorithm, the refinement layer uses a variational autoencoder to perform topological reconstruction on the defect morphology, and the decision layer fuses multi-frame temporal consistency tests based on a probability graph model, and finally outputs the type and location information of welding defects.

2. The welding defect identification method according to claim 1, wherein The multi-scale morphological filtering includes: Defining a set of multi-structural elements, including circular, cross-shaped, and diamond-shaped kernels, and the kernel size is inversely proportional to the resolution of the molten pool image; Performing iterative filtering on the molten pool image using a cascaded combination of opening operation and closing operation, where the opening operation suppresses high-frequency noise and the closing operation fills microscopic pore artifacts; Constructing a morphological gradient map, extracting the oscillation characteristics of the molten pool boundary through the difference operation between the original image and the filtering result, and superimposing it on the preprocessed molten pool image.

3. The welding defect identification method according to claim 1, wherein The depthwise separable convolutional network adopts a dual-branch feature interaction structure, including: The first branch uses an atrous convolutional layer to extract the distribution characteristics of the large-scale temperature field of the molten pool, and the atrous rate increases with the network depth; The second branch uses a channel attention mechanism to enhance local texture detail features, and dynamically adjusts the feature channel weights through a squeeze-and-excitation module; Setting a cross-branch feature fusion layer, splicing the output of the atrous convolution and the channel-weighted features in a tensor, and reducing the dimension through a 1×1 convolution to generate the fusion feature vector.

4. The welding defect identification method according to claim 1, wherein The heterogeneous graph neural network architecture includes: Constructing a molten pool-defect heterogeneous graph, where the nodes in the graph are divided into molten pool main nodes, heat conduction nodes, and defect candidate nodes, and the node attributes include gray value, gradient magnitude, and spectral energy; Adopting a type-aware graph attention mechanism to assign independent attention calculation paths to different node types, and introducing a bidirectional message passing channel between the molten pool nodes and the defect nodes; Performing hierarchical aggregation on the node features through graph pooling operations, using a soft mask mechanism to retain high-confidence defect regions, and generating a defect probability distribution matrix.

5. The welding defect identification method according to claim 1, wherein The adaptive threshold screening algorithm includes: Mapping the defect probability distribution matrix into a three-dimensional space probability field, and the dimensions include horizontal coordinate, vertical coordinate, and probability density; Construct a dynamic energy function, which includes regional connectivity constraints and a probability smoothing term, and use the simulated annealing algorithm to solve the minimum value of the energy function; Adaptive adjust the probability threshold according to the annealing temperature curve, and screen the set of defect coordinates whose connected region volume is greater than the preset threshold.

6. The welding defect identification method according to claim 1, wherein The variational autoencoder's topological reconstruction of the defect morphology includes: The encoder uses a three-dimensional convolutional network to extract the voxelized representation of the defect, and the decoder uses a transposed convolution and a spatial transformation network to generate a continuous surface mesh; Design a reconstruction loss function, which includes voxel cross-entropy loss and surface curvature regularization term; Introduce an adversarial training strategy, and use a discriminator network to distinguish the real defect morphology from the reconstruction result to improve topological consistency.

7. The welding defect identification method according to claim 2, wherein The calculation of the morphological gradient map further includes: Perform non-local mean filtering on the difference result to construct an adaptive weight matrix to suppress the pseudo-gradient response; Use Hessian matrix eigenvalue analysis to enhance the curvature feature of the molten pool boundary, and encode the maximum curvature direction into the channel dimension of the gradient map.

8. The welding defect recognition method according to claim 3, wherein The channel attention mechanism is implemented using a dynamic gating unit, including: Perform global average pooling and max pooling on the input feature map to generate a two-channel global description vector; Learn the temporal dependence relationship of the pooling results through a gated recurrent unit and output the channel weight coefficients; Perform a per-channel multiplication of the weight coefficients and the original feature map to generate enhanced local texture features.

9. The welding defect recognition method according to claim 4, wherein The type-aware graph attention mechanism includes: Construct independent query-key-value pair generation networks for molten pool nodes and defect nodes respectively; During the message passing process, molten pool nodes only receive the feature updates of nodes of the same type, and defect nodes use cross-type attention to fuse the heat conduction path features; Stabilize the propagation process of heterogeneous graph features through residual graph convolutional layers.

10. The welding defect identification method according to claim 1, wherein The probability graph model's fusion of multi-frame temporal consistency checks includes: Construct a hidden Markov model, use the single-frame defect detection result as the observation state, and the real existence of the defect as the hidden state; Decode the optimal state sequence through the Viterbi algorithm, and calculate the state transition probability by combining the overlap degree of the defect regions in adjacent frames; Use the expectation maximization algorithm to iteratively optimize the model parameters until the temporal consistency likelihood function converges.

Images