Laser cladding pore defect prediction method based on multi-view learning

Abstract

The invention provides a laser cladding pore defect prediction method based on multi-view learning, and aims to solve the problems of single data representation, dynamic evolution information loss and the like in the existing pore prediction process. According to the system, a high-speed camera and a thermal infrared imager are integrated, a laser cladding multi-source monitoring platform is constructed, and molten pool images and temperature field data are collected in real time. Through an image processing and deep learning algorithm, geometric physical features, texture features, image depth features and temperature field features of a molten pool are extracted, and a multi-view heterogeneous feature set is constructed. And combined modeling and feature fusion of multi-view sequence data are realized in combination with a self-attention mechanism based on Vision Transform and a cross-view attention guiding module. Experimental results show that the accuracy of pore defect prediction can be effectively improved, and the method is suitable for online quality monitoring and intelligent control in the laser additive manufacturing process.

Description

TECHNICAL FIELD

[0001] The application is related to the field of machine learning and intelligent detection, and particularly relates to a laser cladding porosity defect prediction method based on multi-view learning. BACKGROUND

[0002] As a key technology of additive manufacturing, laser cladding uses high-energy laser to realize metal surface strengthening, has the advantages of low dilution rate, small heat-affected zone, etc., and is widely used in aerospace, energy equipment and other fields. The functional coating prepared by this technology can significantly improve the wear and corrosion resistance of the parts, but the high-sensitivity process can easily cause porosity defects, leading to problems such as decreased mechanical properties and increased corrosion risk. Traditional destructive quality detection methods cannot meet the real-time control requirements, so developing in-situ monitoring and porosity prediction methods for the cladding process has become a key research direction to improve process stability.

[0003] Porosity is a key quality defect of additive manufacturing workpieces, and its prediction accuracy directly affects the service performance of core components. The dynamic molten pool is the core characterization signal of the process quality, and current methods mainly synchronize the acquisition of multi-modal data through optical monitoring and thermal monitoring, and construct a porosity prediction model based on feature engineering. However, the existing methods have two limitations: first, the single-view feature analysis mode breaks the physical coupling relationship between the molten pool image and the temperature field, such as the spatio-temporal correlation between the molten pool morphology and the thermal history, which is not fully explored, resulting in the loss of the collaborative value of multi-source heterogeneous data; second, the static data sampling at discrete time nodes (such as fixed-frequency molten pool snapshot interception) cannot capture the dynamic continuity of porosity evolution, especially the cumulative effect of thermal-mechanical coupling across time scales on defect generation. Breaking through the single-dimensional static analysis framework and establishing a process dynamic modeling method that integrates multi-view spatio-temporal features is an important breakthrough to improve the reliability of laser cladding quality prediction. SUMMARY

[0004] The purpose of the present application is to provide a laser cladding porosity defect prediction method based on multi-view learning to solve the problems of single data representation and missing dynamic evolution information in the existing porosity prediction process, and to improve the accuracy of porosity prediction.

[0005] Note that the recitation of these objects does not preclude the existence of other objects. One mode of the present application does not necessarily achieve all the above-mentioned objects. Other objects can be extracted from the recitations of the specification, drawings, and claims

[0006] To achieve the above-mentioned objects, the technical solutions adopted by the present application are as follows:

[0007] A laser cladding porosity defect prediction method based on multi-view learning, comprising the following steps:

[0008] Step S1, multi-sensor data acquisition: through a multi-sensor data acquisition system, molten pool images and temperature field data in the laser cladding process are acquired;

[0009] Step S2, data preprocessing: the molten pool images and temperature field data collected are subjected to region of interest extraction to remove redundant regions and highlight molten pool information.

[0010] Step S3, multi-view feature extraction: geometric and physical features, texture features, depth features of molten pool images and statistical features, depth features of temperature field are extracted respectively to construct a multi-view heterogeneous feature set.

[0011] Step S4, intra-view time series data modeling: a time series modeling structure based on a deep neural network is constructed to model time dependence of multiple frames of data under the same view.

[0012] Step S5, inter-view information interaction algorithm: a cross-view attention module and a multi-view fusion encoding module are respectively used to interact and fuse heterogeneous features between different views.

[0013] Step S6, residual attention connection: residual attention connection is introduced between adjacent Encoder Blocks of the Vision Transformer to avoid feature collapse, slow down attention globalization, and increase diversity of deep features.

[0014] Step S7, porosity level prediction: the molten pool features finally obtained are classified through a decoding layer to predict the porosity level of the formed cladding layer.

[0015] In the above scheme, the multi-sensor data acquisition system in step S1 is composed of a laser cladding experiment platform and a multi-source information monitoring device. The laser cladding experiment platform is composed of a laser, a powder feeder, metal powder, a gas cylinder and other key components. The multi-source information acquisition device acquires molten pool information in the cladding process by using a high-speed camera and a thermal imager as acquisition equipment, and transmits these information to a computer for processing to achieve the monitoring purpose.

[0016] In the above scheme, the region of interest extraction of the visible light image in step S2 is realized by Mask RCNN model instance segmentation; the ROI extraction of the temperature field data is realized by positioning the highest temperature coordinates and cutting an experienced pixel region with the point as the center.

[0017] In the above scheme, the multi-view feature extraction in step S3 includes four sets of heterogeneous feature extraction methods to construct a multi-view space: the first view extracts physical features based on the image geometry and temperature field information of the molten pool; the second view applies a texture feature extraction method to extract the texture information of the molten pool surface; the third and fourth views are based on deep learning methods to extract high-order semantic features of the molten pool image and temperature field respectively, characterizing the potential patterns of the laser cladding process.

[0018] In the above scheme, the in-view temporal data modeling in step S4 specifically includes:

[0019] Suppose the temporal features of the v-th perspective are represented as follows: ,in Let v be the sequence dimension of the temporal features from the v-th perspective. Let be the sequence length of the temporal features from the v-th perspective. To establish the long-range dependencies between temporal data samples, a learning module based on the Vision Transformer Encoder Block, the Temporal Dependency Encoder Module (TDEM), is designed to perform temporal modeling of the samples. The data processed by the TDEM module can be represented as:

[0020] ;

[0021] For use in the first The TDEM module for viewpoint samples contains independent, non-shared weight parameters for each viewpoint. The learned feature form can be represented as:

[0022] ;

[0023] in, For the first Perspective class token, The patch embedding learned for the v-th time series sample.

[0024] To avoid gradient vanishing or exploding, improve the stability of model training, and accelerate model convergence, layer normalization (LN) is used. Process it.

[0025] In the above scheme, the Cross-View Attention Module (CVAM) in the inter-view information interaction algorithm in step S5 is specifically as follows:

[0026] The multi-view features extracted from the image and temperature field are taken as input, one feature is assigned to the guide feature, and the other feature is assigned to the basic feature. When the view i feature is taken as the basic feature, and the other view j feature is taken as the guide feature, the query matrix (Query, Q) is generated by using the i view feature, and the key matrix (Key, K) and the value matrix (Key, V) are generated by using the other j view feature. Then the guide attention from the guide feature to the basic feature is calculated , and it is added as supplementary information of the basic feature .

[0027] ;

[0028] ;

[0029] ;

[0030] wherein, and , , represent the input basic feature and the guide feature, respectively; , , and are learnable weights; is a bias, is a learnable parameter that adjusts the proportion weight of the guide attention mechanism.

[0031] The sequence length of the first view and the second view is the same, the number of samples within the time sequence is the same, and the dimension has been aligned through the full connection layer; the third view and the fourth view are uniformly mapped through the embedding layer, also with the same sequence length and dimension. The CVAM module needs to match the dimension, so the CVAM module is inserted between the first view and the second view, and between the third view and the fourth view, respectively.

[0032] When passing through the CVAM, the necessary supplementary information is learned from the other view to itself, and a feature with better representation ability is generated.

[0033] ;

[0034] wherein, In order to perform downstream tasks, the class token of each view is extracted: , which is spliced and arranged as a new class token, and is taken as input for subsequent porosity prediction.

[0035] ;

[0036] In the above scheme, the multi-view fusion encoding module (MVFEM) in the view angle information interaction algorithm in step S5 is specifically:

[0037] The learned features of the first view and the second view, the third view and the fourth view are spliced respectively as a new Patch embedding, denoted as , wherein, , represents the new Patch embedding. The MVFEM is divided into two parts, namely MVFEM_1 and MVFEM_2, which are used to process different feature learning needs in two stages. First, the MVFEM_1 is used to learn : ;

[0038] , wherein, , is the learned class token, is the patch embedding. In this way, the class token of one view can learn the context information of different views by interacting with the Patch embedding of other views.

[0039] Next, the class token of different views is fused with the Patch embedding at the same position, and the new fused token embedding is obtained through average pooling, denoted as . respectively collects the key information of two views. In order to further make this feature more representative, the MVFEM_2 is used to learn again, and LN is used to normalize the learned data.

[0040] ;

[0041] Similarly, , is the learned class token, is the patch embedding. In order to perform downstream tasks, the class token of it is extracted: and , and they are merged into a class token by splicing, denoted as , which is used as input for subsequent porosity prediction.

[0042] ​In the scheme, the residual attention connection in the step S6 is specifically:

[0043] Suppose the first Layer Encoder Block generates And The attention aggregation information calculated from them can be represented as: ;

[0044] Where is the projection dimension of the embedding vector. The residual attention connection then supplements the value of to to enrich the information of and avoid feature collapse. This can be represented by the formula:

[0045] ;

[0046] Where . And the attention value of this layer is calculated by multiplying with the matrix.

[0047] ;

[0048] In the scheme, the porosity level prediction in the step S7 is specifically:

[0049] For the final feature representation of the final molten pool , a fully connected layer + Tanh activation function + fully connected layer is used as a classifier, and a softmax layer is used at the end to convert it into a probability distribution. The output of the model can be represented as:

[0050] ;

[0051] Where represents the probability vector of porosity, and represent two fully connected layers respectively, is the number of porosity levels to be predicted.

[0052] Compared with the prior art, the effective effect of the present application is:

[0053] The present application first fuses image geometric and physical features, texture features, depth semantic features, temperature field statistics and depth features, constructs a multi-view heterogeneous feature set, comprehensively represents the dynamic evolution features of molten pool morphology and thermal behavior in the laser cladding process, and significantly improves the recognizability and prediction accuracy of pore defects. Through the time sequence dependence modeling module based on VisionTransformer, the time correlation in the multi-frame sequence is effectively captured, the defects of the traditional static image analysis method in modeling the continuity of pore evolution are made up, and the perception ability of the prediction model to long-term dynamic information in the cladding process is enhanced. The present application designs a cross-view attention mechanism (CVAM) which can guide efficient information interaction between image and temperature views, realize compensatory enhancement between views, avoid redundancy or conflict in the multi-modal information fusion process, and improve the fusion accuracy and model robustness. A multi-view fusion encoding module (MVFEM) is designed to complete the joint modeling of different view classtoken and patch embedding, realize the deep integration of complementary information between views, and effectively enhance the representation ability of the final feature to the pore grade discrimination boundary. An inter-layer residual attention connection mechanism is designed in the Transformer structure to alleviate the problem of attention degradation or feature collapse in deep network, improve the information retention ability and training stability of deep model, and enhance the consistency of pore prediction results under different data scenarios. The prediction system constructed by the present application has end-to-end online inference ability and can be integrated into the actual laser cladding production line as a quality monitoring system to improve the intelligent level and production efficiency of additive manufacturing.

[0054] Note that the description of these effects does not preclude the presence of other effects. One embodiment of the present application does not necessarily have all of the above-mentioned effects. Effects in addition to those described above can be apparent from the description, drawings, claims, and the like. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a flowchart of the laser cladding pore defect prediction method of the present application based on multi-view learning;

[0056] Figure 2 is the original image and temperature field collected by the laser cladding experiment platform of an embodiment of the present application, wherein (a) is the original image and (b) is the temperature field;

[0057] Figure 3 is the result image after the original image and temperature field are processed by the data preprocessing step of an embodiment of the present application, wherein (a) is the original image and (b) is the result image after the temperature field is processed;

[0058] Figure 4 is the overall block diagram of the algorithm of the pore rate prediction model of an embodiment of the present application. Detailed Implementation

[0059] The embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0060] Figure 1 This paper presents a laser cladding porosity defect prediction method based on multi-view learning, which includes the following steps:

[0061] Step S1 Multi-sensor data acquisition: Acquire images of the molten pool and temperature field data during the laser cladding process using a multi-sensor data acquisition system;

[0062] Step S2: Data preprocessing: Extract the region of interest from the acquired molten pool image and temperature field data to remove redundant areas and highlight the molten pool information;

[0063] Step S3 Multi-view feature extraction: Extract the geometric physical features, texture features, and depth features of the molten pool image, as well as the statistical features and depth features of the temperature field, to construct a multi-view heterogeneous feature set;

[0064] Step S4: Temporal data modeling within the same viewpoint: Based on the temporal modeling structure constructed by deep neural networks, time-dependent modeling is performed on multiple frames of data from the same viewpoint.

[0065] Step S5: Interview Information Interaction Algorithm: A cross-view attention module and a multi-view fusion coding module are used to perform information interaction and fusion on heterogeneous features between different viewpoints.

[0066] Step S6 Residual Attention Connection: Residual attention connections are introduced between adjacent Encoder Blocks of the Vision Transformer to avoid feature collapse, slow down attention globalization, and increase the diversity of deep features;

[0067] Step S7 Porosity level prediction: The final molten pool features are classified through the decoding layer to predict the porosity level of the cladding layer after it is formed.

[0068] Preferably, the multi-sensor data acquisition system in step S1 consists of two parts: a laser cladding experimental platform and a multi-source information monitoring device. The laser cladding experimental platform comprises key components such as a laser, a powder feeder, and metal powder. The multi-source information acquisition device uses a high-speed camera and a thermal imager as acquisition devices to obtain information about the molten pool during the cladding process and transmits this information to a computer for processing to achieve the monitoring purpose.

[0069] Preferably, in step S2, the extraction of the region of interest (ROI) of the visible light image is achieved through instance segmentation using the Mask RCNN model. This model introduces a pixel-level segmentation branch based on the Faster RCNN, generates candidate regions through the Region Proposal Network (RPN), and performs object detection (melt pool bounding box regression) and semantic segmentation (melt pool mask generation) within the candidate regions. The ROI extraction of the temperature field data is achieved by searching for the extreme values ​​of the matrix to locate the highest temperature coordinates of the temperature field information at the tail of the molten pool, and the empirical pixel region is extracted centered on these coordinates.

[0070] Preferably, the multi-view feature extraction in step S3 includes four sets of heterogeneous feature extraction methods to construct a multi-view space: The first view extracts physical features based on the image geometry and temperature field information of the molten pool, obtains the minimum bounding rectangle of the molten pool using the minAreaRect function of OpenCV, extracts the length and width of the molten pool, extracts the contour of the molten pool using the Canny edge detection algorithm, calculates the number of contour pixels, extracts the perimeter of the molten pool, and extracts the area of ​​the molten pool by calculating the number of pixels with a gray value of 255 in the binary image; The second view applies the Local Binary Pattern (LBP) texture feature extraction method, divides the LBP texture image into 10×10 non-overlapping regions, calculates the uniform LBP value of all pixels to generate a normalized histogram, and splices the histograms of these regions in spatial order into a high-dimensional feature vector as the final feature of the molten pool texture, extracting the texture information of the molten pool surface; The third and fourth views are based on deep learning methods to extract high-order semantic features of the molten pool image and temperature field, respectively, to characterize the potential patterns of the laser cladding process. High-order semantic feature extraction of the molten pool image is based on ResNet, using a ResNet-50 deep convolutional backbone network as the main feature extraction network. Through successive nonlinear transformations, the high-dimensional feature vector output at the network's end carries the global semantic features of the molten pool morphology. High-order semantic feature extraction of the temperature field adopts a ResNet-50 network that maintains topological consistency with the molten pool morphology feature extraction model. The number of input channels in the initial convolutional layer is adjusted from the standard RGB three-channel to a single-channel grayscale input, while keeping hyperparameters such as the number of filters and the convolutional kernel size unchanged.

[0071] Preferably, the intra-view temporal data modeling in step S4 specifically includes:

[0072] Suppose the temporal features of the v-th perspective are represented as follows: ,in Let v be the sequence dimension of the temporal features from the v-th perspective. Let be the sequence length of the temporal features from the v-th perspective. To establish the long-range dependencies between temporal data samples, a learning module based on the Vision Transformer Encoder Block, the Temporal Dependency Encoder Module (TDEM), is designed to perform temporal modeling of the samples. The data processed by the TDEM module can be represented as:

[0073] ;

[0074] For use in the first The TDEM module for viewpoint samples contains independent, non-shared weight parameters for each viewpoint. The learned feature form can be represented as:

[0075] ;

[0076] in, For the first Perspective class token, The patch embedding learned for the v-th time series sample.

[0077] To avoid gradient vanishing or exploding, improve the stability of model training, and accelerate model convergence, layer normalization (LN) is used. Process it.

[0078] Preferably, the Cross-View Attention Module (CVAM) in the inter-view information interaction algorithm in step S5 specifically comprises:

[0079] Multi-view features extracted from the image and temperature field are used as input. One feature is assigned to the guiding feature, and another to the base feature. When the feature from view i is used as the base feature and other view features are used as guiding features, a query matrix (Query, Q) is generated using the feature from view i, and a key matrix (Key, K) and a value matrix (Key, V) are generated using the other view features from view j. Guided attention from the guiding feature to the base feature is then calculated. And add it as supplementary information as a basic feature. .

[0080] ;

[0081] ;

[0082] ;

[0083] In the formula, and , , These represent the basic input features and the guiding features, respectively. , , and These are learnable weights; It is a bias. It is a learnable parameter that adjusts the weight of the attention-guiding mechanism.

[0084] The first and second perspectives have the same sequence length, which corresponds to the number of samples within the time series, and their dimensions are aligned through a fully connected layer. The third and fourth perspectives are uniformly mapped through an embedding layer and also have the same sequence length and dimensions. CVAM modules require dimension matching; therefore, this study inserts CVAM modules between the first and second perspectives, and between the third and fourth perspectives, respectively.

[0085] When using CVAM, each side learns the supplementary information it needs from the other's perspective, generating features with more representational capabilities.

[0086] ;

[0087] in, To perform downstream tasks, extract their class tokens: The tokens are concatenated and arranged into a new class token, which is then used as input for subsequent porosity prediction.

[0088] ;

[0089] Preferably, the multi-view fusion encoding module (MVFEM) in the inter-view information interaction algorithm in step S5 specifically comprises:

[0090] Learn from first-person, second-person, third-person, and fourth-person perspectives The features are concatenated separately to form a new Patch embedding, denoted as . ,in, , This represents the new patch embedding. MVFEM is divided into two parts, MVFEM_1 and MVFEM_2, to handle the different feature learning needs of the two stages. First, MVFEM_1 is used to... To learn: ;

[0091] in, , For the learned class token, This is for patch embedding. In this way, a class token from one perspective can learn contextual information from different perspectives by interacting with patch embeddings from other perspectives.

[0092] Next, the class tokens from different perspectives are fused with the patch embeddings at the same position. Through average pooling, a new fused token embedding is obtained, denoted as . . This collection gathers key information from two perspectives. To further enhance the representativeness of this feature, MVFEM_2 is used. We then learn again and use LN to normalize the learned data.

[0093] ;

[0094] Similarly, , For the learned class token, This is for patch embedding. To perform downstream tasks, its class token is extracted. and They are then combined into a single class token by concatenation, and this token is denoted as... This is used as input for subsequent porosity prediction.

[0095] Preferably, the residual attention connection in step S6 specifically includes:

[0096] Assume the first Layer Encoder Block Generation and Then the attention aggregation information calculated from them can be represented as: ;

[0097] in Let be the projection dimension of the embedding vector. The residual attention connection will then... The value is added to To enrich To obtain information and avoid feature collapse, the following formula can be used:

[0098] ;

[0099] in And through with The attention value for this layer is calculated by multiplying the matrices.

[0100] ;

[0101] Preferably, the porosity level prediction in step S7 specifically includes:

[0102] The final characteristic representation of the resulting molten pool The model uses a fully connected layer + Tanh activation function + fully connected layer as the classifier, and finally uses a softmax layer to convert it into a probability distribution. The final output of the model can be represented as:

[0103] ;

[0104] in The probability vector representing porosity. and These represent two fully connected layers. It is the number of porosity levels that need to be predicted. Specific Implementation

[0105] This embodiment provides a laser cladding porosity defect prediction method based on multi-view learning, including the following steps:

[0106] Step S101: Multi-sensor data acquisition: The cladding head adopts a three-beam coaxial powder feeding method. The fiber laser power can be adjusted from 1kW to -6kW. A six-axis robot is used to control the trajectory of the cladding head. A high-speed camera equipped with an 808nm filter, combined with an infrared light source, filters out high-light interference during processing. Simultaneously, a lens with up to 8x adjustable magnification is used to capture details of the molten pool. A blackbody-calibrated thermal imager is used to acquire the molten pool temperature field information. The working wavelength of the thermal imager is 8-14 micrometers. Based on the properties of Ni powder in a high-temperature environment, the emissivity of the molten pool is set to 0.8. The camera, thermal imager, and cladding head movement trajectory are at a 45-degree angle to facilitate the capture of global information. The acquired raw images and temperature field data of the molten pool are as follows: Figure 2 As shown.

[0107] Step S102: Data Preprocessing: Region of Interest (ROI) extraction from the visible light image is achieved through instance segmentation using the Mask R-CNN model; ROI extraction from the temperature field data is achieved by locating the highest temperature coordinates and extracting empirical pixel regions centered on that point. The results of data preprocessing are as follows: Figure 3 As shown.

[0108] Step S103: Multi-view feature extraction: Extract multi-view features from the preprocessed image: The first view extracts physical features based on the geometric shape and temperature field information of the molten pool image; the second view extracts texture information of the molten pool surface using a texture feature extraction method; the third and fourth views extract high-order semantic features of the molten pool image and temperature field based on deep learning methods to characterize the potential patterns of the laser cladding process.

[0109] Step 104: Porosity Prediction Model Training: A porosity prediction model for the laser cladding process is trained by combining intra-view temporal data modeling, inter-view information interaction algorithms, residual attention connections, and porosity level prediction steps. Data collected by a high-speed camera and thermal imager within 40ms is selected as a time-series sample. This part of the time-series dataset contains 492 samples, divided into a training set and a test set in an 8:2 ratio. The training set contains 395 time-series samples of images and temperature fields, and the test set contains 97 time-series samples of images and temperature fields.

[0110] First, time-series data is input into a time-series data modeling framework within the same viewpoint to perform time-dependency modeling of multiple frames of data from the same viewpoint. Then, an inter-viewpoint information interaction algorithm is used to exchange and fuse heterogeneous features from different viewpoints. Residual attention connections are used to avoid feature collapse, mitigate attention globalization, and increase the diversity of deep features. Finally, a decoding layer is used to predict the porosity level of the cladding layer. The overall algorithm block diagram is as follows: Figure 4 As shown.

[0111] During training, the Adam optimizer was used to optimize the model, with a learning rate of 0.0001 and 600 training iterations. To balance model performance and parameter count, after multiple experimental analyses, the number of encoder blocks for IFLM was set to 3, TDEM to 3, CVAM to 6, and MVFEM_1 and MVFEM_2 to 3 each.

[0112] Accuracy, macro average precision, macro average recall, and macro average F1 score were used as evaluation metrics to compare the test set accuracy of this model with other machine learning models. The results are shown in Table 1. Experimental results show that the CVAM and MVEFM models proposed in this invention significantly outperform the comparative models in all metrics (accuracy: 97% / 98%; precision: 96% / 97%; recall: 97% / 98%; F1 score: 96% / 97%), verifying the effectiveness of the proposed method in multi-view porosity prediction.

[0113] Table 1:

[0114] ;

[0115] In summary, this invention presents a multi-view learning-based laser cladding porosity defect prediction method that effectively integrates visible light images and temperature field data under multi-sensor data acquisition conditions, improving the prediction accuracy of porosity as the key indicator. Addressing the problem that traditional methods rely on a single viewpoint and cannot capture the dynamic evolution of the molten pool, this invention proposes a multi-view feature extraction method that integrates physical geometry, texture, and deep semantics, and constructs a temporal dependency modeling and viewpoint interaction mechanism to obtain more discriminative spatiotemporal representation features. To address the gradient degradation and information loss issues during deep model training, this invention introduces residual attention connections to improve model stability and feature diversity representation capabilities. Finally, validation results on a real laser cladding experimental dataset demonstrate that the proposed method achieves an accuracy of 98% in porosity level prediction, far exceeding traditional classification models, and can effectively achieve high-precision porosity-based prediction in additive manufacturing processes.

[0116] The present invention provides a detailed description of a laser cladding porosity defect prediction system and method based on multi-view learning. Specific embodiments are used to briefly illustrate the basic principles and implementation methods of the invention, but the scope of protection of the invention is not limited thereto. Those skilled in the art will understand that various changes, modifications, substitutions, and variations can be made to the embodiments described herein without departing from the principles and spirit of the invention. The scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting porosity defects in laser cladding based on multi-view learning, characterized in that, The system includes: Step S1, Multi-sensor data acquisition: Acquire images of the molten pool and temperature field data during the laser cladding process through a multi-sensor data acquisition system; Step S2, Data preprocessing: Extract the region of interest from the acquired molten pool image and temperature field data to remove redundant areas and highlight the molten pool information; Step S3, Multi-view feature extraction: Extract the geometric physical features, texture features, and depth features of the molten pool image, as well as the statistical features and depth features of the temperature field, to construct a multi-view heterogeneous feature set; Step S4, Time-series data modeling within the viewpoint: Based on the time-series modeling structure constructed by the deep neural network, time-dependent modeling is performed on multiple frames of data under the same viewpoint; Step S5, Interview Information Interaction Algorithm: A cross-view attention module and a multi-view fusion coding module are used respectively to perform information interaction and fusion on heterogeneous features between different viewpoints; Step S6, Residual Attention Connection: Residual attention connections are introduced between adjacent Encoder Blocks of the Vision Transformer to avoid feature collapse, slow down attention globalization, and increase the diversity of deep features; Step S7, Porosity level prediction: The final molten pool features are classified through the decoding layer to predict the porosity level of the cladding layer after it is formed.

2. The laser cladding porosity defect prediction method based on multi-view learning according to claim 1, characterized in that, The multi-sensor data acquisition system described in step S1 consists of two parts: a laser cladding experimental platform and a multi-source information monitoring device. The laser cladding experimental platform consists of a laser, a powder feeder, metal powder, and a gas cylinder. The metal powder is loaded into the powder storage box of the powder feeder, and the high-purity nitrogen gas cylinder is connected to the powder feeder through a gas pipe. The powder feeder is integrated into the laser head through a conduit. The multi-source information acquisition device uses a high-speed camera and a thermal imager as acquisition devices to obtain information about the molten pool during the cladding process. Both the high-speed camera and the thermal imager are connected to a computer, and the acquired molten pool information is transmitted to the computer for processing to achieve the monitoring purpose.

3. The laser cladding porosity defect prediction method based on multi-view learning according to claim 1, characterized in that, In step S2, the region of interest extraction of the visible light image is achieved by instance segmentation of the Mask RCNN model; the ROI extraction of the temperature field data is achieved by locating the highest temperature coordinate and extracting an empirical pixel region centered on that point.

4. The laser cladding porosity defect prediction method based on multi-view learning according to claim 1, characterized in that, The multi-view feature extraction described in step S3 includes four sets of heterogeneous feature extraction methods to construct a multi-view space: the first view extracts physical features based on the image geometry and temperature field information of the molten pool; The second perspective uses texture feature extraction methods to extract texture information from the surface of the molten pool; the third and fourth perspectives are based on deep learning methods to extract high-order semantic features of the molten pool image and temperature field, respectively, to characterize the potential patterns of the laser cladding process.

5. The laser cladding porosity defect prediction method based on multi-view learning according to claim 1, characterized in that, The in-view temporal data modeling described in step S4 specifically includes: Suppose the temporal features of the v-th perspective are represented as follows: ,in Let v be the sequence dimension of the temporal features from the v-th perspective. Let be the sequence length of the temporal features at the v-th viewpoint. To establish the long-range dependencies between temporal data samples, a learning module based on a visual Transformer encoder, the temporal dependency encoding module, is designed to perform temporal modeling on the samples. The data processed by the TDEM module can be represented as: ; For use in the first The TDEM module for viewpoint samples contains independent, non-shared weight parameters for each viewpoint. The learned feature form can be represented as: ； in, For the first Perspective class token, The patch embedding learned for the v-th time-series sample; To avoid gradient vanishing or exploding, improve the stability of the model training process, and accelerate model convergence, layer normalization (LN) is used. Process it.

6. The laser cladding porosity defect prediction method based on multi-view learning according to claim 1, characterized in that, The cross-view attention module in the inter-view information interaction algorithm described in step S5 is specifically as follows: Multi-view features extracted from the image and temperature field are used as input. One feature is assigned to the guiding feature, and another feature is assigned to the base feature. When the feature from view i is used as the base feature and other view features are used as guiding features, a query matrix (Query, Q) is generated using the feature from view i, and a key matrix (Key, K) and a value matrix (Key, V) are generated using the other view features from view j. Subsequently, the guiding attention from the guiding feature to the base feature is calculated. And add it as supplementary information as a basic feature. : ； ； ； In the formula, and , , These represent the basic input features and the guiding features, respectively. , , and These are learnable weights; This represents the query matrix from the perspective of i. , These represent the key matrix and value matrix from the perspective of j, respectively. This is a scaling factor to prevent excessively large similarity values ​​from causing gradient vanishing after softmax. It is a bias. It is a learnable parameter that adjusts the weight of the attention-guiding mechanism; The sequence lengths of the first and second perspectives are the same, which is the same as the number of samples in the time series, and the dimensions have been aligned through the fully connected layer; the third and fourth perspectives are uniformly mapped through the embedding layer and also have the same sequence length and dimensions; the CVAM module requires dimension matching, so this study inserts CVAM modules between the first and second perspectives, and between the third and fourth perspectives respectively. When using CVAM, each element learns supplementary information from the other's perspective, generating more representative features. ;in, It is a module consisting of a multi-layer ViT encoder containing CVAM. , They represent The class token and patch embedding, Let v be the sequence dimension of the temporal features from the v-th perspective. Let v be the sequence length of the temporal features from the v-th perspective; in order to perform downstream tasks, extract their class tokens: By concatenating and juxtaposing them into a new class token, this token is used as input for subsequent porosity prediction. ； in, This represents the class token of the i-th perspective.

7. The laser cladding porosity defect prediction method based on multi-view learning according to claim 1, characterized in that, The multi-view fusion coding module in the inter-view information interaction algorithm described in step S5 specifically includes: Learn from first-person, second-person, third-person, and fourth-person perspectives The features are concatenated separately to form a new Patch embedding, denoted as . ,in, , Representing the new Patchembedding, MVFEM is divided into two parts, MVFEM_1 and MVFEM_2, to handle the different feature learning needs of the two stages; firstly, MVFEM_1 is used to... To learn: ; in, , For the learned class token, This is for patch embedding; in this way, a class token from one perspective can learn contextual information from different perspectives by interacting with patch embeddings from other perspectives. Next, the class tokens from different perspectives are fused with the patch embeddings at the same position. Through average pooling, a new fused token embedding is obtained, denoted as . ; This collection gathers key information from two perspectives. To further enhance the representativeness of this feature, MVFEM_2 is used. We then perform another training session, using LN to normalize the learned data: ； Similarly, , For the learned class token, For patch embedding, its class token is extracted for downstream tasks: and They are then combined into a single class token by concatenation, and this token is denoted as... This is used as input for subsequent porosity prediction.

8. The laser cladding porosity defect prediction method based on multi-view learning according to claim 1, characterized in that, The residual attention connection mentioned in step S6 is specifically as follows: Assume the first Encoder Block generates query matrix and Then the attention aggregation information calculated from them can be represented as: ; in, The projection dimension of the embedded vector is used, and the residual attention connection will then... The value is added to To enrich To obtain information and avoid feature collapse, the following formula can be used: ； in , This represents the attention aggregation information of the l-th layer; and through... The attention value for this layer is calculated by multiplying the matrices: .

9. The laser cladding porosity defect prediction method based on multi-view learning according to claim 1, characterized in that, The porosity level prediction in step S7 specifically includes: The final characteristic representation of the resulting molten pool The model uses a fully connected layer + Tanh activation function + fully connected layer as the classifier, and finally uses a softmax layer to convert it into a probability distribution. The final output of the model can be represented as: ; in The probability vector representing porosity. and These represent two fully connected layers. It is the number of porosity levels that need to be predicted.

Images