A method for online real-time prediction of component status based on deep learning

Abstract

The present invention discloses a method for online real-time prediction of component state based on deep learning. The present invention first collects the size and shape of the molten pool and the deposited layer under different process parameters, and calculates the shrinkage of the material from the molten state to the solidified state; then the size and shape of the molten pool and the deposited layer, the shrinkage of the deposited layer, and the process parameters collected are used as input, and the stress value of each point in the manufacturing is calculated in the finite element simulation, and after these parameters are subjected to multi-information feature fusion processing, the real-time stress value of the component in the manufacturing process is output; through a large number of experimental samples, the real-time stress curve diagram and the component morphology diagram in the manufacturing process are obtained and correspond one by one, and the samples are classified into four states of normal, deformation, pre-crack and cracking. Through convolutional neural network training, the state of the output component in the manufacturing process is output. The method can obtain the constructed stress and state, effectively solve the defects such as deformation and cracks in the manufacturing process so as to adjust and optimize the process parameters in time.

Description

Technical Field

[0001] The present invention belongs to the technical field of additive manufacturing, and in particular relates to a method for online real-time prediction of component status based on deep learning. Background Art

[0002] Since laser additive manufacturing is a rapid melting-solidification process, there must be differences in elastic modulus, temperature and thermal expansion coefficient between the matrix and the molding material, which leads to the formation of residual stress inside the component, and residual stress promotes the formation and expansion of cracks. The impact of cracks is relatively fatal, and it is very likely to directly scrap the parts or reduce their service life. Using these parts in important fields such as aerospace and energy and chemical industry will lead to catastrophic consequences. With the development of the demand for intelligent manufacturing, it is possible to monitor the residual stress, microstructure changes and component state changes inside the components during the molding process in real time during the printing process, and make timely process adjustments to reduce the generation of waste when abnormalities are found, which will help to achieve closed-loop control in the manufacturing process.

[0003] To meet this demand, the present invention proposes a method for online real-time prediction of components based on deep learning. Deep learning technology is used to perform real-time calculation of residual stress and real-time judgment of defects and development of morphological states during the manufacturing process, thereby reducing the scrap rate and effectively improving the molding quality of workpieces. Summary of the invention

[0004] In order to solve the above problems, the present invention discloses a method for online real-time prediction of components based on deep learning, the purpose of which is to improve the molding quality of laser additive manufacturing components. The morphology of the molten pool and the deposited layer is captured by the laser melting deposition monitoring system. The real-time stress data inside the component is calculated by finite element simulation in combination with material properties and process parameters, and data fusion is performed to build a deep learning convolutional neural network to predict the real-time stress during the component manufacturing process. By real-time prediction of the stress values ​​of a large number of samples during the manufacturing process, the stress curves are processed in the computer. The manufactured samples are of four types: normal, pre-cracked, cracked, and deformed. The deposited layer morphologies of these four states are established in one-to-one correspondence with their stress curves, which are used as the input of the convolutional neural network for predicting the state of the component, and the state of the component during the manufacturing process is output.

[0005] The method for online real-time prediction of components based on deep learning of the present invention comprises the following steps:

[0006] S1: Data acquisition of the monitoring system: dynamic videos of the molten pool manufactured by laser additive manufacturing under different process parameters are collected, and images of the molten pool in the dynamic videos are continuously collected to obtain the geometric morphology of the molten pool and generate a three-dimensional model of the molten pool. Meanwhile, the shrinkage image of the deposited layer is collected to obtain a three-dimensional model of the deposited layer. Then, the height of the molten pool and the height of the deposited layer after cooling are subtracted at the same pixel coordinates through image processing technology to obtain the displacement difference generated during the cooling and solidification process of the molten pool, and the shrinkage data of the deposited layer are obtained;

[0007] S2: Data acquisition for finite element simulation: Conduct simulation experiment modeling in finite element simulation, use the three-dimensional model of the molten pool and deposited layer obtained in step S1, input the relevant parameters of the material selected for the experiment, simulate the laser heat source and move it with the manufacturing process, and load the displacement difference generated by the cooling and solidification process of the molten pool as the only surface load to the established model, and simulate and calculate the stress generated by the cooling and shrinking of the molten pool;

[0008] S3: Establish a convolutional neural network model 1 for predicting real-time stress: The convolutional neural network needs to fuse the molten pool, the geometric morphology of the deposited layer, the shrinkage data of the deposited layer, the stress values ​​of each point and the process parameters obtained in the above steps S1 and S2 as the input of the convolutional neural network, divide the training data set and the test data set, output the real-time stress value of a certain point in the manufacturing process of the component, and evaluate the trained model;

[0009] S4: Establish a convolutional neural network model 2 for predicting component status: The convolutional neural network model 1 established through the above step S3 predicts the stress values ​​of a large number of samples during the manufacturing process in real time, and processes them into stress curves through computers. The manufactured samples are of four types: normal, pre-cracked, cracked, and deformed. A one-to-one correspondence is established between the deposition layer morphologies of these four states and their stress curves, which are used as the input of the convolutional neural network for predicting component status. The training data set and the test data set are divided, and the output is the state of the component during the manufacturing process.

[0010] Furthermore, the specific process of step S1 is as follows: a coaxial monitoring CCD is arranged on the laser head to obtain a top view of the molten pool, and a side monitoring CCD is arranged to obtain a right view and a main view of the molten pool, thereby synthesizing a three-dimensional model of the molten pool; the three-dimensional model of the deposition layer is obtained by adopting a dual-line laser monitoring system, and the dual-line laser monitoring system is mainly composed of two line lasers generated by two line laser exciters, one line laser is 1 mm away from the molten pool, which is considered to be the state where the molten pool has just been melted, and the other line laser is located 4 mm away from the molten pool, which is considered to be a deposition layer formed after the molten pool is rapidly cooled and contracted, and the contours of the deposition layers are captured respectively by the line laser monitoring CCD, thereby forming a three-dimensional model of the deposition layer; for the captured contours of the deposition layer, edge detection is performed through image processing, and the molten pool height and the deposition layer height after cooling are subtracted at the same pixel coordinates to obtain the displacement difference generated during the cooling and solidification process of the molten pool, and to obtain the deposition layer shrinkage data.

[0011] Furthermore, the data collection of the finite element simulation described in step S2 is specifically carried out as follows: according to the three-dimensional model of the molten pool and the deposited layer obtained in step S1, the process parameters of laser additive manufacturing and the size of the substrate in the experiment, a sequential thermal-mechanical coupling finite element model is established. In the sequential thermal-mechanical coupling finite element model, each layer uses death and death units to simulate the deposition process of powder particles. At the beginning of the simulation, the death unit will not affect the thermo-mechanical model. As the set simulated laser beam heat source moves, the constructed geometric model will be activated unit by unit and layer by layer. The shrinkage value of the deposited layer is loaded as a load onto the real-time model and the stress value of the point is calculated. The calculation results of each step are iteratively updated to obtain the stress value, the required unit nodes are selected, and the stress data is exported and processed.

[0012] Furthermore, the step S3 of establishing the convolutional neural network model 1 for predicting real-time stress is to fuse the data collected in steps S1 and S2. The data collected from the three sources of machine, vision, and finite element must be integrated into a comprehensive data set to be used as training features and targets for the prediction model. All collected signals must be synchronized with a shared timestamp to establish a one-to-one correspondence.

[0013] Furthermore, the convolutional neural network model 1 for predicting real-time stress described in step S3 adopts a hybrid network, which is implemented by stacking two neural networks, a convolutional network and a multi-layer neural network, wherein the convolutional network includes 2 convolutional layers, 2 pooling layers, 1 Dropout layer, and 2 fully connected layers from input to output. The convolutional network is used to process the image data of the molten pool and the deposited layer and extract different feature data of the image through convolution calculation. All convolutional layers are equipped with Relu activation functions. The pooling layer cuts the convolutional features into several regions, takes the maximum value or average value, and obtains new, smaller-dimensional features. In order to avoid overfitting of the model, the Dropout layer allows part of the hidden neurons to work and part of them not to work during the training stage of the model. The fully connected layer is used to connect the image features to obtain the final image features of the molten pool and the deposited layer; the multi-layer neural network includes 1 input layer, 2 hidden layers, and 1 output layer from input to output. The parameters input into the input layer are the image features of the molten pool and the deposited layer, the laser power P and the scanning rate V in the process parameters, and the stress. After two hidden layers, the real-time stress value of a certain point in the component manufacturing process is output.

[0014] Furthermore, the predicted real-time stress convolutional neural network model 1 described in step S3 uses the mean square error MSE for comparison and evaluation, that is, the predicted real-time stress value and the actual value. The mean square error MSE is defined as:

[0015]

[0016] In the formula, n is the number of model samples, i is the data sample of each, and f i is the predicted value of stress at a certain point in the model, y i is the actual stress value at a certain point in the model.

[0017] Furthermore, the convolutional neural network model 2 for predicting the state of the component established in step S4 needs to use the convolutional neural network model 1 established in step S3 to predict the stress values ​​of a large number of samples in the manufacturing process in real time, and process them into stress curves through computers. The manufactured samples are of four types, namely normal, pre-cracked, cracked, and deformed. A one-to-one correspondence is established between the deposition layer morphologies of these four states and their stress curves, which are used as inputs of the convolutional neural network for predicting the state of the component, and the training data set and the test data set are divided to output the state of the component during the manufacturing process.

[0018] Furthermore, step S4 establishes a convolutional neural network model 2 for predicting component status, which includes 2 convolutional layers, 2 pooling layers, and 2 fully connected layers from input to output. At the same time, all convolutional layers are equipped with Relu activation functions. In order to speed up the training and convergence of the network, improve the network gradient and prevent overfitting, a BN layer is added after each convolutional layer; in addition, the Softmax classifier and the cross entropy loss function are used in the final output layer to calculate the probability of each classification of the output value, and the maximum probability is taken as the test result to obtain the final state classification, and the performance of the network is optimized and evaluated.

[0019] Furthermore, the neural network model 2 established in step S4 is to deeply evaluate the model, construct a confusion matrix, and evaluate the false positive and false negative rates of the confusion matrix with precision Precision and recall Recall. The calculation formula is as follows:

[0020]

[0021] Among them, TP means that a sample is predicted to be a certain category, FP means that a sample does not belong to a certain category but is predicted to be a certain category, and FN means that a sample belongs to a certain category but is not predicted to be a certain category.

[0022] The present invention has the advantages that:

[0023] 1. The present invention is based on finite element simulation calculation, laser additive manufacturing forming principle, combined with the thermodynamic finite element model of sequential thermal-mechanical coupling, and directly uses the shrinkage of the deposited layer as the load to directly calculate the stress field, which greatly reduces the calculation time, simplifies the definition steps while ensuring the accuracy of the simulation, and provides a reference for the optimization of laser additive manufacturing process parameters.

[0024] 2. The present invention can directly predict the internal real-time stress of the component manufacturing process based on the molten pool, the size and morphology of the deposited layer, the process parameters and the stress value, and can respond to the changes in the stress inside the deposited layer in real time through computer data processing, which undoubtedly provides great help in timely discovering problems in the manufacturing process.

[0025] 3. The present invention obtains stress curves and component deposition process state data through a large number of experiments, establishes a one-to-one correspondence, and finally uses the network for a comprehensive manufacturing process through convolutional neural network training to determine which component state the component will be in. This can efficiently control additive manufacturing, improve molding quality, and greatly reduce printing costs, which is of great significance to laser additive manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a basic process diagram of online real-time prediction of component status based on deep learning;

[0027] Figure 2 A method and system for online real-time prediction of construction status based on deep learning;

[0028] Figure 3 It is a structural schematic diagram of a convolutional neural network model 1 for predicting real-time stress;

[0029] Figure 4 Schematic diagram of the structure of a convolutional neural network model 2 for predicting component status.

[0030] Specific real-time method

[0031] Depend on Figure 1 As shown, a method for online real-time prediction of component status based on deep learning of the present invention comprises the following steps:

[0032] S1. Data acquisition of the monitoring system: Use the coaxial monitoring CCD and the lateral monitoring CCD to collect dynamic videos of the molten pool manufactured by laser additive manufacturing under different process parameters, continuously collect images of the molten pool in the dynamic video, and obtain the molten pool size and morphology data. At the same time, use the dual-line laser monitoring system to collect the shrinkage image of the deposited layer. Through the image processing technology, the height of the molten pool and the height of the deposited layer after cooling at the same pixel coordinate are subtracted to obtain the displacement difference generated during the cooling and solidification process of the molten pool, and the shrinkage data of the deposited layer are obtained.

[0033] S2. Data acquisition of finite element simulation: Based on the three-dimensional model constructed by monitoring and capturing, the process parameters of directed energy deposition, and the size of the substrate in the experiment, a sequential thermomechanical coupling finite element model was established. In the finite element model, death and death units are used in each layer to simulate the deposition process of powder particles. At the beginning of the simulation, the death unit will not affect the thermomechanical model. As the set simulated laser beam heat source moves, the constructed geometric model will be activated unit by unit and layer by layer. The shrinkage value of the deposited layer is loaded as a load onto the real-time model and the stress value of the point is calculated. The calculation results of each step are iteratively updated to obtain the stress value, the required unit nodes are selected, and the stress data is exported and processed.

[0034] S3. The data collected from the three sources of machine, vision and finite element must be integrated into a comprehensive data set, and all collected signals must be synchronized with the shared timestamp to establish a one-to-one correspondence. Each data in the data set consists of data content and data labels; the data content is the morphology image of the molten pool and the deposited layer, and the process parameters, and the data label is the stress value corresponding to each point in the manufacturing process in real time.

[0035] like Figure 3As shown, the convolutional neural network model 1 for predicting real-time stress adopts a hybrid network, which is mainly realized by stacking two neural networks, namely, a convolutional network and a multi-layer neural network. The convolutional network includes convolutional layer, pooling layer, convolutional layer, pooling layer, Dropout layer, fully connected layer, and fully connected layer from input to output. The convolutional network is mainly used to process the image data of the molten pool and the deposited layer, and extract different feature data of the image through convolution calculation. All convolutional layers are equipped with Relu activation functions. The pooling layer cuts the convolutional features into several regions, takes the maximum value or average value, and obtains new features with smaller dimensions. The Dropout layer is used to avoid overfitting of the model. The fully connected layer is used to connect the image features to obtain the final image features of the molten pool and the deposited layer. The multi-layer neural network includes input layer, hidden layer, hidden layer, and output layer from input to output. The parameters input in the input layer are the image features of the molten pool and the deposited layer, the laser power P and the scanning rate V in the process parameters, and the stress. After two hidden layers, the real-time stress value of a certain point in the component manufacturing process is output.

[0036] Inputting the data in the training data set into the deep learning convolutional neural network model established in step S3, training and optimizing the model, and obtaining an optimized deep learning convolutional neural network model;

[0037] The convolutional neural network model 1 of step S3 uses the mean square error MSE to compare and evaluate, that is, the predicted real-time stress value and the actual value. The mean square error is defined as:

[0038]

[0039] In the formula, n is the number of model samples, i is the data sample of each, and f i is the predicted value of stress at a certain point in the model, y i is the actual stress value at a certain point in the model.

[0040] S4. Figure 4As shown, the convolutional neural network model 1 established by the above step S3 predicts the stress values ​​of a large number of samples in the manufacturing process in real time, and processes them into stress curves by computer. The manufactured samples are normal, pre-cracked, cracked, and deformed. The sedimentary layer morphology of these four states is established in a one-to-one correspondence with their stress curves, which are used as the input of the convolutional neural network for predicting the state of the component, and the training data set and the test data set are divided to output the state of the component in the manufacturing process. The convolutional neural network structure includes 2 convolutional layers, 2 pooling layers, and 2 fully connected layers from input to output. At the same time, all convolutional layers are equipped with Relu activation functions. In order to speed up the training and convergence speed of the network, improve the network gradient and prevent overfitting, a BN layer is added after each convolutional layer. In addition, the Softmax classifier and the cross entropy loss function are used in the final output layer to calculate the probability of each classification of the output value, and the maximum probability is taken as the test result to obtain the final state classification, and the performance of the network is optimized and evaluated.

[0041] The neural network model 2 established in step S4 is used to deeply evaluate the model, construct a confusion matrix, and evaluate the false positive and false negative rates of the confusion matrix with precision and recall.

[0042]

[0043] Among them, TP means that a sample is predicted to be a certain category, FP means that a sample does not belong to a certain category but is predicted to be a certain category, and FN means that a sample belongs to a certain category but is not predicted to be a certain category.

[0044] The above are only preferred embodiments of the invention and are not intended to limit the invention. Unless otherwise specified, the various processes in the text of the invention that appear in a sequential order do not necessarily have a sequential execution order. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the invention.

Claims

1. A method for online real-time prediction of component status based on deep learning, characterized in that: The following steps are involved: S1: Data acquisition of the monitoring system: dynamic videos of the molten pool manufactured by laser additive manufacturing under different process parameters are collected, and images of the molten pool in the dynamic videos are continuously collected to obtain the geometric morphology of the molten pool and generate a three-dimensional model of the molten pool. Meanwhile, the shrinkage image of the deposited layer is collected to obtain a three-dimensional model of the deposited layer. Then, the height of the molten pool and the height of the deposited layer after cooling are subtracted at the same pixel coordinates through image processing technology to obtain the displacement difference generated during the cooling and solidification process of the molten pool, and the shrinkage data of the deposited layer are obtained; The details are as follows: a coaxial monitoring CCD is arranged on the laser head to obtain a top view of the molten pool, and a side monitoring CCD is arranged to obtain a right view and a main view of the molten pool, so as to synthesize a three-dimensional model of the molten pool; the three-dimensional model of the deposition layer is obtained by adopting a dual-line laser monitoring system. The dual-line laser monitoring system consists of two line lasers generated by two line laser exciters. One line laser is 1 mm away from the molten pool, which is considered to be the state where the molten pool has just been melted, and the other line laser is located 4 mm away from the molten pool. It is considered that the deposition layer is formed after the molten pool is rapidly cooled and contracted, and the contours of the deposition layers are captured by the line laser monitoring CCD, thereby forming a three-dimensional model of the deposition layer; for the captured contours of the deposition layer, edge detection is performed through image processing, and the height of the molten pool and the height of the deposition layer after cooling are subtracted at the same pixel coordinates to obtain the displacement difference generated during the cooling and solidification of the molten pool, and the shrinkage data of the deposition layer are obtained; S2: Data acquisition for finite element simulation: Conduct simulation experiment modeling in finite element simulation, use the three-dimensional model of the molten pool and deposited layer obtained in step S1, input the relevant parameters of the material selected for the experiment, simulate the laser heat source and move it with the manufacturing process, and load the displacement difference generated by the cooling and solidification process of the molten pool as the only surface load to the established model, and simulate and calculate the stress generated by the cooling and shrinking of the molten pool; The data collection of the finite element simulation is specifically carried out as follows: according to the three-dimensional model of the molten pool and the deposited layer obtained in step S1, the process parameters of laser additive manufacturing and the size of the substrate in the experiment, a sequential thermal-mechanical coupling finite element model is established. In the sequential thermal-mechanical coupling finite element model, each layer uses death and death units to simulate the deposition process of powder particles. At the beginning of the simulation, the death unit will not affect the thermo-mechanical model. With the movement of the set simulated laser beam heat source, the constructed geometric model will be activated unit by unit and layer by layer. The shrinkage value of the deposited layer is loaded as a load on the real-time model and the stress value of the point is calculated. The calculation result of each step is iteratively updated to obtain the stress value, and the required unit node is selected to export and process the stress data; S3: Establish a convolutional neural network model 1 for predicting real-time stress: The convolutional neural network needs to fuse the molten pool, the geometric morphology of the deposited layer, the shrinkage data of the deposited layer, the stress values ​​of each point and the process parameters obtained in the above steps S1 and S2 as the input of the convolutional neural network, divide the training data set and the test data set, output the real-time stress value of a certain point in the manufacturing process of the component, and evaluate the trained model; S4: Establish a convolutional neural network model 2 for predicting component status: The convolutional neural network model 1 established through the above step S3 predicts the stress values ​​of a large number of samples during the manufacturing process in real time, and processes them into stress curves through computers. The manufactured samples are of four types: normal, pre-cracked, cracked, and deformed. A one-to-one correspondence is established between the deposition layer morphologies of these four states and their stress curves, which are used as the input of the convolutional neural network for predicting component status. The training data set and the test data set are divided, and the output is the state of the component during the manufacturing process.

2. The method for online real-time prediction of component status based on deep learning according to claim 1, characterized in that: The step S3 of establishing the convolutional neural network model 1 for predicting real-time stress is to fuse the data collected in steps S1 and S2. The data collected from the three sources of machine, vision, and finite element must be integrated into a comprehensive data set to be used as the training features and targets of the prediction model. All collected signals must be synchronized with the shared timestamp to establish a one-to-one correspondence.

3. The method for online real-time prediction of component status based on deep learning according to claim 1, characterized in that: The convolutional neural network model 1 for predicting real-time stress described in step S3 adopts a hybrid network, which is implemented by stacking two neural networks, a convolutional network and a multi-layer neural network, wherein the convolutional network includes 2 convolutional layers, 2 pooling layers, 1 Dropout layer, and 2 fully connected layers from input to output. The convolutional network is used to process the image data of the molten pool and the deposited layer, and extracts different feature data of the image through convolution calculation. All convolutional layers are equipped with Relu activation functions. The pooling layer cuts the convolutional features into several regions, takes the maximum value or average value, and obtains new, small-dimensional features. The Dropout layer is used to avoid overfitting of the model. During the training phase of the model, part of the hidden neurons are allowed to work and part are not allowed to work. The fully connected layer is used to connect the image features to obtain the final image features of the molten pool and the deposited layer; the multi-layer neural network includes 1 input layer, 2 hidden layers, and 1 output layer from input to output. The parameters input by the input layer are the image features of the molten pool and the deposited layer, respectively, and the laser power P and scanning rate V in the process parameters, stress, and the real-time stress value of a certain point in the component manufacturing process is output through two hidden layers.

4. The method for online real-time prediction of component status based on deep learning according to claim 1, characterized in that: The predicted real-time stress convolutional neural network model 1 described in step S3 uses the mean square error MSE to compare and evaluate, that is, the predicted real-time stress value and the actual value. The mean square error MSE is defined as: In the formula, n is the number of model samples, i is the data sample of each, and f i is the predicted value of stress at a certain point in the model, y i is the actual stress value at a certain point in the model.

5. The method for online real-time prediction of component status based on deep learning according to claim 1, characterized in that: The convolutional neural network model 2 for predicting the state of the component described in step S4 is established. The convolutional neural network model 1 established in step S3 is required to predict the stress values ​​of a large number of samples in the manufacturing process in real time, and process them into stress curves through computers. The manufactured samples are of four types: normal, pre-cracked, cracked, and deformed. A one-to-one correspondence is established between the deposition layer morphologies of these four states and their stress curves, which are used as the input of the convolutional neural network for predicting the state of the component, and the training data set and the test data set are divided to output the state of the component during the manufacturing process.

6. The method for online real-time prediction of component status based on deep learning according to claim 1, characterized in that: Step S4 establishes a convolutional neural network model 2 for predicting component status, which includes 2 convolutional layers, 2 pooling layers, and 2 fully connected layers from input to output. At the same time, all convolutional layers are equipped with Relu activation functions. In order to speed up the training and convergence of the network, improve the network gradient and prevent overfitting, a BN layer is added after each convolutional layer; in addition, the Softmax classifier and the cross entropy loss function are used in the final output layer to calculate the probability of each classification of the output value, and the maximum probability is taken as the test result to obtain the final state classification, and the performance of the network is optimized and evaluated.

7. The method for online real-time prediction of component status based on deep learning according to claim 1, characterized in that: The neural network model 2 established in step S4 is to deeply evaluate the model, construct a confusion matrix, and evaluate the false positive and false negative rates of the confusion matrix with precision Precision and recall Recall. The calculation formula is as follows: Among them, TP means that a sample is predicted to be a certain category, FP means that a sample does not belong to a certain category but is predicted to be a certain category, and FN means that a sample belongs to a certain category but is not predicted to be a certain category.

Images