Intelligent control method for laser metal deposition process

Abstract

The invention discloses an intelligent control method for a laser metal deposition process, and relates to the technical field of laser metal deposition, the method comprises digital twins capable of representing mapping relations between LMD process parameters and LMD process states and between the LMD process parameters and LMD forming results, and a reinforcement learning training environment is constructed in combination with a reward function. Then, two LMD process control intelligent agents are constructed to be used as a process parameter optimizer in the LMD process intelligent control system and a state controller of the deposition process respectively; the LMD process intelligent control system is a cascade control system and comprises an outer control loop and an inner control loop, and the outer control loop is an LMD forming process staged task planning method based on imitation learning. The intelligent agent constructed by the method can simulate an expert mode of technological parameter optimization to plan technological parameters of the next stage according to the current forming result, and it is guaranteed that the forming process develops towards the expected direction; the inner control loop is an LMD process control method based on reinforcement learning, and stability of the LMD forming process under various working conditions is guaranteed. Therefore, the control system realizes substantial improvement of shape control and performance control capability by effectively regulating and controlling main process parameters.

Description

Technical Field

[0001] The invention relates to the technical field of laser metal deposition, and in particular to an intelligent control method for a laser metal deposition process. Background Art

[0002] Laser Metal Deposition (LMD) is an advanced manufacturing technology that builds three-dimensional objects by adding layers one by one, converting complex three-dimensional forming into two-dimensional stacking forming, and greatly reducing the complexity of production by reducing dimensions. However, due to the temperature variation characteristics of material properties and the uncertainty of certain factors in the LMD working environment, the LMD forming results are unstable and have low repeatability. Therefore, it is necessary to implement closed-loop control of the LMD process to reduce the impact of these factors.

[0003] After searching the publication (announcement) number: CN111545937A, a laser metal additive deposition temperature control method and system are disclosed. The laser metal additive deposition temperature control method includes: obtaining multiple current temperatures when the powder feeding nozzle deposits the current deposition layer with the current laser power; the multiple current temperatures are obtained by collecting the temperature of the deposits ejected by the powder feeding nozzle within the set range during the formation of the current deposition layer; calculating the average temperature of the current deposition layer according to the multiple current temperatures; calculating the next layer laser power of the powder feeding nozzle according to the expected temperature, the average temperature of the current deposition layer and the average temperature of the previous deposition layer, and then using the next layer laser power as the current laser power, and returning to the acquisition of multiple current temperatures when the powder feeding nozzle deposits the current deposition layer with the current laser power until the deposition is completed. This invention improves the control accuracy of the laser metal additive deposition temperature.

[0004] Although the above invention improves the control accuracy of laser metal additive deposition temperature, the currently used LMD process control method basically controls one or two LMD process states, making it difficult to cope with more complex situations in the LMD process, and its shape control and controllability capabilities are relatively weak. For this reason, we propose an intelligent control method for laser metal deposition process. Summary of the invention

[0005] The main purpose of the present invention is to provide an intelligent control method for laser metal deposition process, which can effectively solve the problem of relatively complex situations in the background technology LMD process and its weak shape control and property control capabilities.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is: a method for intelligent control of laser metal deposition process, including establishing a digital twin that can characterize the mapping relationship between LMD process parameters and LMD process states and LMD forming results, and then building a reinforcement learning training environment in combination with a reward function, and then building two LMD process control agents for process parameter optimization and deposition process state control respectively. These two agents constitute the outer loop and inner loop of the LMD process intelligent control system.

[0007] The outer loop is an LMD phased task planning method based on imitation learning; the inner loop is an LMD process control method based on reinforcement learning. The two control loops use different control frequencies. The outer layer optimizes the process parameters for at least one deposition process; while the control frequency of the inner layer is the same as the sampling frequency of the process detection system.

[0008] Preferably, the LMD forming process detection system composed of a set of coaxial molten pool detection and a set of three-dimensional detection devices provides the control system with necessary forming process data.

[0009] Preferably, the coaxial melt pool detection device adopts a coaxial optical path as the main structure, a near-infrared camera and a pyrometer as the basic detection equipment, and an FPGA operation acceleration card for melt pool image processing and agent reasoning. Even if the sampling rate of the detection device is as high as 10kHz, the real-time performance of the image processing algorithm and reasoning operation can be guaranteed. The melt pool image processing refers to the digital image processing process of segmenting the melt pool area from the melt pool image and extracting relevant information such as the pixel value and morphology of the melt pool area. The specific steps involve image segmentation, morphological operation, image moment operation, etc. The so-called agent reasoning refers to a pre-trained agent giving a corresponding control output according to the process state data input in real time.

[0010] Preferably, the three-dimensional detection device is a line structured light scanner using a straight line laser as structured light and an industrial camera as a main detection component. Figure 2After installation in the manner shown, the system needs to be calibrated to obtain the lens distortion parameters and the parameters of the participating light planes inside and outside the camera. After calibration, the scanner can achieve a detection accuracy of up to 0.01mm. It is also characterized in that the point cloud reconstructed by the scanner can convert the deposition height and be combined with the molten pool data as the basic data for training the digital twin. The so-called point cloud reconstruction refers to the method of extracting structured light lines from structured light images, converting the structured light lines into a group of points in 3D space, and then arranging each group of points in the order of collection and spatial intervals to obtain a group of point clouds of the object being measured. The deposition height arranged according to the processing path can be obtained by resampling the group of point clouds according to the processing path, and then the deposition height can be linked to the deposition time according to the scanning speed of the laser processing head, and finally the deposition height corresponding to the molten pool data can be obtained.

[0011] Preferably, the digital twin construction of the laser metal deposition process consists of a forming process state prediction model and a forming result prediction model, and the output end of the state model is connected to the input end of the result model, so as to achieve the purpose of predicting the forming process state and forming result based on process parameters.

[0012] Preferably, the forming process state prediction model can be constructed based on models including but not limited to PatchTST, Timer, GPT2, etc. The main function of the model is to predict the parameters in the forming process including but not limited to the molten pool temperature, molten pool size and the like that directly reflect the forming process state based on given process parameters, and the input and output vector dimensions of the model are determined according to the process parameters and the number of states involved in the training data set.

[0013] Preferably, the forming result prediction model can be constructed based on models including but not limited to GPT2, Bert, etc. The main function of the model is to predict the forming result based on given process parameters and forming process states, including but not limited to parameters such as density and deposition height that directly reflect the forming result, and the input and output vector dimensions of the model are determined according to the process parameters, number of states and types of forming results involved in the training data set. Since process data is difficult to obtain, a pre-trained large model with good generalization such as GPT2 and Bert is used, and better representation can be obtained by training with only a small amount of data.

[0014] Preferably, the process parameter optimization in the process adopts the imitation learning method to construct a parameter optimization agent, so that it can learn and imitate the expert's process parameter design mode, so as to provide an optimized process parameter adjustment plan according to the current forming result. The imitation learning is a reinforcement learning method that enables the agent to learn and imitate the target mode. Imitation learning can project the process parameters into the expert mode of designing process parameters, and establish a mapping from forming requirements to optimal process parameters by dimensionality reduction. This method can narrow the scope of the agent's search action strategy, thereby improving the efficiency of training the agent.

[0015] Preferably, the molten pool temperature control is a method using a reinforcement learning agent as a molten pool temperature controller. The reinforcement learning agent is a network model of a performer-critic structure, which learns the control strategy of the molten pool temperature in an environment based on the digital twin structure. Its adaptability comes from the representation of various working conditions by the digital twin, and it is an effective method to solve the nonlinear control problem of molten pool temperature control.

[0016] Preferably, the laser metal deposition process anomaly detection is a forming process anomaly detection method based on digital twins, which is used to detect the abnormal state and type of the forming process, and can also be used to ensure the safety of the LMD process intelligent control system. The model uses cross entropy as the basis for judging whether an anomaly exists, and judges the type of anomaly based on the state characteristics of the forming process.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. In the present invention, a digital twin that can characterize the mapping relationship between LMD process parameters and LMD process states and LMD forming results is first constructed, and then a reinforcement learning training environment is constructed in combination with a reward function. Then, two LMD process control agents are constructed for process parameter optimization and deposition process state control, respectively. A cascade control system containing two inner and outer loops is constructed by these two agents, so that more process parameters can be incorporated into the control system. The shape control and controllability capabilities of the LMD equipment are effectively improved by real-time optimization of these process parameters.

[0018] 2. In the present invention, a method for detecting anomalies in the LMD process is designed. The method uses cross entropy (CE) to detect the reconstruction error between the measured process state and the estimated value of the digital twin. When CE is greater than a given threshold, it is determined that an anomaly has occurred. At the same time, the method also determines the type of anomaly based on information such as the molten pool temperature, molten pool size, and molten pool image moment actually measured in the LMD process, so that the control system can execute the corresponding anomaly handling process. The method can prevent damage to the LMD equipment or safety accidents caused by anomalies in the LMD process, and can also be used as a bottom-line strategy to ensure that the control agent works within a safe range. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the principle diagram of the LMD intelligent control system of the present invention; Figure 2 A flow chart of the coaxial observation system and the topography scanning device for the LMD process established for the present invention; Figure 3 Schematic diagram for constructing and training the LMD process digital twin for the present invention; Figure 4 A flow chart of establishing an LMD process anomaly detector for the present invention; DETAILED DESCRIPTION

[0020] like Figure 1 , Figure 2 , Figure 3 as well as Figure 4 As shown, the present invention is an intelligent control method for laser metal deposition process, which constructs a two-layer closed-loop control loop based on the principle of cascade control; the method first constructs a digital twin that can characterize the mapping relationship between LMD process parameters and LMD process states and LMD forming results, and then constructs a reinforcement learning training environment in combination with a reward function, and then constructs two LMD process control agents for process parameter optimization and deposition process state control respectively.

[0021] As a key technology for intelligent control of the LMD process, an LMD process intelligent control system similar to the cascade control system was constructed. The cascade control system contains two layers of control loops, the outer layer is used for process parameter optimization, and the inner layer is used to control the deposition process state; the outer layer control loop adopts the LMD phased task planning method based on imitation learning; the inner layer control loop adopts the LMD process control method based on reinforcement learning; the two layers of control loops use different control frequencies, the outer layer optimizes the process parameters for at least one deposition process; and the inner layer control frequency is the same as the sampling frequency of the process detection system.

[0022] As a key technology for intelligent control of LMD process, a digital twin of LMD process is constructed. Digital twin is a deep learning model that can predict the state and forming results of LMD process. The digital twin is mainly composed of a time series prediction model based on PatchTST and a time series prediction model based on GPT2. Figure 2The deposition process model and deposition result model in the paper are as follows; the deposition process model is a PatchTST model, which estimates the state of the deposition process at different times according to the process parameters; the deposition process state refers to the measurable parameters of the LMD process; the deposition result model is a GPT2 model, which estimates the deposition process results according to the process parameters and the deposition process state. The average square of the estimation error can be used as the objective function, and the adaptive moment estimation (Adam) is used to search for the optimal model parameters.

[0023] The LMD process detection method includes morphology scanning and coaxial melt pool detection. Coaxial melt pool detection refers to constructing an optical path device with the same optical axis as the high-energy laser so that the pyrometer and the near-infrared camera can always maintain a top-view observation of the melt pool state in the laser action area, which can ensure that there is no obstruction during observation; the melt pool image uses a threshold-based method to extract the melt pool area, and then calculates its size, image moment, etc.; the morphology scanner refers to a 3D point cloud detection device that uses a line laser as structured light and a camera as a structured light detector. A set of calibration and measurement software has been developed for the device, so that the key parameters of the device can be obtained through calibration after reinstallation, thereby realizing accurate measurement of the point cloud of the object being measured; its detection cycle is a deposition process of one or several layers, while the sampling frequency of the pyrometer and the camera is higher and consistent.

[0024] LMD phased task planner based on imitation learning, LMD phased task planning refers to a method of planning the forming target of the next stage according to the previous forming results, and then adjusting the processing path and process parameters of the next layer or several layers according to the target. The method gradually plans the forming target and optimizes the process parameters so that the forming morphology and mechanical properties gradually approach the design indicators. Imitation learning introduces a discriminator containing expert mode as the reward function of agent training on the basis of reinforcement learning, so as to narrow the search range of the agent strategy and improve the training efficiency. The discriminator refers to a deep learning model composed of a deep convolutional network that can learn the expert mode when setting the LMD process parameters from the expert data set. The trained discriminator can determine whether the process parameters generated by the agent conform to the expert mode and give the corresponding reward value. The task planner contains a reinforcement learning agent, which consists of an actor network and a critic network. Both the actor and the critic network are constructed by LSTM networks, and SAC (Soft Actor-Critic) is used as the strategy training algorithm of the agent. The actor network is used to generate actions that can interact with the reinforcement learning training environment, and the critic network evaluates the output actions of the actor network according to the reward value given by the discriminator and the environmental state. Finally, after training, the agent can imitate the expert mode to generate process parameters suitable for the next stage of deposition.

[0025] A reinforcement learning-based LMD process control method, in which a pre-trained agent directly controls the LMD process, and the agent is trained in a reinforcement learning environment based on the LMD process digital twin. The input vector of the LMD control agent includes LMD process parameters in addition to the molten pool temperature and the molten pool size, which can describe both the current LMD process state and the current working conditions of the LMD; the output of the agent is one or several process parameters that the LMD process can respond quickly, and the adjustable range of these parameters determines the action space of the agent, and these parameters can generally be adjusted continuously, so the present invention uses PPO (Proximal Policy Optimization) to train the agent.

[0026] A method for detecting anomalies in an LMD process is proposed. The method uses cross entropy (CE) to detect the reconstruction error between the measured process state and the estimated value of a digital twin. When CE is greater than a given threshold, it is determined that an anomaly has occurred. At the same time, the method also determines the type of anomaly based on information such as the molten pool temperature, molten pool size, and molten pool image moment actually measured in the LMD process, so that the control system can execute the corresponding anomaly handling process. The method can prevent damage to the LMD equipment or safety accidents caused by anomalies in the LMD process, and can also be used as a bottom-line strategy to ensure that the control agent works within a safe range. Specific implementation steps

[0027] S1: Establishing the coaxial observation system and topography scanning device of LMD process 1) Design the transmittance and reflectivity required by the dichroic mirror in the coaxial optical path and the passband window of the narrow-band filter according to the laser wavelength, the band of the molten pool thermal radiation, the detection band of the camera, and the detection band of the pyrometer.

[0028] 2) Design and manufacture the coaxial optical path structure, install the dichroic mirror, reflector, filter, pyrometer, and camera. During the installation process, fine-tune the orientation of the pyrometer and camera so that the center of their field of view coincides with the guide laser point.

[0029] 3) Set the camera’s windowing parameters, sampling rate, and exposure parameters, and then use the threshold optimization method to obtain the optimal threshold for segmenting the molten pool area.

[0030] 4) Install the topography scanner on the laser cladding processing head, adjust the projection direction of the line laser, adjust the direction and orientation of the camera field of view, and calibrate the camera's internal and external parameter matrices and light plane parameters.

[0031] 5) End S2: Construct and train the LMD process digital twin: 1) Collect LMD process data and process experimental data, and construct the corresponding data set.

[0032] 2) Design the corresponding input layer according to the input vector length of the process model and the result model. The input vector of the process model is the process parameter, while the result model needs to take the process parameter and the process state output by the process model as input.

[0033] 3) Based on PatchTST and GPT2, the hidden layers of the process model and the result model are constructed respectively, the corresponding model structure parameters are designed, and a fully connected layer is added at the end to extract the output features. Finally, a regression layer is used as the output layer to estimate the LMD state or forming result at the next moment from the features.

[0034] 4) Use the Adam optimization algorithm to train the model, and set appropriate parameters such as learning rate and number of training cycles to make the training converge.

[0035] 5) Save the model with the smallest loss value.

[0036] 6) End The types of process parameters used as input parameters in the model can be selected according to the size of the data set. If the existing data is small, only the types that have a significant impact on the process state and forming results of LMD can be selected to train a model with a relatively single applicable scenario. Then, as the data accumulates, the types of process parameters used as input can be gradually increased, and the new model can be trained to make it applicable to more scenarios. In this way, the generalization of the model can be gradually improved.

[0037] S3: Building an LMD process anomaly detector 1) Design the LMD experiment to make the predetermined form of anomaly appear in the experiment without damaging the equipment, and construct the anomaly data set using the process data recorded by the pyrometer and camera.

[0038] 2) Input the same process parameters into the digital twin and record its output, calculate the cross entropy between the twin output and the measured process data, and then merge the measured data, cross entropy, and corresponding anomaly types to form a process anomaly data set.

[0039] 3) Use the abnormal data set to fine-tune the pre-trained process model, and use the abnormal type as the fitting target during training. Set the learning rate and number of training cycles of the Adam optimization method to make the training process converge.

[0040] 4) Save the model with the smallest loss value.

[0041] 5) End S4: Building an LMD phased task planner based on imitation learning Assume that the workpiece to be formed has been divided into layers in the computer-aided manufacturing (CAM) software, and every N layers is a deposition stage, and the topography is scanned once each deposition stage is completed.

[0042] 1) Select ideal forming results from the process experiment data to construct the LMD process expert dataset. Construct a discriminator based on a deep convolutional network to learn the expert mode in the expert dataset. The discriminator consists of an input layer, three linear layers, two ReLU layers, and a Sigmoid layer. The ReLU layer is sandwiched between the three Linear layers, and the Sigmoid layer is used as the output layer.

[0043] 2) Use the Adam algorithm to train the discriminator and optimize the learning rate and number of training cycles to make the training process converge.

[0044] 3) Before the measurement data is input into the actor network, the measured forming morphology is compared with the expected morphology to calculate the forming error, and the forming morphology of the subsequent N layers is added as the forming target for the next stage.

[0045] 4) Build a task planning agent, use two convolutional networks to build the actor network and critic network and their loss functions, and initialize them.

[0046] 5) Initialize the experience buffer area that stores the interaction data between the agent and the environment, and initialize the entropy temperature coefficient α of the randomness of the control strategy.

[0047] 6) Start training and set the initial action of the actor (i.e. a set of process parameters), the digital twin in the environment will predict the process state and forming results of a deposition stage under this setting, and combine these empirical data (state , next state , reward R, action ) and stored in the cache.

[0048] 7) Randomly sample M experience data from the cache as the training data of the policy network and value network, and use the value loss function, policy loss function and entropy weight function as the objective functions of the value network and policy network respectively, and then use the Adam algorithm to optimize their parameters to minimize the objective function value.

[0049] 8) Return to step 6 to start a new round of training. The training is terminated when the scheduled training cycle is completed or the reward value is greater than a given threshold.

[0050] 9) End.

[0051] S5: Building a reinforcement learning based LMD process controller 1) Based on the LMD process model in the digital twin, the OpenAI Gym specification is used to establish the observation state vector and reward value function of the process model, and to construct a training environment for training the LMD process control agent.

[0052] 2) Construct a control agent whose actor network and critic network are composed of two LSTM models respectively. Initialize the actor network and critic network, as well as their optimizers and loss functions.

[0053] 3) Initialize an experience replay buffer to store data generated by the interaction between the agent and the environment.

[0054] 4) According to the initial strategy Output an action Give the environment. The experience data (state , reward R, action etc.) into the buffer area.

[0055] 5) Randomly sample a set of data from the cache, and the critic network calculates the value of the state , and use the Bellman equation to calculate the advantage of an action ; Use the actor network to calculate the probability of each action and compare it with the old probability stored in the buffer to get the importance sampling ratio , calculate the clipping loss value.

[0056] 6) Use the Adam algorithm to update the parameters of the actor network and the critic network. The actor network then outputs new actions to the environment and stores the new states, rewards, and actions in the buffer. Return to step 4 to start a new round of training. The training is terminated when the scheduled training cycle is completed or the reward value is greater than the given threshold.

[0057] 7) End.

[0058] S6: Complete the construction of LMD intelligent control system by combining the above modules.

[0059] S7. End.

[0060] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An intelligent control method for a laser metal deposition process, comprising an LMD forming process control system and a learning environment of an intelligent agent constructed based on a digital twin of the LMD forming process, characterized in that: It also includes the detection of the molten laser metal deposition process, the construction of a digital twin of the laser metal deposition process, the optimization methods of process parameters during the process, the temperature control of the molten pool, and the safety assurance of the intelligent control of the laser metal deposition process; The LMD forming process control system includes a molten pool temperature control loop and a process parameter optimization loop, and two reinforcement learning agents are used as controllers in each of the two loops; The reinforcement learning agent refers to a neural network model that can learn a certain action generation strategy in an environment. The environment can refer to a physically existing system or thing, or a virtualized system simulated by software. The action refers to the output generated by the agent that can interact with the environment based on the state of the environment, and the action generation strategy refers to the action generation pattern learned by the agent based on the feedback characteristics of the environment's reward value.

2. The intelligent control method for laser metal deposition process according to claim 1, characterized in that: This method constructs a LMD forming process detection system that includes a coaxial molten pool detection device and a three-dimensional detection device to provide the control system with necessary forming process data.

3. The intelligent control method for laser metal deposition process according to claim 2, characterized in that: The coaxial molten pool detection device adopts a high-speed camera and a colorimetric pyrometer as basic detection equipment, and adopts an FPGA operation acceleration card to perform molten pool image processing and control quantity calculation, thereby ensuring the real-time performance of the image processing algorithm and the control algorithm.

4. The intelligent control method for laser metal deposition process according to claim 2, characterized in that: The three-dimensional detection device is a line structured light scanner which uses a straight line laser as structured light and an industrial camera as a main detection component.

5. The intelligent control method for laser metal deposition process according to claim 1, characterized in that: The laser metal deposition process digital twin construction consists of a forming process state prediction model and a forming result prediction model, and the output end of the state model is connected to the input end of the result model.

6. The intelligent control method for laser metal deposition process according to claim 5, characterized in that: The forming process state prediction model can be constructed based on a time series prediction model including but not limited to PatchTST, Timer, GPT2 and other models.

7. The intelligent control method for laser metal deposition process according to claim 1, characterized in that: The forming result prediction model can be constructed based on models including but not limited to GPT2 and Bert.

8. The intelligent control method for laser metal deposition process according to claim 7, characterized in that: The process parameter optimization in the process is achieved based on the imitation learning method. This method learns and imitates the expert's process parameter design mode through the constructed parameter optimization intelligent agent, so that it can give an optimized process parameter adjustment plan according to the current forming results.

9. The intelligent control method for laser metal deposition process according to claim 1, characterized in that: The molten pool temperature control adopts a reinforcement learning agent as a controller of the molten pool temperature to keep the molten pool temperature stable at a given level.

10. The intelligent control method for laser metal deposition process according to claim 1, characterized in that: The abnormality detection method of the laser metal deposition process is based on a deep learning network to construct a method for abnormal state detection and classification of the forming process, which is a safety guarantee for intelligent control.

Images