An intelligent self-decision-making molten pool monitoring system

Through the intelligent self-decision melt pool monitoring system, Fourier transformation and deep reinforcement learning are used to realize real-time monitoring and quality control of the welding process, solving the problem that the melt pool status is difficult to monitor in real-time during welding, and improving welding quality and efficiency.

CN113579545BActive Publication Date: 2024-11-15AEROSPACE INTELLIGENT MFG (SHANGHAI) TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110952487.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-19
Publication Date
2024-11-15
Estimated Expiration
2041-08-19

AI Technical Summary

Technical Problem

In modern industrial manufacturing, the lack of an effective online melt pool monitoring system during welding process makes it difficult to control the welding quality in real time and cannot adapt to changes in different materials and temperature environments.

Method used

An intelligent self-decision molten pool monitoring system is designed, including a front-end perception module, a molten pool image processing module, a molten pool state analysis module and a decision learning module. A high dynamic range image is synthesized using the Fourier transform-based gradient reconstruction image method. Combined with deep reinforcement learning, real-time monitoring of the melt pool state and automatic parameter adjustment are achieved.

Benefits of technology

Real-time monitoring and quality control of the welding process are achieved, and the loss of melt pool image details caused by high temperature is avoided. The optimal welding strategy can be formed under different materials and temperature environments, which improves welding quality and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113579545B_ABST
    Figure CN113579545B_ABST
Patent Text Reader

Abstract

The present invention discloses an intelligent self-decision molten pool monitoring system, including a front-end perception module, a molten pool image processing module, a molten pool state analysis module, a decision learning module and a control module. The present invention designs a hardware image acquisition device required for synchronous acquisition of a high dynamic range image of a molten pool image, integrates and implements a method of reconstructing an image from a gradient based on Fourier transform, utilizes the molten pool state and welding parameters represented by the molten pool depth feature, takes the molten pool state as a signal feedback, performs SAC reinforcement learning, and aims to maximize the target image matching degree. It can imitate the trial and error experience learning of manual welding, and perform optimal welding strategy learning of the interaction between the welding operation strategy and the molten pool state environment. It is suitable for online monitoring of molten welding quality control under different parameter environments, and can execute the welding strategy with the optimal welding quality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of welding online control, and in particular to an intelligent self-decision-making molten pool monitoring method. Background Art

[0002] Welding plays an indispensable role in modern industrial manufacturing and is an important business process. The evaluation of welding quality is also crucial to the control of the entire welding process. An effective means to reflect welding quality is to judge the state of the molten pool during welding. There are many kinds of information on the state of the molten pool. Among them, visual information that perceives the state of the molten pool in a non-contact manner can intuitively and effectively reflect the state of the molten pool during the welding process. Therefore, based on visual information, the use of visual monitoring of the welding process will be one of the future development directions.

[0003] During the welding process, welders usually need to estimate the welding quality based on past experience according to the status information of the molten pool, arc, etc., so as to adjust the parameters to achieve real-time welding control and ensure the welding quality. That is, welders have formed a set of experience through long-term welding operations, through the state of the molten pool during welding, and the evaluation of the welding quality after the welding is completed under this molten pool state. After continuous adjustment of strategies, they can reach the level that senior welders can predict the welding quality by observing the molten pool state. However, driven by the development of modern automation and intelligent industrial manufacturing, intelligent and automatic welding equipment will gradually replace manual welding to achieve efficient and stable additive manufacturing welding operations. Therefore, in the field of online sensing and intelligent control of welding quality, an intelligent molten pool monitoring system is urgently needed, which can realize the molten pool monitoring during the welding process of automatic welding equipment such as welding robots, timely evaluate the welding quality according to the molten pool state, and adjust the parameters to achieve welding quality control. Summary of the invention

[0004] The problem to be solved by the present invention is to provide an intelligent self-decision-making molten pool monitoring system.

[0005] To achieve the above object, the technical solution of the present invention is:

[0006] An intelligent self-decision-making molten pool monitoring system comprises a front-end perception module, a molten pool image processing module, a molten pool state analysis module, a decision-making learning module and a control module.

[0007] The front-end perception module is used to collect and preprocess the image information of different light and dark areas in the molten pool.

[0008] The molten pool image processing module is used to synthesize the collected and processed image data into a molten pool high dynamic range image to display the brightness detail characteristics of the molten pool.

[0009] The molten pool state analysis module is used to extract features from the obtained molten pool image, obtain the molten pool state data set to form input parameters, and provide welding machine action reward stimulation according to the target molten pool state.

[0010] The decision learning module is used to output the execution strategy according to the molten pool state and welding machine parameters, and receive the molten pool state feedback reward signal, independently judge the pros and cons of the molten pool changes caused by the output welding strategy, and execute the strategy that can generate the greatest value under the current state.

[0011] The control module is used to receive the control strategy of the decision learning module and control the welding machine to perform corresponding welding operations according to the strategy.

[0012] Preferably, the front-end perception module adopts a hardware camera device suitable for the molten pool image processing module, and the hardware camera device includes a power supply system, a clock system, a protection system, an image sensor and an HDR image signal processor.

[0013] The power supply system is used to output voltages at various levels to provide power for the hardware. It outputs voltages at various levels through a switching power supply or a low-voltage difference linear regulated power supply, adapts to the standard voltage levels of each chip, and meets the power-on and power-off timing to ensure the normal operation of each hardware part.

[0014] The clock system provides internal clock signals for other hardware parts, serving as control signals for the coordinated work of various components within the chip.

[0015] The protection system is used to protect the hardware from damage. Its main function is to notify the HOST when a short circuit occurs in a part of the hardware or the internal chip temperature is too high, and take corresponding measures such as: cutting off the power supply; stopping the chip operation, etc. to protect the device safety, and resume normal operation when the fault is removed.

[0016] The image sensor is used to collect image information and transmit the information to the HDR image signal processor.

[0017] The HDR image signal processor is used to synthesize the melt pool HDR image and perform advanced noise reduction.

[0018] Preferably, the image sensor is controlled by two shutter pointers, which control the integration of odd and even row pairs in a single frame of data. The shutter pointers are used to capture odd and even row pairs of single frame data at different times, and then integrated and output frames.

[0019] Preferably, the HDR image signal processor communicates with the molten pool state analysis module, the decision learning module, and the control module via a two-wire synchronous serial bus.

[0020] Preferably, the molten pool image processing module adopts the Fourier transform-based image reconstruction method from gradient to synthesize a high dynamic range image of the molten pool, which can reflect the brightness details of the molten pool image and avoid the loss of features due to the local area being too bright or too dark due to the high temperature of the molten pool, thereby more accurately analyzing and processing the molten pool state.

[0021] Preferably, the steps for synthesizing a high dynamic range image of the molten pool by reconstructing images from gradients based on Fourier transform are: loading a high-exposure image and a low-exposure image, respectively calculating the gradient vector field of each image, and then taking the maximum gradient point of the partial derivative image of the direction corresponding to the gradient vector field of each image to synthesize a new gradient, using the algorithm of the closed-form solution obtained by Fourier transform, and synthesizing an HDR image with an expanded brightness range.

[0022] Preferably, the molten pool state data set obtained in the molten pool state analysis module includes molten pool state information and corresponding welding parameters.

[0023] Preferably, the decision learning module adopts deep reinforcement learning to learn strategies through the interaction between the welding machine itself and the monitored molten pool state.

[0024] Preferably, the decision learning module performs continuous trial and error action feedback based on the characteristics of the molten pool image, and the reward and punishment mechanism makes decisions on the welding quality to form a set of optimal execution strategies for each state in the welding process.

[0025] The molten pool image processing module implements the image reconstruction method from gradient based on Fourier transform, synthesizes the high dynamic range image of the molten pool, which can reflect the brightness details of the molten pool image and avoid the loss of features due to the high temperature of the molten pool, resulting in the local area being too bright or too dark, so as to more accurately analyze and process the molten pool state. The high dynamic camera designed based on the front-end perception module can collect and process images with significant image features, monitor the molten pool state in real time, and then perform deep CNN image algorithm processing of the molten pool state analysis module to extract the deep features of the molten pool image; the decision learning module adopts deep reinforcement learning, and learns strategies from experience through the interaction between the machine itself and the monitored molten pool state. The machine affects the environmental molten pool state and obtains feedback information by executing welding actions. Through the stimulation of rewards and punishments, the machine gradually learns to respond to stimuli, thereby generating actions that can maximize benefits based on the state. The method of learning while obtaining samples is adopted. The high dynamic range synthetic molten pool image is used as the sample input, the model is updated and the welding operation is performed to obtain a new molten pool state. Feedback is obtained based on the welding quality corresponding to the molten pool state, and the model is further updated. It is continuously iterated until a set of optimal welding execution strategies are formed from the experience of continuous welding, molten pool state changes and feedback on welding quality. The deep features of the collected and pre-processed molten pool high dynamic range synthetic images are trained to achieve feature extraction and continuous trial and error action feedback based on the features, as well as the reward and punishment mechanism to achieve decision-making control of welding quality, forming a set of optimal execution strategies for each state in the welding process; the control module is the action execution mechanism, which controls the welding machine to perform welding according to the strategy provided by the decision learning module.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] (1) The present invention implements a Fourier transform-based image reconstruction method from gradients to synthesize a high dynamic range image of the molten pool, which can avoid the loss of detailed features of the molten pool image due to the high temperature of the molten pool.

[0028] (2) The present invention integrates the molten pool high dynamic range image synthesis algorithm into the front-end camera hardware, which can realize efficient and fast molten pool high dynamic range image acquisition, perceive the molten pool status in real time, and lay the foundation for subsequent intelligent decision-making learning.

[0029] (3) Compared with first welding to obtain a molten pool image and then predicting the welding quality, the present invention uses online monitoring of the molten pool status and real-time decision-making to monitor while welding, obtain the molten pool status based on the molten pool image, and then control the welding to achieve good welding quality.

[0030] (4) The present invention adopts reinforcement learning to achieve welder-like intelligence, and trains and learns welding execution strategies from the continuous trial and error experience of the molten pool state with the best welding quality. There is no need to manually mark samples, and the machine can perform self-interpretation, using high dynamic range synthetic molten pool images as feedback states to achieve optimal decisions. It can make decisions on welding execution based on the current molten pool state, and can be applied to any welding parameter situation, automatically adjusting parameters according to the molten pool state to achieve the best welding quality.

[0031] (5) The present invention is applicable to different materials and can form an optimal welding execution strategy based on experience learning strategies according to different temperature environments and corresponding changing conditions of the molten pool. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0033] In the attached picture:

[0034] Figure 1 This is a hardware block diagram of the front-end perception module in the present invention;

[0035] Figure 2 Obtaining odd and even row pairs schematic diagram for the front-end perception module image sensor;

[0036] Figure 3 Algorithm flow chart for high dynamic range melt pool image synthesis;

[0037] Figure 4 This is a schematic diagram of the overall system framework. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0039] like Figure 1-Figure 4 As shown, in this embodiment, an intelligent self-decision-making molten pool monitoring system is disclosed, which is used to control the molten pool monitoring of a fusion welding machine to perform high-quality welding operations, and includes a front-end perception module, a molten pool image processing module, a molten pool state analysis module, a decision learning module and a control module.

[0040] Among them, the front-end perception module is used to collect different light and dark image information in the molten pool and pre-process it. The molten pool image processing module is used to synthesize the collected and processed image data into a high dynamic range image of the molten pool to display the detailed characteristics of the brightness of the molten pool. The molten pool state analysis module is used to extract features from the obtained molten pool image, obtain the molten pool state data set to form input parameters, and the molten pool state data set specifically includes the molten pool state information and corresponding welding parameters, and at the same time give the welding machine action reward stimulus according to the target molten pool state. The decision learning module is used to output the execution strategy according to the molten pool state and welding machine parameters, and receive the molten pool state feedback reward signal, autonomously judge the pros and cons of the molten pool changes caused by the output welding strategy, and execute the strategy that can generate the greatest value in the current state. The control module is used to receive the control strategy of the decision learning module, and control the welding machine to perform the corresponding welding operation according to the strategy.

[0041] Specifically, the front-end perception module adopts a hardware camera device suitable for the molten pool image processing module, and the hardware camera device includes a power supply system, a clock system, a protection system, an image sensor and an HDR image signal processor.

[0042] The power supply system is used to output voltages at various levels to provide power for the hardware. It outputs voltages at various levels through a switching power supply or a low-voltage difference linear regulated power supply, adapts to the standard voltage levels of each chip, and meets the power-on and power-off timing to ensure the normal operation of each hardware part.

[0043] The clock system provides internal clock signals for other hardware parts, serving as control signals for the coordinated work of various components within the chip.

[0044] The protection system is used to protect the hardware from damage. Its main function is to notify the HOST when a short circuit occurs in a part of the hardware or the internal chip temperature is too high, and take corresponding measures such as: cutting off the power supply; stopping the chip operation, etc. to protect the device safety, and resume normal operation when the fault is removed.

[0045] The image sensor is used to collect image information and transmit the information to the HDR image signal processor. The image sensor uses interlaced multiple exposure reading technology to achieve HDR still and video applications. Figure 2 As shown, the HDR image sensor is controlled by two shutter pointers, which control the integration of odd and even line pairs in a single frame of data. The shutter pointers are used to capture odd and even line pairs of single frame data at different times, and then integrated to output the frame. The output data can be matched with the Fourier transform-based image reconstruction algorithm from gradients.

[0046] The HDR image signal processor is used to synthesize the molten pool HDR image and perform advanced noise reduction. The image and color processing is implemented by the image stream processor encoded with hardware logic. The HDR image signal processor communicates with the molten pool state analysis module, decision learning module and control module of the main controller through a two-wire synchronous serial bus to modify the control and status registers of the signal processor to meet the actual needs of the application.

[0047] Preferably, the molten pool image processing module adopts the Fourier transform-based image reconstruction method from gradient to synthesize a high dynamic range image of the molten pool, which can reflect the brightness details of the molten pool image and avoid the loss of features due to the local area being too bright or too dark due to the high temperature of the molten pool, thereby more accurately analyzing and processing the molten pool state.

[0048] In order to obtain the low-order detail feature information of the image, the first-order derivative is used to form the first-order derivative vector field of the image, that is, the gradient field. Reconstructing the original image from the gradient image can fuse the features and obtain a high-quality image. By selecting a suitable energy minimization model, the gradient reconstruction problem is transformed into a non-homogeneous Poisson method solution problem. The differential operator can be converted into a complex multiplication through Fourier transform, so that the gradient reconstruction algorithm is simplified and a closed-form solution with the input as the gradient vector field can be obtained, so that the gradient image reconstruction can be realized quickly and effectively, and the high dynamic range molten pool image synthesis can be realized.

[0049] The energy minimization model of gradient field reconstruction, that is, after the gradient field v=(v 1 ,v 2 ) T In the process of reconstructing the image, we look for an image whose gradient map is the same as the previously obtained gradient map in terms of the square integrable norm (two-norm). The formula of the optimization problem is expressed as:

[0050]

[0051] in and || || 2 are the gradient operator and the square integrable norm on the two-dimensional rectangular area respectively. The meaning of the above formula is to obtain a target solution u with the minimum two-norm difference from the given vector field v. * .

[0052] The process of transforming Poisson's equation is:

[0053] The energy function is

[0054]

[0055] Using the Hilbert space H 2The dual operator of the upper gradient operator has the property of a negative divergence operator. The first-order variation of (2) is:

[0056]

[0057] Where div is the divergence operator, expressed as △ is the Laplacian operator, expressed as And it is easy to deduce that the Laplace operator has the properties:

[0058] Solution u * The condition that the first-order variation calculated by equation (3) is 0 is satisfied, and div v is: * , then:

[0059] -2(△u * -v * )=0 (4)

[0060] Thus we get the Poisson equation: * =v * Since we are dealing with images, we can assume that the Newman boundary conditions are met.

[0061] The closed form solution is obtained by Fourier transform, and the differential operator in the spatial domain is converted into complex multiplication in the frequency domain by using Fourier transform on the torus.

[0062]

[0063] Among them, ∧ represents the Fourier transform on the torus, ξ is the first component in the two-dimensional frequency domain, corresponding to x in the spatial domain, and y in the spatial domain corresponds to the second component in the frequency domain, represented by η; i is a pure imaginary unit. Using this property, the Fourier transform is applied to both sides of equation (4), and the constant is eliminated to obtain:

[0064] -4π 2 (ξ 2 +η 2 )(u * )^=(v * )^ (6)

[0065] Implement the decoupled differential operator and get u * The closed form solution of is as follows:

[0066]

[0067] where V is the inverse Fourier transform on the torus.

[0068] Since the algorithm needs to be deployed on hardware devices to realize the operation, the above derivation assumes that the solution u is located on the torus with a space H with two-order continuous weak derivatives. 2 The actual processing of digital images is represented by discrete matrices. In order to efficiently implement the closed-form solution of hardware programming, a coefficient matrix is ​​formed by point-by-point division in the frequency domain obtained in formula (7). When the size of the image to be reconstructed is determined, the coefficient matrix is ​​also uniquely determined. When multiple reconstruction operations are required, only one discrete coefficient matrix needs to be constructed. When calling matrix multiplication when reconstructing the image, the efficiency can be greatly improved. The coefficient matrix P is defined as:

[0069]

[0070] Where i and j are the horizontal and vertical coordinates in the discrete case respectively.

[0071] Regarding the divergence operator acting on the vector field, we get v * The key is how to express the differentials in different directions in discrete scenarios. Use the [-1 0 1] matrix to perform convolution operations with the image matrix to obtain the partial derivatives in the y direction in the continuous case and in the down direction in the discrete case. Similarly, use [-1 0 1] T The matrix is ​​convolved with the image matrix to obtain the partial derivatives in the x direction in the continuous case and in the following direction in the discrete case. The programming implementation that satisfies equation (7) in the discrete case.

[0072] HDR images are images with a very wide brightness range. Unlike traditional images that compress brightness information into 8-bit or 16-bit color space, HDR images directly record brightness information. In general, when using computers to represent images, 8 bits (256 levels) are usually used to distinguish image brightness. The brightness level range is too small to truly reflect the ambient brightness. Ordinary image representation is very inaccurate. Overexposure results in a white area that cannot highlight details. Dimming will only reduce the white pixel value and will not improve the details. However, if HDR images are used for recording, detailed features will be presented. The high dynamic range melt pool image synthesis of the present invention uses one or more pictures of the same scene but with different exposures, and uses a Fourier transform-based image reconstruction method from gradients to synthesize a picture with a wider brightness range, such as Figure 3As shown in the figure, by loading high-exposure images and low-exposure images, the gradient vector field of each image is calculated respectively, and then the maximum gradient point of the partial derivative image of the corresponding direction of the gradient vector field of each image is taken to synthesize a new gradient with more details. The algorithm of closed-form solution is obtained by Fourier transform, and the HDR image with greatly expanded brightness range is synthesized. Displaying more details of the molten pool image through HDR images is conducive to feature extraction and accurate judgment of the molten pool state, providing a reliable basis for achieving accurate state feedback and realizing welding strategy learning similar to human continuous exploration and reward mechanism feedback.

[0073] Furthermore, the decision learning module in this embodiment adopts deep reinforcement learning, and learns strategies through the interaction between the welding machine itself and the monitored molten pool state. The reinforcement learning SAC is used to allow the model to automatically explore how to adjust the welding parameters for the best welding quality, thereby maximizing the welding quality under any state. The welder monitors the state of the molten pool, makes intelligent decisions, and performs some actions. The actions then cause the state of the molten pool to change. The welder obtains feedback about the previous actions from these changing states, thereby guiding subsequent action decisions. The specific state of the molten pool in the optimal welding quality range is used as the target image. Based on the image method, the processed and synthesized molten pool image is obtained as both the feedback state and the target image of the reward function. The reward function is defined as the matching degree of the feature map of the current molten pool image and the target image after deep feature extraction. In addition, in order to improve the learning efficiency of the model algorithm, a supervised learning method of imitation learning is added, and the action process is manually demonstrated to the monitoring system, so that the machine can clarify the optimization direction of system learning.

[0074] The decision-making learning module continuously conducts trial-and-error action feedback based on the characteristics of the molten pool image, and uses a reward and punishment mechanism to make decisions and control the welding quality, forming a set of optimal execution strategies for each state in the welding process. Specifically, the machine affects the state of the environmental molten pool and obtains feedback information by performing welding actions. Through the stimulation of rewards and punishments, the machine gradually learns to respond to the stimulation, thereby generating actions that can make the best welding quality according to the state. The method of learning while obtaining samples is adopted, with high dynamic range synthetic molten pool images as sample input, the model is updated and welding operations are performed to obtain a new molten pool state, and feedback is obtained based on the welding quality corresponding to the molten pool state. The model is further updated and iterated continuously until a set of optimal welding execution strategies are learned from the experience of continuous welding, changes in the molten pool state, and feedback on the quality of welding.

[0075] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An intelligent self-decision-making molten pool monitoring system, characterized in that: It includes front-end perception module, molten pool image processing module, molten pool state analysis module, decision learning module and control module; The front-end perception module is used to collect and preprocess the image information of different brightness and darkness in the molten pool; The molten pool image processing module is used to synthesize the collected and processed image data into a molten pool high dynamic range image to display the brightness detail characteristics of the molten pool; The molten pool state analysis module is used to extract features from the obtained molten pool image, obtain the molten pool state data set to form input parameters, and provide welding machine action reward stimulation according to the target molten pool state; The decision learning module is used to output the execution strategy according to the molten pool state and welding machine parameters, and receive the molten pool state feedback reward signal, autonomously judge the pros and cons of the molten pool change caused by the output welding strategy, and execute the strategy that can generate the greatest value under the current state; The control module is used to receive the control strategy of the decision learning module and control the welding machine to perform corresponding welding operations according to the strategy; The front-end perception module uses a hardware camera device suitable for the molten pool image processing module, which includes a power supply system, a clock system, a protection system, an image sensor and an HDR image signal processor; The power supply system is used to output voltages at various levels to provide power for the hardware; The clock system provides the hardware with an internal clock signal as a control signal for coordinated work; The protection system is used to protect the hardware from damage; The image sensor is used to collect image information and transmit the information to the HDR image signal processor; HDR image signal processor is used to synthesize HDR images of the melt pool and perform advanced noise reduction; The image sensor is controlled by two shutter pointers, which control the integration of odd and even line pairs in a single frame of data. The shutter pointers are used to capture odd and even line pairs of a single frame of data at different times, and then integrated and output the frame. The HDR image signal processor communicates with the molten pool state analysis module, the decision learning module, and the control module via a two-wire synchronous serial bus; The molten pool image processing module uses the Fourier transform-based method of reconstructing images from gradients to synthesize high dynamic range images of the molten pool; The steps of synthesizing a high dynamic range image of the melt pool by reconstructing images from gradients based on Fourier transform are as follows: loading a high-exposure image and a low-exposure image, respectively calculating the gradient vector field of each image, then taking the maximum gradient point of the partial derivative image of the corresponding direction of the gradient vector field of each image to synthesize a new gradient, using the closed-form solution algorithm obtained by Fourier transform, and synthesizing an HDR image with an expanded brightness range; The molten pool state data set obtained in the molten pool state analysis module includes molten pool state information and corresponding welding parameters.

2. The intelligent self-decision-making molten pool monitoring system according to claim 1 is characterized in that: The decision learning module uses deep reinforcement learning to learn strategies through the interaction between the welding machine itself and the monitored molten pool state.

3. The intelligent self-decision-making molten pool monitoring system according to claim 1 is characterized in that: The decision-making learning module performs continuous trial-and-error action feedback based on the characteristics of the molten pool image, and uses a reward and punishment mechanism to make decisions and control the welding quality, forming a set of optimal execution strategies for each state in the welding process.