A method and device for controlling the liquid level of an sl a equipment based on deep reinforcement learning

Abstract

The application relates to the technical field of SLA and discloses an SLA equipment liquid level control method and device based on deep reinforcement learning, which comprises the following steps: acquiring the observation state of an SLA printing environment in real time; wherein the observation state is multidimensional information; inputting the observation state into a trained deep reinforcement learning intelligent agent, outputting a continuous control action from the intelligent agent; converting the control action into a driving signal, controlling the displacement of a liquid level compensation execution mechanism, and adjusting the resin liquid surface height. The application comprises the following steps: acquiring the observation state containing multidimensional information in real time, inputting the observation state into a deep reinforcement learning intelligent agent trained by historical interaction data to directly output a continuous control action, and generating a control instruction suitable for instantaneous working conditions without complicated parameter setting for different equipment or materials, so that the liquid level control efficiency is improved on the basis of perceiving multidimensional information.

Description

Technical Field

[0001] This invention relates to the field of stereolithography apparatus (SLA) technology, specifically to a liquid level control method and apparatus for SLA equipment based on deep reinforcement learning. Background Technology

[0002] SLA is an important additive manufacturing technology that uses liquid photosensitive resin as raw material and constructs three-dimensional workpieces by scanning and curing layer by layer with a laser. The stability of the resin liquid surface directly determines the accuracy of the molded part and the printing efficiency. Moreover, the liquid level is affected by the coupling influence of the mechanical characteristics of the equipment, the physical characteristics of the resin, and process parameters (such as cross-sectional area and blade negative pressure), making it significantly difficult to control.

[0003] Currently, mainstream liquid level control technologies typically employ float detection combined with a proportional-integral-derivative (PID) algorithm. This method compensates for resin consumption by adjusting the float. However, it suffers from several key drawbacks: it relies solely on single-dimensional feedback of liquid level deviation, neglecting critical disturbance parameters; PID parameters require manual adjustment, resulting in poor versatility and an inability to adapt to dynamic operating conditions; and the adjustment requires multiple convergence cycles, leading to low efficiency. Therefore, there is an urgent need for an intelligent control method capable of sensing multi-dimensional information, adapting to changes, and rapidly stabilizing the liquid level to overcome these technological limitations. Summary of the Invention

[0004] This invention provides a liquid level control method and device for SLA equipment based on deep reinforcement learning, which solves the comprehensive limitations of related PID control in terms of accuracy, efficiency, stability and versatility, thereby improving the forming quality and production efficiency of SLA printers.

[0005] In a first aspect, the present invention provides a liquid level control method for SLA equipment based on deep reinforcement learning, the method comprising: Real-time acquisition of the observation status of the SLA printing environment; where the observation status is multi-dimensional information; The observation state is input into a trained deep reinforcement learning agent, which then outputs continuous control actions. The control action is converted into a drive signal to control the liquid level compensation actuator to move and adjust the resin liquid level. The agent is trained through the following process: based on empirical data generated during the interaction with the printing environment, consisting of observed states, control actions, immediate rewards calculated according to the composite reward function, and new observed states, the agent's policy network parameters are iteratively updated to optimize the agent's control policy until the policy converges.

[0006] In one optional implementation, the observation state is the observation and measurement values ​​of each dimension of the agent's state space at each time step; the agent's state space includes at least the following observation dimensions: the current liquid level deviation, the rate of change of liquid level deviation, the cross-sectional information of the current layer to be printed, and the negative pressure value of the scraper cavity.

[0007] In one alternative implementation, the observation state is input to a trained deep reinforcement learning agent, which outputs continuous control actions, including: The observed state is input into the agent's policy network; The observation state is nonlinearly transformed by the policy network and mapped to a scalar value within a predetermined continuous interval. The output scalar value is a continuous control action mapped in the state space of the agent; where the sign of the control action corresponds to the direction of movement of the liquid level compensation actuator, and the magnitude of the value corresponds to the amplitude of movement of the liquid level compensation actuator.

[0008] In one optional implementation, the construction of the composite reward function includes: a precision reward term, a direction reward term, an efficiency reward term, and a stability reward term, wherein: The accuracy bonus is configured as follows: when the absolute value of the liquid level deviation is less than the preset error threshold, the bonus value is positive; otherwise, the bonus value is negatively correlated with the absolute value of the liquid level deviation. The reward value for the directional reward item is configured to be positively correlated with the reduction in liquid level deviation; The reward value for efficiency bonus items is configured to a fixed, preset negative value; The reward value for stability bonus items is configured to be negatively correlated with the drasticness of changes in control actions.

[0009] In one optional implementation, the training process of the agent includes an offline training phase and an online adaptation phase, wherein: The offline training phase includes: collecting interaction data by simulating various printing conditions in a simulation environment or experimental equipment and storing it in an experience playback buffer; and training the agent offline using the sampled data. The online adaptation phase includes: deploying the agent model trained offline to the actual SLA device, and continuing to fine-tune the agent online using actual printing data under safety constraints.

[0010] In one alternative implementation, the agent employs an algorithm based on an actor-commentator framework, which includes an actor network and a commentator network. The actor network is used to generate a probability distribution of control actions based on the observed states, and the commentator network is used to evaluate the value of states or state-action pairs and update the network parameters through temporal difference errors.

[0011] Secondly, the present invention provides a liquid level control device for SLA equipment based on deep reinforcement learning, the device comprising: The status acquisition module is used to acquire the observation status of the SLA printing environment in real time; the observation status is multi-dimensional information. The intelligent decision-making module is used to input the observed state into the trained deep reinforcement learning agent, which then outputs continuous control actions. The motion execution module is used to convert control actions into drive signals to control the liquid level compensation actuator to move and adjust the resin liquid level height. The agent in the intelligent decision-making module is trained through the following process: based on the empirical data generated during the interaction with the printing environment, which consists of observed states, control actions, immediate rewards calculated according to the composite reward function, and new observed states, the agent's policy network parameters are iteratively updated to optimize the agent's control policy until the policy converges.

[0012] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the SLA device liquid level control method based on deep reinforcement learning described in the first aspect or any corresponding embodiment.

[0013] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the deep reinforcement learning-based SLA equipment level control method described in the first aspect or any corresponding embodiment thereof.

[0014] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the deep reinforcement learning-based SLA equipment level control method described in the first aspect or any corresponding embodiment thereof.

[0015] In the SLA equipment liquid level control method and apparatus based on deep reinforcement learning in the above embodiments of the present invention, the process of acquiring the observation state containing multi-dimensional information in real time and inputting it into a deep reinforcement learning agent trained from historical interaction data to directly output continuous control actions can generate control commands that adapt to instantaneous working conditions without the need for cumbersome parameter tuning for different equipment or materials, thereby improving the efficiency of liquid level control based on the ability to perceive multi-dimensional information.

[0016] Furthermore, the core feature of the intelligent agent is that its control strategy is dynamically generated and continuously optimized through iterative learning of historical interaction data. This allows the control strategy to adaptively adjust with the printing process, thereby always approaching the optimal solution under the current working conditions. This enables rapid stabilization of the liquid level and improves the efficiency and intelligence level of liquid level control. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a liquid level control method for SLA equipment based on deep reinforcement learning according to an embodiment of the present invention; Figure 2 This is an RDL algorithm framework diagram of the SLA equipment level control method based on deep reinforcement learning according to an embodiment of the present invention; Figure 3 This is a simplified diagram of the Actor-Critic basic algorithm of the SLA equipment level control method based on deep reinforcement learning according to an embodiment of the present invention. Figure 4 This is a structural block diagram of a liquid level control device for SLA equipment based on deep reinforcement learning according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] In the field of additive manufacturing, SLA technology, as an important branch, uses liquid photosensitive resin as raw material and constructs three-dimensional workpieces by scanning and curing layer by layer with a laser beam of a specific wavelength. After each layer is printed, the printing platform needs to be precisely lowered by one layer thickness to ensure that the next layer of resin can be uniformly coated and maintain a precise forming surface. The stability of the resin liquid level directly determines the accuracy, surface quality, and printing efficiency of the formed part. At the same time, the resin liquid level of the SLA equipment is a highly dynamic and complex controlled object. Its stability is affected by the coupling of multiple parameters—including the mechanical characteristics of the equipment (such as Z-axis motion accuracy and storage tank stability), the physical characteristics of the resin (such as viscosity, density, and curing shrinkage rate), and process parameters (such as the doctor blade movement speed, the negative pressure inside the doctor blade cavity, the forming area and shape of the current layer cross-section, and the ambient temperature), which poses a significant challenge to the stable control of the liquid level.

[0023] Therefore, the current mainstream liquid level control solution in the industry adopts a combination of float-based mechanical liquid level detection and PID control algorithm: the liquid level height is monitored in real time by a sensor, and the deviation between the liquid level and the target liquid level is used as the input signal of the PID controller. The liquid level is then maintained near the set value by adjusting the penetration depth of the float to compensate for resin consumption.

[0024] This PID control scheme faces key challenges in SLA level control scenarios, as it struggles to adapt to the dynamic and complex characteristics of liquid levels. The main pain points are reflected in three aspects: Firstly, the debugging cost is high and the versatility is poor. The P, I, and D coefficients of the PID controller need to be manually debugged and calibrated for different equipment models, resin materials, and even different working environments, which makes it impossible to form a universal control strategy. Secondly, the adjustment cycle is long and the efficiency is low. PID control cannot achieve precise adjustment in one step. When faced with significant changes in resin level caused by printing large cross-section layers, it is necessary to calculate and adjust (such as rise, over-adjustment, and pullback) through multiple control cycles to gradually stabilize the liquid level in the target range. This multi-step convergence process prolongs the waiting time before printing each layer, becoming a bottleneck that restricts the overall printing efficiency. Third, the control dimension is singular and the anti-interference capability is weak. The PID controller only uses the current liquid level deviation as a single-dimensional feedback signal, completely ignoring key process variables such as "printing cross-sectional area" and "scalpel negative pressure," which are both key sources of interference and indicators of potential disturbances. Furthermore, its Kp, Ki, and Kd parameters are fixed after pre-tuning and cannot adapt to dynamic conditions during the printing process. For example, when the printing layer cross-section suddenly changes from a small area to a large area, the resin consumption rate changes abruptly. The PID parameters under the original stable conditions will be conservative, resulting in insufficient compensation and slow liquid level recovery. When the scalpel negative pressure undergoes a large change, the fixed parameters cannot be adjusted accordingly, which can easily lead to overshoot or oscillation.

[0025] In response, the accuracy, stability, and robustness of PID-based liquid level control schemes are difficult to guarantee when facing complex and ever-changing printing conditions with sudden disturbances. Related optimization attempts (such as manually readjusting parameters for specific conditions and locally improving detection sensitivity) also have obvious limitations: either they rely on continuous manual intervention, which is inefficient and prone to introducing new problems due to operational deviations; or they are only suitable for a single fixed condition and cannot be compatible with actual printing scenarios with dynamic changes in multiple parameters, making it difficult to fundamentally break through the bottleneck of traditional control methods.

[0026] To address the aforementioned issues, various embodiments of this application provide a deep reinforcement learning-based liquid level control method for SLA equipment. The method includes: real-time acquisition of the observation state of the SLA printing environment; wherein the observation state is multi-dimensional information; inputting the observation state into a trained deep reinforcement learning agent, which outputs continuous control actions; converting the control actions into drive signals to control the liquid level compensation actuator to perform displacement, thereby adjusting the resin level height; wherein the agent is trained through the following process: based on empirical data generated during interaction with the printing environment, consisting of the observation state, control actions, immediate rewards calculated according to a composite reward function, and new observation states, the agent's policy network parameters are iteratively updated to optimize the agent's control strategy until the strategy converges.

[0027] This embodiment provides a liquid level control method for SLA equipment based on deep reinforcement learning, which can be used in control units.

[0028] Here, the control unit can be configured to perform the above method. For example, the control unit may include, but is not limited to, a host computer for the SLA printer, a dedicated intelligent controller integrated into the SLA printer, or a cloud server.

[0029] Figure 1 This is a flowchart of a deep reinforcement learning-based SLA equipment level control method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the process includes the following steps: Step S101: Obtain the real-time observation status of the SLA printing environment.

[0030] In this embodiment, the SLA printing environment can refer to the sum of all internal and external physical conditions that affect the stability of the resin liquid level during the SLA printing process.

[0031] This step aims to provide the agent with a set of observation states that fully describe the current system state and its dynamic trends at the beginning of each control decision cycle. .

[0032] Here, the observation state The data can be obtained by reading measurement data from various sensors and the system's preset process parameters.

[0033] Among them, observation status This is multidimensional information used to structurally describe the current key characteristics of the SLA printing environment.

[0034] For example, observation status It may include at least physical quantities (such as liquid level height, scraper negative pressure) directly or indirectly measured by sensors, as well as preset information from the control system (such as the cross-sectional area of ​​the current layer).

[0035] Step S102: The observation state is input into the trained deep reinforcement learning agent, which then outputs continuous control actions.

[0036] In this embodiment, the multidimensional observation state obtained in step S101 is... Input is fed into a pre-trained, converged Deep Reinforcement Learning (DRL) agent (hereinafter referred to as the agent), enabling the agent to base its observations on the observation states learned during its training. Control Action The mapping rules autonomously analyze and calculate the current operating conditions of the SLA printing environment, and the final output can directly guide the continuous control actions of the liquid level compensation actuator. .

[0037] The intelligent agent, built on the DRL framework, serves as the core decision-making unit. Through continuous interaction with the SLA printing environment, it autonomously learns the optimal liquid level control strategy to achieve rapid, accurate, adaptive, and stable resin level control.

[0038] Continuous control actions output by the intelligent agent Then it can be a form of instruction used to directly guide the action of the liquid level compensation actuator.

[0039] Here, continuous control actions It can output stepless adjustment commands based on the dynamic changes in SLA printing conditions, thus eliminating the need for calculations and adjustments over multiple control cycles to gradually stabilize the liquid level within the target range.

[0040] In step S103, the control action is converted into a drive signal to control the liquid level compensation actuator to move in order to adjust the resin liquid level.

[0041] In this embodiment, SLA technology uses liquid photosensitive resin as raw material and constructs a three-dimensional workpiece by scanning and curing it layer by layer with a laser beam of a specific wavelength. After each layer is printed, the printing platform needs to be precisely lowered by the thickness of one layer. At this time, it is essential to ensure that the resin liquid level can uniformly cover the forming area of ​​the next layer, and the liquid level height must be precisely matched with the reference plane of the laser scanning. Therefore, adjusting the resin liquid level height in SLA printing is primarily to address the liquid level changes caused by resin consumption and operating disturbances during the printing process, ensuring that each layer meets the basic conditions for uniform coating and precise forming.

[0042] In order to adjust the resin level, step S103 follows the continuous control action output from step S102. The core is to transform the abstract control commands output by the intelligent agent into liquid level regulation behaviors that can be directly applied to the physical system through the connection between signal conversion and mechanism drive, and finally achieve precise control of the resin liquid level height.

[0043] Specifically, based on preset signal mapping rules, continuous control actions are first... The signal is converted into a drive signal that is compatible with the drive unit of the liquid level compensation actuator. The converted drive signal is then sent to the drive unit of the actuator (such as a motor driver), which drives the actuator to perform the corresponding displacement operation (rising or falling). By changing the penetration depth of the actuator in the resin, the liquid level change caused by printing consumption or operating condition disturbance is compensated, and the resin liquid level is finally adjusted to the target range.

[0044] The liquid level compensation actuator can refer to a mechanical actuator in an SLA printing device used to achieve dynamic compensation of the resin level through precise displacement operation. For example, the liquid level compensation actuator can be a float lifting mechanism.

[0045] A level compensation actuator drive unit can refer to a hardware unit that provides power and control signal processing capabilities to the level compensation actuator. For example, the drive unit can be a motor driver.

[0046] For example, during SLA printing, if the current liquid level is lower than the target liquid level due to operating disturbances, a control action is generated to trigger the float to rise. and control actions This is converted into a drive signal, causing the float lifting mechanism to raise the float so that the current liquid level rises back to the target liquid level.

[0047] Furthermore, after executing the lifting operation of the float lifting mechanism, the SLA printing environment enters a new observation state. And generate an instant reward. The feedback is sent to the agent, which then updates the mode parameters to optimize the strategy.

[0048] Furthermore, the agent is obtained through the following training process: based on the observed states generated during interaction with the printing environment. Controlling actions Instant reward calculated based on the composite reward function and new observation status The empirical data is used to iteratively update the policy network parameters of the agent to optimize the agent's control policy until the policy converges.

[0049] Among them, empirical data refers to the core data unit generated during the interaction between the intelligent agent and the SLA printing environment, which is used to optimize the control strategy. It can be obtained from the observed state. Controlling actions Instant rewards and new observation status Composition. Observation status This refers to the environmental information perceived by the intelligent agent during interaction, and the control actions. This refers to the agent's assessment of the observed state. Output decision instructions, instant rewards This refers to the feedback from the environment to the effect of the action, a new state of observation. It refers to the new state that the environment presents after an action is performed.

[0050] The composite reward function is a pre-defined function used to calculate immediate rewards. A multi-dimensional evaluation function that comprehensively assesses control actions. The system outputs corresponding instant rewards based on the impact of various preset indicators. This allows the agent to identify which actions are more in line with the control objective, thereby determining the direction of strategy optimization.

[0051] Furthermore, the core purpose of training the agent is to generate effective experience data and iteratively optimize the strategy through the dynamic interaction between the agent and the SLA printing environment, so that the agent can ultimately have the decision-making ability to accurately adapt to the liquid level control requirements.

[0052] As an example, please refer to Figure 2 , Figure 2This is an RDL algorithm framework diagram of the SLA equipment level control method based on deep reinforcement learning according to an embodiment of the present invention, which includes the following process: At each discrete time step The agent obtains an observed state from the environment. The agent, based on its internal policy, determines the current observed state. Select a control action Control Actions The command is passed to the environment for execution. The host computer sends instructions to the motor driver, controlling the float lifting mechanism to perform the corresponding lifting operation; the environment executes the control action. After that, it entered a new observation phase. And generate an instant reward. Feedback is given to the agent; the agent then uses the obtained experience tuples ( , , , The system updates its model parameters to optimize its strategy, with the goal of maximizing long-term cumulative rewards.

[0053] In the SLA equipment level control method and apparatus based on deep reinforcement learning in the above embodiments of the present invention, the observation state containing multi-dimensional information is acquired in real time. The data is then input into a deep reinforcement learning agent trained on historical interaction data to directly output a continuous control process. This eliminates the need for tedious parameter tuning for different equipment or materials, generating control commands that adapt to instantaneous operating conditions. This improves liquid level control efficiency by sensing multi-dimensional information. The core feature of the agent is that its control strategy is dynamically generated and continuously optimized through iterative learning of historical interaction data. This allows the control strategy to adaptively adjust as the printing process progresses, always approaching the optimal solution under the current operating conditions. This enables rapid stabilization of the liquid level, improving the efficiency and intelligence of liquid level control.

[0054] In one possible implementation of step S101 above, the observation state is the observation and measurement value of each dimension of the agent's state space for each time step; the agent's state space includes at least the following observation dimensions: the current liquid level deviation, the rate of change of liquid level deviation, the cross-sectional information of the current layer to be printed, and the negative pressure value of the scraper cavity.

[0055] In this embodiment, the state space of the intelligent agent includes at least the current liquid level deviation acquired in real time. Liquid level deviation change rate Cross-sectional information of the current layer to be printed Negative pressure value inside the scraper cavity .

[0056] in, Let represent the observation value of the corresponding dimension at each time step. Then, the observation state at each time step can be denoted as:

[0057] Current liquid level deviation This can refer to the height difference between the currently monitored liquid level and the target liquid level, which can serve as a core indicator directly reflecting the degree of liquid level deviation and providing a basic basis for liquid level adjustment; the rate of change of liquid level deviation. It can refer to the difference between the current liquid level deviation and the deviation of the previous time step, which is used to characterize the trend of liquid level change. It can intuitively represent the rising / falling trend of the liquid level, help the intelligent agent predict the direction of liquid level change, and avoid lag adjustment.

[0058] Current cross-sectional information L of the layer to be printed t This could refer to the total cross-sectional area of ​​the current layer to be printed. Further, it could refer to the cross-sectional information of the current layer to be printed. It can include the distribution location of the cross section in the resin tank or its equivalent radius, or image information containing the cross section, which can be directly correlated with the resin curing consumption rate and serve as a core indicator for predicting process disturbances.

[0059] Negative pressure value inside the scraper cavity This can refer to the negative pressure value generated inside the scraper during real-time monitoring.

[0060] Here, the magnitude of negative pressure affects the effect of the scraper on the removal / retention of resin (for example, excessive negative pressure can easily lead to a drop in local liquid level), and it is also an important parameter reflecting the disturbance of equipment operating conditions.

[0061] Understandably, the observation state Other important operating parameters may also be included, such as historical operation data, operating temperature, and resin material properties. However, for ease of description, this invention selects the current liquid level deviation. Liquid level deviation change rate Cross-sectional information of the current layer to be printed Negative pressure value inside the scraper cavity As an example, constructing the observation state .

[0062] In the SLA equipment liquid level control method and apparatus based on deep reinforcement learning in the above embodiments of the present invention, the current liquid level deviation is used to... Liquid level deviation change rate Cross-sectional information of the current layer to be printed Negative pressure value inside the scraper cavity Construct observation state It comprehensively covers four core influencing dimensions: deviation, trend, process, and equipment. It can accurately capture key information such as resin consumption and operating condition disturbances, providing the intelligent agent with a complete foundation for operating condition cognition and avoiding adjustment deviations caused by missing information. Liquid level deviation change rate. It can characterize the rising / falling trend of the liquid level in real time, and provide cross-sectional information of the current layer to be printed. The resin curing consumption rate and the negative pressure value inside the scraper cavity can be correlated in advance. The impact of negative pressure changes on resin coating can be predicted. Based on these parameters, the intelligent agent can perceive the liquid level change trend and potential disturbances in advance, realize prediction and early adjustment, and effectively avoid the problems of liquid level overshoot or slow recovery caused by lagging adjustment.

[0063] In one possible implementation of step S102 above, the observation state is input to a trained deep reinforcement learning agent, which then outputs continuous control actions, including: The observed state is input into the agent's policy network; The observation state is nonlinearly transformed by the policy network and mapped to a scalar value within a predetermined continuous interval. The output scalar value serves as a continuous control action; the sign of the control action corresponds to the direction of movement of the liquid level compensation actuator, and the magnitude of the value corresponds to the amplitude of movement of the liquid level compensation actuator.

[0064] In this embodiment, the observation state The input agent's policy network. Here, the policy network can refer to the core functional module in a deep reinforcement learning agent responsible for generating control actions.

[0065] Here, in the input observation state At the same time, it is necessary to ensure that the dimension and data format of the observed state are completely matched with the input layer structure of the policy network to avoid decision bias due to data incompatibility and to provide standardized raw data for subsequent feature processing.

[0066] Furthermore, the policy network receives the observed state Then, the observation state is nonlinearly transformed through the built-in deep neural network structure, which may include: firstly, extracting key features from the observation state through a multi-layer neural network; then further mapping the extracted high-dimensional features to a preset continuous numerical range, and finally outputting a single scalar value.

[0067] For example, the preset continuous numerical range can be [-1, +1]; the range of scalar values ​​is also strictly limited to [-1, +1], and their magnitude and sign are determined by the characteristics of the observed state.

[0068] Furthermore, the scalar values ​​in the range [-1, +1] output by the policy network are directly used as continuous control actions.

[0069] The numerical characteristics of this control action and the motion state of the liquid level compensation actuator have a clear preset mapping rule.

[0070] Specifically, the positive and negative values ​​correspond to the direction of motion. Positive values ​​correspond to the upward movement of the liquid level compensation actuator, while negative values ​​correspond to the downward movement of the liquid level compensation actuator. The magnitude of the value corresponds to the amplitude (or speed) of the motion. The larger the absolute value of the value, the greater the amplitude (or the faster the speed) of the actuator's motion. This ensures that the control action can be steplessly adjusted according to differences in operating conditions, avoiding the adjustment lag problem of traditional discrete actions.

[0071] For example, "+1" means that the actuator rises at its maximum speed for one unit of time, and "-0.3" means that the actuator falls at 30% of its maximum speed for one unit of time.

[0072] In the SLA equipment level control method and device based on deep reinforcement learning in the above embodiments of the present invention, the nonlinear transformation of the policy network can effectively handle the nonlinear relationships of numerous parameters in the observed state. Compared with linear decision-making that relies solely on a single level deviation, this method can more accurately analyze dynamic operating condition characteristics, thereby accurately outputting control actions. The control actions output by the policy network are continuous scalar values ​​in the range [-1, +1], which can be steplessly adjusted by corresponding the magnitude of the value to the motion amplitude / speed. This allows the level compensation actuator to quickly adapt to changes in operating conditions, significantly shortening the level stabilization time and reducing overshoot. The preset rule that "positive and negative values ​​correspond to the direction of motion, and magnitude corresponds to the amplitude" for the control actions is directly associated with the physical actions of the level compensation actuator. It can be analyzed by the drive unit without additional complex conversions, improving the reliability and stability of the control closed loop.

[0073] In one possible implementation of the above embodiments, the construction of the composite reward function includes: a precision reward term, a direction reward term, an efficiency reward term, and a stability reward term, wherein: The accuracy bonus is configured as follows: when the absolute value of the liquid level deviation is less than the preset error threshold, the bonus value is positive; otherwise, the bonus value is negatively correlated with the absolute value of the liquid level deviation. The reward value for the directional reward item is configured to be positively correlated with the reduction in liquid level deviation; The reward value for efficiency bonus items is configured to a fixed, preset negative value; The reward value for stability bonus items is configured to be negatively correlated with the drasticness of changes in control actions.

[0074] In this embodiment, the purpose of constructing the composite reward function is to provide the agent with clear reward and punishment signals, guiding the agent to learn a fast, accurate, and stable liquid level control strategy.

[0075] The agent can be constructed using any deep reinforcement learning algorithm model applicable to continuous action spaces. For example, the agent's algorithm may include, but is not limited to: Deep Deterministic Policy Gradient (DDPG), Twin Delayed Deep Deterministic Policy Gradient (TD3), Proximal Policy Optimization (PPO), or Soft Actor-Critic (SAC).

[0076] Here, each of the above algorithms includes an Actor network (executor) and a Critic network (evaluator). The Actor network is responsible for "decision generation," while the Critic network is responsible for "effect evaluation." Together, they complete a closed loop of "action generation - effect feedback - strategy optimization." The Actor network is used to generate the probability distribution of actions. Network parameters are updated via policy gradient. To maximize long-term returns; Critic networks are used to evaluate the value of states or state-action pairs. or Network parameters are updated using time-series difference error. To approximate the true value of the function.

[0077] Among them, the Actor network uses network parameters (e.g., neural network weights, biases) Construct mapping rules Enter the current liquid level condition. Output each continuous action The probability of the current state. In a Critic network, value refers to the probability of the current state. The expected long-term cumulative reward is the core basis for the Critic network to judge the quality of an action; network parameters These are the neural network parameters of the Critic network.

[0078] Temporal difference error refers to the difference between the value predicted by the Critic network and the reward obtained after the actual interaction plus the value of the next state. It is the core indicator for correcting the accuracy of the Critic evaluation. The true value function refers to the expected true long-term reward of the state / state-action pair.

[0079] As an example, please refer to Figure 3 , Figure 3 This is a simplified diagram of the Actor-Critic basic algorithm for the SLA equipment level control method based on deep reinforcement learning according to an embodiment of the present invention, including the following process: The current liquid level condition of the SLA equipment (environment) is observed to determine the state quantity s (i.e., The data is input synchronously to the Action Network (Actor) and the Evaluation Network (Critic) in the form of (S, S ... The interactive action A (floating block displacement control quantity) is generated and transmitted to the SLA printing device.

[0080] The SLA device executes action A, the liquid level changes, and the next state S is generated. i+1 (Right now The evaluation network Critic simultaneously receives the original observed state s and the next state s. i+1 ,Evaluate And calculate the reward based on the effect of action A. .

[0081] To reward The target value is the temporal difference error, which is calculated through supervised learning to generate gradient information. This gradient information is then passed to the action network (Actor) to update parameters via policy gradients. (to make it easier for subsequently generated A's to obtain high scores) On the other hand, it is passed to the evaluation network Critic to update w (so that the Critic's value assessment is closer to the true target value).

[0082] Repeat the above steps, optimizing the parameters of Actor and Critic through closed-loop optimization after each interaction, until the actions generated by Actor can stably and quickly stabilize the SLA liquid level. At this point, the agent has completed training and can be used for actual liquid level control.

[0083] Furthermore, the core of the precision reward item is to set reward and punishment rules based on whether the liquid level deviation meets the standard, so as to ensure that the agent prioritizes the pursuit of liquid level accuracy.

[0084] First, a preset error threshold that reflects the required accuracy of liquid level control is established. Secondly, based on the absolute value of the liquid level deviation Compared with the preset error threshold The comparison relationship configuration reward logic includes: when When the liquid level deviation is within the allowable range, a positive reward is triggered, which can be achieved through an indicator function. Output a fixed positive value to reflect the reward for achieving the target; when When the liquid level deviation exceeds the allowable range, the reward value is equal to... It is negatively correlated, that is The larger the value, the smaller the reward (even negative), which can be achieved through... (β is the weighting coefficient, such as 0.5) This quantifies the fact that the greater the deviation, the heavier the penalty.

[0085] Finally, the formula for calculating the accuracy bonus can be as follows:

[0086] in, For indicator functions, , These are the weighting coefficients.

[0087] Furthermore, the core of the directional reward is to encourage the agent to take actions that reduce the liquid level deviation, preventing it from lingering inactively within the deviation range or taking the opposite action.

[0088] First, define the decrease in liquid level deviation as the current deviation. Deviation from the previous time step The difference (i.e.) , This indicates that the deviation is increasing. This indicates a reduction in deviation; secondly, the reward value is compared with... Direct association, configured as " The smaller the value, the stronger the positive reward.

[0089] Specifically, the formula for calculating the directional reward item can be as follows:

[0090] in, The weighting coefficient for the directional reward term is used when the action reduces the deviation. ), output positive rewards; when the action does not reduce the deviation ( This design outputs negative rewards, guiding the agent to continuously optimize its actions.

[0091] Furthermore, the core of the efficiency reward is to force the agent to shorten the liquid level stabilization time through time penalties, thereby avoiding multi-cycle convergence.

[0092] First, based on the time required for each control decision cycle, a fixed preset negative reward value (e.g., -0.1) is set; second, it is configured to feed back this fixed negative reward to the agent after each control time step, regardless of the current liquid level status.

[0093] Specifically, the formula for calculating the efficiency bonus can be as follows:

[0094] in, This is the weighting coefficient for efficiency reward items, i.e., the fixed negative reward baseline value.

[0095] To maximize long-term cumulative rewards, the agent must learn to stabilize the liquid level to the target range in the fewest time steps, thereby breaking through the efficiency bottleneck of traditional multi-cycle regulation.

[0096] Furthermore, the core of the stability bonus is to penalize violent movements, avoid liquid level oscillations, and ensure a smooth control process.

[0097] First, define the degree of drastic change in the control action as the current control action. The magnitude of the movement (the greater the amplitude of the movement, the stronger the impact on the actuator, and the more easily the liquid level oscillates). Secondly, the reward value is negatively correlated with the magnitude of the action, configured as "the greater the magnitude of the action, the stronger the negative reward".

[0098] Specifically, the formula for calculating the stability bonus can be as follows:

[0099] in, For the stability reward item weight coefficient, when When approaching ±1 (maximum amplitude), When the value is close to -1, the negative reward is strongest; when When approaching 0 (small movement), With a value close to 0, the negative reward is the weakest. This design forces the agent to output smooth movements, avoiding liquid level overshoot or oscillation caused by violent movements.

[0100] Optionally, to accelerate convergence in the early stages of training, an imitation learning mechanism can be introduced. This reward is constructed by comparing the agent's output actions with expert demonstrations (such as the actions of a PID controller calibrated by an experienced engineer under stable conditions, collected in advance). Positive rewards are given when the agent's behavior closely resembles the expert's behavior, guiding it to quickly learn the basic policy.

[0101] Specifically, the formula for calculating the learning reward can be as follows:

[0102] in, The actions output by the intelligent agent. Demonstrate the actions to the experts; If it is a positive number, it represents the weighting coefficient for that item.

[0103] Ultimately, total reward The sum of the above items can be weighted and calculated as follows:

[0104] in," "Can indicate an optional option."

[0105] In the SLA equipment liquid level control method and apparatus based on deep reinforcement learning in the above embodiments of the present invention, the collaborative design of accuracy reward items and orientation reward items achieves dual protection of liquid level accuracy. This ensures liquid level meets standards while actively approximating deviations towards the target value, significantly improving the forming accuracy of SLA printing. By setting a time penalty mechanism for the efficiency reward item, liquid level can be quickly compensated through actions with appropriate output amplitude, reducing waiting time before each layer is printed and directly improving the overall production efficiency of the LA printer. By setting action amplitude penalty logic for the stability reward item, the agent will prioritize small-amplitude adaptation actions, avoiding liquid level overshoot and reducing mechanical impact on actuators such as the float lifting mechanism, extending equipment lifespan, and ensuring uniform resin coating. By setting optional imitation learning reward items in conjunction with the Actor-Critic framework, the training efficiency and deployment adaptability of the agent are greatly improved.

[0106] In one possible implementation of the above embodiments, the training process of the intelligent agent includes an offline training phase and an online adaptation phase, wherein: The offline training phase includes: collecting interaction data by simulating various printing conditions in a simulation environment or experimental equipment and storing it in an experience playback buffer; and training the agent offline using the sampled data. The online adaptation phase includes: deploying the agent model trained offline to the actual SLA device, and continuing to fine-tune the agent online using actual printing data under safety constraints.

[0107] In this embodiment, the training process of the intelligent agent can be implemented by separating offline training and online training into two stages, with the core logic of first laying the foundation offline and then adapting to the actual device online.

[0108] As a concrete example, the steps can be as follows: I. The offline training phase includes the following steps: Step 1: Based on the actual SLA equipment's hardware parameters (such as the accuracy of the float lifting mechanism and the sensor range), resin physical properties (such as viscosity and density), and printing process range (such as the cross-sectional area variation range and the doctor blade negative pressure adjustment range), build one of the following two offline training environments: Simulation environment: A virtual environment equivalent to the actual SLA equipment is constructed through numerical modeling (such as MATLAB / Simulink, Python Gym framework). It can accurately simulate a variety of typical working conditions such as printing with different cross-sectional areas, sudden changes in doctor blade negative pressure, and changes in resin consumption rate, and can quickly reproduce fault scenarios. Experimental equipment environment: Use an SLA testing machine with the same model or similar parameters as the actual production equipment. Simulate actual working conditions by manually setting process parameters to ensure that the physical characteristics of the environment are consistent with the real scene.

[0109] Step 2: In the established offline environment, allow the agent (an initial DRL model with randomized parameters) to continuously interact with the environment, generating and collecting complete experience tuples. , , , ), will experience tuples ( , , , Store the data in the experience playback buffer and repeat the process until the buffer stores enough data to cover multiple printing conditions.

[0110] Step 3: Randomly sample small batches of experience data from the experience replay buffer (e.g., sample 32-128 tuples each time), and iteratively update the agent's network parameters based on the DRL algorithm.

[0111] II. The online training phase includes the following steps: Step 1: Deploy the offline-trained and converged agent model to the control system (such as a host computer) of the actual SLA production equipment, ensuring that the model is compatible with the equipment's sensor data interface and actuator drive interface, meaning the agent can read the data collected by the equipment's sensors in real time. And the output It can be converted into the actual displacement of the floating block through the drive unit.

[0112] Step 2: To prevent the agent from outputting dangerous actions due to adaptation deviations during the initial online fine-tuning phase, the following two types of safety restrictions are preset: Movement range limitation: the output of the agent The amplitude is limited to ±30% of the convergence action during the offline phase to avoid excessively drastic movements; Liquid level deviation threshold protection: If the real-time liquid level deviation... If the safety threshold is exceeded, an emergency shutdown or switch to manual control will be triggered to prevent equipment damage or printing failure.

[0113] Step 3: In the actual SLA printing production process, the agent continuously optimizes according to the following cycle of control + data collection + fine-tuning: The agent is based on the actual equipment generate Control the float to rise and fall; simultaneously record the actual movement. ,actual ,actual (Calculated based on actual liquid level changes), actual The resulting experience tuples are stored in the online experience buffer. After each 1-5 layers of printing (i.e., collecting a certain amount of practical experience), data is sampled from the online buffer, and the Actor and Critic network parameters are updated with a smaller learning rate (e.g., 1 / 10 of the learning rate in the offline stage). This means only minor adjustments are made to avoid disrupting the core strategies learned during offline training and to ensure stable control performance. If the actions output by the agent do not trigger safety restrictions during 20 consecutive layers of printing, and the liquid level stability index is better than the offline test results, then the action amplitude restrictions will be gradually lifted to allow the agent to fully adapt to the actual equipment characteristics.

[0114] Step 4: When the intelligent agent continuously completes printing of more than 100 layers on the actual equipment, and meets the requirements of "liquid level stabilization time ≤ 1.5 control cycles, deviation..." When the preset conditions of "no safety restrictions triggering and a print workpiece pass rate ≥ 99%" are met, the online adaptation is deemed complete, and the model parameters at this time are saved as the final deployment model for subsequent routine SLA printing liquid level control.

[0115] In the SLA equipment liquid level control method and apparatus based on deep reinforcement learning in the above embodiments of the present invention, the offline training phase uses a "simulation environment / experimental equipment" to replace the actual SLA production equipment for core training. Compared with training from scratch on the production equipment, this design significantly reduces the trial-and-error costs and equipment risks during the training phase, ensuring the continuity of actual production. Furthermore, it can cover most printing scenarios, laying a stable foundation for subsequent online adaptation and reducing the difficulty of adjustments during the online phase. The online adaptation phase adopts a production-while-fine-tuning mode, eliminating the need for dedicated machine shutdowns for training, significantly shortening the implementation cycle of the optimal strategy, and simultaneously improving the efficiency and quality of SLA printing.

[0116] This embodiment also provides a liquid level control device for SLA equipment based on deep reinforcement learning. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0117] This embodiment provides a liquid level control device for SLA equipment based on deep reinforcement learning, such as... Figure 4 As shown, it includes: The status acquisition module 401 is used to acquire the observation status of the SLA printing environment in real time; the observation status is multi-dimensional information. The intelligent decision-making module 402 is used to input the observation state into the trained deep reinforcement learning agent, which then outputs continuous control actions. The action execution module 403 is used to convert control actions into drive signals to control the liquid level compensation actuator to move in order to adjust the resin liquid level height. The agent in the intelligent decision-making module 402 is trained through the following process: based on the empirical data generated during the interaction with the printing environment, which consists of observed states, control actions, immediate rewards calculated according to the composite reward function, and new observed states, the agent's policy network parameters are iteratively updated to optimize the agent's control policy until the policy converges.

[0118] In some optional implementations, the observed status includes at least: the current liquid level deviation, the rate of change of liquid level deviation, the cross-sectional information of the current layer to be printed, and the negative pressure value of the scraper cavity.

[0119] In some alternative implementations, the intelligent decision-making module 402 includes: The observation state input unit is used to input the observation state into the agent's policy network; The nonlinear transformation unit is used to perform a nonlinear transformation on the observed state through the policy network, mapping it to a scalar value within a predetermined continuous interval. The control action output unit is used to output scalar values ​​as continuous control actions; wherein, the sign of the control action value corresponds to the movement direction of the liquid level compensation actuator, and the magnitude of the value corresponds to the movement amplitude of the liquid level compensation actuator.

[0120] In some optional implementations, the construction of the composite reward function includes: a precision reward term, a direction reward term, an efficiency reward term, and a stability reward term, wherein: The accuracy bonus is configured as follows: when the absolute value of the liquid level deviation is less than the preset error threshold, the bonus value is positive; otherwise, the bonus value is negatively correlated with the absolute value of the liquid level deviation. The reward value for the directional reward item is configured to be positively correlated with the reduction in liquid level deviation; The reward value for efficiency bonus items is configured to a fixed, preset negative value; The reward value for stability bonus items is configured to be negatively correlated with the drasticness of changes in control actions.

[0121] In some alternative implementations, the training process of the agent includes an offline training phase and an online adaptation phase, wherein: The offline training phase includes: collecting interaction data by simulating various printing conditions in a simulation environment or experimental equipment and storing it in an experience playback buffer; and training the agent offline using the sampled data. The online adaptation phase includes: deploying the agent model trained offline to the actual SLA device, and continuing to fine-tune the agent online using actual printing data under safety constraints.

[0122] In some alternative implementations, the agent employs an algorithm based on an actor-commentator framework, which includes an actor network and a commentator network; wherein the actor network is used to generate a probability distribution of control actions based on the observed states, and the commentator network is used to evaluate the value of states or state-action pairs and update the network parameters through temporal difference errors.

[0123] The SLA equipment level control device based on deep reinforcement learning provided in this embodiment of the invention can execute the SLA equipment level control method based on deep reinforcement learning provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0124] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0125] The following is a detailed reference. Figure 5 This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0126] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0127] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the deep reinforcement learning-based SLA equipment level control method of the present invention.

[0128] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0129] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the deep reinforcement learning-based SLA equipment level control method shown in the above embodiments is implemented.

[0130] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0131] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A liquid level control method for SLA equipment based on deep reinforcement learning, characterized in that, The method includes: The observation status of the SLA printing environment is acquired in real time; wherein, the observation status is multi-dimensional information; the observation status is the observation and measurement value of each dimension of the agent's state space at each time step; the state space of the agent includes at least the following observation dimensions: the current liquid level deviation, the rate of change of liquid level deviation, the cross-sectional information of the current layer to be printed, and the negative pressure value of the scraper cavity; The observed state is input into a trained deep reinforcement learning agent, which then outputs continuous control actions. The control action is converted into a drive signal to control the liquid level compensation actuator to move, so as to adjust the resin liquid level height. The agent is trained through the following process: based on empirical data generated during interaction with the printing environment, consisting of the observed state, the control action, the immediate reward calculated according to the composite reward function, and the new observed state, the agent's policy network parameters are iteratively updated to optimize the agent's control policy until the policy converges.

2. The method according to claim 1, characterized in that, The step of inputting the observed state into a trained deep reinforcement learning agent, which then outputs continuous control actions, includes: The observed state is input into the agent's policy network; The observation state is nonlinearly transformed by the policy network and mapped to a scalar value within a predetermined continuous interval. The scalar value is output as a continuous control action mapped in the state space of the agent; wherein, the sign of the control action corresponds to the movement direction of the liquid level compensation actuator, and the magnitude of the value corresponds to the movement amplitude of the liquid level compensation actuator.

3. The method according to claim 1, characterized in that, The construction of the composite reward function includes: a precision reward term, a direction reward term, an efficiency reward term, and a stability reward term, wherein: The accuracy bonus is configured such that: when the absolute value of the liquid level deviation is less than a preset error threshold, the bonus value is positive; otherwise, the bonus value is negatively correlated with the absolute value of the liquid level deviation. The reward value for the directional reward item is configured to be positively correlated with the reduction in liquid level deviation; The reward value for the efficiency reward item is configured as a fixed preset negative value; The reward value for the stability reward is configured to be negatively correlated with the degree of change in the control action.

4. The method according to claim 1, characterized in that, The training process of the intelligent agent includes an offline training phase and an online adaptation phase, wherein: The offline training phase includes: collecting interactive data by simulating various printing conditions in a simulation environment or experimental equipment and storing it in an experience playback buffer; and training the agent offline using the sampled data. The online adaptation phase includes: deploying the agent model obtained from offline training to the actual SLA device, and continuing to fine-tune the agent online using actual printing data under safety constraints.

5. The method according to claim 4, characterized in that, The agent employs an algorithm based on an actor-commentator framework, which includes an actor network and a commentator network. The actor network is used to generate a probability distribution of control actions based on the observed states, and the commentator network is used to evaluate the value of states or state-action pairs and update the network parameters through temporal difference errors.

6. A liquid level control device for SLA equipment based on deep reinforcement learning, characterized in that, The device includes: The status acquisition module is used to acquire the observation status of the SLA printing environment in real time; wherein, the observation status is multi-dimensional information; The intelligent decision-making module is used to input the observation state into the trained deep reinforcement learning agent, which outputs continuous control actions; wherein, the observation state is the observation and measurement value of each dimension of the agent's state space at each time step; the state space of the agent includes at least the following observation dimensions: the current liquid level deviation, the rate of change of liquid level deviation, the cross-sectional information of the current layer to be printed, and the negative pressure value of the scraper cavity. The action execution module is used to convert the control action into a drive signal to control the liquid level compensation actuator to move in order to adjust the resin liquid level height; The agent in the intelligent decision-making module is obtained through the following training process: based on the empirical data generated during the interaction with the printing environment, which consists of the observed state, the control action, the immediate reward calculated according to the composite reward function, and the new observed state, the policy network parameters of the agent are iteratively updated to optimize the control policy of the agent until the policy converges.

7. An electronic device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the SLA equipment level control method based on deep reinforcement learning as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the SLA equipment level control method based on deep reinforcement learning as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, Includes computer instructions for causing a computer to execute the SLA device level control method based on deep reinforcement learning as described in any one of claims 1 to 5.

Images