Inkjet printer printing path planning system based on deep reinforcement learning

Abstract

The present invention discloses a coding path planning system for a coding machine based on deep reinforcement learning, and relates to the field of intelligent control technology. The system comprises: a coding machine state acquisition part, a coding machine model construction part, a reinforcement learning control part and a coding path planning part. The coding machine state acquisition part is used to acquire the state data of the coding machine at each time and construct the environmental state vector of the coding machine. The coding machine model construction part is used to construct an inkjet dynamics model based on the environmental state vector to obtain a droplet position. The reinforcement learning control part is used to acquire historical environmental state vectors and pre-train a reinforcement learning model. The coding path planning part is used to control the path running trajectory of the coding machine using an optimal action vector based on a pre-trained reinforcement learning model. The real-time optimization of the coding path and operating parameters is realized, and the inkjet strategy can be adaptively adjusted under dynamically changing environmental conditions.

Description

Technical Field

[0001] The present disclosure relates to but is not limited to the field of intelligent control technology, and in particular to a coding path planning system for a coding machine based on deep reinforcement learning. Background Art

[0002] As a key identification and traceability tool in modern manufacturing, inkjet printers are widely used in many industries such as food, beverage, medicine, electronics and automobiles. The inkjet printer prints logos, codes or patterns on the surface of products through high-speed and precise ink droplet jetting to achieve product identification management and information traceability. With the development of industry and the growth of intelligent manufacturing demand, the accuracy, speed and stability requirements of inkjet printer technology are constantly increasing. How to maintain efficient and high-quality inkjet printer operations in complex industrial environments has become an important technical challenge. Most of the existing inkjet printer technologies rely on traditional control strategies and path planning methods, which are usually based on predefined rules and empirical formulas to control the jetting parameters and paths of the inkjet printer. Traditional control methods, such as proportional-integral-differential (PID) control and fuzzy control, have been widely used in the operation of inkjet printers. These methods achieve precise control of ink droplets by adjusting the jetting speed, pressure and path of the inkjet printer in real time. However, these methods have certain limitations when facing complex and changeable industrial environments and high-precision inkjet printer requirements.

[0003] First, traditional control strategies lack the ability to respond to dynamic changes in the environment in real time. In actual operation, the working environment of the inkjet printer usually has highly dynamic characteristics, such as changes in temperature, humidity, air pressure and wind speed. These environmental factors can significantly affect the behavior of ink droplets and the effect of inkjet printing. For example, in a high humidity environment, the viscosity of the ink droplets will increase, resulting in a decrease in the injection speed, affecting the clarity and consistency of the inkjet printing. On the contrary, in a low temperature environment, the fluidity of the ink is weakened, which may cause the nozzle to be blocked or the ink droplet shape to be irregular. These problems make it impossible for traditional control strategies based on fixed rules to effectively adapt to environmental changes, which can easily lead to a decrease in the quality of inkjet printing or an unstable inkjet process. Secondly, the existing path planning methods for inkjet printers are usually based on static optimization strategies, which are difficult to achieve dynamic optimization in actual operation. Traditional path planning methods, such as the shortest path algorithm and the heuristic-based path search algorithm, can optimize the motion trajectory of the inkjet printer to a certain extent, but often fail to take into account the changes in real-time environmental data and operating conditions. The limitation of this static optimization method is that they assume that all conditions and parameters in the inkjet printing process are known in advance and constant, but in fact, the operating requirements of the inkjet printer on different product surfaces and under different environmental conditions are dynamically changing. Static optimization methods have difficulty handling this complex dynamic environment, resulting in insufficient flexibility and adaptability of the coding path, which ultimately affects the quality and efficiency of coding. Summary of the invention

[0004] The present invention provides a coding path planning system for a coding machine based on deep reinforcement learning, which realizes the real-time optimization of the coding path and operating parameters, and can adaptively adjust the inkjet strategy under dynamically changing environmental conditions. The system significantly improves the adaptive ability, path planning accuracy and inkjet efficiency of the coding machine in complex industrial environments, ensures high-quality coding effects, and improves the intelligence level of the coding machine through comprehensive optimization of multiple factors, providing advanced solutions for the automation and intelligence of modern manufacturing.

[0005] In order to solve the above problems, the technical solution of the present invention is achieved as follows:

[0006] A coding path planning system for a coding printer based on deep reinforcement learning, the system comprising: a coding printer state acquisition part, a coding printer model construction part, a reinforcement learning control part and a coding path planning part; the coding printer state acquisition part is used to collect the state data of the coding printer at each time and construct the environmental state vector of the coding printer; the coding printer model construction part is used to construct an inkjet dynamics model based on the environmental state vector, and use the inkjet dry model to predict the droplet trajectory to obtain the droplet position; the reinforcement learning control part is used to collect historical environmental state vectors and pre-train a reinforcement learning model, specifically including: defining a target reward function for reinforcement learning according to the droplet position, the value of the target reward function represents the coding quality, and the higher the value, the better the coding quality. , the higher the coding quality; define the action vector of reinforcement learning, the action vector is the position of the printer at each time, and at each time, the position of the printer represents the results of all operations performed by the printer from the previous time to the current time; at each time step, calculate the best action vector in the corresponding environment state vector, under the best action vector, the target reward function value is the largest, and use the gradient clipping method to continuously update the parameters of the loss function of the reinforcement learning model until the set number of training times is reached; the coding path planning part is used to calculate the corresponding optimal action vector based on the pre-trained reinforcement learning model and the real-time environment state vector of the printer collected, and use the optimal action vector to control the path trajectory of the printer.

[0007] Furthermore, the status data includes: the nozzle position vector P t , nozzle velocity vector V t , environmental impact factor ξ(t) and substrate surface height H substrate (P t ), and the environment state vector is Among them, P t =[P x (t), P y (t), P z (t)] T , P x(t) represents the X-axis coordinate at time t, P y (t) represents the Y-axis coordinate at time t, P z (t) represents the Z-axis coordinate at time t; V t =[V x (t), V y (t), V z (t)] T , V x (t) represents the velocity in the X-axis direction at time t; V y (t) represents the velocity in the Y-axis direction at time t, V z (t) represents the velocity in the Z-axis direction at time t; H substrate (P t ) indicates that the coordinate is P z The height of the substrate surface at time (t).

[0008] Furthermore, the environmental impact factor ξ(t) is calculated using the following formula:

[0009]

[0010] Among them, P ink (t) is the ink pressure at time t; ρ air (t) is the air density at time t; T nozzle (t) is the nozzle temperature at time t; η ink (t) is the ink viscosity at time t; RH(t) is the relative humidity.

[0011] Furthermore, in the inkjet printer model construction part, based on the environmental state vector, the inkjet dynamics model constructed includes two functions, namely, the inkjet formation stage function and the jetting stage function; the inkjet formation stage is represented by the following formula:

[0012]

[0013] Where R is the droplet radius; η ink (t) / ρ is the kinematic viscosity; σ is the surface tension coefficient; ρ is the ink density; R0 is the initial droplet radius; P ∞ is atmospheric pressure; A piezo is the amplitude of the piezoelectric driver of the inkjet printer; ω is the frequency of the piezoelectric driver of the inkjet printer; β is the attenuation coefficient of the piezoelectric driver of the inkjet printer; |·| is the operation of finding the modulus of a vector or matrix.

[0014] Furthermore, the injection stage is represented by the following formula:

[0015]

[0016]

[0017]

[0018] Among them, V jet is the injection velocity; C d is the flow coefficient; P Laplace is the Laplace pressure; R nozzle is the nozzle radius; L nozzle is the nozzle length; D nozzle is the nozzle diameter; R(·) is the operation for finding the rank of the matrix.

[0019] Furthermore, in the inkjet printer model building part, the inkjet dry model is used to predict the droplet trajectory. The process of obtaining the droplet position specifically includes: defining the initial conditions of the droplet based on the environmental state vector and the inkjet dynamics model:

[0020]

[0021]

[0022] Where r0 is the initial position vector of the droplet; δ nozzle is the distance from the nozzle to the center of the sprinkler; is the unit vector of the nozzle direction; v0 is the initial velocity vector of the droplet; ω t is the nozzle angular velocity; the following droplet motion equation is established:

[0023]

[0024] Where r is the droplet position vector; g is the gravitational acceleration; C D is the resistance adjustment coefficient; v is the droplet velocity vector; Ω is the droplet angular velocity; × is the vector cross product operation.

[0025] Furthermore, the resistance adjustment coefficient is calculated using the following formula:

[0026]

[0027] Where Re is the Reynolds number, which is calculated using the following formula:

[0028]

[0029] Furthermore, the angular velocity Ω of the droplet is calculated by the following formula:

[0030]

[0031] Furthermore, the target reward function R(S t ,r) is expressed using the following formula:

[0032]

[0033] Among them, r target is the target position corresponding to the maximum value of the target reward function; V target is the target speed corresponding to the maximum value of the target reward function; σ r is the norm of the standard deviation of the droplet position; σ V is the standard deviation of the droplet velocity; P target is the ink pressure corresponding to the maximum value of the target reward function; T max is the maximum allowable temperature.

[0034] Furthermore, the loss function L(θ) of reinforcement learning defined in the reinforcement learning control part is expressed using the following formula:

[0035]

[0036] Where Q is the number of time steps per trajectory; N is the number of trajectories; π θ For strategy; i is the droplet position vector at the i-th time step; is the environmental state vector at the i-th time step; is the gradient operation; H(·) is the entropy operation.

[0037] The inkjet printer coding path planning system based on deep reinforcement learning of the present invention has the following beneficial effects:. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the system structure of a printer coding path planning system based on deep reinforcement learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present disclosure clearer and more understandable, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0040] Example 1: Reference Figure 1, a printer inkjet printing path planning system based on deep reinforcement learning, the system includes: a printer state acquisition part, a printer model construction part, a reinforcement learning control part and a printer path planning part; the printer state acquisition part is used to collect the state data of the printer at each time and construct the environmental state vector of the printer; the printer model construction part is used to build an inkjet dynamics model based on the environmental state vector, use the inkjet dry model to predict the droplet trajectory, and obtain the droplet position; the reinforcement learning control part is used to collect historical environmental state vectors and pre-train a reinforcement learning model, specifically including: according to the droplet position, define the target reward function of reinforcement learning, the value of the target reward function represents the quality of inkjet printing, and the higher the value , the higher the coding quality; define the action vector of reinforcement learning, the action vector is the position of the printer at each time, and at each time, the position of the printer represents the results of all operations performed by the printer from the previous time to the current time; at each time step, calculate the best action vector in the corresponding environment state vector, under the best action vector, the target reward function value is the largest, and use the gradient clipping method to continuously update the parameters of the loss function of the reinforcement learning model until the set number of training times is reached; the coding path planning part is used to calculate the corresponding optimal action vector based on the pre-trained reinforcement learning model and the real-time environment state vector of the printer collected, and use the optimal action vector to control the path trajectory of the printer.

[0041] Specifically, in practical applications, the working environment of the inkjet printer is usually more complex and may be affected by a variety of factors, such as changes in ambient humidity, changes in the viscosity of the inkjet medium, fluctuations in the temperature of the nozzle, etc., all of which will directly or indirectly affect the inkjet effect. Therefore, the present invention particularly emphasizes multi-dimensional real-time data acquisition in the state acquisition part to ensure that all factors that may affect the quality of the inkjet printer can be effectively monitored and recorded. Specifically, through the high-precision position sensor and speed sensor built into the inkjet printer, the system can record the motion trajectory of the inkjet printer in three-dimensional space in real time, including the current position and real-time speed changes. These data can not only reflect the motion state of the inkjet printer, but also provide real-time feedback for subsequent path planning to ensure the accuracy and stability of the inkjet path. In addition, the state monitoring of the nozzle is also an important component of the state acquisition part. The state of the nozzle includes parameters such as inkjet pressure, nozzle opening state, and inkjet volume. These data can help the system to judge the working state of the nozzle in real time and adjust the inkjet parameters in time to ensure the stability of the inkjet quality. In order to improve the accuracy and consistency of inkjet, the state acquisition part also integrates an environmental monitoring module to monitor the temperature and humidity around the inkjet printer in real time. These environmental data are crucial for the planning of inkjet paths, because different environmental conditions may cause changes in the physical properties of ink droplets, such as viscosity, evaporation rate, etc., thereby affecting the flight trajectory of ink droplets and the final inkjet effect. Therefore, these external environmental factors must be included in the environmental state vector so that the system can consider the influence of these variables during path planning and make corresponding optimization adjustments. In the data acquisition process, the state acquisition part must not only ensure the accuracy of the data, but also ensure the real-time and continuity of the data. To this end, the present invention adopts advanced sensor fusion technology and data synchronization processing algorithm to uniformly process and integrate data from different sensors to construct a unified environmental state vector. Through this method, the system can obtain the overall state information of the inkjet printer in real time and provide high-quality data input for the deep reinforcement learning model. At the same time, the state acquisition part also integrates a data preprocessing module to reduce noise and standardize the collected raw data, remove noise and outliers in the data, and ensure the accuracy and consistency of the input data. Data preprocessing includes but is not limited to methods such as mean filtering and Kalman filtering. These technologies can effectively remove interference factors such as environmental noise and equipment jitter, and further improve the quality of the data. In order to meet the training requirements of deep reinforcement learning models, the state acquisition part also needs to realize efficient data storage and call functions. The system uses high-performance data storage devices and database management systems to store and manage the real-time collected data in time series. By establishing an efficient data indexing and retrieval mechanism, the system can quickly access and call historical state data, providing massive historical data support for the training of reinforcement learning models.This method can not only speed up the training of the model, but also improve the generalization ability of the model, so that it can better adapt to different working environments and coding requirements.

[0042] The construction of the inkjet dynamics model starts from the basic physical principles of the inkjet printer, and comprehensively considers the various force conditions and motion characteristics of the droplets during the injection process. First, when the inkjet printer is performing inkjet operation, the ink droplets are ejected into the air through the nozzle at a certain initial velocity and injection angle. This initial condition determines the initial kinetic energy and direction of the droplets. Then, the droplets will be affected by a variety of forces during their flight in the air, the most significant of which are gravity and air resistance. Gravity causes the droplets to fall in a vertical direction, while air resistance is related to factors such as the speed of the droplets, the shape of the droplets, and the density of the air. The existence of resistance will cause the droplets to decelerate and deflect their trajectories. These complex physical processes need to be accurately described in the inkjet dynamics model in order to predict the trajectory and landing point position of the droplets under specific environmental conditions.

[0043] In the present invention, the inkjet dynamics model is not just a simple physical motion model, it also incorporates the impact of dynamic changes in environmental factors on the trajectory of droplets. For example, changes in temperature and humidity in the working environment of the inkjet printer will affect the viscosity of the ink droplets, thereby changing the flight stability and trajectory of the droplets. In addition, changes in environmental wind speed and wind direction will also have a significant impact on the motion trajectory of the droplets. These external environmental factors are incorporated into the inkjet dynamics model, and the parameters of the model are dynamically adjusted by analyzing the environmental state vector, so that the motion state and trajectory of the droplets under different environmental conditions can be simulated in real time. The construction of the inkjet dynamics model depends on the full analysis of the historical data of the inkjet printer, and these data are used to train and optimize the model so that it can accurately predict the trajectory and position of the droplets. In order to achieve high-precision droplet trajectory prediction, the inkjet dynamics model adopts advanced numerical calculation methods, including finite element analysis and particle swarm optimization algorithms. These methods can simulate complex droplet motion behaviors and accurately calculate the trajectory of droplets under different boundary conditions. In the process of model construction, the system also considers the evaporation and diffusion effects of droplets, which is of great significance for controlling the flight stability of droplets and the final coding effect. The evaporation effect is mainly manifested as the gradual reduction of the volume of the droplets during flight, which will cause the kinetic energy loss and trajectory deviation of the droplets. The diffusion effect refers to the diffusion phenomenon caused by the collision of hot air flow and air molecules when the droplets move in the air, which will also affect the final landing position of the droplets. The accuracy of the inkjet dynamics model directly affects the effect of the coding path planning. In the present invention, the optimization of the coding path requires not only knowing the position and state of the inkjet printer at a specific time point, but also understanding how the ink droplets will move and distribute under these states. Therefore, the precise construction of the inkjet dynamics model provides a reliable basis for the optimization of the coding path. Through continuous training and optimization of the model, the system can learn the movement laws of droplets under different conditions, so that the position and operating parameters of the inkjet printer can be adjusted predictably during the coding path planning to ensure the best coding quality. In addition, the construction of the inkjet dynamics model also provides an important basis for the definition of the reward function of reinforcement learning. In the reinforcement learning framework of the present invention, the definition of the reward function is directly related to the coding quality, and the coding quality is affected by factors such as droplet landing accuracy, coding uniformity and ink drop shape. The inkjet dynamics model can predict these behavioral characteristics of droplets, thereby helping the system build a reward function that is more in line with the actual situation, so that the reinforcement learning model can more accurately optimize the coding path during training. During the model training process, the system continuously adjusts the inkjet parameters and coding path to maximize the value of the reward function, thereby achieving the best coding effect.

[0044] In the present invention, the environmental state vector of the printer is used as the input state of reinforcement learning. This state vector contains various information of the printer at each time point, such as the position, speed, nozzle state, inkjet parameters, and external environmental conditions such as temperature, humidity, and wind speed of the printer. Based on these state data, the system needs to determine the best action of the printer at each time step. The action here is defined as the operation that the printer needs to perform in the current state, including the moving direction and speed of the printer, the switch state of the nozzle, the inkjet pressure and the injection angle, etc. These actions directly affect the quality and effect of the inkjet. Therefore, it is very important to choose a suitable action, which is also one of the core tasks of the reinforcement learning control part. In order to find the optimal action, the reinforcement learning model needs to be trained with a large amount of historical data. During the training process, the system simulates the results of the printer performing different actions in different states, and evaluates the pros and cons of these actions according to the quality of the inkjet. The reward function is the key to reinforcement learning, which is used to evaluate the effect of each action. In the present invention, the definition of the reward function is closely related to the quality of the inkjet, mainly considering factors such as the landing accuracy of the droplet, the uniformity of the inkjet, and the integrity of the ink droplet shape. The higher the value of the reward function, the better the quality of the inkjet printing, and the system will tend to choose actions that lead to higher rewards. In order to construct an effective reward function, the prediction results of the inkjet dynamics model are also introduced in the present invention. By predicting the droplet trajectory and the final landing position, the system can more accurately evaluate the inkjet printing effect, thereby optimizing the calculation of the reward function. The training process of the reinforcement learning model is a continuous iterative process. First, the system randomly selects and executes some actions, and records the results and corresponding rewards of these actions under the current environmental state. Then, through repeated training and feedback adjustment, the reinforcement learning model can gradually learn which actions can obtain higher rewards under different states. In order to avoid the problem of gradient explosion during training, the present invention adopts gradient clipping technology to stabilize the training process of the model. Gradient clipping is a commonly used optimization technique that prevents excessive gradient updates by limiting the maximum value of the gradient, thereby improving the stability and efficiency of model training. During the training process, the system also uses the experience replay method to store past experiences and randomly extract these experiences for model training to improve sample utilization and the generalization ability of the model. The ultimate goal of the reinforcement learning control part is to enable the printer to quickly calculate the best action according to the real-time environment state vector in actual operation, and guide the printer to optimize the path. In actual applications, the working environment of the printer may be very complex and changeable. The real-time environment state vector provides instant information about the current environment, and the system needs to make the best decision in a very short time. Through the reinforcement learning model trained in advance, the system can quickly determine the best action in the current state, thereby optimizing the path planning of the printer and improving the accuracy and efficiency of inkjet printing.In addition, the reinforcement learning control part also has the ability of adaptive learning. Since the working environment and operating conditions of the inkjet printer may change at any time, a single training model may not be able to cope with all scenario changes. In order to solve this problem, the reinforcement learning model in the present invention can be learned and updated online according to the new environmental data, and the system will continuously feed back the new operating data to the model, so that it can be continuously optimized and adjusted to adapt to new working conditions. This adaptive learning ability ensures that the inkjet printer can achieve the best inkjet path planning in various environments, thereby maintaining high-quality inkjet effects.

[0045] The work of the coding path planning part starts with the decision output of the reinforcement learning model. After being fully trained, the reinforcement learning model can predict the optimal position that the inkjet printer should take under different environmental conditions. In actual operation, the state acquisition part of the inkjet printer acquires the current environmental data in real time, including the current position, speed, nozzle state, inkjet pressure and other parameters of the inkjet printer, to form an environmental state vector. This environmental state vector is input into the reinforcement learning model, and the model uses these input information to calculate the optimal position to which the inkjet printer should move under the current state. This position is not randomly selected, but the result obtained after repeated training of the reinforcement learning model, which can maximize the coding quality under different operating conditions. In the process of coding path planning, the system must not only consider the current coding requirements and environmental conditions, but also foresee the future coding steps. The optimization of the coding path is a dynamic process, which requires real-time adjustment of the position of the inkjet printer in a constantly changing environment to ensure that the coding quality and efficiency of each time step are in the best state. To this end, in the present invention, the coding path planning part adopts a step-by-step path optimization strategy. At each time step, the system calculates the optimal path position between the current position of the inkjet printer and the next time step. In this way, the system can continuously adjust the position of the printer to ensure that the trajectory of the droplets during the inkjet process is accurate and correct, thereby achieving high-quality inkjet effects. Another key principle of the inkjet path planning part is the real-time path adjustment capability. In actual application scenarios, the working environment of the inkjet printer may change rapidly, such as external interference or changes in the state of the nozzle, which will affect the inkjet effect. In this case, the system needs to respond quickly and adapt to new environmental changes by adjusting the position of the inkjet printer in real time. Traditional path planning methods are usually difficult to complete such complex adjustments in a short time. In the present invention, the inkjet path planning part combines the real-time learning ability of the reinforcement learning model, and can immediately recalculate the optimal position of the inkjet printer when receiving new environmental data, so as to adjust the path. Such a dynamic adjustment mechanism can not only significantly improve the flexibility and response speed of inkjet printing, but also effectively avoid the decline in inkjet quality caused by environmental changes. In addition, the inkjet path planning part also has the ability of self-learning and optimization. After each inkjet operation, the system will compare and analyze the actual inkjet results with the expected results, and feed the error back to the reinforcement learning model. Through this feedback mechanism, the system can continuously optimize and adjust the model parameters to make it more intelligent and efficient in future coding operations. This self-learning mechanism enables the coding path planning part to continuously adapt to new operating requirements and environmental changes, and always maintain the optimal coding path planning capabilities. In the coding path planning process, the present invention places special emphasis on the precise control of the position of the printer. Precise position control is essential to achieve high-quality coding because it directly affects the landing position of the droplets and the clarity of the coding pattern.Through the precise calculation and dynamic adjustment of the reinforcement learning model, the system can accurately control the position of the printer at each time step to keep it on the optimal path. This path planning method not only improves the accuracy of inkjet printing, but also significantly reduces the defects caused by equipment errors or environmental changes.

[0046] Embodiment 2: State data includes: nozzle position vector P t , nozzle velocity vector V t , environmental impact factor ξ(t) and substrate surface height H substrate (P t ), and the environment state vector is Among them, P t =[P x (t), P y (t), P z (t)] T , P x (t) represents the X-axis coordinate at time t, P y (t) represents the Y-axis coordinate at time t, P z (t) represents the Z-axis coordinate at time t; V t =[V x (t), V y (t), V z (t)] T , V x (t) represents the velocity in the X-axis direction at time t; V y (t) represents the velocity in the Y-axis direction at time t, V z (t) represents the velocity in the Z-axis direction at time t; H substrate (P t ) indicates that the coordinate is P z The height of the substrate surface at time (t).

[0047] Specifically, the introduction of the nozzle position vector and velocity vector enables the system to understand the dynamic state of the printer in real time. These data reflect the specific position and movement speed of the printer in three-dimensional space, and directly determine the landing point position that the ink drop can reach during the jetting process. By accurately monitoring the movement trajectory of the nozzle, the system can better control the path of the inkjet to ensure that each coding can be carried out according to the predetermined trajectory, thereby reducing the possibility of coding errors and ink drop overlap. This fine path control is particularly important for high-quality coding of complex patterns, because it requires the printer to maintain extremely high coding accuracy during high-speed movement. The introduction of environmental impact factors takes into account the potential impact of the external environment on the coding process. Environmental factors such as temperature, humidity and airflow can affect the physical properties of ink droplets, such as viscosity and evaporation rate, and thus affect the flight stability and landing accuracy of ink droplets. By incorporating these environmental impact factors into the environmental state vector, the system can adjust the coding path and inkjet parameters in real time to adapt to changes in the external environment. This dynamic response capability enables the printer to always maintain the best coding quality in a variety of different working environments. Substrate surface height is another key parameter that provides important information about the surface characteristics of the coding substrate. In practical applications, inkjet printers often need to operate on uneven or height-differentiated substrate surfaces, which places high demands on coding accuracy. Real-time monitoring of the substrate surface height enables the system to dynamically adjust the position and movement trajectory of the printhead to avoid coding distortion or ink droplet scattering caused by changes in substrate height. Especially in application scenarios with high precision requirements, such as fine marking on complex product surfaces, real-time adjustment of the printhead height can significantly improve the consistency and clarity of coding.

[0048] Example 3: The environmental impact factor ξ(t) is calculated using the following formula:

[0049]

[0050] Among them, P ink (t) is the ink pressure at time t; ρ air (t) is the air density at time t; T nozzle (t) is the nozzle temperature at time t; η ink (t) is the ink viscosity at time t; RH(t) is the relative humidity.

[0051] Specifically, the first important parameter in the formula is the ink pressure P ink(t), which directly affects the kinetic energy and speed of the ink droplets at the beginning of the jet. The higher the ink pressure, the faster the ink droplets are ejected and the greater the kinetic energy, which means that the ink droplets can more effectively pass through small obstacles or disturbances in the air. However, too high ink pressure may also cause excessive dispersion of ink droplets, thus affecting the fineness of the inkjet printing. Therefore, controlling the ink pressure within a suitable range is crucial to ensure the accuracy and quality of the inkjet printing. The air density ρ air (t) affects the air resistance encountered by the ink droplets during flight. The greater the air density, the greater the resistance encountered by the ink droplets during flight, which will cause the ink droplets to slow down and deviate from the flight trajectory. The multiplication of ink pressure and air density in the formula reflects their combined effect during the inkjet process, that is, the balance between the initial velocity of the ink droplets and the flight resistance. Nozzle temperature T nozzle (t) is one of the important factors affecting the fluidity of ink. The higher the nozzle temperature, the lower the viscosity of the ink, making it easier to spray the ink and form uniform ink droplets. However, too high a nozzle temperature may cause excessive evaporation of the ink, especially in low ambient humidity, which may cause distortion of the inkjet code or scattering of ink droplets. Therefore, during the inkjet process, the nozzle temperature needs to be precisely controlled to ensure that the viscosity of the ink is within an optimal range to obtain a stable inkjet effect. Ink viscosity η ink(t) further affects the shape and stability of the ink droplets during the jetting process. The higher the viscosity, the less likely the ink droplets are to disperse and the more stable their flight trajectory. However, higher viscosity also increases the difficulty of inkjetting, especially in low temperature environments. Therefore, in the formula, the nozzle temperature and ink viscosity together constitute the denominator, reflecting their combined effect on the behavior of the ink droplets. Relative humidity RH(t) is one of the most variable factors in the external environment and has a significant impact on the inkjet process. High humidity will reduce the evaporation rate of the ink, so that the ink droplets maintain a good shape and weight when flying in the air, avoiding premature dispersion or evaporation in the air. However, too high humidity may also cause the ink to form water beads or clog the nozzle during the coding process. Therefore, changes in relative humidity need to be monitored and adjusted in real time to ensure the stability of inkjet. The humidity factor in the formula is reflected by a weighted term, which indicates the combined effect of humidity changes on the stability of ink droplet flight and coding accuracy. In the entire formula, these parameters work together to form a complex nonlinear relationship network. The calculation of the environmental impact factor ξ(t) is not a simple sum of these factors, but takes into account the interactions and trade-offs between them. For example, although both ink pressure and air density affect the flight trajectory of ink droplets, their influence directions are opposite; similarly, there is an inverse influence relationship between nozzle temperature and ink viscosity. This multiple interaction ensures that the environmental impact factor ξ(t) can fully reflect the various physical effects in the inkjet process, thereby providing an accurate feedback signal for the deep reinforcement learning model.

[0052] Embodiment 4: In the inkjet printer model construction part, based on the environmental state vector, the constructed inkjet dynamics model includes two functions, namely, the inkjet formation stage function and the jetting stage function; the inkjet formation stage is represented by the following formula:

[0053]

[0054] Where R is the droplet radius; η ink (t) / ρ is the kinematic viscosity; σ is the surface tension coefficient; ρ is the ink density; R0 is the initial droplet radius; P ∞ is atmospheric pressure; A piezo is the amplitude of the piezoelectric driver of the inkjet printer; ω is the frequency of the piezoelectric driver of the inkjet printer; β is the attenuation coefficient of the piezoelectric driver of the inkjet printer; |·| is the operation of finding the modulus of a vector or matrix.

[0055] Specifically, during the inkjet formation stage, the formation of droplets is a complex process affected by multiple physical forces. First, the change in the droplet radius R is the core element of the model, which is affected by a series of external and internal factors. The acceleration term in the formula It represents the second-order derivative of the droplet radius with time, that is, the change rate of the droplet radius, which reflects the change of the force on the droplet during its formation. This term is the embodiment of the droplet's kinetic energy, representing the movement trend caused by the unbalanced force on the droplet in the early stage of droplet formation. The following speed-related terms This further refines the change process. Here v = η ink (t) / ρ is the kinematic viscosity, which describes the internal resistance of the droplet during formation due to the viscosity and density of the ink. The surface tension and viscosity of the droplet work together to affect the stability and shape of the droplet during formation. As the droplet gradually forms in the air, this internal resistance will hinder the change in the radius of the droplet, thereby balancing the kinetic energy and surface tension of the droplet to a certain extent and ensuring the stability of the droplet formation. Surface tension plays a vital role in the droplet formation process, which is reflected in another term in the formula. The surface tension term Describes the intrinsic contraction force of the droplet due to surface tension during its formation. The surface tension coefficient σ is the interaction force between molecules on the surface of the droplet, which tends to minimize the surface area of ​​the droplet, thereby forming a stable spherical structure. The expression in the formula This further reflects the difference between the current state of the droplet and its initial state and the influence of the substrate surface height. Especially during the inkjet process, if the droplet radius R is significantly smaller than its initial radius R0, the surface tension will be stronger, causing the droplet to shrink further. In addition, the substrate surface height H substrate (P t ) also need to be considered. When the substrate surface is uneven, this unevenness will disturb the formation of the droplets, making it more difficult for the droplets to maintain a stable shape. These factors work together to determine the shrinkage and deformation behavior of the droplets during the formation process, which is crucial to ensure the morphological stability of the droplets before ejection.

[0056] In the process of droplet formation, the influence of environmental conditions cannot be ignored. The environmental influence factor |ξ(t)| in the formula is related to the ink pressure P ink The product part of |ξ(t)| describes how these environmental factors, together with the ink dynamics, affect the droplet formation process. The environmental factor |ξ(t)| takes into account changes in external environmental conditions including air density, temperature, and humidity, which can significantly affect the flow characteristics of the ink and the stability of the droplet. The ink pressure P ink(t) directly affects the initial velocity and initial kinetic energy of the droplet. The higher the ink pressure, the greater the kinetic energy the droplet gains during the formation process and the faster the injection speed, which will affect the trajectory of the droplet flying in the air and the landing accuracy. This term in the formula combines the influence of environmental factors and ink pressure, dynamically adjusts the droplet formation process, and ensures the best coding effect under various environmental conditions. In addition, the atmospheric pressure P ∞ The effect of is reflected in the formula as a negative pressure term This term reflects the inhibitory effect of the external atmosphere on droplet formation. The existence of atmospheric pressure means that droplets are always under pressure from the external environment during the formation process, which affects the expansion and formation of droplets, especially in high-altitude and low-pressure environments. In order to accurately control the formation and injection characteristics of droplets, the potential impact of changes in atmospheric pressure on droplet behavior must be fully considered. In addition to the above factors, the formula also includes terms related to the piezoelectric drive of the inkjet printer. This term describes the role of the piezoelectric actuator in the droplet formation process. The piezoelectric actuator is controlled by its amplitude A. piezo , frequency ω and attenuation coefficient β to control the droplet generation process. The vibration of the piezoelectric driver will induce pressure fluctuations of the ink inside the nozzle. These pressure fluctuations will affect the speed and shape of droplet formation. In particular, when the frequency of the piezoelectric driver matches the natural frequency of droplet formation, a resonance effect will occur, which will significantly enhance the injection speed and kinetic energy of the droplets. The attenuation coefficient β describes the attenuation of this vibration over time, which is very important for controlling the stability of droplet formation and the consistency of injection. By precisely controlling the parameters of the piezoelectric driver, the system can optimize the droplet formation process so that each inkjet can achieve the expected accuracy and quality.

[0057] Embodiment 5: The injection stage is represented by the following formula:

[0058]

[0059]

[0060]

[0061] Among them, V jet is the injection velocity; C d is the flow coefficient; P Laplace is the Laplace pressure; R nozzle is the nozzle radius; L nozzle is the nozzle length; D nozzle is the nozzle diameter; R(·) is the operation for finding the rank of the matrix.

[0062] Specifically, the injection speed V jetIt is a key parameter when the ink drop leaves the nozzle, which directly affects the behavior of the ink drop flying in the air and the final landing point. In the inkjet process, the jet speed not only determines the trajectory of the ink drop, but also affects the stability and shape of the ink drop during flight. The jet speed calculation in the formula reflects the energy conversion process in the inkjet system, in which the kinetic energy of the ink mainly comes from the ink pressure P ink (t). Ink pressure is the main driving force for accelerating ink droplet jetting. As ink pressure increases, the jetting speed of ink droplets also increases. This relationship shows that controlling ink pressure is a direct means of adjusting jetting speed. Appropriate ink pressure helps to maintain the stability of ink droplets during jetting and avoid the degradation of inkjet quality caused by too fast or too slow jetting. However, ink pressure is not the only factor that affects jetting speed. Atmospheric pressure P ∞ and Laplace pressure P Laplace It also plays an important role in the jetting process. Atmospheric pressure is a uniform pressure exerted by the external environment on the surface of the ink droplets, which has an inhibitory effect on the movement of the ink droplets. When the atmospheric pressure is high, the jetting speed of the ink droplets will be limited to a certain extent, especially in high altitude areas where the atmospheric pressure is low and the ink droplets are more likely to obtain a higher jetting speed, which will affect the accuracy and consistency of the inkjet coding. Laplace pressure is caused by surface tension and is the internal pressure that resists the separation of ink droplets. The size of the Laplace pressure is inversely proportional to the radius of curvature of the droplet. The nozzle radius R nozzle The smaller it is, the greater the Laplace pressure is, and the more difficult it is for the droplet to be ejected from the nozzle. This balance between internal and external pressures is crucial for the precise control of the ejection speed.

[0063] Flow coefficient C d The flow coefficient is also important in the formula, which describes the influence of the geometric characteristics of the nozzle on the flow. nozzle and diameter D nozzle The geometric characteristics of the nozzle directly affect the flow resistance and flow pattern of the ink. The change of the flow coefficient can adjust the size of the injection speed. By changing the length and diameter of the nozzle, the flow coefficient can be optimized to better control the injection speed and flow pattern of the ink droplets. A shorter nozzle length or a larger diameter helps to reduce the flow resistance, thereby increasing the injection speed and improving the quality and efficiency of the inkjet printing. In addition, the environmental state vector S in the formula t The environmental state vector contains various state data of the printer, such as the nozzle position, speed, environmental factors and substrate surface height. The change of the environmental state vector will affect the behavior of the ink droplets during the injection process. By considering the combined influence of these factors, R(S t) item further corrects the injection speed to ensure that the inkjet process can achieve the best effect under different environmental conditions. Especially in a complex working environment, the dynamic changes in environmental conditions will cause significant changes in inkjet behavior. By adjusting the injection speed in real time, the system can maintain high-quality inkjet effects under various conditions. In order to achieve accurate path planning and inkjet optimization, the present invention adopts a deep reinforcement learning model to learn how to optimize the injection speed and inkjet path under different environmental conditions. The reinforcement learning model uses the injection speed calculation in the formula to simulate the inkjet behavior under different operating conditions, and finds the optimal injection speed and path by continuously adjusting the system parameters. As the learning process proceeds, the model can gradually identify how to adjust the injection speed to achieve the optimal inkjet effect under different pressure conditions, environmental factors and nozzle geometric characteristics. This method enables the inkjet printer to automatically optimize its operating parameters in a complex and changeable working environment, thereby significantly improving the efficiency and quality of inkjet coding.

[0064] Example 6: In the inkjet printer model building part, the process of using the inkjet dry model to predict the droplet trajectory and obtain the droplet position specifically includes: defining the initial conditions of the droplet based on the environmental state vector and the inkjet dynamics model:

[0065]

[0066]

[0067] Where r0 is the initial position vector of the droplet; δ nozzle is the distance from the nozzle to the center of the sprinkler; is the unit vector of the nozzle direction; v0 is the initial velocity vector of the droplet; ω t is the nozzle angular velocity; the following droplet motion equation is established:

[0068]

[0069] Where r is the droplet position vector; g is the gravitational acceleration; C D is the resistance adjustment coefficient; v is the droplet velocity vector; Ω is the droplet angular velocity; × is the vector cross product operation.

[0070] Specifically, the injection speed V jet It is the core variable of the entire jetting process, which represents the initial velocity of the ink droplet ejected from the nozzle. This velocity not only determines the trajectory of the ink droplet, but also affects the air resistance and inertia force that the ink droplet is subjected to during flight, thus ultimately determining the landing point of the ink droplet and the coding effect. The jetting velocity calculation in the formula takes into account the combined effects of ink pressure, atmospheric pressure and Laplace pressure. These pressure differences determine the initial kinetic energy of the ink droplet through energy conversion relationships. Ink pressure P ink(t) provides the power source for the ink droplet ejection and is the main driving force for the ejection speed. As the ink pressure increases, the kinetic energy of the ink droplet increases, and the ejection speed also increases, so that it can pass through the nozzle and enter the air more quickly. However, the role of ink pressure is not single. During the inkjet process, the atmospheric pressure P ∞ It also has a certain inhibitory effect on ink droplet ejection. As an external pressure, atmospheric pressure is always applied to the surface of the droplet, preventing it from leaving the nozzle. Therefore, at high altitudes or in low air pressure environments, ink droplets will be relatively easier to eject, and the ejection speed will increase. This relationship is reflected in the formula through the difference between ink pressure and atmospheric pressure: a larger pressure difference means that the ink droplet has higher kinetic energy when ejected, and can more effectively resist external air resistance, maintaining the expected ejection path and flight stability.

[0071] In addition to the ink pressure and atmospheric pressure, the Laplace pressure P Laplace It is also a key factor affecting the injection speed. Laplace pressure is the internal pressure caused by the surface tension of the droplet. Especially under the condition of a small radius nozzle, this pressure will increase significantly and hinder the injection of ink droplets. The definition of Laplace pressure in the formula takes into account the surface tension coefficient σ and the nozzle radius R nozzle The relationship between the internal and external pressures is that a smaller nozzle radius will increase the Laplace pressure, resulting in greater internal resistance, thereby reducing the jetting speed. This dynamic balance between the internal and external pressures is very important. It ensures that the ink droplets can obtain sufficient kinetic energy during jetting, and will not cause jetting instability or ink droplet breakage due to excessive internal and external pressure differences. Therefore, the reasonable design and selection of nozzle geometric characteristics (such as nozzle radius and surface tension coefficient) is the key to optimizing the jetting speed and inkjet quality. Further, the flow coefficient C d It also plays an important role in the formula. It is a dimensionless parameter that describes the flow characteristics of the fluid when it passes through the nozzle. The discharge coefficient reflects the influence of the shape and geometric characteristics of the nozzle on the flow. The length of the nozzle L nozzle and diameter D nozzle Determines the flow pattern and resistance of the ink in the nozzle. Generally, a shorter nozzle length and a larger diameter help reduce the flow resistance of the ink, thereby increasing the jetting speed. The modeling of the flow coefficient in the formula enables the system to optimize the flow conditions, reduce energy loss, and improve inkjet efficiency by changing the geometric characteristics of the nozzle. This method of controlling the flow characteristics by adjusting the nozzle geometry enables the printer to maintain an efficient jetting process under a variety of operating conditions. In addition, an important correction factor is introduced in the formula, which is based on the environmental state vector S t The function R(S t ). Environmental state vector S t Includes various status information of the inkjet printer, such as the position of the printhead P t , speed Vt , environmental impact factor ξ(t) and substrate surface height H substrate (P t ) etc. These information reflect the current operating status of the printer and external environmental conditions, and their impact on the inkjet process is dynamic and complex. t ) corrects the calculation of the jetting speed, allowing the system to dynamically adjust the jetting parameters according to real-time operating and environmental conditions. For example, when the ambient temperature and humidity change, the change of the environmental influence factor ξ(t) will affect the fluidity and jetting behavior of the ink. t ) function, the system can adjust the injection speed in real time to ensure the consistency and accuracy of the coding effect. In practical applications, this dynamic adjustment mechanism based on environmental conditions is particularly important. When operating in different industrial environments, inkjet printers may encounter various unpredictable situations, such as drastic changes in ambient temperature, airflow disturbances, and uneven substrate surfaces. By combining a deep reinforcement learning model, the system can learn and adjust in a constantly changing environment to find the optimal injection speed and path planning strategy. This learning process enables the inkjet printer to adapt to different environmental conditions and operating requirements, continuously optimize its working performance, and thus improve coding quality and production efficiency.

[0072] Example 7: The resistance adjustment coefficient is calculated using the following formula:

[0073]

[0074] Where Re is the Reynolds number, which is calculated using the following formula:

[0075]

[0076] Specifically, the resistance adjustment coefficient C D The formula consists of two parts, which respectively reflect the resistance characteristics of the droplet under different flow conditions. Describes the resistance characteristics under laminar flow conditions. When the Reynolds number Re is small (usually less than 1,000), the flow state around the droplet is laminar, and the air resistance is mainly manifested as viscous resistance, and its size is inversely proportional to the Reynolds number. The term reflects this linear relationship. At the same time, in order to consider the possible transitional flow state of the droplet at a slightly higher Reynolds number, a correction term of 0.15Re is added to the formula 0.687 , this correction term can more accurately describe the change trend of the drag coefficient when the Reynolds number gradually increases. This correction ensures that the model can accurately predict the drag force on the droplet over a wider range of Reynolds numbers. It is mainly used to describe the drag characteristics of droplets under turbulent conditions. When the Reynolds number is large (usually more than 1,000), the flow state around the droplets changes to turbulence, and the air resistance is mainly manifested as inertial resistance. The characteristics of turbulence make the resistance no longer just linearly related to speed, but is affected by the complexity of fluid flow. This part of the formula reflects the change of the drag coefficient under turbulent conditions. In particular, when the Reynolds number continues to increase, the denominator term in the formula

[0077] will approach 1, so that Gradually approaching a constant value reflects the phenomenon that the resistance in the turbulent state tends to be stable. This processing method effectively captures the nonlinear characteristics of the resistance in the turbulent state, making the model more in line with the actual situation.

[0078] The calculation formula of Reynolds number Re further reveals the flow characteristics of droplets in flight. Reynolds number is a dimensionless parameter that represents the ratio between the inertial force and the viscous force of the fluid. It is an important indicator for describing the flow state (laminar or turbulent). The calculation of Reynolds number in the formula involves the diameter of the droplet (expressed as 2R), the relative velocity |vV t |、Air density ρ air (t) and ink viscosity η ink (t). These parameters together determine the flow state around the droplet. Relative velocity |vV t |Describes the speed difference of the droplet relative to the nozzle. The greater the speed difference, the greater the resistance the droplet encounters in the air and the greater the Reynolds number. Air density ρ air (t) reflects the influence of external environmental conditions on the flow state. The greater the density, the greater the inertial force of the fluid and the greater the Reynolds number. Ink viscosity η ink (t) reflects the viscosity of the droplet's own fluid properties. The higher the viscosity, the greater the viscous force of the fluid, and the smaller the Reynolds number. In the inkjet process, accurately calculate the resistance adjustment coefficient C D and Reynolds number Re are crucial for optimizing the path planning of inkjet printers. The jet speed and direction of the inkjet printer are significantly affected by air resistance. Excessive resistance will cause the ink droplet velocity to decay rapidly and deviate from the predetermined trajectory, affecting the accuracy and quality of the inkjet printing. Through the deep reinforcement learning model, the system can learn and adjust these parameters in real time, so that the inkjet process can adapt to different environmental conditions and operating requirements, thereby maintaining optimal jetting performance. The calculation of the resistance adjustment coefficient provides a key input for the inkjet dynamics model, enabling the system to more accurately predict the motion trajectory of the ink droplets, optimize the inkjet path planning, and achieve efficient and high-precision inkjet operations.

[0079] Example 8: The angular velocity Ω of the droplet is calculated by the following formula:

[0080]

[0081] Specifically, the formula for the change in the angular velocity of the droplet is The dynamic changes of the angular velocity of the droplet during flight are described through the relationship between a series of physical parameters. When the droplet moves in the air, it will be affected by multiple forces such as air resistance and fluid viscosity. These forces not only affect the linear velocity of the droplet, but also cause the droplet to rotate and generate angular velocity. The change of angular velocity is affected by multiple factors, including the viscosity of the ink η ink (t), radius of the droplet R, density of air ρ air (t) and environmental influence factor ξ(t). The combined effect of these parameters determines the speed and stability of the droplet's rotation in the air. The main driving force in the formula comes from the viscous force. The viscosity of the ink η ink (t) is a key parameter that reflects the internal friction of the fluid inside the droplet. Higher viscosity will cause the droplet to produce greater resistance when subjected to external forces, thereby affecting the droplet's rotation speed. Specifically, during the droplet's flight, the higher the viscosity, the worse the fluidity of the internal fluid, which will cause the droplet's rotation speed to increase or decrease. This process is expressed by the formula It is reflected that this item represents the relationship between the rate of change of the droplet angular velocity and the ink viscosity, droplet radius, air density and environmental influencing factors. The effect of the droplet radius R on the angular velocity is significant. The larger the radius, the stronger the inertia of the droplet and the slower the change of the angular velocity. Conversely, a smaller droplet radius makes the droplet more susceptible to external forces, resulting in a larger angular velocity change. The environmental influencing factor ξ(t) also plays an important role in the change of the droplet angular velocity. Environmental influencing factors usually include external environmental parameters such as temperature, humidity, and air pressure. These factors affect the air density ρ air (t) and the fluid properties of the droplet surface indirectly affect the rotational behavior of the droplet. Especially in the case of large environmental changes, such as when moving from a high humidity environment to a low humidity environment, the rotational behavior of the droplet may change significantly, and the angular velocity will increase or decrease due to changes in external fluid dynamic conditions. The square of ξ(t) in the formula reflects the significant influence of this environmental factor, indicating that the rate of change of the angular velocity of the droplet under different environmental conditions may be significantly different.

[0082] Ω×(vV t ) further describes the effect of the relative velocity of the droplet relative to the air on the change in angular velocity. After the droplet is ejected, its velocity v is proportional to the velocity V of the nozzle. tThere is a difference between the velocity of the droplet and the speed of the nozzle, which will cause the droplet to rotate. Specifically, when the droplet moves at a high speed relative to the surrounding air, the air resistance will form an uneven pressure distribution on the surface of the droplet, causing the droplet to rotate. The direction and rate of the droplet's rotation are determined by the cross product operation × in the formula, which represents the rotation torque caused by the difference between the droplet velocity and the nozzle velocity. This torque works together with the initial rotation state Ω of the droplet, causing the droplet's angular velocity to change over time. The calculation of the droplet's angular velocity has a direct impact on the coding path planning. By accurately predicting the change in the droplet's angular velocity, the system can better predict the posture and trajectory of the droplet during flight. This is crucial for achieving high-precision coding operations, especially in scenarios where precise control of the droplet's landing point is required. Changes in angular velocity may cause the droplet to deflect during flight, affecting the clarity and consistency of the coding pattern. Therefore, when performing path planning, the printer must consider the rotation behavior of the droplet and control the angular velocity by adjusting the inkjet parameters to ensure that the droplet flies along the predetermined trajectory and finally lands accurately. The deep reinforcement learning model used in the present invention can adjust the coding path and injection parameters in real time by learning the changing law of droplet angular velocity, thereby optimizing the coding quality. For example, when the system detects that the environmental conditions have changed and the droplet angular velocity has increased abnormally, the reinforcement learning model can immediately adjust the injection speed or angular velocity control parameters to prevent the droplet from excessively rotating in flight and ensure the accuracy and consistency of the coding. This ability to adjust and optimize in real time enables the inkjet printer to maintain stable operating performance in a variety of complex working environments.

[0083] Example 9: Target reward function R(S) defined in the reinforcement learning control part t ,r) is expressed using the following formula:

[0084]

[0085] Among them, r target is the target position corresponding to the maximum value of the target reward function; V target is the target speed corresponding to the maximum value of the target reward function; σ r is the norm of the standard deviation of the droplet position; σ V is the standard deviation of the droplet velocity; P target is the ink pressure corresponding to the maximum value of the target reward function; T max is the maximum allowable temperature.

[0086] Specifically, the target reward function R(S t,r) is expressed in two main parts, each of which takes into account the combined influence of multiple factors such as droplet position, velocity, ink pressure, and environmental conditions. The first part measures the actual position r and velocity V of the droplet through an exponential decay function. t and the target position r target and target speed V target The difference between The term describes the squared difference in distance between the current position of the droplet and the target position. The modulus of this standard deviation is σ r It is used to measure the variation range of the droplet position. If the droplet position is close to the target position, this part of the reward function will approach 1, otherwise it will approach 0, thus motivating the system to adjust the inkjet parameters to get as close to the target position as possible. Similarly, the speed deviation term It is used to quantify the difference between the actual droplet velocity and the target velocity. The standard deviation of the velocity σ V The range of variation of the droplet velocity is measured. If the droplet velocity is close to the target velocity, this part of the reward function will tend to 1, which helps to maintain the consistency and accuracy of the coding. The reward function in the first part also combines the environmental influence factor ξ(t), which is corrected by the hyperbolic tangent function tanh(ξ(t)) to reflect the impact of environmental conditions on the coding process. Environmental influence factors include parameters such as temperature, humidity, and air density, which affect the behavior of droplets and inkjet quality. By incorporating environmental influence factors into the reward function, the system can adjust the operating strategy under dynamically changing environmental conditions and optimize the coding path, thereby achieving the best coding effect in different environments. The reward function in the second part focuses on the substrate surface height H substrate (P t ) and ink pressure P ink (t). This part measures the deviation between the substrate surface height and the ink pressure through another exponential decay function. substrate (P t ) reflects the impact of the unevenness of the droplet landing surface on the quality of the inkjet print. Large height differences may cause uneven printing, affecting the clarity and consistency of the pattern. By optimizing the substrate surface height, the system can maintain uniform landing of the droplets and improve the quality of the inkjet print. Similarly, the ink pressure deviation term Used to quantify the difference between the actual ink pressure and the target pressure. If the ink pressure is close to the target value, this part of the reward function will tend to 1, which helps ensure the stability and uniformity of inkjet. The second part of the reward function also combines the nozzle temperature T nozzle The influence of (t), the influence of temperature is expressed by the sine function Correction. Temperature is a key parameter that affects ink viscosity and fluidity. Appropriate nozzle temperature helps maintain inkjet stability and droplet shape. The correction term of the sine function ensures that the temperature is close to the maximum allowable temperature T max When , the reward value reaches the maximum, which encourages the system to operate within an ideal temperature range and avoid inkjet instability caused by overheating or overcooling. In general, the target reward function R(S t ,r) By combining multiple factors such as droplet position, speed, environmental impact, substrate surface height, ink pressure and nozzle temperature, the coding path planning and control strategy are comprehensively optimized. Each part of the formula reflects the optimization requirements and physical limitations under specific operating conditions. By comprehensively considering these factors, the system can adaptively adjust the inkjet parameters under dynamically changing environmental conditions and optimize the coding path, thereby achieving efficient and high-precision coding operations. This multi-factor combined reward function design provides a clear optimization goal for the deep reinforcement learning model, enabling the system to continuously learn and improve its operating strategies, improve coding quality and production efficiency. Through this innovative path planning method, the present invention significantly improves the intelligence level and operational flexibility of the inkjet printer, and provides a more advanced and efficient solution for future industrial automation applications.

[0087] Embodiment 10: The loss function L(θ) of reinforcement learning defined in the reinforcement learning control part is expressed using the following formula:

[0088]

[0089]

[0090] Where Q is the number of time steps per trajectory; N is the number of trajectories; π θ is the strategy; ri is the droplet position vector at the i-th time step; is the environmental state vector at the i-th time step; is the gradient operation; H(·) is the entropy operation.

[0091] Specifically, the loss function L(θ) consists of two parts, which reflect the two core goals of strategy optimization and exploration diversity. The main goal of the first part is to optimize the strategy by maximizing the expected reward so that the printer can obtain the maximum reward at each time step. Specifically, the first part of the loss function Through policy gradient And the reward function The combination of θ and θ guides the model to adjust its policy parameters θ so that the path and operation selected by the model can obtain higher rewards under given environmental conditions. It is the core part of strategy optimization, which means that under the current strategy πθ Next, select a specific action. i The gradient of the logarithmic probability of the policy parameter θ. By calculating and accumulating these gradients, the model can gradually adjust its strategy so that the path and operation it chooses are closer to the optimal solution. In order to further improve the optimization effect of the strategy, a correction factor is introduced into the loss function. This factor uses the average value of the reward function and the smoothness of the hyperbolic tangent function to ensure that the model can obtain a relatively smooth policy update even when the reward does not change much. This approach helps prevent the model from falling into a local optimum and encourages the model to explore more extensively in the policy space. Emphasizes the exploration diversity of strategies, specifically through the entropy of strategies The entropy of the policy measures the randomness and diversity of the policy in a given state. The higher the entropy value, the more diverse the policy, indicating that the model is more exploratory in choosing paths and operations, rather than always choosing the same path or operation. This diversity is very important in reinforcement learning, especially in complex environments, where premature convergence to a suboptimal policy will limit the performance of the model. By introducing the entropy term, the loss function encourages the model to maintain a certain degree of exploration during the policy optimization process in order to find potential better solutions in the policy space. In addition, the loss function in the second part also combines a decay factor This factor decays exponentially as the time step t increases. This decay mechanism ensures that the model pays more attention to early decisions during the path optimization process, and these decisions often have a greater impact on the final effect of the entire path. By focusing on optimizing early decisions, the model can learn more quickly how to make the best decision in the initial state, thereby improving the overall quality and efficiency of inkjet printing.

[0092] The preferred embodiments of the present disclosure are described above with reference to the accompanying drawings, but the scope of the present disclosure is not limited thereby. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the present disclosure shall be within the scope of the present disclosure.

Claims

1. Inkjet printer path planning system based on deep reinforcement learning, It is characterized in that The system includes: a printer state acquisition part, a printer model construction part, a reinforcement learning control part and a printer path planning part; the printer state acquisition part is used to collect the state data of the printer at each time and construct the environmental state vector of the printer; the printer model construction part is used to construct an inkjet dynamics model based on the environmental state vector, use the inkjet dry model to predict the droplet trajectory, and obtain the droplet position; the reinforcement learning control part is used to collect historical environmental state vectors and pre-train a reinforcement learning model, specifically including: defining a target reward function of reinforcement learning according to the droplet position, the value of the target reward function represents the quality of the inkjet, and the higher the value, the higher the quality of the inkjet; defining the action vector of reinforcement learning The action vector is the position of the printer at each time. At each time, the position of the printer represents the results of all operations performed by the printer from the previous time to the current time. At each time step, the optimal action vector of the corresponding environmental state vector is calculated. Under the optimal action vector, the target reward function value is the largest. The parameters of the loss function of the reinforcement learning model are continuously updated using the gradient clipping method until the set number of training times is reached. The inkjet path planning part is used to calculate the corresponding optimal action vector based on the pre-trained reinforcement learning model and the real-time environmental state vector of the inkjet printer collected, and use the optimal action vector to control the path running trajectory of the inkjet printer. The state data includes: the nozzle position vector P t , nozzle velocity vector V t , environmental impact factor ξ(t) and substrate surface height H substrate (P t ), and the environment state vector is Among them, P t =[P x (t),P y (t),P z (t)] T , P x (t) represents the X-axis coordinate at time t, P y (t) represents the Y-axis coordinate at time t, P z (t) represents the Z-axis coordinate at time t; V t =[V x (t),V y (t),V z (t)] T , V x (t) represents the velocity in the X-axis direction at time t; V y (t) represents the velocity in the Y-axis direction at time t, V z (t) represents the velocity in the Z-axis direction at time t; H substrate (P t ) indicates that the coordinate is P z The environmental impact factor ξ(t) is calculated using the following formula: Among them, P ink (t) is the ink pressure at time t; ρ air (t) is the air density at time t; T nozzle (t) is the nozzle temperature at time t; η ink (t) is the ink viscosity at time t; RH(t) is the relative humidity.

2. The inkjet printer coding path planning system based on deep reinforcement learning as claimed in claim 1, It is characterized in that The inkjet printer model construction part, based on the environmental state vector, constructs an inkjet dynamics model that includes two functions, namely, the inkjet formation stage function and the jetting stage function; the inkjet formation stage is represented by the following formula: Where R is the droplet radius; η ink (t) / ρ is the kinematic viscosity; σ is the surface tension coefficient; ρ is the ink density; R 0 is the initial droplet radius; P ∞ is atmospheric pressure; A piezo is the amplitude of the piezoelectric driver of the inkjet printer; ω is the frequency of the piezoelectric driver of the inkjet printer; β is the attenuation coefficient of the piezoelectric driver of the inkjet printer; |·| is the operation of finding the modulus of a vector or matrix.

3. The inkjet printer coding path planning system based on deep reinforcement learning as claimed in claim 2, It is characterized in that The injection phase is represented by the following formula: Among them, V jet is the injection velocity; C d is the flow coefficient; P Laplace is the Laplace pressure; R nozzle is the nozzle radius; L nozzle is the nozzle length; D nozzle is the nozzle diameter; R(·) is the operation for finding the rank of the matrix.

4. The inkjet printer coding path planning system based on deep reinforcement learning as claimed in claim 3, It is characterized in that In the printer model building part, the inkjet dry model is used to predict the droplet trajectory. The process of obtaining the droplet position specifically includes: defining the initial conditions of the droplet based on the environmental state vector and the inkjet dynamics model: Among them, r 0 is the initial position vector of the droplet; δ nozzle is the distance from the nozzle to the center of the sprinkler; is the unit vector of the nozzle direction; v 0 is the initial velocity vector of the droplet; ω t is the nozzle angular velocity; the following droplet motion equation is established: Where r is the droplet position vector; g is the gravitational acceleration; C D is the resistance adjustment coefficient; v is the droplet velocity vector; Ω is the droplet angular velocity; × is the vector cross product operation.

5. The inkjet printer coding path planning system based on deep reinforcement learning as claimed in claim 4, It is characterized in that The drag adjustment factor is calculated using the following formula: Where Re is the Reynolds number, which is calculated using the following formula:

6. The inkjet printer coding path planning system based on deep reinforcement learning as claimed in claim 5, It is characterized in that The angular velocity Ω of the droplet is calculated by the following formula:

7. The inkjet printer coding path planning system based on deep reinforcement learning as claimed in claim 6, It is characterized in that The target reward function R(S) defined in the reinforcement learning control part t ,r) is expressed using the following formula: Among them, r target is the target position corresponding to the maximum value of the target reward function; V target is the target speed corresponding to the maximum value of the target reward function; σ r is the norm of the standard deviation of the droplet position; σ V is the standard deviation of the droplet velocity; P target is the ink pressure corresponding to the maximum value of the target reward function; T max is the maximum allowable temperature.

8. The inkjet printer coding path planning system based on deep reinforcement learning as claimed in claim 7, It is characterized in that The loss function L(θ) of reinforcement learning defined in the reinforcement learning control part is expressed using the following formula: Where Q is the number of time steps per trajectory; N is the number of trajectories; π θ For strategy; i is the droplet position vector at the i-th time step; is the environmental state vector at the i-th time step; is the gradient operation; H(·) is the entropy operation.

Images