Intelligent planning method for painting trajectory based on deep reinforcement learning

Abstract

The application relates to the technical field of artificial intelligence, and discloses a coating track intelligent planning method based on deep reinforcement learning, which abstracts a spraying process as a Markov decision process, and uses the interaction between a spray gun and a simulation environment in the spraying process, so that the determination of the coating track is closely related to the real-time state of the surface of a workpiece to be sprayed; the coating track intelligent planning method based on deep reinforcement learning comprehensively considers the coupling influence of process parameters and the spraying track in the spraying process, effectively obtains an optimal coating track, and effectively solves the problems of improving coating thickness control of a complex surface and calculating a spraying track.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, and computer-readable storage medium for intelligent planning of painting trajectories based on deep reinforcement learning. Background Technology

[0002] Since the successful development of industrial robots in 1959, their high speed, high precision, and repeatability have gradually replaced manual labor in various production activities. Compared to traditional manual spraying, automated spraying using robots can solve problems such as difficulty in controlling coating thickness deviations, difficulty in improving work efficiency, and difficulty in avoiding the harm to human health caused by toxic solvents. Spraying robots have developed rapidly since their introduction to the market and are widely used in aerospace, automotive, and other industrial manufacturing fields.

[0003] Because industrial products such as aerospace products are mostly large in size, have special types of coatings, and are characterized by complex processes and varied production modes, higher requirements are placed on intelligent spraying processes. It is necessary to select the optimal spraying parameters and the best spraying trajectory based on existing spraying equipment and external environment.

[0004] In existing technologies, coating parameters and trajectories in intelligent spraying processes are mostly determined through process experimentation or by solving optimization models with the uniformity of coating thickness on the workpiece surface as the optimization objective. Among these, process experimentation requires significant manpower and resources and is complex, making it difficult to meet the requirements of real-time performance and safety. Currently, commonly used optimization algorithms for optimization models include particle swarm optimization, genetic algorithms, and simulated annealing algorithms. However, the drawbacks of these traditional algorithms are as follows: 1) It has problems such as slow convergence speed and many control variables; 2) Before solving the optimization problem, it is necessary to obtain all possible trajectories. This is a parameter optimization method for known trajectories. It has the problems of low solution efficiency, difficulty in obtaining the global optimal solution, lack of comprehensive consideration of the combination optimization between spraying trajectory and process parameters, poor generalization ability, and lack of universality.

[0005] Therefore, there is an urgent need for an intelligent coating trajectory planning method that can comprehensively consider the influence of spraying process parameters and spraying trajectory optimization. Summary of the Invention

[0006] This invention provides a method, system, electronic device, and storage medium for intelligent planning of coating trajectories based on deep reinforcement learning, in order to solve at least one problem in the prior art.

[0007] To achieve the above objectives, this invention provides an intelligent painting trajectory planning method based on deep reinforcement learning, applied to electronic devices, comprising: The model data of the workpiece to be coated is obtained, and the model of the workpiece to be coated is divided into grids according to the preset coating accuracy standard. The average value of the coating thickness of all point clouds in all grids at a certain moment is vectorized to obtain the initial state vector. The initial state vector is input into the pre-trained coating control strategy model based on deep reinforcement learning to obtain the spraying action vector; and the spraying trajectory at the next moment is determined based on the spraying action vector. In the simulation environment, the spraying action is executed according to the spraying trajectory corresponding to the next moment of the spraying action vector. The influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed is determined by using the paint film thickness deposition model. The state vector for the next moment is updated based on the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed. Based on the state vector at the next moment, the spraying trajectory at the next moment corresponding to the spraying action vector is determined using a pre-trained coating control strategy model based on deep reinforcement learning. The process is repeated until the state vector, spraying action vector, and spraying trajectory corresponding to the spraying action vector at each moment in the spraying process are determined.

[0008] Furthermore, preferably, the coating control strategy model based on deep reinforcement learning is iteratively trained by setting a maximum number of iterations. The iterative training method includes: The parameters of the coating control strategy model based on deep reinforcement learning are randomly initialized, the network training iteration count is initialized to 1, and the experience replay pool of deep reinforcement learning is cleared; wherein, the coating control strategy model based on deep reinforcement learning includes... dimensional state vector For input, 3D action vector For output Network and The network, and, with 3D action vector For input, numerical values For output Network and Network; among which, Network and Each network consists of two networks; the parameters of the coating control strategy model based on deep reinforcement learning include... Network parameters Network parameters, Network parameters and The network parameters; the The network parameters are obtained by copying the corresponding... The network parameters are obtained. The network parameters are obtained by copying the corresponding... The network parameters are obtained; Update state vector Based on state vector Get The network outputs the spraying action vector. ; Based on spraying action vector Calculate the spray trajectory corresponding to the spraying motion vector. The paint film thickness distribution of the workpiece to be coated is updated using a paint film thickness deposition model to obtain the state vector. ; Based on state vector and reward function Calculate real-time rewards ; and with Whether the condition is met determines whether the spraying process has been terminated; Based on the obtained state vector Spraying motion vector Dynamic state vector and real-time rewards Determine the set ; set Stored in the deep learning experience replay pool ,make The process continues in a loop until the experience replay pool is reached. Maximum capacity; From the experience replay pool In the process Second random sampling, to obtain , Calculate the current policy The following theory value; Using the mean squared error loss function, the gradient backpropagation of the neural network is applied to... Network parameters Update; When the number of iterations is an integer multiple of the delayed update base d, the loss function is used to backpropagate the gradients of the neural network. Network parameters Update and provide the updated version. Network parameters and The network parameters are copied to the corresponding... Network and network; Perform iterative training until the maximum number of iterations is reached; obtain a well-trained coating control strategy model based on deep reinforcement learning.

[0009] Furthermore, preferably, the state vector , Indicates array index as Paint film thickness of all data points in the point cloud cluster The average value; Among them, point cloud clusters Medium data points Paint film thickness Obtain it through the following steps: Calculate the spray gun at the trajectory point Duration of stay ; With trajectory points Construct a local coordinate system with the origin as the origin. Using a paint film thickness deposition model to search for point cloud clusters Data points located in the coating area Calculate data points In time The thickness of the paint film obtained inside ; Calculate the trajectory separately Each trajectory point to data point The resulting paint film deposition thickness Therefore, the final paint film thickness for: .

[0010] Furthermore, preferably, the coating thickness deposition model is a free-form surface coating thickness distribution model obtained by converting the planar coating thickness model; The freeform surface coating thickness distribution model, derived from the planar coating thickness model, is achieved through the following formula:

[0011] in, This indicates the point cloud clusters within the current spraying area. Surface data points The coating thickness deposition rate The coating thickness deposition rate on the reference plane is represented by the following: Representing data points Coordinate description; This indicates the distance (height) from the center point of the spray gun to the theoretical plane. The distance from the center point of the spray gun to the reference plane is the height. Representing data points Tangent plane normal vector The angle with respect to the axis of the spray gun, Indicates the angle between the spray gun axis and the direction of the vertical segment; Let these represent the lengths of the major and minor axes of the spraying ellipse on the theoretical plane. The coating deposition rate coefficient representing the theoretical plane. This represents the distribution coefficient of the sedimentation model.

[0012] Further, preferably, the method for acquiring model data of the workpiece to be coated and dividing the workpiece into grids according to a preset accuracy standard includes, Obtain the model data of the workpiece to be coated; wherein, the model data is point cloud data or Model data; Based on the model data of the workpiece to be sprayed, the point cloud data of the three-dimensional space of the workpiece to be sprayed is obtained using a vision sensor, or the CAD model data is converted into point cloud data. Determine the interval size of the segmentation according to the accuracy requirements, and then divide the workpiece to be coated according to the interval size. The data is rasterized and then back-mapped to point cloud data in three-dimensional space. Segmenting the point cloud data in three-dimensional space into The point cloud clusters are composed of an array for each raster cell, and each array contains the numbers of the point cloud data points contained in the current raster cell.

[0013] Furthermore, preferably, through strategy-based The state value function obtains the current policy. The following theory Value; based on strategy State value function This can be achieved through the following formula:

[0014] in, For state The expected reward that can be obtained in the next moment. This is the discount factor.

[0015] Furthermore, preferably, based on state vectors and reward function Calculate real-time rewards This can be achieved through the following formula:

[0016] in, Represents point cloud clusters On the coordinate axes X Minimum value in direction, This indicates the corresponding trajectory point in the spatial coordinate system. Below coordinate, Indicates the ideal paint film thickness. The array index in the simulation model is... data points The thickness of the paint film, This represents the coefficient of the reward function.

[0017] To address the aforementioned problems, this invention also provides an intelligent painting trajectory planning system based on deep reinforcement learning, comprising: The data acquisition unit is used to acquire model data of the workpiece to be coated, and divide the model of the workpiece to be coated into grids according to the preset coating accuracy standard. The average value of the coating thickness of all point clouds in all grids at a certain moment is vectorized to obtain the initial state vector. The state vector update unit is used to input the initial state vector into the pre-trained coating control strategy model based on deep reinforcement learning to obtain the spraying action vector; and to determine the spraying trajectory at the next moment based on the spraying action vector. In the simulation environment, the spraying action is executed according to the spraying trajectory corresponding to the next moment of the spraying action vector. The influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed is determined by using the paint film thickness deposition model. The state vector for the next moment is updated based on the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed. Based on the state vector at the next moment, the spraying trajectory at the next moment corresponding to the spraying action vector is determined using a pre-trained coating control strategy model based on deep reinforcement learning. The execution unit is used to execute cyclically until the state vector, spraying action vector, and spraying trajectory corresponding to the spraying action vector at each moment in the spraying process are determined.

[0018] To address the aforementioned problems, the present invention also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps in the aforementioned intelligent planning method for coating trajectories based on deep reinforcement learning.

[0019] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the above-described intelligent planning method for coating trajectories based on deep reinforcement learning.

[0020] The intelligent coating trajectory planning method based on deep reinforcement learning provided by this invention, based on the coating deposition model, abstracts the spraying process into a Markov decision process. Utilizing the interaction between the spray gun and the simulation environment during the spraying process, the determination of the spraying trajectory is closely related to the real-time state of the workpiece surface. By considering local and global rewards and learning the optimal execution strategy to achieve the goal of uniform paint film thickness distribution, this intelligent coating trajectory planning method based on deep reinforcement learning differs from traditional parameter optimization processes based on known trajectories. It comprehensively considers the coupling influence of process parameters and spraying trajectory during the spraying process, achieving the technical effect of effectively obtaining the optimal coating trajectory and effectively improving the problems of coating thickness control and spraying trajectory calculation for complex curved surfaces. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the intelligent planning method for painting trajectories based on deep reinforcement learning according to an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the principle of converting a CAD model into three-dimensional spatial point cloud data based on a sampling method according to an embodiment of the present invention. Figure 3 This is a schematic diagram illustrating the principle of rasterizing the point cloud to be sprayed according to an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the training principle of the model parameters of the coating control strategy model based on deep reinforcement learning according to an embodiment of the present invention. Figure 5 This is a schematic diagram illustrating the principle of a planar coating thickness deposition distribution model in the prior art. Figure 6 This is a schematic diagram illustrating the principle of a free-form surface coating thickness deposition distribution model according to an embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the principle of obtaining the spraying trajectory based on the slicing method according to an embodiment of the present invention; Figure 8 A logical structure block diagram of a coating trajectory intelligent planning system based on deep reinforcement learning according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the internal structure of an electronic device that implements a deep reinforcement learning-based intelligent planning method for coating trajectories according to an embodiment of the present invention.

[0022] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0024] This application's embodiments can acquire and process relevant data based on artificial intelligence (AI) technology. AI is the theory, methods, technology, and application system that uses digital computers or computers-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. The AI ​​software technology in this invention is machine learning technology based on digital twin models.

[0025] Deep Reinforcement Learning (DRL) is a novel algorithm that combines deep learning with reinforcement learning to achieve end-to-end learning from perception to action. A reinforcement learning system has four main elements: policy, reward function, value function, and optional model of the system. The policy is primarily what is of interest when mapping the perceived state of the system to actions. The reward function defines the objective of the problem as a mapping between states (or state-action pairs) and a single numerical reward that captures the expected nature of the situation. The goal of the system is to identify the policy that maximizes the reward. The value function is a prediction of the future rewards achievable from the current state used to formulate the policy. The optional model is an approximation of the environment that can be used for planning purposes.

[0026] To address the problems of existing technologies that fail to comprehensively consider the combined optimization of spraying trajectory and process parameters, resulting in poor generalization ability and lack of versatility, this invention abstracts the spraying process into a Markov decision process. By utilizing the interaction between the spray gun and the simulation environment during the spraying process, the determination of the coating trajectory is closely related to the real-time state of the workpiece surface to be sprayed. The intelligent coating trajectory planning method based on deep reinforcement learning in this invention comprehensively considers the coupled influence of process parameters and spraying trajectory during the spraying process, effectively obtaining the optimal coating trajectory and significantly improving the problems of coating thickness control and spraying trajectory calculation for complex curved surfaces.

[0027] Specifically, as an example, Figure 1 This is a flowchart illustrating an embodiment of the intelligent painting trajectory planning method based on deep reinforcement learning provided by the present invention. (Refer to...) Figure 1 As shown, the present invention provides an intelligent planning method for painting trajectories based on deep reinforcement learning. This method can be executed by a device, which can be implemented by software and / or hardware.

[0028] In this embodiment, the intelligent planning method for painting trajectory based on deep reinforcement learning includes steps S110 to S150.

[0029] S110. Obtain the model data of the workpiece to be coated, and divide the model of the workpiece to be coated into grids according to the preset coating accuracy standard. Arrange the average value of the coating thickness of all point clouds in all grids at a certain moment into vectors to obtain the initial state vector.

[0030] It should be noted that, in the specific implementation process, the vision sensor can be, but is not limited to, LiDAR or a scanner. Specifically, for acquiring the three-dimensional point cloud data of the workpiece to be coated using a vision sensor... Data points.

[0031] Figure 2 This is a schematic diagram illustrating the principle of converting a CAD model into three-dimensional spatial point cloud data based on a sampling method according to an embodiment of the present invention; for example... Figure 2 As shown, for data that is inconvenient to collect or already exists... The workpiece of the model can be sampled using, but is not limited to, uniform sampling methods. The model is converted into 3D point cloud data, specifically... Data points.

[0032] Specifically, the method for acquiring model data of the workpiece to be coated and dividing the workpiece into grids according to a preset accuracy standard includes: acquiring model data of the workpiece to be coated; wherein, the model data is point cloud data or... Model data; based on the model data of the workpiece to be coated, use a vision sensor to acquire point cloud data of the three-dimensional space of the workpiece to be coated, or convert the CAD model data into point cloud data; determine the segmentation interval size according to the accuracy requirements, and segment the workpiece to be coated according to the segmentation interval size. The data is rasterized and then back-mapped to point cloud data in 3D space; the point cloud data in 3D space is then segmented into... The point cloud clusters are composed of an array for each raster cell, and each array contains the numbers of the point cloud data points contained in the current raster cell.

[0033] In other words, the workpiece to be coated is divided into grids according to accuracy requirements, which can be done using, but is not limited to, principal component analysis. In the specific implementation process, the geometric principal direction and attitude transformation center point of the workpiece to be coated are calculated, with the geometric principal direction as... An axis is used to construct a three-dimensional spatial coordinate system with the center point of attitude transformation as the origin. .like Figure 2 As shown, according to the interval Will The projection region is divided into The raster region is then mapped back to the point cloud data in three-dimensional space, dividing the point cloud data into... The point cloud clusters are composed of an array for each raster cell, and each array contains the numbers of the point cloud data points contained in the current raster cell.

[0034] dotted cloud clusters Medium data points Array index The value is obtained through the following formula:

[0035] in, Representing data points On the coordinate axis Coordinates in direction, They represent point cloud clusters respectively. On the coordinate axes Minimum value in direction, sign This represents the floor function.

[0036] After dividing the model of the workpiece to be coated into grids according to the preset coating accuracy standard, the average value of the coating thickness of all point clouds in all grids at a certain moment is vectorized to obtain the initial state vector.

[0037] Specifically, the state vector , Indicates array index as The average paint film thickness of all data points in the grid, i.e.:

[0038] S120. Input the initial state vector into the pre-trained coating control strategy model based on deep reinforcement learning to obtain the spraying action vector; and determine the spraying trajectory at the next moment based on the spraying action vector.

[0039] The spraying process is abstracted as a Markov reward decision process, and a system is established with variable parameters such as spraying distance, spraying speed, and overlap spacing as independent variables. Vectors are established, with global paint film thickness distribution and local paint film thickness influence as independent variables. Functions; based on deep reinforcement learning models and different The thickness distribution of the coating film changes in the simulation environment under vector input, and the loss function is applied to... and Update the network parameters and then update the data. Network parameters and The network parameters are copied to the corresponding... Network and network.

[0040] It should be noted that the spraying motion vector can include, but is not limited to, parameters such as spraying distance, spraying speed, and overlap spacing.

[0041] Figure 4 This is a schematic diagram illustrating the training principle of the model parameters for a coating control strategy model based on deep reinforcement learning according to an embodiment of the present invention; as shown. Figure 4 As shown, The coating control strategy model based on deep reinforcement learning includes two main networks and two target networks; the main network includes... Network and Network; and the target network includes Network and Network. In other words, the coating control strategy model based on deep reinforcement learning includes... dimensional state vector For input, 3D action vector For output Network and The Internet, and 3D action vector For input, numerical values For output Network and Network; wherein, the Critic network and the Critic-Target network each contain two networks, namely Critic network 1 and Critic network 2, Critic-Target network 1, Critic Target Network 2, in which, The network parameters are obtained by copying the corresponding numbered Critic network. Actor Target The network parameters are also obtained by copying the Actor network. The parameters of the coating control strategy model based on deep reinforcement learning include... Network parameters Network parameters Network parameters and Network parameters.

[0042] The main function of the experience replay pool is to address the issues of correlation and non-static distribution during model training. Specifically, it uses the transition samples obtained from the agent's interaction with the environment at each time step. Stored in the playback memory unit, some are randomly retrieved during training. minibatch(This involves breaking down the training process into fragments for storage, and then randomly selecting data during training to avoid correlation issues.)

[0043] Using the mean squared error loss function, the gradient backpropagation of the neural network is applied to... Network parameters Update; Use The loss function is applied through backpropagation of gradients in the neural network. Network parameters Update.

[0044] By setting the target network, specifically... This represents the output of the current network (Main Net), a value function used to evaluate the current state-action pair; This represents the output of the target network (Target Net), used to obtain the target Q-value. The parameters of the current network (Main Net) are updated using their respective loss functions. Every N iterations, the parameters of the current network (Main Net) are copied to the target network (Target Net), thus periodically updating the parameters of the target network (Target Net). This reduces the correlation between the current Q-value and the target Q-value to some extent, improving the algorithm's stability.

[0045] In one specific embodiment, the coating control strategy model based on deep reinforcement learning is iteratively trained by setting a maximum number of iterations. The iterative training method includes steps S121-S128.

[0046] S121. Randomly initialize the parameters of the coating control strategy model based on deep reinforcement learning. Initialize the network training iteration count to 1 and clear the deep reinforcement learning experience replay pool. D It should be noted that the network update rate is also initialized, typically set to 0.001; and the number of training epochs is also initialized to 1, thus initializing the training time. t= 1 ; S122, Update the state vector Based on state vector Get The network outputs the spraying action vector. It should be noted that the state vector is... It is a random state vector.

[0047] S123, Based on spraying motion vector Calculate the spray trajectory corresponding to the spraying motion vector. The paint film thickness distribution of the workpiece to be coated is updated using a paint film thickness deposition model to obtain the state vector. ; S124, Based on state vector and reward function Calculate real-time rewards ; and with Whether the condition is met determines whether the spraying process should be terminated; it should be noted that this is done using Boolean values. To describe, that is Whether the condition is met determines whether the spraying process should be terminated.

[0048] S125. Based on the obtained state vector Spraying motion vector Dynamic state vector and real-time rewards Determine the set ; set Stored in the deep learning experience replay pool ,make The process continues in a loop until the experience replay pool is reached. Maximum capacity; S126, From the experience replay pool In the process Second random sampling, to obtain , Calculate the current policy The following theory Value (State Action) Q (Function). It should be noted that if the experience replay pool... If the number of samples in the pool exceeds a set value, then a set number of samples are selected from the experience replay pool for sampling. This is based on state s. j Get Actor Target Network output a j , using s j and a j Calculate two Critic Target The network outputs Q1 and Q2, and the minimum value is taken as Q.

[0049] Current Strategy The following theory value Obtain it using the following formula:

[0050] in, This represents the reward decay factor. Indicates network parameters as of The network in the state vector The corresponding output spraying action vector under the input. , Indicates network parameters as of The network outputs corresponding to the corresponding inputs.

[0051] S127. Using the mean squared error loss function Through gradient backpropagation of neural networks Network parameters Perform an update; use the loss function. Through gradient backpropagation of neural networks Network parameters Update; and the updated Network parameters and The network parameters are copied to the corresponding... Network and The network involves using a soft update method to copy the parameters of the Actor network and Critic network 1 and Critic network 2 to the corresponding Target network.

[0052] Specifically, after a certain number of iterations For the coating control strategy model based on deep reinforcement learning The update is performed using the following formula:

[0053] in, , To enhance the learning rate during the learning process.

[0054] S128. Perform iterative training until the set maximum number of iterations is reached; obtain the trained coating control strategy model based on deep reinforcement learning. It should be noted that if at time... t A training round ends when the set maximum number of training sessions is reached. If eps reaches the set maximum number of rounds, offline training ends, and the agent that has completed offline training is obtained.

[0055] In summary, the deep reinforcement learning model and different... The thickness distribution of the coating film changes in the simulation environment under vector input, and the loss function is applied to... and Update the network parameters and then update the data. Network parameters and The network parameters are copied to the corresponding... Network and Network; is the number of grid divisions based on a point cloud model. and known Vector Dimension , construct dimensional state vector For input, 3D action vector For output and Multilayer network model and 3D action vector For input, numerical values For output and A multi-layer network model, and trained according to the deep reinforcement learning process. Network parameters Update.

[0056] Specifically, the deep reinforcement learning-based coating control strategy model is trained until it converges. Then, the trained model is used to obtain intelligent planning data for the coating trajectory.

[0057] It should be noted that the paint film thickness deposition model and model parameters used in the simulation environment are set. By extracting key factors affecting the robot spraying process, a multivariable spray gun coating thickness deposition model is constructed, and the model parameters are initialized to simulate the changes in the paint film thickness on the workpiece surface during the robot spraying process. Commonly used thickness deposition models include... Distribution models, analytical sedimentation models, parabolic sedimentation models, and elliptical double sedimentation models Distribution models, etc.

[0058] In one specific embodiment, the coating thickness deposition model is a freeform surface coating thickness distribution model obtained by converting a planar coating thickness model. This conversion is based on an elliptic double... A coating thickness deposition model and differential amplification principle can be implemented.

[0059] Figure 5 and Figure 6 A general description of the paint film thickness deposition model was provided; among which, Figure 5 This is a schematic diagram illustrating the principle of a planar coating thickness deposition distribution model in the prior art. Figure 6 This is a schematic diagram illustrating the principle of a freeform surface coating thickness deposition distribution model according to an embodiment of the present invention. Figure 5 and Figure 6 As shown, The freeform surface coating thickness distribution model, derived from the planar coating thickness model, is achieved through the following formula:

[0060] in, This indicates the point cloud clusters within the current spraying area. Surface data points The coating thickness deposition rate The coating thickness deposition rate on the reference plane is represented by the following: Representing data points Coordinate description; This indicates the distance (height) from the center point of the spray gun to the theoretical plane. The distance from the center point of the spray gun to the reference plane is the height. Representing data points Tangent plane normal vector The angle with respect to the axis of the spray gun, Indicates the angle between the spray gun axis and the direction of the vertical segment; Let these represent the lengths of the major and minor axes of the spraying ellipse on the theoretical plane. The coating deposition rate coefficient representing the theoretical plane. This represents the distribution coefficient of the sedimentation model.

[0061] The above can be achieved through elliptic bipolar... Point cloud data points of paint film thickness deposition model Paint film thickness The numerical calculation can be performed as follows: Calculate the spray gun at the trajectory point Duration of stay ,have:

[0062] With trajectory points Construct a local coordinate system with the origin as the origin. and elliptical double Paint film thickness deposition model, finding point clouds Data points located within the elliptical coating area Calculate data points In time The thickness of the paint film deposited inside is:

[0063] Repeat the above process to calculate the trajectory respectively. Each trajectory point to data point The resulting paint film deposition thickness The final paint film thickness It can be represented as:

[0064] In the specific implementation process, the method of determining the spray trajectory at the next moment based on the spraying action vector can be, but is not limited to, the slicing method. Figure 7 This is a schematic diagram illustrating the principle of obtaining the spraying trajectory based on the slicing method according to an embodiment of the present invention; as shown. Figure 7 As shown, based on the spraying action vector Discretized spraying trajectories can be calculated; specifically, spatially scattered point clouds, i.e., point cloud clusters, can be computed. Intersecting a set of parallel planes, and translated along the normal vector direction of the tangent plane at the intersection point. Distance allows us to obtain the spraying motion vector. Corresponding spraying trajectory Spraying trajectory Consisting of several intervals Discrete trajectory points The composition is such that the direction of the normal vector of the aforementioned parallel plane is consistent with the orientation of the spraying trajectory.

[0065] S130. In the simulation environment, the spraying action is executed according to the spraying trajectory corresponding to the next moment of the spraying action vector. Using the paint film thickness deposition model, the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed is determined.

[0066] Because the spraying process is abstracted as a Markov reward decision process, and a system is established with variable parameters such as spraying distance, spraying speed, and overlap spacing as independent variables. Vectors are established, with global paint film thickness distribution and local paint film thickness influence as independent variables. Therefore, it is necessary to determine the impact of each spraying trajectory on the paint film thickness distribution of the workpiece under simulation using a paint film thickness deposition model.

[0067] Specifically, the exemplary robotic painting problem solved by reinforcement learning (RL) can be represented by a quintuple of Markov decision processes (MDPs).<S,A,T,R,γ> Let S be the state space, A be the action space, T be the state transition model, R be the reward function, and γ be the discount factor for the cumulative reward. Reinforcement learning is used to optimize policy π to achieve a specific goal by enabling an agent to learn from its interactions with the environment. The agent observes a state S from the environment, chooses action A given by policy π to be executed in the environment, observes the next state, and simultaneously receives a reward R, until the endpoint state is reached. Therefore, the expected cumulative reward obtained by the optimal policy π is maximized. The goal of reinforcement learning methods is to find the optimal policy π. For some implementations, reinforcement learning requires interaction with the environment to understand the benefits of each policy, and then combines the policies for sequential decision-making.

[0068] In this embodiment, the actual spraying process is arranged in an alternating chronological order, and described using the following state-action chain (Markov chain):

[0069] in, express The choice of spraying method at any given time can be determined by multiple factors, including whether to spray, the location of the spraying point, and the spraying process parameters. express The distribution of paint film thickness on the surface of the workpiece to be coated at any given time. Status The coating thickness deposition model constructed as described above is used for the description.

[0070] The spraying process is abstracted as a Markov decision process, and the time steps are extracted. The spraying action constitutes a deep reinforcement learning network Network input Vector, spraying action vector This may include, but is not limited to, spraying speed. Spraying distance Trajectory overlap distance wait.

[0071] Right now ,and:

[0072] in, These represent the variable parameters in the spraying trajectory. The theoretical minimum and maximum values, These represent the variable parameters in the spraying trajectory. The theoretical minimum and maximum values, These represent the variable parameters in the spraying trajectory. The theoretical minimum and maximum values.

[0073] In order to obtain a deep reinforcement learning strategy It is necessary to specify the vector for each spraying action. An objective evaluation of the actual effects produced should be made, and the corresponding reward function should be determined according to the established method. R .

[0074] Policy-based approaches can be introduced. State value function :

[0075] in, For state The expected reward that can be obtained in the next moment. This is the discount factor. It should be noted that the state value function is the expected value of the reward obtained from a given state when the action is determined according to policy π. V π (s) This means that the state value function represents the expected cumulative reward of the painting robot in state s until the painting action is completed. The policy π of the policy network aims to maximize the state value function. When s is in state s t Choose action a t As a result, the reward r was observed. t+1 and the state of the next transition t+1 At that time, the action a chosen from the state after transitioning to a one-hour step size. t+1 To update the action value Q (st，at) .

[0076] For the spraying process, the spraying direction is used as the spatial coordinate system. Down Taking direction as an example, the reward function It can be determined, but is not limited to, by the following method:

[0077] in, , The specific value depends on the position of the currently calculated trajectory; Represents point cloud clusters On the coordinate axes X Minimum value in direction, This indicates the corresponding trajectory point in the spatial coordinate system. Below coordinate, Indicates the ideal paint film thickness. The array index in the simulation model is... Medium data points The thickness of the paint film, The coefficients of the reward function are typically: .

[0078] It should be noted that when satisfy Sometimes, Established, This represents a variable distance parameter.

[0079] S140. Update the state vector for the next moment based on the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed. S150. Based on the state vector for the next moment, use a pre-trained deep reinforcement learning-based coating control strategy model to determine the spraying trajectory corresponding to the spraying action vector for the next moment; repeat this process until the state vector, spraying action vector, and spraying trajectory corresponding to the spraying action vector for each moment in the spraying process are determined.

[0080] In other words, initialize the state vector. ; in state As The input to the network model is used to obtain the network output spraying action vector. Based on spraying motion vector The given spraying speed Spraying distance Trajectory overlap distance The actual discretized spraying trajectory is obtained using the slicing method. Based on spraying trajectory and elliptical double The paint film thickness deposition model updates the paint film thickness distribution of the workpiece to be coated and obtains the state vector. ;make Repeat the above steps until the condition is met. Thus, it is possible to monitor every moment (step). Below the spraying speed Spraying distance Trajectory overlap distance The coating process parameters were determined, and a series of discrete spraying trajectories were obtained. .

[0081] In summary, the intelligent coating trajectory planning method based on deep reinforcement learning in this invention addresses the problems of low solution efficiency, difficulty in obtaining the global optimum, and poor versatility in existing optimization model solving methods. Based on a paint deposition model, the spraying process is abstracted as a Markov decision process. By utilizing the interaction between the spray gun and the simulation environment during the spraying process, and considering both local and global rewards, the optimal execution strategy is learned to achieve the goal of uniform paint film thickness distribution. Unlike traditional parameter optimization processes based on known trajectories, this method effectively obtains the optimal coating trajectory by comprehensively considering the coupling effects of process parameters and coating trajectory during the spraying process. This significantly improves the technical performance of coating thickness control and spray trajectory calculation for complex curved surfaces.

[0082] Corresponding to the above-mentioned intelligent planning method for coating trajectories based on deep reinforcement learning, the present invention also provides an intelligent planning system for coating trajectories based on deep reinforcement learning. Figure 6The functional modules of a coating trajectory intelligent planning system based on deep reinforcement learning according to an embodiment of the present invention are shown.

[0083] like Figure 8 As shown, the intelligent painting trajectory planning system 800 based on deep reinforcement learning provided by this invention can be installed in an electronic device. Depending on the functions implemented, the intelligent painting trajectory planning system 800 based on deep reinforcement learning may include a data acquisition unit 810, a state vector update unit 820, and an execution unit 830. The unit described in this invention can also be called a module, referring to a series of computer program segments that can be executed by the processor of an electronic device and can perform a specific fixed function, stored in the memory of the electronic device.

[0084] In this embodiment, the functions of each module / unit are as follows: The data acquisition unit 810 is used to acquire model data of the workpiece to be coated, and divide the model of the workpiece to be coated into grids according to the preset coating accuracy standard. The average value of the coating thickness of all point clouds in all grids at a certain moment is vectorized to obtain the initial state vector. The state vector update unit 820 is used to input the initial state vector into the pre-trained coating control strategy model based on deep reinforcement learning to obtain the spraying action vector; and determine the spraying trajectory at the next moment based on the spraying action vector; in the simulation environment, the spraying action is executed according to the spraying trajectory at the next moment corresponding to the spraying action vector; the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed is determined using the paint film thickness deposition model; the state vector at the next moment is updated based on the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed; and the spraying trajectory at the next moment corresponding to the spraying action vector is determined using the pre-trained coating control strategy model based on deep reinforcement learning based on the state vector at the next moment. The execution unit 830 is used to execute cyclically until the state vector, spraying action vector, and spraying trajectory corresponding to the spraying action vector at each moment in the spraying process are determined.

[0085] More specific implementations of the intelligent painting trajectory planning system based on deep reinforcement learning provided by this invention can all refer to the above-described embodiments of the intelligent painting trajectory planning method based on deep reinforcement learning, and will not be listed one by one here.

[0086] like Figure 9 As shown, this invention provides an electronic device 9 based on a deep reinforcement learning-based intelligent planning method for coating trajectories.

[0087] The electronic device 9 may include a processor 90, a memory 91 and a bus, and may also include a computer program stored in the memory 91 and capable of running on the processor 90, such as a painting trajectory intelligent planning program 92 based on deep reinforcement learning.

[0088] The memory 91 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 91 can be an internal storage unit of the electronic device 9, such as a portable hard drive. In other embodiments, the memory 91 can be an external storage device of the electronic device 9, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, the memory 91 can include both internal and external storage units of the electronic device 9. The memory 91 can be used not only to store application software and various types of data installed on the electronic device 9, such as the code of a painting trajectory intelligent planning program based on deep reinforcement learning, but also to temporarily store data that has been output or will be output.

[0089] In some embodiments, the processor 90 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 90 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 91 (e.g., a deep reinforcement learning-based intelligent painting trajectory planning program) and calls data stored in the memory 91 to perform various functions and process data in the electronic device 9.

[0090] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 91 and at least one processor 90, etc.

[0091] Figure 9 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 9 The structure shown does not constitute a limitation on the electronic device 9, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0092] For example, although not shown, the electronic device 9 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 90 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 9 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0093] Furthermore, the electronic device 9 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 9 and other electronic devices.

[0094] Optionally, the electronic device 9 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 9 and to display a visual user interface.

[0095] Optionally, the external sensor may include an imaging sensor, a temperature sensor, and / or a humidity sensor. The imaging sensor may be a visual sensor including an RGB sensor, a monochrome sensor, an infrared sensor, a haze sensor, a reflectivity sensor, and / or a diffuser sensor, or a terrain sensor including RGB-D (structured light, time-of-flight, and / or stereo photogrammetry), stereo refraction, profilometry, and / or microscopy. The external sensor may also include a tactile sensor for elastomer imaging (i.e., GelSight); a temperature sensor, including thermocouples and / or IR thermal imaging, may also be used.

[0096] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0097] The deep reinforcement learning-based intelligent planning program 92 for coating trajectories, stored in the memory 91 of the electronic device 9, is a combination of multiple instructions. When run in the processor 90, it can: acquire model data of the workpiece to be coated, and divide the model of the workpiece to be coated into grids according to a preset coating accuracy standard; vectorize the average value of the coating thickness of all point clouds in all grids at a certain moment to obtain an initial state vector; input the initial state vector into a pre-trained deep reinforcement learning-based coating control strategy model to obtain a spraying action vector; and determine the spraying trajectory at the next moment based on the spraying action vector; in simulation... In a real-world environment, the spraying action is executed according to the spraying trajectory corresponding to the next moment of the spraying action vector. Using a paint film thickness deposition model, the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed is determined. Based on the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed, the state vector for the next moment is updated. Based on the state vector for the next moment, a pre-trained coating control strategy model based on deep reinforcement learning is used to determine the spraying trajectory corresponding to the spraying action vector for the next moment. This process is repeated until the state vector, spraying action vector, and spraying trajectory corresponding to the spraying action vector are determined at each moment in the spraying process.

[0098] Specifically, the processor 90's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here. It should be emphasized that, to further ensure the privacy and security of the above-mentioned intelligent painting trajectory planning program based on deep reinforcement learning, the above-mentioned intelligent painting trajectory planning program based on deep reinforcement learning is stored in the node of the blockchain where this server cluster is located.

[0099] Furthermore, if the modules / units integrated in the electronic device 9 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0100] This invention also provides a computer-readable storage medium, which may be non-volatile or volatile. The storage medium stores a computer program, which, when executed by a processor, performs the following: acquiring model data of the workpiece to be coated, and dividing the model of the workpiece to be coated into grids according to a preset coating accuracy standard; vectorizing the average coating thickness of all point clouds within all grids at a certain moment to obtain an initialized state vector; inputting the initialized state vector into a pre-trained coating control strategy model based on deep reinforcement learning to obtain a spraying action vector; and determining the spraying action vector for the next moment. The process involves several steps: First, in a simulation environment, the spraying action is executed according to the spraying trajectory corresponding to the next moment's spraying action vector. Using a paint film thickness deposition model, the influence of each spraying trajectory on the paint film thickness distribution of the workpiece is determined. Based on the influence of each spraying trajectory on the paint film thickness distribution of the workpiece, the state vector for the next moment is updated. Then, based on the state vector for the next moment, a pre-trained deep reinforcement learning-based coating control strategy model is used to determine the spraying trajectory corresponding to the next moment's spraying action vector. This process is repeated until the state vector, spraying action vector, and corresponding spraying trajectory for each moment in the spraying process are determined.

[0101] Specifically, the specific implementation method of the computer program when executed by the processor can be found in the description of the relevant steps in the embodiment of the intelligent planning method for painting trajectory based on deep reinforcement learning, which will not be repeated here.

[0102] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0103] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0105] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0106] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0107] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying platform, a platform product service layer, and an application service layer. Blockchain can store medical data, such as personal health records, kitchen records, and examination reports.

[0108] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A painting trajectory intelligent planning method based on deep reinforcement learning, applied to electronic devices, characterized in that, The method includes: The model data of the workpiece to be coated is obtained, and the model of the workpiece to be coated is divided into grids according to the preset coating accuracy standard. The average value of the coating thickness of all point clouds in all grids at a certain moment is arranged into vectors to obtain the initial state vector. The initial state vector is input into a pre-trained coating control strategy model based on deep reinforcement learning to obtain a spraying action vector; and the spraying trajectory at the next moment is determined based on the spraying action vector; wherein, the coating control strategy model based on deep reinforcement learning includes... dimensional state vector For input, 3D action vector For output Network and The network, and, with 3D action vector For input, numerical values For output Network and Network; wherein, the aforementioned Critic Network and Critic Target Each network comprises two networks; In the simulation environment, the spraying action is executed according to the spraying trajectory corresponding to the next moment of the spraying action vector. The influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed is determined using the paint film thickness deposition model. The paint film thickness deposition model is a free-form surface coating thickness distribution model converted from the planar coating thickness model. The freeform surface coating thickness distribution model, derived from the planar coating thickness model, is achieved through the following formula: in, This indicates the point cloud clusters within the current spraying area. Surface data points The coating thickness deposition rate The coating thickness deposition rate on the reference plane is represented by the following: Representing data points Coordinate description; This indicates the distance (height) from the center point of the spray gun to the theoretical plane. The distance from the center point of the spray gun to the reference plane is the height. Representing data points Tangent plane normal vector The angle with respect to the axis of the spray gun, Indicates the angle between the spray gun axis and the direction of the vertical segment; Let these represent the lengths of the major and minor axes of the spraying ellipse on the theoretical plane. The coating deposition rate coefficient representing the theoretical plane. Indicates the distribution coefficient of the sedimentation model; The state vector for the next moment is updated based on the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed. Based on the state vector at the next moment, the spraying trajectory corresponding to the spraying action vector at the next moment is determined using a pre-trained coating control strategy model based on deep reinforcement learning; this process is repeated until the state vector, spraying action vector, and spraying trajectory corresponding to the spraying action vector at each moment in the spraying process are determined.

2. The intelligent planning method for coating trajectory based on deep reinforcement learning as described in claim 1, characterized in that, The coating control strategy model based on deep reinforcement learning is iteratively trained by setting a maximum number of iterations. The iterative training method includes: The parameters of the deep reinforcement learning-based coating control strategy model are randomly initialized, the network training iteration count is initialized to 1, and the deep reinforcement learning experience replay pool is cleared; wherein, the parameters of the deep reinforcement learning-based coating control strategy model include Network parameters, Network parameters, Network parameters and The network parameters; the The network parameters are obtained by copying the corresponding... The network parameters are obtained. The network parameters are obtained by copying the corresponding... The network parameters are obtained; Update state vector Based on state vector Get The network outputs the spraying action vector. ; Based on spraying action vector Calculate the spray trajectory corresponding to the spraying motion vector. The paint film thickness distribution of the workpiece to be coated is updated using a paint film thickness deposition model to obtain the state vector. ; Based on state vector and reward function Calculate real-time rewards ; and with Whether the condition is met determines whether the spraying process has been terminated; Based on the obtained state vector Spraying motion vector Dynamic state vector and real-time rewards Determine the set ; The set Stored in the deep learning experience replay pool ,make The process continues in a loop until the experience replay pool is reached. The maximum capacity; where t is the residence time of the spray gun; From the experience replay pool In the process Second random sampling, to obtain , Calculate the current policy The following theory value; Using the mean squared error loss function, the gradient backpropagation of the neural network is applied to... Network parameters Update; When the number of iterations is an integer multiple of the delayed update base d, the loss function is used to backpropagate the gradients of the neural network. Network parameters Update and provide the updated version. Network parameters and The network parameters are copied to the corresponding... Network and network; Perform iterative training until the maximum number of iterations is reached; obtain a well-trained coating control strategy model based on deep reinforcement learning.

3. The intelligent planning method for coating trajectory based on deep reinforcement learning as described in claim 2, characterized in that, State vector , Indicates array index as Paint film thickness of all data points in the point cloud cluster The average value; Among them, point cloud clusters Medium data points Paint film thickness Obtain it through the following steps: Calculate the spray gun at the trajectory point Duration of stay ; With trajectory points Construct a local coordinate system with the origin as the origin. Using a paint film thickness deposition model to search for point cloud clusters Data points located in the coating area Calculate data points In time The thickness of the paint film obtained inside ; Calculate the trajectory separately Each trajectory point to data point The resulting paint film deposition thickness Therefore, the final paint film thickness for: .

4. The intelligent planning method for coating trajectory based on deep reinforcement learning as described in claim 2, characterized in that, A method for acquiring model data of a workpiece to be coated and dividing the workpiece into grids according to a preset accuracy standard includes, Obtain model data of the workpiece to be coated; wherein, the model data is point cloud data or Model data; Based on the model data of the workpiece to be coated, the point cloud data of the three-dimensional space of the workpiece to be coated is obtained using a vision sensor, or the CAD model data is converted into point cloud data. The interval size for segmentation is determined according to the accuracy requirements, and the workpiece to be coated is then segmented according to the interval size. The data is rasterized and then back-mapped to point cloud data in three-dimensional space. The point cloud data in the three-dimensional space is segmented into The point cloud clusters are composed of an array for each raster cell, and each array contains the numbers of the point cloud data points contained in the current raster cell.

5. The intelligent planning method for coating trajectory based on deep reinforcement learning as described in claim 2, characterized in that, Through policy-based The state value function obtains the current policy. The following theory Value; the strategy-based State value function This can be achieved through the following formula: in, For state The expected reward that can be obtained in the next moment. This is the discount factor.

6. The intelligent planning method for coating trajectories based on deep reinforcement learning as described in claim 1, characterized in that, Based on state vector and reward function Calculate real-time rewards This can be achieved through the following formula: in, Represents point cloud clusters On the coordinate axes X Minimum value in direction, This indicates the corresponding trajectory point in the spatial coordinate system. Below coordinate, Indicates the ideal paint film thickness. The array index in the simulation model is... data points The thickness of the paint film, This represents the coefficient of the reward function.

7. A coating trajectory intelligent planning system based on deep reinforcement learning, characterized in that, include: The data acquisition unit is used to acquire model data of the workpiece to be coated, and divide the model of the workpiece to be coated into grids according to the preset coating accuracy standard. The average value of the coating thickness of all point clouds in all grids at a certain moment is vectorized to obtain the initial state vector. A state vector update unit is used to input the initialized state vector into a pre-trained deep reinforcement learning-based coating control strategy model to obtain a spraying action vector; and to determine the spraying trajectory at the next moment based on the spraying action vector; wherein, the deep reinforcement learning-based coating control strategy model includes... dimensional state vector For input, 3D action vector For output Network and The network, and, with 3D action vector For input, numerical values For output Network and Network; wherein, the aforementioned Critic Network and Critic Target Each network comprises two networks; In a simulation environment, the spraying action is executed according to the spraying trajectory corresponding to the next moment of the spraying action vector. Using a paint film thickness deposition model, the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed is determined. The paint film thickness deposition model is a freeform surface coating thickness distribution model converted from a planar coating thickness model. The freeform surface coating thickness distribution model converted from the planar coating thickness model is achieved through the following formula: in, This indicates the point cloud clusters within the current spraying area. Surface data points The coating thickness deposition rate The coating thickness deposition rate on the reference plane is represented by the following: Representing data points Coordinate description; This indicates the distance (height) from the center point of the spray gun to the theoretical plane. The distance from the center point of the spray gun to the reference plane is the height. Representing data points Tangent plane normal vector The angle with respect to the axis of the spray gun, Indicates the angle between the spray gun axis and the direction of the vertical segment; Let these represent the lengths of the major and minor axes of the spraying ellipse on the theoretical plane. The coating deposition rate coefficient representing the theoretical plane. Indicates the distribution coefficient of the sedimentation model; The state vector for the next moment is updated based on the influence of each spraying trajectory on the paint film thickness distribution of the workpiece to be sprayed. Based on the state vector at the next moment, the spraying trajectory at the next moment corresponding to the spraying action vector is determined using a pre-trained coating control strategy model based on deep reinforcement learning. The execution unit is used to execute cyclically until the state vector, spraying action vector, and spraying trajectory corresponding to the spraying action vector at each moment in the spraying process are determined.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps in the intelligent painting trajectory planning method based on deep reinforcement learning as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent planning method for painting trajectory based on deep reinforcement learning as described in any one of claims 1 to 6.

Images