Ball screw feeding system servo parameter optimization method and device based on flexible action-evaluation algorithm

Abstract

The application belongs to the technical field of servo control optimization, and discloses a servo parameter optimization method and equipment for a ball screw feeding system based on a flexible action-evaluation algorithm, which comprises the following steps: (1) constructing an agent network of an SAC algorithm based on an Actor-Critic framework, wherein the agent network introduces maximum entropy; (2) associating a simulation model of the ball screw feeding system and the agent network of the SAC algorithm through an environment search interface, training the agent network of the SAC algorithm, and then obtaining optimized servo parameters by using the trained agent network. The SAC algorithm used introduces the concept of maximum entropy, explores the maximum entropy of a random strategy while exploring the maximum cumulative reward value, widens the exploration space of the agent network, improves the randomness of exploration, and is less likely to fall into local optimization compared with heuristic algorithms and other deep reinforcement learning algorithms.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of servo control optimization, and more particularly to a servo parameter optimization method and device for a ball screw feeding system based on a flexible action-evaluation algorithm. BACKGROUND

[0002] As a commonly used driving system of general numerical control machine tools, the ball screw feeding system plays a crucial role in ensuring and improving the machining precision, work efficiency and comprehensive performance of the entire numerical control machine tool. The dynamic performance optimization of the ball screw system is a hot research topic. By designing a servo control strategy and control algorithm, the dynamic performance of the feeding driving system can be estimated and optimized, thereby effectively improving the manufacturing precision. However, the performance of the control algorithm is greatly affected by key servo parameters. Only by selecting appropriate parameter values can the improvement of the dynamic performance of the feeding driving system be ensured. Therefore, it is of great significance to study the motion control parameter optimization technology of the ball screw feeding system.

[0003] According to the different principles of servo parameter optimization methods, they can be generally divided into parameter optimization methods based on control models and intelligent parameter optimization methods. The parameter optimization method based on the control model first needs to determine the system model of the control object. The specific methods include the number frequency characteristic method, the root locus method, the decay curve method, etc. Then, based on the transfer function or state space equation of the control object, the theoretical value of the servo parameter can be calculated according to the control principle. This parameter optimization method is suitable for scenarios where the working environment disturbance is small and the control object is simple. However, manual fine-tuning is usually required after obtaining the theoretical value, so there may be problems of long time consumption and low efficiency in actual application, which cannot meet the needs of high precision and high efficiency in industrial production. With the in-depth study of optimization problems by scholars, various intelligent parameter optimization methods have been proposed. Among them, the heuristic parameter optimization algorithm has been widely used in industrial production. At present, the heuristic algorithm is mainly based on natural algorithm, such as particle swarm optimization algorithm and genetic optimization algorithm. The particle swarm optimization algorithm has few hyperparameters and fast convergence, and is suitable for continuous problems. The genetic optimization algorithm has large computational complexity and is suitable for continuous or discrete problems. The Bayesian optimization algorithm is suitable for low-dimensional optimization problems. The goal of the heuristic parameter optimization algorithm is to find the global optimal solution of the objective function, which can effectively adapt to the control optimization problem of complex systems. However, this type of algorithm has high computational cost, strong dependence on hyperparameters, and is prone to fall into the dilemma of local optimum in the parameter search process.

[0004] Recently, deep learning and reinforcement learning have become the research focus of scholars. The introduction of deep reinforcement learning algorithm into the servo parameter optimization of control system can help improve the intelligence of the control system tuning process. At the same time, the agent constructed by the algorithm can realize the migration of optimization strategy across devices and working conditions, effectively reducing the time of repeated training. The common deep reinforcement learning model, Actor-Critic algorithm, has fast parameter adjustment speed and strong anti-interference ability. Compared with heuristic algorithms, it can adapt to more complex environments and uncertainties, and has strong generalization ability. However, the Actor-Critic algorithm has large calculation amount in the training process, slow network convergence, and the optimization result is easy to fall into local optimum. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a ball screw feeding system servo parameter optimization method and device based on a flexible action-evaluation algorithm, which aims to solve the problem that the existing bionic algorithm easily falls into local optimum.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, a ball screw feeding system servo parameter optimization method based on a flexible action-evaluation algorithm is provided, which comprises the following steps:

[0007] (1) Constructing an agent network of SAC algorithm based on Actor-Critic framework, the agent network introduces maximum entropy, and the formula of the Actor policy network is:

[0008]

[0009] Wherein, R(s t ,a t ) is the reward value, H(π(·∣s t )) is the entropy value, γ represents the discount factor, and α is the temperature coefficient;

[0010] (2) Associating the simulation model of the ball screw feeding system and the agent network of SAC algorithm through the environment search interface, and training the agent network of SAC algorithm, and then obtaining the optimized servo parameters by using the trained agent network.

[0011] Further, the SAC algorithm agent network simultaneously learns two Critic networks and an Actor policy network, and each Critic network also has its corresponding target network. The parameters of the target network are updated using a soft update strategy during the training process.

[0012] Further, the state value function is:

[0013]

[0014] The action value function of the Critic network is:

[0015]

[0016] Where s t 、a t and r t are the system state value, output action and reward value at the current moment, i.e., time t; γ is the discount factor, which is used to control the weight of future rewards; R(s t ,a t ) is state s t Next take action a t The reward value; s0 is the initial state; a0 is the initial action; α is the temperature coefficient.

[0017] Furthermore, the search environment of the SAC algorithm agent network includes an agent-environment interaction function and a search environment initialization function. During agent training, the done value is used to determine whether the current training round is complete, that is, whether the target convergence value has been reached. The default value of done is 0.

[0018] The agent-environment interaction function calculates the current action a based on the objective function of the reward value t The reward value r t , compare the reward value with the target value goal to get the done value, the calculation formula is:

[0019]

[0020] The agent performs action a t After the state is updated, s t+1 , the state update function is:

[0021] s t+1 ~P(s t+1 ∣s t ,a t )

[0022] The search environment initialization function is:

[0023] s t =s0

[0024] Where s0 is the initial state.

[0025] Furthermore, the agent network needs to initialize the environment for each round of search. In each round of search, the agent network is used to calculate the reward value of the current step action and the state of the next step, and to determine whether the current round has reached the target convergence value.

[0026] Further, the training parameters of the agent network include the number of training rounds, the learning rate and the maximum entropy value; the preset servo parameter value is taken as input to train the agent network, the next action is selected according to the current state, the reward value is calculated after each round of search is completed, and the training is stopped until the set number of rounds is completed.

[0027] Further, the update formula of the critic network is:

[0028]

[0029] The update formula of the actor network is:

[0030]

[0031] The update formula of the alpha value is:

[0032]

[0033] In the formula, Q θ (s t ,a t ) is the expected cumulative reward of the critic network under the state s t and the action a t ; D is the data of the replay buffer; r(s t ,a t ) is the reward value of the action a t under the state s t ; is the expected cumulative reward of the actor network under the state s t+1 ; π φ is the current policy sampling; f φ (ε t ;s t ) uses the reparameterization trick, and the action is sampled from the policy Gaussian distribution, that is, a t =f φ (ε t ;s t ), wherein f φ is the policy neural network, the parameter is φ, ε t is the initialization parameter of the network; Q θ (s t ,f φ (ε t ;s t )) is the expected cumulative reward of the target critic network under the state s t and the action f φ (ε t ;s t ); π t is the state s tTake action a t The probability of; The target value of entropy, used to automatically adjust the entropy regular term.

[0034] Further, the target function reward of the reward value is:

[0035] reward = -(tr+overshot+ts+td).

[0036] The application also provides a ball screw feeding system servo parameter optimization system based on a flexible action-evaluation algorithm, which comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to execute the ball screw feeding system servo parameter optimization method based on the flexible action-evaluation algorithm as described above.

[0037] The application also provides a computer readable storage medium, which stores machine executable instructions, and the machine executable instructions make the processor realize the ball screw feeding system servo parameter optimization method based on the flexible action-evaluation algorithm when the machine executable instructions are called and executed by the processor.

[0038] Overall, compared with the prior art, the ball screw feeding system servo parameter optimization method and device based on the flexible action-evaluation algorithm provided by the application mainly have the following beneficial effects:

[0039] 1. The SAC algorithm used in the application introduces the concept of maximum entropy, explores the maximum entropy of the random strategy while exploring the maximum cumulative reward value, widens the exploration space of the agent network, improves the randomness of exploration, and is less likely to fall into local optimization compared with heuristic algorithms and other deep reinforcement learning algorithms.

[0040] 2. The SAC algorithm used in the application introduces a replay buffer and a target network, the experience replay area stores historical samples, randomly extracts a small batch of samples for training and updating, reduces the correlation of the samples, and improves the utilization rate of the samples; the target network and the estimation network have the same structure, and the parameters thereof are periodically copied from the estimation network, so that the training process of the agent is more stable, and the oscillation problem caused by the rapid update of the estimation network parameters is avoided.

[0041] 3. The SAC algorithm used in the application has generalization, the agent trained thereby can be migrated to different devices and different working conditions for servo parameter optimization, reduces the repeated training time of the model, can effectively help improve the motion accuracy and response speed of the ball screw system, and has good industrial application value.

[0042] 4. The SAC algorithm uses two Critic networks to alleviate the overestimation problem of Q values. The structure of the algorithm target network is the same as that of the Critic network, which is used to stabilize the Q value update. The target network maintains the stability of training through soft update (i.e. only part of the parameters is updated after each training).

[0043] 5. Considering that it takes a long time to interact with the real ball screw feeding system and there are many disturbances, a simulation model is selected to improve the interaction efficiency. By analyzing the motion control process of the system, the attribute parameters of the system are obtained, and a simulation model is built to simulate the real ball screw feeding system, thereby shortening the interaction time between the environment and the agent. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a flowchart of a ball screw feeding system servo parameter optimization method based on a flexible action-evaluation algorithm provided by the present application;

[0045] Figure 2 is a reward function value change graph of the agent training process of the SAC algorithm in the present application;

[0046] Figure 3 is an average reward function value change graph of the agent training process of the SAC algorithm in the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0048] The present application provides a ball screw feeding system servo parameter optimization method based on a flexible action-evaluation algorithm, which mainly comprises the following steps:

[0049] Step 1: Construct an agent network of the SAC algorithm based on the Actor-Critic framework, which introduces maximum entropy, and the formula of the Actor policy network is:

[0050]

[0051] wherein R(s t ,a t ) is the reward value, H(π(·∣s t )) is the entropy value, γ represents the discount factor, and α is the temperature coefficient.

[0052] Specifically, the SAC algorithm is a modified algorithm based on the Actor-Critic framework. The SAC algorithm simultaneously learns two Critic networks and an Actor policy network, and each Critic network also has its corresponding target network. The parameters of the target network are updated using a soft update strategy during the training process. At the same time, the SAC algorithm introduces entropy into the reinforcement learning algorithm, thereby avoiding the policy falling into a local optimum and exploring multiple feasible solutions to complete the specified task, improving the anti-interference ability of the algorithm.

[0053] The goal of deep reinforcement learning algorithm is to learn a policy with the highest expected cumulative reward value. In order to randomize the policy, i.e., the probability of each action output is as dispersed as possible, rather than falling into a local optimum, the SAC algorithm introduces the concept of maximum entropy, which requires the entropy of each output action of the policy to be maximum. The formula of the Actor policy network obtained is:

[0054]

[0055] where R(s t ,a t ) is the reward value, H(π(·∣s t )) is the entropy value, γ represents the discount factor, and α is the temperature coefficient.

[0056] The state value function is changed to:

[0057]

[0058] The action value function of the Critic network is changed to:

[0059]

[0060] The SAC algorithm uses two Critic networks to alleviate the overestimation problem of Q value. The structure of the algorithm target network is the same as that of the Critic network, which is used to stabilize the Q value update. The target network maintains the stability of the training through soft update (i.e., only updating part of the parameters after each training).

[0061] Due to different goals of industrial production, the performance requirements of the ball screw feeding system are also different, so it is necessary to set different reward values for different performance requirements. The setting of the objective function can adopt single index, multi-index weighted sum, multi-index weighted sum with constraints, etc.

[0062] The search environment of the SAC algorithm agent network is built, including the agent environment interaction function and the search environment initialization function. The done value is used to judge whether the current training round is completed, i.e., whether the target convergence value is reached, during the agent training process. The done value is 0 by default.

[0063] The agent environment interaction function calculates the current action a according to the target function of the reward value r t t The done value is obtained by comparing the reward value with the target value goal, and the calculation formula is:

[0064]

[0065] The agent performs the action a t The state is updated to s t+1 , and the state update function is:

[0066] s t+1 ~ P(s t+1 | s t , a t )

[0067] The search environment initialization function is:

[0068] s t = s0

[0069] The search of the agent network needs to initialize the environment every round, and in each round of search, the reward value of the current step action and the next step state are calculated by using the agent network, and it is judged whether the current round reaches the target convergence value.

[0070] Step two, through the environment search interface, the simulation model of the ball screw feeding system and the agent network of the SAC algorithm are associated, and the agent network of the SAC algorithm is trained, and then the optimized servo parameters are obtained by using the trained agent network.

[0071] Among them, considering that the interaction time required for training the agent by using the real ball screw feeding system is long, and there are many disturbances, therefore, the simulation model is selected to improve the interaction efficiency, by analyzing the motion control process of the system, the attribute parameters of the system are obtained, and the simulation model is built to simulate the real ball screw feeding system, so as to shorten the interaction time of the environment and the agent.

[0072] The training parameters of the agent network are set, including the number of training rounds, the learning rate and the maximum entropy value, the preset servo parameter value is taken as the input to train the agent network, the next action is selected according to the current state, the reward value is calculated after each round of search is completed, and the training is stopped after the set number of rounds is completed.

[0073] The agent training process of the SAC algorithm is: (1) the actor network selects the action a t according to the current state s t , and delivers the action to the environment; (2) the environment executes the action to obtain the reward r t and the new state s t+1 ​; (3) the Actor network stores the state transition process (current step state s t , action a t , and reward r t , next state s t+1 ) into the replay buffer as a dataset for training the estimation network; (4) N state transition process data are randomly sampled from the replay buffer as a mini-batch of training data sets for the estimation network and the target network; (5) the minimum value of the average values of the two Critic target networks and is calculated; (6) the and of the two Critic networks are updated; (7) the Actor network is updated; and (8) the value of a is updated. The above process is repeated until convergence.

[0074] The update formula of the Critic network is as follows:

[0075]

[0076] The update formula of the Actor network is as follows:

[0077]

[0078] The update formula of the value of a is as follows:

[0079]

[0080] The application will be further described in detail below with specific embodiments.

[0081] The embodiment provides a ball screw servo parameter optimization method based on a flexible action-evaluation algorithm, and specific implementation steps are as follows:

[0082] Step 1, a simulation model of the ball screw system is built.

[0083] A simple ball screw motion speed control system was built by using the GUI of FMPy library in Python, setting reasonable ball screw structure parameters and adopting basic PID control method. The input of the simulation model is the servo control parameters KP, KI and KD of the device, i.e. the parameters to be optimized, and the returned system performance evaluation indexes include the rise time tr, overshoot overshot, adjustment time ts and setting time td of the system speed response curve. According to the relevant knowledge of control theory, it can be known that the rise time tr is in antagonistic relationship with the other three indexes, and when the four outputs are small, the speed following performance of the system is better. At the same time, the simulation model provides a set of self-tuning optimal parameters (also the initial parameters) and the corresponding evaluation indexes for testing the performance of SAC parameter optimization algorithm.

[0084] Step 2, build the SAC algorithm agent network based on the Actor-Critic framework.

[0085] The SAC algorithm is composed of five networks, and the agent network learns two Critic networks and an Actor policy network at the same time. Each Critic network also has its corresponding target network, and the parameters of the target network are updated using the soft update strategy during the training process. The training flow chart of SAC algorithm is shown in Figure 1 .

[0086] Figure 1 s t , a t and r t are the system state value, output action and reward value at the current time, i.e. t time; s t+1 , a t+1 are the system state value and output action at the next step, i.e. (t+1) time; batch_size is the size of the sample extracted from the experience pool, i.e. the replay buffer; is the policy network, i.e. the Actor network; the two Critic networks are and The target networks of the two Critic networks are and

[0087] The formula of the Actor policy network is:

[0088]

[0089] Among them, R(s t ,a t ) is the reward value, H(π(·∣s t )) is the entropy value, γ represents the discount factor, and α is the temperature coefficient.

[0090] The state value function is changed to:

[0091]

[0092] The action value function of the Critic network is changed to:

[0093]

[0094] The SAC algorithm uses two Critic networks to alleviate the overestimation problem of Q values. The target network has the same structure as the Critic network and is used to stabilize Q value updates. The target network is updated by soft update to maintain the stability of training.

[0095] Step 3, set the target function of the SAC algorithm reward value according to the performance requirements of the ball screw feeding system.

[0096] The target function of the reward value is used to evaluate the quality of the action selected by the agent in the current round, so as to train the agent to obtain the optimal policy network. In this embodiment, the target function reward is:

[0097] reward=-(tr+overshot+ts+td)

[0098] Step 4, build the search environment of the SAC algorithm agent network.

[0099] The search environment of the SAC algorithm agent includes the agent state update function, the agent action selection function and the search environment initialization function.

[0100] The done value is used to determine whether the current training round is completed during the agent training process, that is, whether the target convergence value is reached, and the done value is 0 by default.

[0101] The agent environment interaction function calculates the reward value r t of the current action a t according to the target function of the reward value, compares the reward value with the target value goal to obtain the done value, and the calculation formula is:

[0102]

[0103] The state update function after the agent executes the action a t is obtained s t+1 , and in this embodiment, the state update function is:

[0104] s t+1 =a t

[0105] The search environment initialization function is:

[0106] s t =s0

[0107] In the formula, s0 is the initial state, that is, the initial parameter.

[0108] The search of the agent in each round needs to initialize the environment. In each round of search, the next state and reward value are calculated by using the agent network, and it is determined whether the current round reaches the target convergence value. In addition, the range of PID parameter optimization is set to the initial PID parameter ±10%.

[0109] Step 5, training the agent network of the SAC algorithm.

[0110] The simulation model and the agent are associated through the environment search interface, and the training parameters of the agent are set, including the number of training rounds, the learning rate and the maximum entropy value. The preset initial servo parameter value is input, the agent of the SAC algorithm is trained, and the training steps are as follows: (1) the Actor network obtains the current step state s t Select an action a t , and input the ball screw motion speed control system; (2) the environment executes the action to obtain the reward r t and the new state s t+1 , that is, the ball screw motion speed control system generates the corresponding speed response curve according to the input PID parameter of the current step, calculates the reward value of the current step according to the response curve, and takes the PID parameter of the current step as the next step state; (3) the Actor network stores the state transition process (current step state s t , action a t and reward r t , next step state s t+1 ) into the replay buffer as a data set for training the estimation network; (4) 64 state transition process data samples are randomly extracted from the replay buffer as a mini-batch of training data set for the estimation network and the target network; (5) the minimum value of the average value of the two Critic target networks and is calculated; (6) the two Critic networks and are updated; (7) the Actor network is updated ; (8) the value of a is updated. Repeat the above process until convergence.

[0111] The update formula of the Critic network is:

[0112]

[0113] The update formula of the actor network is:

[0114]

[0115] The update formula of the alpha value is:

[0116]

[0117] The reward value and the average reward value of the agent training process are shown in Figure 2 and Figure 3 .

[0118] Step 6, test the trained SAC algorithm agent network, obtain the optimized servo parameters, and verify the effectiveness of the algorithm.

[0119] After training the SAC algorithm agent network, the PID parameter tuning test is performed on the same ball screw motion speed control system simulation model, the performance index changes of the control system before and after optimization are compared and analyzed, and the effectiveness of the algorithm is verified.

[0120] The test results are shown in Table 1, and it can be seen that after the SAC algorithm optimizes the servo parameters of the system, although the rise time tr is slightly increased, the overshoot overshot, the regulation time ts and the setting time td are significantly reduced, and the overall performance of the system is obviously improved.

[0121] Table 1 Comparison of SAC algorithm agent parameter tuning results on the training device

[0122] KP KI KD tr overshot ts td Initial parameters / indicators 32000 500 5e-05 0.068 0.055 0.103 0.104 Optimized parameters / indicators 30484.99 513.84 5.19e-05 0.07 0.021 0.071 0.092 Optimization percentage — — — -2.94％ 61.82％ 31.07％ 11.54％

[0123] In addition, in order to verify that the agent network trained by the SAC algorithm has good migration application ability, the structure parameters of the ball screw in the simulation model are modified, other conditions are kept unchanged, and the trained agent network is directly used for cross-device PID parameter tuning test. The test results are shown in Table 2, and the comparison and analysis show that the servo parameters optimized by the agent have a certain effect on the performance of the new device, and the possibility of cross-device application of the agent is successfully verified.

[0124] Table 2 Comparison of SAC algorithm agent cross-device parameter tuning results

[0125] KP KI KD tr overshot ts td Initial parameters / indicators 32000 500 5e-05 0.081 0.178 0.161 0.162 Optimized parameters / indicators 29502.03 524.57 5.01e-05 0.082 0.14 0.155 0.157 Optimization percentage — — — -1.23％ 21.35％ 3.73％ 3.09％

[0126] The application also provides a ball screw feeding system servo parameter optimization system based on a flexible action-evaluation algorithm, which comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to execute the ball screw feeding system servo parameter optimization method based on the flexible action-evaluation algorithm as described above.

[0127] The application further provides a computer readable storage medium, which stores machine executable instructions, when the machine executable instructions are called and executed by a processor, the machine executable instructions cause the processor to implement the flexible action-evaluation algorithm based ball screw feeding system servo parameter optimization method as described above.

[0128] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A servo parameter optimization method for a ball screw feed system based on a flexible motion-evaluation algorithm, characterized in that: The method comprises the following steps: (1) Construct an agent network based on the SAC algorithm of the Actor-Critic framework. The agent network introduces maximum entropy, and the formula of its Actor strategy network is: in, is the reward value, is the entropy value, γ represents the discount factor, is the temperature coefficient; (2) The simulation model of the ball screw feed system is associated with the intelligent agent network of the SAC algorithm through the environment search interface, and the intelligent agent network of the SAC algorithm is trained. The trained intelligent agent network is then used to obtain the optimized servo parameters. The update formula of the critic network is: The update formula for the Actor network is: α The update formula of the value is: Where, For the Critic network in state Take action Expected cumulative rewards; D The data of the playback buffer; Status Take action The reward value; For Actor network in state Expected cumulative rewards under Sampling for the current strategy; Using the reparameterization technique, actions are sampled from the policy Gaussian distribution, i.e. ,in is the policy neural network, and its parameters are , is the initialization parameter of the network; The target critic network status Take action Expected cumulative rewards; Status Take action probability; is the target value of entropy, which is used to automatically adjust the entropy regularization term.

2. The servo parameter optimization method for a ball screw feed system based on a flexible motion-evaluation algorithm according to claim 1, characterized in that: The SAC algorithm agent network simultaneously learns two Critic networks and one Actor policy network. Each Critic network also has its corresponding target network. During the training process, a soft update strategy is used to update the parameters of the target network.

3. The servo parameter optimization method for a ball screw feed system based on a flexible motion-evaluation algorithm according to claim 2, wherein: The state value function is: The action value function of the Critic network is: Where, 、 and are the system state value, output action and reward value at the current moment, i.e., time t; is the discount factor used to control the weight of future rewards; Status Take action The reward value; is the initial state; is the initial action; is the temperature coefficient.

4. The method for optimizing servo parameters of a ball screw feed system based on a flexible motion-evaluation algorithm according to claim 1, wherein: The search environment of the SAC algorithm agent network includes the agent-environment interaction function and the search environment initialization function.

5. The method for optimizing servo parameters of a ball screw feed system based on a flexible motion-evaluation algorithm according to claim 4, wherein: The agent network needs to initialize the environment for each round of search. In each round of search, the agent network is used to calculate the reward value of the current step action and the state of the next step, and to determine whether the current round has reached the target convergence value.

6. The method for optimizing servo parameters of a ball screw feed system based on a flexible motion-evaluation algorithm according to claim 1, wherein: The training parameters of the agent network include the number of training rounds, learning rate and maximum entropy value; the preset servo parameter values ​​are used as input to train the agent network, and the next action is selected according to the current state. The reward value is calculated after each round of search is completed until the set number of rounds is completed and the training stops.

7. The method for optimizing servo parameters of a ball screw feed system based on a flexible motion-evaluation algorithm according to any one of claims 1 to 6, characterized in that: The objective function of the reward value reward for: Where, 、 、 and td are four evaluation indicators of the system speed response curve, among which is the rise time of the system speed response curve, is the overshoot of the system speed response curve, is the adjustment time of the system speed response curve, It is the setting time of the system speed response curve.

8. A servo parameter optimization system for a ball screw feed system based on a flexible motion-evaluation algorithm, characterized by: The system includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it executes the servo parameter optimization method of the ball screw feed system based on the flexible motion-evaluation algorithm described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions prompt the processor to implement the ball screw feed system servo parameter optimization method based on the flexible motion-evaluation algorithm described in any one of claims 1-7.

Images