A machining parameter optimization method based on multi-agent evolutionary reinforcement learning

Abstract

The present invention belongs to the technical field related to cutting parameter processing optimization, and discloses a processing parameter optimization method based on multi-agent evolutionary reinforcement learning. The method comprises the following steps: (1) collecting processing vibration data under variable parameter cutting conditions and measuring the surface roughness of the workpiece, while extracting the time domain characteristics and frequency domain characteristics of the vibration signal; (2) constructing a processing process knowledge graph and establishing a processing quality estimation model based on an integrated graph attention network; (3) formalizing the processing parameter optimization problem as a Markov decision process, determining the state, action space and reward function, and then constructing a multi-agent reinforcement learning model; (4) determining a fitness function, and combining an evolutionary learning method to train the multi-agent reinforcement learning model in the process of interacting with the processing quality estimation model, and then using the multi-agent reinforcement learning model to achieve processing parameter optimization. The present invention improves prediction performance.

Description

Technical Field

[0001] The present invention belongs to the technical field related to cutting parameter processing optimization, and more specifically, relates to a processing parameter optimization method based on multi-agent evolutionary reinforcement learning. Background Art

[0002] Properly selecting machining parameters plays a crucial role in ensuring product quality, reducing machining costs, and improving production efficiency. Traditionally, this process involves adjusting feed rate, cutting speed, and depth of cut, and conducting extensive trial cutting experiments to explore the parameters that achieve high material removal rates and machining quality. However, this traditional trial cutting method is costly, time-consuming, and difficult to achieve optimal parameters.

[0003] There are two main challenges in machining parameter optimization: Ⅰ) Correlation analysis and modeling between cutting parameters and optimization objectives. Ⅱ) Improvement and innovation of traditional optimization methods. Currently, the accuracy of numerical simulation is poor in the actual optimization stage. Therefore, gray correlation, response surface methodology, and artificial neural networks are often used to analyze the relationship between machining parameters, cutting forces, cutting power, and surface roughness. These studies use experimental data to establish a "parameter variable → objective function" mapping model based on statistical models, empirical models, or artificial intelligence models, often ignoring the dominant role of machining vibration in the correlation between machining parameters and machining quality. In addition, most machining parameter optimization methods use traditional optimization methods such as genetic algorithms, particle swarm optimization, simulated annealing, etc., and assume that the environment is stationary. However, once machining vibration is taken into account, the optimization of machining parameters becomes a high-dimensional optimization problem, and traditional optimization methods often perform poorly in solving such high-dimensional complex optimization problems.

[0004] Therefore, this field urgently needs to study a new, efficient and stable machining parameter optimization method to solve the cutting machining parameter optimization problem in high-precision machining scenarios. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a processing parameter optimization method based on multi-agent evolutionary reinforcement learning, which utilizes knowledge graph technology to integrate the mapping information and structural information between processing parameters, processing vibration characteristics and processing surface roughness data to establish a more accurate processing quality estimation model. It further combines the evolutionary learning method to train the established multi-agent reinforcement learning model during the interaction with the processing quality estimation model, and then explores effective and stable processing parameter optimization strategies to solve the processing parameter optimization problem in high-precision processing scenarios.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for optimizing machining parameters based on multi-agent evolutionary reinforcement learning is provided, the method comprising the following steps:

[0007] (1) Collect machining vibration data under variable parameter cutting conditions and measure the surface roughness of the workpiece, while extracting the time domain and frequency domain characteristics of the vibration signal;

[0008] (2) constructing a machining process knowledge graph and establishing a machining quality estimation model based on an integrated graph attention network; wherein the machining process knowledge graph includes machining parameters, machining vibration time-frequency domain characteristics, and surface roughness;

[0009] (3) The processing parameter optimization problem is formalized as a Markov decision process, and the state, action space and reward function are determined to construct a multi-agent reinforcement learning model;

[0010] (4) Determine the fitness function, and train the multi-agent reinforcement learning model in the process of interacting with the processing quality estimation model in combination with the evolutionary learning method, and then use the multi-agent reinforcement learning model to achieve processing parameter optimization.

[0011] Furthermore, the processing quality estimation model includes two modules, and the corresponding expressions are:

[0012]

[0013] Where, is the first module of the model, whose function includes missing feature completion, and θ1 is the network parameter of the first module; is the second module of the model, which uses the completed features and processing parameters to estimate the roughness value of the machined surface. θ2 is the network parameter of the second module.

[0014] Furthermore, when the processing parameter optimization problem is transformed into a Markov decision problem, the corresponding formula is:

[0015]

[0016] Where, Γ i represents the optimization objective of the i-th production task, subject to (ω1,ω2) i Influence, ω1 and ω2 are the weight coefficients of material removal rate MRR and surface roughness Ra respectively.

[0017] Furthermore, the calculation formulas for MRR and Ra are:

[0018]

[0019]

[0020] Where n, f represent the spindle speed and feed speed respectively, v c Indicates cutting speed, a pIndicates the depth of cut, D is the diameter of the workpiece; That is, the two modules of the processing quality estimation model.

[0021] Furthermore, the state S is:

[0022] s=[tff,n,f]

[0023] The state vector fully reflects the environmental state and consists of two parts: the processing parameters n,f and the corresponding processing features tff, where tff is The module is calculated;

[0024] Action a is:

[0025] a=[n,f]

[0026] The upper and lower selection limits of n and f are as follows;

[0027] Constraint g i for:

[0028] g i ∈[x min ,x max ]

[0029] where g i represents the i-th constraint, including spindle speed and feed rate, x min and x max Represent the upper and lower bounds of the i-th constraint respectively;

[0030] The reward function is:

[0031]

[0032] Where (n′, f′) is the processing parameter combination obtained in the previous round of optimization.

[0033] Furthermore, the multi-agent reinforcement learning model includes a simulation environment framework and a multi-agent framework, wherein the simulation environment framework is based on the established processing quality estimation model based on the integrated graph attention network, and the environment inputs the processing parameter combination (n, f), firstly through The module completes the corresponding processing feature tff; concatenates (n,f) and tff vectors as The input of the module calculates the surface roughness Ra of the processed surface; in the environment output, the spliced ​​processing parameters and feature vectors are used as the environment state vector s t , the machining surface roughness Ra output by the environment is used for agent reward calculation.

[0034] Furthermore, the multi-agent framework includes two agents n and Agent f, the output processing parameters n and f are calculated according to the environmental state s, and the processing parameters (n, f) are passed to the simulation environment framework for execution. The simulation environment framework feeds back the new roughness Ra and the new environmental state s.

[0035] Furthermore, according to the reward function, and And feedback to the agent n and Agent f , which is used to adjust the agent's processing parameter optimization strategy.

[0036] Furthermore, the fitness function is:

[0037]

[0038] Where α1 and α2 represent the weight coefficients of the objective function, and α1 + α2 = 1; ∑(*) represents the test return of each agent; when evolving multiple agents, the population, weight of the objective function, and fitness of the multi-agent combination are first initialized; then the best individual, that is, the agent combination, is selected based on the fitness value.

[0039] Furthermore, agent crossover: each agent combination, as an individual, has a certain probability of obtaining an agent from other individuals to replace the corresponding agent in the parent generation, forming a new agent combination, namely the offspring;

[0040] Agent mutation: During the mutation process, the optimal individual agent is used as a template to partially update the network parameters of the offspring agent;

[0041] After generating the subpopulation, the fitness value is updated through agent testing, the optimal individual is saved and a new population is screened out; then, each individual in the population is trained based on the multi-agent framework, and the fitness value of the trained individual is updated.

[0042] In general, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:

[0043] 1. Based on limited experimental data, the present invention establishes a machining quality estimation model based on an integrated graph attention neural network. First, a variable parameter cutting experiment is designed to collect machining vibration data and measure the machining surface roughness, and extract the time domain and frequency domain characteristics of the vibration signal. Then, a machining process knowledge graph is constructed, and an integrated attention network method is proposed to establish a machining quality estimation model. This method uses the adjacency matrix to fully utilize the correlation between approximate machining parameters, and introduces an integration method to reduce the prediction variance on a limited data set, thereby improving the prediction performance of the model.

[0044] 2. This paper transforms the machining parameter optimization problem into a Markov decision problem, defines the corresponding states, actions, constraints, and reward functions, so that the machining parameter optimization problem can be solved by reinforcement learning methods. A machining quality estimation model based on an integrated graph attention neural network is used to establish a simulated machining environment, and a new multi-agent framework for machining parameter optimization is proposed to optimize the machining parameters through interaction with the simulation environment.

[0045] 3. The present invention combines the idea of ​​evolutionary learning in the multi-agent training stage. This method can solve the problem of multi-agent reward function coupling, improve the stability of the convergence process, and effectively improve the optimization effect of multi-agent reinforcement learning processing parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flowchart of a processing parameter optimization method based on multi-agent evolutionary reinforcement learning provided by the present invention;

[0047] Figure 2 It is a structural diagram of the knowledge graph of the processing process constructed by the present invention;

[0048] Figure 3 It is a schematic diagram of the structure of the integrated graph attention network model constructed by the present invention;

[0049] Figure 4 Schematic diagram of the multi-agent reinforcement learning framework for machining parameter optimization provided by the present invention;

[0050] Figure 5 It is a multi-agent evolutionary flow chart provided by the present invention;

[0051] Figure 6 is a vibration signal of the machining process involved in the embodiment of the present invention, wherein (a) is the original vibration signal, (b) is the vibration signal of the steady-state cutting stage, and (c) is the spectrum obtained by Fourier transforming the vibration signal of the steady-state cutting stage;

[0052] Figure 7 1 is a schematic diagram of surface roughness prediction results of different models in Case 1 and Case 2 involved in an embodiment of the present invention, wherein (a) is the prediction result of the Auto-XGB model in Case 1, (b) is the prediction result of the Auto-XGB model in Case 2, (c) is the prediction result of the MLP model in Case 1, (d) is the prediction result of the MLP model in Case 2, (e) is the prediction result of the GAT model in Case 1, (f) is the prediction result of the GAT model in Case 2, (g) is the prediction result of the Bagging-GAT model in Case 1, (h) is the prediction result of the Bagging-GAT model in Case 2, and (i) is a box plot of roughness prediction errors of the four models in the two cases;

[0053] Figure 8 It is a fitness value curve during the training process of the Multi-TD3 and MERAL-TD3 reinforcement learning processing parameter optimization models involved in the embodiments of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0055] See also Figure 1 and Figure 2 The present invention provides a processing parameter optimization method based on multi-agent evolutionary reinforcement learning, which mainly includes the following steps:

[0056] S1, collects machining vibration data under variable parameter cutting conditions and measures the surface roughness of the workpiece, while extracting the time domain characteristics and frequency domain characteristics of the vibration signal.

[0057] Among them, a variable parameter cutting experiment was designed and implemented, and at the same time, processing vibration data was collected, the surface roughness of the workpiece was measured, and the time domain characteristics and frequency domain characteristics of the vibration signal were extracted.

[0058] S2, constructing a machining process knowledge graph and establishing a machining quality estimation model based on an integrated graph attention network method; wherein, the machining process knowledge graph includes machining parameters, machining vibration time-frequency domain characteristics and surface roughness.

[0059] See also Figure 3 ,The machining process knowledge graph ontology is constructed by the following triples:

[0060] {parm,tff,ra}

[0061] Wherein, parm represents the processing parameters, tff represents the time domain and frequency domain characteristics of the processing vibration signal, and ra represents the roughness value of the processed surface. The corresponding data are all provided by step S1.

[0062] The processing quality estimation model based on the integrated graph attention network consists of two modules, and the corresponding expressions are:

[0063]

[0064] Where, is the first module of the model, whose main function is to complete missing features, and θ1 is the network parameter of the first module; is the second module of the model, which uses the completed features and processing parameters to estimate the roughness value of the machined surface. θ2 is the network parameter of the second module.

[0065] S3, formalizes the processing parameter optimization problem as a Markov decision process, determines the state, action space and reward function, and then constructs a multi-agent reinforcement learning model.

[0066] See also Figure 4 and Figure 5 , when the processing parameter optimization problem is transformed into a Markov decision problem, the corresponding formula is:

[0067]

[0068] Where, Γ i represents the optimization objective of the i-th production task, which is mainly affected by (ω1,ω2) i Influence, ω1, ω2 are the weight coefficients of material removal rate MRR and surface roughness Ra respectively. The calculation formulas of MRR and Ra are as follows:

[0069]

[0070]

[0071] Where n, f represent the spindle speed and feed speed respectively, v c Indicates cutting speed, a p Indicates the depth of cut, D is the diameter of the workpiece; That is, the two modules of the processing quality estimation model.

[0072] In the Markov decision problem for machining parameter optimization, states, actions, constraints, and rewards are also defined. The state S is defined as:

[0073] s=[tff,n,f]

[0074] The state vector fully reflects the environmental state and consists of two parts: the processing parameters n,f and the corresponding processing features tff, where tff is Module calculated.

[0075] Action a definition:

[0076] a=[n,f]

[0077] The upper and lower selection limits of n and f are usually determined by the manual.

[0078] Constraint g i definition:

[0079] g i ∈[x min ,xmax ]

[0080] where g i Represents the i-th constraint, mainly including spindle speed and feed rate, x min and x max Represent the upper and lower bounds of the i-th constraint, respectively, and are provided by the machine tool and tool manuals.

[0081] Reward function definition:

[0082]

[0083] Where (n′, f′) is the processing parameter combination obtained in the previous round of optimization.

[0084] The multi-agent reinforcement learning model includes a simulation environment framework and a multi-agent framework. The simulation environment framework is based on the established processing quality estimation model based on the integrated graph attention network. The environment inputs the processing parameter combination (n, f), firstly through The module completes the corresponding processing feature tff; concatenates (n,f) and tff vectors as The input of the module calculates the surface roughness Ra of the processed surface; in the environment output, the spliced ​​processing parameters and feature vectors are used as the environment state vector s t , the machining surface roughness Ra output by the environment is used for agent reward calculation.

[0085] The multi-agent framework includes two agents n and Agent f , calculate the output processing parameters n and f according to the environment state s, and pass the processing parameters (n, f) to the simulation environment framework for execution. The simulation environment framework feeds back the new roughness Ra and the new environment state s. According to the definition of the reward function, calculate and And feedback to the agent n and Agent f , which is used to adjust the agent's processing parameter optimization strategy.

[0086] S4, determining the fitness function, and combining the evolutionary learning method to train the multi-agent reinforcement learning model in the process of interacting with the processing quality estimation model, and then using the multi-agent reinforcement learning model to achieve processing parameter optimization.

[0087] The fitness function is:

[0088]

[0089] Where α1, α2 represent the weight coefficients of the objective function, and α1+α2=1; ∑(*) represents the test return of each agent.

[0090] See also Figure 5 When evolving a multi-agent system, the population, objective function weights, and fitness of the multi-agent system are initialized first. Then, the best individual, i.e., the agent system, is selected based on the fitness value. Crossover and mutation of the agent population are performed according to the following rules:

[0091] Agent crossover: Each agent combination, as an individual, has a certain probability of obtaining an agent from other individuals to replace the corresponding agent in the parent generation, forming a new agent combination, namely the offspring.

[0092] Agent mutation: The mutation probability of each agent in the offspring is very low. During the mutation process, the agent of the optimal individual is used as a template to partially update the network parameters of the offspring agent.

[0093] After generating a subpopulation, the fitness value is updated through agent testing, the best individual is saved, and a new population is selected. Then, each individual in the population is trained based on the multi-agent framework, and the fitness value of the trained individual is updated.

[0094] The present invention is further described in detail below by taking the external cylindrical machining process of a commutator production line as an example.

[0095] The specific implementation steps of the present invention are as follows:

[0096] (1) Design, implementation and data processing of variable parameter cutting experiments

[0097] The experiment was carried out on the external cylindrical turning machine tool of the commutator production line. The vibration acceleration signal of the tool holder during the cutting process was collected using Dewesoft at a sampling frequency of 10kHz. The roughness value of the machined surface was measured using a roughness meter, and the experimental data was stored in a laptop.

[0098] Table 1 Surface roughness measurement values ​​of workpieces

[0099]

[0100] On the experimental platform, a series of variable parameter cutting experiments were carried out at different spindle speeds (3500, 4000, 4500, 5000, 5500 rpm) and different feed rates (6, 7, 8, 9, 10, 11, 12 mm / s). The surface roughness measurement results of the machined workpiece are shown in Table 1. The vibration signal of the machining process is shown in Figure 6 As shown in (a), the signal in the dotted box is intercepted as follows Figure 6 As shown in (b), it is the vibration signal of the steady-state cutting stage, from which the 11-dimensional time domain features are extracted. The spectrum of the vibration signal of the steady-state cutting stage is obtained by Fourier transform, as shown in Figure 6As shown in (c) in the figure, 14-dimensional frequency domain features are extracted from it. Then, the time domain features and frequency features are spliced ​​together to obtain vibration feature data.

[0101] (2) Establish a processing quality estimation model and compare and verify

[0102] To investigate the impact of vibration information on the prediction accuracy of machining quality estimation models, a set of comparative cases was designed. In Case 1, machining vibration information was not considered, and only machining parameters were used to predict machined surface roughness. In Case 2, a prediction model for machining vibration characteristics was first established to obtain a pseudo-feature space. Then, machining parameters and pseudo-vibration characteristics were used as input to predict machined surface roughness.

[0103] Using limited experimental data, firstly, a pseudo-feature space generation model is established with machining parameters as model input and machining vibration features as labels to generate machining vibration pseudo-features without measured machining parameters.

[0104] Table 2 Comparison of the effects of the four models in two cases

[0105]

[0106] The machining quality estimation model in Case 2 is established based on the pseudo-vibration feature space obtained by the pseudo-feature prediction model. In Case 1, the machining quality estimation model was established using the same experimental data except for the vibration signal. Figure 7 As shown in Figure 1, the prediction results of different models in Case 1 and Case 2 are represented by dotted lines, and the experimental results are marked with asterisks. The height of the histogram represents the absolute error between the model prediction results and the experimental results. In order to further analyze the prediction errors of each model in the two cases, the absolute error box plot is shown in Figure 1. Figure 7 As shown in (i) of Figure 2, the results show that the proposed Bagging-GAT model has a smaller and more concentrated prediction error in Case 2. In addition, Table 2 shows the performance comparison of the four models in the two cases, using three different evaluation metrics: mean absolute error (MAE), maximum error (MME), and root mean square error (RMSE) to compare the performance of the models. The proposed Bagging-GAT method has the smallest MAE and RMSE for predicting the machined surface roughness in both cases. In Case 2, the Bagging-GAT model achieved the best prediction results among the four models and two cases. The results show that the introduction of vibration information can greatly improve the roughness prediction performance of the Bagging-GAT model. In addition, compared with the AutoXGB and MLP models, the GAT and Bagging-GAT models have better prediction performance. The graph-based method better utilizes the structural relationship between the data and improves the modeling accuracy. In addition, the Bagging method can effectively reduce the variance of the graph-based model on small datasets.

[0107] (3) Establish a multi-agent evolutionary reinforcement learning model and conduct comparative verification

[0108] A simulated machining environment was established using a well-trained vibration signature generation model and machining quality estimation model. Before interactive learning with the environment, a suitable reinforcement learning agent needed to be designed for the multi-agent evolutionary reinforcement learning model. The double-delayed deep deterministic policy gradient algorithm (TD3) is one of the most advanced deep reinforcement learning methods, achieving good performance in continuous control and optimization tasks. The machining parameter optimization task to be solved is a typical continuous optimization task with a continuous action space and a continuous state space. Therefore, TD3 was used as the agent for the multi-agent evolutionary reinforcement learning model.

[0109] Table 3 Fitness values ​​of the two model optimization results under different weight coefficients

[0110]

[0111] To verify the effectiveness of evolutionary thinking on multi-agent reinforcement learning, we compared a multi-agent reinforcement learning model and a multi-agent evolutionary reinforcement learning model, referred to as Multi-TD3 and MERAL-TD3, respectively, using TD3 as the agent. On a computer with the same configuration, Multi-TD3 and MERAL-TD3 interacted with the same simulated processing environment. The fitness curves tested during training are shown in Figure 2. Figure 8 The horizontal axis represents training time, and the vertical axis represents fitness. Triangles represent test points for MERAL-TD3, and circles represent test points for Multi-TD3. Because MERAL-TD3 performs testing after one round of population iteration, the testing interval is relatively long. Furthermore, the weight coefficients of the fitness function are variable (0-1500s: α1 = 0.5, α2 = 0.5; 1500-3000s: α1 = 0.8, α2 = 0.2; 3000-4500s: α1 = 0.2, α2 = 0.8). The results in the figure show that MERAL-TD3 achieves optimal optimization after three iterations, while Multi-TD3 exhibits unstable convergence at the beginning of training. The evolutionary method selects the best individuals for testing, improving the stability of the training process. Table 3 provides more detailed optimization results and fitness values ​​for the two methods under different weight coefficients. MERAL-TD3 achieves higher fitness values ​​than Multi-TD3. A drawback of Multi-TD3 is that it is difficult to achieve an overall optimal result when the reward functions of different agents are coupled. However, the proposed MEARL-TD3 method can better handle the coupling of reward functions and escape local optima by selecting the best individual through a global fitness function.

[0112] The present invention takes reinforcement learning as its core. First, a variable parameter cutting experiment is designed to collect machining vibration data and measure the machining surface roughness, extracting the time domain and frequency domain features of the vibration signal. Second, a machining process knowledge graph is constructed, and an integrated graph attention network method is proposed to establish a machining quality estimation model. Then, the machining parameter optimization problem is formalized as a Markov decision process, defining the state, action space and reward function, and establishing a multi-agent reinforcement learning model. Finally, combined with the evolutionary learning method, a fitness function is defined. In the process of interacting with the machining quality model, the constructed multi-agent reinforcement learning model is trained to explore machining parameter optimization strategies. In this way, the present invention utilizes knowledge graph technology to integrate the mapping information and structural information between machining parameters, machining vibration characteristics and machining surface roughness data to establish a more accurate machining quality estimation model. Further combined with the evolutionary learning method, the established multi-agent reinforcement learning model is trained in the process of interacting with the machining quality estimation model, which can explore effective and stable machining parameter optimization strategies and solve the machining parameter optimization problem.

[0113] In summary, based on the method proposed in this invention, the surface roughness of the external cylindrical machining tool of the rectifier production line is modeled. The machining parameter optimization problem is then formalized as a Markov decision process, with corresponding state, action, reward, and constraint functions defined. A roughness estimation model is established based on the Bagging-GAT method. As a simulation environment, a multi-agent reinforcement learning framework is constructed, and evolutionary learning is combined to solve the Markov decision problem of machining parameter optimization. This method establishes a machining quality estimation model based on an integrated graph attention neural network based on limited experimental data. The adjacency matrix is ​​used to fully utilize the correlation between approximate machining parameters, and an integration method is introduced to reduce the prediction variance on the limited data set, thereby improving the model's prediction performance. The average prediction absolute error is only 0.077. In combination with the idea of ​​evolutionary learning, this method can solve the problem of multi-agent reward function coupling, improve the stability of the convergence process, and effectively improve the optimization effect of multi-agent reinforcement learning machining parameters.

[0114] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A processing parameter optimization method based on multi-agent evolutionary reinforcement learning, characterized in that: The method comprises the following steps: (1) Collect machining vibration data under variable parameter cutting conditions and measure the surface roughness of the workpiece, while extracting the time domain and frequency domain characteristics of the vibration signal; (2) constructing a machining process knowledge graph and establishing a machining quality estimation model based on an integrated graph attention network; wherein the machining process knowledge graph includes machining parameters, machining vibration time-frequency domain characteristics, and surface roughness; (3) The processing parameter optimization problem is formalized as a Markov decision process, and the state, action space and reward function are determined to construct a multi-agent reinforcement learning model; (4) determining a fitness function, and training the multi-agent reinforcement learning model in the process of interacting with the processing quality estimation model in combination with an evolutionary learning method, and then using the multi-agent reinforcement learning model to optimize the processing parameters; The machining quality estimation model consists of two modules, and the corresponding expressions are: Where, is the first module of the model, whose function includes missing feature completion, and θ1 is the network parameter of the first module; is the second module of the model, which uses the completed features and processing parameters to estimate the roughness value of the machined surface, and θ2 is the network parameter of the second module; When the machining parameter optimization problem is transformed into a Markov decision problem, the corresponding formula is: Where, Γ i represents the optimization objective of the i-th production task, subject to (ω1,ω2) i Influence, ω1 and ω2 are the weight coefficients of material removal rate MRR and surface roughness Ra respectively.

2. The processing parameter optimization method based on multi-agent evolutionary reinforcement learning according to claim 1, characterized in that: The calculation formulas for MRR and Ra are: Where n, f represent the spindle speed and feed speed respectively, v c Indicates cutting speed, a p Indicates the depth of cut, D is the diameter of the workpiece; That is, the two modules of the processing quality estimation model.

3. The processing parameter optimization method based on multi-agent evolutionary reinforcement learning according to claim 2, characterized in that: The state S is: s=[tff,n,f] The state vector fully reflects the environmental state and consists of two parts: the processing parameters n,f and the corresponding processing features tff, where tff is The module is calculated; Action a is: a=[n,f] The upper and lower selection limits of n and f are as follows; Constraint g i for: g i ∈[x min ,x max ] where g i represents the i-th constraint, including spindle speed and feed rate, x min and x max Represent the upper and lower bounds of the i-th constraint respectively; The reward function is: Where (n ′ ,f ′ ) is the processing parameter combination obtained in the previous round of optimization.

4. The processing parameter optimization method based on multi-agent evolutionary reinforcement learning according to claim 1, characterized in that: The multi-agent reinforcement learning model includes a simulation environment framework and a multi-agent framework, wherein the simulation environment framework is based on the established processing quality estimation model based on the integrated graph attention network, and the environment inputs the processing parameter combination (n, f), firstly through The module completes the corresponding processing feature tff; concatenates (n,f) and tff vectors as The input of the module calculates the surface roughness Ra of the processed surface; in the environment output, the spliced ​​processing parameters and feature vectors are used as the environment state vector s t , the machining surface roughness Ra output by the environment is used for agent reward calculation.

5. The processing parameter optimization method based on multi-agent evolutionary reinforcement learning according to claim 4 is characterized in that: The multi-agent framework includes two agents n and Agent f , the output processing parameters n and f are calculated according to the environmental state s, and the processing parameters (n, f) are passed to the simulation environment framework for execution. The simulation environment framework feeds back the new roughness Ra and the new environmental state s.

6. The processing parameter optimization method based on multi-agent evolutionary reinforcement learning according to claim 5, characterized in that: Calculate according to the reward function and And feedback to the agent n and Agent f , which is used to adjust the agent's processing parameter optimization strategy.

7. The processing parameter optimization method based on multi-agent evolutionary reinforcement learning according to any one of claims 1 to 6, characterized in that: The fitness function is: Where α1 and α2 represent the weight coefficients of the objective function, and α1 + α2 = 1; ∑(*) represents the test return of each agent; when evolving multiple agents, the population, weight of the objective function, and fitness of the multi-agent combination are first initialized; then the best individual, that is, the agent combination, is selected based on the fitness value.

8. The processing parameter optimization method based on multi-agent evolutionary reinforcement learning according to claim 7, characterized in that: Agent crossover: Each agent combination, as an individual, has a certain probability of obtaining an agent from other individuals to replace the corresponding agent in the parent generation, forming a new agent combination, that is, the offspring; Agent mutation: During the mutation process, the optimal individual agent is used as a template to partially update the network parameters of the offspring agent; After generating the subpopulation, the fitness value is updated through agent testing, the optimal individual is saved and a new population is screened out; then, each individual in the population is trained based on the multi-agent framework, and the fitness value of the trained individual is updated.

Images