Cutting parameter dynamic autonomous decision-making method and system based on deep reinforcement learning

Abstract

The invention discloses a cutting parameter dynamic autonomous decision-making method and system based on deep reinforcement learning. The method comprises the following steps: establishing a multi-target cutting parameter dynamic optimization model under a time-varying workpiece deformation constraint; according to the principle that the rigidity of the workpiece changes along with the residual cutting thickness of each layer, a time-varying workpiece deformation constraint threshold value is designed; establishing a constraint function of elastic deformation of the workpiece caused by the time-varying cutting force according to a workpiece material strain change rule, and establishing a multi-target cutting parameter dynamic optimization model of the aviation complex part by taking the machining efficiency as an optimization target; the function of formulating optimal cutting parameters for each pass of machining under the current deformation constraint is achieved; the dynamic characteristics of the optimization model in the decision making process are considered, the optimization model is reconstructed into an MDP model, a model solving architecture based on a soft actor-commentator algorithm is designed, and autonomous decision making of five-axis side milling multi-target cutting parameters of the aviation complex part is achieved.

Description

Background Art

[0002] Considering the weak rigidity of complex aviation parts, the surface of the part is usually divided into multiple layers along the tool axis in the semi-finishing and finishing processes, and the tool is gradually deepened layer by layer to weaken the cutting load generated during processing and avoid deformation of the workpiece during side milling. As the processing deepens, the residual cutting thickness after each cutting pass decreases, and the material strain and cutting load of the workpiece will continue to change. Since the cutting parameters directly affect the cutting load, if the cutting parameters are not adjusted in time, it is easy to cause dynamic elastic deformation of the workpiece. However, the traditional cutting parameter decision method usually relies on experience, and the value is too conservative, which cannot adapt to the dynamic strain of the workpiece material during processing. It is not only difficult to ensure the processing quality, but also easy to cause low processing efficiency. In addition, in order to adapt to the decreasing weak rigidity of thin walls and the complex curvature change characteristics, the process parameters (cutting parameters, tool path) of each pass are also different. Although multi-pass machining reduces the cutting energy consumption within a single step, during the entire machining process, carbon emissions will evolve dynamically and accumulate layer by layer with the continuous adjustment of process parameters, showing dynamic and hierarchical characteristics. This not only increases carbon emissions overall, but also causes the machining process to be seriously time-consuming, making it impossible to balance machining carbon emissions, machining efficiency and machining quality. It should be noted that the multi-objective cutting parameter optimization problem of five-axis side milling is a complex and dynamic decision-making problem. Not only is the tool-workpiece contact area large and the cutting load high, but the material properties also change layer by layer with the selection of cutting parameters (especially cutting depth), resulting in continuous fluctuations in the cutting load, which can easily cause elastic dynamic deformation of the workpiece, thereby affecting the processing quality. Therefore, in the multi-objective cutting parameter optimization of five-axis side milling, the time-varying deformation of the workpiece must be taken into consideration, the law of the change of workpiece deformation with cutting parameters must be clarified, and the multi-objective cutting parameter optimization mechanism under the constraint of the time-varying deformation of the workpiece must be explored. However, the introduction of this dynamic constraint indicator in the cutting parameter optimization problem will make traditional models and methods inapplicable. In this regard, it is necessary to study new theories, models and methods to explore the multi-objective cutting parameter optimization mechanism of the five-axis side milling process under the constraint of the time-varying deformation of the workpiece. Summary of the invention

[0003] The technical problem to be solved by the present invention is to address the deficiencies in the above-mentioned prior art and provide a method and system for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning, which is used to solve the technical problem that the material strain and cutting load of the workpiece will continue to change during the processing, resulting in dynamic elastic deformation of the workpiece.

[0004] The present invention adopts the following technical solutions: The dynamic autonomous decision-making method of cutting parameters based on deep reinforcement learning includes the following steps: Determine optimization variables, establish objective functions and design constraints; Construct a multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on objective functions and constraints; A multi-pass cutting parameter optimization solution framework is established, and the multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts is solved based on deep reinforcement learning to achieve autonomous decision-making on low-carbon cutting parameters.

[0005] Preferably, processing efficiency and processing carbon emissions are taken into consideration. To optimize the target, the objective function is as follows:

[0006] in, is the material removal rate.

[0007] Preferably, the optimization variables are determined to be: In the five-axis side milling of complex aviation parts, It is i The cutting depth of each pass, is the feed rate, is the spindle speed, It is the key variable to be optimized in cutting parameters. and is considered as the decision variable in each cut .

[0008] Preferably, the constraints are specifically: Constraint inequality for the theoretical maximum cutting force:

[0009] Correlation between the current actual cutting force and deformation:

[0010] in, is the safety factor, The cutting force limit is determined by factors such as the workpiece or tool material properties, fixing method, and machine tool capabilities during processing; is the theoretical maximum cutting force threshold, It is the lateral load per unit area on the workpiece surface in the current pass; is the surface area of ​​the workpiece; D is the bending stiffness, E is the elastic modulus; μ is Poisson's ratio; The sum of the cutting depths of each pass must be equal to the thickness to be cut:

[0011] in, I is the number of multi-pass cutting, For thei The cutting depth of each pass, is the thickness to be cut; Key variables meet machine tool and cutting tool specifications:

[0012]

[0013]

[0014]

[0015] in, is the cutting power per pass, The spindle efficiency is is the rated power of the spindle motor / W, are the upper and lower limits of the cutting parameters selected for each pass, For the i The cutting parameter combination selected for each pass, Respectively i The spindle speed, feed rate, cutting depth of each pass, For the i Processing time in one pass / min, is the preset tool life / min, is the maximum surface roughness constraint / μm, For the i Surface roughness of each pass, , It is the tool rake angle / ° and clearance angle / °.

[0016] Preferably, the multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts is as follows:

[0017] in, For the i Carbon emissions from each processing pass, For the i Pass material removal rate, For the i The total cutting force generated by the contact between the tool and the workpiece during the pass, is the current maximum cutting force, are the upper and lower limits of the cutting parameters selected for each pass, For the i The cutting parameter combination selected for each pass, is the cutting power per pass, The spindle efficiency is is the rated power of the spindle motor / W, For thei Processing time in one pass / min, is the preset tool life / min, is the maximum surface roughness constraint / μm, For the i Surface roughness of each pass.

[0018] Preferably, the multi-pass cutting parameter optimization solution framework includes: Based on the agent model's environmental modeling, an agent model for functional simulation of the machining environment is constructed; The SAC method is applied as the intelligent agent of DRL to solve the dynamic cutting parameter optimization problem for strategy improvement and evaluation, and a replay buffer is set to store experience.

[0019] Preferably, the proxy model environment of digital-analog linkage includes: Process performance data: i The 、MRR i 、 , which is used to reflect the specific impact of the selected cutting parameters on carbon emissions, material removal rate, processing quality, and the current maximum cutting force threshold , Processing power , Processing time , as well as the relevant parameters of the machine tool and the tool, to determine whether the selected cutting parameters meet the current constraints; WIP physical information: current stiffness of the part , which varies with the residual thickness Decrease, reflecting the dynamic strain of the material of the parts during the processing, thus supporting Calculation of Characterize the current surface quality status of the part.

[0020] Preferably, solving the dynamic cutting parameter optimization problem includes: Given the current state and behavior, the state at the next moment is independent of all previous states, and the cutting parameter optimization decision problem is constructed as an MDP model. S , A , P , R > The SAC method is used as the intelligent agent of DRL to solve the MDP problem. In the network structure of the SAC method, the input layer of the executor network has 4 neurons, corresponding to 4 element space, and contains multiple hidden layers; it has two output layers, each with 2 neurons, which respectively produce the mean and logarithmic standard deviation of the action; the input layer of the critic network has 7 neurons, which is and The output layer has only one neuron, which reflects the single action value of a specific state-action pair and uses linear output.

[0021] Preferably, the state space : The thickness to be cut after each cutting pass is taken as the key state variable:

[0022] in, Indicates the last cutting pass in the machining process and the number of cutting passes that have been processed , cumulative carbon emissions And the cumulative processing time Determines the environmental status of the current pass, therefore, ; Action space A: is a three-dimensional vector , used to characterize Processing parameters during cutting; Reward Function :Includes optimization objectives and constraint functions; introduces penalty function method, adds penalty items that violate constraints in reward function, introduces The function normalizes it, and the formula for the weighted combination of the three is expressed as:

[0023] in, is the normalization term based on the Sigmoid function, , is the constant coefficient for the trade-off between processing carbon emissions and processing efficiency, is the penalty coefficient, is the penalty function, For Carbon emissions from cutting at all times, For Cutting efficiency at all times.

[0024] In a second aspect, an embodiment of the present invention provides a cutting parameter dynamic autonomous decision-making system based on deep reinforcement learning, comprising: Parameter module, determines the optimization variables, establishes the objective function and designs the constraints; Building module, constructing a multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on objective functions and constraints; The optimization module establishes a multi-pass cutting parameter optimization solution framework, solves the multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on deep reinforcement learning, and realizes autonomous decision-making of low-carbon cutting parameters.

[0025] In a third aspect, a computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning when executing the computer program.

[0026] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning.

[0027] In a fifth aspect, a chip comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning when executing the computer program.

[0028] In a sixth aspect, an embodiment of the present invention provides an electronic device, comprising a computer program, which, when executed by the electronic device, implements the steps of the above-mentioned method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning.

[0029] Compared with the prior art, the present invention has at least the following beneficial effects: A dynamic autonomous decision-making method for cutting parameters based on deep reinforcement learning is proposed. In the process of machining complex aviation parts, the residual cutting thickness after each cutting is reduced, and the material strain and cutting load between the tool and the workpiece will also change continuously, which may cause dynamic elastic deformation of the workpiece. Based on the idea of ​​dynamic optimization of deep reinforcement learning and the principle of layer-by-layer decrease of workpiece stiffness, a dynamic cutting force threshold model is designed, and a dynamic optimization model of cutting parameters under the constraint of time-varying workpiece deformation is established.

[0030] Furthermore, five-axis side milling often uses a multi-pass processing method that goes deeper layer by layer in the semi-finishing and finishing stages. In order to adapt to the decreasing weak stiffness of thin walls and the complex curvature changes, the cutting parameters of each pass are also different. Although multi-pass processing reduces the cutting energy consumption within a single step, the entire processing process not only increases carbon emissions overall, but also causes the processing process to be seriously time-consuming, making it impossible to balance processing carbon emissions, processing efficiency and processing quality. Therefore, optimization with processing efficiency and processing carbon emissions as the goal is of great significance to solving the above problems. Furthermore, in the five-axis side milling of complex aviation parts, assuming It is i The cutting depth of each pass, It is i The thickness to be cut in each pass. As the cutting force increases, the cutting force also increases, thereby increasing the elastic deformation of the workpiece. If the setting is too large, not only will it produce larger processing carbon emissions, but it will also cause the normal stress on the workpiece to exceed the range of elastic deformation, making the deformation irreversible. Smaller settings will remove less material per cut, increasing total cuts, reducing MRR, and making To this end, the invention will is considered as a key variable to be optimized in cutting parameters. and It also affects the deformation of the workpiece caused by cutting force, MRR and Therefore, both are also considered as variables to be optimized in each cutting.

[0031] Furthermore, workpiece deformation should be considered as a key process constraint in process planning to ensure the machining quality of complex aviation parts. Secondly, the continuity constraint of multi-pass machining should be considered to ensure the feasibility of the cutting parameter strategy. Thirdly, it is necessary to consider the basic machining constraints so that the cutting parameters can meet the machine tool performance and surface quality requirements.

[0032] Furthermore, the multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts proposed in the present invention aims to improve processing efficiency, reduce carbon emissions and ensure processing quality by dynamically optimizing cutting parameters. The model takes into account the layer-by-layer decrease in workpiece stiffness and the change in material removal rate during the cutting process, and establishes a time-varying workpiece deformation constraint threshold, thereby effectively controlling the elastic deformation of the workpiece caused by changes in cutting force. Through a multi-objective optimization method, the model comprehensively considers multiple objectives such as processing efficiency, carbon emissions and processing quality to ensure that both efficiency and environmental protection are improved while meeting quality requirements. In addition, the use of deep reinforcement learning (DRL) technology enables the model to adjust cutting parameters in real time during continuous learning and feedback to adapt to the dynamically changing processing environment, further improving the level of intelligent decision-making.

[0033] Furthermore, an autonomous decision-making architecture for cutting parameters based on SAC is designed to explore the potential of autonomous decision-making of cutting parameters under the current workpiece deformation constraints.

[0034] Furthermore, the present invention constructs a digital-analog agent model environment to perceive the environmental state of the multi-pass machining process, thereby helping the SAC agent to interact with it and learn the optimal cutting parameter strategy under the current machining environment. , MRR, and Therefore, the output response of the proxy model includes these three. , it is necessary to predict the cutting power At the same time, in order to meet the current workpiece deformation constraints, The input variables of the proxy model are basically the same as the optimization variables, that is, , and In addition, the cutting width It will also affect The size of. Likewise Therefore, in order to ensure The prediction accuracy of and as additional predictor variables.

[0035] Furthermore, the present invention uses the SAC algorithm as the intelligent agent of DRL to solve the MDP problem. SAC is an actor-critic algorithm based on the maximum entropy reinforcement learning framework, which has strong exploratory and robustness. The SAC algorithm can effectively handle the uncertainty and changes in the multi-pass cutting process, and has good adaptability to the problem of continuous action space, so that the intelligent agent can effectively guide the intelligent agent to make autonomous optimal decisions in complex cutting parameter optimization tasks.

[0036] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0037] In summary, the multi-objective cutting parameter dynamic autonomous decision-making method based on deep reinforcement learning in the present invention establishes a constraint function of the elastic deformation of the workpiece caused by the time-varying cutting force according to the law of the workpiece material strain change, and takes improving the machining process and machining efficiency as the optimization goal to construct a multi-objective cutting parameter dynamic decision-making model under the constraint of time-varying workpiece deformation; further, a model solving architecture based on the Soft Actor Critic (SAC) algorithm is designed to realize the autonomous decision-making of multi-objective cutting parameters in five-axis side milling of complex aviation parts.

[0038] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is the overall framework diagram of the present invention; Figure 2 A diagram of the process of constructing the agent model environment for digital-analog linkage of the present invention; Figure 3 A reinforcement learning framework diagram for process decision making of the present invention; Figure 4 The network structure diagram of the SAC method of the present invention; Figure 5 This is the CAD model diagram of the aviation thin-walled part that is the research object of the present invention; Figure 6A schematic diagram of a computer device provided by an embodiment of the present invention; Figure 7 The block diagram is a chip provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0042] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0043] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0044] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0045] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0046] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0047] The present invention provides a method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning. According to the principle of layer-by-layer decrease of workpiece stiffness, a dynamic cutting force threshold model is designed. Based on this model, a multi-objective dynamic optimization model of cutting parameters under time-varying workpiece deformation constraints is established. A dynamic autonomous decision-making framework for cutting parameters based on the Soft Actor Critic (SAC) algorithm is constructed. Under the current workpiece deformation constraints, the autonomous decision-making potential of multi-objective cutting parameters is explored. The optimal decision of multi-objective cutting parameters is achieved, and the processing efficiency is improved and the carbon emission of processing is reduced while ensuring the processing quality.

[0048] See also Figure 1 The present invention provides a cutting parameter dynamic decision method based on deep reinforcement learning, which respectively describes the multi-objective cutting parameter optimization decision problem and solves the optimization model based on deep reinforcement learning. The description of the multi-objective cutting parameter optimization decision-making problem includes the establishment of the optimization objective function, the design of constraint conditions, and the establishment of the optimization decision-making model; the construction of the multi-objective cutting parameter optimization model based on deep reinforcement learning includes the construction of a digital-analog linkage agent model environment and the multi-objective cutting parameter optimization solution based on deep reinforcement learning.

[0049] The construction process of the dynamic decision framework for cutting parameters is as follows: Figure 1 As shown in the figure, the construction process of the agent model environment for digital-analog linkage is as follows Figure 2 As shown, the specific steps are as follows: The present invention provides a cutting parameter dynamic autonomous decision-making method based on deep reinforcement learning, comprising the following steps: S1. Determine optimization variables, establish objective function, and design constraints; S101, Assumptions in five-axis side milling of complex aviation parts It is i The cutting depth of each pass, is the feed rate, is the spindle speed, It is the key variable to be optimized in cutting parameters. and It is also regarded as the decision variable in each cutting, that is, ; S102, based on machining efficiency (measured by introducing material removal rate MRR) and machining carbon emissions To optimize the goal; S103. Based on step S102, The lower the value, the higher the MRR value, indicating that the performance of the multi-objective cutting parameter scheme is better. The objective function is thus established, namely: ; S104. In process planning, workpiece deformation is taken as a key process constraint to ensure the machining quality of complex aviation parts; secondly, the continuity constraint of multi-pass machining is considered to ensure the feasibility of the cutting parameter strategy; thirdly, the basic machining constraints are considered so that the cutting parameters can meet the machine tool performance and surface quality requirements; The specific constraints are: The constraint inequality that defines the theoretical maximum cutting force is:

[0050] The differential equation of the elastic surface of the thin plate is defined as follows: By solving , the relationship between the current actual cutting force and deformation can be established

[0051] in, is the safety factor, which is generally set to 0.8; The cutting force limit / N is determined by factors such as the workpiece or tool material properties, fixing method, and machine tool capacity during processing; is the theoretical maximum cutting force threshold / N, It is the lateral load per unit area of ​​the workpiece surface in the current pass / N·mm²; is the workpiece surface area / mm²; D is the bending stiffness / N·m, which varies with the thickness of the sheet to be cut; E is the elastic modulus / Pa; μ is Poisson's ratio.

[0052] To ensure the continuity of cutting, the sum of the cutting depths of each pass must be equal to the thickness to be cut:

[0053] in, I The number of multi-pass cutting.

[0054] To ensure the reliability of the machining process, key variables must meet the specifications of the machine tool and cutting tool:

[0055]

[0056]

[0057] .

[0059] S2, constructing an optimization model based on the objective function and constraint conditions obtained in step S1; The multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts is established as follows:

[0060] Among them, the solution methods mostly use static optimization methods (such as genetic algorithms, etc.), but their effects may be limited when facing a multi-pass cutting parameter model that requires dynamic optimization. Therefore, a solution method based on deep reinforcement learning is adopted to build an agent model environment driven by data and mechanism hybrid, so that the intelligent agent can interact with it, and gradually optimize its decision-making and action strategies through continuous learning and accumulation of experience, so as to adapt to the current environment and explore the optimal strategy of multi-objective cutting parameters under the current constraints.

[0061] S3, establishing a multi-pass cutting parameter optimization solution framework; A multi-pass cutting parameter autonomous decision framework based on SAC was established, which includes two main processes: environment modeling based on agent model and optimization solution of SAC method. Based on agent model environment modeling, an agent model of machining environment functional simulation was constructed to help deep reinforcement learning interact with dynamic environment. SAC method was applied as the intelligent agent of DRL for strategy improvement and evaluation, and a replay buffer was set to store experience.

[0062] S4, build agent model environment; A digital-analog agent model environment (DMDE) is constructed to perceive the environmental state of the multi-pass machining process, thereby helping the SAC agent to interact with it and learn the optimal cutting parameter strategy under the current machining environment. The construction of DMDE includes two modules: an integrated transfer learning prediction module and an empirical model calculation module. The carbon source that needs to be calculated using empirical models and the theoretical calculation of MRR.

[0063] See also Figure 2, the proxy environment construction process is as follows: S401, constructing an agent model environment for digital-analog linkage; S4011, Process performance data: including i The 、MRR i 、 , which is used to reflect the specific impact of the selected cutting parameters on carbon emissions, material removal rate, and processing quality. In addition, the current maximum cutting force threshold should also be included , Processing power , Processing time , as well as the relevant parameters of the machine tool and tool to determine whether the selected cutting parameters meet the current constraints; S4012, WIP physical information: including current stiffness of parts , which varies with the residual thickness Decrease, which can reflect the dynamic strain of the material of the parts during the processing, thus supporting Calculation of Characterize the current surface quality status of the part.

[0064] S5. Solve the dynamic cutting parameter optimization problem based on deep reinforcement learning SAC method to formulate multi-pass cutting parameter optimization strategy.

[0065] The dynamic cutting parameter optimization problem is defined as a Markov decision model problem and solved using deep reinforcement learning to develop a multi-pass cutting parameter optimization strategy.

[0066] The optimization solution mainly includes: MDP problem description: Given the current state and behavior, the state at the next moment is independent of all previous states. Therefore, the cutting parameter optimization decision problem is constructed as an MDP model< S , A , P , R > Figure 3 shown.

[0067] State Space : The thickness to be cut after each cutting pass is taken as the key state variable, that is: .in, Indicates the last cutting pass in the machining process. In addition, the number of cutting passes that have been processed , cumulative carbon emissions And the cumulative processing time It also determines the environmental status of the current pass, so .

[0068] Action space A: The action space is a three-dimensional vector , used to characterize The action space is continuous so that the DRL agent can fully explore it under the given constraints.

[0069] Reward Function :It mainly includes two parts: optimization objective and constraint function. Since SAC cannot directly solve MDP with constraints, the penalty function method is introduced, that is, a penalty term for violating the constraint is added to the reward function. At the same time, due to the different dimensions of the optimization objective and the penalty function, the penalty function is introduced. The function normalizes it, and the formula for the weighted combination of the three is expressed as:

[0070] in, It is a normalization term based on the Sigmoid function, which normalizes the targets of different dimensions to the uniform range of [-1, 1] to keep the reward bounded and prevent divergence; , The constant coefficient for the trade-off between processing carbon emissions and processing efficiency is set to 1 to emphasize the importance of reducing environmental impact while improving production efficiency; is the penalty coefficient. In order to indicate the importance of constraints and ensure the feasibility of the optimization scheme in practical application, Set to 100; is the penalty function:

[0071] In order to support the acquisition of process performance data and physical information of work-in-progress, two modules are set up in DMIE: (1) Integrated transfer learning (ETL) prediction module to achieve and The prediction of Obtained from the trained data model in Chapter 2, It needs to be obtained by ETL after prediction training. The predictor variables include and ,Therefore, according to the ETL process, relevant experimental designs are arranged to collect data to complete the prediction; (2) Empirical model calculation module, mainly including Carbon sources that need to be quantified using empirical models, and MRR i and wait.

[0072] S4022 and SAC methods are used as intelligent agents of DRL to solve MDP problems. SAC is an actor-critic algorithm based on the maximum entropy reinforcement learning framework, which has strong exploration and robustness.

[0073] The SAC method is as follows:

[0074] See also Figure 4 , is the network structure of the SAC method. The input layer of the executor network has 4 neurons, corresponding to 4 element space and contains multiple hidden layers. It has two output layers, each with 2 neurons, which produce the mean and logarithmic standard deviation of the action respectively. The input layer of the critic network has 7 neurons, which is and The output layer has only one neuron, which reflects the single action value of a specific state-action pair, that is, , and linear output is used. In addition, the choice of discount factor plays a decisive role in the learning effect of the SAC agent. To this end, several common discount factor values ​​are selected, and different seeds are used. Taking into account the inherent randomness of SAC, the average cumulative reward and the number of convergence iterations are used as performance indicators to determine the optimal value of the discount factor and the hyperparameter configuration.

[0075] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0076] In yet another embodiment of the present invention, a cutting parameter dynamic autonomous decision-making system based on deep reinforcement learning is provided. The system can be used to implement the above-mentioned cutting parameter dynamic autonomous decision-making method based on deep reinforcement learning. Specifically, the cutting parameter dynamic autonomous decision-making system based on deep reinforcement learning includes a parameter module, a construction module and an optimization module.

[0077] Among them, the parameter module determines the optimization variables, establishes the objective function and designs the constraints; Building module, constructing a multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on objective functions and constraints; The optimization module establishes a multi-pass cutting parameter optimization solution framework, and solves the multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on deep reinforcement learning, realizing autonomous decision-making of low-carbon cutting parameters, effectively reducing processing carbon emissions and improving processing efficiency.

[0078] In another embodiment of the present invention, a terminal device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the dynamic autonomous decision-making method of cutting parameters based on deep reinforcement learning, including: Determine the optimization variables, establish the objective function and design constraints; construct a multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on the objective function and constraints; establish a multi-pass cutting parameter optimization solution framework, and solve the multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on deep reinforcement learning, realizing autonomous decision-making of low-carbon cutting parameters, effectively reducing machining carbon emissions and improving machining efficiency.

[0079] See also Figure 6 , the terminal device is a computer device. The computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the dynamic autonomous decision-making method for cutting parameters based on deep reinforcement learning in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the dynamic autonomous decision-making system for cutting parameters based on deep reinforcement learning in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0080] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 6This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0081] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0082] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0083] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0084] See also Figure 7 , the terminal device is a chip, and the chip 600 of this embodiment includes a processor 622, which may be one or more, and a memory 632 for storing a computer program executable by the processor 622. The computer program stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 622 may be configured to execute the computer program to execute the above-mentioned dynamic autonomous decision-making method for cutting parameters based on deep reinforcement learning.

[0085] In addition, the chip 600 may further include a power supply component 626 and a communication component 650, wherein the power supply component 626 may be configured to perform power management of the chip 600, and the communication component 650 may be configured to implement communication, for example, wired or wireless communication, of the chip 600. In addition, the chip 600 may further include an input / output interface 658. The chip 600 may operate based on an operating system stored in the memory 632.

[0086] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-VolatileMemory), such as at least one disk memory.

[0087] The processor may load and execute one or more instructions stored in a computer-readable storage medium to implement the corresponding steps of the method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning in the above embodiment; the processor may load and execute the following steps of one or more instructions in the computer-readable storage medium: Determine the optimization variables, establish the objective function and design constraints; construct a multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on the objective function and constraints; establish a multi-pass cutting parameter optimization solution framework, and solve the multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on deep reinforcement learning, realizing autonomous decision-making of low-carbon cutting parameters, effectively reducing machining carbon emissions and improving machining efficiency.

[0088] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0089] In order to verify the method proposed in the present invention, the JDGR400_A13S machining center of the Intelligent Manufacturing Innovation Center of the Innovation Port Campus of Xi'an Jiaotong University was used as the experimental platform, and the material 7050-T7451 high-strength aluminum alloy thin-walled parts were used as the test objects (such as Figure 5 As shown in the figure, a multi-objective cutting parameter dynamic autonomous decision-making method based on deep reinforcement learning proposed in the present invention is verified. Through the specific case of aviation thin-walled parts processing process, it can be verified that compared with the benchmark method, the proposed method can significantly reduce machining carbon emissions and improve material removal rate in the prediction results. Among them, machining carbon emissions are reduced by 12.41% compared with the benchmark method, and the material removal rate is increased by 33.03% compared with the benchmark method, indicating that deep reinforcement learning is more effective in multi-pass cutting parameter optimization.

[0090] On the other hand, compared with the currently popular algorithms, the proposed method achieves a better balance between improving material removal rate and reducing machining carbon emissions, which proves that the proposed optimization method can effectively support autonomous decision-making of multi-objective cutting parameters. The specific statistical data are shown in the following table:

[0091] The present invention studies the influence of cutting parameters on the elastic deformation of the workpiece under the action of cutting force; secondly, according to the law of strain change of the workpiece material, a constraint function of the elastic deformation of the workpiece caused by the time-varying cutting force is established, and with the improvement of carbon emissions and machining efficiency in the machining process as the optimization goal, a multi-objective dynamic decision-making model of cutting parameters under the constraint of time-varying workpiece deformation is constructed; the model solving framework of the SAC method is designed to realize the autonomous decision-making of multi-objective cutting parameters in the five-axis side milling of complex aviation parts.

[0092] In summary, the present invention provides a method and system for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning. According to the principle that the workpiece stiffness changes with the remaining cutting thickness of each layer, a time-varying workpiece deformation constraint threshold is designed. Taking processing efficiency as the optimization goal, a multi-objective dynamic optimization model of cutting parameters for complex aviation parts is established, which allows the optimal cutting parameters to be formulated for each processing pass under the current deformation constraints. Considering the dynamic characteristics of the optimization model in the decision-making process, it is reconstructed into an MDP model, and a multi-objective autonomous decision-making framework for cutting parameters based on deep reinforcement learning is designed.

[0093] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0094] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0095] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0096] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0098] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0099] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0100] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0103] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A dynamic autonomous decision-making method for cutting parameters based on deep reinforcement learning, characterized in that: The following steps are involved: Determine optimization variables, establish objective functions and design constraints; Construct a multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on objective functions and constraints; A multi-pass cutting parameter optimization solution framework is established, and the multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts is solved based on deep reinforcement learning to achieve autonomous decision-making on low-carbon cutting parameters.

2. The method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning according to claim 1 is characterized in that: Processing efficiency and carbon emissions To optimize the target, the objective function is as follows: in, is the material removal rate.

3. The method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning according to claim 2 is characterized in that: Determine the optimization variables as follows: In the five-axis side milling of complex aviation parts, It is i The cutting depth of each pass, is the feed rate, is the spindle speed, It is the key variable to be optimized in cutting parameters. and Considered as the decision variable in each cut .

4. The method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning according to claim 2 is characterized in that: The specific constraints are: Constraint inequality for the theoretical maximum cutting force: Correlation between the current actual cutting force and deformation: in, is the safety factor, The cutting force limit is determined by factors such as the workpiece or tool material properties, fixing method, and machine tool capabilities during processing; is the theoretical maximum cutting force threshold, It is the lateral load per unit area on the workpiece surface in the current pass; is the surface area of ​​the workpiece; D is the bending stiffness, E is the elastic modulus; μ is Poisson’s ratio; The sum of the cutting depths of each pass must be equal to the thickness to be cut: in, I is the number of multi-pass cutting, For the i The cutting depth of each pass, is the thickness to be cut; Key variables meet machine tool and cutting tool specifications: in, is the cutting power per pass, The spindle efficiency is is the rated power of the spindle motor, are the upper and lower limits of the cutting parameters selected for each pass, For the i The cutting parameter combination selected for each pass, Respectively i The spindle speed, feed rate, cutting depth of each pass, For the i Processing time within a pass, is the preset tool life, is the maximum surface roughness constraint, For the i Surface roughness of each pass, , are the front and back angles of the tool.

5. The method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning according to claim 1, characterized in that: The multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts is as follows: in, For the i Carbon emissions from each processing pass, For the i Material removal rate per pass, For the i The total cutting force generated by the contact between the tool and the workpiece during the pass, is the current maximum cutting force, are the upper and lower limits of the cutting parameters selected for each pass, For the i The cutting parameter combination selected for each pass, is the cutting power per pass, The spindle efficiency is is the rated power of the spindle motor, For the i Processing time within a pass, is the preset tool life, is the maximum surface roughness constraint, For the i Surface roughness of each pass.

6. The method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning according to claim 1, characterized in that: The multi-pass cutting parameter optimization solution framework includes: Based on the agent model's environmental modeling, an agent model for functional simulation of the machining environment is constructed; The SAC method is applied as the intelligent agent of DRL to solve the dynamic cutting parameter optimization problem for strategy improvement and evaluation, and a replay buffer is set to store experience.

7. The method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning according to claim 6, characterized in that: The proxy model environment of digital-analog linkage includes: Process performance data: i The 、MRR i 、 , which is used to reflect the specific impact of the selected cutting parameters on carbon emissions, material removal rate, processing quality, and the current maximum cutting force threshold , Processing power , Processing time , as well as the relevant parameters of the machine tool and the tool, to determine whether the selected cutting parameters meet the current constraints; WIP physical information: current stiffness of the part , which varies with the residual thickness Decrease, reflecting the dynamic strain of the material of the parts during the processing, thus supporting Calculation of Characterize the current surface quality status of the part.

8. The method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning according to claim 1, characterized in that: Solving the dynamic cutting parameter optimization problem includes: Given the current state and behavior, the state at the next moment is independent of all previous states, and the cutting parameter optimization decision problem is constructed as an MDP model. S , A , P , R > The SAC method is used as the intelligent agent of DRL to solve the MDP problem. In the network structure of the SAC method, the input layer of the executor network has 4 neurons, corresponding to 4 element space, and contains multiple hidden layers; it has two output layers, each with 2 neurons, which respectively produce the mean and logarithmic standard deviation of the action; the input layer of the critic network has 7 neurons, which is and The output layer has only one neuron, which reflects the single action value of a specific state-action pair and uses linear output.

9. The method for dynamic autonomous decision-making of cutting parameters based on deep reinforcement learning according to claim 8, characterized in that: State Space : The thickness to be cut after each cutting pass is taken as the key state variable: in, Indicates the last cutting pass in the machining process and the number of cutting passes that have been processed , cumulative carbon emissions And the cumulative processing time Determines the environmental status of the current pass, therefore, ; Action space A: is a three-dimensional vector , used to characterize Processing parameters during cutting; Reward Function :Includes optimization objectives and constraint functions; introduces penalty function method, adds penalty items that violate constraints in reward function, introduces The function normalizes it, and the formula for the weighted combination of the three is expressed as: in, is the normalization term based on the Sigmoid function, , is the constant coefficient for the trade-off between processing carbon emissions and processing efficiency, is the penalty coefficient, is the penalty function, For Carbon emissions from cutting at all times, For Cutting efficiency at all times.

10. A cutting parameter dynamic autonomous decision-making system based on deep reinforcement learning, characterized in that: include: Parameter module, determines the optimization variables, establishes the objective function and designs the constraints; Building module, constructing a multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on objective functions and constraints; The optimization module establishes a multi-pass cutting parameter optimization solution framework, solves the multi-objective cutting parameter optimization model for five-axis side milling of complex aviation parts based on deep reinforcement learning, and realizes autonomous decision-making of low-carbon cutting parameters.

Images