Hydrostatic bearing orifice arrangement and size optimization method, system and equipment based on deep reinforcement learning

Abstract

The invention discloses a hydrostatic bearing throttling hole arrangement and size optimization method, system and device based on deep reinforcement learning, and relates to the technical field of high-end equipment core part design and artificial intelligence, and the method comprises the steps: constructing a deep reinforcement learning agent; constructing a hydrostatic bearing simulation environment, receiving a mixed action output by the deep reinforcement learning agent based on the hydrostatic bearing simulation environment, calculating a multi-dimensional performance index, and calculating a reward value by adopting a multi-target reward function based on the multi-dimensional performance index; performing sequential interaction on the deep reinforcement learning agent and the hydrostatic bearing simulation environment to obtain a trained deep reinforcement learning agent; the trained deep reinforcement learning intelligent body is used for generating an optimal design scheme of the hydrostatic bearing throttling hole; and the arrangement and the size of the throttling holes are optimized based on the optimization design scheme. According to the method, the wide design space of the layout and the size of the throttling holes of the hydrostatic bearing can be automatically and efficiently explored.

Description

Technical Field

[0001] This application relates to the fields of core component design and artificial intelligence technology for high-end equipment, and in particular to a method, system and equipment for optimizing the arrangement and size of throttling holes in hydrostatic bearings based on deep reinforcement learning. Background Technology

[0002] As core supporting components of national strategic equipment such as ultra-precision machine tools, large radio telescopes, and heavy-duty rotary tables, the performance of hydrostatic bearings directly determines the accuracy, stability, and service life of the entire equipment. The design of hydrostatic bearings is essentially an extremely complex multi-objective optimization problem, requiring designers to seek the optimal balance among multiple conflicting and even mutually restrictive performance indicators (static load capacity, static stiffness, power, and dynamic characteristics).

[0003] In design practice, there are inherent contradictions among these performance indicators. For example, in order to pursue higher static load capacity and static stiffness, designers often increase the oil supply pressure or the number of throttling orifices, but this will inevitably lead to a sharp increase in pumping power.

[0004] Traditional hydrostatic bearing design methods heavily rely on the experience, intuition, and extensive trial-and-error iterations of design engineers. The design process typically involves simplified analytical models or simulations comparing a few pre-defined layout schemes using the finite element / finite difference method. These methods have significant limitations: (1) Insufficient exploration of design space: The design space has extremely high dimensions (number of throttling holes, two-dimensional coordinates of each hole, hole diameter, etc.). Traditional methods can only explore a very small part of the huge design space, which is very easy to get stuck in local optimal solutions and cannot find global optimal or non-traditional, high-performance design solutions.

[0005] (2) Inefficient and costly: The repeated manual modeling, simulation analysis and scheme adjustment process is time-consuming and laborious, which seriously prolongs the R&D cycle of high-end equipment.

[0006] (3) Unable to systematically handle multi-objective conflicts: When faced with multiple conflicting performance indicators, traditional methods often rely on engineers’ trade-off decisions and lack a systematic and quantifiable method to reveal the inherent performance constraints between different design choices.

[0007] (4) Ignoring dynamic characteristics: Traditional design processes often focus on static performance optimization and do not give enough consideration to dynamic characteristics, which may lead to instability of the designed bearings during actual high-speed operation. Summary of the Invention

[0008] The purpose of this application is to provide a method, system, and device for optimizing the layout and size of hydrostatic bearing throttling holes based on deep reinforcement learning. This method can automatically and efficiently explore a wide design space for the layout and size of hydrostatic bearing throttling holes. It can not only systematically generate multi-objective Pareto optimal fronts covering static and dynamic performance, but also directly apply the optimization results to downstream rotor system dynamics analysis, thereby realizing a complete closed loop from intelligent optimization to engineering verification.

[0009] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a method for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning, including: Construct a deep reinforcement learning agent; the deep reinforcement learning agent includes a policy network and a value network, the policy network is used to output a hybrid action based on the current state in the state space, and the value network is used to evaluate the value of the hybrid action; wherein, the state space is a design scheme of the throttle orifice of a hydrostatic bearing, including the number of throttle orifices, the position parameters of each throttle orifice, and the geometric dimension parameters; A hydrostatic bearing simulation environment is constructed, and the hybrid actions output by the deep reinforcement learning agent are received based on the hydrostatic bearing simulation environment. Multidimensional performance indicators are calculated, and a multi-objective reward function is used to calculate the reward value based on the multidimensional performance indicators. The multidimensional performance indicators include static performance indicators and dynamic performance indicators. The deep reinforcement learning agent sequentially interacts with the hydrostatic bearing simulation environment, and updates the parameters of the policy network and the value network through an experience replay mechanism until the preset iteration termination condition is met, thus obtaining the trained deep reinforcement learning agent. An optimized design scheme for the throttling orifice of a hydrostatic bearing is generated using a trained deep reinforcement learning agent. The arrangement and size of the throttling orifice are optimized based on the aforementioned optimized design scheme.

[0010] Secondly, this application provides a system for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning, including: A deep reinforcement learning agent construction module is used to construct a deep reinforcement learning agent. The deep reinforcement learning agent includes a policy network and a value network. The policy network is used to output a mixed action based on the current state in the state space, and the value network is used to evaluate the value of the mixed action. The state space is a design scheme of the throttle orifice of a hydrostatic bearing, including the number of throttle orifices, the position parameters of each throttle orifice, and the geometric dimension parameters. A hydrostatic bearing simulation environment construction module is used to construct a hydrostatic bearing simulation environment, and receive the mixed actions output by the deep reinforcement learning agent based on the hydrostatic bearing simulation environment, calculate multi-dimensional performance indicators, and calculate the reward value based on the multi-dimensional performance indicators using a multi-objective reward function; the multi-dimensional performance indicators include static performance indicators and dynamic performance indicators; An interaction module is used for sequential interaction between the deep reinforcement learning agent and the hydrostatic bearing simulation environment. The parameters of the policy network and the value network are updated through an experience replay mechanism until a preset iteration termination condition is met, thereby obtaining the trained deep reinforcement learning agent. An optimized design scheme generation module is used to generate optimized design schemes for hydrostatic bearing throttling orifices using a trained deep reinforcement learning agent. The optimization module is used to optimize the arrangement and size of the throttling orifice based on the optimized design scheme.

[0011] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for optimizing the arrangement and size of hydrostatic bearing throttle holes based on deep reinforcement learning.

[0012] According to the specific embodiments provided in this application, this application has the following technical effects: (1) This application realizes the automated and systematic exploration of the design space of the throttle hole of the hydrostatic bearing through a deep reinforcement learning agent, which breaks through the dependence of traditional methods on experience, can efficiently cover the high-dimensional design space, avoid getting trapped in local optima, and help to discover the global optimum or non-traditional high-performance design schemes. (2) By leveraging the sequential interaction between deep reinforcement learning agents and the hydrostatic bearing simulation environment, automated iteration is achieved, replacing the traditional manual modeling and trial-and-error process, which greatly improves design efficiency and shortens the R&D cycle of high-end equipment. (3) By generating multi-objective reward functions and Pareto fronts, the conflict between static and dynamic performance indicators is systematically quantified, providing a quantifiable basis for design decisions and overcoming the limitations of traditional methods that rely on manual trade-offs. (4) Integrate dynamic performance indicators for optimization to make up for the shortcomings of traditional design that focuses on static performance and ignores dynamic characteristics, and reduce the risk of instability of bearings when running at high speed. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating a method for optimizing the arrangement and size of throttle holes in hydrostatic bearings based on deep reinforcement learning, provided as an embodiment of this application; Figure 2 To optimize the training process, the convergence curves of various performance indicators as a function of the number of training iterations are shown in the following diagrams: (a) is the convergence curve of static bearing capacity as a function of the number of training iterations; (b) is the convergence curve of static stiffness as a function of the number of training iterations; (c) is the convergence curve of friction power as a function of the number of training iterations; (d) is the convergence curve of pumping power consumption (i.e., pumping power) as a function of the number of training iterations; and (e) is the convergence curve of dynamic performance indicators as a function of the number of training iterations. Figure 3 A schematic diagram of a specific optimal throttle orifice spatial layout scheme determined for the evaluation model; Figure 4 To and Figure 3 A schematic diagram showing the bearing oil film pressure field distribution corresponding to the optimal layout scheme; Figure 5 This is a schematic diagram of a Campbell's diagram; Figure 6 The diagram shows the Pareto optimal frontier; where (a) represents the constraint and trade-off relationship between pumping power and stiffness (i.e., static stiffness), (b) represents the constraint and trade-off relationship between pumping power and bearing capacity (i.e., static bearing capacity), (c) represents the constraint and trade-off relationship between steady-state performance index (i.e., dynamic performance index) and stiffness (i.e., static stiffness), and (d) represents the constraint and trade-off relationship between steady-state performance index (i.e., dynamic performance index) and bearing capacity (i.e., static bearing capacity). Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] The purpose of this application is to overcome the shortcomings of existing technologies and provide a broad design space that can automatically and efficiently explore the layout and size of the throttling orifice in hydrostatic bearings. This not only systematically generates a multi-objective Pareto optimal front covering both static and dynamic performance, but also directly applies the optimization results to downstream rotor system dynamics analysis, thus achieving a complete closed loop from intelligent optimization to engineering verification. Based on this, this application constructs the complex bearing design process as a Markov Decision Process (MDP). This process involves a closed-loop interaction between a deep reinforcement learning agent and a high-fidelity bearing physics simulation environment. The agent is responsible for sequentially proposing design modification "actions," while the simulation environment evaluates the performance changes caused by these actions and returns a "reward" signal. The agent learns the optimal design strategy by maximizing the cumulative reward.

[0017] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] In one exemplary embodiment, a method for optimizing the arrangement and size of throttle holes in hydrostatic bearings based on deep reinforcement learning is provided. The design problem of throttle holes in hydrostatic bearings is formalized as a Markov decision process, the key elements of which include: State space: the state at any given time t. This is a complete mathematical description of the current hydrostatic bearing orifice design scheme. It is defined as a normalized, fixed-length continuous floating-point vector containing the number of orifices, the position of each orifice (e.g., polar coordinates r, θ), and key geometric dimensions (e.g., diameter d). Its dimensions are 1 (number) + N_max (activation flag) + 3 × N_max (position and size parameters), where N_max is the maximum allowed number of orifices. This state representation method allows design schemes with different topologies to be compared in the same vector space.

[0019] Hybrid Action Space: To simultaneously optimize the topology and geometric parameters of a hydrostatic bearing, this application designs a novel hybrid action space. The hybrid action output by the agent (i.e., a deep reinforcement learning agent) at time t is... It contains two parts: Discrete actions: determine what type of macroscopic design modification to perform, such as adding, deleting, or modifying orifices, for example, "adding a new orifice", "deleting an existing orifice", or "modifying a specific orifice".

[0020] Continuous action: Determines the parameters of discrete actions. For example, in an "Add" action, continuous action will determine the position and size of the new orifice; in a "Modify" action, it will determine the adjustment amount to the existing parameters.

[0021] Specifically, discrete action values ​​and continuous action vectors are extracted from the mixed actions. Then, depending on the type of discrete action, the continuous action vector is used in different ways: if added, the continuous action vector in [-1,1] is transformed to [0,1], then denormalized to the physical position and size parameters, and added to the orifice list; if deleted, the first value of the continuous action vector is used to determine which index of the orifice in the list to delete; if modified, the continuous action vector in [-1,1] is used as the normalization increment for the existing parameters of the specified orifice, modified in the normalization space, and then denormalized, thereby achieving fine-tuning of the existing design.

[0022] Multi-objective Reward Function: To guide the agent in learning high-performance designs, the multi-objective reward function is designed as a weighted sum of multiple performance metrics. These metrics include not only static performance metrics (static load capacity, static stiffness, pumping power, and friction power), but also, innovatively, a quantified dynamic performance metric. This dynamic performance metric is directly derived from the complete set of eight dynamic characteristic coefficients (stiffness matrix and damping matrix) calculated in the simulation environment. The expression for the multi-objective reward function is: in, Let be the reward value for the t-th iteration. , , , Static stiffness Static bearing capacity Pumping power Friction power Dynamic stability index The weighting coefficients, (•) is the normalization function. By adjusting the weights, the agent can be guided to optimize towards different design preferences.

[0023] like Figure 1 As shown, the above method specifically includes the following steps.

[0024] S1: Construct a deep reinforcement learning agent; the deep reinforcement learning agent includes a policy network and a value network, the policy network is used to output a hybrid action based on the current state in the state space, and the value network is used to evaluate the value of the hybrid action.

[0025] The agent acts as a "decision-maker," and this application employs the advanced SAC (Soft Actor-Critic) algorithm as its core. The SAC algorithm is particularly suitable for handling the mixed action space proposed in this application, and its objective function of maximizing entropy helps the agent to explore more thoroughly. The agent's goal is to learn an optimal policy π. | This strategy can adjust according to the current state. Provide a hybrid action that maximizes long-term cumulative rewards. .

[0026] (1) Network architecture.

[0027] Policy Network: Its core is a multilayer perceptron (MLP). The key difference lies in its output layer being divided into two heads: Discrete Action Header: The output of the MLP is passed through a linear layer to obtain logits representing each discrete action.

[0028] Continuous Action Head: The output of the MLP passes through two additional parallel linear layers to predict the mean μ and log-standard deviation log(σ) of the Gaussian distribution for continuous actions, respectively. The mean output is activated by the tanh activation function to ensure its range is [-1, 1]. During forward propagation, discrete and continuous actions are sampled using categorical and normal distributions, respectively.

[0029] Value Network: Also inherits from the neural network base class. It receives a concatenated vector of states and actions as input, passes it through an MLP, and finally outputs a single Q-value. To implement the double Q-learning technique in SAC to alleviate the Q-value overestimation problem, two independent value networks, Q1 and Q2, are instantiated in the agent.

[0030] (2) Algorithm flow.

[0031] 1) Initialization: During initialization, the agent creates a policy network, two value networks, and their corresponding target networks. It also initializes the optimizer and the experience replay buffer.

[0032] 2) Action selection: This method calls the forward propagation method of the policy network to obtain the action distribution, and decides whether to perform random sampling (exploration during training) or select the mean of the distribution (utilization during evaluation) based on the evaluation flag.

[0033] 3) Training Function: This method is invoked when the amount of data in the experience replay pool exceeds the batch size, and it performs the following steps.

[0034] ① Randomly sample a batch of data (s,a,r,s',d) from the experience replay buffer, where s is the current state, a is the action performed, r is the reward obtained, s' is the next state, and d is the end-of-round flag, which is 1 if the round has ended and 0 otherwise.

[0035] ② Update the value network: First, calculate the objective Q-value y. The objective Q-value of SAC includes the Q-value of the next state and the entropy of the policy to encourage exploration: in, The target Q-value is used to guide the updating of the value network. This is a discount factor used to balance the importance of current rewards and future rewards. The state calculated for the target value network Next action The SAC algorithm uses two Q-networks (i=1,2) and takes the smaller one to suppress the problem of overestimation of the Q-value. This is the entropy regularization coefficient, used to control the balance between exploration and exploitation. For policy networks, It is the logarithmic probability of the policy, representing the entropy of the policy.

[0036] ③ Then calculate the mean squared Bellman error (MSE) loss function for the two value networks. It then performs backpropagation and gradient updates.

[0037] in, Let i be the loss function of the i-th value network. This represents the expectation of (state, action) pairs sampled from the experience playback buffer D. This is the Q-value prediction for the current value network for state s and action a.

[0038] ④ Update the policy network: The goal of the policy network is to maximize the "soft" Q-value of its output action. Its loss function... Defined as: in, Let the loss function be the policy network. This represents the expectation of a state s sampled from the experience replay buffer D and an action a sampled according to the current policy. The entropy term of the policy encourages the policy to explore more diverse actions. The smaller Q-value in the two value networks is used to guide the policy to update towards a higher Q-value.

[0039] ⑤ Update entropy coefficient α: If automatic entropy tuning is enabled, it will be based on the current policy entropy and the target entropy. The difference is used to update α, and its loss function J(α) is defined as: .

[0040] in, Entropy coefficient loss function, This represents the expectation of action 'a' sampled according to the current policy. The entropy of the current strategy.

[0041] 4) Model saving and loading: Save the state dictionary of all networks to a file or load it from a file.

[0042] S2: Construct a hydrostatic bearing simulation environment, and receive the mixed actions output by the deep reinforcement learning agent based on the hydrostatic bearing simulation environment, calculate multi-dimensional performance indicators, and calculate the reward value based on the multi-dimensional performance indicators using a multi-objective reward function; the multi-dimensional performance indicators include static performance indicators and dynamic performance indicators.

[0043] Step S2 specifically includes: updating the hydrostatic bearing geometric model in the hydrostatic bearing simulation environment based on the hybrid actions output by the deep reinforcement learning agent; calculating the pressure distribution using the generalized Reynolds equation including the extrusion film effect based on the updated hydrostatic bearing geometric model; and calculating multidimensional performance indicators based on the pressure distribution.

[0044] The high-fidelity hydrostatic bearing simulation environment receives an action from the intelligent agent. Subsequently, the hydrostatic bearing simulation environment will: (a) update its internal hydrostatic bearing geometry model; (b) solve for the new pressure distribution; (c) calculate all static and dynamic performance indices; and (d) calculate the reward value based on the aforementioned multi-objective reward function. It returns the reward value and the next state. .

[0045] The hydrostatic bearing simulation environment serves as a bridge between the physical world and artificial intelligence. It is implemented as an environment class that conforms to a standard interface and encapsulates a sophisticated physics solver.

[0046] Physical Solver: The solver is based on the generalized Reynolds equation, which incorporates the squeezing film effect and describes the pressure distribution within the lubricating oil film. Where p is the oil film pressure, h is the oil film thickness, μ is the dynamic viscosity of the lubricating oil, and ω is the angular velocity of the bearing. θ and θ are the radial and angular coordinates in polar coordinates. The first term on the right-hand side of the equation The wedge effect term, generated by the rotation of the axis, is the second term. This is the extrusion film effect term, generated by the dynamic displacement of the axis, and is crucial for dynamic characteristic analysis.

[0047] Oil film thickness: The oil film thickness function depends not only on the position, but also on the small displacement (x, y) of the journal center relative to the bearing center and the velocity. related: in, For any position and time Oil film thickness, It is the average radial clearance of the bearing. Let t be the displacement of the axis in the x and y directions at time t.

[0048] Its derivative with respect to time is: Pressure field solution: The Reynolds equations are discretized using the finite difference method. The solver creates a two-dimensional discrete grid in polar coordinates and obtains the pressure field by iteratively solving the linear equations until the maximum relative error between two iterations is less than the preset convergence criterion. To accelerate convergence, an over-relaxation iterative method is used. Boundary conditions are: the pressure at the inner and outer diameters of the bearing is the ambient pressure, and the pressure at the outlet oil cavity of each throttling orifice is the oil cavity pressure to be solved.

[0049] Static performance index calculation: After obtaining the convergent pressure field, the bearing capacity in the x and y directions can be obtained by performing two-dimensional numerical integration on the pressure field. , The static bearing capacity L and static stiffness S are respectively: Static load capacity refers to the total load that a bearing can withstand in a static state, which is obtained by combining forces in the x and y directions. Static stiffness is defined as static bearing capacity. Relative to the central film thickness The rate of change characterizes the bearing's ability to resist deformation.

[0050] Pumping power refers to the power consumed by a hydraulic pump to overcome the resistance of the throttle orifice and maintain the pressure in the bearing oil chamber. Its core calculation formula is as follows: Pumping power (unit: watts, W). The oil supply pressure is expressed in Pascals (Pa). The total flow rate into the bearing (unit: cubic meters per second) ).

[0051] The total flow rate is the sum of the flow rates through all the orifices. For the first... A throttling orifice, its flow rate The flow rate formula for thin-walled orifices is usually used for calculation: is the flow coefficient of the orifice, a dimensionless constant determined by the shape of the orifice and the flow state. For the first The cross-sectional area of ​​each throttling orifice (unit: square meters) For circular holes, ,in It is the diameter of the throttling orifice, which is a "geometric dimension parameter" in the state space of this embodiment. The density of lubricating oil (unit: kilograms per cubic meter). ). For the first The pressure (in Pascals, Pa) supplied to the oil chamber by each orifice is obtained by solving the generalized Reynolds equation.

[0052] Therefore, the calculation process for pumping power can be summarized as follows: obtain the pressure of each oil chamber by solving the Reynolds equation, further calculate the flow rate of each throttling orifice, then sum them to obtain the total flow rate, and finally calculate the pumping power.

[0053] Frictional power refers to the power loss caused by the viscous shear of the lubricating oil film between the rotating journal and the bearing surface. Its calculation formula is: Among them, shear stress Determined by Newton's law of internal friction: Frictional power (unit: watt, W). The shear stress in the oil film (unit: Pascal, Pa). The linear velocity of the journal surface (unit: meters per second). ), and rotational angular velocity Related ( ). : Micro-element area (unit: square meters) (), integral over the entire lubrication area The above will be carried out. The velocity gradient along the oil film thickness direction (unit: per second). Under the simple Cuyet flow assumption, this gradient can be approximated as... .

[0054] Dynamic performance index calculation: The complete set of eight dynamic characteristic coefficients (stiffness matrix K and damping matrix C) is calculated using the numerical perturbation method. The calculation is performed at the central equilibrium position. Using this as a reference, small perturbations are applied to the displacement and velocity respectively (e.g. Each disturbance triggers a new solution for the pressure field and a recalculation of the resultant force change. The partial derivatives of the force are then calculated using the central difference method to obtain the stiffness coefficient. and damping coefficient : in, For stiffness matrix coefficients, represent the rate of change of force in direction i relative to displacement in direction j (i, j can be x or y). The damping matrix coefficients represent the rate of change of force in the i-direction relative to velocity in the j-direction (i and j can be x or y). Let i be the oil film force in the i-direction; The displacement and velocity at the equilibrium position (usually 0); Let i and j be the small perturbations applied to the displacement and velocity. i and j can take the values ​​x or y, resulting in a total of 8 coefficients that form the stiffness matrix K and the damping matrix C. This formula uses the central difference method to approximate the partial derivatives of the forces, thereby obtaining the dynamic characteristic coefficients.

[0055] S3: The deep reinforcement learning agent sequentially interacts with the hydrostatic bearing simulation environment, and updates the parameters of the policy network and the value network through an experience replay mechanism until the preset iteration termination condition is met, thus obtaining the trained deep reinforcement learning agent.

[0056] S4: Utilize the trained deep reinforcement learning agent to generate an optimized design scheme for the throttling orifice of the hydrostatic bearing.

[0057] S5: Optimize the arrangement and size of the throttling orifice based on the optimized design scheme.

[0058] After step S5, the method further includes: performing dynamic analysis and Pareto front generation on the optimized design scheme.

[0059] (1) Dynamic analysis can extract the stiffness matrix and damping matrix corresponding to the optimized design scheme and integrate them into a complete rotor system finite element model, automatically generating a Campbell diagram to evaluate the stability of the design at the system level. The dynamic analysis process includes: extracting the dynamic performance index corresponding to the optimized design scheme; integrating the corresponding dynamic performance index into the preset rotor system finite element model to construct the global mass matrix, gyro effect matrix, system stiffness moment and system damping matrix of the rotor-bearing coupled system; solving the eigenvalues ​​of the rotor-bearing coupled system based on the global mass matrix, the gyro effect matrix, the system stiffness moment and the system damping matrix; and generating a Campbell diagram within a preset speed range based on the eigenvalues.

[0060] The implementation process of dynamic analysis: 1) Coefficient extraction: After the dynamic analysis is started, a trained agent is first loaded, and then the evaluation function is called to perform an evaluation to find the optimal design that the model can produce. The stiffness matrix K and damping matrix C corresponding to the design are then extracted from the returned information dictionary.

[0061] 2) Rotor System Modeling: In the dynamic analysis function, a finite element model of the rotor system is defined, including nodes, disks, and specific geometric and material properties. The extracted K and C matrices are assigned to specific nodes of this model as bearing supports.

[0062] 3) System Matrix Integration: Based on finite element theory, the mass and stiffness of each part of the rotor, as well as the stiffness and damping matrices of the bearings, are assembled into the global mass matrix M, gyroscopic effect matrix G, and stiffness matrix of the entire rotor-bearing system. and damping matrix .

[0063] 4) Critical speed solution: The eigenvalues ​​λ of the system are calculated by solving a generalized eigenvalue problem in state-space form. This problem describes the free vibration of the system. This equation is an eigenvalue problem in state-space form for a rotor-bearing coupled system. is the characteristic value of the system, whose imaginary part corresponds to the natural frequency of the system, and whose real part is related to the damping ratio of the system. The system's state vector includes the displacement vector. and velocity vector ; The state matrix of the system; It is an identity matrix.

[0064] The system scans within a range of rotational speeds and calculates the eigenvalues ​​at each speed. The imaginary part of the eigenvalues ​​corresponds to the system's natural frequency, while the real part is related to the damping ratio.

[0065] 5) Campbell plot drawing: Finally, the calculated series of natural frequencies as a function of rotational speed are visualized and plotted as a Campbell plot. An additional 1X synchronous excitation line can be plotted, allowing designers to intuitively identify the critical speed and unbalanced response region of the system.

[0066] (2) Pareto front generation provides designers with a global multi-objective optimization perspective. The process of Pareto front generation includes: adjusting the weights of the multi-objective reward function to obtain multiple optimal design schemes; using a non-dominated sorting algorithm to screen all optimal design schemes to obtain a non-dominated solution set; and visualizing the non-dominated solution set in the performance index space to generate the Pareto optimal front.

[0067] The Pareto frontier generation process is as follows.

[0068] 1) Receives a predefined list of weight combinations.

[0069] 2) Iterate through each weight vector in this list.

[0070] 3) For each weight vector, create a brand new environment instance (configured with the current weights) and agent instance, and execute a complete, independent training process.

[0071] 4) After each independent training session, record the performance index vector of the optimal design found under that weight.

[0072] 5) After traversing all weights, calculate the set of non-dominated solutions from these solutions, i.e., the Pareto optimal frontier, and call the visualization function to draw it into a chart.

[0073] By running the optimization process multiple times independently, using a different set of reward function weights w in each run, a series of optimal solutions with different performance emphases can be systematically obtained. Connecting these non-dominated solutions allows us to depict the Pareto optimal front of the design. The Pareto front is a set of non-dominated solutions in which any improvement in any performance metric of any solution necessarily comes at the expense of at least one other performance metric. By obtaining the complete Pareto front, design engineers can have a global design perspective and make the wisest and most reasonable final choice among all the optimal trade-offs based on actual engineering needs (such as energy budgets, cost constraints, etc.).

[0074] The overall framework of this application is scheduled by a main control program that integrates all steps and provides a flexible operating mode through a command-line interface.

[0075] (1) Entry and Configuration: Started through a main function, it parses command-line arguments and supports multiple running modes, including: Training mode: Executes the training process for the agent; Evaluation mode: Load a pre-trained model and evaluate its performance; Dynamics analysis mode: Based on the evaluation of the optimal design, further dynamics analysis is performed; Pareto optimization mode: Run the Pareto frontier generation process; Sensitivity Analysis Mode: Performs parameter sensitivity analysis; all outputs, including model files, logs, and result images, are saved to a unique directory named after the mode and timestamp for easy traceability and management. Upon system startup, detailed environment information, including the operating environment, working directory, and executed commands, is recorded in the log file and console.

[0076] (2) Core implementation process.

[0077] 1) Initialization: Based on the operating mode, a corresponding hydrostatic bearing simulation environment (hereinafter referred to as the simulation environment) and a deep reinforcement learning agent are created. All key parameters are read from a central configuration file, ensuring the consistency and configurability of system behavior.

[0078] 2) Action selection: During the training or evaluation cycle, the agent's policy network receives the current state. Output a blended action .

[0079] 3) Environment Interaction: The simulation environment executes actions. The internal hydrostatic bearing geometry model is updated, and its physical solver is called to calculate the multidimensional performance indicators corresponding to the new design.

[0080] 4) Reward Calculation and Learning: The simulation environment calculates rewards based on multi-dimensional performance indicators and preset weights. The intelligent agent will use experience tuples ( , , , The data is stored in the experience replay pool and sampled from it for network updates.

[0081] 5) Iteration and convergence: Repeat steps 2) to 4) until the termination condition is met. After training, the final model is saved.

[0082] 6) Analysis and Verification: Based on different operating modes, call the corresponding post-processing programs (such as Pareto analysis and dynamic analysis) to conduct in-depth analysis and visualization of the optimized design scheme, and finally generate design reports and charts.

[0083] Compared with existing hydrostatic bearing design methods, this application has the following significant advantages.

[0084] (1) Automation and intelligence of the design process: This application transforms the traditional bearing design process, which relies on a lot of human experience and tedious trial and error, into an automated intelligent optimization process. The intelligent agent can systematically explore the extremely broad and complex design space that is difficult for the human brain to reach.

[0085] (2) Systematic generation of Pareto optimal frontier: This application does not merely provide a single, compromise design solution, but rather systematically generates a series of Pareto optimal solutions representing different performance trade-offs by adjusting the weights in the multi-objective reward function. This provides design engineers with unprecedented decision support, enabling them to make well-founded and optimal design choices based on the unique constraints of specific application scenarios.

[0086] (3) The ability to discover innovative and non-intuitive designs: Deep reinforcement learning agents are not bound by traditional design paradigms and "rules of thumb" during the exploration process, and therefore have the potential to discover new and non-intuitive orifice layouts and size combinations that may outperform any traditional design.

[0087] (4) Significantly shorten the R&D cycle and reduce costs: By fully automating the iterative cycle of "design-simulation-analysis", this application can shorten the design time that originally required several weeks or even months to several days or hours, which greatly accelerates the R&D process of core components of high-end equipment and reduces R&D costs.

[0088] The method proposed in this application will be described in detail below through a specific hydrostatic bearing optimization task. This embodiment demonstrates the complete technical process from parameter configuration, startup optimization to analysis and verification.

[0089] 1. Parameter configuration.

[0090] In this embodiment, all configurable parameters are defined in a central configuration unit. This unit stores all preset values ​​required to implement the method of this application in a structured data format, mainly including: Bearing geometry and physical parameter set: Defines the basic geometric dimensions of the bearing to be optimized (such as inner and outer diameters, nominal clearance), as well as constraints on design variables (such as the upper and lower limits of the allowable number, diameter, and location of orifices). Operating conditions and fluid properties set: Defines the expected operating conditions of the bearing (such as speed and oil supply pressure) and the physical properties of the lubricant used (such as viscosity and density). Optimization objective parameter set: Defines the relative weight coefficients of each performance index (such as stiffness, stability, power, etc.) in the multi-objective reward function; Agent hyperparameter set: Defines the internal parameters of a deep reinforcement learning agent, such as the learning rate of the neural network, the capacity of the experience replay memory, and the discount factor in the decision-making process.

[0091] 2. Optimize the implementation of training.

[0092] After configuration, the user can start the optimized training protocol and specify the total number of iterations (rounds) for the training process.

[0093] (1) Protocol execution process.

[0094] 1) Based on the parameters in the configuration unit, initialize an instance of a high-fidelity hydrostatic bearing simulation environment and an instance of a deep reinforcement learning agent.

[0095] 2) Enter the iterative loop of sequential decision-making. In each iteration, the agent generates a decision action (i.e., a design modification plan) based on its internal policy network and the current environmental state (i.e., the current design scheme).

[0096] 3) The simulation environment receives this action, updates its internal hydrostatic bearing geometric model, and calls its physical solver to calculate the complete performance index corresponding to the new design scheme.

[0097] 4) The simulation environment combines multi-dimensional performance indicators into a single reward value based on preset weights, and returns it to the agent along with the next state.

[0098] 5) The agent stores the experience of this interaction (current state, action, reward, next state) in its experience replay memory and samples data from it to update the parameters of its policy network and value network.

[0099] 6) This iterative cycle continues until the preset total number of iterations is reached.

[0100] (2) Process output.

[0101] 1) After the training protocol is completed, the final trained agent parameters are saved to the specified storage location.

[0102] 2) Simultaneously, based on the key performance data recorded during training, charts describing the evolution trends of various performance indicators as the optimization process progress are generated, providing users with an intuitive understanding of the optimization process, such as... Figure 2As shown in (a)-(e).

[0103] 3. Design evaluation and system-level dynamics verification.

[0104] To verify the effectiveness of the optimized design at the system level, users can initiate a dynamics analysis protocol and specify a pre-trained agent as input.

[0105] (1) Protocol execution process.

[0106] 1) Load the specified agent parameters.

[0107] 2) First, a deterministic evaluation process is executed: the loaded agent makes a complete decision to obtain the optimal design scheme that the agent can produce.

[0108] 3) Extract the core dynamic characteristics from the performance data of the optimized design scheme, namely the eight dynamic coefficient matrices representing the cross-coupling stiffness and damping.

[0109] 4) Input the extracted stiffness and damping matrices into the preset finite element model of the rotor system, which includes mass, geometry, and material properties.

[0110] 5) The dynamic performance indicators of the bearing are used as supporting conditions and integrated into the finite element model of the rotor system to construct a complete global dynamic equation of motion for the rotor-bearing coupled system.

[0111] 6) By solving the eigenvalues ​​of the equation, the natural frequencies and corresponding damping ratios of the coupled system at a series of rotational speeds are calculated.

[0112] (2) Process output.

[0113] 1) Generate visualized performance graphs for this optimized design scheme, such as orifice layout diagrams and oil film pressure distribution diagrams, e.g. Figure 3 and Figure 4 As shown.

[0114] 2) More importantly, the system plots the results of the dynamic analysis into a Campbell diagram. For example... Figure 5 As shown in the figure, this diagram intuitively illustrates the relationship between the system's natural frequency and rotational speed, enabling designers to clearly identify potential resonance zones and instability risks, thereby completing a closed loop from optimizing individual components to verifying the performance of the entire system.

[0115] 4. Generation of Pareto optimal frontier.

[0116] To provide designers with comprehensive decision-making support, this application also provides a protocol for generating multi-objective Pareto optimal frontiers.

[0117] (1) Protocol execution process.

[0118] 1) The user initiates the Pareto optimization protocol.

[0119] 2) Preset a series of weight combinations for multi-objective reward functions. Each combination represents a different design preference (for example, one set of weights favors high stiffness, while another set favors low power).

[0120] 3) Traverse these weight combinations. For each weight combination, execute the aforementioned optimization training protocol completely once.

[0121] 4) After all the independent optimization training processes are completed, summarize the optimization design schemes produced by all processes.

[0122] 5) Finally, a set of non-dominated solutions is selected from these aggregated solutions using a non-dominated sorting algorithm. This set of solutions constitutes the Pareto optimal front.

[0123] (2) Process output.

[0124] This ultimately generates a Pareto front plot. (See diagram below.) Figure 6 As shown in (a)-(d) in the figure, this diagram visualizes all non-dominated solutions in a two-dimensional or three-dimensional performance space (e.g., with "stiffness" as one axis and "power" as another axis). Figure 6 (a)-(d) in the paper reveal the inherent and unavoidable constraints and trade-offs between different performance indicators, providing a global and quantitative scientific basis for engineers to make final design decisions based on the constraints of specific application scenarios.

[0125] The above embodiments fully demonstrate how this application can combine deep reinforcement learning, high-fidelity simulation and system dynamics analysis to achieve an automated and intelligent closed loop from intelligent optimization to engineering verification for a complex engineering design problem.

[0126] Based on the same inventive concept, this application also provides a system for optimizing the arrangement and size of throttle holes in hydrostatic bearings based on deep reinforcement learning. The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the system for optimizing the arrangement and size of throttle holes in hydrostatic bearings based on deep reinforcement learning provided below can be found in the limitations of the method for optimizing the arrangement and size of throttle holes in hydrostatic bearings based on deep reinforcement learning described above, and will not be repeated here.

[0127] In one exemplary embodiment, a hydrostatic bearing throttling orifice arrangement and size optimization system based on deep reinforcement learning is provided, including the following modules.

[0128] A deep reinforcement learning agent construction module is used to construct a deep reinforcement learning agent. The deep reinforcement learning agent includes a policy network and a value network. The policy network is used to output a mixed action based on the current state in the state space, and the value network is used to evaluate the value of the mixed action. The state space is a design scheme of the throttling orifice of a hydrostatic bearing, including the number of throttling orifices, the position parameters of each throttling orifice, and the geometric dimension parameters.

[0129] The hydrostatic bearing simulation environment construction module is used to construct a hydrostatic bearing simulation environment, and receive the mixed actions output by the deep reinforcement learning agent based on the hydrostatic bearing simulation environment, calculate multi-dimensional performance indicators, and calculate the reward value based on the multi-dimensional performance indicators using a multi-objective reward function; the multi-dimensional performance indicators include static performance indicators and dynamic performance indicators.

[0130] The interaction module is used for sequential interaction between the deep reinforcement learning agent and the hydrostatic bearing simulation environment. It updates the parameters of the policy network and the value network through an experience replay mechanism until a preset iteration termination condition is met, thereby obtaining the trained deep reinforcement learning agent.

[0131] The optimization design scheme generation module is used to generate optimized design schemes for the throttling orifice of hydrostatic bearings using a trained deep reinforcement learning agent.

[0132] The optimization module is used to optimize the arrangement and size of the throttling orifice based on the optimized design scheme.

[0133] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments. The computer device can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, memory, and I / O are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device stores data to be processed. The I / O interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with an external terminal via a network connection. When the computer program is executed by the processor, it implements a method for optimizing the arrangement and size of throttle holes in hydrostatic bearings based on deep reinforcement learning.

[0134] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0135] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0136] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning, characterized in that, include: Constructing deep reinforcement learning intelligent agents; The deep reinforcement learning agent includes a policy network and a value network. The policy network is used to output a hybrid action based on the current state in the state space, and the value network is used to evaluate the value of the hybrid action. The state space is a design scheme of the throttle orifice of a hydrostatic bearing, including the number of throttle orifices, the position parameters of each throttle orifice, and the geometric dimension parameters. A hydrostatic bearing simulation environment is constructed, and the mixed actions output by the deep reinforcement learning agent are received based on the hydrostatic bearing simulation environment. Multidimensional performance indicators are calculated, and a multi-objective reward function is used to calculate the reward value based on the multidimensional performance indicators. The multidimensional performance indicators include static performance indicators and dynamic performance indicators; The deep reinforcement learning agent sequentially interacts with the hydrostatic bearing simulation environment, and updates the parameters of the policy network and the value network through an experience replay mechanism until the preset iteration termination condition is met, thus obtaining the trained deep reinforcement learning agent. An optimized design scheme for the throttling orifice of a hydrostatic bearing is generated using a trained deep reinforcement learning agent. The arrangement and size of the throttling orifice are optimized based on the aforementioned optimized design scheme.

2. The method for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning according to claim 1, characterized in that, The hybrid action includes discrete actions and continuous actions; the discrete actions are used to add, delete, or modify the orifice, and the continuous actions are used to determine the parameters of the discrete actions.

3. The method for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning according to claim 1, characterized in that, Based on the hydrostatic bearing simulation environment, the system receives the mixed actions output by the deep reinforcement learning agent and calculates multi-dimensional performance indicators, specifically including: The hydrostatic bearing geometric model in the hydrostatic bearing simulation environment is updated based on the hybrid actions output by the deep reinforcement learning agent. Based on the updated hydrostatic bearing geometric model, the pressure distribution is calculated using the generalized Reynolds equation that includes the squeezing film effect; Multidimensional performance indicators are calculated based on the pressure distribution.

4. The method for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning according to claim 1, characterized in that, The static performance indicators include static bearing capacity, static stiffness, pumping power, and friction power, while the dynamic performance indicators include stiffness matrix and damping matrix.

5. The method for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning according to claim 1, characterized in that, The expression for the multi-objective reward function is: in, Let be the reward value for the t-th iteration. , , , Static stiffness Static bearing capacity Pumping power Friction power Dynamic stability index The weighting coefficients, (•) is the normalization function.

6. The method for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning according to claim 1, characterized in that, After generating an optimized design scheme for the throttling orifice of a hydrostatic bearing using a trained deep reinforcement learning agent, the following steps are also included: Dynamic analysis and Pareto front generation are performed on the optimized design scheme.

7. The method for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning according to claim 6, characterized in that, The process of the kinetic analysis includes: Extract the dynamic performance indicators corresponding to the optimized design scheme; The corresponding dynamic performance indicators are integrated into the preset rotor system finite element model to construct the global mass matrix, gyro effect matrix, system stiffness moment and system damping matrix of the rotor-bearing coupled system; The eigenvalues ​​of the rotor-bearing coupled system are solved based on the global mass matrix, the gyro effect matrix, the system stiffness moment, and the system damping matrix. Based on the characteristic values, a Campbell diagram is generated within a preset rotational speed range.

8. The method for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning according to claim 6, characterized in that, The process of generating the Pareto front includes: By adjusting the weights of the multi-objective reward function, multiple optimized design schemes are obtained; A non-dominated sorting algorithm is used to filter all optimal design schemes to obtain a set of non-dominated solutions; The non-dominated solution set is visualized in the performance index space to generate the Pareto optimal frontier.

9. A system for optimizing the arrangement and size of throttling orifices in hydrostatic bearings based on deep reinforcement learning, characterized in that, include: The Deep Reinforcement Learning Agent Construction Module is used to construct deep reinforcement learning agents. The deep reinforcement learning agent includes a policy network and a value network. The policy network is used to output a hybrid action based on the current state in the state space, and the value network is used to evaluate the value of the hybrid action. The state space is a design scheme of the throttle orifice of the hydrostatic bearing, including the number of throttle orifices, the position parameters of each throttle orifice, and the geometric dimension parameters. A hydrostatic bearing simulation environment construction module is used to construct a hydrostatic bearing simulation environment, and receive the mixed actions output by the deep reinforcement learning agent based on the hydrostatic bearing simulation environment, calculate multi-dimensional performance indicators, and calculate the reward value based on the multi-dimensional performance indicators using a multi-objective reward function; the multi-dimensional performance indicators include static performance indicators and dynamic performance indicators; An interaction module is used for sequential interaction between the deep reinforcement learning agent and the hydrostatic bearing simulation environment. The parameters of the policy network and the value network are updated through an experience replay mechanism until a preset iteration termination condition is met, thereby obtaining the trained deep reinforcement learning agent. An optimized design scheme generation module is used to generate optimized design schemes for hydrostatic bearing throttling orifices using a trained deep reinforcement learning agent. The optimization module is used to optimize the arrangement and size of the throttling orifice based on the optimized design scheme.

10. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method for optimizing the arrangement and size of the hydrostatic bearing throttle hole based on deep reinforcement learning as described in any one of claims 1-8.

Images