Space truss vibration control method and system based on deep reinforcement learning

Abstract

The present application relates to the technical field of mechanical control, and particularly relates to a space truss vibration control method and system based on deep reinforcement learning, which utilizes ATMD to control the truss, and collects the displacement and velocity of the truss nodes and the displacement and velocity output by the ATMD in the control process; the four data currently collected are used to form a state vector, and the displacement and velocity of the truss nodes and the control force output by the ATMD are used to form a reward function, reinforcement learning is performed, and the control force generated by the ATMD at the next moment is output. The present application combines deep reinforcement learning and ATMD control technology, and proposes an innovative multi-modal vibration control method, which effectively solves the limitations of existing methods in multi-modal vibration control, system adaptability, real-time optimization and energy efficiency optimization. Through the technical scheme, efficient vibration suppression can be realized in a dynamic complex environment, and the robustness and high efficiency of the system are ensured.

Description

Technical Field

[0001] This invention belongs to the field of mechanical control technology, and in particular relates to a method and system for controlling the vibration of a space truss based on deep reinforcement learning. Background Technology

[0002] In many high-precision engineering applications, particularly in space optical structures and precision instrument systems, vibration control remains a critical technical challenge. Large space truss structures are crucial connecting components for the on-orbit assembly of large spacecraft; currently, space solar power station battery antenna arrays and on-orbit assembled space telescopes all require large space truss structures. Once a large space truss structure is disturbed, and without air damping, vibrations on the truss cannot decay quickly enough, affecting the normal operation of the payloads on the truss. Furthermore, the complex space environment, with its alternating high and low temperatures and the presence of space debris, can easily induce prolonged vibrations. These structural vibrations not only affect the structure's safety but also directly impact the system's performance and stability. In space telescopes, even minute vibrations can affect the accuracy of optical imaging, leading to blurred and distorted images, thus affecting the effectiveness of scientific experiments.

[0003] In space truss structures, the resonance of certain natural modes can significantly amplify vibrations, leading to local or overall structural instability and even affecting the overall system's function. In engineering practice, space truss structures are often complex multi-degree-of-freedom systems with multiple natural frequencies and modes, which are usually coupled and influence each other. Therefore, control requires not only controlling individual modes but also considering the interactions between multiple modes. Furthermore, the vibration sources in practical applications are often time-varying, and the system response often exhibits nonlinear characteristics.

[0004] Currently, vibration isolation methods for space truss structures mainly include passive control methods such as dampers and isolation systems, and active control methods such as active mass tuned dampers (ATMD), PID control, and LQR (linear quadratic regulator) control. Passive control methods primarily absorb vibration energy by adding damping elements; however, this method is often ineffective in responding to external changes or dynamic disturbances. Active control methods can adjust the control force in real time according to the system state to achieve precise vibration control. However, traditional active control methods typically rely on accurate system models, and in multi-degree-of-freedom or nonlinear systems, the control effect is often difficult to achieve optimally.

[0005] Existing active vibration damping (ATMD) systems are widely used in various vibration control tasks as active vibration control systems to suppress modal vibrations. ATMDs typically control specific modes of a structure through a coupled system of a mass block and a damper, with the controller reducing vibrations by outputting a control force. PID control is widely used in ATMDs. PID control optimizes the control system's response by adjusting proportional, integral, and derivative parameters. However, the effectiveness of PID control may fail or produce unsatisfactory control results when faced with time-varying disturbances and system nonlinearities. LQR control calculates the optimal control force by optimizing the control objective and cost function. In simple systems, LQR control can effectively achieve modal suppression. However, this method also relies on an accurate system model, and its effectiveness is significantly reduced when the system exhibits nonlinear behavior. Summary of the Invention

[0006] In view of this, the present invention aims to provide a space truss vibration control method and system based on deep reinforcement learning. By combining deep reinforcement learning with multi-ATMD control, it effectively solves the limitations of existing methods in multimodal vibration control, system adaptability, real-time optimization and energy efficiency optimization. It can achieve efficient vibration suppression in dynamic and complex environments and ensure the robustness and high efficiency of the system.

[0007] To achieve the above objectives, the technical solution created by this invention is implemented as follows: A method for controlling the vibration of a space truss based on deep reinforcement learning includes: S1: Use ATMD to control the truss and collect the displacement and velocity at the truss nodes during the control process, as well as the displacement and velocity output by ATMD; S2: The state vector is composed of the four data collected in step S1. The reward function is composed of the displacement and velocity at the truss node and the control force output by ATMD. Reinforcement learning is performed to output the control force generated by ATMD in the next moment.

[0008] Furthermore, the reward function in step S2 is: ; Where R(t) represents the reward function at the t-th learning step, x i (t) represents the displacement at the truss node during the t-th learning step, v i (t) represents the velocity at the truss node during the t-th learning step, M represents the total number of truss nodes, and F ATMD,j α represents the control force exerted by the j-th ATMD, N represents the total number of ATMDs, and α and β represent the weighting coefficients.

[0009] Furthermore, the reinforcement learning process in step S2 includes: the actor network generating the actual control force of ATMD at the current moment based on the state vector of the previous moment; the control force at the current moment controls the truss movement, forming the actual state vector at the current moment; the commenting network calculating the actual value of the actual control force at the current moment based on the actual state vector at the current moment, and obtaining the reward value used to measure the performance of the truss through the reward function; the target actor network generating the target control force of ATMD at the current moment based on the state vector of the previous moment, and generating the target state vector at the current moment based on the target control force; the target commenting network calculating the target value of the target control force at the current moment based on the target state vector at the current moment and the reward value; calculating the loss between the actual value and the target value, using the loss to guide the actor network to adjust the strategy, and storing the actual control force, actual state vector, reward value, and actual state vector at the current moment in the experience cache.

[0010] Furthermore, a soft update mechanism is used to update the target actor network and the target comment network.

[0011] Furthermore, Ornstein-Uhlenbeck noise was used to facilitate exploration.

[0012] A space truss vibration control system based on deep reinforcement learning includes: multiple ATMDs randomly arranged on the space truss for outputting control forces to the space truss; sensors for sensing the motion of the space truss nodes and each ATMD; and a control unit for receiving and, based on the data collected by the sensors, using the space truss vibration control method based on deep reinforcement learning provided by this invention, outputting control signals to each ATMD, so that the multiple ATMDs control the overall motion of the space truss according to the control signals.

[0013] Furthermore, each ATMD includes: a mass block for balancing the weight distribution of the entire ATMD; a mounting interface for fixed connection to the connecting rod of the space truss; a force actuator, with its mounting end set on the mass block and its output end connected to the mounting interface, for outputting control force to the connecting rod through the mounting interface; a damper, with its two ends connected to the mass block and the mounting interface respectively, for constraining the movement of the mass block when the actuator outputs control force; and an ATMD sensor, set on the mounting interface, for sensing the movement of each ATMD and transmitting the sensed information to the control unit.

[0014] Furthermore, each ATMD also includes a spring, with its two ends connected to the mass block and the mounting interface, respectively.

[0015] Compared with the prior art, the present invention can achieve the following beneficial effects: This invention presents a space truss vibration control method and system based on deep reinforcement learning. Leveraging deep reinforcement learning, the method automatically optimizes control strategies even without a precise model, and possesses online learning and adaptive adjustment capabilities. It adapts to dynamic environments and external disturbances, and can simultaneously suppress vibrations of multiple intrinsic modes. In contrast, traditional methods typically optimize only a single mode, failing to effectively address complex multimodal vibration problems. This invention not only improves the accuracy of vibration control but also effectively reduces interference and coupling effects between multiple modes, enhancing overall control performance. Furthermore, by introducing an energy consumption term into the reward function, this invention minimizes energy consumption while suppressing vibration, improving system energy efficiency and overcoming the energy waste problem of traditional methods, making it more suitable for space applications. Attached Figure Description

[0016] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 A flowchart illustrating the space truss vibration control method based on deep reinforcement learning as described in an embodiment of the present invention; Figure 2 A schematic diagram of the structure of the space truss vibration control system based on deep reinforcement learning as described in an embodiment of the present invention; Figure 3 A schematic diagram of the structure of each ATMD described in the embodiments of the present invention.

[0017] Explanation of reference numerals in the attached figures: 1. ATMD; 2. Sensor; 3. Control unit; 4. Secondary mirror; 5. Primary mirror; 6. Connecting rod; 7. Mass block; 8. Mounting interface; 9. Force actuator; 10. Damping; 11. ATMD sensor; 12. Spring. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0019] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0020] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] like Figure 1 As shown in the embodiment of the present invention, the space truss vibration control method based on deep reinforcement learning includes: S1: Use ATMD to control the truss, and collect the displacement and velocity at the truss nodes during the control process, as well as the displacement and velocity output by ATMD.

[0023] The truss structure consists of multiple nodes and connecting rods, and the movement between the nodes is mechanically constrained by the connecting rods. In this embodiment of the invention, multiple connecting rods are randomly selected from all the connecting rods of the truss, and ATMDs (Automatic Traction Devices) are installed at these selected connecting rods. The ATMDs output control forces to these connecting rods, thereby controlling the overall movement of the truss. Simultaneously, this embodiment of the invention uses sensors to centrally collect the motion information of all nodes. To describe the dynamic response of the truss structure, this embodiment of the invention uses the following dynamic model: Ma(t) + Cv(t) + Kx(t) = F e (t)+F ATMD (t); Where a(t), v(t), and x(t) represent the acceleration, velocity, and displacement at the truss node at time t, respectively; M, C, and K represent the mass matrix, damping matrix, and stiffness matrix of the truss, respectively; and F... e (t) represents the external disturbance force F experienced by the truss at time t. ATMD (t) represents the control force that ATMD outputs to the truss at time t.

[0024] Each ATMD is considered a single-degree-of-freedom system. Its dynamic model is as follows: ma ATMD (t)+cv ATMD (t)+kx ATMD (t)=F ATMD (t); Among them, a ATMD (t), v ATMD (t) and x ATMD (t) represents the acceleration, velocity and displacement output by each ATMD at time t, respectively, and m, c and k represent the mass matrix, damping matrix and stiffness matrix of each ATMD, respectively.

[0025] S2: The state vector is composed of the four data collected in step S1. The reward function is composed of the displacement and velocity at the truss node and the control force output by ATMD. Reinforcement learning is performed to output the control force generated by ATMD in the next moment.

[0026] In this embodiment of the invention, the state vector s(t) can be represented as s(t)=[x(t),v(t),x ATMD (t),v ATMD (t)]. The control force output by each ATMD is a continuous variable; therefore, the actions performed by all ATMDs are N-dimensionally continuous, where N is the number of ATMDs. This embodiment of the invention assumes that the total number of truss nodes is P, and the dimension of the state space composed of state vectors is (4P+2N).

[0027] In some embodiments, the reward function is: ; Where R(t) represents the reward function at the t-th learning step, x i (t) represents the displacement at the truss node during the t-th learning step, v i (t) represents the velocity at the truss node during the t-th learning step, F ATMD,j α represents the control force applied by the j-th ATMD, α represents the vibration suppression weight, and β represents the energy consumption weight. The two weight coefficients are used to balance vibration suppression and energy consumption.

[0028] Correspondingly, in the reward function R(t), This is the vibration penalty item. This refers to the energy penalty term. The vibration suppression coefficient α and energy consumption weight β essentially assign priorities to two competing objectives within the reward function. Specifically: The vibration suppression weight α is used to amplify the influence of the vibration penalty term. The larger the value of the vibration suppression weight α, the more the reinforcement learning strategy will tend to reduce the displacement and velocity of the truss at all costs, and place vibration suppression in a higher priority. The energy consumption weight β is used to amplify the influence of the energy penalty term. The larger the value of the energy consumption weight β, the more the reinforcement learning policy will tend to conserve control and place energy efficiency in a higher priority, but may tolerate greater residual vibrations.

[0029] In this embodiment of the invention, the adjustment strategies for the vibration suppression coefficient α and the energy consumption weight β are as follows: Set the initial vibration suppression coefficient α and energy consumption weight β; Reinforcement learning is initiated, and the changing trend of the reward component is observed during the learning process: if the vibration penalty term decreases very slowly, the vibration inhibition coefficient α is increased; if the control force penalty term decreases very slowly or the system always outputs a very small force, the energy consumption weight β is increased. The trained reinforcement learning strategy is validated in a test environment to evaluate key metrics (such as vibration attenuation rate, steady-state error, maximum control force, and total energy consumption). If the preset requirements are not met, reinforcement learning is repeated and the vibration suppression coefficient α and energy consumption weight β are adjusted until the trained reinforcement learning strategy meets the preset evaluation criteria.

[0030] This invention employs the Deep Deterministic Policy Gradient (DDPG) algorithm for reinforcement learning. This learning method is a reinforcement learning algorithm based on the actor-critic architecture and is suitable for problems with continuous action spaces. Further elaborating, the DDPG algorithm comprises four neural networks: an actor network, a critic network, a target actor network, and a target critic network.

[0031] The input to the actor network is the state vector s(t), and the output is the deterministic control force vector a. out (t)=[F ATMD,1 ,F ATMD,2 ,...,F ATMD,j ,...,F ATMD,N In this embodiment of the invention, a multilayer fully connected network (MLP) is preferably used as the actor network. Specifically, the state vector s(t) is processed by a fully connected layer with 256 neurons and then ReLU activated. The activated features are then input into a fully connected layer with 256 neurons for further processing. The processed features are then activated by ReLU and tanh, and the output control force vector a is obtained. out (t). The data processing procedure for the actor network can be represented by the following formula: aout (t)=μ(s(t)|θ μ ); Where μ represents the actor network, θ μ This represents the learning parameters of neurons in the actor network.

[0032] The input to the comment network is the state vector s(t) and the control force vector a. out The concatenation of (t) outputs a scalar Q value, which represents the control force vector a executed under the state vector s(t). out The long-term expected return of (t). In this embodiment of the invention, a multi-layer fully connected network with the same structure as the actor network is preferably used. The data processing process of the comment network can be represented by the following formula: Q out (t)=Q((s(t),a out (t))|θ Q ); Among them, Q out (t) represents the scalar value of the comment network output at time t, Q represents the comment network, and θ Q This represents the learning parameters of neurons in the comment network.

[0033] The target networks (i.e., the target actor network and the target commentary network) are delayed copies of the main networks (i.e., the corresponding actor network and commentary network) used for stable learning during training. In this embodiment of the invention, the specific network structures of the target actor network and the target commentary network are exactly the same as those of the aforementioned actor network and commentary network, but they have independent corresponding learning parameters θ. μ ' and θ Q '.

[0034] Specifically, in some embodiments, the reinforcement learning process includes: The actor network generates the actual control force of ATMD at the current moment based on the state vector of the previous moment; The control force at the current moment controls the movement of the truss, forming the actual state vector at the current moment; The comment network calculates the actual value of the actual control force at the current moment based on the actual state vector at the current moment, and obtains the reward value used to measure the performance of the truss through the reward function; The target actor network generates the target control force of ATMD at the current moment based on the state vector of the previous moment, and generates the target state vector at the current moment based on the target control force; The target evaluation network calculates the target value of the target control force at the current moment based on the target state vector at the current moment and the reward value; Calculate the loss between the actual value and the target value, use the loss to guide the actor network to adjust its strategy, and store the actual control power, actual state vector, reward value of the previous moment, and actual state vector of the current moment in the experience cache.

[0035] In this embodiment of the invention, the reinforcement learning process further includes: The target scalar Q-value of the target review network is obtained by the following formula: y(t)=R(t)+γQ'((s(t+1),μ'(t+1))|θ Q '); Where y(t) represents the target scalar Q value output by the target comment network Q' at time t, μ' represents the target actor network, and γ represents the discount factor used to weigh the importance of current rewards versus future rewards, 0≤γ<1.

[0036] The loss function L(θ) of the comment network Q )for: L(θ Q )=E[(Q((s,a out )|θ Q )-y) 2 ]; Here, E represents the expected value calculation. The comment network parameters θ are updated by minimizing the mean squared error between the predicted Q value and the target Q value y. Q .

[0037] The expected gradient of the actor network, E, is calculated on the data sampled from the experience replay pool using the following formula: ; in, This represents the gradient of the actor network. Indicates the control force vector a out Perform gradient calculation on the comment network. This indicates that the gradient calculation is performed on the actor network using the comment network.

[0038] In some embodiments, a soft update mechanism is used to update the target actor network and the target comment network. Specifically, in this embodiment of the invention: ; Wherein, τ represents the update rate, which is set to 0.001 in this embodiment of the invention. This embodiment of the invention slowly updates the parameters of the main network to the target network, rather than directly copying them. This significantly improves the stability of the learning process through a very small update rate τ.

[0039] In some embodiments, Ornstein-Uhlenbeck noise is used to facilitate exploration. Reinforcement learning is essentially an "exploration-exploitation dilemma" process, meaning that reinforcement learning requires trying different actions to understand the environment (i.e., the exploration process) while simultaneously utilizing known experience to maximize rewards (i.e., the exploitation process). Utilizing only the existing best policy may lead to local optima; overexploration results in inefficient learning. Designing effective exploration mechanisms in continuous action spaces (such as the continuous control force output by ATMD in this invention) is more complex than in discrete action spaces. The Ornstein-Uhlenbeck process, being a stochastic process, generates noise with two key characteristics that make it particularly suitable for physical control tasks (such as the vibration control of this invention). Specifically, Ornstein-Uhlenbeck noise fluctuates around a long-term mean (usually set to 0) and has a "pull" that makes it tend to revert to the mean. This means that the output action (i.e. the output control force) will not drift indefinitely to an out-of-range range due to the accumulation of noise, increasing the stability of the exploration. At the same time, the Ornstein-Uhlenbeck noise values ​​of two adjacent moments are correlated, rather than being completely independent Gaussian white noise. This makes the generated exploration actions smooth and inertial in the time series, which is closer to many physical systems (such as motors and actuators).

[0040] In this embodiment of the invention, the process of Ornstein-Uhlenbeck noise participating in reinforcement learning specifically includes: the actor network outputs a basic deterministic action (i.e., basic deterministic control force) based on the current state; during the Ornstein-Uhlenbeck process, a noise is randomly sampled and added to the basic deterministic action to obtain an exploratory action for environmental interaction; this exploratory action is executed to obtain a reward and the next state from the environment; the actual control force, actual state vector, reward value from the previous time step, and the actual state vector from the current time step are stored in an experience cache for subsequent network updates. The participation of Ornstein-Uhlenbeck noise in training can gradually reduce the amplitude of the noise. Through the above training methods and multiple iterations, the system will gradually converge to the optimal strategy, enabling ATMD to minimize vibration and maximize energy efficiency under multimodal vibration, thereby reducing the vibration of the truss structure and optimizing the control of energy consumption.

[0041] This invention also provides a space truss vibration control system based on deep reinforcement learning, such as... Figure 2As shown, the system includes multiple ATMD1s, sensors 2, and a control unit 3. The multiple ATMD1s are randomly positioned on the space truss to output control forces to the space truss. Sensors 2 are used to sense the movement of the space truss nodes and each ATMD1. The control unit 3 receives the data collected by sensors 2 and, using the deep reinforcement learning-based space truss vibration control method provided in this invention, outputs control signals to each ATMD1, enabling the multiple ATMD1s to control the overall movement of the space truss according to the control signals. This invention integrates multiple ATMD1s with a deep reinforcement learning model to form an end-to-end control system. It acquires the state of the truss structure and ATMD1s through real-time sensor data, inputs the state information into the trained deep reinforcement learning model, outputs a control strategy, and applies it to the ATMD1s for real-time vibration control.

[0042] This embodiment of the invention takes a space truss equipped with a three-mirror astigmatism correction system as an example. Specifically, the space truss adopts a triangular structure. The secondary mirror 4 in the three-mirror astigmatism correction system is installed at a vertex of the triangular structure. The three primary mirrors 5 in the three-mirror astigmatism correction system are arranged on opposite sides of the secondary mirror 4 in the triangular structure, and the back of each primary mirror 5 is installed on the intersecting space truss. At this time, the distribution of multiple ATMD1s is as follows: Figure 2 As shown, one of the ATMD1 is installed on the connecting rod 6 connected to the secondary mirror 4, and this ATMD1 is close to the secondary mirror 4 to control the movement of the secondary mirror 4. The remaining ATMD1s are evenly distributed on the space truss connected to the primary mirror 5 to control the movement of the three primary mirrors 5.

[0043] In some embodiments, the structure of each ATMD1 is as follows: Figure 3 As shown, it includes a mass block 7, a mounting interface 8, a force actuator 9, a damper 10, and an ATMD sensor 11. The mass block 7 is used to balance the weight distribution of the entire ATMD1; the mounting interface 8 is fixedly connected to the connecting rod 6 of the space truss; the mounting end of the force actuator 9 is located on the mass block 7, and the output end of the force actuator 9 is connected to the mounting interface 8, used to output control force to the connecting rod 6 through the mounting interface 8; the two ends of the damper 10 are connected to the mass block 7 and the mounting interface 8 respectively, used to constrain the movement of the mass block 7 when the actuator outputs control force; the ATMD sensor 11 is located on the mounting interface 8, used to sense the movement of each ATMD1 and transmit the sensed information to the control unit 3.

[0044] In some embodiments, each ATMD1 further includes a spring 12, the two ends of which are connected to the mass block 7 and the mounting interface 8, respectively. The spring 12 mainly serves as a guide, ensuring that the control force is directed axially.

[0045] This invention optimizes the vibration control strategy of a multi-ATMD system using deep reinforcement learning (DDPG). By combining a multi-degree-of-freedom dynamic model of the truss structure with a single-degree-of-freedom vibration model of the ATMD, it effectively achieves multi-modal vibration control and energy efficiency optimization. Through steps such as state-space modeling, reward function design, and training process implementation, this invention can optimize the control strategy in real time in a dynamic environment, providing a highly efficient, adaptive, and energy-efficient vibration control solution.

[0046] A space truss is a complex elastic body with multiple natural frequencies (such as the first-order bending frequency, the second-order torsional frequency, etc.). When subjected to external disturbances (such as micrometeorite impacts or thermal deformation), the structure does not move in the same phase as a whole, but rather excites a variety of vibration modes with different spatial forms. These modes are called modes. When multiple such vibration modes are excited and coupled together in a space truss, multimodal vibration is formed in the space truss (for example, a disturbance may simultaneously excite a low-order overall bending (mode 1) and a high-order local vibration (modes 2, 3, etc.)).

[0047] Different modes have different frequencies, spatial distributions, and energies. Suppressing only one vibration mode might unintentionally excite other modes or be ineffective against them. Therefore, an ideal control strategy needs to sense, decouple, and simultaneously suppress multiple modes of vibration. For controller design feasibility, traditional methods often require significant order reduction of high-order, continuous systems or design for a single dominant mode (such as the fundamental frequency). This results in the controller's inability to effectively sense and suppress other unmodeled modes. Furthermore, the parameters (gain) of PID or LQR controllers are fixed, and both are designed based on linear, time-invariant system models, making it difficult to adapt to inter-modal coupling effects and time-varying disturbances. When multiple modes are excited, the fixed control law cannot dynamically adjust its response strategy, potentially leading to decreased control effectiveness or even instability. The innovation of this invention lies in: In terms of control paradigm, the approach shifts from "design based on accurate physical models" to "end-to-end self-learning based on interactive data," abandoning the traditional method's reliance on system matrices. This allows deep reinforcement learning agents to directly interact with truss systems containing multiple ATMDs and automatically learn the optimal control strategy from high-dimensional state data, thereby enabling them to adaptively handle unmodeled dynamics, nonlinearities, and time-varying disturbances. In terms of problem definition, the complex task of "multimodal suppression" is innovatively transformed into a single optimization objective of "minimizing global vibration energy." By using a reward function to simultaneously penalize both the total vibration energy (sum of squared displacements and velocities) of all nodes and the total control force consumption of all actuators, the driving strategy automatically learns to coordinate and schedule multiple ATMDs to suppress overall vibration in the most economical way, rather than performing independent and fixed control on individual modes. These two principles complement each other, jointly achieving efficient and adaptive suppression of complex multimodal vibrations.

[0048] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0049] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A spatial truss vibration control method based on deep reinforcement learning, characterized in that, include: S1: Use ATMD to control the truss and collect the displacement and velocity at the truss nodes during the control process, as well as the displacement and velocity output by ATMD; S2: The state vector is formed by the four data collected in step S1. The reward function is formed by the displacement and velocity at the truss node and the control force output by ATMD. Reinforcement learning is performed to output the control force generated by ATMD in the next moment. The reward function in step S2 is: ； Where R(t) represents the reward function at the t-th learning step, x i (t) represents the displacement at the truss node during the t-th learning step, v i (t) represents the velocity at the truss node during the t-th learning step, M represents the total number of truss nodes, and F ATMD,j α represents the control force exerted by the j-th ATMD, N represents the total number of ATMDs, and α and β represent the weighting coefficients. The reinforcement learning process in step S2 includes: The actor network generates the actual control force of ATMD at the current moment based on the state vector of the previous moment; The control force at the current moment controls the movement of the truss, forming the actual state vector at the current moment; The comment network calculates the actual value of the actual control force at the current moment based on the actual state vector at the current moment, and obtains the reward value used to measure the performance of the truss through the reward function; The target actor network generates the target control force of ATMD at the current moment based on the state vector of the previous moment, and generates the target state vector at the current moment based on the target control force; The target evaluation network calculates the target value of the target control force at the current moment based on the target state vector at the current moment and the reward value; Calculate the loss between the actual value and the target value, use the loss to guide the actor network to adjust its strategy, and store the actual control power, actual state vector, reward value of the previous moment, and actual state vector of the current moment in the experience cache.

2. The spatial truss vibration control method based on deep reinforcement learning according to claim 1, characterized in that, The target actor network and the target comment network are updated using a soft update mechanism.

3. The spatial truss vibration control method based on deep reinforcement learning according to claim 1, characterized in that, Using Ornstein-Uhlenbeck noise to facilitate exploration.

4. A space truss vibration control system based on deep reinforcement learning, characterized in that, include: Multiple ATMDs are randomly positioned on the space truss to output control forces to the space truss; Sensors are used to detect the movement at the nodes of the space truss and at each ATMD; The control unit is used to receive and, based on the data collected by the sensors, use the deep reinforcement learning-based space truss vibration control method as described in any one of claims 1 to 3 to output a control signal to each ATMD, so that the multiple ATMDs control the overall movement of the space truss according to the control signal.

5. The space truss vibration control system based on deep reinforcement learning according to claim 4, characterized in that, Each ATMD includes: Mass blocks are used to balance the gravity distribution across the entire ATMD; The mounting interface is fixedly connected to the connecting rod of the space truss. A force actuator is mounted on the mass block, and its output end is connected to the mounting interface to output control force to the connecting rod through the mounting interface. The damper, connected at both ends to the mass block and the mounting interface respectively, is used to constrain the movement of the mass block when the actuator outputs control force; The ATMD sensor, installed on the mounting interface, is used to sense the movement of each ATMD and transmit the sensed information to the control unit.

6. The space truss vibration control system based on deep reinforcement learning according to claim 5, characterized in that, Each ATMD also includes: The spring is connected at both ends to the mass block and the mounting interface, respectively.

Images