Intelligent suspension control method, medium and device considering suspension system time delay

Abstract

The present invention relates to the field of artificial intelligence technology, and more particularly to an intelligent suspension control method, medium, and device that considers suspension system time lag. The method includes pre-training two evaluation networks and a control strategy network using meta-learning to obtain two evaluation models and a control strategy model. The control strategy model determines the current control force output by the intelligent suspension based on the current state of the vehicle. The current control force is applied to the vehicle to obtain the vehicle's next state. The current reward is determined based on the current control force and the current state. The two evaluation models and the control strategy model are re-trained using multiple sets of data to obtain two final evaluation models and a final control strategy model. The intelligent suspension is operated and controlled using the two final evaluation models and the final control strategy model. The present invention incorporates a delay link to guide the intelligent agent to obtain a more robust control strategy in a time-lag environment.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to an intelligent suspension control method, medium and device taking into account the time lag of the suspension system. Background Art

[0002] With the continuous advancement and innovation of microprocessor, sensor, and actuator technologies, intelligent suspension systems have become a research hotspot in vehicle engineering. Intelligent suspension systems can adaptively adjust output force based on driver input and real-time road conditions by precisely controlling actuators. This significantly improves vehicle ride quality and performance, meeting the high standards of comfort and safety demanded by modern vehicles. However, because intelligent suspension systems are closed-loop systems comprising sensors, controllers, and actuators, inherent time delays in signal measurement and actuation are unavoidable. In most cases, this time delay may be negligible and have minimal impact on system performance. However, in certain scenarios, when time delays are comparable to or even longer than the control cycle, their impact becomes significant and requires careful consideration. Time delays can often weaken control effectiveness and may even lead to control system instability, posing a potential threat to vehicle safety.

[0003] While deep reinforcement learning has shown initial success in the field of suspension control, research on time lag remains insufficient. Notably, time lag has a particularly significant negative impact on actual control performance in sequential decision-making processes, such as those involving deep reinforcement learning. Therefore, exploring how to effectively address time lag within the deep reinforcement learning framework remains an emerging and pressing challenge.

[0004] Current research mainly focuses on the feasibility analysis of deep reinforcement learning technology in suspension control. It is still in a relatively superficial exploration stage and has not deeply considered the impact of time delay of intelligent suspension systems on control effects, nor has it proposed an effective solution to the time delay problem. Summary of the Invention

[0005] In view of this, the present invention aims to provide an intelligent suspension control method, medium and equipment that takes into account the time delay of the suspension system. The present invention incorporates a delay link, aiming to guide the intelligent agent to explore and obtain a more robust control strategy in a time-delay environment, thereby effectively suppressing the negative impact of uncertain delays on the performance of the intelligent suspension system, and providing strong support for the efficient and stable control of the intelligent suspension system.

[0006] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0007] An intelligent suspension control method considering suspension system time lag, comprising:

[0008] S1: Use meta-learning to pre-train the first evaluation network, the second evaluation network, and the control policy network to obtain the first evaluation model, the second evaluation model, and the control policy model, and initialize the experience pool;

[0009] S2: The control strategy model obtained in step S1 determines the current control force output by the intelligent suspension based on the current state of the vehicle; the current control force is applied to the vehicle to obtain the next state of the vehicle;

[0010] S3: Determine the current reward based on the current control power and the current state; save the current state, current control power, current reward, and next state as a set of samples in the experience pool;

[0011] S4: Repeat steps S2 to S3 multiple times, and save multiple groups of samples in the experience pool; use the multiple groups of samples to train the first evaluation model, the second evaluation model, and the control strategy model obtained in step S1 again to obtain the final first evaluation model, the final second evaluation model, and the final control strategy model;

[0012] S5: Run the intelligent suspension and use the final first evaluation model, the final second evaluation model, and the final control strategy model obtained in step S4 to control the intelligent suspension.

[0013] Furthermore, step S1 includes: initializing the first evaluation network, the second evaluation network, the control strategy network and the experience pool; constructing a meta-training set, and performing meta-learning on the initialized first evaluation network, the second evaluation network and the control strategy network according to the meta-training set to obtain the corresponding first evaluation model, the second evaluation model and the control strategy model.

[0014] Furthermore, in step S2, the process of applying the control force at the current moment to the vehicle includes: hard constraints on the control force output by the control strategy model to obtain the control force at the current moment; setting the low speed range, medium speed range and high speed range of the vehicle; setting the low acceleration range, medium acceleration range and high acceleration range of the vehicle; setting the low control force change range, medium control force change range and high control force change range corresponding to the control force output by the control strategy model; measuring the current speed and current acceleration of the vehicle, and calculating the control force change rate between the current moment control force and the previous moment control force; comparing the current speed with the low speed range, medium speed range and high speed range, comparing the current acceleration with the low acceleration range, medium acceleration range and high acceleration range, and comparing the control force change rate with the low control force change range, medium control force change range and high control force change range, and determining the time delay of the control strategy model when outputting the control force at the current moment based on the three comparison results; the control strategy model controls the intelligent suspension to apply the current moment control force to the vehicle based on the time delay.

[0015] Furthermore, a hard constraint is imposed on the control force output by the following control strategy model:

[0016] ;

[0017] in, represents the control strategy model, represents the network weight of the control strategy model, represents the interception function, and Respectively represent the minimum and maximum values ​​of the control force at the current moment, Indicates the control power at the current moment.

[0018] Furthermore, in step S3: the current state ,in, and They represent the acceleration and velocity of the vehicle at time t, represents the dynamic travel of the smart suspension at time t, represents the vertical displacement of the vehicle at time t, represents the unsprung displacement of the vehicle at time t; represents the velocity difference between the sprung and unsprung masses of the vehicle at time t, , represents the unsprung speed of the vehicle at time t;

[0019] The current reward is

[0020] ;

[0021] in, represents the reward at time t, 、 、 and represents the reward coefficient, represents the dynamic wheel load of the vehicle at time t, , represents the road displacement of the vehicle at time t.

[0022] Furthermore, the dynamic itinerary satisfies:

[0023] ;

[0024] in, Indicates the ultimate dynamic deflection of the intelligent suspension.

[0025] Further, satisfy:

[0026] ;

[0027] in, Indicates the vehicle's tire stiffness; Express the static tire load of the vehicle as:

[0028] ;

[0029] in, represents the acceleration due to gravity, Indicates the vehicle's body mass, Indicates the unsprung mass.

[0030] Furthermore, in step S4: in each training time step, multiple groups of samples are randomly selected from the experience pool , calculate the corresponding target value by the following formula:

[0031] ;

[0032] in, represents the target value, represents the discount factor, represents the j-th evaluation network, Represents the network weight of the j-th evaluation network; combined with the target value, the loss function of training the two evaluation networks is calculated:

[0033] ;

[0034] in, Represents the loss function of the jth evaluation network, and N represents the time step. The two evaluation networks are trained using their loss functions. When the training process of the two evaluation networks meets the preset triggering conditions for training the control strategy model, the control strategy model is trained using the following formula:

[0035] .

[0036] A readable storage medium stores a computer program. When the computer program is executed by a processor, the computer program implements the steps of the intelligent suspension control method considering the time lag of the suspension system provided by the present invention.

[0037] An electronic device, comprising:

[0038] memory for storing computer programs;

[0039] The processor is configured to implement the steps of the intelligent suspension control method considering the time lag of the suspension system provided by the present invention when executing the computer program.

[0040] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0041] (1) The present invention creates an intelligent suspension control method that takes into account the time lag of the suspension system. A meta-learning module for complex road conditions is introduced to pre-train the evaluation network and the policy network at the initial stage of training. By constructing a meta-training set covering a variety of typical complex road conditions, the network is enabled to learn the common features and control strategies under different complex road conditions using a meta-learning algorithm, which greatly enhances the universality of the control strategy. In actual driving, facing various complex road conditions (such as potholes, bumps, muddy sections, etc.), the control strategy of the present invention can quickly adapt, effectively improving the vehicle's driving performance and ride comfort under complex road conditions, further broadening the scope of application of the present invention under different road conditions;

[0042] (2) In the intelligent suspension control method that takes into account the time delay of the suspension system described in the present invention, the time delay characteristics of the suspension system are innovatively considered in the design of the control strategy, including various situations such as deterministic delay, semi-regular delay and uncertain delay. By constructing multiple sets of simulation experiments to test the control performance of the algorithm under different time delay conditions, the various time delay characteristics of the control system and the uncertainty under their influence are fully considered, making the simulation results closer to the actual working conditions; at the same time, the uniquely designed action delay mechanism, especially the use of fuzzy delay control rules, can dynamically adjust the action delay time according to the vehicle driving state, significantly improving the robustness and reliability of the control strategy. This advantage enables the present invention to maintain more stable and effective suspension control performance when dealing with complex and changing driving environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0044] Figure 1 A flow chart of an intelligent suspension control method considering suspension system time lag according to an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of a framework of an intelligent suspension control method considering suspension system time lag according to an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of a control strategy network according to an embodiment of the present invention;

[0047] Figure 4 Schematic diagram of two evaluation networks according to an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of a dynamic model of an intelligent suspension system according to an embodiment of the present invention;

[0049] Figure 6 A schematic structural diagram of an electronic device according to an embodiment of the present invention.

[0050] Description of reference numerals:

[0051] 1. Electronic device; 2. External device; 3. Processing unit; 4. Bus; 5. Network adapter; 6. Display; 7. (I / O) interface; 8. System memory; 9. Random access memory; 10. Cache memory; 11. Storage system; 12. Utility; 13. Program module. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0053] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0054] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second" and the like are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first", "second" and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0055] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0056] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0057] like Figures 1 to 4 As shown, the intelligent suspension control method considering the time lag of the suspension system according to the embodiment of the present invention includes:

[0058] S1: Use meta-learning to pre-train the first evaluation network, the second evaluation network, and the control policy network to obtain the first evaluation model, the second evaluation model, and the control policy model, and initialize the experience pool.

[0059] Meta-learning, a method for learning how to learn, enables models to quickly adapt to new tasks and environments. Under complex road conditions, different road conditions (such as potholes, bumps, and muddy sections) place vastly different demands on suspension system control. To improve the universality of the control strategy, this paper introduces a meta-learning module to pre-train the evaluation network and policy network at the initial stage of training.

[0060] In some embodiments, step S1 includes:

[0061] S11: Initialize the first evaluation network, the second evaluation network, the control strategy network and the experience pool.

[0062] In one embodiment, the control strategy network is as follows: Figure 3 As shown in the figure, the vehicle state is input into a fully connected layer composed of 128 neurons for processing. The processed data is processed by the ReLU activation function and then input into a fully connected layer composed of 256 neurons. The output data is processed by the ReLU activation function and then input into a fully connected layer composed of 1 neuron. Finally, the output data is activated by the Tanh activation layer and scaled by the scaling layer to obtain the control force output by the intelligent suspension. The structures of the first evaluation network and the second evaluation network are shown in the figure. Figure 4 As shown in the figure, the vehicle state is input into a fully connected layer composed of 128 neurons for processing. The processed data is processed by the ReLU activation function and then input into a fully connected layer composed of 256 neurons to obtain the first feature. At the same time, the output of the control strategy network is processed by a fully connected layer composed of 256 neurons to obtain the second feature. After the corresponding elements of the first and second features are added, the ReLU activation function is processed and then input into a fully connected layer composed of 1 neuron to output the evaluation result. The initialization operations of the three networks include: The network weights in , First Rating Network The network weights in , and control strategy network The network weight Perform random initialization, s represents the state of the vehicle, and a represents the control force output by the intelligent suspension.

[0063] S12: Construct a meta-training set, and perform meta-learning on the initialized first evaluation network, second evaluation network, and control strategy network according to the meta-training set to obtain the corresponding first evaluation model, second evaluation model, and control strategy model.

[0064] In one embodiment, a large amount of vehicle driving data under various complex road conditions, including but not limited to vehicle acceleration, vehicle speed, suspension deflection, and road surface condition information, is collected to construct a meta-training set. This meta-training set covers a variety of typical complex road conditions to fully simulate the various situations likely to be encountered in actual driving. A gradient-based meta-learning algorithm (MAML: Model-Agnostic Meta-Learning) is used to pre-train the first evaluation network, the second evaluation network, and the control strategy network. The goal of pre-training is to find a set of initial network parameters that enable the network to quickly adjust the parameters using a small number of samples when faced with new complex road conditions, thereby achieving better control results.

[0065] During pre-training, the meta-learning rate was set to 0.001 and the number of meta-training steps was set to 500. During each meta-training iteration, multiple subtasks with different road conditions were randomly selected from the meta-training dataset, each containing a certain number of training samples. For each subtask, the evaluation network and the policy network were trained on that subtask, calculating the loss and updating the network parameters. Through multiple iterations, the network gradually learned the common features and control strategies for different complex road conditions, completing the pre-training process.

[0066] S2: The control strategy model obtained in step S1 determines the current control force output by the intelligent suspension according to the current state of the vehicle; the current control force is applied to the vehicle to obtain the next state of the vehicle.

[0067] In some embodiments, the process of applying the current control force to the vehicle in step S2 includes:

[0068] Hard constraints are imposed on the control force output by the control strategy model to obtain the control force at the current moment;

[0069] Set the vehicle's low-speed range, medium-speed range, and high-speed range;

[0070] Set the vehicle's low acceleration range, medium acceleration range, and high acceleration range;

[0071] Set a low control force variation interval, a medium control force variation interval, and a high control force variation interval corresponding to the control force output by the control strategy model;

[0072] Measuring the current speed and current acceleration of the vehicle, and calculating the control force change rate between the current control force and the previous control force; comparing the current speed with the low speed interval, the medium speed interval, and the high speed interval, comparing the current acceleration with the low acceleration interval, the medium acceleration interval, and the high acceleration interval, and comparing the control force change rate with the low control force change interval, the medium control force change interval, and the high control force change interval, and determining the time delay of the control strategy model in outputting the current control force based on the three comparison results;

[0073] The control strategy model controls the intelligent suspension to apply the current control force to the vehicle according to the time delay.

[0074] In one embodiment, the control force output by the control strategy model is hard constrained by the following formula:

[0075] ;

[0076] in, represents the interception function, and Respectively represent the minimum and maximum values ​​of the control force at the current moment, Indicates the control force at the current moment. Among them, the minimum value of the control force at the current moment and maximum value The value of is obtained according to the maximum force limit that the intelligent suspension can output, and in a certain embodiment, a minimum value is set. , maximum value .

[0077] In one embodiment, in determining the time delay of the control strategy model in outputting the control force at the current moment:

[0078] Set the vehicle's low speed range to [0, 45 km / h], medium speed range to (45 km / h, 90 km / h], and high speed range to (90 km / h, ∞);

[0079] The low acceleration of the vehicle can reflect the degree of bumpiness of the road. Therefore, the three acceleration intervals of the vehicle are set to reflect the different degrees of bumpiness of the road. That is, the low acceleration interval of the vehicle is set to [0, 0.5m / s 2 ], corresponding to a slight degree of bumpiness; the medium acceleration range is (0.5m / s 2 ,1.5m / s 2 ], corresponding to moderate turbulence; high acceleration range is (1.5m / s 2 ,∞), corresponding to the degree of severe turbulence;

[0080] The control force change rate can reflect the degree of change of the output control force, which can be obtained as follows:

[0081] ;

[0082] in, represents the rate of change of control force at time t, Indicates the time interval. Based on the above formula, we can set

[0083] The low control force change range is [0,10 kN / s], corresponding to a slow control force change; the medium control force change range is (10 kN / s,30 kN / s], corresponding to a medium control force change; and the high control force change range is (30 kN / s,∞), corresponding to a drastic control force change.

[0084] After completing the above range settings for speed, acceleration, and control force change rate, measure the vehicle's current speed and current acceleration, and calculate the control force change rate between the current control force and the previous control force; compare the current speed with the low speed range, medium speed range, and high speed range; compare the current acceleration with the low acceleration range, medium acceleration range, and high acceleration range; and compare the control force change rate with the low control force change range, medium control force change range, and high control force change range. Based on the three comparison results, determine the time delay of the control strategy model when outputting the current control force. The specific process is as follows:

[0085] If the current vehicle speed is low, the road is slightly bumpy, and the control signal rate of change is slow, the delay time is short, that is, 0.01s. This is because at low speeds, slight bumps, and slow control signal changes, the system response is relatively easy, and a shorter delay time can meet control requirements while ensuring vehicle stability.

[0086] If the vehicle is traveling at a low speed, the road is slightly bumpy, and the control signal rate of change is moderate, the delay is short, at 0.015 seconds. Although the control signal changes more quickly, the low speed and slightly bumpy road conditions still allow the system to respond effectively within a short delay. Therefore, the delay is increased appropriately to accommodate the change in the control signal.

[0087] If the vehicle is traveling at a low speed, the road is slightly bumpy, and the control signal rate of change is drastically changing, the delay is set to medium, or 0.02s. Because the control signal changes drastically, it takes time for the system to adapt. Furthermore, a longer delay won't cause serious consequences in low-speed, slightly bumpy road conditions, so a medium delay is used.

[0088] If the vehicle is traveling at a low speed, the road surface is moderately bumpy, and the control signal rate of change is slow, the delay is medium, or 0.02 seconds. Moderate bumps require more immediate system adjustments, but the system has more time to process them at low speeds, so a medium delay is used to balance control effectiveness and system stability.

[0089] If the vehicle is traveling at a low speed, the road surface is moderately bumpy, and the control signal rate of change is moderate, the delay is medium, or 0.025 seconds. In this case, both moderate bumps and a moderate control signal rate of change require a certain amount of system response time, so increasing the delay appropriately ensures control accuracy.

[0090] If the vehicle is traveling at a low speed, the road surface is moderately bumpy, and the control signal rate of change is drastic, the delay is long, at 0.03s. Drastic control signal changes and moderate bumps place high demands on the system's response, and a long delay helps the system better integrate information and make more appropriate control decisions.

[0091] If the vehicle is currently traveling at a low speed, the road is severely bumpy, and the control signal rate of change is slow, the delay is medium, or 0.025 seconds. Severe bumps require a fast system response, but the control signal changes slowly, so a medium delay is used to balance system stability and bump suppression.

[0092] If the vehicle is traveling at a low speed, the road is severely bumpy, and the control signal rate of change is moderate, the delay is long, or 0.035 seconds. Severe bumps and a moderate control signal rate of change require the system to adjust, and a long delay allows for better coordination between the two.

[0093] If the vehicle is currently traveling at a low speed, the road is severely bumpy, and the control signal rate of change is drastically changing, the delay will be long, at 0.04s. In this case, the system faces significant challenges, and a long delay helps the system more accurately respond to severe bumps and drastic control signal changes.

[0094] If the current vehicle speed is medium, the road surface is slightly bumpy, and the control signal rate of change is slow, the delay time is short, that is, 0.012s. Medium speed and slight bumps give the system a certain degree of responsiveness, and the control signal changes slowly, so a short delay slightly longer than that used for low speed, slight bumps, and slow changes is used.

[0095] If the vehicle is traveling at a medium speed, the road is slightly bumpy, and the control signal rate of change is moderate, the delay is medium, or 0.02s. Under moderate speed and slightly bumpy road conditions, a moderate control signal rate of change requires a certain amount of system reaction time. Medium delay balances control accuracy and response speed.

[0096] If the vehicle speed is medium, the road is slightly bumpy, and the control signal rate of change is drastic, the delay is medium, or 0.025 seconds. Although the road is slightly bumpy, the drastic control signal change requires sufficient time for the system to process. A medium delay ensures a stable system response.

[0097] If the current vehicle speed is medium, the road surface is moderately bumpy, and the control signal change rate is slow, the delay is medium, that is, 0.022s. Moderate bumps and medium speeds require a certain system response speed, and the control signal changes slowly. Medium delay can meet control requirements and ensure vehicle stability.

[0098] If the current vehicle speed is medium, the road surface is moderately bumpy, and the control signal rate of change is medium, then the delay is medium, or 0.028s. In this case, both the moderate bumps and the moderate control signal rate of change require a timely system response, so increasing the delay appropriately can improve control effectiveness.

[0099] If the current vehicle speed is medium, the road surface is moderately bumpy, and the control signal change rate is drastic, the delay is long, i.e., 0.035s. Drastic control signal changes and moderate bumps place higher demands on the system, and a long delay helps the system make more appropriate control decisions.

[0100] If the current vehicle speed is medium, the road is severely bumpy, and the control signal rate of change is slow, the delay is long, i.e., 0.03s. Severe bumps require a fast system response, so a long delay helps the system better cope with bumps even if the control signal changes slowly.

[0101] If the current vehicle speed is medium, the road is severely bumpy, and the control signal rate of change is medium, the delay is long, that is, 0.04s. Severe bumps and a medium control signal rate of change require the system to have sufficient time to adjust, and a long delay can improve the system's adaptability.

[0102] If the current vehicle speed is medium, the road is severely bumpy, and the control signal change rate is drastic, the delay time will be long, that is, 0.045s. In this extreme case, the long delay allows the system to fully integrate information and make more accurate control actions;

[0103] If the vehicle is traveling at high speed, the road is slightly bumpy, and the control signal rate of change is slow, the delay is medium, or 0.02s. At high speeds, system response time is relatively tight, and even with slight bumps and slow control signal changes, a certain delay is required to ensure control accuracy.

[0104] If the vehicle is traveling at high speed, the road is slightly bumpy, and the control signal rate of change is moderate, the delay is medium, or 0.025 seconds. Under high speed and slightly bumpy road conditions, a moderate control signal rate of change requires the system to have adequate reaction time. Medium delay balances control and vehicle stability.

[0105] If the vehicle is traveling at high speed, the road is slightly bumpy, and the control signal rate of change is drastic, the delay is long, i.e., 0.03s. Drastic control signal changes require more time to process at high speeds, and a long delay helps stabilize the system's response.

[0106] If the current vehicle speed is high, the road surface is moderately bumpy, and the control signal rate of change is slow, the delay is long, i.e., 0.035s. High speed and moderate bumps require a high level of system response, and even slow control signal changes cannot be ignored. A long delay ensures the system can effectively respond.

[0107] If the vehicle is traveling at high speed, the road is moderately bumpy, and the control signal rate of change is moderate, the delay is long, or 0.04s. In this case, high speed, moderate bumps, and a moderate control signal rate of change all require sufficient time for the system to respond, so a long delay improves control effectiveness.

[0108] If the vehicle is traveling at high speed, the road surface is moderately bumpy, and the control signal rate of change is drastically changing, the delay is long, i.e., 0.05s. Drastic control signal changes, high speed, and moderately bumpy road conditions pose significant challenges to the system, and a long delay allows the system to adjust the suspension more accurately.

[0109] If the vehicle is traveling at high speed, the road is severely bumpy, and the control signal rate of change is slow, the delay is long, i.e., 0.04s. Severe bumps and high speeds require a fast system response, so a long delay helps stabilize the system even if the control signal changes slowly.

[0110] If the vehicle is traveling at high speed, the road is severely bumpy, and the control signal rate of change is moderate, the delay is long, i.e., 0.05s. Severe bumps, high speed, and moderate control signal rate of change require sufficient system adjustment, and a long delay ensures system adaptability.

[0111] If the vehicle is traveling at high speed, the road is severely bumpy, and the control signal rate of change is drastically changing, the delay is long, at 0.06 seconds. This is the most complex operating condition, and a long delay allows the system to better handle the influence of various factors, ensuring safe and comfortable driving.

[0112] The center of gravity method is used as a clarification method to convert the fuzzy output delay time level into an actual delay time value. Specifically, after a series of judgments based on conditions such as vehicle speed, acceleration, and the rate of change of control force, a fuzzy output delay time level (such as short delay, medium delay, or long delay) is obtained. The center of gravity method is a commonly used clarification method for converting fuzzy outputs into precise numerical values. Specifically, for each delay time level, a corresponding membership function (which describes the degree to which a value belongs to that level) is pre-set. Taking delay time as an example, short delay, medium delay, and long delay each correspond to a different membership function curve. When the system outputs a fuzzy delay time level, a "center of gravity" position is calculated based on these membership functions. The horizontal coordinate value of this "center of gravity" position is the actual delay time value obtained by the center of gravity method. For example, assuming the membership function for short delay covers a time range of 0 to 0.02 seconds, medium delay covers a range of 0.02 to 0.04 seconds, and long delay covers a range of 0.04 to 0.06 seconds. When the system fuzzy output is somewhere between short and medium delays, calculating the centroid of the membership functions for these two levels yields a precise delay value, such as 0.025 seconds. This fuzzy delay control rule dynamically adjusts the action delay time based on the vehicle's current driving state, enabling the system to more effectively handle time lags under varying operating conditions and improve control effectiveness.

[0113] In one embodiment, the control force output by the intelligent suspension is added with a Gaussian random distribution. The disturbance amount, that is, the current control force output by the intelligent suspension is:

[0114] .

[0115] S3: Determine the current reward based on the current control power and the current state; save the current state, current control power, current reward, and next state as a set of samples in the experience pool.

[0116] In some embodiments, the intelligent suspension system dynamics model is as follows: Figure 5 As shown, Figure 5 in Indicates the vertical displacement of the vehicle at the current moment, represents the unsprung displacement of the vehicle at the current moment, Indicates the vehicle's body mass, represents the unsprung mass, Indicates the vehicle's current road displacement, Indicates the vehicle's tire stiffness, represents the suspension spring stiffness of the smart suspension, Indicates the damping coefficient of the smart suspension. Current state ,in, and They represent the acceleration and velocity of the vehicle at time t, respectively, and the vertical displacement of the vehicle at time t is Perform the first and second derivatives respectively to obtain, represents the dynamic travel of the smart suspension at time t; , represents the unsprung velocity of the vehicle at time t, which is determined by the unsprung displacement of the vehicle at time t. In one embodiment, in order to improve the generalization performance of the control strategy model, the state is normalized, that is, the current state ,in 、 、 and represents the normalization coefficient, which is adaptively selected and adjusted according to actual conditions. In a certain embodiment, it can be set to 2, 0.2, 0.4 and 0.15.

[0117] In some embodiments, the current reward is:

[0118] ;

[0119] in, represents the reward at time t, 、 、 and represents the reward coefficient, Represents the dynamic wheel load of the vehicle at time t. Specifically, . Reward coefficient 、 、 and Adaptive adjustment is made according to actual conditions. In one embodiment, a reward coefficient can be set. 、 、 and They are 0.7, 0.1, 0.1, and 0.1 respectively. It should be noted that the rewards in the training phase refer to the state information of the system after the control force is applied after the delay.

[0120] Among them, dynamic stroke and Satisfy respectively:

[0121] ;

[0122] ;

[0123] in, Indicates the ultimate dynamic deflection of the intelligent suspension; Express the static tire load of the vehicle as:

[0124] ;

[0125] in, In one embodiment, the ultimate dynamic deflection of the suspension is is 0.15m.

[0126] S4: Repeat steps S2 to S3 multiple times, and save multiple groups of samples in the experience pool accordingly; use the multiple groups of samples to retrain the first evaluation model, the second evaluation model, and the control strategy model obtained in step S1 to obtain the final first evaluation model, the final second evaluation model, and the final control strategy model.

[0127] In some embodiments, the training process in step S4 includes:

[0128] In each training time step, set the number of training episodes to M and the time step of each episode to N. Randomly extract multiple groups of samples from the experience pool , calculate the corresponding target value by the following formula:

[0129] ;

[0130] in, represents the target value, represents the discount factor, represents the j-th evaluation network, Represents the network weight of the j-th evaluation network; combined with the target value, the loss function of training the two evaluation networks is calculated:

[0131] ;

[0132] in, Represents the loss function of the jth evaluation network, and N represents the time step. The two evaluation networks are trained using their loss functions. When the training process of the two evaluation networks meets the preset triggering conditions for training the control strategy model, the control strategy model is trained using the following formula:

[0133] .

[0134] In one embodiment, the number of training episodes M is set to 4000, the time step N is set to 2000, and in each training time step, 256 groups of samples are randomly selected from the experience pool. , discount factor The value is 0.99. Using the stochastic gradient descent method, combined with the loss function The two evaluation networks are trained as follows:

[0135] ;

[0136] During the training process, the network weights of the two evaluation networks are adjusted using soft updates. and To update, that is:

[0137] ;

[0138] in, and Represent the updated network weights of the two evaluation networks, Indicates the soft update frequency, the value is 0.001;

[0139] The preset trigger condition is that the two evaluation networks are updated twice. That is, when both evaluation networks are updated twice, the control strategy network is trained using the stochastic gradient ascent method. During the training process, the network weights of the control strategy network are also updated using the soft update method. To update, that is:

[0140] ;

[0141] in, Indicates the network weight after the control strategy network is updated. The soft update frequency here is The value is still 0.001.

[0142] S5: Run the intelligent suspension and use the final first evaluation model, the final second evaluation model, and the final control strategy model obtained in step S4 to control the intelligent suspension.

[0143] Based on the provided intelligent suspension control method that takes into account suspension system time lag, the present invention also provides an intelligent suspension control system that takes into account suspension system time lag. The system includes an intelligent agent, a car or test bench equipped with an intelligent suspension and related sensors, a state observer and state estimator, and an intrinsic reward function. The intelligent agent controls the intelligent suspension, and the intelligent suspension control method that takes into account suspension system time lag provided by the present invention is integrated into the intelligent agent. The related sensors include an acceleration sensor and a displacement sensor inertial measurement unit (IMU). The operating process of the intelligent suspension control system is as follows:

[0144] The intelligent agent obtains data such as vehicle body acceleration, vehicle body speed, suspension dynamic deflection and derivative of suspension dynamic deflection from the entire vehicle system through sensors, and combines these data into the current state information of the vehicle through a state observer and a state estimator. The intelligent agent then decides how much control force the intelligent suspension should output in the current state based on this state information. Under the action of the control force, the state of the entire vehicle changes. The intelligent agent generates a reward value to evaluate its control action based on the new system state and the intrinsic reward function. The intelligent agent performs self-iteration and control strategy optimization with reference to the reward value in accordance with the intelligent suspension control method considering the time lag of the suspension system provided by the present invention.

[0145] Figure 6 Schematic diagram of the structure of an electronic device 1 provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary electronic device 1 suitable for implementing embodiments of the present invention is shown. Figure 6 The electronic device 1 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0146] like Figure 6 As shown, electronic device 1 is represented in the form of a general-purpose computing device. Electronic device 1 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0147] The components of the electronic device 1 may include, but are not limited to: one or more processors or processing units 3, a system memory 8, and a bus 4 connecting different system components (including the system memory 8 and the processing unit 3).

[0148] Bus 4 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0149] The electronic device 1 typically includes a variety of computer system readable media, which can be any available media that can be accessed by the electronic device 1, including volatile and non-volatile media, removable and non-removable media.

[0150] The system memory 8 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 9 and / or cache memory 10. The electronic device 1 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 11 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 Not shown, often called a "hard drive"). Although Figure 6 Not shown, a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 4 via one or more data medium interfaces. System memory 8 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0151] A program / utility 12 having a set (at least one) of program modules 13 may be stored, for example, in system memory 8. Such program modules 13 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 13 generally implement the functions and / or methods of the embodiments described herein.

[0152] The electronic device 1 may also communicate with one or more external devices 2 (e.g., a keyboard, a pointing device, a display 6, etc.), one or more devices that enable a user to interact with the electronic device 1, and / or any device that enables the electronic device 1 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 7. Furthermore, the electronic device 1 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 5. Figure 6 As shown, the network adapter 5 communicates with other modules of the electronic device 1 via the bus 4. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0153] The processing unit 3 executes various functional applications and data processing by running programs stored in the system memory 8, such as implementing the intelligent suspension control method considering the time delay of the suspension system provided by the embodiment of the present invention.

[0154] An embodiment of the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, on which a computer program is stored. When the program is executed by a processor, the intelligent suspension control method considering the suspension system time delay provided in all the inventive embodiments of this application is implemented.

[0155] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device.

[0156] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0157] The program code that comprises on the computer-readable medium can be transmitted with any appropriate medium, includes but not limited to wireless, electric wire, optical cable, RF etc., or above-mentioned any suitable combination.Can write the computer program code that is used to carry out the operation of the present invention with one or more programming languages ​​or its combination, described programming language comprises object-oriented programming language such as Java, Smalltalk, C++, also comprises conventional procedural programming language--such as " C " language or similar programming language.Program code can be carried out on user's computer completely, partly on user's computer, carry out as an independent software package, partly on user's computer partly on remote computer, or carry out completely on remote computer or server.In the situation that relates to remote computer, remote computer can comprise local area network (LAN) or wide area network (WAN) to be connected to user's computer by the network of any kind, perhaps, can be connected to external computer (for example, utilize Internet service provider to come to connect by Internet).

[0158] An embodiment of the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned intelligent suspension control method considering the time delay of the suspension system.

[0159] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0160] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. An intelligent suspension control method considering suspension system time lag, characterized in that: include: S1: Use meta-learning to pre-train the first evaluation network, the second evaluation network, and the control policy network to obtain the first evaluation model, the second evaluation model, and the control policy model, and initialize the experience pool; Step S1 includes: collecting a large amount of vehicle driving data under different complex road conditions to construct a meta-training set; S2: The control strategy model obtained in step S1 determines the current control force output by the intelligent suspension based on the current state of the vehicle; the current control force is applied to the vehicle to obtain the next state of the vehicle. In step S2, the process of applying the current control force to the vehicle includes: Performing hard constraints on the control force output by the control strategy model to obtain the control force at the current moment; Setting a low-speed range, a medium-speed range, and a high-speed range for the vehicle; Setting a low acceleration range, a medium acceleration range, and a high acceleration range of the vehicle; Setting a low control force variation interval, a medium control force variation interval, and a high control force variation interval corresponding to the control force output by the control strategy model; measuring a current speed and a current acceleration of the vehicle, and calculating a control force change rate between the control force at a current moment and the control force at a previous moment; comparing the current speed with the low speed interval, the medium speed interval, and the high speed interval, comparing the current acceleration with the low acceleration interval, the medium acceleration interval, and the high acceleration interval, and comparing the control force change rate with the low control force change interval, the medium control force change interval, and the high control force change interval, and determining a time delay of the control strategy model in outputting the control force at the current moment based on the three comparison results; The control strategy model controls the intelligent suspension to apply the current control force to the vehicle according to the time delay; In the process of determining the time delay of the control strategy model when outputting the control force at the current moment based on the three comparison results: determining a fuzzy output level based on the three comparison results, the fuzzy output level including short delay, medium delay, and long delay; using a center of gravity method as a clarification method to convert the fuzzy output delay time level into an actual delay time value; S3: determining a current reward based on the current control power and the current state; and storing the current state, the current control power, the current reward, and the next state as a set of samples in the experience pool; S4: Repeat steps S2 to S3 multiple times, and save multiple groups of samples in the experience pool; use the multiple groups of samples to train the first evaluation model, the second evaluation model, and the control strategy model obtained in step S1 again to obtain the final first evaluation model, the final second evaluation model, and the final control strategy model; S5: running the intelligent suspension, and controlling the intelligent suspension using the final first evaluation model, the final second evaluation model, and the final control strategy model obtained in step S4.

2. The intelligent suspension control method considering suspension system time lag according to claim 1, characterized in that: Step S1 includes: Initializing the first evaluation network, the second evaluation network, the control strategy network, and the experience pool; A meta-training set is constructed, and the initialized first evaluation network, second evaluation network, and control strategy network perform meta-learning according to the meta-training set to obtain the corresponding first evaluation model, second evaluation model, and control strategy model.

3. The intelligent suspension control method considering suspension system time lag according to claim 1, characterized in that: A hard constraint is imposed on the control force output by the control strategy model described in the following formula: ； in, represents the control strategy model, represents the network weight of the control strategy model, represents the interception function, and Respectively represent the minimum and maximum values ​​of the control force at the current moment, Indicates the control force at the current moment.

4. The intelligent suspension control method considering suspension system time lag according to claim 3, characterized in that: In step S3: Current state ,in, and represent the acceleration and speed of the vehicle at time t, respectively, represents the dynamic travel of the smart suspension at time t, represents the vertical displacement of the vehicle at time t, represents the unsprung displacement of the vehicle at time t; represents the velocity difference between the sprung mass and the unsprung mass of the vehicle at time t, , represents the unsprung speed of the vehicle at time t; The current rewards are: ； in, represents the reward at time t, 、 、 and represents the reward coefficient, represents the dynamic wheel load of the vehicle at time t, , represents the road displacement of the vehicle at time t.

5. The intelligent suspension control method considering suspension system time delay according to claim 4, characterized in that: The dynamic stroke satisfies: ； in, It represents the ultimate dynamic deflection of the intelligent suspension.

6. The intelligent suspension control method considering suspension system time lag according to claim 4, characterized in that: described satisfy: ； in, represents the tire stiffness of the vehicle; Denotes the static tire load of the vehicle, which is: ； in, represents the acceleration due to gravity, represents the body mass of the vehicle, Indicates the unsprung mass.

7. The intelligent suspension control method considering suspension system time lag according to claim 4, characterized in that: In step S4: In each training time step, multiple groups of samples are randomly drawn from the experience pool , calculate the corresponding target value by the following formula: ； in, represents the target value, represents the discount factor, represents the j-th evaluation network, represents the network weight of the j-th evaluation network; The loss function for training the two evaluation networks is calculated based on the target values: ； in, represents the loss function of the j-th evaluation network, and N represents the time step; the two evaluation networks are trained using their loss functions; When the training process of the two evaluation networks meets the preset triggering conditions for training the control strategy model, the control strategy model is trained using the following formula: 。 8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the intelligent suspension control method considering the time delay of the suspension system as claimed in any one of claims 1 to 7.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor is configured to implement the steps of the intelligent suspension control method considering the time delay of the suspension system as claimed in any one of claims 1 to 7 when executing the computer program.

Images