Cab suspension control method, system, device and storage medium

Abstract

The present application relates to a cab suspension control method, system, device and storage medium. By adopting a DQN model cab suspension control model, combined with multimodal input, namely, inputting vehicle speed, vertical vibration acceleration of four cab suspension points, cab roll angle, and a photo of the road surface in front of the vehicle, the optimal decision action regarding the vertical vibration acceleration of the four suspension points and the cab roll angle is output among multiple possible decisions. The suspension stiffness and damping are adjusted according to the optimal decision action to reduce cab vibration and simultaneously control the vertical vibration acceleration and cab roll angle of the four suspension points to improve driving comfort and vehicle stability. In addition, the suspension stiffness and damping are dynamically adjusted according to complex real-time road conditions and driving conditions to improve the adaptability and robustness of the system. A reward function that comprehensively considers the cab vibration acceleration and roll angle is designed to comprehensively evaluate the performance of the system, accurately control, and guide the optimization direction.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle suspension control, and in particular to a cab suspension control method, system, and computer-readable storage medium. Background Art

[0002] At present, during the driving process of the vehicle, the cab suspension system plays a vital role in improving the driver's comfort and reducing vehicle vibration.

[0003] In related technologies, traditional cab suspension control methods are usually based on physical models or empirical formulas, which are difficult to adapt to complex driving conditions and dynamically changing road conditions. Summary of the Invention

[0004] The embodiments of the present application provide a cab suspension control method, system, and computer-readable storage medium to address the problem in related technologies that traditional cab suspension control methods are generally based on physical models or empirical formulas and are difficult to adapt to complex driving conditions and dynamically changing road conditions.

[0005] In a first aspect, a cab suspension control method is provided, comprising:

[0006] Acquire status information during vehicle driving; status information includes vehicle speed, vertical vibration acceleration of the four cab suspension points, cab roll angle, and photos of the road ahead of the vehicle;

[0007] Inputting the state information into the DQN model cab suspension control model to calculate multiple potential decision actions and the actual reward of each potential decision action relative to the vertical vibration acceleration and cab roll angle of the four cab suspension points;

[0008] The actual reward of multiple potential decision actions is analyzed to output the optimal decision action; then, according to the optimal decision action, the spring stiffness and shock absorber damping of the four suspension points of the cab are adjusted.

[0009] In some embodiments, training the DQN model cab suspension control model includes the following steps:

[0010] Initialize the DQN network structure, network weight parameters, and experience replay buffer of the DQN model cab suspension control model;

[0011] Obtain a data sampling set and store it in an experience replay buffer; the data sampling set includes multiple data points distributed in chronological order;

[0012] A batch of data points are randomly extracted from the experience replay buffer; then, the DQN network structure is offline trained using the batch of data points, and the network weight parameters are updated until the DQN model cab suspension control model converges to a state that meets preset performance indicators.

[0013] In some embodiments, obtaining data points follows the following process:

[0014] Using the state information of the current time step, combined with the ε-greedy strategy, a decision action is selected;

[0015] Executing the selected decision action to adjust the spring stiffness and shock absorber damping of the four suspension points of the cab, and then obtaining the vertical vibration acceleration of the four suspension points and the roll angle of the cab after executing the selected decision action;

[0016] Calculating an instant reward based on the vertical vibration acceleration of the four suspension points and the roll angle of the cab in combination with a predefined reward function;

[0017] The state information of the current time step, the decision action of the current time step, the immediate reward after executing the decision action of the current time step, and the state information of the next time step are recorded as the data points of the current time step.

[0018] In some embodiments, calculating the instant reward based on the vertical vibration accelerations of the four suspension points and the roll angle of the cab in combination with a predefined reward function includes the following steps:

[0019] Obtaining an average value of weighted root mean square values ​​of vertical vibration accelerations of four mounting points, as well as a threshold value and a comfort value of the average value of the weighted root mean square values;

[0020] Obtaining the threshold and comfort value of the cab roll angle;

[0021] An immediate reward corresponding to a decision action is calculated based on the average value of the weighted root mean square value, the threshold and comfort value of the average value of the weighted root mean square value, the cab roll angle, the threshold and comfort value of the cab roll angle, and a predefined reward function.

[0022] In some embodiments, the DQN network structure is trained using a batch of the data points, and the network weight parameters are updated until the DQN model cab suspension control model converges to a state that meets preset performance indicators, including the following steps:

[0023] For each data point, the DQN network structure and initial network weight parameters are used to calculate the predicted reward value of the state information at the current time step, as well as the future reward value of the state information at the next time step;

[0024] Based on the first predefined function and combining the immediate reward and future reward value of each data point, the actual return reward of each data point is calculated;

[0025] Using a gradient descent algorithm, the network weight parameters are updated in combination with the actual reward and the predicted reward estimate until the difference calculated using the updated network weight parameters combined with the DQN network structure satisfies the first or second condition, and the predicted reward estimate satisfies the vehicle comfort requirement.

[0026] The DQN model cab suspension control model is formed using the final network weight parameters and the DQN network structure.

[0027] In some embodiments, the first condition is that the difference between the predicted reward estimate calculated using the updated network weight parameters combined with the DQN network structure and the actual reward is lower than a first preset threshold;

[0028] The second condition is that the error between the difference obtained after the last update of the network weight parameters and the difference obtained after the last update of the network weight parameters is less than a second preset threshold;

[0029] The preset performance indicators include a first condition, a second condition, and a predicted return reward valuation meeting vehicle comfort requirements.

[0030] In some embodiments, parsing the actual rewards of multiple potential decision actions to output the optimal decision action includes the following steps:

[0031] Calculate the actual reward for each potential decision action;

[0032] The decision action with the largest actual reward is selected as the optimal decision action; the potential decision actions include the adjustment values ​​of the spring stiffnesses and the shock absorber damping adjustment values ​​of the four suspension points.

[0033] In a second aspect, a cab suspension control system is provided, comprising:

[0034] A sensor module is used to obtain vehicle speed, vertical vibration acceleration of the four cab suspension points, cab roll angle, and a photo of the road ahead of the vehicle during driving, and generate status information;

[0035] The controller module is used to input the state information obtained from the sensor module into the DQN model cab suspension control model, and then calculate multiple potential decision actions and the actual rewards corresponding to the potential decision actions; the actual rewards of multiple potential decision actions are analyzed to output the optimal decision action;

[0036] The actuator module is used for controlling the suspension stiffness adjuster and the suspension damping adjuster according to the optimal decision action.

[0037] In a third aspect, a cab suspension control device is provided, which includes a processor, a memory, and a cab suspension control program stored in the memory and executable by the processor, wherein when the cab suspension control program is executed by the processor, the steps of the cab suspension control method are implemented.

[0038] In a fourth aspect, a computer-readable storage medium is provided, on which a cab suspension control program is stored, wherein when the cab suspension control program is executed by a processor, the steps of the cab suspension control method are implemented.

[0039] The beneficial effects of the technical solution provided by this application include:

[0040] The embodiments of the present application provide a cab suspension control method, system, device and storage medium. Due to the DQN model cab suspension control model, combined with multi-modal input, namely, input vehicle speed, vertical vibration acceleration of four cab suspension points, cab roll angle, and a photo of the road surface in front of the vehicle, the optimal decision action regarding the vertical vibration acceleration of the four suspension points and the cab roll angle can be output from multiple possible decisions, and the suspension stiffness and damping are adjusted according to the optimal decision action, thereby reducing cab vibration and simultaneously controlling the vertical vibration acceleration and cab roll angle of the four suspension points to improve driving comfort and vehicle stability; in addition, it also realizes dynamic adjustment of suspension stiffness and damping according to complex real-time road conditions and driving conditions, thereby improving the adaptability and robustness of the system; a reward function that comprehensively considers the cab vibration acceleration and roll angle is designed, which can comprehensively evaluate the performance of the system, accurately control, and guide the optimization direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 A schematic flow chart of the cab suspension control method provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of the network structure of the DQN model cab suspension control model provided in an embodiment of the present application;

[0044] Figure 3A schematic diagram of a cab suspension control system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0046] It should be understood that:

[0047] Reinforcement learning (RL) is a machine learning method in which an agent learns optimal behavior strategies by interacting with its environment. In RL, the agent maximizes cumulative rewards through trial and error. Q-Learning is a RL algorithm used to learn the optimal action-value function Q*(s,a). A deep Q-network (DQN) Q(s,a;w) is typically used to approximate the optimal action-value function, and the temporal difference (TD) algorithm is used to train the DQN.

[0048] The general steps are:

[0049] 1. Observe a quadruple (s t ,a t ,r t ,s t+1 ), called a transition;

[0050] where s t represents the state of the environment at time step t, a t represents the action chosen by the agent at time step t, r t Represents the reward, indicating that the agent performs action a at time step t t Immediate feedback from the environment.

[0051] 2. TD Target y t calculate

[0052] The TD objective is calculated by combining the immediate reward and future reward values. It guides the update of the reward estimate, enabling the agent to learn the optimal action policy. Using the target network improves training stability and avoids fluctuations in training caused by frequent updates to the reward estimate.

[0053] Among them, y t =rt +γmaxQ(S t+1 ,a;w);

[0054] γ is the discount rate, which indicates the importance of future rewards; maxQ(S t+1 ,a;w) means in the next state S t+1 The maximum reward estimate for all possible actions a under the given set. w is the network weight parameter of the target network.

[0055] 3. TD errorδ t calculate

[0056] δ t =Q(S t ,a t ;w)-y t TD error is the predicted value Q(s) of the deep Q network t ,a t ; w) and actual return y t The difference between.

[0057] 4. Gradient descent to update the neural network parameters w

[0058] Where α is a hyperparameter representing the learning rate.

[0059] The above is an introduction to the Deep Q Network (DQN) in the prior art, but there is no relevant implementation plan for how to apply it to suspension control.

[0060] In addition, the comfort of the vehicle is a relatively subjective point of view. There is currently no relevant solution on how to reflect comfort through certain data and guide adjustments in real time and accurately. This application associates the vertical vibration acceleration of the four suspension points of the cab, the cab roll angle and comfort, and participates in the adjustment of the suspension.

[0061] Therefore, the embodiments of the present application provide a cab suspension control method, system, device and storage medium to solve the problem that traditional cab suspension control methods in related technologies are usually based on physical models or empirical formulas and are difficult to adapt to complex driving conditions and dynamically changing road conditions.

[0062] First, see, Figure 1 and Figure 2 , a cab suspension control method, comprising the following steps:

[0063] Step 100: Acquire status information while the vehicle is traveling; the status information includes vehicle speed, vertical vibration acceleration of the four cab suspension points, cab roll angle, and a photograph of the road surface in front of the vehicle; the photograph of the road surface in front of the vehicle can be used to determine the height of the road surface. This application does not explain how to obtain the height, as this is a common technical method.

[0064] Step 200: Input the state information into the cab suspension control model of the DQN model to calculate multiple potential decision actions and actual rewards of the potential decision actions with respect to the vertical vibration acceleration of the four cab suspension points and the cab roll angle;

[0065] Step 300: Analyze the actual reward of multiple potential decision actions to output the optimal decision action; then adjust the spring stiffness and shock absorber damping of the four suspension points of the cab according to the optimal decision action.

[0066] Wherein, in step 300, the actual reward of each potential decision action is calculated;

[0067] The decision action with the largest actual reward is selected as the optimal decision action; the potential decision actions include the adjustment values ​​of the spring stiffnesses and the shock absorber damping adjustment values ​​of the four suspension points.

[0068] The above steps enable intelligent control. Using the DQN algorithm, the system automatically learns the optimal suspension stiffness and damping adjustment strategy, improving cabin comfort and vehicle stability. It responds to road surface changes in real time, dynamically adjusting suspension parameters to effectively reduce cabin vibration. Adaptive control automatically adjusts the control strategy based on varying road and driving conditions, improving system robustness and adaptability. The system offers flexible design, enabling adjustments to the DQN network structure and reward function.

[0069] Specifically:

[0070] Through the DQN model cab suspension control model, combined with multi-modal input, namely input vehicle speed, vertical vibration acceleration of the four suspension points of the cab, cab roll angle, and photos of the road surface in front of the vehicle, the optimal decision action regarding the vertical vibration acceleration of the four suspension points and the cab roll angle can be output among multiple possible decisions. The suspension stiffness and damping are adjusted according to the optimal decision action, thereby reducing cab vibration and simultaneously controlling the vertical vibration acceleration and cab roll angle of the four suspension points to improve driving comfort and vehicle stability; in addition, dynamic adjustment of suspension stiffness and damping according to complex real-time road conditions and driving conditions is also realized to improve the adaptability and robustness of the system; a reward function that comprehensively considers the cab vibration acceleration and roll angle is designed, which can comprehensively evaluate the performance of the system, accurately control, and guide the optimization direction.

[0071] In addition, the above correlation between the vertical vibration acceleration of the four suspension points, the cab roll angle and comfort provides clear standards and adjustment directions for how to specifically adjust to a comfortable state.

[0072] In some preferred embodiments, in step 100, the network structure of the DQN model cab suspension control model includes a first input layer and a second input layer;

[0073] The output end of the first input layer is connected to the convolutional network layer and the first fully connected layer in sequence; the first input layer is used to input environmental state information;

[0074] The second input layer is connected to the second fully connected layer, which is used to obtain the vertical vibration acceleration of the four suspension points and the cab roll angle; the second fully connected layer and the first fully connected layer are both connected to the decision action layer through the third fully connected layer.

[0075] In some preferred embodiments, how to associate the vertical vibration acceleration of the four suspension points, the cab roll angle and comfort, and participate in the DQN model cab suspension control model training is introduced as follows:

[0076] The specific training method of the DQN model cab suspension control model in step 200 includes the following steps:

[0077] Step 200-1: Initialize the DQN network structure, network weight parameters, and experience replay buffer of the DQN model cab suspension control model;

[0078] Step 200-2: Obtain a data sampling set and store it in an experience playback buffer; the data sampling set includes multiple data points distributed in chronological order;

[0079] Step 200-3: randomly extract a batch of data points from the experience replay buffer; then use the batch of data points to train the DQN network structure and update the network weight parameters until the DQN model cab suspension control model converges to a state that meets the preset performance indicators.

[0080] In step 200-2, the data points obtained in each interaction cycle follow the following process: The interaction cycle includes multiple time steps

[0081] In step 200-21, the state information at the current time step is used in conjunction with the ε-greedy strategy to select a decision action. This means that there is a certain probability of randomly selecting an action for exploration, and the remaining probability of selecting the action with the highest current reward estimate. The observation information includes the current environmental state, the vertical vibration acceleration of the four mounting points, and the cab roll angle. The ε-greedy strategy is a common exploration and exploitation strategy in reinforcement learning that aims to balance exploration and exploitation. At each time step, the agent randomly selects an action (exploration) with a probability of ε and selects the currently known optimal action (exploitation) with a probability of 1-ε.

[0082] Step 200-22: Execute the selected decision action to adjust the spring stiffness and shock absorber damping of the four suspension points of the cab, and then obtain the vertical vibration acceleration of the four suspension points and the roll angle of the cab after the selected decision action is executed;

[0083] Step 200-23: Calculate an immediate reward based on the vertical vibration acceleration of the four suspension points and the roll angle of the cab, combined with a predefined reward function. The specific steps are:

[0084] Obtaining an average value of weighted root mean square values ​​of vertical vibration accelerations of four mounting points, as well as a threshold value and a comfort value of the average value of the weighted root mean square values;

[0085] Obtaining the threshold and comfort value of the cab roll angle;

[0086] An immediate reward corresponding to a decision action is calculated based on the average value of the weighted root mean square value, the threshold and comfort value of the average value of the weighted root mean square value, the cab roll angle, the threshold and comfort value of the cab roll angle, and a predefined reward function.

[0087] Among them, we mainly focus on the vibration acceleration of the cabin and the stability of the vehicle, and design the following predefined reward function:

[0088] r t =-k1×(aa threshold )+k2×(a comfort- a)-k3×(θ-θ threshold )+k4×(θ comfort -θ)

[0089] Where: a is the average of the weighted RMS values ​​of the vertical vibration acceleration at the four cab suspension points. The weighted reference frequency range of 4-12.5 Hz, to which the human body is most sensitive to vertical vibration, is used. The real-time RMS acceleration value is calculated after a 4-12.5 Hz bandpass filter.

[0090] a threshold and a comfortThe threshold and comfort value of a are respectively. A reward exceeding the threshold is negative, and a reward below the comfort value is positive. The threshold and comfort value refer to the five levels of comfort requirements for different vehicle models. The uncomfortable state is the threshold, and the comfortable state is the comfort value.

[0091] θ is the cab's roll angle θ threshold and θ comfort These are the roll angle threshold and comfort level, respectively. Similarly, rewards are negative for values ​​exceeding the threshold and positive for values ​​below the comfort level. The roll angle threshold and comfort level are determined based on the handling requirements of different vehicle models. k1, k2, k3, and k4 are positive constants used to adjust the weights of the various indicators.

[0092] Step 200-24: Record the state information of the current time step, the decision action of the current time step, the immediate reward obtained after executing the decision action of the current time step, and the state information of the next time step as the data point of the current time step. The data point of the next time step is obtained in the same way.

[0093] Above, step 200-3 specifically includes the following steps:

[0094] Step 200-31: For each data point, use the DQN network structure and initial network weight parameters to calculate the predicted reward value of the state information of the current time step and the future reward value of the state information of the next time step;

[0095] Step 200-32: Calculate the actual reward for each data point based on the first predefined function and in combination with the immediate reward and future reward value of each data point;

[0096] The first predefined function is: t =r t +γmaxQ(S t+1 ,a;w);

[0097] The actual reward is the sum of the immediate reward plus the expected discount on the value of future rewards.

[0098] Step 200-33, using the gradient descent algorithm, and combining the actual reward and the predicted reward valuation to update the network weight parameters, until the difference calculated by combining the updated network weight parameters with the DQN network structure meets the first condition or the first condition, and the predicted reward valuation meets the vehicle comfort requirements; the first condition is that the difference between the predicted reward valuation calculated by combining the updated network weight parameters with the DQN network structure and the actual reward is lower than the first preset threshold; the second condition is that the error between the difference obtained after the last update of the network weight parameters and the difference obtained after the last update of the network weight parameters is less than the second preset threshold; the preset performance indicators include the first condition, the second condition and the predicted reward valuation meeting the vehicle comfort requirements. The predicted reward valuation that meets the vehicle comfort requirements is different for each vehicle or model and needs to be preset in advance;

[0099] The loss function can be minimized by using the gradient descent algorithm to update the network weight parameters. This process is repeated multiple times until the preset performance indicators or convergence conditions are reached. That is, the above data collection, storage, training and optimization processes are continuously iterated until the performance of the model is stable and can produce behaviors that meet the vehicle comfort requirements.

[0100] In step 200-33, the difference between the mean square error between the actual reward obtained with the initial network weight parameters and the predicted reward estimate is calculated using a second predefined function; the second predefined function is: δ t =Q(S t ,a t ;w)-y

[0101] The gradient descent algorithm is: Use the gradient descent algorithm to iterate continuously to reduce the error value.

[0102] Step 200-34: forming the DQN model cab suspension control model using the final network weight parameters and the DQN network structure.

[0103] In a second aspect, the present application further proposes a cab suspension control system, which includes:

[0104] A sensor module is used to obtain vehicle speed, vertical vibration acceleration of the four cab suspension points, cab roll angle, and a photo of the road ahead of the vehicle during driving, and generate status information;

[0105] The controller module is used to input the state information obtained from the sensor module into the DQN model cab suspension control model, and then calculate multiple potential decision actions and the actual rewards corresponding to the potential decision actions; the actual rewards of multiple potential decision actions are analyzed to output the optimal decision action;

[0106] an actuator module for controlling a suspension stiffness regulator and a suspension damping regulator according to an optimal decision action;

[0107] The communication module is used to transmit data between the controller module, sensor module, and actuator module. This communication module ensures real-time data exchange between modules and supports efficient system operation.

[0108] The system collects environmental status information through the sensor module, uses the DQN model in the controller module to select the optimal suspension control strategy, and applies these strategies to the actual suspension system through the actuator module, thereby optimizing the comfort and stability of the cab.

[0109] Above, the sensor module is responsible for collecting environmental data, which will serve as the input of DQN and includes the following components:

[0110] Acceleration sensor: measures the acceleration of the cab suspension.

[0111] Camera: Captures images of the road ahead and is used to identify road conditions.

[0112] Vehicle speed sensor: monitors the current vehicle speed.

[0113] The controller module contains a microprocessor, memory, and software that runs the DQN algorithm. The actuator module adjusts the state of the cab suspension according to the instructions issued by the controller. It includes:

[0114] Spring rate adjuster: controls the cab air spring pressure to adjust the air spring rate.

[0115] Shock absorber damping adjuster: adjusts the damping force of the cab shock absorber.

[0116] Through the above description, this application has the following effects:

[0117] (1) Optimize the performance of the suspension system through reinforcement learning technology, reduce cabin vibration, and improve driving comfort and vehicle stability.

[0118] (2) Multimodal input processing: A multimodal input processing method was designed, including image input and numerical input, which can comprehensively consider road conditions, vehicle speed and suspension point vibration acceleration to improve control accuracy.

[0119] (3) Dynamic adjustment strategy: The system can dynamically adjust the suspension stiffness and damping according to real-time road conditions and driving conditions, thereby improving the system's adaptability and robustness.

[0120] (4) Efficient training algorithm: Using experience replay and target network technology to improve the stability and efficiency of training and accelerate convergence.

[0121] (5) Comprehensive reward function: A reward function that comprehensively considers the vibration acceleration and roll angle of the cab is designed to comprehensively evaluate the performance of the system and guide the optimization direction.

[0122] Deploying the above cab suspension control system includes the following steps

[0123] Sensor installation:

[0124] Install the accelerometer, camera, vehicle speed sensor, and height sensor in appropriate locations to ensure they can accurately collect the required data.

[0125] Controller configuration:

[0126] Install the operating system and necessary development environment on the microprocessor.

[0127] Load the pre-trained DQN model into memory.

[0128] Write the control logic code that will read sensor data, calculate actions through the DQN model, and send control commands to the actuators.

[0129] Communication Settings:

[0130] Configure the CAN bus and ensure that all modules can communicate with each other correctly.

[0131] Test the communication link to ensure that data is transmitted correctly.

[0132] Actuator connection:

[0133] Connect the spring rate adjuster and the shock absorber damping adjuster.

[0134] Calibrate actuators to ensure they operate precisely as commanded by the control system.

[0135] System integration testing:

[0136] Conduct system integration testing in a safe environment to verify that sensor data acquisition, DQN decision-making, control command generation, and actuator responses meet expectations. Adjust parameters and conduct multiple tests until the system performs stably and reliably.

[0137] Actual deployment:

[0138] The entire system is installed on a real vehicle and final commissioning is performed.

[0139] Monitor system performance and perform further tuning as needed.

[0140] In a third aspect, an embodiment of the present application provides a cab suspension control device, which may be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0141] In the embodiment of the present application, the cab suspension control device may include a processor, a memory, a communication interface and a communication bus.

[0142] The communication bus may be of any type and is used to interconnect the processor, memory, and communication interface.

[0143] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces, used to interconnect components within the cab mount control system and to connect the cab mount control system to other devices (such as other computing devices or user devices). Physical interfaces can be Ethernet, fiber, or ATM interfaces; user devices can be displays or keyboards.

[0144] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical memory, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0145] The processor may be a general-purpose processor that can call a cab suspension control program stored in a memory and execute the cab suspension control method provided in the embodiments of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the cab suspension control program is called can be referred to in the various embodiments of the cab suspension control method of the present application and will not be further described here.

[0146] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0147] The computer-readable storage medium of the present application stores a cab suspension control program, wherein when the cab suspension control program is executed by a processor, the steps of the cab suspension control method as described above are implemented.

[0148] Among them, the method implemented when the cab suspension control program is executed can refer to the various embodiments of the cab suspension control method of the present application, and will not be repeated here.

[0149] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0150] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit the "first", "second" and "third" to different types.

[0151] In the description of the embodiments of this application, the words "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0152] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0153] In some processes described in the embodiments of the present application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be performed in the order in which they appear in the embodiments of the present application or may be performed in parallel. The sequence numbers of the operations are only used to distinguish between different operations, and the sequence numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations or steps may be performed in sequence or in parallel, and these operations or steps may be combined.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of the present application.

[0155] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A cab suspension control method, characterized in that: It includes: Acquire status information during vehicle driving; status information includes vehicle speed, vertical vibration acceleration of the four cab suspension points, cab roll angle, and photos of the road ahead of the vehicle; Inputting the state information into the DQN model cab suspension control model to calculate multiple potential decision actions and the actual reward of each potential decision action relative to the vertical vibration acceleration and cab roll angle of the four cab suspension points; The actual rewards of multiple potential decision actions are analyzed to output the optimal decision action; then, the spring stiffness and shock absorber damping of the four suspension points of the cab are adjusted according to the optimal decision action; Training the DQN model cab suspension control model includes the following steps: Initialize the DQN network structure, network weight parameters, and experience replay buffer of the DQN model cab suspension control model; A data sampling set is obtained and stored in an experience replay buffer; the data sampling set includes multiple data points distributed in chronological order; obtaining the data points follows the following process: using the state information of the current time step, in combination with the ε-greedy strategy, a decision action is selected; executing the selected decision action to adjust the spring stiffness and shock absorber damping of the four suspension points of the cab, and then obtaining the vertical vibration acceleration of the four suspension points and the roll angle of the cab after executing the selected decision action; calculating an immediate reward based on the vertical vibration acceleration of the four suspension points and the roll angle of the cab in combination with a predefined reward function; recording the state information of the current time step, the decision action of the current time step, the immediate reward obtained after executing the decision action of the current time step, and the state information of the next time step as the data point of the current time step; A batch of data points are randomly extracted from the experience replay buffer; the batch of data points are then used to perform offline training on the DQN network structure, and network weight parameters are updated until the DQN model cab suspension control model converges to a state that meets preset performance indicators; the batch of data points are used to train the DQN network structure, and network weight parameters are updated until the DQN model cab suspension control model converges to a state that meets preset performance indicators, including the following steps: for each data point, using the DQN network structure and initial network weight parameters, calculating a predicted reward estimate for state information at a current time step and a future reward value for state information at a next time step; calculating an actual reward for each data point based on a first predefined function and combining the immediate reward and future reward value of each data point; updating the network weight parameters using a gradient descent algorithm and combining the actual reward and the predicted reward estimate until a difference calculated using the updated network weight parameters and the DQN network structure meets a first condition or a second condition, and the predicted reward estimate meets vehicle comfort requirements; and forming the DQN model cab suspension control model using the final network weight parameters and the DQN network structure.

2. The cab suspension control method according to claim 1, wherein: Calculating an immediate reward based on the vertical vibration accelerations of the four suspension points and the roll angle of the cab in combination with a predefined reward function includes the following steps: Obtaining an average value of weighted root mean square values ​​of vertical vibration accelerations of four mounting points, as well as a threshold value and a comfort value of the average value of the weighted root mean square values; Obtaining the threshold and comfort value of the cab roll angle; An immediate reward corresponding to a decision action is calculated based on the average value of the weighted root mean square value, the threshold and comfort value of the average value of the weighted root mean square value, the cab roll angle, the threshold and comfort value of the cab roll angle, and a predefined reward function.

3. The cab suspension control method according to claim 1, wherein: The first condition is that the difference between the predicted reward estimate calculated using the updated network weight parameters combined with the DQN network structure and the actual reward is lower than a first preset threshold; The second condition is that the error between the difference obtained after the last update of the network weight parameters and the difference obtained after the last update of the network weight parameters is less than a second preset threshold; The preset performance indicators include a first condition, a second condition, and a predicted return reward valuation meeting vehicle comfort requirements.

4. The cab suspension control method according to claim 1, wherein: Parsing the actual rewards of multiple potential decision actions to output the optimal decision action includes the following steps: Calculate the actual reward for each potential decision action; The decision action with the largest actual reward is selected as the optimal decision action; the potential decision actions include the adjustment values ​​of the spring stiffnesses and the shock absorber damping adjustment values ​​of the four suspension points.

5. A cab suspension control system, characterized in that: It includes: A sensor module is used to obtain vehicle speed, vertical vibration acceleration of the four cab suspension points, cab roll angle, and a photo of the road ahead of the vehicle during driving, and generate status information; The controller module is used to input the state information obtained from the sensor module into the DQN model cab suspension control model, and then calculate multiple potential decision actions and the actual rewards corresponding to the potential decision actions; the actual rewards of multiple potential decision actions are analyzed to output the optimal decision action; Training the DQN model cab suspension control model includes the following steps: Initialize the DQN network structure, network weight parameters, and experience replay buffer of the DQN model cab suspension control model; A data sampling set is obtained and stored in an experience replay buffer; the data sampling set includes multiple data points distributed in chronological order; obtaining the data points follows the following process: using the state information of the current time step, in combination with the ε-greedy strategy, a decision action is selected; executing the selected decision action to adjust the spring stiffness and shock absorber damping of the four suspension points of the cab, and then obtaining the vertical vibration acceleration of the four suspension points and the roll angle of the cab after executing the selected decision action; calculating an immediate reward based on the vertical vibration acceleration of the four suspension points and the roll angle of the cab in combination with a predefined reward function; recording the state information of the current time step, the decision action of the current time step, the immediate reward obtained after executing the decision action of the current time step, and the state information of the next time step as the data point of the current time step; A batch of data points are randomly extracted from the experience replay buffer; the batch of data points are then used to perform offline training on the DQN network structure, and network weight parameters are updated until the DQN model cab suspension control model converges to a state that meets preset performance indicators; the batch of data points are used to train the DQN network structure, and network weight parameters are updated until the DQN model cab suspension control model converges to a state that meets preset performance indicators, including the following steps: for each data point, using the DQN network structure and initial network weight parameters to calculate a predicted reward estimate of state information at a current time step, and a future reward value of state information at a next time step; based on a first predefined function, and combining the immediate reward and future reward value of each data point, an actual reward is calculated for each data point; using a gradient descent algorithm, and combining the actual reward and the predicted reward estimate, the network weight parameters are updated until a difference calculated using the updated network weight parameters and the DQN network structure meets a first condition or a second condition, and the predicted reward estimate meets vehicle comfort requirements; and forming the DQN model cab suspension control model using the final network weight parameters and the DQN network structure; The actuator module is used for controlling the suspension stiffness adjuster and the suspension damping adjuster according to the optimal decision action.

6. A cab suspension control device, characterized in that: The cab suspension control device includes a processor, a memory, and a cab suspension control program stored in the memory and executable by the processor, wherein when the cab suspension control program is executed by the processor, the steps of the cab suspension control method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a cab suspension control program, wherein when the cab suspension control program is executed by a processor, the steps of the cab suspension control method according to any one of claims 1 to 4 are implemented.

Images