Cab suspension control method, system and equipment and storage medium

Abstract

The invention relates to a cab suspension control method, system and device and a storage medium, through a DQN model cab suspension control model and in combination with multi-mode input, namely, the input vehicle speed, the vertical vibration acceleration of four suspension points of a cab, the cab roll angle and the picture of the pavement in front of the vehicle; the optimal decision action about the vertical vibration acceleration and the cab roll angle of the four suspension points is output in a plurality of possible decisions, the suspension rigidity and damping are adjusted according to the optimal decision action, cab vibration is reduced, and the vertical vibration acceleration and the cab roll angle of the four suspension points are controlled at the same time; the driving comfort and the vehicle stability are improved; in addition, the suspension rigidity and damping are dynamically adjusted according to complex real-time road conditions and driving conditions, and the adaptability and robustness of the system are improved; the award function comprehensively considering the vibration acceleration and the roll angle of the cab is designed, the performance of the system can be comprehensively evaluated, control is accurate, and the optimization direction is guided.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle suspension control, and in particular to a cab suspension control method, system, and computer-readable storage medium. Background Art

[0002] At present, during vehicle driving, the cab suspension system plays a vital role in improving driver comfort and reducing vehicle vibration.

[0003] In the related art, traditional cab suspension control methods are usually based on physical models or empirical formulas, which are difficult to adapt to complex driving conditions and dynamically changing road conditions. Summary of the invention

[0004] The embodiments of the present application provide a cab suspension control method, system, and computer-readable storage medium to solve the problem that traditional cab suspension control methods in related technologies are usually based on physical models or empirical formulas and are difficult to adapt to complex driving conditions and dynamically changing road conditions.

[0005] In a first aspect, a cab suspension control method is provided, comprising:

[0006] Acquire status information during vehicle driving; the status information includes vehicle speed, vertical vibration acceleration of four suspension points of the cab, cab roll angle, and photos of the road surface in front of the vehicle;

[0007] Inputting the state information into the cab suspension control model of the DQN model to calculate multiple potential decision actions and actual reward of each potential decision action relative to the vertical vibration acceleration of the four suspension points of the cab and the cab roll angle;

[0008] The actual return rewards of multiple potential decision actions are analyzed to output the optimal decision action; then, the spring stiffness and shock absorber damping of the four suspension points of the cab are adjusted according to the optimal decision action.

[0009] In some embodiments, training the DQN model cab suspension control model includes the following steps:

[0010] Initialize the DQN network structure, network weight parameters and experience replay buffer of the DQN model cab suspension control model;

[0011] Obtain a data sampling set and store it in an experience playback buffer; the data sampling set includes a plurality of data points distributed in chronological order;

[0012] A batch of data points are randomly extracted from the experience replay buffer; then the DQN network structure is offline trained using the batch of data points, and the network weight parameters are updated until the DQN model cab suspension control model converges to a state that meets the preset performance indicators.

[0013] In some embodiments, obtaining data points follows the following process:

[0014] Using the state information of the current time step, a decision action is selected in combination with the ε-greedy strategy;

[0015] Execute the selected decision action to adjust the spring stiffness and shock absorber damping of four suspension points of the cab, and then obtain the vertical vibration acceleration of the four suspension points and the roll angle of the cab after the selected decision action is executed;

[0016] Calculating an instant reward based on the vertical vibration acceleration of the four suspension points and the roll angle of the cab in combination with a predefined reward function;

[0017] The state information of the current time step, the decision action of the current time step, the immediate reward after executing the decision action of the current time step, and the state information of the next time step are recorded as the data points of the current time step.

[0018] In some embodiments, the instant reward is calculated based on the vertical vibration acceleration of the four suspension points and the roll angle of the cab in combination with a predefined reward function, including the following steps:

[0019] Obtaining an average value of weighted root mean square values ​​of vertical vibration accelerations of four suspension points, as well as a threshold value and a comfort value of the average value of the weighted root mean square values;

[0020] Obtaining the threshold and comfort value of the cab roll angle;

[0021] Based on the average value of the weighted root mean square value, the threshold and comfort value of the average value of the weighted root mean square value, the cab roll angle, the threshold and comfort value of the cab roll angle, and in combination with a predefined reward function, an immediate reward corresponding to a decision action is calculated.

[0022] In some embodiments, the DQN network structure is trained using a batch of the data points, and the network weight parameters are updated until the DQN model cab suspension control model converges to a state that meets the preset performance indicators, including the following steps:

[0023] For each data point, the predicted return reward estimate of the state information of the current time step and the future reward value of the state information of the next time step are calculated using the DQN network structure and the initial network weight parameters;

[0024] Based on the first predefined function and combining the immediate reward and future reward value of each data point, calculate the actual return reward for each data point;

[0025] Using a gradient descent algorithm, and combining the actual reward and the predicted reward estimate, the network weight parameters are updated until the difference calculated by combining the updated network weight parameters with the DQN network structure satisfies the first condition or the second condition, and the predicted reward estimate meets the vehicle comfort requirement;

[0026] The DQN model cab suspension control model is formed with the final network weight parameters and the DQN network structure.

[0027] In some embodiments, the first condition is that the difference between the predicted reward estimate calculated by combining the updated network weight parameters with the DQN network structure and the actual reward is lower than a first preset threshold;

[0028] The second condition is that the error between the difference obtained after the last update of the network weight parameter and the difference obtained after the last update of the network weight parameter is less than a second preset threshold;

[0029] The preset performance indicators include a first condition, a second condition and a predicted return reward valuation meeting vehicle comfort requirements.

[0030] In some embodiments, parsing actual rewards of multiple potential decision actions to output an optimal decision action includes the following steps:

[0031] Calculate the actual reward for each potential decision action;

[0032] The decision action with the largest actual reward is selected as the optimal decision action; the potential decision actions include adjustment values ​​of the spring stiffness of the four suspension points and the shock absorber damping adjustment values.

[0033] In a second aspect, a cab suspension control system is provided, comprising:

[0034] A sensor module is used to obtain the vehicle speed, the vertical vibration acceleration of the four suspension points of the cab, the cab roll angle, and the road surface photo in front of the vehicle during the vehicle's driving process, and form status information;

[0035] A controller module is used to input the state information obtained from the sensor module into the DQN model cab suspension control model, and then calculate multiple potential decision actions and actual reward rewards corresponding to the potential decision actions; analyze the actual reward rewards of multiple potential decision actions to output the optimal decision action;

[0036] The actuator module is used for controlling the suspension stiffness adjuster and the suspension damping adjuster according to the optimal decision action.

[0037] In a third aspect, a cab suspension control device is provided, comprising a processor, a memory, and a cab suspension control program stored in the memory and executable by the processor, wherein when the cab suspension control program is executed by the processor, the steps of the cab suspension control method are implemented.

[0038] In a fourth aspect, a computer-readable storage medium is provided, on which a cab suspension control program is stored, wherein when the cab suspension control program is executed by a processor, the steps of the cab suspension control method are implemented.

[0039] The beneficial effects of the technical solution provided by this application include:

[0040] The embodiments of the present application provide a cab suspension control method, system, device and storage medium. Due to the DQN model cab suspension control model, combined with multi-modal input, namely, input vehicle speed, vertical vibration acceleration of four suspension points of the cab, cab roll angle, and a photo of the road surface in front of the vehicle, the optimal decision action about the vertical vibration acceleration of the four suspension points and the cab roll angle can be output from multiple possible decisions, and the suspension stiffness and damping are adjusted according to the optimal decision action, so as to reduce the vibration of the cab and simultaneously control the vertical vibration acceleration of the four suspension points and the cab roll angle to improve driving comfort and vehicle stability; in addition, it also realizes dynamic adjustment of suspension stiffness and damping according to complex real-time road conditions and driving conditions, so as to improve the adaptability and robustness of the system; a reward function that comprehensively considers the vibration acceleration and roll angle of the cab is designed, which can comprehensively evaluate the performance of the system, accurately control, and guide the optimization direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A schematic diagram of the general flow of the cab suspension control method provided in the embodiment of the present application;

[0043] Figure 2 A schematic diagram of the network structure of the DQN model cab suspension control model provided in an embodiment of the present application;

[0044] Figure 3A schematic diagram of a cab suspension control system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0046] It should be understood that:

[0047] Reinforcement Learning (RL) is a machine learning method in which an agent learns the optimal behavior strategy by interacting with the environment. In reinforcement learning, the agent maximizes the cumulative reward through trial and error. Q-Learning is a reinforcement learning algorithm used to learn the optimal action-value function Q*(s,a). A deep Q network (DQN) Q(s,a;w) is usually used to approximate the optimal action-value function, and the temporal difference (TD) algorithm is used to train the DQN.

[0048] The general steps are:

[0049] 1. Observe a quaternion (s t ,a t ,r t ,s t+1 ), called a transition;

[0050] where s t represents the state of the environment at time step t, a t represents the action chosen by the agent at time step t, r t Represents the reward, indicating that the agent performs action a at time step t t Immediate feedback from the environment.

[0051] 2. TD Target y t calculate

[0052] The TD target is calculated by combining the immediate reward and the future reward value. It is used to guide the update of the return reward valuation so that the agent can learn the optimal action strategy. By using the target network, the stability of training can be improved and the training fluctuations caused by frequent updates of the return reward valuation can be avoided.

[0053] Among them, y t =rt +γmaxQ(S t+1 ,a;w);

[0054] γ is the discount rate, which indicates the importance of future rewards; maxQ(S t+1 ,a;w) means in the next state S t+1 The maximum reward estimate for all possible actions a under . w is the network weight parameter of the target network.

[0055] 3. TD errorδ t calculate

[0056] δ t =Q(S t ,a t ;w)-y t TD error is the predicted value Q(s) of the deep Q network t ,a t ; w) and actual return y t The difference between.

[0057] 4. Gradient descent to update the neural network parameters w

[0058] Where α is a hyperparameter representing the learning rate.

[0059] The above is an introduction to the deep Q network (DQN) in the prior art, but there is no relevant implementation plan for how to apply it in suspension control.

[0060] In addition, the comfort of the vehicle is a relatively subjective point of view. There is currently no relevant solution on how to reflect comfort through certain data and guide adjustments in real time and accurately. This application associates the vertical vibration acceleration of the four suspension points of the cab, the roll angle of the cab with comfort, and participates in the adjustment of the suspension.

[0061] Therefore, the embodiments of the present application provide a cab suspension control method, system, device and storage medium to solve the problem that traditional cab suspension control methods in related technologies are usually based on physical models or empirical formulas and are difficult to adapt to complex driving conditions and dynamically changing road conditions.

[0062] First, please see, Figure 1 and Figure 2 , a cab suspension control method, comprising the following steps:

[0063] Step 100, obtaining status information during vehicle driving; the status information includes vehicle speed, vertical vibration acceleration of four suspension points of the cab, cab roll angle, and a photo of the road surface in front of the vehicle; the photo of the road surface in front of the vehicle can determine the height of the road surface, and this application does not explain how to obtain the height, which is a common technical means;

[0064] Step 200, inputting the state information into the cab suspension control model of the DQN model to calculate a plurality of potential decision actions and actual reward of potential decision actions with respect to vertical vibration acceleration of four cab suspension points and cab roll angle;

[0065] Step 300: Analyze the actual reward of multiple potential decision actions to output the optimal decision action; then adjust the spring stiffness and shock absorber damping of the four suspension points of the cab according to the optimal decision action.

[0066] Wherein, in step 300, the actual return reward of each potential decision action is calculated;

[0067] The decision action with the largest actual reward is selected as the optimal decision action; the potential decision actions include adjustment values ​​of the spring stiffness of the four suspension points and the shock absorber damping adjustment values.

[0068] Through the above steps, intelligent control can be achieved. That is, through the DQN algorithm, the optimal suspension stiffness and damping adjustment strategy can be automatically learned to improve the comfort of the cab and the stability of the vehicle. It can respond to road changes in real time, dynamically adjust suspension parameters, and effectively reduce cab vibration. It is adaptive and can automatically adjust the control strategy according to different road conditions and driving conditions to improve the robustness and adaptability of the system. The system design is flexible and can be adjusted by adjusting the DQN network structure and reward function.

[0069] Specifically:

[0070] Through the DQN model cab suspension control model, combined with multi-modal input, namely input vehicle speed, vertical vibration acceleration of four cab suspension points, cab roll angle, and photos of the road surface in front of the vehicle, the optimal decision action about the vertical vibration acceleration of four suspension points and cab roll angle can be output among multiple possible decisions, and the suspension stiffness and damping are adjusted according to the optimal decision action, so as to reduce the vibration of the cab and simultaneously control the vertical vibration acceleration of four suspension points and cab roll angle to improve driving comfort and vehicle stability; in addition, it also realizes dynamic adjustment of suspension stiffness and damping according to complex real-time road conditions and driving conditions to improve the adaptability and robustness of the system; a reward function that comprehensively considers the vibration acceleration and roll angle of the cab is designed, which can comprehensively evaluate the performance of the system, control accurately, and guide the optimization direction.

[0071] In addition, the above correlation between the vertical vibration acceleration of the four suspension points, the cab roll angle and comfort provides clear standards and adjustment directions for how to specifically adjust to a comfortable state.

[0072] In some preferred embodiments, in step 100, the network structure of the DQN model cab suspension control model includes a first input layer and a second input layer;

[0073] The output end of the first input layer is connected to the convolutional network layer and the first fully connected layer in sequence; the first input layer is used to input environmental state information;

[0074] The second input layer is connected to the second fully connected layer, and the second input layer is used to obtain the vertical vibration acceleration of the four suspension points and the roll angle of the cab; the second fully connected layer and the first fully connected layer are both connected to the decision action layer through the third fully connected layer.

[0075] In some preferred embodiments, how to associate the vertical vibration acceleration of the four suspension points, the cab roll angle and the comfort, and participate in the DQN model cab suspension control model training, is introduced as follows:

[0076] The specific training method of the DQN model cab suspension control model in step 200 includes the following steps:

[0077] Step 200-1, initializing the DQN network structure, network weight parameters and experience playback buffer of the DQN model cab suspension control model;

[0078] Step 200-2, obtaining a data sampling set and storing it in an experience playback buffer; the data sampling set includes a plurality of data points distributed in chronological order;

[0079] Step 200-3, randomly extract a batch of data points from the experience playback buffer; then use the batch of data points to train the DQN network structure and update the network weight parameters until the DQN model cab suspension control model converges to a state that meets the preset performance indicators.

[0080] In step 200-2, the data points acquired in each interaction cycle follow the following process: The interaction cycle includes multiple time steps

[0081] In step 200-21, the state information of the current time step is used to select a decision action in combination with the ε-greedy strategy; that is, there is a certain probability of randomly selecting an action for exploration, and the remaining probability is to select the action with the largest current reward valuation; the observation information includes the environmental state information at the current time, the vertical vibration acceleration of the four suspension points, and the cab roll angle; the ε-greedy strategy is an exploration and utilization strategy commonly used in reinforcement learning, which aims to balance the relationship between exploration and utilization. In each time step, the agent randomly selects an action (exploration) with a probability of ε, and selects the currently known optimal action (utilization) with a probability of 1-ε.

[0082] Step 200-22, executing the selected decision action to adjust the spring stiffness and shock absorber damping of the four suspension points of the cab, and then obtaining the vertical vibration acceleration of the four suspension points and the roll angle of the cab after executing the selected decision action;

[0083] Step 200-23, calculating an instant reward based on the vertical vibration acceleration of the four suspension points and the roll angle of the cab in combination with a predefined reward function; the specific steps are:

[0084] Obtaining an average value of weighted root mean square values ​​of vertical vibration accelerations of four suspension points, as well as a threshold value and a comfort value of the average value of the weighted root mean square values;

[0085] Obtaining the threshold and comfort value of the cab roll angle;

[0086] Based on the average value of the weighted root mean square value, the threshold and comfort value of the average value of the weighted root mean square value, the cab roll angle, the threshold and comfort value of the cab roll angle, and in combination with a predefined reward function, an immediate reward corresponding to a decision action is calculated.

[0087] Among them, we mainly focus on the vibration acceleration of the cab and the stability of the vehicle, and design the following predefined reward function:

[0088] r t =-k1×(aa threshold )+k2×(a comfort- a)-k3×(θ-θ threshold )+k4×(θ comfort -θ)

[0089] Where: a is the average value of the weighted root mean square value of the vertical vibration acceleration of the four suspension points of the cab. The weighted reference frequency range of 4-12.5 Hz, to which the human body is most sensitive to vertical vibration, is used. The real-time root mean square value of acceleration is calculated after 4-12.5 Hz bandpass filtering.

[0090] a threshold and a comfortThe threshold and comfort value of a are respectively. The reward is negative if it exceeds the threshold and positive if it is less than the comfort value. The threshold and comfort value refer to the 5-level comfort requirements of different models. The uncomfortable state is the threshold and the comfortable state is the comfort value.

[0091] θ is the roll angle of the cab threshold and θ comfort They are the threshold and comfort value of the roll angle. Similarly, the reward is negative if it exceeds the threshold, and positive if it is less than the comfort value. The threshold and comfort value of the roll angle are determined according to the handling requirements of different models. k1, k2, k3, k4 are positive constants used to adjust the weights of various indicators.

[0092] Step 200-24, record the state information of the current time step, the decision action of the current time step, the immediate reward obtained after executing the decision action of the current time step, and the state information of the next time step as the data point of the current time step. The data point of the next time step is obtained in the same way.

[0093] Above, step 200-3 specifically includes the following steps:

[0094] Step 200-31, for each data point, using the DQN network structure and the initial network weight parameters to calculate the predicted return reward estimate of the state information of the current time step, and the future reward value of the state information of the next time step;

[0095] Step 200-32, based on the first predefined function and in combination with the immediate reward and future reward value of each data point, calculate the actual return reward for each data point;

[0096] The first predefined function is: t =r t +γmaxQ(S t+1 ,a;w);

[0097] The actual reward is the sum of the immediate reward plus the expected discount on the value of future rewards.

[0098] Step 200-33, using the gradient descent algorithm, and combining the actual reward and the predicted reward valuation to update the network weight parameters, until the difference calculated by combining the updated network weight parameters with the DQN network structure meets the first condition or the first condition, and the predicted reward valuation meets the vehicle comfort requirements; the first condition is that the difference between the predicted reward valuation calculated by combining the updated network weight parameters with the DQN network structure and the actual reward is lower than the first preset threshold; the second condition is that the error between the difference obtained after the last update of the network weight parameters and the difference obtained after the last update of the network weight parameters is less than the second preset threshold; the preset performance indicators include the first condition, the second condition and the predicted reward valuation meeting the vehicle comfort requirements. The predicted reward valuation that meets the vehicle comfort requirements is different for each vehicle or model and needs to be preset in advance;

[0099] The loss function can be minimized by using the gradient descent algorithm to update the network weight parameters; this process is repeated multiple times until the preset performance index or convergence condition is reached, that is, the above data collection, storage, training and optimization process is continuously iterated until the performance of the model is stable and can produce behaviors that meet the vehicle comfort requirements.

[0100] In step 200-33, the difference between the mean square error of the actual reward obtained with the initial network weight parameters and the predicted reward estimate is calculated using a second predefined function; the second predefined function is: δ t =Q(S t ,a t ;w)-y

[0101] The gradient descent algorithm is: The gradient descent algorithm is used to iterate continuously to reduce the error value.

[0102] Step 200-34, forming the DQN model cab suspension control model with the final network weight parameters and the DQN network structure.

[0103] In a second aspect, the present application also proposes a cab suspension control system, which includes:

[0104] A sensor module is used to obtain the vehicle speed, the vertical vibration acceleration of the four suspension points of the cab, the cab roll angle, and the road surface photo in front of the vehicle during the vehicle's driving process, and form status information;

[0105] A controller module is used to input the state information obtained from the sensor module into the DQN model cab suspension control model, and then calculate multiple potential decision actions and actual reward rewards corresponding to the potential decision actions; analyze the actual reward rewards of multiple potential decision actions to output the optimal decision action;

[0106] An actuator module, which is used to control a suspension stiffness regulator and a suspension damping regulator according to an optimal decision action;

[0107] The communication module is used to transmit data between the controller module, the sensor module and the actuator module. The communication module ensures real-time data exchange between the modules and supports the efficient operation of the system.

[0108] The system collects environmental status information through the sensor module, uses the DQN model in the controller module to select the optimal suspension control strategy, and applies these strategies to the actual suspension system through the actuator module, thereby optimizing the comfort and stability of the cab.

[0109] Above, the sensor module is responsible for collecting environmental data, which will be used as input for DQN, including the following components:

[0110] Acceleration sensor: measures the acceleration of the cab suspension.

[0111] Camera: Captures images of the road ahead to identify road conditions.

[0112] Vehicle speed sensor: monitors the current vehicle speed.

[0113] The controller module contains a microprocessor, memory, and software that runs the DQN algorithm; the actuator module adjusts the state of the cab suspension according to the instructions issued by the controller, and it includes:

[0114] Spring Rate Adjuster: Controls the cab air spring pressure to adjust the air spring rate.

[0115] Shock absorber damping adjuster: adjusts the damping force of the cab shock absorber.

[0116] Through the above description, this application has the following effects:

[0117] (1) Optimize the performance of the suspension system through reinforcement learning technology, reduce cabin vibration, and improve driving comfort and vehicle stability.

[0118] (2) Multimodal input processing: A multimodal input processing method was designed, including image input and numerical input, which can comprehensively consider road conditions, vehicle speed, and suspension point vibration acceleration to improve control accuracy.

[0119] (3) Dynamic adjustment strategy: The system can dynamically adjust the suspension stiffness and damping according to real-time road conditions and driving conditions, thereby improving the system's adaptability and robustness.

[0120] (4) Efficient training algorithm: Experience replay and target network technology are used to improve the stability and efficiency of training and accelerate convergence.

[0121] (5) Comprehensive reward function: A reward function that comprehensively considers the vibration acceleration and roll angle of the cab is designed to comprehensively evaluate the performance of the system and guide the optimization direction.

[0122] Deploying the above cab suspension control system includes the following steps

[0123] Sensor Installation:

[0124] Install the accelerometer, camera, vehicle speed sensor, and height sensor in appropriate locations to ensure they can accurately collect the required data.

[0125] Controller configuration:

[0126] Install the operating system and necessary development environment on the microprocessor.

[0127] Load the pre-trained DQN model into memory.

[0128] Write the control logic code that will read the sensor data, calculate the actions through the DQN model, and send control commands to the actuators.

[0129] Communication Settings:

[0130] Configure the CAN bus etc. to ensure all modules can communicate with each other correctly.

[0131] Test the communication link to ensure that data is transmitted correctly.

[0132] Actuator connection:

[0133] Connect the spring rate adjuster and the shock absorber damping adjuster.

[0134] Calibrate actuators to ensure they work precisely as commanded.

[0135] System Integration Testing:

[0136] Perform system integration tests in a safe environment to verify that sensor data acquisition, DQN decision-making process, control command generation, and actuator response are as expected. Adjust parameters and perform multiple tests until the system performs stably and reliably.

[0137] Actual deployment:

[0138] Install the entire system on a real vehicle and perform final commissioning.

[0139] Monitor the performance of the system and perform further tuning as needed.

[0140] In a third aspect, an embodiment of the present application provides a cab suspension control device, which may be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0141] In the embodiment of the present application, the cab suspension control device may include a processor, a memory, a communication interface and a communication bus.

[0142] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0143] The communication interface includes input / output (I / O) interface, physical interface and logical interface, etc., which are used to realize the interconnection of devices inside the cab suspension control device, and the interface used to realize the interconnection between the cab suspension control device and other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.

[0144] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0145] The processor may be a general-purpose processor, which may call the cab suspension control program stored in the memory and execute the cab suspension control method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the cab suspension control program is called may refer to the various embodiments of the cab suspension control method of the present application, which will not be described in detail here.

[0146] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0147] The computer-readable storage medium of the present application stores a cab suspension control program, wherein when the cab suspension control program is executed by a processor, the steps of the cab suspension control method as described above are implemented.

[0148] Among them, the method implemented when the cab suspension control program is executed can refer to the various embodiments of the cab suspension control method of the present application, and will not be repeated here.

[0149] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0150] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0151] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0152] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0153] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0155] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A cab suspension control method, characterized in that: It includes: Acquire status information during vehicle driving; the status information includes vehicle speed, vertical vibration acceleration of four suspension points of the cab, cab roll angle, and photos of the road surface in front of the vehicle; Inputting the state information into the cab suspension control model of the DQN model to calculate multiple potential decision actions and actual reward of each potential decision action relative to the vertical vibration acceleration of the four suspension points of the cab and the cab roll angle; The actual return rewards of multiple potential decision actions are analyzed to output the optimal decision action; then, according to the optimal decision action, the spring stiffness and shock absorber damping of the four suspension points of the cab are adjusted.

2. The cab suspension control method according to claim 1, characterized in that: Training the DQN model cab suspension control model includes the following steps: Initialize the DQN network structure, network weight parameters and experience replay buffer of the DQN model cab suspension control model; Obtain a data sampling set and store it in an experience playback buffer; the data sampling set includes a plurality of data points distributed in chronological order; A batch of data points are randomly extracted from the experience replay buffer; then the DQN network structure is offline trained using the batch of data points, and the network weight parameters are updated until the DQN model cab suspension control model converges to a state that meets the preset performance indicators.

3. The cab suspension control method according to claim 2, characterized in that: The process of obtaining data points follows the following procedure: Using the state information of the current time step, a decision action is selected in combination with the ε-greedy strategy; Execute the selected decision action to adjust the spring stiffness and shock absorber damping of four suspension points of the cab, and then obtain the vertical vibration acceleration of the four suspension points and the roll angle of the cab after the selected decision action is executed; Calculating an instant reward based on the vertical vibration acceleration of the four suspension points and the roll angle of the cab in combination with a predefined reward function; The state information of the current time step, the decision action of the current time step, the immediate reward after executing the decision action of the current time step, and the state information of the next time step are recorded as the data points of the current time step.

4. The cab suspension control method according to claim 3, characterized in that: The instant reward is calculated according to the vertical vibration acceleration of the four suspension points and the roll angle of the cab in combination with a predefined reward function, including the following steps: Obtaining an average value of weighted root mean square values ​​of vertical vibration accelerations of four suspension points, as well as a threshold value and a comfort value of the average value of the weighted root mean square values; Obtaining the threshold and comfort value of the cab roll angle; Based on the average value of the weighted root mean square value, the threshold and comfort value of the average value of the weighted root mean square value, the cab roll angle, the threshold and comfort value of the cab roll angle, and in combination with a predefined reward function, an immediate reward corresponding to a decision action is calculated.

5. The cab suspension control method according to claim 3, characterized in that: Using a batch of the data points to train the DQN network structure and update the network weight parameters until the DQN model cab suspension control model converges to a state that meets the preset performance indicators, including the following steps: For each data point, the predicted return reward estimate of the state information of the current time step and the future reward value of the state information of the next time step are calculated using the DQN network structure and the initial network weight parameters; Based on the first predefined function and combining the immediate reward and future reward value of each data point, calculate the actual return reward for each data point; Using a gradient descent algorithm, and combining the actual reward and the predicted reward estimate, the network weight parameters are updated until the difference calculated by combining the updated network weight parameters with the DQN network structure satisfies the first condition or the second condition, and the predicted reward estimate meets the vehicle comfort requirement; The DQN model cab suspension control model is formed with the final network weight parameters and the DQN network structure.

6. The cab suspension control method according to claim 5, characterized in that: The first condition is that the difference between the predicted reward valuation calculated by combining the updated network weight parameters with the DQN network structure and the actual reward is lower than a first preset threshold; The second condition is that the error between the difference obtained after the last update of the network weight parameter and the difference obtained after the last update of the network weight parameter is less than a second preset threshold; The preset performance indicators include a first condition, a second condition and a predicted return reward valuation meeting vehicle comfort requirements.

7. The cab suspension control method according to claim 1, characterized in that: Parsing the actual rewards of multiple potential decision actions to output the optimal decision action includes the following steps: Calculate the actual reward for each potential decision action; The decision action with the largest actual reward is selected as the optimal decision action; the potential decision actions include adjustment values ​​of the spring stiffness of the four suspension points and the shock absorber damping adjustment values.

8. A cab suspension control system, characterized in that: It includes: A sensor module is used to obtain the vehicle speed, the vertical vibration acceleration of the four suspension points of the cab, the cab roll angle, and the road surface photo in front of the vehicle during the vehicle's driving process, and form status information; A controller module is used to input the state information obtained from the sensor module into the DQN model cab suspension control model, and then calculate multiple potential decision actions and actual reward rewards corresponding to the potential decision actions; analyze the actual reward rewards of multiple potential decision actions to output the optimal decision action; The actuator module is used for controlling the suspension stiffness adjuster and the suspension damping adjuster according to the optimal decision action.

9. A cab suspension control device, characterized in that: The cab suspension control device includes a processor, a memory, and a cab suspension control program stored in the memory and executable by the processor, wherein when the cab suspension control program is executed by the processor, the steps of the cab suspension control method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a cab suspension control program, wherein when the cab suspension control program is executed by a processor, the steps of the cab suspension control method according to any one of claims 1 to 7 are implemented.

Images