A brake smoothness control method for a line control brake system based on reinforcement learning

Abstract

The application provides a kind of based on reinforcement learning's line control brake system electromechanical compound brake smoothness control method, this method includes the following steps: obtaining the current operating state data of vehicle and constructing state space vector;State space vector is composed of sliding window state sequence, input to the reinforcement learning intelligent agent model pre-trained, output basic distribution proportion coefficient;Target total brake torque is analyzed into target motor brake torque and target line control friction brake torque;In the transition window period of dynamic handover of motor brake and line control brake, the dynamic error between target deceleration and actual longitudinal deceleration is calculated in real time;When dynamic error is greater than zero and exceeds preset dead zone threshold, generate transient compensation torque instruction and send to motor controller, so that the actual output torque of drive motor is the sum of target motor brake torque and transient compensation torque.The application realizes the continuous smooth transition of vehicle braking force in brake mode dynamic switching process.

Description

Technical Field

[0001] This invention relates to a control method for a brake-by-wire system, specifically a method for controlling the ride comfort of a brake-by-wire system based on reinforcement learning, belonging to the field of braking control technology for new energy vehicles. Background Technology

[0002] With the continuous evolution of new energy vehicle technology, electromechanical composite braking systems that combine regenerative braking of drive motors with hydraulic steerable friction braking have become one of the core technological paths to improve the energy utilization efficiency of the entire vehicle. However, under actual braking conditions, the drive motor responds to braking torque commands extremely quickly, with the establishment and adjustment of braking torque taking only tens of milliseconds; while the steerable hydraulic braking system, from receiving the electronic control command to the motor pump building pressure, the opening and closing of the hydraulic circuit, the caliper piston pushing the friction pads to clamp the brake disc, and finally to the actual formation of effective brake fluid pressure in the wheel cylinder, the entire physical pressure building process usually requires an incompressible physical delay of 100 to 200 milliseconds. This asymmetry in response speed means that during the transition window between electric motor braking and brake-by-wire, when the brake-by-wire system is in the physical pressure-building phase, if the regenerative braking torque from the electric motor has already begun to disengage according to the command, the total braking force of the vehicle will inevitably experience a momentary insufficiency. This will cause the actual longitudinal deceleration of the vehicle to suddenly fall below the driver's expected value, resulting in a noticeable interruption of braking force that can be perceived by the occupants. Conversely, if the disengagement of the electric motor torque lags behind the establishment of the brake-by-wire force, it will cause a brief overload of the total braking force, similarly inducing a sudden and impactful change in longitudinal acceleration. The aforementioned problems faced by the electromechanical composite braking system seriously impair ride comfort and pose a potential threat to driving safety under extreme conditions.

[0003] In existing technologies, solutions to the aforementioned problems mainly fall into two categories. The first category is the traditional rule-based calibration control method. This typically involves engineers pre-calibrating fixed torque transition timing and ramp curve parameters under specific operating conditions, then superimposing a fixed feedforward compensation term based on the hysteresis of the brake-by-wire system to bridge the time gap during braking force switching. This method relies on offline calibration lookup logic, and while its control effect is acceptable under calibration conditions, it is essentially a static fit to a specific scenario. It cannot flexibly handle dynamic changes in driver pedal input intensity, real-time differences in road surface adhesion conditions, the constraint of the battery's state of charge on the motor's torque capability, and complex dynamic factors such as the drift in hydraulic system response characteristics due to temperature changes. Its robustness is severely insufficient under non-calibrated conditions, and significant braking force fluctuation risks remain under extreme conditions. The second category introduces purely data-driven deep reinforcement learning or deep neural network algorithms. Their advantage lies in achieving strong generalization capabilities through training on massive amounts of data. However, due to the uninterpretability of its decision-making process, pure deep neural networks cannot provide verifiable safety guarantees for physical hard limits such as the maximum torque capacity of the motor and the maximum regenerative power of the battery. The safety boundary of its output action depends on the coverage of the training data, which may lead to the output of dangerous commands under distributed operating conditions. It is difficult to meet the fundamental constraints of verifiable and analyzable functional safety requirements for the highest safety integrity level of the chassis control system as required by automotive functional safety standards. Therefore, there are significant certification and engineering obstacles at the engineering implementation level. Summary of the Invention

[0004] Based on the above background, the purpose of this invention is to provide a method for smooth control of electromechanical composite braking in a brake-by-wire system based on reinforcement learning. This method achieves a continuous and smooth transition of the vehicle's braking force during dynamic switching of braking modes. Under the premise of establishing a hard physical boundary for functional safety, it fully utilizes the energy recovery potential of regenerative braking by the motor. This solves the technical problems in the prior art where the asymmetry of response speed in the transition between motor braking and brake-by-wire systems causes instantaneous fluctuations in total braking force, inducing sudden changes in vehicle longitudinal acceleration, and the lack of robustness of traditional calibration control algorithms under all operating conditions and the inability of pure data-driven algorithms to meet the verifiability requirements of functional safety standards.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0006] A method for smooth braking of a brake-by-wire system based on reinforcement learning, comprising the following steps:

[0007] Acquire the current operating status data of the vehicle and construct a state space vector, which includes the current vehicle speed, brake pedal depth change rate, power battery state of charge, actual longitudinal deceleration, and target total braking torque.

[0008] The state space vectors containing the current time and the past preset time steps are used to form a sliding window state sequence, which is then input into a pre-trained reinforcement learning agent model. The basic allocation ratio coefficient is output through the reinforcement learning agent model.

[0009] Based on the aforementioned basic allocation ratio coefficient, the target total braking torque is analyzed into the target motor braking torque and the target steer-by-wire friction braking torque.

[0010] During the transition window period when the motor braking and the brake-by-wire braking dynamically change, the dynamic error between the target deceleration and the actual longitudinal deceleration is calculated in real time.

[0011] When the dynamic error is greater than zero and exceeds the preset dead zone threshold, a transient compensation torque command is generated and sent to the motor controller so that the actual output torque of the drive motor is the sum of the target motor braking torque and the transient compensation torque.

[0012] As the braking force of the brake-by-wire system gradually builds up, the transient compensation torque is controlled to decay and exit according to an exponential smooth decay function.

[0013] The reinforcement learning agent model includes an Actor network and a Critic network. The output of the Actor network is connected in series with a differentiable dynamic safety projection layer. The dynamic safety projection layer is configured to obtain the maximum allowable motor braking ratio under the current physical limit, and compare the original allocation ratio output by the Actor network with the maximum motor braking ratio, taking the smaller value of the two as the final output basic allocation ratio coefficient.

[0014] Preferably, the input end of the reinforcement learning agent model is provided with a gated recurrent unit feature extraction layer; the step of forming a sliding window state sequence from the state space vectors containing the current time and the past preset time steps, and inputting it into the pre-trained reinforcement learning agent model, specifically includes:

[0015] The sliding window state sequence is input into the feature extraction layer of the gated loop unit to extract the hidden state vector containing the physical pressure build-up trend of the brake-by-wire system;

[0016] The hidden state vector is input into the Actor network and the Critic network for action evaluation and output.

[0017] Preferably, the maximum allowable motor braking ratio under the current physical limits is obtained in real time by calculating the maximum allowable regenerative power limit of the battery broadcast in real time by the chassis domain controller and the external characteristic curve of the drive motor at the current speed.

[0018] Preferably, the reinforcement learning agent model employs a multi-objective fusion reward function during the training phase. The reward function includes a ride comfort penalty term, which is configured to calculate the derivative of the vehicle's longitudinal acceleration in real time as an impulsivity value. When the impulsivity value is greater than a preset comfort threshold, an exponentially amplified negative penalty is applied to the reward function.

[0019] Preferably, the mathematical expression used for the reward function of the multi-objective fusion is:

[0020] ;

[0021] in, For the reward function of multi-objective fusion, The tracking reward is calculated based on the error between the target deceleration and the actual longitudinal deceleration. The energy reward is calculated based on the actual recovered power of the drive motor. For the smoothness penalty term, To switch the jitter penalty term, the jitter penalty term is used to penalize the high-frequency oscillations of the base allocation ratio coefficient in adjacent time steps. All are weighting coefficients.

[0022] Preferably, the mathematical expression used to control the transient compensation torque to decay and exit according to an exponential smooth decay function is as follows:

[0023] ;

[0024] in, for Transient compensation torque at time t, The initial maximum transient compensation torque, The attenuation coefficient;

[0025] The attenuation coefficient The dynamic adjustment is based on the current pressure build-up rate of the brake-by-wire system, and the mathematical expression used is as follows:

[0026] ;

[0027] in, This refers to the real-time pressure change rate of the brake-by-wire system. This is the calibration gain coefficient.

[0028] Preferably, the method further includes the following steps:

[0029] Real-time monitoring of the vehicle chassis stability;

[0030] When the Anti-lock Braking System (ABS), Electronic Stability Control (ESC), or the current road surface adhesion coefficient is detected to be triggered, the output of the reinforcement learning agent model is forcibly bypassed, the basic allocation ratio coefficient is forcibly set to zero, and the transient compensation torque is cleared, so that the braking force is provided independently by the brake-by-wire system.

[0031] Preferably, the current vehicle speed is obtained through a wheel speed sensor or a vehicle controller local area network bus, the brake pedal depth change rate is obtained by acquiring the pedal displacement through an electronic pedal displacement sensor and performing differential processing, and the actual longitudinal deceleration is obtained through an inertial measurement unit.

[0032] Preferably, when the dynamic error is greater than zero and exceeds a preset dead zone threshold, a transient compensation torque command is generated and sent to the motor controller, specifically including:

[0033] When the dynamic error is detected to be greater than zero and exceeds the preset dead zone threshold, it is determined that the brake-by-wire system is in the physical pressure build-up delay stage, resulting in insufficient total braking force.

[0034] The power battery is instantaneously discharged, controlling the drive motor to break through the steady-state distribution limit and output the positive transient compensation torque.

[0035] Preferably, the reinforcement learning agent model is constructed based on the SAC algorithm; during the training process of the reinforcement learning agent model, its optimization objective is to maximize the sum of cumulative reward and policy entropy.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] This invention discloses a reinforcement learning-based electromechanical hybrid braking smoothness control method for brake-by-wire systems. By introducing a gated recurrent unit temporal memory mechanism into the reinforcement learning agent model, the controller can predict the pressure build-up trend of the brake-by-wire system, overcoming the adverse effects of brake-by-wire physical hysteresis on braking smoothness. Through the synergistic effect of transient compensation torque based on error gradient and exponential smooth decay function, it actively fills the braking force gap during the transition window and smoothly exits without disturbance, suppressing abrupt changes in longitudinal impulse during braking mode switching. By embedding a differentiable dynamic safety projection layer at the Actor network output, functional safety constraints are embedded into the decision link end of the reinforcement learning agent model in the form of mathematically differentiable operators. This does not interfere with the end-to-end gradient optimization training of the policy network and provides rigid protection for system safety, thereby obtaining the full-condition adaptive advantage of reinforcement learning algorithms while establishing a hard physical safety boundary that meets high-level functional safety standards. Through a chassis limit arbitration mechanism, it provides the highest priority safety degradation guarantee for the entire hybrid braking system, ensuring the absolute reliability of the braking system under extreme adhesion conditions. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the control flow of a reinforcement learning-based electromechanical composite braking smoothness control method for a brake-by-wire system according to the present invention.

[0040] Figure 2 This is a schematic diagram of the network architecture of the reinforcement learning agent model in this invention;

[0041] Figure 3 This is a schematic diagram of the transient compensation torque generation and exponential decay exit control logic during the transition window period in this invention;

[0042] Figure 4 This is a schematic diagram of the logical architecture of the chassis limit arbitration mechanism in this invention. Detailed Implementation

[0043] The technical solution of the present invention will be further described in detail below through specific embodiments and in conjunction with the accompanying drawings. It should be understood that the implementation of the present invention is not limited to the following embodiments, and any modifications and / or alterations made to the present invention will fall within the protection scope of the present invention.

[0044] In this invention, unless otherwise specified, all parts and percentages are by weight, and the equipment and raw materials used are commercially available or commonly used in the art. Unless otherwise specified, the methods in the following embodiments are conventional methods in the art. Unless otherwise specified, the components or equipment in the following embodiments are general standard parts or components known to those skilled in the art, and their structures and principles can be learned by those skilled in the art through technical manuals or conventional experimental methods.

[0045] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. In this detailed description, numerous specific details are set forth to facilitate explanation and provide a thorough understanding of the embodiments of the present invention. However, one or more embodiments may be practiced by those skilled in the art without these specific details.

[0046] like Figure 1As shown, an embodiment of the present invention discloses a method for smooth braking of an electromechanical composite braking system based on reinforcement learning. This method includes the following control steps in a sequential logical progression: multi-dimensional vehicle state feature perception and acquisition, dynamic allocation of basic torque based on improved reinforcement learning, transient active compensation control based on error gradient, and finally, execution of electromechanical composite braking. Specifically, it includes the following steps:

[0047] Acquire the vehicle's current operating status data and construct a state space vector, which includes the current vehicle speed, brake pedal depth change rate, power battery state of charge, actual longitudinal deceleration, and target total braking torque.

[0048] The state space vectors containing the current time and the past preset time steps are used to form a sliding window state sequence, which is then input into a pre-trained reinforcement learning agent model. The reinforcement learning agent model outputs the basic allocation ratio coefficient.

[0049] Based on the basic allocation ratio coefficient, the target total braking torque is analyzed into the target motor braking torque and the target steer-by-wire friction braking torque.

[0050] During the transition window period when the motor braking and the brake-by-wire braking dynamically change, the dynamic error between the target deceleration and the actual longitudinal deceleration is calculated in real time.

[0051] When the dynamic error is greater than zero and exceeds the preset dead zone threshold, a transient compensation torque command is generated and sent to the motor controller so that the actual output torque of the drive motor is the sum of the target motor braking torque and the transient compensation torque.

[0052] As the braking force of the brake-by-wire system gradually builds up, the transient compensation torque decreases and exits according to an exponentially smooth decay function.

[0053] Furthermore, to ensure the chassis stability control system operates independently with the highest priority under extreme conditions approaching the physical adhesion limit, providing ultimate safety degradation protection for the entire electromechanical composite braking system, this method also includes a chassis limit arbitration mechanism with the highest priority. Specifically, the stability status of the vehicle chassis is monitored in real time; when the Anti-lock Braking System (ABS), Electronic Stability Control (ESC), or the current road surface adhesion coefficient is detected to be triggered, the output of the reinforcement learning agent model is forcibly bypassed, the basic distribution ratio coefficient is forcibly set to zero, and the transient compensation torque is cleared, allowing the brake-by-wire system to independently provide braking force, that is, the braking power is completely transferred to the brake-by-wire system for independent operation.

[0054] The complete technical solution of the present invention will be described in detail below, combining the internal mechanisms and interrelationships of each step.

[0055] The control system collects vehicle operating status data in real time through various sensors and an on-board communication bus at a fixed sampling period, and converts the collected multidimensional status variables into data in each control cycle. Each moment is constructed as a state space vector It is represented as follows:

[0056] ;

[0057] in, Current vehicle speed, in meters per second; The brake pedal depth change rate is the first derivative of the pedal displacement signal with respect to time, measured in millimeters per second, reflecting the speed and intensity of the driver's action of pressing the brake pedal. The current state-of-charge percentage of the power battery and the corresponding maximum allowable regenerative power constraint are calculated in real time by the battery management system and broadcast through the controller area network bus. The actual longitudinal deceleration of the vehicle is measured directly by the inertial measurement unit along the longitudinal axis of the vehicle, and the unit is meters per square second. The target total braking torque for the entire vehicle is calculated in real time by the vehicle controller based on the driver's pedal displacement and pedal force, and is measured in Newton-meters. These five components comprehensively cover the key decision-making information required for optimal braking torque distribution from three dimensions: driver intent, vehicle motion state, and energy constraints.

[0058] Regarding the specific data collection methods for each component, the current vehicle speed The vehicle speed signal is obtained either by directly measuring the rotational speed of each wheel using wheel speed sensors and then calculating it, or by obtaining a fused vehicle speed signal from the chassis domain controller via the vehicle controller's local area network bus. Both methods provide redundancy and backup to improve the reliability of the speed signal source. Brake pedal depth change rate. The actual longitudinal deceleration is obtained by numerically differentiating the pedal displacement data collected by an electronic pedal displacement sensor. Outputted directly from the inertial measurement unit. Current vehicle speed. Simultaneously, as a prerequisite input for subsequent dynamic safety projection layer queries of the motor's external characteristic curve, the state of charge of the power battery... The corresponding maximum allowable feedback power directly constrains the upper limit of the maximum motor braking ratio that the dynamic safety projection layer can permit, ensuring that the allocation strategy always operates within the physical energy constraint boundary of the battery in the full state of charge range.

[0059] like Figure 2 As shown, the control system will include the current time. and past preset time steps The state space vectors at each time step are arranged in chronological order to form a sliding window state sequence. It is represented as:

[0060] ;

[0061] in, The preset time steps are determined by comprehensively considering the typical physical pressure build-up hysteresis time of the brake-by-wire system (usually 100-200 milliseconds) and the sampling period of the control system, to ensure that the sliding window fully covers the complete physical process of brake-by-wire from command issuance to stable braking force establishment. The entire sliding window state sequence is fed into a pre-trained offline reinforcement learning agent model for inference calculation, and finally outputs the basic allocation ratio coefficient.

[0062] Using a sliding window state sequence Instead of a single-frame state snapshot As model input, this is because the physical pressure build-up process of the brake-by-wire system within 100-200 milliseconds is a dynamic process with clear temporal evolution characteristics. It involves multiple consecutive physical stages, including command issuance, pump and motor start-up and pressure build-up, gradual opening of the hydraulic circuit, gradual increase in wheel cylinder pressure, and finally, the gradual establishment of caliper clamping force. Each stage retains different temporal characteristics in observable measurements such as vehicle longitudinal deceleration and motor torque response mode. If only a single frame state is considered... The controller cannot distinguish between the two fundamentally different system states: the brake-by-wire just beginning to build up voltage and the brake-by-wire having completed voltage buildup. Therefore, it cannot make differentiated allocation decisions. However, by inputting a series of historical state frames through a sliding window, the controller can identify the precise voltage buildup stage of the current brake-by-wire system from the timing characteristics. This allows it to proactively adjust the motor braking ratio in advance, achieving predictive compensation for physical hysteresis.

[0063] The reinforcement learning agent model incorporates a gated recurrent unit feature extraction layer at its input. (Sliding window state sequence) First, the data is fed into the gated recurrent unit (GRU) feature extraction layer for temporal feature extraction and encoding. The GRU is a recurrent neural network unit with a gated adaptive memory mechanism, internally incorporating both update and reset gates. Let the first GRU be... At each time step, the input to the gated loop unit is... The previous hidden state was Then update the door. Reset door With the current hidden state The recurrence relation satisfies:

[0064]

[0065] ;

[0066] in, It is the sigmoid activation function; It is the hyperbolic tangent activation function; This indicates element-wise multiplication; , , and , , These are the input weight matrix and hidden state weight matrix calculated for the update gate, reset gate, and candidate state, respectively. , , For the corresponding bias vector, all the above parameters were optimized and determined during the offline training phase using the backpropagation algorithm. Update gate Controls the percentage of currently input information that is incorporated into the hidden state, and resets the door. By controlling the proportion of the historical hidden state that was forgotten in the previous moment, the gated loop unit effectively alleviates the problem of long sequence gradient vanishing through the above-mentioned gating mechanism. It can selectively memorize key dynamic features within a relatively long time span of 100 to 200 milliseconds, and is particularly suitable for capturing the slowly changing trend features in the hydraulic pressure building process.

[0067] In the application scenario of this invention, the gated loop unit feature extraction layer sequentially performs the above-mentioned loop processing on the state space vector at each time step within the sliding window, and then processes the last time step... Output hidden state vector The compressed, encoded representation of the temporal features of the entire sequence is fed into the downstream network. Hidden state vector It is not simply equal to the state of the last frame. It is not a feature representation, but a high-dimensional abstract summary of the dynamic evolution law of the system revealed by the entire historical time series, among which, particularly important, is the quantitative expression of the process-predictive feature of the current physical pressure build-up trend of the linear braking system. The hidden state vector extracted through the gated recurrent unit... The reinforcement learning agent model can make torque allocation decisions by predicting the current pressure build-up stage based on history, rather than relying on static mapping reasoning without time sequence memory based on a pure current state snapshot.

[0068] Hidden state vector The data is then synchronously fed into the Actor network and the Critic network. The Actor network is a feedforward deep neural network responsible for receiving... It also outputs the torque distribution action at the current moment, specifically, it outputs an original distribution ratio coefficient representing the desired motor braking ratio. ,satisfy The Critic network is also a feedforward deep neural network, responsible for receiving... The Actor network outputs actions to evaluate the long-term cumulative expected return of the current state-action pair and outputs a state-action value estimate, which provides a value guidance signal for the policy gradient update of the Actor network. The two achieve iterative and collaborative optimization of policy evaluation and policy improvement through the actor-critic framework.

[0069] A differentiable dynamic safety projection layer is connected in series at the output of the Actor network, which is one of the core innovations of this invention in terms of intelligent agent architecture design. The working mechanism of the dynamic safety projection layer is that the chassis domain controller continuously broadcasts the maximum allowable regenerative power limit value of the battery fed back by the current battery management system in real time through the controller area network bus. Meanwhile, the dynamic safety projection layer is based on the current vehicle speed. The converted drive motor speed Query the motor's external characteristic curve data to determine the current speed. The physical limit of the maximum braking torque that the lower motor can output Taking into account the two physical constraints mentioned above, calculate the maximum allowable motor braking ratio under the current physical limits. Its computational logic is expressed as follows:

[0070] ;

[0071] in, The current angular velocity of the drive motor is determined by the vehicle speed. It is calculated by converting to the transmission ratio, and the unit is radians per second; This represents the maximum regenerative power currently allowed by the battery management system, in watts. To the current motor speed The maximum braking torque is determined by the external characteristic curve, and the unit is Newton-meter. The target total braking torque is expressed in Newton-meters. It is not a fixed constant calibrated offline, but is dynamically calculated based on two real-time changing physical parameters: battery state of charge and motor speed. Therefore, it always reflects the actual physical safety boundary across the entire operating range.

[0072] Subsequently, the dynamic security projection layer assigns the original scaling factor to the Actor network output. The upper limit of the maximum motor braking ratio calculated in real time above Perform the following projection operation to obtain the final output base allocation ratio coefficient. :

[0073] ;

[0074] in, For Actor networks based on hidden state vectors The output of the original allocation ratio coefficient; The maximum motor braking ratio is calculated in real time from the battery's maximum allowable regenerative power and the motor's external characteristic curve; The basic distribution ratio coefficient, after being processed by safety projection, is finally output to the torque analysis unit, satisfying... The key engineering characteristic of this pointwise underfimum operation is that when When the layer is transparent, it transmits the output of the Actor network, and the gradient can be completely propagated back to the Actor network parameters; while when... At that time, the layer will truncate the output to the safe limit. Even at the truncation point, a subgradient exists, and the gradient signal can still be transmitted to the Actor network to a certain extent, guiding it to learn not to output policy tendencies that exceed physical limits in high allocation ratios. This differentiability allows the entire policy network with safety constraints to undergo end-to-end gradient optimization training, overcoming the problem of traditional hard constraint methods where non-differentiable conditional branches are independent of the network, thus severing the policy training and safety constraint feedback paths. In particular, regardless of the reinforcement learning policy output... After being processed by the dynamic security projection layer, it is transmitted downstream. It will never exceed the hard safety boundary determined by the physical state of the battery and the external characteristics of the motor at any time, thus eliminating the possibility of the intelligent agent issuing motor braking torque commands that exceed the physical capacity limit.

[0075] After processing by the dynamic safety projection layer, the basic allocation ratio coefficient The torque is fed into the torque analysis and distribution unit to determine the target total braking torque. Perform the following linear analysis:

[0076] ;

[0077] in, The target motor braking torque is expressed in Newton-meters. The motor controller drives the motor to perform regenerative braking based on this instruction. The target friction braking torque is measured in Newton-meters. The brake-by-wire actuator drives the hydraulic main pump to build up pressure to apply the clamping force of the caliper according to this instruction. The basic allocation ratio coefficient output through the dynamic safety projection layer satisfies And always Under steady-state operating conditions, the braking force requirements of the entire vehicle are strictly ensured to be met.

[0078] For the offline training mechanism of reinforcement learning agent models, this invention employs a soft actor-critic algorithm to construct the reinforcement learning agent model and perform offline training. The core optimization objective of the soft actor-critic algorithm is to maximize the policy's information entropy while maximizing the policy's cumulative discount reward, i.e.:

[0079] ;

[0080] in, This is the optimal strategy; Discount factor; The immediate reward for the current moment; For the strategy in the current hidden state Information entropy under the given conditions is defined as follows: ; The temperature parameter is automatically updated through an adaptive adjustment mechanism within the soft actor-critic algorithm framework. The introduction of the information entropy maximization objective allows the agent to traverse the braking torque allocation policy space with greater breadth of exploration in the early stages of training, avoiding premature trapping in suboptimal local policies. In the later stages of training, the temperature parameter... By adaptively adjusting automatic decay, the policy gradually converges towards high-confidence, high-quality allocation decisions, exhibiting better generalization robustness across all operating conditions compared to deterministic policy algorithms.

[0081] The reinforcement learning agent model employs a customized reward function that integrates multiple objectives during the training phase, at each training time step. Comprehensive instant reward value Satisfy the following formula:

[0082] ;

[0083] in, To track rewards, based on target deceleration Compared with actual longitudinal deceleration The absolute value of the error between them is calculated; the smaller the error, the better. The larger the value, the greater the ability of the intelligent agent to learn and accurately track braking intentions, which can be defined as... Or its normalized form; The energy reward is based on the actual mechanical power recovered by the drive motor at the current moment. Calculate, satisfy ,in, To normalize the reference maximum recovery power, the agent is continuously stimulated to maximize energy utilization efficiency without sacrificing braking effect; As a ride comfort penalty, it is based on the vehicle's longitudinal acceleration. Regarding the calculation of the derivative with respect to time, when the impulse value Exceeding the preset comfort threshold When an exponentially amplified negative penalty is applied, its penalty function can be defined as:

[0084] ;

[0085] in, To determine the penalty amplification factor and control the exponential growth rate of the penalty intensity as the impulse exceeds the limit, calibration is required in conjunction with the subjective comfort evaluation criteria of the target vehicle model. This non-linear penalty design means that even slight acceleration abrupt changes will incur additional penalties far exceeding those of conventional tracking errors, driving the agent to prioritize smooth braking at the policy level. To switch the jitter penalty item, used to penalize the base allocation ratio coefficient. High-frequency oscillations between adjacent time steps can be defined as Or its power form, to guide the agent to output a smooth and continuous torque distribution strategy curve; , , , These are all weighting coefficients for the corresponding components. The specific values ​​of each weighting coefficient are jointly calibrated and optimized based on the target vehicle's dynamic parameters, energy recovery strategy objectives, and subjective evaluation standards for ride comfort.

[0086] Through the above-mentioned multi-objective fusion reward function design, the trained reinforcement learning strategy can achieve a dynamic balance between braking effect, energy recovery efficiency and braking smoothness, and receive a strong negative feedback signal when each braking mode switch causes a sudden acceleration change, so that it can eventually learn a predictive allocation strategy that can actively avoid braking force fluctuations during the handover transition period.

[0087] While temporal memory can predict and reduce braking force gaps caused by pressure build-up hysteresis in linear braking to some extent, due to the complexity and variability of real physical systems, predictive allocation based on learning strategies cannot eliminate braking force gaps with zero error under all operating conditions. Therefore, as... Figure 3 As shown, transient active compensation control is performed during the transition window period to measure longitudinal deceleration. As a closed-loop feedback quantity, it actively fills the remaining braking force gap after the dynamic distribution of the basic torque in real time.

[0088] When the control system detects a dynamic torque handover event between motor braking and brake-by-wire, it enters the dynamic handover transition window and simultaneously activates the transient compensation control logic. During the transition window, the control system calculates the target deceleration in real time with a high-frequency sampling period. Compared with actual longitudinal deceleration Dynamic error between Its definition is:

[0089] ;

[0090] in, The target deceleration currently analyzed by the vehicle controller is expressed in meters per square second, representing the desired braking effect corresponding to the driver's pedal operation intention. The actual longitudinal deceleration measured in real time by the inertial measurement unit is expressed in meters per square second, representing the vehicle's current true braking response. The difference between the two, expressed in meters per square second, quantifies the degree of inadequacy of the total braking force relative to the target value at the current moment. When this occurs, it indicates that the actual deceleration is lower than the target deceleration, meaning there is a shortfall in the total braking force. When this occurs, it indicates that the actual braking force has met or exceeded the demand, and there is no need to activate the compensation logic.

[0091] To avoid false triggering of the compensation logic due to non-physical braking force gaps such as measurement noise from the inertial measurement unit and vibration disturbances caused by minor road surface irregularities, this invention sets a preset dead zone threshold. Mechanism, only if the dynamic error satisfies Only then was it determined that the current insufficient total braking force was caused by the physical pressure build-up delay of the brake-by-wire system, thus activating the transient compensation torque generator. Dead zone threshold. The magnitude can be calibrated based on the trade-off between noise suppression and compensation response sensitivity. Typically, the deceleration value corresponding to a noise standard deviation that can effectively filter the sensor's conventional noise amplitude is selected as the reference starting point.

[0092] Once the compensation trigger condition is met. The control system coordinates with the battery management system through the chassis domain controller to utilize the instantaneous discharge power margin that can be supported under the current battery state of charge, and sends a signal to the motor controller that is superimposed on the target motor braking torque. Positive transient compensation torque above At this time, the actual output braking torque of the drive motor satisfy:

[0093] ;

[0094] in, The actual output braking torque of the drive motor during the transition window period, expressed in Newton-meters; The target motor braking torque output by the reinforcement learning agent model is expressed in Newton-meters. This is a transient compensation torque, measured in Newton-meters, and directed in the positive direction (i.e., the direction of increasing braking force). It is used to temporarily fill the gap in total braking force before the online control brakes complete pressure build-up. The initial value is based on the current dynamic error. Vehicle equivalent mass Parameters are calculated and determined to ensure that the initial amplitude of the compensation response matches the magnitude of the braking force gap, avoiding excessive initial compensation that could introduce reverse deceleration overshoot. Utilizing the extremely fast torque response speed of the drive motor at the millisecond level, transient compensation torque... It can almost in real time fill the braking force gap caused by the brake-by-wire pressure build-up phase with the motor torque, ensuring that the vehicle's longitudinal deceleration remains close to the target value and runs smoothly throughout the entire transition window.

[0095] As the pressure in the hydraulic circuit of the brake-by-wire system continues to build up, the brake-by-wire power gradually increases and approaches its steady-state target value. To prevent the combined overload of total braking force caused by the simultaneous action of the increase in brake-by-wire power and the motor compensation torque, this invention designs an exponentially smooth decay exit mechanism that is physically linked to the brake-by-wire voltage build-up rate. During the compensation exit phase, the transient compensation torque... It decays over time according to an exponentially smooth decay function, satisfying the following mathematical model:

[0096]

[0097] in, For the first time since the compensation trigger time The transient compensation torque value per second, expressed in Newton-meters; The initial maximum transient compensation torque, measured in Newton-meters, is determined based on the current dynamic error at the moment of logic activation and represents the initial amplitude of the exponential decay function. This is the attenuation coefficient, measured per second. The larger the value, the faster the compensation torque attenuates and the quicker the exit. The relative duration accumulated since the compensation trigger moment, in seconds; It is a natural constant. The exponential function form guarantees that... In theory, it asymptotically approaches zero in a smooth and continuous manner, without introducing a sudden change in torque at the end of the exit phase.

[0098] Attenuation coefficient It is not a fixed calibration constant, but is dynamically adjusted according to the current real-time pressure change rate of the brake-by-wire system, satisfying the following formula:

[0099] ;

[0100] in, This represents the adaptive decay coefficient at the current moment, expressed in seconds. The real-time pressure change rate of the master cylinder or wheel cylinder of the brake-by-wire system is obtained by numerically differentiating the sampled values ​​of the wheel cylinder pressure sensor or master cylinder pressure sensor with time, and directly reflects the actual growth rate of the brake-by-wire power at the current moment. To calibrate the gain coefficient, the unit is . Their dimensions are inverse, making The dimensions are expressed in seconds. The specific values ​​are determined through joint calibration of bench tests and real vehicle road tests, based on the pressure-torque conversion characteristics of the hydraulic braking system of the target vehicle model, the dynamic characteristics of the motor torque response, and the subjective driving evaluation targets of the whole vehicle.

[0101] when At higher pressure levels (e.g., when a driver applies emergency braking with significant force, the hydraulic pump motor operates at high speed, resulting in a higher pressure build-up rate), To obtain a larger value, The rate of exponential decay accelerates, and the compensating torque smoothly exits at a rate matching the rapid increase in linear control power, resulting in a stable total braking force. When At smaller times (e.g., when the hydraulic system experiences a decrease in pressure build-up rate due to increased oil viscosity caused by low temperature, or when the driver applies gentle braking, causing the pump motor to build up pressure at a low speed). To obtain a smaller value, Withdrawal should be gradual to avoid premature and significant removal of motor compensation support before sufficient online control power has been established, which could create a new braking force gap. This is achieved by adjusting the attenuation coefficient. Real-time pressure change rate of the brake-by-wire system Direct physical bonding enables transient torque compensation under any operating condition. The exit curve can be dynamically synchronized with the actual establishment curve of the brake-by-wire power on the time axis. The total braking force formed by the sum of the two remains smooth and continuous throughout the entire transition window period, thereby eliminating sudden changes in the degree of impulsiveness during the braking mode switching process.

[0102] like Figure 4 As shown, the chassis limit arbitration mechanism has the highest execution weight in the priority architecture of the entire control system. Its judgment result can forcibly cover and bypass the output of the basic torque dynamic distribution step and the transient active compensation step, providing an unavoidable safety degradation bottom line for the entire electromechanical composite braking control scheme.

[0103] Specifically, at a sampling rate no less than the main cycle frequency of the control system, three types of chassis stability status information broadcast from the chassis domain controller are continuously monitored in real time: the current activation status flag of the anti-lock braking system, the current activation status flag of the electronic stability program, and the current road adhesion coefficient estimated in real time by the chassis domain controller based on wheel speed differences and lateral acceleration. .

[0104] When the chassis limit arbitration module detects that any of the following judgment conditions are met, it immediately triggers a forced degradation protection action.

[0105] The first trigger condition is the activation of the anti-lock braking system (ABS). ABS activation means that the braking slip ratio of at least one wheel has exceeded a safe threshold, and that wheel is in a critical state approaching lock-up. In this situation, any additional braking torque from the drive motor will further increase the wheel's slip ratio, exacerbating the risk of wheel lock-up and interfering with the ABS's fine-tuning logic for adjusting braking pressure on each wheel.

[0106] The second trigger condition is the activation of the electronic stability program. The activation of the electronic stability program means that the vehicle is currently facing a risk of lateral instability, and chassis stability control should be completely transferred to the electronic stability program, which will coordinate the distribution of braking force to each wheel to restore vehicle stability. The independent torque distribution output of the electromechanical composite brake controller may interfere with the active intervention of the electronic stability program.

[0107] The third triggering condition is the current road surface adhesion coefficient. Below the set threshold In low-traction road conditions such as snow and ice cover, and slippery surfaces, the maximum adhesion that tires can provide is significantly reduced. The adhesion margin assumption on which the torque distribution strategy dominated by the reinforcement learning agent is based no longer holds, and continuing to let the agent dominate the distribution may lead to exceeding the road adhesion limit.

[0108] Once any of the above conditions are triggered, the chassis limit arbitration module immediately issues a forced zeroing command to the torque analysis and distribution unit, transient compensation torque generator, and motor controller to perform the following operations: reset the basic distribution ratio coefficient. Forced to zero, while simultaneously reducing transient compensation torque. Synchronous zeroing ensures that the regenerative braking torque of the drive motor immediately returns to zero. At this point, the target drive-by-wire friction braking torque satisfies... All braking force requirements are transferred to the brake-by-wire system for independent operation. Under the direct control of the anti-lock braking system or electronic stability program, the brake-by-wire system independently exercises braking authority. All outputs of the reinforcement learning agent model are completely bypassed, ensuring that the active intervention of the chassis stability safety system is not interfered with by the compound braking intelligent distribution logic.

[0109] If both the anti-lock braking system and the electronic stability program are deactivated, and the road adhesion coefficient is estimated to be... Restore to the set threshold The above, and maintain a stable judgment lag time for a period of time. If the extreme operating condition is lifted, the chassis limit arbitration module will release the forced degradation state, and the control system will gradually reduce the load using a preset smooth ramp method. The system recovers from zero to the normal output value of the reinforcement learning agent model, avoiding the introduction of new torque step changes by the control mode switching itself.

[0110] In this invention, the gated loop unit feature extraction layer extracts features from the historical state sequence of the sliding window. The extraction of hidden linear control impulse build-up timing features endows the Actor network with timing-aware look-ahead predictive capabilities. The Actor network can thus learn to proactively adjust the original output allocation coefficients when typical initial transition features appear in the sliding window. This proactively compresses the magnitude of the braking force gap that needs to be actively filled, allowing subsequent transient active compensation steps to quickly address only the smaller residual gap, thus reducing the transient compensation torque. Required response amplitude and initial value The magnitude.

[0111] Dynamic security projection layer for original allocation scaling factor implement Real-time constraint projection ensures that regardless of the strategy output by the Actor network, the torque resolution unit receives the correct response. Never exceed and The jointly determined physical safety boundary, while preserving the feasibility of end-to-end gradient optimization training, establishes a rigid safety boundary for the entire intelligent allocation framework, making up for the deficiencies of pure data-driven algorithms in terms of verifiable functional safety.

[0112] Attenuation coefficient Compensating torque The exit dynamic characteristics are physically coupled with the actual pressure build-up dynamic characteristics of the brake-by-wire system, ensuring that under any pressure build-up rate condition... The resultant force curve (where, The actual braking force curve (established for linear control) remains flat and smooth throughout, ensuring effective active smoothness protection for the entire total braking force curve during the transition window, corresponding to the reward function. The suppression objective was fully achieved at the execution level.

[0113] The chassis limit arbitration mechanism serves as a rigid safety safety net. When the chassis stability has reached its limit, it immediately removes the intervention of all intelligent allocation and compensation logic, ensuring that the chassis active safety system operates independently without any interference under the highest priority.

[0114] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A method for controlling the ride comfort of a brake-by-wire system based on reinforcement learning, characterized in that: The method includes the following steps: Acquire the current operating status data of the vehicle and construct a state space vector, which includes the current vehicle speed, brake pedal depth change rate, power battery state of charge, actual longitudinal deceleration, and target total braking torque. The state space vectors containing the current time and the past preset time steps are used to form a sliding window state sequence, which is then input into a pre-trained reinforcement learning agent model. The basic allocation ratio coefficient is output through the reinforcement learning agent model. Based on the aforementioned basic allocation ratio coefficient, the target total braking torque is analyzed into the target motor braking torque and the target steer-by-wire friction braking torque. During the transition window period when the motor braking and the brake-by-wire braking dynamically change, the dynamic error between the target deceleration and the actual longitudinal deceleration is calculated in real time. When the dynamic error is greater than zero and exceeds the preset dead zone threshold, a transient compensation torque command is generated and sent to the motor controller so that the actual output torque of the drive motor is the sum of the target motor braking torque and the transient compensation torque. As the braking force of the brake-by-wire system gradually builds up, the transient compensation torque is controlled to decay and exit according to an exponential smooth decay function. The reinforcement learning agent model includes an Actor network and a Critic network. The output of the Actor network is connected in series with a differentiable dynamic safety projection layer. The dynamic safety projection layer is configured to obtain the maximum allowable motor braking ratio under the current physical limit, and compare the original allocation ratio output by the Actor network with the maximum motor braking ratio, taking the smaller value of the two as the final output basic allocation ratio coefficient.

2. The method for smooth braking of a brake-by-wire system based on reinforcement learning according to claim 1, characterized in that: The input end of the reinforcement learning agent model is equipped with a gated recurrent unit feature extraction layer; The step of forming a sliding window state sequence from the state space vectors containing the current time and past preset time steps, and inputting it into a pre-trained reinforcement learning agent model, specifically includes: The sliding window state sequence is input into the feature extraction layer of the gated loop unit to extract the hidden state vector containing the physical pressure build-up trend of the brake-by-wire system; The hidden state vector is input into the Actor network and the Critic network for action evaluation and output.

3. The method for smooth braking of a brake-by-wire system based on reinforcement learning according to claim 1, characterized in that: The maximum permissible motor braking ratio under the current physical limits is obtained in real time by calculating the maximum permissible regenerative power limit of the battery broadcast in real time by the chassis domain controller and the external characteristic curve of the drive motor at the current speed.

4. The method for smooth braking of a brake-by-wire system based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning agent model employs a multi-objective fusion reward function during the training phase. The reward function includes a ride comfort penalty term, which is configured to calculate the derivative of the vehicle's longitudinal acceleration in real time as an impulsivity value. When the impulsivity value exceeds a preset comfort threshold, an exponentially amplified negative penalty is applied to the reward function.

5. The method for smooth braking of a brake-by-wire system based on reinforcement learning according to claim 4, characterized in that: The mathematical expression used for the reward function of the multi-objective fusion is: ； in, For the reward function of multi-objective fusion, The tracking reward is calculated based on the error between the target deceleration and the actual longitudinal deceleration. The energy reward is calculated based on the actual recovered power of the drive motor. For the smoothness penalty term, To switch the jitter penalty term, the jitter penalty term is used to penalize the high-frequency oscillations of the base allocation ratio coefficient in adjacent time steps. All are weighting coefficients.

6. The method for smooth braking of a brake-by-wire system based on reinforcement learning according to claim 1, characterized in that: The mathematical expression used to control the transient compensation torque to decay and exit according to an exponential smooth decay function is as follows: ； in, for Transient compensation torque at time t, The initial maximum transient compensation torque, The attenuation coefficient; The attenuation coefficient The dynamic adjustment is based on the current pressure build-up rate of the brake-by-wire system, and the mathematical expression used is as follows: ； in, This refers to the real-time pressure change rate of the brake-by-wire system. This is the calibration gain coefficient.

7. The method for smooth braking of a brake-by-wire system based on reinforcement learning according to claim 1, characterized in that: The method also includes the following steps: Real-time monitoring of the vehicle chassis stability; When the Anti-lock Braking System (ABS), Electronic Stability Control (ESC), or the current road surface adhesion coefficient is detected to be triggered, the output of the reinforcement learning agent model is forcibly bypassed, the basic allocation ratio coefficient is forcibly set to zero, and the transient compensation torque is cleared, so that the braking force is provided independently by the brake-by-wire system.

8. The method for smooth braking of a brake-by-wire system based on reinforcement learning according to claim 1, characterized in that: The current vehicle speed is obtained through wheel speed sensors or the vehicle controller local area network bus; the brake pedal depth change rate is obtained by acquiring pedal displacement through electronic pedal displacement sensors and performing differential processing; and the actual longitudinal deceleration is obtained through an inertial measurement unit.

9. The method for smooth braking of a brake-by-wire system based on reinforcement learning according to claim 1, characterized in that: When the dynamic error is greater than zero and exceeds a preset dead zone threshold, a transient compensation torque command is generated and sent to the motor controller, specifically including: When the dynamic error is detected to be greater than zero and exceeds the preset dead zone threshold, it is determined that the brake-by-wire system is in the physical pressure build-up delay stage, resulting in insufficient total braking force. The power battery is instantaneously discharged, controlling the drive motor to break through the steady-state distribution limit and output the positive transient compensation torque.

10. The method for smooth braking of a brake-by-wire system based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning agent model is constructed based on the SAC algorithm; during the training process of the reinforcement learning agent model, its optimization objective is to maximize the sum of cumulative reward and policy entropy.

Images