Vehicle inching control method and device, electronic equipment and storage medium

Abstract

The present application relates to a kind of vehicle crawl control method, device, electronic equipment and storage medium, wherein vehicle crawl control method includes: obtaining self-vehicle state perception data and front vehicle state perception data;According to the self-vehicle state perception data and front vehicle state perception data determine basic torque data;Obtain the correction torque data determined according to the individualized crawl control information of driver, the basic torque data is corrected using the correction torque data, obtain target output torque;According to the target output torque, vehicle crawl control is carried out.The embodiments of the present application can guarantee the safety of vehicle crawl process, improve the safety of driving under traffic jam and the overall traffic efficiency of road traffic, and can adapt to the crawl driving style of driver, without the frequent operation of driver throttle and brake pedal, greatly reduce the operation burden of driver under traffic jam, improve driving comfort.

Description

Technical Field

[0001] This application relates to the field of electric vehicle technology, and in particular to a vehicle crawl control method, device, electronic device, and storage medium. Background Technology

[0002] With increasing urban traffic congestion, the frequent stop-and-go low-speed driving conditions place higher demands on driving comfort and safety. Crawl control allows vehicles to automatically maintain a low speed when the driver is not operating the accelerator or brake pedals, reducing driving workload. However, current crawl control technology still has many limitations.

[0003] Most existing crawl control systems are based on fixed rules or finite state machines, which perform poorly in real congested traffic conditions. Drivers still need to intervene frequently, failing to fully realize the original design goal of reducing driving burden through crawl functionality. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this application provides a vehicle crawling control method, device, electronic device and storage medium.

[0005] In a first aspect, this application provides a vehicle creep control method, including:

[0006] Acquire vehicle status perception data and the status perception data of the vehicle in front;

[0007] The basic torque data is determined based on the self-vehicle status perception data and the front vehicle status perception data;

[0008] Obtain the corrected torque data determined based on the driver's personalized crawl control information, and use the corrected torque data to correct the base torque data to obtain the target output torque;

[0009] Vehicle creep control is performed based on the target output torque.

[0010] Optionally, the basic torque data is determined based on the self-vehicle state perception data and the preceding vehicle state perception data, including:

[0011] The vehicle speed data and the slope data of the road the vehicle is traveling on are determined based on the vehicle status perception data.

[0012] The speed data and distance data of the vehicle in front are determined based on the aforementioned vehicle in front status perception data.

[0013] The vehicle speed data, the gradient data, the speed data of the vehicle in front, and the distance data of the vehicle in front are input into a preset offline network model so that the offline network model outputs the basic torque data.

[0014] Optionally, the method for updating the model parameters of the offline network model includes:

[0015] Obtain the current state perception information of the first self-vehicle and the state perception information of the first preceding vehicle, and determine the first torque based on the current state perception information of the first self-vehicle and the state perception information of the first preceding vehicle and the offline network model;

[0016] The second vehicle state perception information at the next moment is calculated based on the first torque, and the third vehicle state perception information at the next moment is obtained as the second front vehicle state perception information. The first reward score is determined based on the first front vehicle state perception information, and the model parameters of the offline network model are updated based on the first vehicle state perception information, the first front vehicle state perception information, the first torque, the second vehicle state perception information, the second front vehicle state perception information, and the first reward score.

[0017] Optionally, acquiring the current vehicle state perception information and the first preceding vehicle state perception information, and determining the first torque based on the first vehicle state perception information, the first preceding vehicle state perception information, and the offline network model, includes:

[0018] Initialize the current speed information of the first vehicle and the distance information of the first vehicle in front;

[0019] Collect the speed information of the second vehicle at the current moment as the speed information of the first vehicle in front, and collect the first gradient information of the vehicle in front as the first gradient information of the vehicle in front;

[0020] The first vehicle speed information, first slope information, first preceding vehicle speed information, and first preceding vehicle distance information are input into the offline network model to obtain the first torque.

[0021] Optionally, the second vehicle state perception information at the next moment is calculated based on the first torque, and the third vehicle state perception information at the next moment is obtained as the second preceding vehicle state perception information. A first reward score is determined based on the first preceding vehicle state perception information. The model parameters of the offline network model are updated based on the first vehicle state perception information, the first preceding vehicle state perception information, the first torque, the second vehicle state perception information, the second preceding vehicle state perception information, and the first reward score, including:

[0022] Calculate the vehicle acceleration based on the first torque;

[0023] Calculate the second vehicle speed information for the next moment based on the vehicle's acceleration;

[0024] The second vehicle speed information, the first vehicle speed information, the first preceding vehicle speed information, and the first preceding vehicle distance information are used to calculate the second preceding vehicle distance information at the next moment.

[0025] Collect the third speed information of the vehicle at the next moment as the second speed information of the vehicle in front, and collect the slope information of the vehicle at the next moment as the second slope information of the vehicle in front.

[0026] The first reward score is determined based on the first distance information of the preceding vehicle;

[0027] The offline network model parameters are updated based on the first vehicle speed information, the first slope information, the first preceding vehicle speed information, the first preceding vehicle distance information, the first torque, the first reward score, the second vehicle speed information, the second preceding vehicle distance information, the second preceding vehicle speed information, the second slope information, and the first reward score.

[0028] Optionally, obtain corrected torque data determined based on the driver's personalized crawl control information, including:

[0029] Obtain brake pedal travel data and accelerator pedal travel data for historical time periods;

[0030] The corrected torque data is determined based on the brake pedal travel data, the accelerator pedal travel data, and a preset online network model.

[0031] Optionally, the corrected torque data is determined based on the brake pedal travel data, the accelerator pedal travel data, and a preset online correction network model, including:

[0032] The vehicle speed data and the slope data of the road the vehicle is traveling on are determined based on the vehicle status perception data.

[0033] The speed data and distance data of the vehicle in front are determined based on the aforementioned vehicle in front status perception data.

[0034] The brake pedal travel data, accelerator pedal travel data, vehicle speed data, gradient data, preceding vehicle speed data, and preceding vehicle distance data are input into the online network model to obtain the corrected torque data.

[0035] Optionally, the method for updating the model parameters of the online network model includes:

[0036] Obtain combined status information including first brake pedal information, first accelerator pedal travel information, vehicle speed information, slope information, preceding vehicle speed information, and preceding vehicle distance information;

[0037] The combined state information is input into the online network model so that the online network model can determine the candidate torque information based on the combined state information and the preset greedy strategy.

[0038] The candidate torque information is used to correct the first torque output by the offline network model to obtain corrected torque information. Vehicle creep control is performed based on the corrected torque information, and the second brake pedal information and the second accelerator pedal travel information input by the user during the vehicle creep process are obtained.

[0039] The second bonus score is determined based on the second brake pedal information and the second accelerator pedal travel information.

[0040] The experience replay sequence is determined based on the combined state information, the corrected torque information, and the second reward score, and the experience replay sequence is stored in the experience replay pool.

[0041] When the total number of experience replay sequences in the experience replay pool reaches a preset threshold, a first preset number of experience replay sequences are obtained from the experience replay pool, and the model parameters of the online network model are updated. Every second preset number of training steps, the network parameters of the target network in the online network are updated. When the training termination condition is met, the training terminates.

[0042] Optionally, the model parameters of the online network model include: online network parameters and target network parameters. The model parameters of the online network model are updated by acquiring a first preset number of experience replay sequences from the experience replay pool, including:

[0043] For any of the aforementioned experience replay sequences, calculate the state-action value based on the experience replay sequence and the experience replay sequence at the next moment;

[0044] Calculate the error loss value based on the state action value and the experience replay sequence;

[0045] The online network parameters are then updated based on the error loss.

[0046] The target network parameters are updated every third preset number of training steps.

[0047] Secondly, this application provides a vehicle crawl control device, comprising:

[0048] The data acquisition module is used to acquire the vehicle's status perception data and the vehicle's status perception data in front;

[0049] The torque determination module is used to determine basic torque data based on the self-vehicle state perception data and the front vehicle state perception data.

[0050] The torque correction module is used to acquire the corrected torque data determined based on the driver's personalized crawl control information, and to correct the base torque data using the corrected torque data to obtain the target output torque.

[0051] The creep control module is used to control vehicle creep based on the target output torque.

[0052] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0053] Memory, used to store computer programs;

[0054] The processor, when executing a program stored in memory, implements the vehicle crawl control method described in any of the first aspects.

[0055] Fourthly, this application provides a computer-readable storage medium storing a program for a vehicle crawl control method, wherein when the program for the vehicle crawl control method is executed by a processor, it implements the steps of the vehicle crawl control method described in any of the first aspects.

[0056] The beneficial effects of this invention are:

[0057] In this embodiment, the vehicle can automatically acquire its own vehicle status perception data and the status perception data of the vehicle in front, and calculate the basic torque data that matches both. It can adjust the torque in a timely manner according to the status of the vehicle in front, ensuring the following safety during the crawling process, avoiding the risk of collision with the vehicle in front, improving the safety of driving in congested road conditions and the overall traffic efficiency of the road. Moreover, by using the corrected torque data determined according to the driver's personalized crawling control information, the basic torque data is corrected to obtain the target output torque, which can be adapted to the driver's crawling driving style. The driver does not need to frequently operate the accelerator and brake pedals, which greatly reduces the driver's operating burden in congested road conditions and improves driving comfort. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 A structural diagram of an adaptive creep control system provided in an embodiment of this application;

[0061] Figure 2 A flowchart of a vehicle creep control method provided in this application embodiment;

[0062] Figure 3 for Figure 2 Flowchart of step S102;

[0063] Figure 4 A flowchart illustrating a method for updating model parameters of an offline network model, provided in an embodiment of this application;

[0064] Figure 5 for Figure 4 Flowchart of step S301;

[0065] Figure 6 for Figure 4 Flowchart of step S302;

[0066] Figure 7 for Figure 2 Flowchart of step S103;

[0067] Figure 8 for Figure 7 Flowchart of step S602;

[0068] Figure 9 A flowchart illustrating a method for updating model parameters of an online network model, provided in an embodiment of this application;

[0069] Figure 10 for Figure 9 Flowchart of step S806;

[0070] Figure 11 A flowchart illustrating the training process of an offline network model provided in this application embodiment;

[0071] Figure 12 A flowchart illustrating the training process of an online network model provided in this application embodiment;

[0072] Figure 13 A structural diagram of a vehicle crawl control device provided in an embodiment of this application;

[0073] Figure 14 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0075] Since most existing crawl control systems are based on fixed rules or finite state machines, they perform poorly in real congested traffic conditions, requiring frequent driver intervention and failing to fully realize the original design goal of reducing driving burden through crawl functionality. Therefore, this application provides a vehicle crawl control method, device, electronic device, and storage medium. This application embodiment can achieve adaptive following control without driver intervention while ensuring safety, adapting to different drivers' individual driving styles, significantly improving the driving experience in congested traffic, reducing driver fatigue, and improving road traffic efficiency.

[0076] To facilitate understanding of this solution, the following technical terms are explained before formally introducing the technical solution of this application:

[0077] 1. DDPG (Deep Deterministic Policy Gradient): A reinforcement learning algorithm applicable to continuous action spaces, combining policy gradient methods and value function methods.

[0078] 2. DQN (Deep Q-Network): A Q-Learning algorithm based on deep learning, suitable for reinforcement learning problems in discrete action spaces.

[0079] 3. Experience Replay: A mechanism used in reinforcement learning to store training data. It breaks data correlations through random sampling, thereby improving training stability.

[0080] 4. Creep Control: The function of automatically maintaining a following distance when the vehicle is traveling at low speeds without relying on the driver's accelerator and brake operations.

[0081] 5. Torque Correction Value: Discrete value output by the online network for adjusting the base torque, ranging from -50Nm to +50Nm, with 11 selectable levels.

[0082] like Figure 1As shown in the illustration, this application provides an adaptive crawl control system, mainly comprising an environmental perception module, an offline DDPG strategy network, an online DQN correction network, a torque synthesis module, and a motor actuator. Specifically, the distance and speed signals of the vehicle ahead sensed by the camera and radar, the vehicle speed signal read from the CAN bus, and the road gradient signal acquired by the inertial measurement unit (IMU) are input to the pre-trained offline DDPG strategy network. This network calculates the base torque value, which is then superimposed on the torque correction value calculated by the online DQN correction network based on real-time pedal travel (accelerator and brake pedals) in the torque synthesis module. Finally, the synthesized torque command is sent to the motor controller to drive the drive motor, achieving adaptive crawl control.

[0083] This application provides a vehicle creep control method, such as... Figure 2 As shown, the following steps may be included:

[0084] Step S101: Obtain the vehicle's status perception data and the vehicle's status perception data in front;

[0085] In this embodiment, the vehicle status perception data refers to the vehicle's own operation-related data, including: vehicle speed data and slope data of the road the vehicle is traveling on, etc. The vehicle status perception data can reflect the real-time status information of the vehicle; the preceding vehicle status perception data refers to the operation-related data of the adjacent vehicles in front of the vehicle, including: preceding vehicle speed data and preceding vehicle distance data, etc. The preceding vehicle distance data refers to the distance between the preceding vehicle and the vehicle, and the preceding vehicle status perception data can reflect the preceding vehicle's status information.

[0086] In this step, the vehicle's speed data can be read via the CAN bus, and the slope data can be obtained via the accelerometer as the vehicle's status perception data. The distance to the vehicle in front can be output via the vehicle's camera perception system and / or radar perception system. The 2D and 3D images output by the vehicle's camera perception system and radar perception system are then used to determine the speed data of the vehicle in front via a camera fusion algorithm as the vehicle's status perception data. The distance to the vehicle in front can be detected by replacing the camera perception system with LiDAR, or by combining LiDAR with the camera perception system. This can improve the ranging accuracy and adaptability to adverse weather conditions.

[0087] In other words, the vehicle can collect information such as its own speed, road gradient, the speed of the vehicle in front, and the distance between the vehicle and the vehicle in front through onboard CAN bus, IMU accelerometer, forward-facing camera, radar, and other devices.

[0088] Step S102: Determine the basic torque data based on the self-vehicle status perception data and the front vehicle status perception data;

[0089] In this embodiment, the basic torque data is the basic power control data for vehicle crawling, calculated by the offline network model based on the state perception data of the vehicle and the vehicle in front.

[0090] In this step, the collected state perception data of the vehicle and the vehicle in front can be input into a pre-trained offline network model, which will then calculate and output the basic torque data under the corresponding operating conditions.

[0091] Step S103: Obtain the corrected torque data determined based on the driver's personalized crawl control information, and use the corrected torque data to correct the basic torque data to obtain the target output torque;

[0092] In this embodiment of the application, the driver's personalized crawl control information includes: personalized crawl driving mode setting information and / or information on the driver's operation of the accelerator pedal or brake pedal during crawl control in the past preset time period, etc.

[0093] For platforms with limited computing resources, only the offline network can be retained as the basic control strategy, while the online learning module can be changed to a parameter-adjustable mode, allowing the driver to manually select a preset personalized crawl driving mode (such as close following distance, moderate following distance, or long following distance). In other words, the corrected torque data can be a fixed value found based on the user's pre-selected personalized crawl driving mode, corresponding to the personalized crawl driving mode. Alternatively, the corrected torque data can be supplementary data generated by the online network model in combination with factors such as the driver's personalized operation habits of the accelerator or brake pedal, used to adjust the basic torque. The target output torque is the final torque command data used for vehicle crawl control obtained after correcting the basic torque data with the corrected torque data.

[0094] In this step, the corrected torque data determined based on the driver's personalized crawl control information can be obtained. Then, the corrected torque data is combined with the basic torque data for calculation and other processing to obtain the target output torque that conforms to the driver's personalized habits.

[0095] Step S104: Perform vehicle creep control based on the target output torque.

[0096] In this step, the target output torque can be converted into a motor control command and sent to the motor controller, which then drives the motor to output the corresponding torque, thereby enabling the vehicle to adaptively creep.

[0097] The application scenarios of this application can be extended to low-speed scenarios such as commercial vehicle queue control and automatic parking, requiring only adjustments to the reward function and state space definition.

[0098] In this embodiment, the vehicle can automatically acquire its own vehicle status perception data and the status perception data of the vehicle in front, and calculate the basic torque data that matches both. It can adjust the torque in a timely manner according to the status of the vehicle in front, ensuring the following safety during the crawling process, avoiding the risk of collision with the vehicle in front, improving the safety of driving in congested road conditions and the overall traffic efficiency of the road. Moreover, by using the corrected torque data determined according to the driver's personalized crawling control information, the basic torque data is corrected to obtain the target output torque, which can be adapted to the driver's crawling driving style. The driver does not need to frequently operate the accelerator and brake pedals, which greatly reduces the driver's operating burden in congested road conditions and improves driving comfort.

[0099] In one embodiment of this application, such as Figure 3 As shown, step S102 determines the basic torque data based on the self-vehicle state perception data and the preceding vehicle state perception data, including:

[0100] Step S201: Determine the vehicle speed data and the slope data of the road the vehicle is traveling on based on the vehicle status perception data;

[0101] In this embodiment, the vehicle speed data is a parameter used to characterize the speed of the vehicle, which can be obtained by reading data from devices such as wheel speed sensors via the CAN bus. The slope data is a parameter of the inclination of the road on which the vehicle travels, which can be calculated by devices such as IMU acceleration sensors.

[0102] Step S202: Determine the speed data and distance data of the vehicle in front based on the vehicle in front status perception data;

[0103] In this embodiment, the speed data of the vehicle in front is the driving speed parameter of the vehicle in front, which can be obtained by means of radar and camera fusion algorithms, etc., and the distance data of the vehicle in front is the distance parameter between the vehicle and the vehicle in front, which can be detected by sensing devices such as forward-facing cameras.

[0104] Step S203: Input the vehicle speed data, the slope data, the speed data of the vehicle in front, and the distance data of the vehicle in front into a preset offline network model so that the offline network model outputs the basic torque data.

[0105] In this embodiment, the offline network model is a DDPG policy network model that is pre-trained on the server and deployed to the vehicle for calculating the basic torque. In practical applications, the offline network model can also be an on-policy (Proximal Policy Optimization, PPO) algorithm based on policy gradients. The PPO algorithm has advantages in training stability and is suitable for scenarios with higher safety requirements.

[0106] like Figure 1 As shown, the input states of the offline network model include: four continuous variables—distance to the vehicle in front (output of the camera perception system), speed of the vehicle in front (output of the camera perception system), vehicle speed (read from the CAN bus), and road gradient (acquired by the acceleration sensor). The output action is one continuous variable—torque at the vehicle's motor (range -150Nm to 150Nm, negative values ​​represent regenerative braking). The network structure of the offline network model is a multi-layer fully connected neural network, trained through deep reinforcement learning to obtain the basic control strategy.

[0107] In this step, the vehicle's speed data, gradient data, the speed data of the vehicle in front, and the distance data to the vehicle in front can be input into the pre-trained offline network model. Based on its own training strategy, the model calculates and outputs basic torque data that is adapted to the current following vehicle and road conditions.

[0108] This implementation calculates the base torque by inputting the vehicle's speed data, gradient data, the speed data of the vehicle in front, and the distance data to the vehicle in front into an offline network model. The offline network model generates base torque data that matches the actual driving conditions by taking into account the vehicle's speed data, gradient data, the speed data of the vehicle in front, and the distance data to the vehicle in front. This can effectively deal with following scenarios, ensure the rationality and accuracy of following based on the base torque data, reduce problems such as vehicle jerking or following too closely caused by unreasonable base torque, and improve the stability and safety of creep control.

[0109] In yet another embodiment of this application, as Figure 4 As shown, the method for updating the model parameters of the offline network model includes:

[0110] Step S301: Obtain the first vehicle state perception information and the first preceding vehicle state perception information at the current moment, and determine the first torque based on the first vehicle state perception information, the first preceding vehicle state perception information, and the offline network model.

[0111] In this embodiment, the first vehicle state perception information refers to the vehicle-related state data collected at the current moment that triggers the model parameter update, including information such as vehicle speed and road slope; the first preceding vehicle state perception information refers to the preceding vehicle-related state data collected at the current moment that triggers the model parameter update, including information such as preceding vehicle speed and preceding vehicle distance; and the first torque refers to the torque data output by the offline network model at the current moment that triggers the model parameter update.

[0112] During the offline network model training process, the current state perception information of the first self vehicle and the state perception information of the first preceding vehicle are collected, and then combined with the offline network model, the model outputs the first torque corresponding to that moment.

[0113] Step S302: Calculate the second vehicle state perception information at the next moment based on the first torque, obtain the third vehicle state perception information at the next moment as the second front vehicle state perception information, determine the first reward score based on the first front vehicle state perception information, and update the model parameters of the offline network model based on the first vehicle state perception information, the first front vehicle state perception information, the first torque, the second vehicle state perception information, the second front vehicle state perception information, and the first reward score.

[0114] In this embodiment, the second vehicle state perception information refers to the vehicle state information at the next moment calculated based on the first torque at the current moment; the third vehicle state perception information refers to the vehicle state information actually collected at the next moment; the second preceding vehicle state perception information is the third vehicle state perception information of the vehicle at the next moment; and the first reward score is a score used to evaluate the quality of the current torque action, calculated based on the preceding vehicle state perception information at the current moment through a preset reward function.

[0115] The preset reward function (piecewise function) provided in this application embodiment is as follows:

[0116] 1) When the distance to the vehicle in front is less than 3.8m: Reward = (55 / 9) × distance to the vehicle in front - 200 / 9.

[0117] 2) When the distance to the vehicle in front is 3.8-4.2m: reward = 1 (optimal range).

[0118] 3) When the distance to the vehicle in front is 4.2-10m: Reward = (-5 / 29) × distance to the vehicle in front + 50 / 29.

[0119] 4) When the distance to the vehicle in front is greater than 10m: Reward = 0.

[0120] This function encourages vehicles to maintain a safe following distance of 3.8-4.2m, ensuring that the trained strategy is both safe and efficient.

[0121] In this step, a vehicle dynamics model can be used to calculate the theoretical state information of the vehicle at the next moment, i.e., the second vehicle state perception information, based on the first torque. Simultaneously, the third vehicle state perception information of the vehicle at the next moment is collected as the second front vehicle state perception information. Then, based on a preset piecewise reward function and combined with the front vehicle distance in the first front vehicle state perception information, a first reward score is calculated. Finally, the model parameters of the offline network model are updated based on the first vehicle state perception information, the first front vehicle state perception information, the first torque, the second vehicle state perception information, the second front vehicle state perception information, and the first reward score.

[0122] The parameter update method for the offline network model in this embodiment combines the current torque, the predicted state at the next moment, and the actual collected state information, and introduces a reward score to optimize the model parameters. This allows the offline network model to continuously learn better control strategies, enabling the trained offline network model to output a safer base torque that is more suitable for actual following conditions in actual crawling scenarios, thereby enhancing the model's environmental adaptability and control reliability.

[0123] In yet another embodiment of this application, as Figure 5 As shown, step S301 includes:

[0124] Step S401: Initialize the speed information of the first vehicle and the distance information of the first vehicle ahead at the current moment;

[0125] In this step, at the beginning of each round of offline network model training, the first vehicle speed information and the first distance information of the vehicle in front can be randomly set according to a preset range.

[0126] Step S402: Collect the speed information of the second vehicle at the current moment as the speed information of the first vehicle in front, and collect the first gradient information of the vehicle as the first gradient information of the vehicle in front.

[0127] In this step, the current speed information of the second vehicle can be collected as the speed information of the first vehicle in front, and the first gradient information of the vehicle can be collected as the first gradient information of the vehicle in front.

[0128] Step S403: Input the first vehicle speed information, first slope information, first preceding vehicle speed information, and first preceding vehicle distance information into the offline network model to obtain the first torque.

[0129] In this step, the randomly initialized first vehicle speed information and first preceding vehicle distance information, as well as the equivalent first slope information and first preceding vehicle speed information, can be input into the offline network model, and the offline network model can calculate and output the first torque according to its own strategy.

[0130] This embodiment of the application uses the self-vehicle as the preceding vehicle at the current moment, and uses the second self-vehicle speed information and the first slope information collected by the self-vehicle as the first preceding vehicle speed information and the first preceding vehicle slope information, respectively. In this way, the speed information and slope information collected by the self-vehicle can be used to supplement the speed information and slope information of the preceding vehicle, without the need to build a preceding vehicle model, which greatly reduces the difficulty of building a preceding vehicle model. When collecting data in actual working conditions, only the self-vehicle is needed to collect preceding vehicle data, without relying on other vehicles. At the same time, it can also simulate preceding vehicles with various types of driving styles.

[0131] In yet another embodiment of this application, as Figure 6 As shown, step S302 includes:

[0132] Step S501: Calculate the vehicle acceleration based on the first torque;

[0133] In this step, the first torque can be substituted into the driving force calculation formula based on the mechanical balance equation of vehicle dynamics, and then combined with parameters such as rolling resistance, wind resistance, and slope resistance to calculate the acceleration of the vehicle.

[0134] Key parameters in the mechanical equilibrium equations include: vehicle mass m, total transmission efficiency η, transmission ratio igear, wheel radius r, rolling coefficient fr, and air resistance coefficient Cd. Vehicle acceleration calculations are based on the balance between driving force and resistance (rolling resistance, wind resistance, and gradient resistance).

[0135] 1) Driving force Ft = T × igear × η / r

[0136] 2) Rolling resistance Ff = m × g × fr × cosθ

[0137] 3) Wind resistance Fw = 0.5 × ρ × Cd × A × v1²

[0138] 4) Slope resistance Fi = m × g × sinθ

[0139] 5) Acceleration a = (Ft - Ff - Fw - Fi) / m

[0140] Where T is the motor output torque, θ is the road slope, ρ is the air density, A is the vehicle's frontal area, v1 is the vehicle speed, and g is the acceleration due to gravity.

[0141] Step S502: Calculate the second vehicle speed information at the next moment based on the vehicle acceleration;

[0142] The formula for calculating the vehicle speed update is: v1 = v1_pre + a × Ts

[0143] Where Ts is the simulation time step (0.01s), v1_pre is the vehicle speed at the current moment, and v1 is the vehicle speed at the next moment.

[0144] In this step, the vehicle speed at the current moment and the calculated acceleration can be used, combined with the simulation time step, to calculate the second vehicle speed information at the next moment through numerical integration.

[0145] Step S503: Calculate the second forward vehicle distance information at the next moment based on the second vehicle speed information, the first vehicle speed information, the first forward vehicle speed information, and the first forward vehicle distance information;

[0146] The formula for calculating the distance to the vehicle in front is: s = s_pre + [(v2_pre + v2) × Ts / 2 - (v1_pre + v1) × Ts / 2]

[0147] Where Ts is the simulation time step (0.01s), v1_pre and v2_pre are the current vehicle speed and the speed of the vehicle in front, respectively, and v1 and v2 are the vehicle speed and the speed of the vehicle in front, respectively, at the next moment.

[0148] In this step, based on the principle of relative motion of vehicles, and combining the speed changes of the vehicle and the vehicle in front at previous and next moments, as well as the initial distance to the vehicle in front, the theoretical distance between the vehicle and the vehicle in front at the next moment can be calculated, that is, the second distance information of the vehicle in front.

[0149] Step S504: Collect the third speed information of the vehicle at the next moment as the second speed information of the vehicle in front, and collect the slope information of the vehicle at the next moment as the second slope information of the vehicle in front.

[0150] In other words, at the next moment, the vehicle itself is taken as the vehicle in front, the third vehicle speed information collected by the vehicle itself is taken as the second vehicle speed information in front, and the gradient information collected by the vehicle itself is taken as the second gradient information in front.

[0151] Step S505: Determine the first reward score based on the first distance information of the preceding vehicle;

[0152] In this step, the distance information of the first vehicle ahead at the current moment can be substituted into a preset segmented reward function to calculate the corresponding reward score, which is used to evaluate the quality of the current torque action.

[0153] Step S506: Update the model parameters of the offline network model based on the first vehicle speed information, first slope information, first preceding vehicle speed information, first preceding vehicle distance information, first torque, first reward score, second vehicle speed information, second preceding vehicle distance information, second preceding vehicle speed information, second slope information, and the first reward score.

[0154] This application embodiment uses the self-vehicle as the preceding vehicle at the next moment, and equates the self-vehicle data to the corresponding state data of the preceding vehicle. In this way, the speed and slope information collected by the self-vehicle can be used to supplement the speed and slope information of the preceding vehicle. There is no need to build a preceding vehicle model, which greatly reduces the difficulty of building a preceding vehicle model. When collecting data in actual working conditions, only the self-vehicle is needed to collect preceding vehicle data, without relying on other vehicles. At the same time, it can also simulate preceding vehicles with various driving styles.

[0155] This application embodiment combines precise vehicle dynamics calculations with actual collected data to enable model training that better reflects the actual operating state of the vehicle. By evaluating torque action through a piecewise reward function, the model is guided to learn a safe and efficient following strategy, allowing the trained offline network model to output basic torque more accurately in actual creep control, thus ensuring the safety and comfort of the following process.

[0156] In yet another embodiment of this application, as Figure 7 As shown, step S103 obtains the corrected torque data determined based on the driver's personalized crawl control information, including:

[0157] Step S601: Obtain brake pedal travel data and accelerator pedal travel data for historical time periods;

[0158] In this embodiment of the application, the brake pedal travel data for a historical time period refers to the travel distance and frequency of the driver pressing the brake pedal over a past period; the accelerator pedal travel data refers to the travel distance and frequency of the driver pressing the accelerator pedal over a past period.

[0159] In this step, the pedal sensor can collect relevant data on the driver's operation of the brake pedal and accelerator pedal over a period of time. This data can reflect the driver's driving habits and operating preferences regarding following distance during vehicle crawling.

[0160] Step S602: Determine the corrected torque data based on the brake pedal travel data, the accelerator pedal travel data, and the preset online network model.

[0161] In this embodiment, the online network model refers to a DQN correction network model or a reinforcement learning algorithm such as the off-policy algorithm (Soft Actor-Critic, SAC) that is deployed on the vehicle and can perform online learning and parameter updates, used to generate personalized torque correction data.

[0162] like Figure 1 As shown, the input states of the online network model include: two additional state variables—accelerator pedal travel and brake pedal travel—in addition to the offline network inputs, for a total of six inputs. Output actions: 11 discrete torque correction values ​​(-50, -40, -30, -20, -10, 0, 10, 20, 30, 40, 50). Learning mechanism: Online real-time updates of network parameters to adapt to the driver's individual driving style.

[0163] In this step, the collected historical pedal operation data can be input into the online network model. The model combines the driver's operating habits it has learned to output corresponding corrective torque data, which is used to make personalized adjustments to the base torque.

[0164] This application embodiment generates corrected torque by collecting historical pedal operation data from the driver and combining it with an online network model. This allows the torque control strategy to adapt to the personalized driving styles of different drivers regarding following distance. Since intelligent following control ensures that the vehicle and the vehicle in front always maintain the following distance desired by the driver, the driver does not need to frequently operate the brake and accelerator pedals, minimizing the frequency of accelerator and brake pedal operations and improving driving comfort. At the same time, torque correction based on the driver's operating habits can reduce the number of times the driver intervenes during creeping, further reducing the driving burden and making creep control more in line with actual driving needs.

[0165] In yet another embodiment of this application, as Figure 8 As shown, step S602 determines the corrected torque data based on the brake pedal travel data, the accelerator pedal travel data, and the preset online correction network model, including:

[0166] Step S701: Determine the vehicle speed data and the slope data of the road the vehicle is traveling on based on the vehicle status perception data;

[0167] Step S702: Determine the speed data and distance data of the vehicle in front based on the vehicle in front status perception data;

[0168] Step S703: Input the brake pedal travel data, the accelerator pedal travel data, the vehicle speed data, the slope data, the speed data of the vehicle in front, and the distance data of the vehicle in front into the online network model to obtain the corrected torque data.

[0169] In this step, the driver's real-time brake pedal travel data, accelerator pedal travel data, vehicle speed data, gradient data, speed data of the vehicle in front, and distance data of the vehicle in front can be integrated and input into the online network model. The model combines its own learning strategy and driver habits to output corrective torque data adapted to the current working conditions, thereby realizing personalized correction of the base torque.

[0170] This application embodiment integrates the driver's real-time brake pedal travel data, accelerator pedal travel data, the vehicle's own speed data, the slope data, the speed data of the vehicle in front, and the distance data to the vehicle in front, and inputs them into the online network model. This allows the corrected torque to not only adapt to the driver's pedal operation habits in following other vehicles, but also to fit the real-time driving conditions, making the final target torque more accurate and reasonable. This multi-dimensional data input improves the decision-making accuracy of the online network model, further optimizes the control effect in crawling following scenarios, and enhances the vehicle's adaptive capability in complex road conditions.

[0171] In yet another embodiment of this application, as Figure 9As shown, the method for updating the model parameters of the online network model includes:

[0172] Step S801: Obtain combined state information including first brake pedal information, first accelerator pedal travel information, vehicle speed information, slope information, preceding vehicle speed information, and preceding vehicle distance information;

[0173] Step S802: Input the combined state information into the online network model so that the online network model can determine the candidate torque information based on the combined state information and the preset greedy strategy;

[0174] In this embodiment, the candidate torque information is a preliminary torque information output by the online network model based on the combined state information and the ε-greedy greedy strategy. While ensuring the utilization of the optimal strategy, appropriate exploration is carried out to output the corresponding candidate torque information.

[0175] Step S803: The candidate torque information is used to correct the first torque output by the offline network model to obtain corrected torque information. Vehicle creep control is performed based on the corrected torque information, and the second brake pedal information and the second accelerator pedal travel information input by the user during vehicle creep are obtained.

[0176] In this embodiment, the corrected torque information is the final torque correction data obtained by correcting the first torque output by the offline network using candidate torque information.

[0177] In this step, the candidate torque information can be synthesized with the base torque output from the offline network to obtain the corrected torque information, which is then used for vehicle creep control. At the same time, new pedal operation data of the driver during the control process is collected.

[0178] Step S804: Determine the second bonus score based on the second brake pedal information and the second accelerator pedal travel information;

[0179] In this embodiment of the application, the second reward score is a reward score calculated by a preset online reward function based on the driver's new pedal operation information during vehicle creep.

[0180] In this step, the newly collected pedal operation data can be substituted into the online reward function to calculate the corresponding reward score, which is used to evaluate the effect of this torque correction action.

[0181] Step S805: Determine the experience replay sequence based on the combined state information, the corrected torque information, and the second reward score, and store the experience replay sequence in the experience replay pool;

[0182] In this embodiment, the experience replay sequence is a data sequence for model training composed of combined state information, corrected torque information, and a second reward score. The experience replay pool is a database for storing experience replay sequences, and its capacity can be preset.

[0183] In this step, the combined state information, corrected torque information, and second reward score can be integrated into an experience replay sequence and stored in the experience replay pool to provide data for subsequent model parameter updates.

[0184] Step S806: When the total number of experience replay sequences in the experience replay pool reaches a preset threshold, a first preset number of experience replay sequences are obtained from the experience replay pool, and the model parameters of the online network model are updated. Every second preset number of training steps, the network parameters of the target network in the online network are updated. When the training termination condition is met, the training is terminated.

[0185] In this embodiment of the application, the training termination condition refers to the vehicle speed exceeding 15 km / h exiting the crawl mode, or the average reward of 10 consecutive training cycles converging and the fluctuation being less than ±0.1.

[0186] In this step, when the amount of data in the experience replay pool meets a preset threshold, a certain number of sequences can be selected and the model parameters of the online network can be updated using the DQN algorithm. The target network parameters are updated synchronously after each training step. When the training termination condition is met, the training of the online network is terminated to ensure that the model parameters are stable and the strategy is optimal.

[0187] The online network parameter update method in this application balances the exploration and utilization of the model through an ε-greedy strategy, enabling the model to continuously learn new driving habits and operating condition characteristics. The use of an experience replay pool improves data utilization and training stability, and updating the target network parameters by step ensures a smooth training process. Based on the reward mechanism of real-time driver operation, the model can be guided to quickly adapt to the driver's style, reduce the number of pedal interventions, significantly improve the personalization and intelligence level of crawl control, and optimize the driving experience.

[0188] In another embodiment of this application, the model parameters of the online network model include: online network parameters and target network parameters, such as... Figure 10 As shown, step S806 involves obtaining a first preset number of experience replay sequences from the experience replay pool and updating the model parameters of the online network model, including:

[0189] Step S901: For any of the experience replay sequences, calculate the state action value based on the experience replay sequence and the experience replay sequence at the next moment.

[0190] In this embodiment of the application, the state action value, i.e. the target Q value, is used to evaluate the value of performing the corresponding action in the current state, and can be calculated by combining the immediate reward and the maximum Q value of the next state.

[0191] In this step, for each experience replay sequence, the immediate reward is extracted, and combined with the maximum Q-value of the experience replay sequence in the target network at the next time step, using the formula... Calculate the state-action value, where It's Q. Instant rewards =0.99 is the discount factor. Indicates a new state The largest Q value among all possible corrective torques.

[0192] Step S902: Calculate the error loss value based on the state action value and the experience replay sequence;

[0193] In this embodiment, the error loss value is the mean square error between the Q value predicted by the online network and the target Q value, which is used to measure the accuracy of the network prediction.

[0194] In this step, the Q-value predicted by the online network for the current experience replay sequence can be compared with the calculated state-action value, using the formula... The mean squared error loss value was calculated, where, Where j is the sequence number of the currently used sequence data in the experience pool, and s is the state of all inputs in the j-th data. Let s be the data in the next time step of the j-th data, N be the number of mini-batches, and a be the correction torque in the j-th data (one of the selected action numbers 1 to 11). Let 'a' be the data in the next time step of the j-th data.

[0195] Step S903: Update the online network parameters based on the error loss.

[0196] In this embodiment, the online network parameters are the network parameters used for real-time inference and learning in the online DQN network.

[0197] In this step, the error loss value can be used as the optimization objective. The parameters of the online network can be adjusted using the gradient descent algorithm to reduce the prediction error and improve the network's prediction accuracy.

[0198] Step S904: Every third preset number of training steps, update the target network parameters.

[0199] In this embodiment, the target network parameter is a network parameter used in an online DQN network to provide a stable target Q value, and its update frequency is lower than that of the online network parameter.

[0200] To ensure training stability, a fixed number of steps can be set in this step. When the online network reaches this number of steps, the parameters of the online network are synchronized to the target network, so that the parameters of the target network are updated in a timely manner to adapt to the new learning strategy.

[0201] This application's embodiments update online network parameters by calculating state action value and error loss value, which can accurately measure the accuracy of network prediction. The parameters are continuously optimized through gradient descent algorithm to improve the network's decision-making ability. Updating the target network parameters by a fixed number of steps ensures the stability of the training process and avoids training divergence caused by frequent fluctuations in the target value. This parameter update method allows the online network model to quickly learn and adapt to the driver's operating habits, generating more accurate correction torque, and further improving the personalization and reliability of vehicle creep control.

[0202] For ease of understanding, this application also provides an example of offline network model and online network model training in a practical application, as follows:

[0203] (I) Construction of Offline Training Environment and Data Preparation

[0204] 1. Training data collection specifications: Collect time series data of vehicle speed and gradient on real roads, and treat the collecting vehicle as the preceding vehicle.

[0205] 2. Data Standards: A time series with a vehicle speed below 15km / h and lasting for more than 1 minute is considered a valid sample, and the total number of samples must reach more than 10,000.

[0206] 3. Data preprocessing: The collected raw data is filtered to eliminate the influence of sensor noise and ensure the quality of training data.

[0207] 4. Simulation Environment Setup and Parameter Initialization: Parameters are initialized for each training round: vehicle speed (random, 1-10 km / h) and distance to the preceding vehicle (random, 2-20 m). The speed and gradient of the preceding vehicle are obtained sequentially from the dataset at simulation time steps (0.01 s). Vehicle dynamics parameters: vehicle mass m = 1500 kg, total transmission efficiency η = 0.95, transmission ratio igear = 8.5, wheel rolling radius r = 0.32 m, rolling coefficient fr = 0.015, air damping coefficient ρ = 1.225 kg / m³, drag coefficient Cd = 0.3, and frontal area A = 2.2 m².

[0208] 5. Training termination condition and reward function design:

[0209] Round termination conditions: Training terminates when the current vehicle distance is ≤2m or >20m, or when the relative vehicle speed difference within the distance interval exceeds a threshold. Specific conditions include:

[0210] 1) When the distance between your vehicle and the vehicle in front is 2~5m, your vehicle speed - the speed of the vehicle in front > 5km / h.

[0211] 2) When the distance between your vehicle and the vehicle in front is 5~10m, your vehicle speed - the speed of the vehicle in front > 10km / h.

[0212] 3) When the distance between your vehicle and the vehicle in front is greater than 10m, your vehicle speed - the speed of the vehicle in front > 15km / h.

[0213] By using training termination conditions based on relative vehicle speed and distance, as well as a torque output limiting mechanism, the system's safety boundaries can be ensured.

[0214] 6. Reward function design (piecewise function):

[0215] 1) When the distance to the vehicle in front is less than 3.8m: Bonus = (55 / 9) × Distance to the vehicle in front - 200 / 9

[0216] 2) When the distance to the vehicle in front is 3.8-4.2m: Reward = 1 (optimal range)

[0217] 3) When the distance to the vehicle in front is 4.2-10m: Bonus = (-5 / 29) × distance to the vehicle in front + 50 / 29

[0218] 4) When the distance to the vehicle in front is greater than 10m: Reward = 0

[0219] This function encourages vehicles to maintain a safe following distance of 3.8-4.2m, ensuring that the trained strategy is both safe and efficient.

[0220] (II) Online learning and personalized adaptation mechanism

[0221] 1. Experience pool construction and data management

[0222] Collect real-time driving data sequences: speed and distance of the vehicle in front, speed of the vehicle itself, gradient, pedal travel, and corrected torque index. Network training is triggered after each complete sample is accumulated (vehicle speed <15km / h for 1 minute).

[0223] 2. Experience replay mechanism

[0224] We employ a priority-based experience replay approach to focus on learning driving scenarios with high intervention frequencies.

[0225] 3. Online reward function design

[0226] 1) Accelerator pedal penalty mechanism

[0227] Trip < 10%: Reward r = 0 (no penalty); Trip 10%-30%: Reward r = -1 (mild penalty); Trip > 30%: Reward r = -2 (severe penalty)

[0228] 2) Brake pedal penalty mechanism

[0229] Trip <10%: Reward r=-1 (mild penalty); Trip 10%-30%: Reward r=-2 (moderate penalty); Trip >30%: Reward r=-5 (severe penalty)

[0230] The goal of the online reward function design is to minimize pedal operation frequency and promote autonomous following. This reward function design enables the system to learn following strategies that align with driver preferences.

[0231] Offline training takes place on a server, and after training, the data is deployed to the vehicle. Figure 11 As shown, at the start of offline training, the simulation environment initialization module randomly sets the vehicle's speed (1-10 km / h) and initial distance to the vehicle in front (2-20 m). In each training step, the state observation module acquires the current environmental state (distance to the vehicle in front, vehicle speed, etc.) and inputs it into the DDPG network. The torque action T output by the network is used by the vehicle dynamics model to calculate the state at the next moment (vehicle speed and distance to the vehicle in front), and the reward calculation module evaluates the quality of this torque action according to a preset piecewise function. The experience replay pool stores this data (the four states at the current moment, the model's output torque, the reward value, and the four states at the next moment) for network updates. The termination judgment module continuously monitors whether the round end condition is met (such as distance being too close or too large). Once triggered, the current round ends and a new round of training begins. Through a large number of such rounds of training, the network parameters are continuously optimized.

[0232] Online DQN network training and inference both occur on the vehicle side, such as... Figure 12 As shown, before the online training process begins, the parameters of the online DQN network are initialized to their default values ​​(output correction torque is 0), and the pre-trained offline DDPG network is loaded as the basic control policy. Simultaneously, the experience replay buffer is initialized with a capacity of 10,000 data sequences.

[0233] Once the process begins, the first step is environmental perception and data acquisition. Real-time status data is continuously collected through onboard sensors, including: distance to the vehicle in front (output of the camera perception system), speed of the vehicle in front (calculated by radar or camera fusion algorithm), vehicle speed (read from CAN bus), road gradient (acquired by IMU sensor), and accelerator / brake pedal travel (read from pedal sensor). The data is cached in time series form and updated every 0.01 seconds.

[0234] Then, the collected data is input into the DQN network in the form of combined states. Next is action selection and execution. The online DQN network selects torque correction actions based on the current state input using an ε-greedy strategy: it estimates the Q-value of different torque actions through a deep neural network (DQN), randomly selects exploratory actions with probability ε (e.g., random correction values ​​from -50 to 50 Nm, with 20 values ​​every 5 N), and selects the optimal action (e.g., correction value 0 or ±10 Nm) with probability 1-ε, which has the largest current Q value (i.e., the action with the highest probability).

[0235] Next, the offline network baseline torque and the online correction value are combined to form the final torque. Crawl control is then implemented. Again, the driver's accelerator and brake pedal travel is detected, and the reward function r is calculated: accelerator pedal travel > 30%: r = -2 (severe penalty); brake pedal travel > 30%: r = -5 (safety priority penalty); no pedal operation: r = +1 (encouraging autonomous control). Generally, only one of the accelerator or brake pedal travel can occur at any given time.

[0236] Next, the combined state, corrected torque, and reward r are stored in the experience replay buffer. Once the experience replay pool has accumulated enough data (512 sequences), a small batch (for example, batch size=32, i.e., 32 sequences) is randomly sampled for training. The key training steps are:

[0237] Calculate the target Q value

[0238] in, It's Q. Instant rewards =0.99 is the discount factor. Indicates a new state The largest Q value among all possible corrective torques. This refers to the DQN model, which updates the online network parameters using mean squared error loss. Where j is the sequence number of the currently used sequence data in the experience pool, and s is the state of all inputs in the j-th data. Let s be the data in the next time step of the j-th data, N be the number of mini-batches, and a be the correction torque in the j-th data (one of the selected action numbers 1 to 11). For the parameter 'a' in the next time step of the j-th data, the target network parameters are updated synchronously every 100 training steps. , It needs to be updated after each training session.

[0239] Finally, the training termination condition is determined by: exiting the crawl mode (vehicle speed greater than 15km / h; if the vehicle exits and then re-enters the crawl mode, training needs to be restarted), or the average reward converges after 10 consecutive training cycles (fluctuation less than ±0.1).

[0240] The offline network in this application provides basic safe following capability to ensure basic safety under various road conditions; the online network, on the other hand, learns the driver's operating habits in real time and makes personalized fine adjustments to the basic torque output.

[0241] To facilitate understanding, this application also provides an embodiment for inference using a trained offline network model and an online network model in a practical application, as follows:

[0242] The proposed scheme will be implemented under typical urban congestion conditions (average vehicle speed <15km / h, vehicle distance 5-15m).

[0243] Implementation environment setup:

[0244] Road conditions: Urban main road, flat surface, gradient <1%.

[0245] Traffic conditions: Congested roads during the morning rush hour, with vehicles frequently starting and stopping in front.

[0246] Vehicle status: The electric vehicle is fully loaded, the air conditioning is on, and the battery is fully charged.

[0247] 1. System Initialization: Load pre-trained offline network parameters; initialize online network parameters to zero. After passing the system self-test, enter standby mode. Crawl mode activation conditions: Automatically enter crawl mode when the accelerator and brake pedals are not depressed, the gear is in drive, and the vehicle speed is below the threshold (15km / h).

[0248] 2. Environmental perception and data acquisition: Distance to the vehicle in front: 10m (detected by a forward-facing camera), speed of the vehicle in front: 8km / h (calculated by a radar and camera fusion algorithm), speed of the vehicle itself: 10km / h (read from wheel speed sensor data via CAN bus), road gradient: 0° (calculated by an IMU accelerometer).

[0249] 3. Basic torque calculation: The offline network calculates the basic torque based on the input state: the network forward propagation calculation outputs a basic torque value of 45Nm, and the torque range is checked and limited to ensure that the output is within the range of -150Nm to 150Nm.

[0250] 4. Personalized correction: The online network outputs a torque correction value based on the current status and the driver's operation history. The input status includes: distance to the vehicle in front, vehicle speed, slope and recent pedal operations. The network inference output is a +10Nm correction value (corresponding to discrete action index 7).

[0251] 5. Torque Synthesis and Execution: Combining the base torque and the corrected torque, the final motor control command of 55Nm is output. The torque is smoothed to avoid sudden shocks. The motor controller receives the command and drives the motor to output the corresponding torque.

[0252] 6. Effect Evaluation and Learning: The online network experience pool is updated based on driver intervention, and regular training and optimization are performed. When frequent pedal operations are detected, the online network automatically adjusts the torque strategy, gradually reducing reliance on manual intervention, and ultimately achieving natural following behavior that matches the driver's style.

[0253] This application's embodiments achieve creep adaptive control through a hierarchical reinforcement learning strategy, significantly improving environmental adaptability: overcoming the limitations of fixed-rule strategies, it can adapt to complex and ever-changing road environments and driver styles. Through an online learning mechanism, the system can continuously optimize the control strategy based on actual driving conditions. The operational burden is greatly reduced: real-world data shows that the number of pedal operations by the driver can be reduced in congested traffic. Through personalized following strategies, the system can accurately predict driver intentions, reducing unnecessary intervention. Safety is comprehensively improved: integrating forward vehicle distance perception and multiple safety constraints avoids potential collision risks. The reward function design ensures that the system prioritizes safety, especially in terms of rapid response capabilities under emergency conditions.

[0254] In another embodiment of this application, a vehicle crawl control device is also provided, such as... Figure 13 As shown, it includes:

[0255] Data acquisition module 11 is used to acquire the vehicle's status perception data and the vehicle's status perception data in front;

[0256] The torque determination module 12 is used to determine basic torque data based on the self-vehicle state perception data and the front vehicle state perception data.

[0257] The torque correction module 13 is used to acquire the correction torque data determined based on the driver's personalized crawl control information, and use the correction torque data to correct the basic torque data to obtain the target output torque.

[0258] The creep control module 14 is used to perform vehicle creep control based on the target output torque.

[0259] In another embodiment of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0260] Memory, used to store computer programs;

[0261] When the processor executes the program stored in the memory, it implements the vehicle crawl control method described in any of the foregoing method embodiments.

[0262] The electronic device provided in this embodiment of the invention enables the vehicle to automatically acquire its own vehicle status perception data and the status perception data of the vehicle in front by executing a program stored in the memory. It then calculates basic torque data that matches both data and calculates the torque data. The torque can be adjusted in a timely manner according to the status of the vehicle in front, ensuring the safety of following the vehicle during creeping, avoiding the risk of collision with the vehicle in front, improving the safety of driving in congested traffic conditions and the overall traffic efficiency. Moreover, by using the corrected torque data determined based on the driver's personalized creeping control information, the basic torque data is corrected to obtain the target output torque, which can adapt to the driver's creeping driving style. This eliminates the need for the driver to frequently operate the accelerator and brake pedals, significantly reducing the driver's workload in congested traffic conditions and improving driving comfort.

[0263] The communication bus 1140 mentioned in the above-mentioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 14 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0264] The communication interface 1120 is used for communication between the above-mentioned electronic device and other devices.

[0265] The memory 1130 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0266] The processor 1110 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0267] In another embodiment of this application, a computer-readable storage medium is provided, on which a program for a vehicle crawl control method is stored. When the program for the vehicle crawl control method is executed by a processor, it implements the steps of the vehicle crawl control method described in any of the foregoing method embodiments.

[0268] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0269] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A vehicle creep control method, characterized in that, include: Acquire vehicle status perception data and the status perception data of the vehicle in front; The basic torque data is determined based on the self-vehicle status perception data and the front vehicle status perception data; The basic torque data is determined based on the self-vehicle state perception data and the preceding vehicle state perception data, including: The vehicle speed data and the slope data of the road the vehicle is traveling on are determined based on the vehicle status perception data. The speed data and distance data of the vehicle in front are determined based on the aforementioned vehicle in front status perception data. The vehicle speed data, the slope data, the speed data of the vehicle in front, and the distance data of the vehicle in front are input into a preset offline network model so that the offline network model outputs the basic torque data. Obtain the corrected torque data determined based on the driver's personalized crawl control information, and use the corrected torque data to correct the base torque data to obtain the target output torque; Acquire corrected torque data determined based on the driver's personalized crawl control information, including: Obtain brake pedal travel data and accelerator pedal travel data for historical time periods; The corrected torque data is determined based on the brake pedal travel data, the accelerator pedal travel data, and a preset online network model; Vehicle creep control is performed based on the target output torque.

2. The vehicle creep control method according to claim 1, characterized in that, The method for updating the model parameters of the offline network model includes: Obtain the current state perception information of the first self-vehicle and the state perception information of the first preceding vehicle, and determine the first torque based on the current state perception information of the first self-vehicle and the state perception information of the first preceding vehicle and the offline network model; The second vehicle state perception information at the next moment is calculated based on the first torque, and the third vehicle state perception information at the next moment is obtained as the second front vehicle state perception information. The first reward score is determined based on the first front vehicle state perception information, and the model parameters of the offline network model are updated based on the first vehicle state perception information, the first front vehicle state perception information, the first torque, the second vehicle state perception information, the second front vehicle state perception information, and the first reward score.

3. The vehicle creep control method according to claim 2, characterized in that, Acquire the current vehicle state perception information and the first preceding vehicle state perception information, and determine the first torque based on the first vehicle state perception information, the first preceding vehicle state perception information, and the offline network model, including: Initialize the current speed information of the first vehicle and the distance information of the first vehicle in front; Collect the speed information of the second vehicle at the current moment as the speed information of the first vehicle in front, and collect the first gradient information of the vehicle in front as the first gradient information of the vehicle in front; The first vehicle speed information, the first slope information, the first preceding vehicle speed information, and the first preceding vehicle distance information are input into the offline network model to obtain the first torque.

4. The vehicle creep control method according to claim 2, characterized in that, The second vehicle state perception information at the next moment is calculated based on the first torque, and the third vehicle state perception information at the next moment is obtained as the second preceding vehicle state perception information. A first reward score is determined based on the first preceding vehicle state perception information. The model parameters of the offline network model are updated based on the first vehicle state perception information, the first preceding vehicle state perception information, the first torque, the second vehicle state perception information, the second preceding vehicle state perception information, and the first reward score, including: Calculate the vehicle acceleration based on the first torque; Calculate the second vehicle speed information for the next moment based on the vehicle's acceleration; The second vehicle speed information, the first vehicle speed information, the first preceding vehicle speed information, and the first preceding vehicle distance information are used to calculate the second preceding vehicle distance information at the next moment. Collect the third speed information of the vehicle at the next moment as the second speed information of the vehicle in front, and collect the slope information of the vehicle at the next moment as the second slope information of the vehicle in front. The first reward score is determined based on the first distance information of the preceding vehicle; The offline network model parameters are updated based on the first vehicle speed information, the first slope information, the first preceding vehicle speed information, the first preceding vehicle distance information, the first torque, the first reward score, the second vehicle speed information, the second preceding vehicle distance information, the second preceding vehicle speed information, the second slope information, and the first reward score.

5. The vehicle creep control method according to claim 1, characterized in that, The corrected torque data is determined based on the brake pedal travel data, the accelerator pedal travel data, and a preset online correction network model, including: The vehicle speed data and the slope data of the road the vehicle is traveling on are determined based on the vehicle status perception data. The speed data and distance data of the vehicle in front are determined based on the aforementioned vehicle in front status perception data. The brake pedal travel data, accelerator pedal travel data, vehicle speed data, gradient data, preceding vehicle speed data, and preceding vehicle distance data are input into the online network model to obtain the corrected torque data.

6. The vehicle creep control method according to claim 5, characterized in that, The method for updating the model parameters of the online network model includes: Obtain combined status information including first brake pedal information, first accelerator pedal travel information, vehicle speed information, slope information, preceding vehicle speed information, and preceding vehicle distance information; The combined state information is input into the online network model so that the online network model can determine the candidate torque information based on the combined state information and the preset greedy strategy. The candidate torque information is used to correct the first torque output by the offline network model to obtain corrected torque information. Vehicle creep control is performed based on the corrected torque information, and the second brake pedal information and the second accelerator pedal travel information input by the user during the vehicle creep process are obtained. The second bonus score is determined based on the second brake pedal information and the second accelerator pedal travel information. The experience replay sequence is determined based on the combined state information, the corrected torque information, and the second reward score, and the experience replay sequence is stored in the experience replay pool. When the total number of experience replay sequences in the experience replay pool reaches a preset threshold, a first preset number of experience replay sequences are obtained from the experience replay pool, and the model parameters of the online network model are updated. Every second preset number of training steps, the network parameters of the target network in the online network are updated. When the training termination condition is met, the training terminates.

7. The vehicle creep control method according to claim 6, characterized in that, The model parameters of the online network model include: online network parameters and target network parameters. A first preset number of experience replay sequences are obtained from the experience replay pool, and the model parameters of the online network model are updated, including: For any of the aforementioned experience replay sequences, calculate the state-action value based on the experience replay sequence and the experience replay sequence at the next moment; Calculate the error loss value based on the state action value and the experience replay sequence; The online network parameters are then updated based on the error loss. The target network parameters are updated every third preset number of training steps.

8. A vehicle creep control device, characterized in that, include: The data acquisition module is used to acquire the vehicle's status perception data and the vehicle's status perception data in front; The torque determination module is used to determine basic torque data based on the self-vehicle state perception data and the front vehicle state perception data. Determining basic torque data based on the self-vehicle state perception data and the preceding vehicle state perception data includes: determining self-vehicle speed data and the slope data of the road the self-vehicle is traveling on based on the self-vehicle state perception data; determining the preceding vehicle speed data and the preceding vehicle distance data based on the preceding vehicle state perception data; and inputting the self-vehicle speed data, the slope data, the preceding vehicle speed data, and the preceding vehicle distance data into a preset offline network model so that the offline network model outputs the basic torque data. The torque correction module is used to acquire corrected torque data determined based on the driver's personalized crawl control information, and to correct the base torque data using the corrected torque data to obtain the target output torque. Acquiring the corrected torque data determined based on the driver's personalized crawl control information includes: acquiring brake pedal travel data and accelerator pedal travel data over a historical time period; and determining the corrected torque data based on the brake pedal travel data, the accelerator pedal travel data, and a preset online network model. The creep control module is used to control vehicle creep based on the target output torque.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor, when executing a program stored in memory, implements the vehicle crawl control method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for a vehicle crawl control method, which, when executed by a processor, implements the steps of the vehicle crawl control method according to any one of claims 1-7.

Images