Self-adaptive cruise model training method, self-adaptive cruise control method and equipment

Abstract

The invention relates to the technical field of vehicles, in particular to an adaptive cruise model training method and an adaptive cruise control method and device, and the method comprises the steps: obtaining predefined driving cycle data of a target vehicle; a self-adaptive cruise model is trained through the predefined driving cycle data, the self-adaptive cruise model comprises an action network and an evaluation network, and the action network outputs an acceleration control strategy of the target vehicle at the next moment based on the state data of the front vehicle at the current moment and the state data of the target vehicle; the evaluation network evaluates a reward value of the acceleration control strategy based on a multi-target reward function; in the training process, weights of a plurality of sub-rewards in the multi-target reward function are optimized based on the reward value, training is stopped until an acceleration control strategy output by the adaptive cruise model meets a preset requirement, a trained adaptive cruise model is obtained, and adaptive cruise of the target vehicle is controlled through the trained adaptive cruise model. The sub-award is an optimization target of the acceleration control strategy.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a training method for an adaptive cruise model, an adaptive cruise control method and a device. Background Art

[0002] With the development of intelligent vehicles and automated transportation, Adaptive Cruise Control (ACC) has become a core technology in the field of intelligent driving. This system uses sensors such as radar and cameras to sense the status of the vehicle ahead in real time, automatically adjusting the vehicle's speed and distance to the driver, enabling intelligent following. It is widely used in various vehicle types, including passenger cars and commercial vehicles.

[0003] However, traditional adaptive cruise control systems often use a single target or fixed rules for control. The control target is single and it is difficult to take into account multiple targets at the same time, such as safety, driving comfort and speed compliance. Summary of the Invention

[0004] The present application provides a training method for an adaptive cruise model, an adaptive cruise control method and a device to solve the problem in related technologies that adaptive cruise cannot simultaneously optimize multiple objectives.

[0005] A first aspect embodiment of the present application provides a training method for an adaptive cruise model, comprising the following steps: obtaining predefined driving cycle data of a target vehicle, wherein the predefined driving cycle data includes status data of a preceding vehicle and status data of the target vehicle; using the predefined driving cycle data to train an adaptive cruise model, wherein the adaptive cruise model includes an action network and an evaluation network, wherein the action network outputs an acceleration control strategy of the target vehicle at the next moment based on the status data of the preceding vehicle and the status data of the target vehicle at the current moment, and the evaluation network evaluates the reward value of the acceleration control strategy based on a multi-objective reward function; during the training process, optimizing the weights of multiple sub-rewards in the multi-objective reward function based on the reward value, stopping the training until the acceleration control strategy output by the adaptive cruise model meets preset requirements, and obtaining a trained adaptive cruise model, and using the trained adaptive cruise model to control the adaptive cruise of the target vehicle, wherein the sub-reward is the optimization target of the acceleration control strategy.

[0006] Optionally, the multi-objective reward function includes: a speed reward function, a safety reward function and a comfort reward function, wherein, The speed reward function is: ; in, is the speed of the target vehicle, is the preset speed, is the speed bonus value; The safety reward function is: ; in, is the relative distance between the target vehicle and the preceding vehicle, is the lower limit of the safety distance, is the following speed of the target vehicle, is the following acceleration of the target vehicle, As the basic safety reward value; The comfort reward function is: ; in, is the rate of change of the acceleration of the target vehicle, is the maximum acceleration of the target vehicle, is the minimum acceleration of the target vehicle, is the comfort bonus value; The multi-objective reward function is: ; in, is the multi-objective reward value, is the speed bonus value, is the weight of the speed reward value, is the safety bonus value, is the weight of the safety reward value, is the comfort bonus value, is the weight of the comfort reward value.

[0007] Optionally, the state data includes speed, acceleration, and relative distance. Before the evaluation network evaluates the reward value of the acceleration control strategy based on the multi-objective reward function, it also includes: calculating the estimated time of collision between the target vehicle and the front vehicle based on the relative distance and speed between the target vehicle and the front vehicle at the current moment; and adjusting the basic safety reward value output by the safety reward function based on the estimated time.

[0008] Optionally, before the evaluation network evaluates the reward value of the acceleration control strategy based on the multi-objective reward function, it also includes: using the target intelligent driving model to correct the acceleration control strategy.

[0009] Optionally, the adaptive cruise control model is trained using predefined driving cycle data, including: normalizing the state data of the preceding vehicle and the state data of the target vehicle to obtain a state observation vector; and inputting the state observation vector into the adaptive cruise control model.

[0010] Optionally, the method further includes: calculating the upper limit and lower limit of the safety distance between the target vehicle and the preceding vehicle based on the status data of the target vehicle at the current moment.

[0011] A second aspect of the present application provides an adaptive cruise control method, comprising the following steps: obtaining status data of a target vehicle and status data of a preceding vehicle at a current moment, wherein the target vehicle is deployed with an adaptive cruise model trained using the training method of the adaptive cruise model of the above-mentioned embodiment; inputting the status data of the target vehicle and status data of the preceding vehicle into the trained adaptive cruise model, and the trained adaptive cruise model outputting an acceleration control strategy for the target vehicle at the next moment; and controlling the adaptive cruise of the target vehicle based on the acceleration control strategy.

[0012] In a third aspect, an embodiment of the present application provides a training device for an adaptive cruise model, comprising: a first acquisition module for acquiring predefined driving cycle data of a target vehicle, wherein the predefined driving cycle data includes state parameters of a preceding vehicle and state parameters of the target vehicle; a training module for training an adaptive cruise model using the predefined driving cycle data, wherein the adaptive cruise model includes an action network and an evaluation network, wherein the action network outputs an acceleration control strategy of the target vehicle at the next moment based on the state data of the preceding vehicle and the state data of the target vehicle at the current moment, and the evaluation network evaluates the reward value of the acceleration control strategy based on a multi-objective reward function; an adjustment module for optimizing the weights of multiple sub-rewards in the multi-objective reward function based on the reward value during training, until the acceleration control strategy output by the adaptive cruise model meets preset requirements, and then stopping the training to obtain a trained adaptive cruise model, and using the trained adaptive cruise model to control the adaptive cruise of the target vehicle, wherein the sub-reward is the optimization target of the acceleration control strategy.

[0013] Optionally, the multi-objective reward function includes: a speed reward function, a safety reward function and a comfort reward function, wherein, The speed reward function is: ; in, is the speed of the target vehicle, is the preset speed, is the speed bonus value; The safety reward function is: ; in, is the relative distance between the target vehicle and the preceding vehicle, is the lower limit of the safety distance, is the following speed of the target vehicle, is the following acceleration of the target vehicle, As the basic safety reward value; The comfort reward function is: ; in, is the rate of change of the acceleration of the target vehicle, is the maximum acceleration of the target vehicle, is the minimum acceleration of the target vehicle, is the comfort bonus value; The multi-objective reward function is: ; in, is the multi-objective reward value, is the speed bonus value, is the weight of the speed reward value, is the safety bonus value, is the weight of the safety reward value, is the comfort bonus value, is the weight of the comfort reward value.

[0014] Optionally, the status data includes speed, acceleration, and relative distance.

[0015] Optionally, it also includes: a first calculation module, which is used to calculate the estimated time of collision between the target vehicle and the front vehicle based on the relative distance and speed between the target vehicle and the front vehicle at the current moment before the evaluation network evaluates the reward value of the acceleration control strategy based on the multi-objective reward function; and adjust the basic safety reward value output by the safety reward function based on the estimated time.

[0016] Optionally, it also includes: a correction module, which is used to correct the acceleration control strategy using the target intelligent driving model before the evaluation network evaluates the reward value of the acceleration control strategy based on the multi-objective reward function.

[0017] Optionally, the training module is further used to: normalize the state data of the preceding vehicle and the state data of the target vehicle to obtain a state observation vector; and input the state observation vector into the adaptive cruise model.

[0018] Optionally, the system further includes: a second calculation module, configured to calculate the upper limit and lower limit of the safety distance between the target vehicle and the preceding vehicle based on the status data of the target vehicle at a current moment.

[0019] An embodiment of the fourth aspect of the present application provides an adaptive cruise control device, including: a second acquisition module, used to obtain the status data of the target vehicle and the status data of the preceding vehicle at the current moment, wherein the target vehicle is deployed with an adaptive cruise model trained by a training device for the adaptive cruise model as in the above-mentioned embodiment; an input module, used to input the status data of the target vehicle and the status data of the preceding vehicle into the trained adaptive cruise model, and the trained adaptive cruise model outputs the acceleration control strategy of the target vehicle at the next moment; and a control module, used to control the adaptive cruise of the target vehicle based on the acceleration control strategy.

[0020] The fifth aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein the processor executes the program to perform the training method of the adaptive cruise model as described in the above embodiment.

[0021] The sixth aspect of the present application provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the adaptive cruise control method as described in the above embodiment.

[0022] The seventh aspect of the present application provides a computer-readable storage medium on which a computer program or instruction is stored. The computer program or instruction is executed by a processor to perform the training method of the adaptive cruise model or the adaptive cruise control method as described in the above embodiment.

[0023] Therefore, this application has at least the following beneficial effects: The embodiments of the present application can train an adaptive cruise control model using predefined driving cycle training data. During the training process, the weights of multiple sub-rewards in the multi-objective reward function are optimized based on the multi-objective reward function, enabling evaluation of the acceleration control strategy output by the adaptive cruise control model from multiple perspectives. This results in training an adaptive cruise control model that can balance and optimize multiple objectives. The trained adaptive cruise control model is then used to implement adaptive cruise control of a target vehicle, further achieving multi-objective comprehensive optimization of vehicle-following behavior, improving the collaborative optimization capabilities of the vehicle's adaptive cruise control, avoiding vehicle collisions, and enhancing vehicle safety and the user's driving experience. This solves the technical problem in related technologies that adaptive cruise control cannot simultaneously optimize multiple objectives.

[0024] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart of a training method for an adaptive cruise model provided in an embodiment of the present application; Figure 2 This is a flowchart of an adaptive cruise control method according to an embodiment of the present application; Figure 3 A flowchart of the training and application of an adaptive cruise control model according to one embodiment of the present application; Figure 4This is an example diagram of a training device for an adaptive cruise model provided according to an embodiment of the present application; Figure 5 This is an example diagram of an adaptive cruise control device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0027] Before describing the solution of this application, the related technologies of this application are first introduced.

[0028] Traditional automatic cruise control systems often use a single objective or fixed rules for control, making it difficult to simultaneously address multiple objectives: 1. Safety: Fixed safety distances or simple collision avoidance strategies may not adequately address emergencies in dynamic traffic scenarios. 2. Driving comfort: Sharp acceleration and deceleration, as well as frequent speed fluctuations, can reduce ride comfort, and traditional systems lack effective means of controlling jerk. 3. Speed ​​compliance and energy conservation: Some systems, while pursuing speed tracking, overlook energy consumption control and the risk of exceeding speed limits.

[0029] In addition, although some advanced research has begun to focus on the coordinated optimization of energy management and eco-driving of hybrid vehicles, multi-objective coordinated control still has problems such as low training efficiency, prone to collisions (or extreme states), and unstable regulation of some state variables.

[0030] To this end, the present application proposes a training method for an adaptive cruise model, an adaptive cruise control method and a device to solve at least one of the above technical problems.

[0031] Specifically, Figure 1 A flowchart of a training method for an adaptive cruise model provided in an embodiment of the present application.

[0032] like Figure 1 As shown, the training method of the adaptive cruise model includes the following steps: In step S101 , predefined driving cycle data of a target vehicle is acquired, wherein the predefined driving cycle data includes state data of a preceding vehicle and state data of the target vehicle.

[0033] The state data includes at least one of speed, acceleration, and relative distance; the predefined driving cycle data may be pre-collected time series data including the preceding vehicle and the target vehicle, where the target vehicle may also be referred to as a following vehicle.

[0034] In step S102, an adaptive cruise control model is trained using predefined driving cycle data, wherein the adaptive cruise control model includes an action network and an evaluation network. The action network outputs an acceleration control strategy for the target vehicle at the next moment based on the state data of the preceding vehicle and the state data of the target vehicle at the current moment. The evaluation network evaluates the reward value of the acceleration control strategy based on a multi-objective reward function.

[0035] Among them, the adaptive cruise model is a model based on reinforcement learning.

[0036] It can be understood that the embodiment of the present application uses predefined driving cycle data to train the adaptive cruise model, wherein the adaptive cruise model uses the actor-critic framework in reinforcement learning to train the model, specifically including an action actor network and an evaluation critic network. The action network is used to output the acceleration control strategy of the target vehicle at the next moment based on the state data of the preceding vehicle and the state data of the target vehicle at the current moment, and the evaluation network is used to evaluate the reward value of the acceleration control strategy based on a multi-objective reward function.

[0037] In the embodiment of the present application, the multi-objective reward function includes: a speed reward function, a safety reward function and a comfort reward function, wherein: The speed reward function is: ; in, is the speed of the target vehicle, is the preset speed, is the speed bonus value; The safety reward function is: ; in, is the relative distance between the target vehicle and the preceding vehicle, is the lower limit of the safety distance, is the following speed of the target vehicle, is the following acceleration of the target vehicle, As the basic safety reward value; The comfort reward function is: ; in, is the rate of change of the acceleration of the target vehicle, is the maximum acceleration of the target vehicle, is the minimum acceleration of the target vehicle, is the comfort bonus value; The multi-objective reward function is: ; in, is the multi-objective reward value, is the speed bonus value, is the weight of the speed reward value, is the safety bonus value, is the weight of the safety reward value, is the comfort bonus value, is the weight of the comfort reward value.

[0038] It can be understood that the embodiments of the present application can evaluate the acceleration control strategy output by the adaptive cruise model through a multi-objective reward function, thereby realizing the evaluation of the acceleration control strategy output by the adaptive cruise model from multiple angles, so as to achieve joint optimization of the target vehicle's following behavior and realize safe, efficient and energy-saving control of the target vehicle's following behavior.

[0039] In an embodiment of the present application, predefined driving cycle data is used to train an adaptive cruise control model, including: normalizing the state data of the preceding vehicle and the state data of the target vehicle to obtain a state observation vector; and inputting the state observation vector into the adaptive cruise control model.

[0040] It can be understood that the embodiment of the present application can normalize the state data of the preceding vehicle and the state data of the target vehicle to obtain a state observation vector, input the state observation vector into the adaptive cruise model, map the state parameters of different dimensions to a unified range, avoid gradient vanishing or explosion, and accelerate model training.

[0041] In an embodiment of the present application, before the evaluation network evaluates the reward value of the acceleration control strategy based on the multi-objective reward function, it also includes: calculating the estimated time of collision between the target vehicle and the front vehicle based on the relative distance and speed between the target vehicle and the front vehicle at the current moment; and adjusting the basic safety reward value output by the safety reward function based on the estimated time.

[0042] Among them, the estimated time can be measured by TTC (Time To Collision).

[0043] It is understandable that the embodiment of the present application can calculate the estimated time of collision between the target vehicle and the preceding vehicle based on the relative distance and speed between the target vehicle and the preceding vehicle at the current moment; adjust the basic safety reward value output by the safety reward function based on the estimated time, specifically, TTC measures the estimated time it takes for the current vehicle to collide with the vehicle ahead. When TTC is less than 1, it indicates a high risk of collision. This application adjusts the safety reward by adjusting the TTC value: ; The TTC formula is calculated as the distance between the target vehicle and the preceding vehicle divided by their relative speed. This calculation allows us to determine the expected time to collision and make timely adjustments to the impending collision. If the TTC is less than 1 second, a large negative reward is used; if the TTC value is between 1 and 3 seconds, a logarithmic adjustment of the reward is used: .

[0044] An embodiment of the present application provides a multi-objective reward function, which consists of a speed reward, a safety reward, and a comfort reward.

[0045] Speed ​​Reward: When the vehicle's predicted speed exceeds the preset speed limit When , a large negative reward is given, and the reward is set to - , otherwise the reward is 0.

[0046] ; (1) Safety bonus: The safety bonus is calculated by comparing the actual distance between the vehicle and the vehicle ahead with the dynamic safety distance ( and ) deviations; at the same time, TTC (Time To Collision) is combined to make segmented adjustments to the state close to collision.

[0047] If the distance between the vehicles is too small or too large, a negative reward or a smaller reward will be given respectively: ; (2) TTC measures the estimated time it takes for the current vehicle to collide with the vehicle ahead. When TTC is less than 1, it indicates a high risk of collision. This application adjusts the safety reward by adjusting the TTC value: ; (3) The TTC formula is calculated as the distance between the vehicle and the preceding vehicle divided by their relative speed. This calculation allows us to determine the expected time to collision and make timely adjustments to the impending collision. If the TTC is less than 1 second, a large negative reward is used; if the TTC value is between 1 and 3 seconds, a logarithmic adjustment of the reward is used: ; (4) Comfort Reward: Normalizes the vehicle's jerk to avoid sudden acceleration or deceleration. If the vehicle's jerk is high, a negative reward is given to discourage aggressive driving and improve ride comfort.

[0048] ; (5) If the system is in a good security state (i.e. >0), the penalty coefficient for jerk is further increased to ensure a smooth and comfortable driving process.

[0049] The total reward formula is: .

[0050] In an embodiment of the present application, before the evaluation network evaluates the reward value of the acceleration control strategy based on the multi-objective reward function, it also includes: using the target intelligent driving model to correct the acceleration control strategy.

[0051] Among them, the target intelligent driving model is the improved IDM model.

[0052] It can be understood that the embodiments of the present application can use the target intelligent driving model, that is, the improved IDM model to correct the acceleration control signal output by the adaptive cruise model to ensure that deceleration is prioritized when the vehicle spacing is insufficient, and combine reinforcement learning with the traditional model. The prior knowledge of IDM is used to make up for the shortcomings of the pure data-driven method, which can accelerate model convergence and reduce training time. At the same time, it can also improve the robustness of the model in extreme scenarios (such as vehicles suddenly cutting in) and avoid unstable behavior that may occur in pure reinforcement learning.

[0053] In step S103, during the training process, the weights of multiple sub-rewards in the multi-objective reward function are optimized based on the reward value until the acceleration control strategy output by the adaptive cruise model meets the preset requirements, and the training is stopped to obtain a trained adaptive cruise model. The trained adaptive cruise model is used to control the adaptive cruise of the target vehicle, wherein the sub-reward is the optimization target of the acceleration control strategy.

[0054] Among them, the preset requirement is that the output acceleration control strategy can achieve zero collision.

[0055] It can be understood that the embodiment of the present application can adjust the weights of multiple sub-rewards of the multi-objective reward function based on the reward value until the acceleration control strategy output by the adaptive cruise model meets the preset requirements and stops training, thereby training an adaptive cruise model that can balance multiple objectives, and using the trained adaptive cruise model to implement adaptive cruise control of the target vehicle, thereby achieving multi-objective comprehensive optimization of vehicle following behavior, avoiding vehicle collisions, and improving vehicle safety and user driving experience.

[0056] In an embodiment of the present application, it also includes: calculating the upper and lower limits of the safety distance between the target vehicle and the preceding vehicle based on the status data of the target vehicle at the current moment.

[0057] It can be understood that the embodiment of the present application can dynamically calculate the upper and lower limits of the safety distance based on the status data of the target vehicle at the current moment. Compared with a fixed safety distance, dynamic adjustment is more in line with actual driving needs and improves driving safety.

[0058] According to the training method of the adaptive cruise model proposed in the embodiment of the present application, the adaptive cruise model can be trained using predefined driving cycle training data, and the weights of multiple sub-rewards in the multi-objective reward function can be optimized based on the multi-objective reward function during the training process, so as to realize the evaluation of the acceleration control strategy output by the adaptive cruise model from multiple angles, thereby training an adaptive cruise model that can balance multiple objectives, and using the trained adaptive cruise model to realize adaptive cruise control of the target vehicle, thereby realizing multi-objective comprehensive optimization of the vehicle following behavior, avoiding vehicle collisions, and improving vehicle safety and the user's driving experience.

[0059] The above embodiment focuses on explaining adaptive cruise control from the perspective of model training, while the following embodiment focuses on describing adaptive cruise control from the perspective of specific application of the model. The embodiments may refer to each other for any incomplete details.

[0060] An embodiment of the present application also provides an adaptive cruise control method.

[0061] like Figure 2 As shown, the adaptive cruise control method includes the following steps: In step S201, the state data of the target vehicle and the state data of the preceding vehicle at the current moment are obtained, wherein the target vehicle is deployed with an adaptive cruise model trained by the adaptive cruise model training method of the above embodiment.

[0062] The status data includes speed, acceleration and relative distance.

[0063] In step S202, the state data of the target vehicle and the state data of the preceding vehicle are input into the trained adaptive cruise control model, and the trained adaptive cruise control model outputs the acceleration control strategy of the target vehicle at the next moment.

[0064] In step S203 , the adaptive cruise control of the target vehicle is controlled based on the acceleration control strategy.

[0065] According to the adaptive cruise control method proposed in the embodiment of the present application, adaptive cruise control of the vehicle can be achieved by deploying a trained adaptive cruise model on the target vehicle, thereby achieving multi-objective comprehensive optimization of the vehicle's following behavior, avoiding vehicle collisions, and improving vehicle safety and the user's driving experience.

[0066] The following describes the training and application process of the adaptive cruise model of the embodiment of the present application through a specific embodiment. It can be deployed in an adaptive cruise control system and described in the form of modules in the adaptive cruise control system, including a data acquisition module, a state preprocessing module, a control decision module, a state update module and a reward calculation module, and a closed-loop feedback is formed between the modules.

[0067] The specific process of this application is described in combination with the above modules, such as Figure 3 Shown, including: S1: Data collection and model building.

[0068] Use an external module to obtain predefined driving cycle data, including the speed list (SPEED_LIST) and acceleration list (ACC_LIST) of the preceding vehicle.

[0069] A normalization function is defined to normalize the state variables such as the speed, acceleration and relative distance of the following vehicle to the leading vehicle, and to construct an observation vector for use in the subsequent control decision module.

[0070] S2: Construction of dynamic vehicle following model.

[0071] This application establishes a vehicle kinematic model that dynamically updates velocity, acceleration, and jerk. Within this model, an improved Intelligent Driving Model (IDM) is used to modify the acceleration control signal output by the reinforcement learning model to prioritize deceleration when the distance between vehicles is insufficient. Furthermore, the lower and upper safety distance limits are dynamically calculated based on the current speed of the following vehicle, ensuring safe driving under various operating conditions.

[0072] S3: Multi-objective reward function design.

[0073] The core of this application is to design a multi-objective reward function, which consists of speed reward, safety reward, and comfort reward.

[0074] Speed ​​Reward: When the vehicle's predicted speed exceeds the preset speed limit When , a large negative reward is given, and the reward is set to - , otherwise the reward is 0.

[0075] ; (1) Safety bonus: The safety bonus is calculated by comparing the actual distance between the vehicle and the vehicle ahead with the dynamic safety distance ( and ) deviations; at the same time, TTC (Time To Collision) is combined to make segmented adjustments to the state close to collision.

[0076] If the distance between the vehicles is too small or too large, a negative reward or a smaller reward will be given respectively: ; (2) TTC measures the estimated time it takes for the current vehicle to collide with the vehicle ahead. When TTC is less than 1, it indicates a high risk of collision. This application adjusts the safety reward by adjusting the TTC value: ; (3) The TTC formula is calculated as the distance between the vehicle and the preceding vehicle divided by their relative speed. This calculation allows us to determine the expected time to collision and make timely adjustments to the impending collision. If the TTC is less than 1 second, a large negative reward is used; if the TTC value is between 1 and 3 seconds, a logarithmic adjustment of the reward is used: ; (4) Comfort Reward: Normalizes the vehicle's jerk to avoid sudden acceleration or deceleration. If the vehicle's jerk is high, a negative reward is given to discourage aggressive driving and improve ride comfort.

[0077] ; (5) If the system is in a good security state (i.e. >0), the penalty coefficient for jerk is further increased to ensure a smooth and comfortable driving process.

[0078] The total reward formula is: ,This reward function comprehensively considers the safety, comfort, and speed compliance of the vehicle, ensuring collaborative optimization in different driving environments, thereby achieving the best driving experience and energy saving effect.

[0079] In summary, the adaptive cruise control model training method and the trained adaptive cruise control model of the present application can produce the following effects, including: 1. Improved safety: Dynamically calculated lower and upper limits for safe distances, combined with segmented penalties for the Time to Collision (TTC) metric, provide strong negative rewards for potential collisions, significantly reducing accident risk. 2. Improved driving comfort: Introducing constraints on the vehicle's acceleration rate of change (jerk) into the reward function to suppress sudden acceleration and deceleration, improving driving smoothness and ride comfort; 3. Speed ​​compliance and energy-saving optimization: When a vehicle exceeds the preset speed limit, a large negative reward is applied. By comprehensively considering distance and speed changes during the following process, speed errors are corrected in real time to ensure that the vehicle operates within a safe and energy-saving range. 4. Better collaborative optimization performance: This application combines a rule-based control algorithm (such as the IDM model) with a multi-objective reinforcement learning algorithm to organically integrate the vehicle's current state with future driving information, thereby achieving joint optimization of the vehicle's following behavior, thereby achieving zero collisions and ensuring state stability (such as SOC stability) during training, while improving training efficiency.

[0080] This application minimizes energy consumption while maintaining safety and achieves comprehensive optimization of vehicle following behavior. It is suitable for ecological driving control of hybrid and other types of vehicles.

[0081] Next, the training device for the adaptive cruise model and the adaptive cruise control device according to the embodiments of the present application will be described with reference to the accompanying drawings.

[0082] Figure 4 4 is a block diagram of a training device for an adaptive cruise model according to an embodiment of the present application.

[0083] like Figure 4 As shown, the training device 10 of the adaptive cruise model includes: a first acquisition module 101 , a training module 102 and an adjustment module 103 .

[0084] Among them, the first acquisition module 101 is used to obtain predefined driving cycle data of the target vehicle, wherein the predefined driving cycle data includes state parameters of the preceding vehicle and state parameters of the target vehicle; the training module 102 is used to train the adaptive cruise model using the predefined driving cycle data, wherein the adaptive cruise model includes an action network and an evaluation network, wherein the action network outputs the acceleration control strategy of the target vehicle at the next moment based on the state data of the preceding vehicle and the state data of the target vehicle at the current moment, and the evaluation network evaluates the reward value of the acceleration control strategy based on a multi-objective reward function; the adjustment module 103 is used to optimize the weights of multiple sub-rewards in the multi-objective reward function based on the reward value during the training process, until the acceleration control strategy output by the adaptive cruise model meets the preset requirements, then stop training, and obtain a trained adaptive cruise model, and use the trained adaptive cruise model to control the adaptive cruise of the target vehicle, wherein the sub-reward is the optimization target of the acceleration control strategy.

[0085] In the embodiment of the present application, the multi-objective reward function includes: a speed reward function, a safety reward function and a comfort reward function, wherein: The multi-objective reward function includes: speed reward function, safety reward function and comfort reward function, among which, The speed reward function is: ; in, is the speed of the target vehicle, is the preset speed, is the speed bonus value; The safety reward function is: ; in, is the relative distance between the target vehicle and the preceding vehicle, is the lower limit of the safety distance, is the following speed of the target vehicle, is the following acceleration of the target vehicle, As the basic safety reward value; The comfort reward function is: ; in, is the rate of change of the acceleration of the target vehicle, is the maximum acceleration of the target vehicle, is the minimum acceleration of the target vehicle, is the comfort bonus value; The multi-objective reward function is: ; in, is the multi-objective reward value, is the speed bonus value, is the weight of the speed reward value, is the safety bonus value, is the weight of the safety reward value, is the comfort bonus value, is the weight of the comfort reward value.

[0086] In the embodiment of the present application, the status data includes speed, acceleration, and relative distance.

[0087] In the embodiment of the present application, the device 10 of the embodiment of the present application further includes: a first calculation module.

[0088] Among them, the first calculation module is used to calculate the estimated time when the target vehicle will collide with the preceding vehicle based on the relative distance and speed between the target vehicle and the preceding vehicle at the current moment before the evaluation network evaluates the reward value of the acceleration control strategy based on the multi-objective reward function; and adjust the basic safety reward value output by the safety reward function based on the estimated time.

[0089] In the embodiment of the present application, the device 10 of the embodiment of the present application further includes: a correction module.

[0090] Among them, the correction module is used to use the target intelligent driving model to correct the acceleration control strategy before the evaluation network evaluates the reward value of the acceleration control strategy based on the multi-objective reward function.

[0091] In the embodiment of the present application, the training module 200 is further used to: normalize the state data of the preceding vehicle and the state data of the target vehicle to obtain a state observation vector; and input the state observation vector into the adaptive cruise model.

[0092] In the embodiment of the present application, the device 10 of the embodiment of the present application further includes: a second calculation module.

[0093] The second calculation module is used to calculate the upper limit and lower limit of the safety distance between the target vehicle and the preceding vehicle according to the status data of the target vehicle at the current moment.

[0094] It should be noted that the above explanation of the embodiment of the training method of the adaptive cruise model is also applicable to the training device of the adaptive cruise model of this embodiment, and will not be repeated here.

[0095] According to the training device of the adaptive cruise model proposed in the embodiment of the present application, the adaptive cruise model can be trained using predefined driving cycle training data, and the weights of multiple sub-rewards in the multi-objective reward function are optimized based on the multi-objective reward function during the training process, so as to realize the evaluation of the acceleration control strategy output by the adaptive cruise model from multiple angles, thereby training an adaptive cruise model that can balance multiple objectives, and using the trained adaptive cruise model to realize adaptive cruise control of the target vehicle, thereby realizing multi-objective comprehensive optimization of vehicle following behavior, avoiding vehicle collisions, and improving vehicle safety and user driving experience.

[0096] Figure 5 4 is a block diagram of an adaptive cruise control device according to an embodiment of the present application.

[0097] like Figure 5 As shown, the adaptive cruise control device 20 includes: a second acquisition module 201 , an input module 202 and a control module 203 .

[0098] Among them, the second acquisition module 201 is used to obtain the status data of the target vehicle and the status data of the preceding vehicle at the current moment, wherein the target vehicle is deployed with an adaptive cruise model trained by the training device of the adaptive cruise model as in the above embodiment; the input module 202 is used to input the status data of the target vehicle and the status data of the preceding vehicle into the trained adaptive cruise model, and the trained adaptive cruise model outputs the acceleration control strategy of the target vehicle at the next moment; the control module 203 is used to control the adaptive cruise of the target vehicle based on the acceleration control strategy.

[0099] According to the adaptive cruise control device proposed in the embodiment of the present application, adaptive cruise control of the vehicle can be achieved by deploying a trained adaptive cruise model on the target vehicle, thereby achieving multi-objective comprehensive optimization of the vehicle's following behavior, avoiding vehicle collisions, and improving vehicle safety and the user's driving experience.

[0100] An embodiment of the present application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the above-mentioned training method for the adaptive cruise model.

[0101] An embodiment of the present application further provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the above-described adaptive cruise control method.

[0102] An embodiment of the present application also provides a computer-readable storage medium having a computer program or instruction stored thereon. When the computer program or instruction is executed by a processor, the training method of the adaptive cruise model or the adaptive cruise control method as described above is implemented.

[0103] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0104] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0105] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0106] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, it can be implemented using any one or a combination of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0107] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

Claims

1. A training method for an adaptive cruise model, characterized in that: The following steps are involved: Acquiring predefined driving cycle data of a target vehicle, wherein the predefined driving cycle data includes status data of a preceding vehicle and status data of the target vehicle; Training an adaptive cruise control model using the predefined driving cycle data, wherein the adaptive cruise control model includes an action network and an evaluation network, wherein the action network outputs an acceleration control strategy for the target vehicle at a next moment based on current state data of the preceding vehicle and state data of the target vehicle, and the evaluation network evaluates a reward value for the acceleration control strategy based on a multi-objective reward function; During the training process, the weights of multiple sub-rewards in the multi-objective reward function are optimized based on the reward value until the acceleration control strategy output by the adaptive cruise model meets preset requirements, and the training is stopped to obtain a trained adaptive cruise model. The trained adaptive cruise model is used to control the adaptive cruise of the target vehicle, wherein the sub-reward is the optimization target of the acceleration control strategy.

2. The training method of the adaptive cruise model according to claim 1, characterized in that: The multi-objective reward function includes: a speed reward function, a safety reward function and a comfort reward function, wherein: The speed reward function is: ； in, is the speed of the target vehicle, is the preset speed, is the speed bonus value; The safety reward function is: ； in, is the relative distance between the target vehicle and the preceding vehicle, is the lower limit of the safety distance, is the following speed of the target vehicle, is the following acceleration of the target vehicle, As the basic safety reward value; The comfort reward function is: ； in, is the rate of change of the acceleration of the target vehicle, is the maximum acceleration of the target vehicle, is the minimum acceleration of the target vehicle, is the comfort bonus value; The multi-objective reward function is: ； in, is the multi-objective reward value, is the speed bonus value, is the weight of the speed reward value, is the safety bonus value, is the weight of the safety reward value, is the comfort bonus value, is the weight of the comfort reward value.

3. The training method of the adaptive cruise model according to claim 2, characterized in that: The state data includes speed, acceleration, and relative distance. Before evaluating the reward value of the acceleration control strategy based on the multi-objective reward function, the state data also includes: Calculating an estimated time of collision between the target vehicle and the preceding vehicle based on the relative distance and speed between the target vehicle and the preceding vehicle at the current moment; A base safety reward value output by the safety reward function is adjusted based on the estimated time.

4. The training method of the adaptive cruise model according to claim 1, characterized in that: Before the evaluation network evaluates the reward value of the acceleration control strategy based on the multi-objective reward function, the method further includes: The acceleration control strategy is modified using a target intelligent driving model.

5. The training method of the adaptive cruise model according to claim 1, characterized in that: The method of training the adaptive cruise control model using the predefined driving cycle data includes: Normalizing the state data of the preceding vehicle and the state data of the target vehicle to obtain a state observation vector; The state observation vector is input into the adaptive cruise model.

6. The training method of the adaptive cruise model according to claim 1, characterized in that: Also includes: The upper limit and lower limit of the safety distance between the target vehicle and the preceding vehicle are calculated based on the state data of the target vehicle at the current moment.

7. An adaptive cruise control method, characterized in that: The following steps are involved: Acquire state data of a target vehicle and state data of a preceding vehicle at a current moment, wherein the target vehicle is deployed with an adaptive cruise model trained using the adaptive cruise model training method according to any one of claims 1 to 6; Inputting the state data of the target vehicle and the state data of the preceding vehicle into a trained adaptive cruise control model, the trained adaptive cruise control model outputting an acceleration control strategy for the target vehicle at a next moment; The adaptive cruise control of the target vehicle is controlled based on the acceleration control strategy.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the training method for the adaptive cruise model according to any one of claims 1 to 6.

9. A vehicle, characterized in that: It is characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the adaptive cruise control method according to claim 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: The computer program or instruction is executed by a processor to implement the training method of the adaptive cruise model according to any one of claims 1 to 6, or the adaptive cruise control method according to any one of claim 7.

Images