A Coupling Control Method for Gear Shifting and Speed ​​Tracking in a Car Driving Robot

Abstract

This invention discloses a coupled control method for gear shifting and speed tracking in a car driving robot. The method includes: the car driving robot acquiring a trajectory tracking task and collecting environmental state vectors related to the car driving robot and the vehicle under test at the current moment; inputting the state vectors into a trained strategy model; the strategy model outputting the action command for the car driving robot at the next moment; the driving robot executing the action command, generating the state vector for the next moment, and updating the shared state space; wherein, the strategy model includes a gear shifting agent strategy model and a pedal agent strategy model; the gear shifting agent strategy model generates a discrete action probability distribution and selects the action with the highest probability as the expected gear shifting action for the next moment; the pedal agent strategy model concatenates the discrete action probability distribution with the shared state information and outputs the pedal action value for the next moment. This invention reduces overall vehicle fuel consumption, improves vehicle speed tracking accuracy, and enhances engineering stability.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more specifically to a method for coupled control of gear shifting and speed tracking in a car driving robot. Background Technology

[0002] The automotive industry requires extensive fuel consumption and emissions testing during product development, quality inspection, and regulatory review. These tests are typically conducted on a chassis dynamometer test bench, involving vehicle speed tracking according to a pre-defined target speed curve. The target curve adheres to relevant testing standards, commonly including WLTC, NEDC, CLTC, and EPA.

[0003] If manual driving is performed on a chassis dynamometer test bench to track the speed of a vehicle, there are problems such as high safety risks, high labor intensity, high labor costs, large test errors, and poor repeatability, which are particularly prominent for vehicles with manual shifting, resulting in serious deficiencies in test accuracy and repeatability. Based on this, in recent years, vehicle testing driving robots have gradually replaced manual driving. The driving robot system consists of a gear-shifting robot and a pedal robot: the gear-shifting robot is responsible for clutch operation, gear selection, and gear engagement; the pedal robot controls the accelerator and brake pedal actions; the two mechanisms work closely together to achieve a smooth and continuous accelerator, clutch, and gear-shifting process. Driving robots can be universally installed in the vehicle's cockpit without requiring modifications to the test vehicle and have now become one of the important automotive testing equipment.

[0004] Currently, the mainstream method for controlling driving robots to perform speed tracking in manual transmission (MT) vehicles is to decouple the pedal robot from the gear shifting robot: the gear shifting robot makes gear shifting decisions based on a pre-calibrated gear shifting rule table and real-time engine speed readings; the pedal robot collects vehicle speed signals and executes throttle or braking operations based on PID control. However, due to the nonlinear characteristics of the vehicle itself, the speed tracking accuracy of simple PID control still has considerable room for improvement; calibrating the gear shifting rule table for MT vehicles is labor-intensive and difficult to achieve optimal fuel consumption. Furthermore, this completely decoupled control architecture severs the intrinsic connection between throttle / braking and gear shifting decisions, making it difficult to synergistically improve speed tracking accuracy and overall fuel efficiency.

[0005] Achieving human-like control and comprehensive optimization of multi-objective performance is an important development direction for the control of driving robots. Human-like control refers to taking into account human driving habits and coordinating gear shifting decisions and throttle operation to achieve coupled and coordinated control of the two; multi-objective optimization covers the coordinated optimization of comprehensive performance such as tracking accuracy, fuel consumption, and operation smoothness.

[0006] Deep reinforcement learning (DRL) is an important method for solving multi-input, multi-objective optimization control problems. DRL learns policies by continuously interacting with the environment, aiming to maximize cumulative rewards. It does not require a precise mathematical model of the system and possesses strong nonlinear adaptability, demonstrating significant advantages in complex control tasks. However, the control problem of driving robots involves both continuous throttle / brake control and discrete decision-making regarding gear shifting timing, forming a hybrid action space. When a single DRL agent handles such problems, it requires special network structure design or action space parameterization for both continuous and discrete actions, resulting in high algorithm complexity. Furthermore, policy convergence is difficult under the sequential temporal coupling constraints of gear shifting and pedal control, hindering multi-objective collaborative optimization. Multi-agent reinforcement learning (MARL) offers a new approach by decomposing the hybrid action space into multiple subspaces, with different agents undertaking continuous or discrete decision-making tasks, fundamentally reducing the action space complexity of a single agent. However, the decision-making logic of gear shifting and pedal control in driving robots has a sequential dependency, which differs from the parallel decision-making paradigm of standard MARL. Currently, there is no mature methodological paradigm in the MARL field that can be directly adopted for complex control scenarios that simultaneously involve serial timing dependencies, continuous-discrete hybrid action spaces, and multi-objective collaborative optimization. Summary of the Invention

[0007] In view of this, the present invention provides a coupling control method for gear shifting and speed tracking of an automotive driving robot, which can solve the above-mentioned technical problems.

[0008] To solve the above-mentioned technical problems, the present invention is implemented as follows.

[0009] A method for coupled control of gear shifting and speed tracking in a car driving robot includes:

[0010] Step S1: The car driving robot acquires the trajectory tracking task and collects the environmental state vectors related to the car driving robot and the vehicle under test at the current moment; the vehicle under test is the vehicle driven by the car driving robot. Step S2: Input the state vector into the trained policy model; the policy model outputs the action command for the car driving robot at the next moment; Step S3: Drive the robot to execute action commands, generate the state vector for the next moment, and update the shared state space; Among them, the strategy models include the shift agent strategy model and the pedal agent strategy model; The shifting agent strategy model generates a discrete action probability distribution based on the shared state information in the shared state space at the current moment and selects the action with the highest probability as the expected shifting action at the next moment. The pedal agent strategy model concatenates the discrete action probability distribution with the shared state information, and outputs the pedal action value at the next moment based on the concatenated state information. The shift filter determines the confidence level corresponding to the expected shift action at the next moment, and combines it with the shift action at the current moment to determine the actual shift action at the next moment; The actual shift command and pedal command for the next moment are determined based on the actual shift action and pedal action value for the next moment, respectively. The driving robot executes the actual gear shifting and pedal operation commands for the next moment.

[0011] Preferably, a shared state space is used. In 9-dimensional space, it is represented as follows:

[0012] in, For vehicle speed error, This is the actual vehicle speed. For the current acceleration, Engine speed, For the previous gear, The value of the pedal action at the previous moment. This is the time elapsed since the last gear shift. and These are the desired acceleration and the desired jerk, respectively.

[0013] Preferably, the desired acceleration and desired jerk are calculated based on the reference vehicle speed trajectory corresponding to the prediction time window of the trajectory tracking task. Specifically, the instantaneous acceleration of each sampling step within the prediction time window is weighted by the first exponential decay coefficient and the average value is calculated to obtain the desired acceleration; the acceleration difference between adjacent sampling steps is weighted by the second exponential decay coefficient and the average value is calculated to obtain the desired jerk. Expected acceleration With expected acceleration The calculation formula is as follows:

[0014]

[0015] in, The sampling interval is... To predict the number of sampling steps within the time window, and They are respectively Time and The target speed at any given moment and They are respectively Time and The target acceleration at any given moment; This is the first exponential decay coefficient. The second exponential decay coefficient is defined as follows:

[0016] in, This is the attenuation coefficient.

[0017] Preferably, the action space of the shift agent strategy model is a discrete set of shift actions, and the action space of the pedal agent strategy model is a continuous set of accelerator pedal and brake pedal actions. Action space of shifting agent strategy model These correspond to three types of actions: downshifting, holding, and upshifting; the action space of the pedal agent strategy model is a continuous interval. Positive values ​​correspond to the normalized accelerator pedal opening, and negative values ​​correspond to the normalized brake pedal opening.

[0018] in, , These refer to the travel of the accelerator and brake pedals, respectively. , These represent the corresponding maximum travel distances.

[0019] Preferably, at each decision-making moment, the shifting agent's policy model outputs a discrete action probability distribution of the complete action space based on the current shared state information st. The pedal agent policy model combines it with splicing to form an enhanced observation vector :

[0020] in, This indicates splicing; the shift agent policy model and the pedal agent policy model execute their respective actions synchronously, and the joint action probability distribution is decomposed according to the chain rule as follows:

[0021] in, For the current gear shift action, For the current pedal action, This represents the output probability of the gear shift action at the current moment. This represents the output probability of the pedal action at the current moment.

[0022] Preferably, the shift filter determines the confidence level corresponding to the expected shift action at the next moment, and combines it with the shift action at the current moment to determine the actual shift action at the next moment, wherein: The formula for calculating confidence level is as follows:

[0023] in, Predict actions for the next moment; Confidence level greater than or equal to preset threshold If the expected gear shift is to be performed at the next moment, then the actual gear shift will be performed at the next moment; otherwise, the gear shift will be performed at the current moment.

[0024] Preferably, the reward function used by the policy model during training is as follows:

[0025] in, Represents the agent policy model. , express Share rewards at all times; Values hour, express The constantly shifting intelligent agent strategy model enjoys exclusive rewards; Values hour, express The instant pedal intelligent agent strategy model enjoys exclusive rewards; The allocation coefficients for shared rewards and exclusive rewards are as follows:

[0026]

[0027]

[0028] in, , , , , , These are the weighting coefficients for each reward or penalty item. for Real-time vehicle speed tracking reward items, for Fuel economy incentives at all times for Constant engine speed constraint penalty item, for Constant gear shifting constraint penalty item, for Always pedal smoothness bonus, for Constraints on the speed of pedal operation at all times.

[0029] A coupling control device for gear shifting and speed tracking in a car driving robot includes: Data acquisition module: configured for the trajectory tracking task of the car driving robot, collecting environmental state vectors related to the car driving robot and the vehicle under test at the current moment; the vehicle under test is the vehicle driven by the car driving robot; Action command generation module: configured to input the state vector into the trained policy model; the policy model outputs the action command for the car driving robot at the next moment; State update module: Configured to execute action commands for the driving robot, generate the state vector for the next moment, and update the shared state space; Among them, the strategy models include the shift agent strategy model and the pedal agent strategy model; The shifting agent strategy model generates a discrete action probability distribution based on the shared state information in the shared state space at the current moment and selects the action with the highest probability as the expected shifting action at the next moment. The pedal agent strategy model concatenates the discrete action probability distribution with the shared state information, and outputs the pedal action value at the next moment based on the concatenated state information. The shift filter determines the confidence level corresponding to the expected shift action at the next moment, and combines it with the shift action at the current moment to determine the actual shift action at the next moment; The actual shift command and pedal command for the next moment are determined based on the actual shift action and pedal action value for the next moment, respectively. The driving robot executes the actual gear shifting and pedal operation commands for the next moment.

[0030] The present invention provides an electronic device, characterized in that the electronic device comprises: A processor is used to execute multiple instructions; Memory, used to store multiple instructions; The plurality of instructions are to be stored in the memory and loaded and executed by the processor as described above.

[0031] The present invention provides an electronic device, characterized in that the electronic device comprises: A processor is used to execute multiple instructions; Memory, used to store multiple instructions; The plurality of instructions are to be stored in the memory and loaded and executed by the processor as described above.

[0032] The present invention provides a non-transitory computer-readable storage medium storing computer instructions, characterized in that the computer instructions are used to cause the computer to perform the method described above.

[0033] This invention breaks down the overall vehicle control task of the driving robot into two cooperative reinforcement learning agents: a gear shifting agent is used to select the optimal discrete gear, and a pedal agent is responsible for the continuous quantity control of throttle and brake. At the same time, it is designed with four core mechanisms: shared state space between the two agents, interactive communication for action decision, joint credit allocation, and gear shift decision filtering execution.

[0034] The present invention has the following beneficial effects: (1) This invention effectively reduces the fuel consumption of the vehicle. By having the shifting agent autonomously learn the optimal shifting strategy, under the premise that the vehicle speed tracking accuracy meets the standard requirements, it can achieve a significant reduction in fuel consumption compared with the traditional standard shifting scheme and the lookup table shifting scheme. The fuel economy is close to the global optimal level of dynamic programming, and at the same time, it does not require a high-precision vehicle model and offline global calculation.

[0035] (2) This invention improves vehicle speed tracking accuracy. This invention uses deep reinforcement learning to train the pedal agent, which learns the nonlinear dynamic mapping relationship of the vehicle through continuous interaction with the environment, thus significantly improving vehicle speed tracking accuracy. At the same time, it simultaneously optimizes multiple control objectives such as vehicle speed tracking accuracy and pedal operation smoothness within a single strategy framework, overcoming the inherent limitations of single-objective optimization in traditional piecewise PID schemes.

[0036] (3) This invention improves the engineering stability of the training and deployment of the control system. Through the collaborative design of the action interaction communication mechanism and the joint credit allocation mechanism, the instability and convergence difficulties that are prone to occur in single agent training in the mixed action space are solved, and the stability of dual agent collaborative training is guaranteed. The trained strategy can be directly deployed on the physical driving robot platform, and the simulated gear shifting behavior is highly consistent with that of the real vehicle, which has good engineering usability. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the coupling control method for gear shifting and speed tracking of the car driving robot of the present invention.

[0038] Figure 2 This is another schematic diagram of the coupling control method for gear shifting and speed tracking of the car driving robot of the present invention.

[0039] Figure 3 This is a schematic diagram of the training strategy model of the present invention. Detailed Implementation

[0040] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] like Figures 1-2 As shown, specifically, the control steps are as follows: Figure 1 As shown, the specific control execution logic after system deployment is as follows: Figure 2 As shown, this invention proposes a coupled control method for gear shifting and speed tracking in a car driving robot, comprising: Step S1: The car driving robot acquires the trajectory tracking task and collects the environmental state vectors related to the car driving robot and the vehicle under test at the current moment; the vehicle under test is the vehicle driven by the car driving robot. Step S2: Input the state vector into the trained policy model; the policy model outputs the action command for the car driving robot at the next moment; Step S3: Drive the robot to execute action commands, generate the state vector for the next moment, and update the shared state space; Among them, the strategy models include the shift agent strategy model and the pedal agent strategy model; The shifting agent strategy model generates a discrete action probability distribution based on the shared state information in the shared state space at the current moment and selects the action with the highest probability as the expected shifting action at the next moment. The pedal agent strategy model concatenates the discrete action probability distribution with the shared state information, and outputs the pedal action value at the next moment based on the concatenated state information. The shift filter determines the confidence level corresponding to the expected shift action at the next moment, and combines it with the shift action at the current moment to determine the actual shift action at the next moment; The actual shift command and pedal command for the next moment are determined based on the actual shift action and pedal action value for the next moment, respectively. The driving robot executes the actual gear shifting and pedal operation commands for the next moment.

[0042] Furthermore, the state vector stored in the shared state space is a global environment state vector, including the vehicle speed signal and engine speed signal collected by the control computer, pre-stored target vehicle speed information, etc., and also includes the previous historical actions of the two intelligent agent policy models embedded in the shared state, thus constituting a two-way exchange of historical action information, providing a unified environmental perception basis for collaborative decision-making. In the embodiments of the present invention, the shared state space... The design is based on 9-dimensional space, represented as follows:

[0043] in, For vehicle speed error, This is the actual vehicle speed. For the current acceleration, Engine speed, For the previous gear, The value of the pedal action at the previous moment. This is the time elapsed since the last gear shift. and These are the desired acceleration and the desired jerk, respectively.

[0044] Furthermore, the expected acceleration and expected jerk are calculated based on the trajectory tracking task and the corresponding reference vehicle speed trajectory within the prediction time window. Specifically, the instantaneous acceleration of each sampling step within the prediction time window is weighted and averaged according to the first exponential decay coefficient to obtain the expected acceleration; the acceleration difference between adjacent sampling steps is weighted and averaged according to the second exponential decay coefficient to obtain the expected jerk; wherein, the prediction time window is set to 1 second.

[0045] Expected acceleration With expected acceleration The calculation formula is as follows:

[0046]

[0047] in, The sampling interval is... To predict the number of sampling steps within the time window, and They are respectively Time and The target speed at any given moment and They are respectively Time and The target acceleration at any given moment; This is the first exponential decay coefficient. The second exponential decay coefficient is defined as follows:

[0048] In the above formula, As the attenuation coefficient, in this embodiment of the invention, The value is set to 0.7. The attenuation coefficient design gives a higher weight to the proximal sampling step, making the state variable more sensitive to the immediate motion trend. The larger the attenuation coefficient, the higher the weight of the proximal sampling step, making the state variable more sensitive to the immediate motion trend.

[0049] Furthermore, the action space of the shift agent strategy model is a discrete set of shift actions, while the action space of the pedal agent strategy model is a continuous set of accelerator pedal and brake pedal actions.

[0050] In this embodiment of the invention, the action space of the shifting agent strategy model These correspond to three types of actions: downshifting, holding, and upshifting; the action space of the pedal agent strategy model is a continuous interval. Positive values ​​correspond to the normalized accelerator pedal opening, and negative values ​​correspond to the normalized brake pedal opening, specifically defined as follows:

[0051] in , These refer to the travel of the accelerator and brake pedals, respectively. , These represent the corresponding maximum travel distances.

[0052] Furthermore, the two agent policy models created in this invention mimic human driving habits, employing a sequential reasoning structure that prioritizes the shifting agent policy model. To this end, an action interaction communication mechanism is designed between the agent policy models. At each decision-making moment, the shifting agent policy model first outputs the probability distribution of the action space based on the current state st. The pedal agent policy model combines it with Concatenate to form an enhanced observation vector:

[0053] The pedal agent policy model simultaneously perceives the global state and the complete intent distribution of the shift agent policy model during decision-making, achieving unidirectional forward transmission of shift intent to pedal decision. The two agent policy models then synchronously execute their respective actions, and the joint action probability distribution is decomposed according to the chain rule as follows:

[0054] in, For the current gear shift action, For the current pedal action, This represents the output probability of the gear shift action at the current moment. This represents the output probability of the pedal action at the current moment.

[0055] Furthermore, this invention designs a shift gating mechanism to suppress unnecessary frequent shifts that may be induced by small fluctuations in the probability distribution, thus providing redundant protection for the driving robot. The confidence level corresponding to the expected shift action at the next moment is determined, and the actual shift action at the next moment is determined based on this confidence level. The formula for calculating the confidence level is as follows:

[0056] in, Predict actions for the next moment; Confidence level greater than or equal to preset threshold If the expected gear shift is made at the next moment, the actual gear shift will be made at the next moment; otherwise, the gear shift will be made at the current moment. That is:

[0057] In the embodiments of the present invention, .

[0058] Furthermore, action commands are generated based on the actual gear shifting action and the pedal action value at the next moment. In other words, the action commands include gear shifting commands and pedal action commands after gating filtering. These commands are sent to the gear shifting module and pedal module of the driving robot, and the action commands are sent to the pedal robot module. The servo drive mechanisms of each module then perform the corresponding physical operations.

[0059] The policy model in this invention requires offline simulation training using Deep Reinforcement Learning (DRL) before it can be deployed and executed. The training algorithm flow is as follows: Figure 3 As shown: Based on the Proximal Policy Optimization (PPO) algorithm, independent experience replay buffers and policy update modules are configured for the heterogeneous characteristics of the two agent policy models. The shift agent policy model adopts an exploration strategy with entropy coefficients that decay linearly with training progress. In the early stages of training, it maintains high exploration entropy to fully search the discrete action space, and gradually converges to a deterministic policy as training progresses. The pedal agent policy model adopts an exploration strategy with action standard deviations that decay linearly with training progress. In the early stages, it maintains a large exploration range to cover the continuous action space, and gradually narrows it in the later stages to refine the policy output. The above heterogeneous exploration configuration matches the exploration rhythm of the shift agent and the pedal agent with the characteristics of their respective discrete and continuous action spaces. At each decision step, the shift agent policy model first infers based on the current shared state, outputs the action probability distribution of each gear, and generates a shift action accordingly. Then, the shared state and the probability distribution are concatenated into an augmented observation and passed to the pedal agent, which generates a pedal action accordingly. After executing the joint action, the shared reward and the individual reward of each agent policy model are calculated and superimposed, and the complete trajectory is stored in their respective buffers. After the trajectory acquisition phase, the two agents synchronously update their policies. At the end of each round, the sum of the cumulative exclusive rewards of each agent's policy model is used as the evaluation metric, and the policy parameters corresponding to the optimal metric are saved.

[0060] Furthermore, to improve the stability and policy convergence of the dual-agent collaborative optimization, this invention constructs a joint credit allocation mechanism. The total reward of each agent's policy model at time t is composed of a weighted sum of the shared reward and the agent's specific reward. The shared reward reflects the contribution of the joint actions of the dual-agent policy models to the global task, while the specific reward is precisely attributed to the independent decision-making responsibilities of each agent. That is, at time t... The total reward obtained by each strategy model as follows:

[0061] in, Represents the agent policy model. , express Share rewards at all times; Values hour, express The constantly shifting intelligent agent strategy model enjoys exclusive rewards; Values hour, express The instant pedal intelligent agent strategy model enjoys exclusive rewards; The allocation coefficients for shared rewards and exclusive rewards are as follows:

[0062]

[0063]

[0064] in, , , , , , These are the weighting coefficients for each reward or penalty item. for Real-time vehicle speed tracking reward items, for Fuel economy incentives at all times for Constant engine speed constraint penalty item, for Constant gear shifting constraint penalty item, for Always pedal smoothness bonus, for Constraints on the speed of pedal operation at all times.

[0065] In this invention, vehicle speed tracking accuracy is defined as a performance indicator jointly managed by the shift agent and the pedal agent, with a shared reward. The vehicle speed tracking reward uses real-time vehicle speed error as input; a positive reward is given when the error is within the tolerance band, and a penalty is applied when it exceeds the tolerance band, guiding the two agents to collaboratively optimize vehicle speed tracking performance.

[0066] Pedal operation smoothness is defined as a performance indicator exclusively responsible for the pedal agent, with corresponding rewards specific to the pedal agent. Among them, the pedal smoothness reward penalizes excessive pedal action change rate and imposes additional penalties for frequent pedal direction switching; the pedal operation rate constraint penalty penalizes situations where the pedal action rate exceeds the upper limit, ensuring that pedal operation conforms to the constraints of human driving characteristics.

[0067] Fuel economy, engine speed rationality, and shift constraints are defined as performance indicators specifically responsible for the shifting agent, with corresponding rewards also specific to the shifting agent. The fuel economy reward employs a conditional activation design, requiring both positive pedal action and vehicle speed error within the tolerance band to trigger. The reward value is calculated based on instantaneous fuel consumption within a control step, with a higher reward coefficient as the error approaches zero, guiding the shifting agent to optimize fuel economy while maintaining tracking accuracy. Engine speed constraint penalties are imposed on conditions where the engine speed is below the idle limit or above the maximum speed, guiding the shifting agent to maintain the engine within a reasonable speed range. Shift frequency constraint penalties are imposed on shift intervals below the minimum threshold and situations where the starting phase does not involve starting in first gear, ensuring that shifting behavior conforms to engineering constraints.

[0068] In this embodiment, the reward functions for each item are defined as follows: (1) Vehicle speed tracking reward item Defined as:

[0069] This is for vehicle speed error; (2) Fuel economy incentive The conditional activation design requires that the pedal action be positive and the vehicle speed error be within the tolerance zone simultaneously before triggering:

[0070] in To control fuel consumption (kg) within a step, the segmentation coefficient is... Determined by vehicle speed error:

[0071] (3) Engine speed constraint penalty item at all times Defined as:

[0072] in , These are the engine idle speed and maximum speed, respectively. for Constant engine speed.

[0073] (4) Constant shifting constraint penalty item Defined as:

[0074] In the formula, for……, for……, A penalty will be imposed for shift intervals of less than 3 seconds. First gear must be used during the forced start phase.

[0075] (5) Always-on pedal smoothness bonus Defined as:

[0076] in This represents the change in pedal position. For positive operators, The function is an indicator function. The function is a symbolic function. Penalize excessive pedal change rate. The direction of the penalty pedal changes frequently.

[0077] (6) Pedal operation rate constraint penalty item Defined as:

[0078] in This is the upper limit of the manual pedal operation rate.

[0079] The numerical coefficients in the above reward function were determined through iterative debugging during the training process. In this embodiment, we take... .

[0080] This invention designs corresponding training scenarios based on the tracked target operating condition. These scenarios fully cover the vehicle speed range, acceleration / deceleration range, and typical characteristics of each typical operating condition segment of the target operating condition, ensuring that the trained strategy is sufficiently adaptable to the target operating condition. The training scenarios can be representative segments extracted from the target operating condition. The selected segments should cover the speed and acceleration / deceleration characteristics at low, medium, and high speeds, reducing redundant samples while ensuring coverage, thereby improving training convergence efficiency. In this embodiment, the training scenarios use the tail segment from 1470s to 1800s of the complete WLTC operating condition.

[0081] In this embodiment of the invention, the offline training of the policy model was completed on the CarSim and Python co-simulation platform. A reinforcement learning simulation environment was built based on OpenAI Gym, and a dual-agent neural network model was implemented using PyTorch. The trained policy was directly deployed on a commercial driving robot platform, communicating with the driving robot in real time via Ethernet to achieve vehicle speed tracking and gear shift coupling control under standard test conditions such as WLTC. The vehicles used in both the simulation modeling and the real-vehicle tests were passenger cars with a 6-speed manual transmission from a certain brand. Specific training hyperparameters are shown in Table 1.

[0082] Table 1 Core Hyperparameters for Dual-Agent PPO Training

[0083] Note: t is the current time step; T = 1 × 10 6 This represents the total number of time steps.

[0084] During the simulation and real-vehicle verification phase, a comparative test was conducted with the baseline scheme (UN GTR No. 15 standard shifting scheme combined with pedal segmented PID decoupling control) under complete WLTC conditions. Results showed that the proposed scheme achieved 7.84% lower fuel consumption and 12.36% lower average absolute error in vehicle speed tracking compared to the baseline scheme under complete WLTC conditions. Real-vehicle verification was completed at a brand's testing center's vehicle energy consumption laboratory. Test results showed that the proposed scheme achieved 7.51% lower fuel consumption and 10.27% lower average absolute error in vehicle speed tracking compared to the baseline scheme. The shifting timing of the real vehicle closely matched the simulation results, with only minor timing deviations at a few local operating points and no shifting logic disorder, verifying the engineering feasibility of this invention from simulation training to real-vehicle deployment.

[0085] The present invention also provides a gear shifting and speed tracking coupling control device for an automotive driving robot, comprising: Data acquisition module: configured for the trajectory tracking task of the car driving robot, collecting environmental state vectors related to the car driving robot and the vehicle under test at the current moment; the vehicle under test is the vehicle driven by the car driving robot; Action command generation module: configured to input the state vector into the trained policy model; the policy model outputs the action command for the car driving robot at the next moment; State update module: Configured to execute action commands for the driving robot, generate the state vector for the next moment, and update the shared state space; Among them, the strategy models include the shift agent strategy model and the pedal agent strategy model; The shifting agent strategy model generates a discrete action probability distribution based on the shared state information in the shared state space at the current moment and selects the action with the highest probability as the expected shifting action at the next moment. The pedal agent strategy model concatenates the discrete action probability distribution with the shared state information, and outputs the pedal action value at the next moment based on the concatenated state information. The shift filter determines the confidence level corresponding to the expected shift action at the next moment, and combines it with the shift action at the current moment to determine the actual shift action at the next moment; The actual shift command and pedal command for the next moment are determined based on the actual shift action and pedal action value for the next moment, respectively. The driving robot executes the actual gear shifting and pedal operation commands for the next moment.

[0086] The specific embodiments described above only illustrate the design principles of the present invention. The shapes and names of the components in this description may differ and are not limited. Therefore, those skilled in the art can modify or make equivalent substitutions to the technical solutions described in the foregoing embodiments; and these modifications and substitutions do not depart from the inventive spirit and technical solutions of the present invention, and should all fall within the protection scope of the present invention.

Claims

1. A method for coupled control of gear shifting and speed tracking in a car driving robot, characterized in that, include: Step S1: The car driving robot acquires the trajectory tracking task and collects the environmental state vectors related to the car driving robot and the vehicle under test at the current moment; the vehicle under test is the vehicle driven by the car driving robot. Step S2: Input the state vector into the trained policy model; the policy model outputs the action command for the car driving robot at the next moment; Step S3: Drive the robot to execute action commands, generate the state vector for the next moment, and update the shared state space; Among them, the strategy models include the shift agent strategy model and the pedal agent strategy model; The shifting agent strategy model generates a discrete action probability distribution based on the shared state information in the shared state space at the current moment and selects the action with the highest probability as the expected shifting action at the next moment. The pedal agent strategy model concatenates the discrete action probability distribution with the shared state information, and outputs the pedal action value at the next moment based on the concatenated state information. The shift filter determines the confidence level corresponding to the expected shift action at the next moment, and combines it with the shift action at the current moment to determine the actual shift action at the next moment; The actual shift action command and pedal action command for the next moment are determined based on the actual shift action and pedal action value for the next moment, respectively. The driving robot executes the actual gear shifting and pedal operation commands for the next moment.

2. The method as described in claim 1, characterized in that, Shared state space In 9-dimensional space, it is represented as follows: in, For vehicle speed error, This is the actual vehicle speed. For the current acceleration, Engine speed, The gear was in position at the previous moment. The value of the pedal action at the previous moment. This is the time elapsed since the last gear shift. and These are the desired acceleration and the desired jerk, respectively.

3. The method of claim 2, wherein, The expected acceleration and expected jerk are calculated based on the reference vehicle speed trajectory within the prediction time window of the trajectory tracking task. Specifically, the instantaneous acceleration of each sampling step within the prediction time window is weighted by the first exponential decay coefficient and the average is calculated to obtain the expected acceleration; the acceleration difference between adjacent sampling steps is weighted by the second exponential decay coefficient and the average is calculated to obtain the expected jerk. Desired acceleration The formula for calculating the desired jerk is as follows: in, The sampling interval is... To predict the number of sampling steps within the time window, and They are respectively Time and The target speed at any given moment and They are respectively Time and The target acceleration at any given moment; This is the first exponential decay coefficient. The second exponential decay coefficient is defined as follows: wherein is the attenuation coefficient.

4. The method of claim 3, wherein, The action space of the shift agent strategy model is a discrete set of shift actions, while the action space of the pedal agent strategy model is a continuous set of accelerator pedal and brake pedal actions. Action space of shifting agent strategy model These correspond to three types of actions: downshifting, holding, and upshifting; the action space of the pedal agent strategy model is a continuous interval. Positive values ​​correspond to the normalized accelerator pedal opening, and negative values ​​correspond to the normalized brake pedal opening. wherein, , are the accelerator and brake pedal travel, respectively, , are the maximum travel, respectively.

5. The method as described in claim 4, characterized in that, At each decision-making moment, the shifting agent's policy model outputs a discrete action probability distribution of the complete action space based on the current shared state information st. The pedal agent policy model combines it with splicing to form an enhanced observation vector : in, This indicates splicing; the shift agent policy model and the pedal agent policy model execute their respective actions synchronously, and the joint action probability distribution is decomposed according to the chain rule as follows: in, For the current gear shift action, For the current pedal action, This represents the output probability of the gear shift action at the current moment. This represents the output probability of the pedal action at the current moment.

6. The method as described in claim 5, characterized in that, The shift filter determines the confidence level corresponding to the expected shift action at the next moment, and combines it with the shift action at the current moment to determine the actual shift action at the next moment, wherein: The formula for calculating confidence level is as follows: in, Predict actions for the next moment; Confidence level greater than or equal to preset threshold If the expected gear shift is to be performed at the next moment, then the actual gear shift will be performed at the next moment; otherwise, the gear shift will be performed at the current moment.

7. The method as described in claim 6, characterized in that, The reward function used by the policy model during training is as follows: in, Represents the agent policy model. , express Share rewards at all times; Values hour, express The constantly shifting intelligent agent strategy model enjoys exclusive rewards; Values hour, express The instant pedal intelligent agent strategy model enjoys exclusive rewards; The allocation coefficients for shared rewards and exclusive rewards are as follows: in, , , , , , These are the weighting coefficients for each reward or penalty item. for Real-time vehicle speed tracking reward items, for Fuel economy incentives at all times for Constant engine speed constraint penalty item, for Constant gear shifting constraint penalty item, for Always pedal smoothness bonus, for Constraints on the speed of pedal operation at all times.

8. A coupling control device for gear shifting and speed tracking in a car driving robot, characterized in that, include: Data acquisition module: configured for the trajectory tracking task of the car driving robot, collecting environmental state vectors related to the car driving robot and the vehicle under test at the current moment; the vehicle under test is the vehicle driven by the car driving robot; Action command generation module: configured to input the state vector into the trained policy model; the policy model outputs the action command for the car driving robot at the next moment; State update module: Configured to execute action commands for the driving robot, generate the state vector for the next moment, and update the shared state space; Among them, the strategy models include the shift agent strategy model and the pedal agent strategy model; The shifting agent strategy model generates a discrete action probability distribution based on the shared state information in the shared state space at the current moment and selects the action with the highest probability as the expected shifting action at the next moment. The pedal agent strategy model concatenates the discrete action probability distribution with the shared state information, and outputs the pedal action value at the next moment based on the concatenated state information. The shift filter determines the confidence level corresponding to the expected shift action at the next moment, and combines it with the shift action at the current moment to determine the actual shift action at the next moment; The actual shift action command and pedal action command for the next moment are determined based on the actual shift action and pedal action value for the next moment, respectively. The driving robot executes the actual gear shifting and pedal operation commands for the next moment.

9. An electronic device, characterized in that, The electronic device includes: A processor is used to execute multiple instructions; Memory, used to store multiple instructions; The plurality of instructions are to be stored in the memory and loaded by the processor and executed as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method as described in any one of claims 1-7.

Images