An automatic driving racing car control method based on integral compensation

Abstract

The application discloses an automatic driving racing car control method based on integral compensation, which comprises the following steps: establishing a vehicle dynamics model and a kinematics equation; defining a state space and an action space; constructing a reinforcement learning control strategy based on a double-delay deep deterministic policy gradient method, combining an action mapping mechanism to maximize vehicle performance under the premise that the control output satisfies the tire friction force constraint; fitting a track center line trajectory according to a sampling point sequence of a track center line in front of the vehicle; designing an integral compensation mechanism to generate a compensation error state, adjust the vehicle control output and realize control strategy optimization. The application enhances the utilization ability of the system to historical yaw information by introducing the integral compensation technology, enables the control strategy to adaptively adjust based on the accumulated error trend, makes the driving trajectory of the racing car on the track more smooth and coherent, reduces unnecessary oscillation and deviation, improves the overall lap speed and improves the control performance of the automatic driving racing car in the high-speed limit driving environment.

Description

TECHNICAL FIELD

[0001] The application relates to an integral compensation-based automatic racing car control method and belongs to the technical field of automatic driving. BACKGROUND

[0002] Current automatic driving technology has covered most driving scenarios, including highway driving, urban driving and automatic parking, etc. In highway and urban driving scenarios, vehicles mostly run near the equilibrium point, and the dynamics thereof can be approximated as a linear model; in racing driving scenarios, vehicles run near their physical limits, and therefore complex nonlinear dynamics and constraint conditions need to be considered. These factors make automatic racing car control more challenging and valuable for research than other driving scenarios.

[0003] Existing automatic racing car control methods mainly include model-based optimization control methods and reinforcement learning (RL)-based policy learning methods. Model-based optimization control methods, such as model predictive control (MPC), predict future trajectories using a vehicle dynamics model and calculate optimal control outputs through rolling optimization. Although MPC can handle constraints and obtain relatively optimal solutions in theory, it has high computational complexity and is difficult to meet the real-time requirements of high-speed racing cars in extreme states. RL-based policy learning methods learn end-to-end control strategies through the interaction between agents and environments, and can achieve complex control without an accurate model. However, the control actions directly output by RL methods are often not physically constrained, which may cause the vehicle to slip or lose stability.

[0004] To solve the problem of lack of physical constraints in RL methods, some scholars have proposed an action mapping (AM) mechanism. This mechanism maps the unconstrained actions output by the neural network to control outputs that satisfy the tire friction limit by predefining a safe action space. The TD3-AM method combines the efficient learning ability of the twin delayed deterministic policy gradient (TD3) algorithm and the safety constraint characteristics of the AM mechanism, and can effectively utilize tire grip to improve the performance of racing cars. However, from the perspective of state input, the control strategies of existing methods mostly make decisions based on current observation states, lacking effective use of historical information, which affects the stability and smoothness of vehicles in high-speed extreme working conditions. SUMMARY

[0005] The technical problem to be solved by the application is to provide an integral compensation-based automatic racing car control method, solve the problem of insufficient use of historical information under the existing TD3-AM control framework, improve the smoothness of racing car trajectories, and further improve the control performance of automatic racing cars in high-speed extreme driving environments.

[0006] The application adopts the following technical solutions to solve the above technical problems:

[0007] An automatic driving racing car control method based on integral compensation, comprising the following steps:

[0008] Step 1, four wheels of the racing car are simplified into two equivalent wheels at the center of the front and rear axles, and a dynamics model and kinematics equation of the vehicle are established based on a single-track model;

[0009] Step 2, a state space and an action space of a reinforcement learning control strategy are defined, the state space includes basic states of the vehicle, relative positions of the track, and a sequence of sampling points of the track center line in front of the vehicle, and the action space, i.e., control output, includes lateral control output and longitudinal control output;

[0010] Step 3, a reinforcement learning control strategy is constructed based on a double-delay deep deterministic policy gradient method, and a action mapping mechanism is combined to maximize the vehicle performance under the premise of satisfying the tire friction force constraint;

[0011] Step 4, a quintic polynomial method is used to fit the track center line trajectory according to the sequence of sampling points of the track center line in front of the vehicle;

[0012] Step 5, an integral compensation mechanism is designed, a compensation error state is generated, the compensation error state is used to replace the angle between the vehicle heading and the track direction in the state space, the vehicle control output is adjusted, and the racing car control strategy optimization is realized.

[0013] Compared with the prior art, the application has the following technical effects:

[0014] 1. The integral compensation (IC) mechanism is introduced to improve the precision and stability of the control system; the integral compensation provides the ability to use historical yaw information for the control system, so that the control strategy can be adaptively adjusted based on the cumulative error trend, rather than relying on the current state only, which is suitable for processing frequent acceleration, deceleration and steering operations on the track.

[0015] 2. The quintic polynomial is used to fit the track center line, which can more accurately describe the complex track geometry compared with low-order polynomials, especially for high-curvature regions; compared with higher-order polynomials, the problem of overfitting is avoided, the calculation efficiency is maintained, and the real-time control requirements are met.

[0016] 3. The original observation state is directly replaced by the integral compensated state, which avoids the problems of increased state-action space exploration difficulty and function fitting complexity caused by increasing the input state dimension. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flow chart of an automatic driving racing car control method based on integral compensation according to the present application;

[0018] Figure 2 is a state space definition schematic diagram according to an embodiment of the present application;

[0019] Figure 3 is a double-delay deep deterministic policy gradient algorithm framework diagram;

[0020] Figure 4 is a simulation track shape schematic diagram according to an embodiment of the present application;

[0021] Figure 5 is a racing car running speed distribution cloud diagram before introducing the integral compensation technology according to an embodiment of the present application, wherein (a) is starting from the starting point at an initial speed of 10 m / s to start the first lap, and (b) is the second lap;

[0022] Figure 6 is a racing car running speed distribution cloud diagram after introducing the integral compensation technology according to an embodiment of the present application, wherein (a) is starting from the starting point at an initial speed of 10 m / s to start the first lap, and (b) is the second lap;

[0023] Figure 7 is a simulation racing car speed comparison schematic diagram according to an embodiment of the present application;

[0024] Figure 8 is a simulation racing car throttle / brake usage comparison schematic diagram according to an embodiment of the present application;

[0025] Figure 9 is a simulation racing car track segment brake usage comparison schematic diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The embodiments of the present application will be described in detail below, examples of which are shown in the accompanying drawings. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be explained as a limitation of the present application.

[0027] As Figure 1 shown, the present application proposes an automatic driving racing car control method based on integral compensation, the model is unknown to the track condition in advance, the current state is modeled by using the state space and the action space, the control strategy is constructed by combining the deep reinforcement learning algorithm; the track center point is obtained by the sensor during training and the path is fitted, the integral compensation mechanism is introduced to correct the control deviation, and the precise control of the racing car movement is realized. The specific steps are as follows:

[0028] Step 1, Establish the vehicle dynamics model, adopt single-track model to describe the dynamics characteristics of the racing car. The specific process is as follows:

[0029] The single-track model simplifies the four wheels of the racing car into two equivalent wheels, which are located at the center of the front and rear axles. This simplification is based on the following assumptions: 1) the body can be regarded as a rigid body, ignoring the influence of suspension dynamics; 2) the dynamics characteristics of the left and right sides of the front and rear wheels are symmetrical; 3) the track surface is flat and the friction coefficient is uniform; 4) ignore the longitudinal and lateral load transfer effect.

[0030] The model uses two reference coordinate systems at the same time: the ground fixed coordinate system And the body fixed coordinate system . Among them, the origin of the ground fixed coordinate system is set at the starting point of the track, The axis is along the forward direction of the track, The axis points to the left side, The axis is vertical upward, used to describe the global position of the vehicle on the track. In the fixed body coordinate system, the origin is located at the vehicle mass center, The axis points to the front direction, The axis points to the left side, The axis is vertical upward, used to describe the local motion state of the vehicle. The state of the vehicle is described by the following variables:

[0031] Position coordinates : the position of the vehicle mass center in the ground coordinate system;

[0032] Heading angle : the angle between the body longitudinal axis and the ground coordinate system The axis;

[0033] Longitudinal velocity : the velocity component of the vehicle mass center along the body coordinate system The axis;

[0034] Lateral velocity : the velocity component of the vehicle mass center along the body coordinate system The axis;

[0035] Yaw rate : the angular velocity of the vehicle around the axis;

[0036] Front wheel steering angle : the angle between the front wheel and the body longitudinal axis.

[0037] The longitudinal acceleration of the vehicle is determined by the longitudinal force acting on the vehicle, according to Newton's second law:

[0038] ,

[0039] Where, is the mass of the vehicle, Longitudinal force generated for the front wheels, Longitudinal force generated for the rear wheels, Lateral force generated for the front wheels, the last term is the Coriolis force term, reflecting the centrifugal effect due to vehicle turning.

[0040] Total longitudinal force along the longitudinal direction of the vehicle body Further decomposed into multiple components:

[0041] ,

[0042] Driving force Directly related to the motor characteristics:

[0043] ,

[0044] Where, Transmission efficiency, Motor output torque, Gearbox reduction ratio, Effective radius of the wheels.

[0045] Motor torque Affected by the motor control signal and the motor characteristic curve, with different performances in different speed intervals. Braking force Proportional to the brake control signal :

[0046] ,

[0047] Where, Brake system gain.

[0048] Rolling resistance Approximately proportional to the ground pressure:

[0049] ,

[0050] Where, Rolling resistance coefficient, Gravitational acceleration.

[0051] Air resistance Proportional to the square of the vehicle speed:

[0052] ,

[0053] Where, Air density, Aerodynamic drag coefficient, Vehicle frontal area.

[0054] The lateral motion includes vehicle lateral displacement and yaw motion, which is described by the following equation:

[0055] ,

[0056] ,

[0057] where, is the lateral force generated by the rear wheel, is the distance from the front axle to the center of mass, is the distance from the rear axle to the center of mass, is the rotational inertia of the vehicle around the axis.

[0058] The motion of the vehicle in the ground coordinate system can be described by the following kinematic equations:

[0059] ,

[0060] ,

[0061] where, and represent the position coordinates of the vehicle center of mass in the ground fixed coordinate system, represents the side slip angle of the vehicle, that is, the angle between the center of mass velocity direction and the vehicle longitudinal axis.

[0062] Combining the above dynamics and kinematics equations, the motion characteristics of the racing car in the limit state can be completely described, which provides a theoretical basis for the subsequent control strategy design and reinforcement learning algorithm. In this invention, the above single-track dynamics model and its related formulas are used as the basis for state updating, which can accurately reflect the key nonlinear characteristics of the racing car in the limit state, and provide accurate reference for the introduction and application of integral compensation technology.

[0063] Step 2, define state space and action space

[0064] In order to enable the vehicle to perceive the curvature change of the front trajectory in advance, the state space ~ is designed as follows: Figure 2 ~ The basic state of the vehicle includes speed , steering wheel angle and steering angular velocity ; and represent the relative position of the vehicle in the track, represents the lateral distance of the vehicle to the center line of the track , represents the angle between the vehicle and the track direction​​​ ; is a series of point coordinates of the center line of the track in front of the racing car . The sampling distance of the point coordinates is designed as: 5m, 10m, 20m, 30m, 40m, 50m, 60m, 70m, 80m, 100m, 120m, 160m and 200m. Among them, the near distance points (5m-10m) are used to accurately judge the current position; the middle distance points (20m-100m) provide the coming curve information; the distance points (120m-200m) help the system to predict the large curvature change. In order to ensure the numerical stability of the state space and the training efficiency, all state variables are normalized to control within a reasonable numerical range.

[0065] The action space is the output of the designed racing car controller, which is selected by imitating the operation mode of human drivers and designed as a two-dimensional continuous action space, wherein the lateral control output controls the steering action of the vehicle; and the longitudinal control output controls the acceleration (positive value) and braking (negative value) of the vehicle. In order to ensure the training efficiency, the upper and lower limits of the action output are uniformly normalized to . This design helps the neural network to learn and output standardized control signals, improves the training efficiency and stability.

[0066] Step 3, constructing a reinforcement learning control policy based on a double-delay deep deterministic policy gradient method, as shown in Figure 3 , combined with an action mapping mechanism to ensure that the control output meets the tire friction force constraint. The specific process is as follows:

[0067] The TD3 algorithm is adopted to construct the autonomous racing car control policy, which effectively solves the value function overestimation problem through the double Q network structure and the delay policy update mechanism.

[0068] The policy network adopts a three-layer fully connected neural network structure, the number of hidden layer neurons is 256-256-2, the input is a 31-dimensional state vector , and the output is a normalized two-dimensional action vector ; the value network includes two neural networks with the same structure but independent parameters, each network adopts a three-layer fully connected structure, the number of hidden layer neurons is 256-256-1, the input is a splicing vector of state and action , and the output is the Q value estimation of the corresponding state-action pair; the target network includes a target policy network and two target Q networks, which have the same structure as the corresponding current network and the parameters are slowly updated from the current network through a soft update mechanism.

[0069] All hidden layers use ReLU activation function, which has the advantages of high computational efficiency, alleviating gradient vanishing and promoting sparse expression; the output layer of the policy network uses Tanh activation function, which strictly limits the output in the range of [-1, 1], meeting the requirements of normalized action space.

[0070] The replay buffer with a capacity of 1,000,000 stores interaction data , , for the next time state, and 256 samples are randomly sampled to form a training batch, breaking the temporal correlation between samples; when calculating the target Q value, clipping noise is added to the target action; every 2 Q network updates slowly track the current network parameters through soft update, further stabilizing the training process.

[0071] The present application uses as the reward function, wherein, is the vehicle speed, is the angle between the vehicle direction and the track direction, encourages the vehicle to travel along the track direction. This reward function balances the dual goals of maximizing speed and trajectory tracking, guiding the vehicle to maintain high speed while adhering to the target trajectory.

[0072] The action value output by the controller is converted into an actual control signal through an action mapping mechanism, ensuring that the output control signal can fully utilize the performance of the racing car and will not cause the vehicle to lose control.

[0073] The maximum force that the tire can provide is limited by the tire-road friction. According to the friction circle model, the resultant force generated by the tire must satisfy:

[0074] ,

[0075] where, is the longitudinal force vector of a single tire, is the lateral force vector of a single tire, is the friction coefficient between the tire and the road, is the vertical load of a single tire.

[0076] This constraint is particularly important in extreme driving conditions, as it limits the maximum acceleration, maximum braking force, and maximum lateral acceleration that the vehicle can achieve. In racing car control, effectively utilizing the maximum friction without exceeding the friction limit is the key to improving the performance of the racing car.

[0077] Through the above control framework, the present application realizes efficient control of the autonomous racing car, which can maximize the performance of the vehicle while meeting the tire friction constraint.

[0078] Step 4, fitting the track centerline trajectory, a quintic polynomial method is adopted to fit the track geometry according to the observed points, as shown in Figure 4 . The specific process is as follows:

[0079] Obtain the coordinates of multiple key points on the track centerline through the environmental perception system, which represent the geometry of the track in front of the car. The sampling distance of the key points is designed in step 2, which can ensure that the track information at near, medium and far distances is fully captured. The obtained key point coordinates are represented as , where represents the index of the point, from 1 to (in this embodiment ). These coordinate points are relative coordinates relative to the current position and orientation of the vehicle. Calculate the Euclidean distance between each adjacent point:

[0080] ,

[0081] Calculate the cumulative length of the path:

[0082] ,

[0083] where , represents the starting point position, is the summation index term.

[0084] Fit the horizontal coordinates and vertical coordinates of the track key points respectively using a quintic polynomial method:

[0085] ,

[0086] ,

[0087] Solve the polynomial coefficients and by least squares method, so that the sum of squared errors between the fitted curve and the sampled track key points is minimized:

[0088] ,

[0089] Using the solved polynomial coefficients, the coordinate point at any path length and the tangent direction of the point can be calculated:

[0090]

[0091] where and are the first-order derivatives of and with respect to :

[0092] ,

[0093] .

[0094] To ensure the real-time and accuracy of the fitted trajectory, the system re-performs polynomial fitting every 100 time steps of driving, ensuring that the fitted curve always reflects the latest track shape in front of the vehicle. The quintic polynomial curve obtained by fitting has more degrees of freedom than the cubic polynomial, which can more accurately describe complex curves and high-curvature areas; it ensures the smoothness and continuity of the curve, avoiding the discontinuity problem that may occur at the connection of the segmented linear method; compared with higher-order polynomials, the quintic polynomial avoids overfitting problems, ensuring accuracy while maintaining computational efficiency.

[0095] Step 5, design integral compensation mechanism, calculate the cumulative effect of yaw angle error, and generate compensation signal; use the compensated state instead of the original observation state to adjust the vehicle control input to optimize the racing control strategy. The specific process is as follows:

[0096] Obtain the reference path direction through the track center line fitted in step 4 This value represents the tangent direction of the target path at the current position.

[0097] Obtain the current real-time yaw angle of the vehicle Calculate the yaw angle error:

[0098] ,

[0099] This error represents the angle difference between the current heading of the vehicle and the expected path direction.

[0100] Normalize the calculated yaw angle error to ensure it always remains within the range of :

[0101] ,

[0102] Integrate the normalized yaw angle error to calculate the cumulative effect:

[0103] ,

[0104] Where, represents the integral value at time .

[0105] Introduce a decay factor to control the growth rate of the integral error, and perform exponential decay on the integral value:

[0106] ,

[0107] wherein, is in the range of , in this embodiment, it is 0.99, used to maintain sufficient integral action while avoiding infinite growth of error.

[0108] A weighting coefficient is applied to the integral error to generate the final compensation value:

[0109] ,

[0110] wherein, is the integral compensation gain coefficient, by adjusting the size of , the balance between compensation strength and system response speed is achieved, preventing oscillation caused by excessive compensation. When is small, the system response is more stable, but the speed of following the trajectory is slower; when is large, the system can react more quickly, but may cause overshoot and oscillation. Through experimental optimization, provides a good balance point.

[0111] The yaw angle state in the original observation state is replaced with the compensated state, and the state space is updated as:

[0112] ,

[0113] The advantage of this state replacement method is that it does not need to increase the input state dimension, avoiding the increase of state space exploration difficulty and computational complexity; without the need to redesign or train the neural network, the stability of the original control framework is maintained; directly guiding the control strategy to adjust in the direction of a better trajectory:

[0114] ,

[0115] Through the integral compensation mechanism, the system can learn a more stable control strategy, improving the performance of the racing car on continuous curves and high-speed straight lines. Compared with pure random exploration, this guided approach based on historical information improves the learning efficiency, enabling the racing car to better master the driving strategy.

[0116] In order to evaluate the effectiveness and innovation of the designed control strategy, simulation verification is carried out based on the embodiment. The parameters are shown in Table 1:

[0117] Table 1 Main parameters of the simulation racing car

[0118]

[0119] Figures 5 to 9 is the simulation result of the system using the controller based on the integral compensation technology.Figure 5 (a) and (b) Figure 6 (a) and (b) show the trajectory performance and speed distribution of the race car on the track before and after the introduction of integral compensation. After using integral compensation, the trajectory is closer to the route planning of a professional race car driver, and the average lap time is also faster. The simulation results are shown in Table 2. It can be seen that, whether in the first lap time or the second lap, the TD3-AM-IC method combined with integral compensation proposed in this invention performs better than before the improvement.

[0120] Table 2 Comparison of simulation results before and after improvement

[0121]

[0122] Figure 7 The results show that the control strategy designed in this invention improves the average speed and enhances system performance compared to the previous method. Figure 8 The results show that the output signal fluctuations of the throttle and brake in the improved model are significantly reduced, especially in the braking section. This indicates that the vehicle's control system has been optimized in terms of the balance between acceleration and braking, avoiding unnecessary violent operations, reducing energy waste and improving driving efficiency. Figure 8 This indicates that the algorithm designed in this invention uses slightly more braking force in curves than before the improvement, while using significantly less braking force in straight sections. This shows that the improved algorithm can better utilize the curvature of the curve and the friction of the tires to decelerate, and adjust the trajectory in the turning phase, thereby giving full play to the acceleration performance of the race car on the straight track. Figure 9 The braking usage across different track sections is more rationally distributed after the introduction of points compensation: braking is more concentrated and smoother in corners, avoiding frequent emergency braking; there is virtually no unnecessary braking on straight sections, fully utilizing acceleration performance. The overall braking strategy is closer to that of professional drivers, further improving lap times and energy efficiency.

[0123] Based on the same inventive concept, this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned autonomous driving racing car control method based on integral compensation.

[0124] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aforementioned autonomous driving racing car control method based on integral compensation.

[0125] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0126] The present application is described in reference to the flow diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It will be understood that each block of the flow diagrams, and combinations of blocks in the flow diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams block or blocks. Figure One The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams block or blocks.

[0127] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams block or blocks. Figure One The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams block or blocks.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams block or blocks. Figure One The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams block or blocks.

[0129] The above embodiments are only illustrative of the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical solution falls within the protection scope of the present application.

Claims

1. A control method for an autonomous racing car based on integral compensation, characterized in that, The method comprises the following steps: Step 1, four wheels of the racing car are simplified into two equivalent wheels at the center of the front and rear axles of the racing car, and a dynamic model and kinematic equation of the racing car are established based on a single-track model; Step 2, a state space and an action space of a reinforcement learning control strategy are defined, the state space comprises basic states of the racing car, a relative position of the racing car on a track and a sequence of sampling points of a center line of the track in front of the racing car, and the action space, i.e. a control output, comprises a lateral control output and a longitudinal control output; Step 3, the reinforcement learning control strategy is constructed based on a double-delay deep deterministic policy gradient method, and an action mapping mechanism is combined to maximize the performance of the racing car under the premise of satisfying a tire friction force constraint; In the method, a network structure of the reinforcement learning control strategy comprises a policy network, a first value network and a second value network, a structure of a target policy network corresponding to the policy network is the same as that of the policy network, a structure of a target first value network corresponding to the first value network is the same as that of the first value network, and a structure of a target second value network corresponding to the second value network is the same as that of the second value network; during network training, an experience replay buffer is used to store interaction data; an input of the policy network is the state space at a current time, and an output of the policy network is the action space at the current time; an environment generates a current reward and a state space at a next time according to the state space at the current time and the action space at the current time; inputs of the first value network and the second value network are a concatenation of the state space at the current time and the action space at the current time, and outputs of the first value network and the second value network are Q values at the current time; an input of the target policy network is the state space at the next time, and an output of the target policy network is a target action space at the next time; inputs of the target first value network and the target second value network are a concatenation of the state space at the next time and the target action space at the next time, and outputs of the target first value network and the target second value network are a first target Q value and a second target Q value corresponding to a state-action pair, respectively; a time difference target is calculated based on the current reward and the first target Q value and the second target Q value, and the first value network and the second value network are updated; the policy network is updated by a gradient ascent method based on the first value network; the target policy network, the target first value network and the target second value network are updated through a delayed policy updating mechanism; that is, the policy network is updated once every two times of updating of the first value network and the second value network, and parameters of the target policy network, the target first value network and the target second value network are executed for soft updating; the action space output by the policy network is converted into control outputs satisfying the tire friction force constraint through the action mapping mechanism, a lateral action is mapped into a steering angle rate of change, and a longitudinal action is mapped into an acceleration rate of change, and the steering angle rate of change and the acceleration rate of change are jointly constrained by a tire friction ellipse to ensure that the friction limit is not exceeded in an extreme driving state; a tire friction force constraint equation is as follows: ， wherein, is the longitudinal force vector of the individual tire, is the lateral force vector of the individual tire, is the modulus of the resultant force vector on the individual tire, is the friction coefficient between the tire and the road surface, is the vertical load of the individual tire; Step 4, a track trajectory of the center line of the track in front of the racing car is fitted by using a quintic polynomial method according to the sequence of sampling points of the center line of the track in front of the racing car. Step 5, design integral compensation mechanism, and generate compensation error state, use compensation error state to replace the angle between vehicle orientation and track direction in state space, adjust vehicle control output, realize racing car control strategy optimization; The specific process of designing integral compensation mechanism and generating compensation error state is as follows: Obtaining a reference path direction by fitting a track centerline i.e. the tangent direction of the fitted track centerline at the current position of the vehicle; Acquiring vehicle real-time yaw angle , calculating yaw angle error : ， The yaw angle error is normalized, and the normalized yaw angle error Integration is performed to calculate the cumulative effect: ， wherein, represents the integral error value at the time point represents the yaw angle error at the time point represents represents the yaw angle error at the time point Introducing a decay factor Controlling the rate of growth of the integral error, Taking values between 0 and 1: ， wherein is the integral error value after the introduction of the attenuation factor; applying an integral compensation gain factor to the integral error value , generating a final compensation value : ， Compensating for vehicle yaw angle with a compensation value added, constituting a compensation error state ; Utilizing compensation error states Replacing the angle between the vehicle heading and the track direction in the state space Implementing race car control strategy optimization.

2. The integral compensation-based control method for an autonomous racing car according to claim 1, wherein, In step 1, the dynamic model of the vehicle includes the longitudinal acceleration equation, the lateral acceleration equation and the yaw acceleration equation of the vehicle, wherein the longitudinal acceleration equation is as follows: ， wherein is the vehicle mass; is the longitudinal velocity of the vehicle, i.e. the velocity component of the vehicle's center of mass along the body-fixed coordinate system is the velocity component of the vehicle's center of mass along the body-fixed coordinate system is the derivative of is the derivative of is the body-fixed coordinate system with the origin at the vehicle's center of mass, is the body-fixed coordinate system with the origin at the vehicle's center of mass, is the body-fixed coordinate system with the origin at the vehicle's center of mass, is the longitudinal force generated by the front wheels, is the lateral force generated by the front wheels, is the longitudinal force generated by the rear wheels; is the front wheel steering angle, i.e. the angle between the front wheels and the body longitudinal axis; is the lateral velocity of the vehicle, i.e. the velocity component of the vehicle's center of mass along the body-fixed coordinate system is the lateral velocity of the vehicle, i.e. the velocity component of the vehicle's center of mass along the body-fixed coordinate system is the yaw rate; The lateral acceleration equation is as follows: ， wherein is derivative of is the lateral force generated by the rear wheels; The yaw acceleration equation is as follows: ， wherein is the derivative of , is the moment of inertia of the vehicle about the axis, is the distance from the front axle to the center of mass, is the distance from the rear axle to the center of mass; The kinematic equation of the vehicle is as follows: ， ， wherein, and denote the lateral and longitudinal coordinate values of the position coordinate of the vehicle center of mass in the ground coordinate system, respectively; are the derivatives of , respectively; the ground coordinate system has the start of the track as its origin, the x-axis is in the direction of progress of the track, the y-axis points to the left of the direction of progress of the track, the z-axis is perpendicular to the ground upwards; is the vehicle speed; is the heading angle, i.e. the angle between the longitudinal axis of the vehicle body and the x-axis of the ground coordinate system; is the yaw angle, i.e. the angle between the longitudinal axis of the vehicle body and the y-axis of the ground coordinate system; is the side slip angle of the vehicle, i.e. the angle between the direction of the center of mass velocity and the longitudinal axis of the vehicle body.

3. The integral compensation based control method for an autonomous racecar as recited in claim 1, wherein, In the step 2, the defined state space specifically includes: vehicle basic state, track relative position, and a sequence of coordinates of sampling points of the track center line in front of the vehicle, wherein the vehicle basic state includes vehicle speed , front wheel steering angle , and steering angular velocity ; the track relative position includes lateral distance of the vehicle to the track center line and included angle of the vehicle to the track direction ; and the sequence of sampling points of the track center line in front of the vehicle is , , wherein each sampling point is 5 m, 10 m, 20 m, 30 m, 40 m, 50 m, 60 m, 70 m, 80 m, 100 m, 120 m, 160 m, and 200 m away from the center point of the vehicle, respectively. The defined action space is a two-dimensional continuous action space, where a lateral control output is used to control the steering action of the vehicle; and a longitudinal control output is used to control the acceleration and braking of the vehicle.

4. The integral compensation-based control method for an autonomous racing car according to claim 3, wherein, The specific process of step 4 is as follows: Real-time acquisition of the coordinates of the sampling points of the center line of the track in front of the vehicle is performed, and the coordinates are represented by , , wherein i represents the index of the sampling point. Calculate the Euclidean distance between adjacent sampling points: ， wherein denotes the Euclidean distance between the sampling points and ; Calculating the cumulative length of the track from the start to the first sampling point: ， wherein, is the sum index item, is the Euclidean distance, is the cumulative length from the start of the track to the sample point. The abscissa and ordinate of the i-th sampling point are fitted by a quintic polynomial method :​​ ， ， Solving the polynomial coefficients by least squares and minimizing the sum of squared errors between the fitted track centerline and the sampled points of the track centerline: ， wherein, is the fitting coordinate of the i-th sampling point calculated from the fitting polynomial; and is the fitting coordinate of the i-th sampling point calculated from the fitting polynomial. According to the fitted polynomial, the coordinates of any sampling point on the center line in front of the track are calculated and the tangent direction of the center line ， wherein respectively the abscissa , ordinate derivative; The racing car performs polynomial fitting once every 100 time steps, ensuring that the fitted track center line always reflects the track shape in front of the vehicle.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The processor executes the computer program to realize the steps of the integral compensation based automatic driving racing car control method according to any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 5. The computer program is executed by the processor to realize the steps of the integral compensation based automatic driving racing car control method according to any one of claims 1 to 4.

Images