Vehicle control method, vehicle and storage medium

Abstract

The embodiment of the invention provides a vehicle control method, a vehicle and a storage medium. The method comprises the steps of obtaining current state data of a target vehicle; performing prediction analysis based on the current state data to obtain a first prediction result, a second prediction result and a third prediction result, the first prediction result being used for representing the steering wheel steering angle predicted by the target fitting neural network model, the second prediction result being used for representing the steering wheel steering angle predicted by the reinforcement learning model, and the third prediction result being used for representing the steering wheel steering angle predicted by the reinforcement learning model; the third prediction result is used for representing a steering wheel steering angle predicted by the vehicle motion physical model; determining steering control data corresponding to the target vehicle based on the first prediction result, the second prediction result and the third prediction result; and performing lateral control on the target vehicle by using the steering control data. The technical problems of low control efficiency and poor reliability during vehicle steering control in the prior art are solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of automatic driving, in particular, relate to a vehicle control method, a vehicle and a storage medium. BACKGROUND

[0002] With the rapid development of automatic driving technology, the model predictive control (MPC) algorithm has become a powerful tool for realizing precise path tracking of vehicles due to its forward-looking and real-time optimization capabilities. MPC relies on an accurate vehicle kinematic model to predict the future state of the vehicle and solve the optimal control sequence accordingly. However, the traditional vehicle kinematic model is limited by simplifying assumptions and performs poorly in complex and variable environments, making it difficult to accurately reflect the dynamic characteristics of the vehicle. In related technologies, a neural network can be used to optimize the vehicle dynamics model, which can improve the adaptability of the model, but ignores the dynamic changes of vehicle parameters and the influence of road environment factors. In related technologies, a deep reinforcement learning can also be used to improve the prediction model in MPC, but it is essentially more focused on strategy learning rather than fine description of physical models. When facing highly nonlinear vehicle dynamics and extreme road conditions, it cannot provide sufficient robust and accurate control instructions. Therefore, while pursuing model adaptability and intelligent optimization, related technologies fail to effectively integrate real-time feedback of vehicle physical characteristics and environmental variables, resulting in significant shortfalls in control performance in specific scenarios, low control efficiency and poor reliability of vehicles.

[0003] At present, there is no good solution to the above problems. SUMMARY

[0004] Embodiments of the present application provide a vehicle control method, a vehicle and a storage medium to at least solve the technical problems of low control efficiency and poor reliability in vehicle steering control in related technologies.

[0005] According to an aspect of an embodiment of the present application, a vehicle control method is provided, comprising: obtaining current state data of a target vehicle, wherein the current state data is used to represent a vehicle state of the target vehicle in a current sampling period; performing prediction analysis based on the current state data to obtain a first prediction result, a second prediction result and a third prediction result, wherein the first prediction result is used to represent a steering wheel steering angle predicted via a target fitting neural network model, the second prediction result is used to represent a steering wheel steering angle predicted via a reinforcement learning model, and the third prediction result is used to represent a steering wheel steering angle predicted via a vehicle motion physical model; determining steering control data corresponding to the target vehicle based on the first prediction result, the second prediction result and the third prediction result, wherein the steering control data is used to represent a steering wheel steering angle of the target vehicle in a future sampling period; and performing lateral control on the target vehicle by using the steering control data.

[0006] Optionally, the performing prediction analysis based on the current state data to obtain the first prediction result comprises: performing state prediction on the current state data by using the target fitting neural network model to obtain future state data of the target vehicle, wherein the future state data is used to represent a vehicle state of the target vehicle in a plurality of prediction sampling periods; determining a prediction angle control sequence based on the future state data, wherein the prediction angle control sequence is used to represent a steering angle vector corresponding to the plurality of prediction sampling periods; performing verification processing on the prediction angle control sequence by using a preset constraint condition to obtain a verification processing result; and determining the first prediction result based on the verification processing result.

[0007] Optionally, the preset constraint condition comprises a steering angle constraint condition and an angle change rate constraint condition, the steering angle constraint condition is used to determine whether the prediction angle control sequence is in a preset angle range, and the angle change rate constraint condition is used to determine whether an angle change rate corresponding to the prediction angle control sequence is in a preset change rate range.

[0008] Optionally, the determining the first prediction result based on the verification processing result comprises: in response to determining that the prediction angle control sequence satisfies the preset constraint condition based on the verification processing result, determining an optimization objective function based on the future state data and the prediction angle control sequence; and in response to a function value corresponding to the optimization objective function satisfying a preset numerical condition, determining the first prediction result based on the prediction angle control sequence.

[0009] Optionally, the reinforcement learning model comprises a policy prediction network model and a value estimation network model, and the vehicle control method further comprises: performing analysis and processing on the sample state data by using the policy prediction network model to obtain an action prediction result; performing evaluation processing on the action prediction result by using the value estimation network model to obtain a value evaluation result, wherein the value evaluation result is used to determine a mapping relationship between the sample state data, the action prediction result and the target action reward; and performing update processing on the policy prediction network model and the value estimation network model based on the value evaluation result until the reinforcement learning model meets a preset convergence condition.

[0010] Optionally, determining the target vehicle corresponding steering control data based on the first prediction result, the second prediction result and the third prediction result comprises: obtaining fusion weight data, wherein the fusion weight data is determined according to the driving scene of the target vehicle; and performing fusion processing on the first prediction result, the second prediction result and the third prediction result by using the fusion weight data to obtain the steering control data.

[0011] Optionally, performing lateral control on the target vehicle by using the steering control data comprises: generating a steering control instruction based on the steering control data, wherein the steering control instruction is used to adjust the current steering angle of the target vehicle; and performing lateral control on the target vehicle by using the steering control instruction.

[0012] Optionally, the current state data comprises: lateral deviation, rate of change of lateral deviation, heading angle deviation, rate of change of heading angle deviation, longitudinal speed, lateral speed, longitudinal acceleration, lateral acceleration, heading angle, rate of change of heading angle, left front wheel slip rate, right front wheel slip rate, left rear wheel slip rate, right rear wheel slip rate, ground adhesion coefficient and current steering angle.

[0013] According to another aspect of the embodiments of the present application, a vehicle is further provided, comprising: a memory storing an executable program; and a processor configured to execute the program, wherein the program is configured to execute the method in the embodiments of the present application when executed.

[0014] According to another aspect of the embodiments of the present application, a computer readable storage medium is further provided, comprising a stored executable program, wherein the computer readable storage medium is configured to control a device where the computer readable storage medium is located to execute the method in the embodiments of the present application when the executable program is executed.

[0015] According to another aspect of the embodiments of the present application, a computer program product is further provided, comprising a computer program, wherein the computer program is configured to implement the method in the embodiments of the present application when executed by a processor.

[0016] According to a further aspect of the embodiments of the present application, a computer program product is also provided, including a non-volatile computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the method in various embodiments of the present application.

[0017] According to a further aspect of the embodiments of the present application, a computer program is also provided, the computer program being executed by a processor to implement the method in various embodiments of the present application.

[0018] In the embodiments of the present application, by acquiring the current state data of the target vehicle, and then performing prediction analysis based on the current state data to obtain the first prediction result, the second prediction result and the third prediction result, the first prediction result is used to represent the steering wheel steering angle predicted via the target fitting neural network model, the second prediction result is used to represent the steering wheel steering angle predicted via the reinforcement learning model, and the third prediction result is used to represent the steering wheel steering angle predicted via the vehicle motion physical model, then the steering control data corresponding to the target vehicle is determined based on the first prediction result, the second prediction result and the third prediction result, and finally the target vehicle is controlled in the lateral direction by using the steering control data, thereby predicting the steering wheel steering angle of the target vehicle in the next sampling period by fusing three different prediction models, i.e., the target fitting neural network model, the reinforcement learning model and the vehicle motion physical model, which significantly improves the adaptability, robustness and control precision of the automatic driving system, significantly improves the vehicle control efficiency and reliability during steering control, and further solves the technical problems of low control efficiency and poor reliability during vehicle steering control in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate the illustrative embodiments of the present application and its description, which serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0020] Figure 1 is a flowchart of a vehicle control method according to an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of a vehicle control method according to an embodiment of the present application;

[0022] Figure 3 is a structural block diagram of a vehicle control device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should be within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", and the like in the description and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0025] According to the embodiments of the present application, an embodiment of a vehicle control method is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0026] In the present embodiment, a vehicle control method is provided, Figure 1 is a flowchart of a vehicle control method according to an embodiment of the present application, as Figure 1 shown, the flow includes the following steps:

[0027] Step S11, obtaining current state data of the target vehicle, wherein the current state data is used to represent the vehicle state of the target vehicle in the current sampling period;

[0028] Step S12, performing prediction analysis based on the current state data to obtain a first prediction result, a second prediction result and a third prediction result, wherein the first prediction result is used to represent the steering angle of the steering wheel predicted by the target fitting neural network model, the second prediction result is used to represent the steering angle of the steering wheel predicted by the reinforcement learning model, and the third prediction result is used to represent the steering angle of the steering wheel predicted by the vehicle motion physical model;

[0029] Step S13, determining the steering control data corresponding to the target vehicle based on the first prediction result, the second prediction result and the third prediction result, wherein the steering control data is used to represent the steering angle of the steering wheel of the target vehicle in the future sampling period;

[0030] Step S14, performing lateral control on the target vehicle by using the steering control data.

[0031] The current state data of the target vehicle includes a series of key parameters, including but not limited to lateral deviation, speed, acceleration and steering angle, and is collected by the vehicle-mounted sensor in real time as the basis for subsequent prediction analysis, aiming to accurately capture the real-time state of the vehicle in the current sampling period and provide accurate data support for the next control decision.

[0032] Further, based on the current state data of the target vehicle, a target fitting neural network model, a reinforcement learning model and a vehicle motion physical model are called respectively for prediction analysis. The target fitting neural network model can learn and predict the complex and variable vehicle dynamics characteristics through deep learning technology, and output the first prediction result, i.e. the steering angle of the steering wheel in the future sampling period. The reinforcement learning model generates the second prediction result, i.e. the steering angle of the steering wheel predicted by considering road information, vehicle state and system stability, based on the interaction with the environment. The vehicle motion physical model calculates the third prediction result, i.e. the steering angle prediction based on traditional physical modeling, by using known vehicle parameters and physical laws, to provide a theoretical basis for control decision.

[0033] The steering angles of the steering wheel predicted by the above three models are comprehensively analyzed, and the steering control data of the target vehicle in the future sampling period is determined by a set of weighting algorithms, so as to fully integrate the prediction ability of neural network, the strategy optimization of reinforcement learning and the theoretical support of physical model, ensuring the comprehensiveness and robustness of the steering control data and improving the adaptability of the autonomous vehicle in various environments.

[0034] The lateral control of the target vehicle is implemented by using the calculated steering control data. The lateral control can directly act on the steering system of the vehicle, and by accurately adjusting the steering angle of the steering wheel, the vehicle can effectively track the predetermined path, reduce the lateral deviation and improve the stability and safety of driving. The embodiments of the present application realize fine prediction and control of the future state of the target vehicle by integrating multiple prediction models in the automatic driving control process, which significantly improves the accuracy and response speed of the lateral control.

[0035] Based on the steps S11 to S14, by obtaining the current state data of the target vehicle, and then based on the current state data, the first prediction result, the second prediction result and the third prediction result are obtained through prediction analysis, the first prediction result is used to represent the steering wheel steering angle predicted by the target fitting neural network model, the second prediction result is used to represent the steering wheel steering angle predicted by the reinforcement learning model, and the third prediction result is used to represent the steering wheel steering angle predicted by the vehicle motion physical model. Then, the steering control data corresponding to the target vehicle is determined based on the first prediction result, the second prediction result and the third prediction result, and finally the target vehicle is controlled in the lateral direction by using the steering control data. Thus, by fusing three different prediction models, i.e., the target fitting neural network model, the reinforcement learning model and the vehicle motion physical model, the steering wheel steering angle of the target vehicle in the next sampling period is comprehensively predicted, which significantly improves the adaptability, robustness and control accuracy of the automatic driving system, significantly improves the vehicle control efficiency and reliability during steering control, and further solves the technical problems of low control efficiency and poor reliability in vehicle steering control in the related art.

[0036] The vehicle control method in the embodiments of the present application will be further introduced below.

[0037] In an optional embodiment, in step S12, the prediction analysis based on the current state data to obtain the first prediction result includes:

[0038] In step S121, the target fitting neural network model is used to predict the state of the current state data to obtain the future state data of the target vehicle, wherein the future state data is used to represent the vehicle state of the target vehicle in a plurality of prediction sampling periods;

[0039] In step S122, the prediction angle control sequence is determined based on the future state data, wherein the prediction angle control sequence is used to represent the steering angle vector corresponding to the plurality of prediction sampling periods;

[0040] In step S123, the prediction angle control sequence is verified by using the preset constraint condition to obtain a verification result;

[0041] In step S124, the first prediction result is determined based on the verification result.

[0042] Specifically, by calling the target fitting neural network model, the current state data collected is subjected to deep learning state prediction, the target fitting neural network model can fit complex nonlinear vehicle dynamics, based on the vehicle state data of the current sampling period, the vehicle state data of the target vehicle in the subsequent multiple sampling periods is predicted, such as the position, speed and direction of the vehicle and other key state information. Through state prediction, prospective data support can be provided for control decision, compared with the traditional control method based on real-time data, the response ability of the autonomous vehicle to road changes and the path tracking accuracy are significantly improved.

[0043] In the process of training the target fitting neural network model, an initial fitting neural network model can be defined, the initial fitting neural network model can be a multilayer perceptron (MLP) network, the input is a 12-dimensional vector, respectively representing lateral deviation, lateral deviation rate, heading angle deviation, heading angle deviation rate, longitudinal speed, lateral speed, longitudinal acceleration, lateral acceleration, heading angle, heading angle rate, left front wheel slip rate, right front wheel slip rate, left rear wheel slip rate, right rear wheel slip rate, ground adhesion coefficient and current steering angle. The above input is calculated and output by the vehicle-mounted sensor device, such as positioning sensor, inertial measurement unit, visual camera, wheel speed sensor, steering wheel angle sensor, etc. The output of the initial fitting neural network is i.e. the prediction results of lateral deviation, lateral deviation rate, heading angle deviation, heading angle deviation rate in the next sampling period. By collecting the actual driving data of the vehicle, a data set sample is obtained, and the target fitting neural network is trained and recorded as N. It should be noted that the input and output of the target fitting neural network are not limited in the embodiments of the present application, the input vector of the target fitting neural network can be modified according to the sensor configuration of the vehicle, and the output of each dimension can be adjusted according to the solving target of the MPC algorithm.

[0044] The state variable The system state equation is x k+1 = predict(N, [x k , L k ]), the output of the system is defined as where L k is a vector composed of the last 12-dimensional vector of the input I of the target fitting neural network.

[0045]

[0046] Wherein, u is the control variable, in the lateral MPC control algorithm, it is the steering angle of the steering wheel.

[0047] The target fitting neural network is used to predict the state variables of the system in the next sampling period. Similarly, the state variables of the system in the future n time steps are predicted in the prediction time domain n of the MPC, and the following is obtained:

[0048]

[0049] where N i represents that the target fitting neural network is used for i times of loop reasoning.

[0050] After obtaining the first prediction result, a model predictive control algorithm can be used based on the first prediction result to calculate a predicted angle control sequence:

[0051]

[0052] The predicted angle control sequence is a multi-dimensional vector that includes the steering angles that the target vehicle should execute in the future multiple predicted sampling periods. Combining the prediction advantages of the target fitting neural network model and the optimization solution of the MPC algorithm, both the uncertainty of the dynamic environment and the optimization goal of the control instruction are considered, thereby improving the lateral control effect of the autonomous vehicle, especially under complex road conditions, the steering can be adjusted more accurately and stably.

[0053] The predicted angle control sequence is verified by using a preset constraint condition, which can include but is not limited to vehicle physical limitations (such as limits of steering angles), road safety specifications (such as lane keeping, avoiding obstacles, etc.), and passenger comfort considerations (such as avoiding sharp turns). The purpose of the verification process is to ensure that the predicted angle control sequence meets the physical and safety limitations of the vehicle and provides a smooth driving experience. The introduction of the verification process reflects the strict compliance of the embodiments of the present application to system constraints when making control decisions, enhancing the safety and feasibility of autonomous driving and avoiding invalid or potentially dangerous control instructions.

[0054] According to the verification result, the first prediction result is determined, that is, under the premise of meeting all the constraint conditions, the first steering angle in the predicted angle control sequence becomes the target steering wheel steering angle output in the current sampling period. The determination of the first prediction result ensures the real feasibility of the control instruction, directly guides the lateral control action of the autonomous vehicle, and further improves the adaptability and safety of the autonomous vehicle in the dynamic environment.

[0055] Based on the above optional embodiment, by utilizing the target fitting neural network model to perform state prediction on the current state data, the future state data of the target vehicle is obtained, and then the prediction angle control sequence is determined based on the future state data, and then the preset constraint condition is used to verify the prediction angle control sequence, and the verification processing result is obtained, and finally the first prediction result is determined based on the verification processing result, which realizes efficient prediction and constraint verification of the future state of the target vehicle, ensures the rationality and effectiveness of the prediction angle control sequence, and further improves the accuracy, stability and safety of the lateral control of the autonomous vehicle, and provides key technical support for accurate path tracking and response to complex environmental changes of the vehicle.

[0056] In an optional embodiment, the preset constraint condition includes a steering angle constraint condition and an angle change rate constraint condition, the steering angle constraint condition is used to determine whether the prediction angle control sequence is within a preset angle range, and the angle change rate constraint condition is used to determine whether the angle change rate corresponding to the prediction angle control sequence is within a preset change rate range.

[0057] The above steering angle constraint condition is mainly used to limit each steering angle value in the prediction angle control sequence, to ensure that it is within the preset angle range u min ≤ u ≤ u max By accurately setting the maximum steering angle and the minimum steering angle, the risk of physical limit exceeding that may occur when the autonomous vehicle executes the steering instruction, such as tire side slip, vehicle out of control, etc., is avoided, thereby improving the safety and feasibility of the overall control process.

[0058] The above angle change rate constraint condition focuses on the rate of change of the steering angle in the prediction angle control sequence, to ensure that the change is kept within the preset change rate range Therefore, not only the stability of the vehicle dynamic response is considered, to avoid the discomfort or potential safety problems caused by sudden change of the steering angle, but also the adaptability of the autonomous vehicle in complex road environment is optimized, to ensure that the turning and lane keeping actions are smoother and more natural, and the passenger's comfort experience is improved.

[0059] The preset constraint condition plays a key role in the control process of the embodiments of the present application when performing verification processing. After the prediction angle control sequence is calculated, the steering angle constraint condition and the angle change rate constraint condition can be used to strictly verify each angle value and its change rate of the prediction angle control sequence. Specifically, for the steering angle of each prediction sampling period, it is checked whether it falls within the preset angle range; for the change rate of the steering angle, it is ensured that the change trend and amplitude do not exceed the preset change rate range. When the prediction angle control sequence completely meets the above two constraint conditions, the prediction angle control sequence can be used as the basis for determining the first prediction result.

[0060] Based on the above optional embodiments, by introducing preset constraints, especially steering angle constraints and angle change rate constraints, the safety and stability of the lateral control instructions of the autonomous driving vehicle are effectively guaranteed, avoiding driving safety problems caused by unreasonable settings of the steering angle or change rate. At the same time, it also significantly improves the adaptability of the autonomous driving vehicle in complex road environments and ensures the riding comfort of passengers.

[0061] In an optional embodiment, in step S124, determining the first prediction result based on the verification processing result includes:

[0062] Step S1241 , in response to determining based on the verification processing result that the predicted angle control sequence satisfies the preset constraint condition, determining an optimization objective function based on the future state data and the predicted angle control sequence;

[0063] Step S1242 : In response to the function value corresponding to the optimization objective function satisfying a preset numerical condition, a first prediction result is determined based on the prediction angle control sequence.

[0064] Specifically, when the verification process results indicate that the predicted angle control sequence satisfies the preset constraints, an optimization objective function is defined and calculated based on the future state data and the predicted angle control sequence. The optimization objective function is a criterion for measuring the effectiveness of the predicted angle control sequence and typically encompasses multiple aspects such as minimizing vehicle path tracking error, saving control energy, and considering passenger comfort. Thus, embodiments of the present application are able to screen out candidate predicted angle control sequences from among numerous possible steering wheel angle sequences that both adhere to the preset constraints and achieve a good control effect.

[0065] When the value of the optimization objective function satisfies preset numerical conditions, the predicted angle control sequence is directly assigned as the first prediction result, representing the steering angle that the vehicle should execute during the current control cycle. These preset numerical conditions ensure that the selected predicted angle control sequence not only satisfies the preset constraints but also reaches or approaches the target value of the optimization objective function. This serves as a judgment criterion, helping to avoid suboptimal solutions and ensuring that each lateral control decision is based on optimal path planning and vehicle state prediction.

[0066] The above optimization objective function can be:

[0067] J=Y T QY+U T RU

[0068] in Represents a vector of system output y of a set of sequences in the prediction time domain n, It represents the vector of control variables u in the prediction time domain n. Q and R are weights.

[0069] According to the calculation principle of the MPC algorithm, a set of U is selected to make the value of J minimum under the preset constraint condition. The value of u1 is taken as the control variable to be executed at the current time step, and the same calculation and optimization operation is taken at the next time step.

[0070] The kinematic model of the vehicle is simulated and predicted by fitting the neural network, and in the process of solving the optimal solution, the action space can be discretized according to the complexity of the calculation, and the optimal control sequence U is found based on the search method. For example, between u min and u max , m actions are discretely and equidistantly set, and the optimal solution is searched in the time domain n.

[0071] Based on the above optional embodiments, by responding to the determination that the predicted angle control sequence meets the preset constraint condition based on the verification processing result, the optimization objective function is determined based on the future state data and the predicted angle control sequence, and then the first prediction result is determined based on the predicted angle control sequence in response to the function value corresponding to the optimization objective function satisfying the preset numerical condition, thereby improving the accuracy and response speed of the automatic driving vehicle lateral control, reducing the path deviation, improving the driving stability and safety of the vehicle, and at the same time, the comfort of the passengers is taken into account.

[0072] In an optional embodiment, the reinforcement learning model includes a policy prediction network model and a value estimation network model, and the vehicle control method in the embodiment of the application further includes:

[0073] The sample state data is analyzed and processed by using the policy prediction network model to obtain an action prediction result;

[0074] The action prediction result is evaluated by using the value estimation network model to obtain a value evaluation result, wherein the value evaluation result is used to determine the mapping relationship between the sample state data, the action prediction result and the target action reward;

[0075] The policy prediction network model and the value estimation network model are updated based on the value evaluation result until the reinforcement learning model meets the preset convergence condition.

[0076] The main responsibility of the above-mentioned policy prediction network model is to receive sample state data, i.e. the state information of the automatic driving vehicle in a specific environment, and output an action prediction result after internal calculation and processing. The policy prediction network model can quickly predict the most suitable action (such as the steering angle of the steering wheel) based on the complex and variable vehicle environment state, and continuously optimize its own prediction strategy as the training deepens, and finally achieve high-precision control instruction prediction.

[0077] The value estimation network model is responsible for evaluating the action prediction result, and outputs a value evaluation result. The value evaluation result essentially reflects the expected return after performing a certain action in a specific state, i.e., the target action reward. The above evaluation process establishes a mapping relationship between the sample state data, the action prediction result and the target action reward, which is extremely crucial for guiding the learning direction of the reinforcement learning model. Through continuous iterative learning, the value estimation network model of the embodiments of the present application can accurately quantify the actual utility of the action, and promote the optimization and upgrading of the policy prediction network model.

[0078] In combination with the application scenario of autonomous driving, the action prediction result is the immediate reaction of the policy prediction network model to a given vehicle state, aiming to provide appropriate vehicle control instructions for the vehicle; while the value evaluation result is the evaluation of the control instruction execution effect by the value estimation network model, both of which jointly drive the learning process of the reinforcement learning model. In each round of training, the policy prediction network model and the value estimation network model will be updated based on the latest value evaluation result, and the above process will continue until the reinforcement learning model reaches the preset convergence condition, i.e., the change of the model parameters tends to be stable and no longer significantly improves the prediction or evaluation accuracy.

[0079] The above preset convergence condition serves as the determination standard for terminating the training of the reinforcement learning model, ensuring that the final form of the policy prediction network model and the value estimation network model in the embodiments of the present application is stable and efficient, and can provide reliable and accurate control suggestions in the field of autonomous driving.

[0080] Specifically, to accelerate the calculation of the control variable, the embodiments of the present application introduce deep reinforcement learning. The state variable x is defined as the environment S in reinforcement learning, and the optimal control variable u is defined as the action a in reinforcement learning. The reward function in reinforcement learning is defined as A reinforcement learning agent is constructed, and the policy prediction network model N policy and the value estimation network model N value are initialized. The target fitting neural network N is used to calculate the change of the environment state, and the agent randomly takes an action a in the environment, and the environment state becomes S ′ , S ′ = predict(N, S). The agent repeatedly explores in the environment, and the value estimation network model N value learns the mapping relationship between (S, a) and the target action reward according to the data records in the exploration process of the agent. Through the experience accumulated in the exploration process of the agent, the policy prediction network model N policy and the value estimation network model N valueFinally, the preset convergence condition is met. When the agent calculates the optimal execution action a under the environment state S through the policy prediction network model, the optimal reward can be obtained in the time domain n. It should be noted that the embodiments of the present application do not limit the reinforcement learning algorithm, and the above method is only used as a specific embodiment description, and other reinforcement learning algorithms such as DeepQ-Network (DQN), Proximal Policy Optimization (PPO) and the like can also be used.

[0081] Based on the above optional embodiments, by using the policy prediction network model to analyze and process the sample state data, the action prediction result is obtained, and then the value estimation network model is used to evaluate and process the action prediction result, and the value evaluation result is obtained, and finally the policy prediction network model and the value estimation network model are updated based on the value evaluation result until the reinforcement learning model meets the preset convergence condition, which further improves the driving performance of the autonomous vehicle, significantly enhances the accuracy and stability of the path following, and improves the vehicle control efficiency and reliability.

[0082] In an optional embodiment, in step S13, the steering control data corresponding to the target vehicle is determined based on the first prediction result, the second prediction result and the third prediction result, which comprises:

[0083] In step S131, the fusion weight data is obtained, wherein the fusion weight data is determined according to the driving scene of the target vehicle;

[0084] In step S132, the fusion weight data is used to fuse the first prediction result, the second prediction result and the third prediction result to obtain the steering control data.

[0085] The fusion weight data is essentially a series of numerical coefficients, which measures the relative importance and reliability of three different prediction control methods, i.e. the MPC algorithm based on the traditional vehicle kinematics model, the MPC algorithm based on the neural network model prediction and the control model based on the deep reinforcement learning strategy, when processing the current driving scene. The fusion weight data is not static and unchangeable, but dynamically adjusted, which can automatically adjust the weight proportion of various prediction methods according to the real-time changes of the driving scene (such as urban congestion, highway, rainy and snowy weather, etc.), so as to obtain a suitable control strategy. Thus, the adaptability and control accuracy of the autonomous vehicle in different environments are improved, and additional information is not required, and the control decision can be effectively optimized.

[0086] The obtained fusion weight data is used to fuse the first prediction result, the second prediction result and the third prediction result, and then the final steering control data is obtained. Specifically, the results obtained by the three prediction control methods can be weighted and averaged according to the preset weight, so as to obtain a steering control instruction that comprehensively integrates the advantages of various control strategies. In the actual application scene of the autonomous vehicle, the fusion weight data will be loaded into the control system first, and then the first prediction result, the second prediction result and the third prediction result will be weighted and synthesized according to the proportion of the fusion weight data. Therefore, the final steering control data not only considers the accuracy of the vehicle dynamics and the prediction model, but also integrates the flexibility of the intelligent decision system, so as to provide more stable and effective control commands in complex driving environments.

[0087] Based on the above optional embodiment, by obtaining fusion weight data and then using the fusion weight data to fuse the first prediction result, the second prediction result and the third prediction result, the steering control data is obtained, thereby enhancing the road condition adaptability of the autonomous vehicle. Especially in the face of unforeseen or extreme driving scenarios, the decision basis can be flexibly adjusted to ensure that the control system can always make decisions based on the most suitable information, avoiding the limitations of a single algorithm in some scenarios and improving the safety and efficiency of vehicle operation.

[0088] In an optional embodiment, in step S14, using the steering control data to perform lateral control on the target vehicle includes:

[0089] In step S141, a steering control instruction is generated based on the steering control data, wherein the steering control instruction is used to adjust the current steering angle of the target vehicle.

[0090] In step S142, the target vehicle is controlled laterally using the steering control instruction.

[0091] Specifically, the above-mentioned steering control data is comprehensive data obtained by fusing the outputs of multiple prediction control methods and processing according to the fusion weight in a specific scene in the embodiments of the application. The steering control data contains reasonable suggestions for the lateral control of the target vehicle, that is, the steering angle adjustment information of the steering wheel after considering the vehicle kinematics, neural network prediction and deep reinforcement learning strategy. The generated steering control instruction is used to guide the steering system of the target vehicle to make corresponding angle adjustment, so as to ensure that the vehicle can accurately follow the predetermined path. Therefore, complex data can be efficiently converted into specific and executable control instructions, and the path following accuracy and response speed of the autonomous vehicle are improved.

[0092] The target vehicle is controlled laterally by using the generated steering control instruction, which can be applied to the steering system of the vehicle to adjust the angle of the steering wheel and keep the target vehicle in the correct driving direction in the current lane. In the automatic driving scenario, the steering control instruction is sent to the actuator of the vehicle, such as the electronic power steering system, the electronic stability program, etc., to directly intervene in the steering behavior of the vehicle and achieve fine control of the lateral movement of the vehicle.

[0093] Based on the above optional embodiments, by generating steering control instructions based on steering control data and then using the steering control instructions to control the target vehicle laterally, the accuracy and real-time performance of steering control are ensured, and even in high-speed driving or complex road conditions, the steering angle can be quickly adjusted to improve the safety of vehicle driving and the accuracy of path following.

[0094] In an optional embodiment, the current state data includes: lateral deviation, rate of change of lateral deviation, heading angle deviation, rate of change of heading angle deviation, longitudinal speed, lateral speed, longitudinal acceleration, lateral acceleration, heading angle, rate of change of heading angle, left front wheel slip rate, right front wheel slip rate, left rear wheel slip rate, right rear wheel slip rate, ground adhesion coefficient, and current steering angle.

[0095] Specifically, the input current state data can be a 16-dimensional vector, which respectively represents the lateral deviation, rate of change of lateral deviation, heading angle deviation, rate of change of heading angle deviation, longitudinal speed, lateral speed, longitudinal acceleration, lateral acceleration, heading angle, rate of change of heading angle, left front wheel slip rate, right front wheel slip rate, left rear wheel slip rate, right rear wheel slip rate, ground adhesion coefficient, and current steering angle in the automatic driving lateral control process.

[0096] The lateral deviation refers to the horizontal distance difference between the current position of the vehicle and the predetermined driving trajectory, which is used to evaluate the degree of deviation of the vehicle from the expected path. The rate of change of the lateral deviation represents the speed of change of the lateral deviation over time, revealing the dynamic information of the deviation tendency of the vehicle. The heading angle deviation is the difference between the heading angle of the vehicle and the target heading angle, reflecting the deviation between the direction of the vehicle and the expected path. The rate of change of the heading angle deviation shows the instantaneous change rate of the heading angle deviation, which helps to understand the degree of urgency of the vehicle heading adjustment. The longitudinal speed is the speed of the vehicle along the driving direction, which is used to evaluate the speed state of the vehicle advancing. The lateral speed is the speed perpendicular to the driving direction, which embodies the characteristics of the lateral dynamics of the vehicle, such as side slip, etc. The longitudinal acceleration is the acceleration or deceleration degree of the vehicle along the driving direction, which reveals the current working state of the vehicle power system. The lateral acceleration is the acceleration of the vehicle in the lateral direction, which is used to monitor the lateral dynamics of the vehicle, such as the acceleration during turning. The heading angle is the rotation angle between the vehicle and the fixed reference direction (such as north), which provides basic information of the direction of the vehicle. The rate of change of the heading angle is the instantaneous change speed of the heading angle, which helps to understand and predict the turning tendency of the vehicle. The left front wheel slip rate reflects the difference between the actual rolling state and the expected rolling state of the left front wheel relative to the road surface, which is used to monitor the tire grip. The right front wheel slip rate is similar to the left front wheel slip rate, but it is for the right front wheel, which is an important reference information for vehicle dynamic control. The left rear wheel slip rate monitors the slip state of the left rear wheel, which is closely related to the stability and handling characteristics of the vehicle. The right rear wheel slip rate focuses on the slip situation of the right rear wheel, which is used together with the left rear wheel slip rate to understand the longitudinal and lateral dynamic balance of the vehicle. The ground adhesion coefficient describes the degree of friction between the tire and the ground, which directly affects the handling stability and braking performance of the vehicle. The current steering angle indicates the current steering wheel rotation angle of the vehicle, which is the key state information in the lateral control algorithm.

[0097] Based on the above optional embodiments, the rich information of the current state data can be used to evaluate and adjust the lateral control strategy of the vehicle in real time, especially in complex driving environments, such as nonlinear kinematic models that are difficult to accurately predict the state of the vehicle. The comprehensiveness of the current state data provides more accurate feedback signals for the control system, ensuring that the vehicle can still travel smoothly and safely when facing various challenges.

[0098] Figure 2 is a schematic diagram of a vehicle control method according to an embodiment of the present application, such as Figure 2As shown, with the preview distance, a preview point and a reference point can be determined on the vehicle predicted path. The preview point is an observation point in front of the vehicle predicted path set according to the target preview distance, and the reference point is the corresponding point of the preview point on the actual reference trajectory, which is usually obtained by matching the preview point position with the pre-planned route map to find the trajectory point closest to the preview point. The lateral deviation and the heading angle deviation can be calculated according to the preview point and the reference point.

[0099] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0100] According to an embodiment of the present application, a device embodiment of a vehicle control method is provided. It should be noted that the device can be used to execute the vehicle control method described above. Figure 3 is a structural block diagram of a vehicle control device according to an embodiment of the present application, as shown in the figure, the device comprises: Figure 3

[0101] The acquisition module 301 is configured to acquire current state data of the target vehicle, wherein the current state data is used to represent the vehicle state of the target vehicle in the current sampling period.

[0102] The prediction module 302 is configured to perform prediction analysis based on the current state data to obtain a first prediction result, a second prediction result and a third prediction result, wherein the first prediction result is used to represent the steering wheel steering angle predicted by the target fitting neural network model, the second prediction result is used to represent the steering wheel steering angle predicted by the reinforcement learning model, and the third prediction result is used to represent the steering wheel steering angle predicted by the vehicle motion physical model.

[0103] The determination module 303 is configured to determine the steering control data corresponding to the target vehicle based on the first prediction result, the second prediction result and the third prediction result, wherein the steering control data is used to represent the steering wheel steering angle of the target vehicle in the future sampling period.

[0104] The control module 304 is configured to perform lateral control on the target vehicle by using the steering control data.

[0105] ​Optionally, the prediction module 302 is further configured to: perform state prediction on the current state data by using a target fitting neural network model to obtain future state data of the target vehicle, wherein the future state data is used to represent vehicle states of the target vehicle in a plurality of prediction sampling periods; determine a predicted angle control sequence based on the future state data, wherein the predicted angle control sequence is used to represent a steering angle vector corresponding to the plurality of prediction sampling periods; perform verification processing on the predicted angle control sequence by using a preset constraint condition to obtain a verification processing result; and determine the first prediction result based on the verification processing result.

[0106] Optionally, the preset constraint condition comprises a steering angle constraint condition and an angle change rate constraint condition, the steering angle constraint condition is used to determine whether the predicted angle control sequence is within a preset angle range, and the angle change rate constraint condition is used to determine whether an angle change rate corresponding to the predicted angle control sequence is within a preset change rate range.

[0107] Optionally, the prediction module 302 is further configured to: in response to determining that the predicted angle control sequence satisfies the preset constraint condition based on the verification processing result, determine an optimization objective function based on the future state data and the predicted angle control sequence; and in response to a function value corresponding to the optimization objective function satisfying a preset numerical condition, determine the first prediction result based on the predicted angle control sequence.

[0108] Optionally, the reinforcement learning model comprises a policy prediction network model and a value estimation network model, and the vehicle control device comprises: an analysis module 305 configured to perform analysis processing on the sample state data by using the policy prediction network model to obtain an action prediction result; an evaluation module 306 configured to perform evaluation processing on the action prediction result by using the value estimation network model to obtain a value evaluation result, wherein the value evaluation result is used to determine a mapping relationship between the sample state data, the action prediction result and a target action reward; and an update module 307 configured to perform update processing on the policy prediction network model and the value estimation network model based on the value evaluation result until the reinforcement learning model satisfies a preset convergence condition.

[0109] Optionally, the determination module 303 is further configured to: obtain fusion weight data, wherein the fusion weight data is determined according to a driving scene of the target vehicle; and perform fusion processing on the first prediction result, the second prediction result and the third prediction result by using the fusion weight data to obtain steering control data.

[0110] Optionally, the control module 304 is further configured to: generate a steering control instruction based on the steering control data, wherein the steering control instruction is used to adjust a current steering angle of the target vehicle; and perform lateral control on the target vehicle by using the steering control instruction.

[0111] Optionally, the current state data comprises: lateral deviation, rate of change of lateral deviation, heading angle deviation, rate of change of heading angle deviation, longitudinal velocity, lateral velocity, longitudinal acceleration, lateral acceleration, heading angle, rate of change of heading angle, left front wheel slip ratio, right front wheel slip ratio, left rear wheel slip ratio, right rear wheel slip ratio, ground adhesion coefficient, and current steering angle.

[0112] Embodiments of the present application further provide a vehicle, comprising: a memory storing an executable program; and a processor configured to execute the program, wherein the program, when executed, performs the method in any of the embodiments of the present application.

[0113] Embodiments of the present application further provide a computer readable storage medium, comprising a stored executable program, wherein the executable program, when executed, controls a device in which the computer readable storage medium is located to perform the method in any of the embodiments of the present application.

[0114] Embodiments of the present application further provide a computer program product, comprising a computer program which, when executed by a processor, performs the method in any of the embodiments of the present application.

[0115] Embodiments of the present application further provide a computer program product, comprising a non-volatile computer readable storage medium for storing a computer program which, when executed by a processor, performs the method in any of the embodiments of the present application.

[0116] Embodiments of the present application further provide a computer program which, when executed by a processor, performs the method in any of the embodiments of the present application.

[0117] In the above-described embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0118] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other means. Among them, the above-described device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0119] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0120] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0121] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various program code storage media.

[0122] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A vehicle control method, characterized in that: include: Acquiring current state data of the target vehicle, wherein the current state data is used to represent the vehicle state of the target vehicle in a current sampling period; Performing a prediction analysis based on the current state data to obtain a first prediction result, a second prediction result, and a third prediction result, wherein the first prediction result is used to represent the steering wheel steering angle predicted by the target fitting neural network model, the second prediction result is used to represent the steering wheel steering angle predicted by the reinforcement learning model, and the third prediction result is used to represent the steering wheel steering angle predicted by the vehicle motion physics model; Determining steering control data corresponding to the target vehicle based on the first prediction result, the second prediction result, and the third prediction result, wherein the steering control data is used to represent a steering wheel steering angle of the target vehicle in a future sampling period; The target vehicle is laterally controlled using the steering control data.

2. The method according to claim 1, characterized in that Performing a prediction analysis based on the current state data to obtain the first prediction result includes: Performing state prediction on the current state data using the target fitting neural network model to obtain future state data of the target vehicle, wherein the future state data is used to represent the vehicle state of the target vehicle within a plurality of prediction sampling periods; determining a predicted angle control sequence based on the future state data, wherein the predicted angle control sequence is used to represent the steering angle vectors corresponding to the multiple prediction sampling periods; Performing verification processing on the predicted angle control sequence using preset constraint conditions to obtain a verification processing result; The first prediction result is determined based on the verification processing result.

3. The method according to claim 2, characterized in that The preset constraints include: a steering angle constraint and an angle change rate constraint. The steering angle constraint is used to determine whether the predicted angle control sequence is within a preset angle range, and the angle change rate constraint is used to determine whether the angle change rate corresponding to the predicted angle control sequence is within a preset change rate range.

4. The method according to claim 2, characterized in that Determining the first prediction result based on the verification processing result includes: In response to determining that the predicted angle control sequence satisfies the preset constraint condition based on the verification processing result, determining an optimization objective function based on the future state data and the predicted angle control sequence; In response to a function value corresponding to the optimization objective function satisfying a preset numerical condition, the first prediction result is determined based on the prediction angle control sequence.

5. The method according to claim 1, wherein The reinforcement learning model includes: a strategy prediction network model and a value estimation network model, and the method includes: Analyzing and processing the sample state data using the strategy prediction network model to obtain action prediction results; Using the value estimation network model to evaluate the action prediction result to obtain a value evaluation result, wherein the value evaluation result is used to determine a mapping relationship between the sample state data, the action prediction result, and the target action reward; The strategy prediction network model and the value estimation network model are updated based on the value evaluation result until the reinforcement learning model meets the preset convergence condition.

6. The method according to claim 1, characterized in that Determining the steering control data corresponding to the target vehicle based on the first prediction result, the second prediction result, and the third prediction result includes: Acquiring fusion weight data, wherein the fusion weight data is determined according to a driving scenario of the target vehicle; The first prediction result, the second prediction result and the third prediction result are fused using the fusion weight data to obtain the steering control data.

7. The method according to claim 1, characterized in that Performing lateral control on the target vehicle using the steering control data includes: generating a steering control instruction based on the steering control data, wherein the steering control instruction is used to adjust a current steering angle of the target vehicle; The target vehicle is laterally controlled using the steering control instruction.

8. The method according to claim 1, characterized in that The current state data includes: lateral deviation, rate of change of lateral deviation, heading angle deviation, rate of change of heading angle deviation, longitudinal speed, lateral speed, longitudinal acceleration, lateral acceleration, heading angle, rate of change of heading angle, left front wheel slip rate, right front wheel slip rate, left rear wheel slip rate, right rear wheel slip rate, ground adhesion coefficient, and current steering angle.

9. A vehicle, characterized in that: include: a memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 8 when running.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 8.

Images