Vehicle control model training method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle control model training method and device, electronic equipment and a storage medium, and relates to the technical field of automatic driving, and the method comprises the steps: obtaining the first expected trajectory information of a target vehicle at the current moment and the current simulation vehicle state information; inputting the first expected trajectory information and the current simulation vehicle state information into a basic control model of a controller in a basic vehicle to obtain a first control quantity of an actuator; inputting the first control quantity, the historical vehicle state information of the target vehicle and the historical execution quantity output by the target execution model into the target execution model to predict the current execution quantity of a target vehicle actuator; updating simulation vehicle track information of the target vehicle according to the current execution quantity; and adjusting model parameters of the basic control model according to the deviation between the simulated vehicle trajectory information and the first expected trajectory information to obtain a target control model of the target vehicle controller. By applying the technical scheme provided by the embodiment of the invention, the matching efficiency of the controller and the actuator can be improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a vehicle control model training method, device, electronic device and storage medium. Background Art

[0002] ADAS (Advanced Driving Assistant System) is deployed in autonomous vehicles. ADAS includes an upper-layer controller and the lower-layer ESC (Electronic Stability Control System) and EPS (Electronic Power Steering System). ESC and EPS each include corresponding actuators. The above controller controls the actuators in the lower-layer ESC and EPS to execute instructions, thereby realizing the control of the lateral and longitudinal movements of the vehicle, such as vehicle driving, braking, steering, etc.

[0003] However, due to the differences in chassis hardware of different vehicles, the actuators of different vehicles have different vehicle motion control effects when executing the same instructions from the controller. In order to control the vehicle to drive along the desired trajectory, it is necessary to adaptively adjust the control strategy of the controller for different vehicles. In this way, the controller can issue corresponding instructions based on the execution effect of the vehicle actuator, so that the vehicle actuator can achieve the desired vehicle motion control effect after executing the instructions issued by the controller.

[0004] In the traditional way, the control strategy in the controller is adjusted by manual parameter adjustment to adapt to the lower-level actuator. This method is dependent on the personal experience of the parameter adjustment engineer, and the parameter adjustment results need to be repeatedly verified on the actual vehicle. The overall debugging cycle is long, and the matching efficiency of the controller and the actuator is not high. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a vehicle control model training method, device, electronic device and storage medium to improve the matching efficiency between the controller and the actuator. The specific technical solution is as follows:

[0006] In a first aspect of an embodiment of the present application, a vehicle control model training method is provided, the method comprising:

[0007] Obtaining first expected trajectory information of the target vehicle at the current moment and current simulated vehicle state information;

[0008] Inputting the first expected trajectory information and the current simulated vehicle state information into a basic control model of a controller in a base vehicle, and obtaining a first control amount of an actuator output by the basic control model, wherein the base vehicle is: a vehicle in which a controller matches an actuator, the input data of the basic control model includes: the expected trajectory information and vehicle state information of the base vehicle, and the output data of the basic control model includes: the control amount that the actuator of the base vehicle needs to execute when the base vehicle travels according to the trajectory indicated in the expected trajectory information;

[0009] The first control amount, the historical vehicle state information of the target vehicle and the historical execution amount output by the target execution model are input into the target execution model to predict the current execution amount of the actuator in the target vehicle, wherein the target execution model is: a working model of the actuator in the target vehicle, and the target execution model outputs the execution amount of the target vehicle after executing the control amount according to the input control amount;

[0010] According to the current execution amount, updating the simulated vehicle trajectory information of the target vehicle;

[0011] According to the deviation between the simulated vehicle trajectory information and the first expected trajectory information, the model parameters of the basic control model are adjusted to obtain a target control model of the controller in the target vehicle.

[0012] In a possible implementation manner, adjusting the model parameters of the basic control model according to the deviation between the simulated vehicle trajectory information and the first expected trajectory information to obtain the target control model of the controller in the target vehicle includes:

[0013] Calculating a deviation between the simulated vehicle trajectory information and the first expected trajectory information;

[0014] Obtaining a first value return value corresponding to the first control amount based on the deviation, wherein the first value return value represents: the accuracy of using the first control amount to control the actuator in the target vehicle at the next moment;

[0015] The model parameters of the basic control model are adjusted based on the first value return value to obtain a target control model of the controller in the target vehicle.

[0016] In a possible implementation manner, the simulated vehicle trajectory information includes: trajectory information of the target vehicle at each moment in a first time period, wherein the first time period is: a time period of a preset time period from the current moment backward;

[0017] The step of adjusting the model parameters of the basic control model based on the first value return value to obtain a target control model of the controller in the target vehicle includes:

[0018] A second value return value corresponding to the first control amount is obtained based on the deviation, wherein the second value return value represents: a matching degree of the first vehicle trajectory information relative to the first expected trajectory information when the first control amount is used to control the actuator in the target vehicle, and the first vehicle trajectory information is: simulated vehicle trajectory information of the target vehicle at each moment except the current moment and the next moment within the first time period;

[0019] Generate a cumulative value return value of the first value return value and the second value return value;

[0020] The model parameters of the basic control model are adjusted based on the cumulative value return value to obtain a target control model of the controller in the target vehicle.

[0021] In a possible implementation manner, generating a cumulative value return value of the first value return value and the second value return value includes:

[0022] The cumulative value return value is generated according to the following expression:

[0023] R t =r t+1 +γr t+2 +…+γ n-1 r t+n ;

[0024] Among them, R t is the cumulative value return value, t is the current time, γ is the set attenuation coefficient, n is the preset time length, r t+1 is the first value return value, r t+2 ,……,r t+n Return value for the second value.

[0025] In a possible implementation, the current simulated vehicle state information includes: a vehicle posture obtained by updating the vehicle posture at the previous moment based on the previous execution amount, and a simulated vehicle state of the target vehicle after the actuator executes the previous execution amount predicted by the target execution model, wherein the previous execution amount is: an execution amount predicted by the target execution model based on the first control amount at the previous moment, historical vehicle state information, and historical execution amount; or

[0026] The current simulated vehicle state information includes: simulated vehicle state information obtained by updating the simulated vehicle state information at the last moment based on the last execution amount.

[0027] In a possible implementation manner, the basic control model is trained in the following manner:

[0028] For each sample moment, obtaining sample expected trajectory information of the base vehicle, sample vehicle state information, and an actual control amount output by a controller in the base vehicle;

[0029] Inputting sample expected trajectory information and sample vehicle state information at each sample time into the basic control model respectively, and obtaining a second control amount of the actuator corresponding to each sample time output by the basic control model;

[0030] According to the difference between the second control amount corresponding to each sample moment and the actual control amount, the model parameters of the basic control model are adjusted.

[0031] In a possible implementation manner, the target execution model is trained in the following manner:

[0032] For each sample moment, obtaining sample vehicle state information of the target vehicle, an actual control amount and an actual execution amount input by an actuator in the target vehicle;

[0033] Inputting the sample vehicle state information, the actual control amount, the actual execution amount, and the actual control amount at each sample moment corresponding to the historical moment of each sample moment into the target execution model respectively, and obtaining the first vehicle state information and the first execution amount corresponding to each sample moment output by the target execution model;

[0034] The model parameters of the target execution model are adjusted according to the difference between the first vehicle state information corresponding to each sample moment and the sample vehicle state information corresponding to each sample moment, and the difference between the first execution amount corresponding to each sample moment and the actual execution amount corresponding to each sample moment.

[0035] In a possible implementation manner, after obtaining the target control model, the method further includes:

[0036] Obtaining the second expected trajectory information and the current actual vehicle state information of the target vehicle at the current moment;

[0037] Inputting the second desired trajectory information and the current actual vehicle state information into the target control model to obtain a third control variable of the actuator output by the target control model;

[0038] Obtaining actual vehicle trajectory information of the target vehicle after the actuator of the target vehicle performs an operation under the third control amount;

[0039] According to the deviation between the actual vehicle trajectory information and the second expected trajectory information, the model parameters of the target control model are adjusted.

[0040] In a possible implementation manner, the actuator includes at least one of the following actuators: a steering actuator, a power actuator, a brake actuator; and / or

[0041] The target execution model includes at least one of the following models: a steering execution model, a power execution model, and a braking execution model.

[0042] In a second aspect of an embodiment of the present application, a vehicle control model training device is provided, the device comprising:

[0043] A first acquisition module is used to obtain the first expected trajectory information of the target vehicle at the current moment and the current simulated vehicle state information;

[0044] a second obtaining module, for inputting the first expected trajectory information and the current simulated vehicle state information into a basic control model of a controller in a basic vehicle, and obtaining a first control amount of an actuator output by the basic control model, wherein the basic vehicle is a vehicle in which a controller matches an actuator, the input data of the basic control model includes the expected trajectory information and vehicle state information of the basic vehicle, and the output data of the basic control model includes the control amount that the actuator of the basic vehicle needs to execute when the basic vehicle travels along the trajectory indicated in the expected trajectory information;

[0045] a prediction module, configured to input the first control amount, the historical vehicle state information of the target vehicle and the historical execution amount output by the target execution model into the target execution model, and predict the current execution amount of the actuator in the target vehicle, wherein the target execution model is: a working model of the actuator in the target vehicle, and the target execution model outputs the execution amount of the target vehicle after executing the control amount according to the input control amount;

[0046] An updating module, used for updating the simulated vehicle trajectory information of the target vehicle according to the current execution amount;

[0047] The first adjustment module is used to adjust the model parameters of the basic control model according to the deviation between the simulated vehicle trajectory information and the first expected trajectory information to obtain the target control model of the controller in the target vehicle.

[0048] In a possible implementation manner, the first adjustment module is specifically configured to:

[0049] Calculating a deviation between the simulated vehicle trajectory information and the first expected trajectory information;

[0050] Obtaining a first value return value corresponding to the first control amount based on the deviation, wherein the first value return value represents: the accuracy of using the first control amount to control the actuator in the target vehicle at the next moment;

[0051] The model parameters of the basic control model are adjusted based on the first value return value to obtain a target control model of the controller in the target vehicle.

[0052] In a possible implementation manner, the simulated vehicle trajectory information includes: trajectory information of the target vehicle at each moment in a first time period, wherein the first time period is: a time period of a preset time period from the current moment backward;

[0053] The first adjustment module is specifically used to:

[0054] A second value return value corresponding to the first control amount is obtained based on the deviation, wherein the second value return value represents: a matching degree of the first vehicle trajectory information relative to the first expected trajectory information when the first control amount is used to control the actuator in the target vehicle, and the first vehicle trajectory information is: simulated vehicle trajectory information of the target vehicle at each moment except the current moment and the next moment within the first time period;

[0055] Generate a cumulative value return value of the first value return value and the second value return value;

[0056] The model parameters of the basic control model are adjusted based on the cumulative value return value to obtain a target control model of the controller in the target vehicle.

[0057] In a possible implementation manner, the first adjustment module is specifically configured to:

[0058] The cumulative value return value is generated according to the following expression:

[0059] R t =r t+1 +γr t+2 +…+γ n-1 r t+n ;

[0060] Among them, R t is the cumulative value return value, t is the current time, γ is the set attenuation coefficient, n is the preset time length, r t+1 is the first value return value, r t+2 ,……,r t+n Return value for the second value.

[0061] In a possible implementation, the current simulated vehicle state information includes: a vehicle posture obtained by updating the vehicle posture at the previous moment based on the previous execution amount, and a simulated vehicle state of the target vehicle after the actuator executes the previous execution amount predicted by the target execution model, wherein the previous execution amount is: an execution amount predicted by the target execution model based on the first control amount at the previous moment, historical vehicle state information, and historical execution amount; or,

[0062] The current simulated vehicle state information includes: simulated vehicle state information obtained by updating the simulated vehicle state information at the last moment based on the last execution amount.

[0063] In a possible implementation manner, the basic control model is trained in the following manner:

[0064] For each sample moment, obtaining sample expected trajectory information of the base vehicle, sample vehicle state information, and an actual control amount output by a controller in the base vehicle;

[0065] Inputting sample expected trajectory information and sample vehicle state information at each sample time into the basic control model respectively, and obtaining a second control amount of the actuator corresponding to each sample time output by the basic control model;

[0066] According to the difference between the second control amount corresponding to each sample moment and the actual control amount, the model parameters of the basic control model are adjusted.

[0067] In a possible implementation manner, the target execution model is trained in the following manner:

[0068] For each sample moment, obtaining sample vehicle state information of the target vehicle, an actual control amount and an actual execution amount input by an actuator in the target vehicle;

[0069] Inputting the sample vehicle state information, the actual control amount, the actual execution amount, and the actual control amount at each sample moment corresponding to the historical moment of each sample moment into the target execution model respectively, and obtaining the first vehicle state information and the first execution amount corresponding to each sample moment output by the target execution model;

[0070] The model parameters of the target execution model are adjusted according to the difference between the first vehicle state information corresponding to each sample moment and the sample vehicle state information corresponding to each sample moment, and the difference between the first execution amount corresponding to each sample moment and the actual execution amount corresponding to each sample moment.

[0071] In a possible implementation manner, the device further includes a second adjustment module, configured to:

[0072] Obtaining the second expected trajectory information and the current actual vehicle state information of the target vehicle at the current moment;

[0073] Inputting the second desired trajectory information and the current actual vehicle state information into the target control model to obtain a third control variable of the actuator output by the target control model;

[0074] Obtaining actual vehicle trajectory information of the target vehicle after the actuator of the target vehicle performs an operation under the third control amount;

[0075] According to the deviation between the actual vehicle trajectory information and the second expected trajectory information, the model parameters of the target control model are adjusted.

[0076] In a possible implementation manner, the actuator includes at least one of the following actuators: a steering actuator, a power actuator, a brake actuator; and / or

[0077] The target execution model includes at least one of the following models: a steering execution model, a power execution model, and a braking execution model.

[0078] In a third aspect of the embodiments of the present application, an electronic device is provided, including:

[0079] Memory, used to store computer programs;

[0080] The processor is used to implement the steps of any method described in the first aspect when executing the program stored in the memory.

[0081] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of any method described in the first aspect are implemented.

[0082] In a fifth aspect of the embodiments of the present application, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the steps of any one of the methods described in the first aspect above.

[0083] Beneficial effects of the embodiments of the present application:

[0084] In the technical solution provided by the embodiment of the present application, the basic control model of the controller with a relatively mature control strategy in the base vehicle is first migrated to the target vehicle as the basic control model of the controller in the target vehicle; then the target execution model reflecting the execution effect of the actuator in the target vehicle is used to predict the execution result according to the first control quantity output by the basic control model; then the model parameters of the basic control model are adjusted according to the predicted execution result, so that the adjusted basic control model matches the execution effect of the target vehicle; then the adjusted basic control model is used as the target control model of the target vehicle controller, so that the controller of the target vehicle is matched with the actuator. In this way, there is no need to train the control model of the target vehicle controller from scratch, which speeds up the convergence speed of the target control model training, thereby improving the efficiency of matching the controller with the actuator.

[0085] In addition, the above scheme uses simulated vehicle state information and target execution model to perform simulation training of the vehicle control model. Compared with real vehicle training, simulation training is more efficient in obtaining training data, the training process is safer, and no human intervention is required, which reduces the cost and safety cost required for vehicle control model training and further improves the efficiency of model training, thereby further improving the efficiency of matching the controller and the actuator.

[0086] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0088] Figure 1 A schematic diagram of the relationship between a controller and an actuator in an autonomous driving vehicle provided in an embodiment of the present application;

[0089] Figure 2 A schematic diagram of a flow chart of a vehicle control model training method provided in an embodiment of the present application;

[0090] Figure 3 A detailed flow chart of step S205 provided in an embodiment of the present application;

[0091] Figure 4 A detailed flow chart of step S303 provided in an embodiment of the present application;

[0092] Figure 5A flow chart of a training method of a basic control model provided in an embodiment of the present application;

[0093] Figure 6 A training framework diagram of a basic control model provided in an embodiment of the present application;

[0094] Figure 7 A flowchart of a training method for a target execution model provided in an embodiment of the present application;

[0095] Figure 8 A training framework diagram of a target execution model provided in an embodiment of the present application;

[0096] Fig. 9 An interactive schematic diagram of an example of applying the vehicle control model training method provided by an embodiment of the present application;

[0097] Fig.10 A schematic diagram of a flow chart of a vehicle control model training method based on a real vehicle provided in an embodiment of the present application;

[0098] Fig.11 An interactive schematic diagram of an example of applying the above-mentioned vehicle control model training method based on a real vehicle;

[0099] Fig.12 A flowchart of an example of combining simulation training and real vehicle training provided in an embodiment of the present application;

[0100] Fig.13 A schematic diagram of the structure of a vehicle control model training device provided in an embodiment of the present application;

[0101] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0102] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application. It should be noted that the processing operations of the present application for data are all performed under the premise that the relevant users are aware and authorized.

[0103] ADAS is deployed in autonomous vehicles. ADAS includes an upper-layer controller and lower-layer ESC and EPS. ESC and EPS each include corresponding actuators. The above controller controls the actuators in the lower-layer ESC and EPS to execute instructions, thereby achieving control of the vehicle's lateral and longitudinal movements, such as vehicle driving, braking, steering, etc. Figure 1 As shown in the figure, the upper controller, also called the intelligent assisted driving controller, issues control instructions to the lower-level steering actuators, power actuators, brake actuators, etc. After the steering actuator executes the control instruction, the vehicle's wheels rotate a certain angle; after the power actuator executes the control instruction, the vehicle's driving torque changes; after the brake actuator executes the control instruction, the vehicle's acceleration changes.

[0104] However, due to the differences in chassis hardware and execution software of different vehicles, the actuators of different vehicles have different vehicle motion control effects when executing the same control instructions from the controller. In order to control the vehicle to drive along the desired trajectory, it is necessary to adaptively adjust the control strategy of the controller for different vehicles. In this way, the controller can issue corresponding control instructions based on the execution effect of the vehicle actuator, so that the vehicle actuator can achieve the desired vehicle motion control effect after executing the control instructions issued by the controller.

[0105] In the traditional way, the control strategy in the controller is adjusted by manual parameter adjustment to adapt to the lower-level actuator. This method is dependent on the personal experience of the parameter adjustment engineer, and the parameter adjustment results need to be repeatedly verified on the actual vehicle. The overall debugging cycle is long, and the matching efficiency of the controller and the actuator is not high.

[0106] In order to solve the above problems, the present application embodiment provides a vehicle control model training method, such as Figure 2 As shown, the method includes the following steps S201-S205.

[0107] The vehicle control model training method provided in the embodiment of the present application can be applied to electronic devices such as desktop computers, laptop computers or servers. For ease of description, the execution subject of the vehicle control model training method provided in the embodiment of the present application is collectively referred to as the first electronic device.

[0108] Step S201, obtaining the first expected trajectory information of the target vehicle at the current moment and the current simulated vehicle state information.

[0109] The target vehicle is the vehicle to be matched with the controller and the actuator. The first expected trajectory is the trajectory that the target vehicle is expected to travel after the current moment. The first expected trajectory information includes the curvature of the expected target vehicle's travel trajectory and the speed, acceleration, position, and heading angle of the target vehicle. The current simulated vehicle state information is the vehicle state information of the target vehicle in the simulation environment at the current moment. The current simulated vehicle state information includes the speed, acceleration, position, and heading angle of the target vehicle at the current moment.

[0110] For the target vehicle, a simulated environment model can be pre-built, and the environment model includes the target vehicle and the simulated road conditions on which the target vehicle is located. The vehicle state information of the target vehicle in the environment model can change over time. The first electronic device can obtain the current simulated vehicle state information of the target vehicle at the current moment from the environment model.

[0111] Based on the simulated road condition information and simulated vehicle state information in the constructed environment model, the existing decision planning model can be used to plan the driving trajectory of the target vehicle to generate the first expected trajectory information. The first electronic device can obtain the first expected trajectory information generated by the decision planning model.

[0112] Step S202: input the first expected trajectory information and the current simulated vehicle state information into a basic control model of a controller in a basic vehicle to obtain a first control variable of an actuator output by the basic control model.

[0113] Among them, the basic vehicle is: a vehicle in which the controller and the actuator are matched. ADAS is deployed on the basic vehicle, and ADAS includes an upper-layer controller and a bottom-layer ESC and EPS. ESC and EPS each include a corresponding actuator. The actuator may include at least one of the following: a steering actuator, a power actuator, and a braking actuator. The steering actuator can change the heading angle of the vehicle, the power actuator can change the speed and acceleration of the vehicle, and the braking actuator can change the speed and acceleration of the vehicle. The controller can send a control instruction to at least one of these actuators according to the desired trajectory, and the actuator executes the received control instruction to change the driving trajectory of the vehicle. If the degree of matching between the driving trajectory of the vehicle and the desired trajectory after the actuator executes the control instruction meets the preset matching requirements, it means that the controller and the actuator are matched.

[0114] Specifically, the matching degree between the driving trajectory and the expected trajectory may include: the accuracy of the vehicle position at each moment in the driving trajectory relative to the vehicle position at each moment in the expected trajectory, and the stability of the vehicle in the driving trajectory. The stability of the vehicle may be determined by combining one or more of the following factors: whether the speed of the vehicle changes suddenly, whether the driving direction of the vehicle swings greatly, etc.

[0115] Correspondingly, the preset matching requirement may be a quantitative threshold set for the above matching degree.

[0116] For example, with respect to the accuracy of the vehicle's position at each moment in the driving trajectory relative to the vehicle's position at each moment in the expected trajectory, the preset matching requirements may include the following settings: the accuracy of the vehicle's position at each moment in the driving trajectory relative to the vehicle's position at each moment in the expected trajectory is not less than 97%, where 97% is the set quantization threshold.

[0117] For another example, regarding whether the speed of the vehicle changes suddenly, the preset matching requirements may include the following settings: the acceleration of the vehicle at each moment is no greater than 6 meters per second squared, where 6 meters per second squared is the set quantization threshold.

[0118] For another example, regarding whether the driving direction of the vehicle swings significantly, the preset matching requirements may include the following settings: the heading angle of the vehicle does not swing more than 5 degrees, where 5 degrees is a set quantization threshold.

[0119] In one example, the preset matching requirements may include the above three settings. If, after the actuator executes the control quantity in the control instruction issued by the controller, the accuracy of the basic vehicle position at each moment in the vehicle's driving trajectory relative to the vehicle position at the corresponding moment in the expected trajectory is 99.9%, which is greater than the given quantization threshold of 97%, and the acceleration of the vehicle at each moment is less than 6 meters per second squared, and the heading angle of the vehicle does not swing more than 5 degrees, then it can be determined that the controller and the actuator are matched.

[0120] The basic control model is a working model of the controller in the base vehicle. The basic control model may be a deployed control model directly obtained from the controller of the base vehicle, and the deployed control model in the base vehicle controller may be generated based on the driving data of the base vehicle when debugging the base vehicle. The above-mentioned basic control model may also be generated based on the expected trajectory information and vehicle state of the base vehicle when the basic control model is needed in the embodiments of the present application.

[0121] When generating a basic control model based on the expected trajectory and vehicle state of the basic vehicle, the basic control model can be obtained by manual debugging according to the expected trajectory information and vehicle state of the basic vehicle, or the basic control model can be obtained by training the basic control model using equipment according to the vehicle trajectory information and vehicle state of the basic vehicle. The training method of the basic control model is described below and will not be repeated here.

[0122] The basic control model reflects the control strategy of the controller in the basic vehicle. Specifically, the basic control model can output the control amount that the actuator of the basic vehicle needs to execute when driving according to the trajectory indicated in the expected trajectory information based on the input expected trajectory information and the vehicle state information. That is, for a vehicle in the input vehicle state information, in order to make the vehicle drive according to the trajectory indicated in the expected trajectory information, the basic control model can be used to output the control amount that the actuator of the vehicle needs to execute, and then the actuator of the vehicle can execute the control amount output by the basic control model to change the driving trajectory of the vehicle.

[0123] The first electronic device can use the basic control model to obtain the first control amount performed by the actuator in order to drive according to the first expected trajectory information and the current simulated vehicle state information. The first expected trajectory is the trajectory described by the first expected trajectory information.

[0124] The first control amount includes a control amount for a steering actuator, a control amount for a power actuator, and a control amount for a brake actuator. The control amount for the steering actuator may be a steering wheel angle request value, the control force for the power actuator or the brake actuator may be a longitudinal acceleration request value, and the control amount for the power actuator may be a power torque request value.

[0125] The basic control model can determine the control amount for the steering actuator according to the heading angle in the first expected trajectory information. The basic control model can also determine the control amount for the power actuator or the brake actuator according to the speed, acceleration, etc. in the first expected trajectory information.

[0126] Step S203, inputting the first control amount, the historical vehicle state information of the target vehicle and the historical execution amount output by the target execution model into the target execution model to predict the current execution amount of the actuator in the target vehicle.

[0127] The target execution model is: a working model of an actuator in the target vehicle. The target execution model can be generated based on historical data of the target vehicle. The target execution model can include at least one of the following models: a steering execution model, a power execution model, and a braking execution model.

[0128] The target execution model reflects the execution effect of the actuator in the target vehicle. Specifically, the target execution model can output the execution amount of the target vehicle after executing the control amount according to the input control amount. The execution amount represents the state change of the target vehicle after the control amount is executed. The execution amount may include the heading angle change, speed change, acceleration change, etc. of the target vehicle.

[0129] The execution effects of the same control quantity executed by the actuators of different vehicles may be different, that is, they are manifested as different execution quantities. For example, the steering actuators of vehicle A and vehicle B execute the same control quantity "steering wheel turns 72 degrees clockwise". After the steering actuator of vehicle A executes this control quantity, the heading angle of the vehicle actually turns 4.75 degrees to the right, that is, the execution quantity of the steering actuator of vehicle A is "heading angle turns right 4.75 degrees"; after the steering actuator of vehicle B executes this control quantity, the heading angle of the vehicle actually turns right 4.8 degrees, that is, the execution quantity of the steering actuator of vehicle B is "heading angle turns right 4.8 degrees".

[0130] The current execution amount is a state change of the target vehicle after the actuator in the target vehicle executes the first control amount at the current moment, predicted by the target execution model.

[0131] The first electronic device may use the target execution model to predict a current execution amount of the first control amount executed by the actuator in the target vehicle.

[0132] Step S204: updating the simulated vehicle trajectory information of the target vehicle according to the current execution amount.

[0133] The simulated vehicle trajectory is the trajectory of the target vehicle in the simulation environment model. The simulated vehicle trajectory information includes the curvature, speed, acceleration, position and heading angle of the target vehicle in the simulation environment model.

[0134] The first electronic device can calculate the speed, position, heading angle and other information of the target vehicle at the next moment based on the current execution amount. For example, the heading angle of the target vehicle at the current moment is 0 degrees, and the first electronic device can calculate the heading angle of the target vehicle at the next moment, that is, 1 second later, to be 4.75 degrees based on the predicted steering execution amount "turn right 4.75 degrees". For another example, the speed of the target vehicle at the current moment is 50 kilometers per hour, and the acceleration is 0 meters per second squared, then the first electronic device can calculate the speed of the target vehicle after 1 second to be 50 kilometers per hour-6 meters per second squared × 1 second = 21.6 kilometers per hour based on the predicted braking execution amount "-6 meters per second squared". The first electronic device can also calculate the position of the target vehicle after 1 second based on the above-mentioned speed, acceleration, heading angle and other information, combined with the position of the target vehicle at the current moment. By analogy, the first electronic device can obtain the speed, acceleration, position and heading angle of the target vehicle at multiple moments after the current moment, and then generate the driving trajectory of the target vehicle based on the position of the target vehicle at these multiple moments, and then obtain the curvature information of the driving trajectory of the target vehicle.

[0135] Afterwards, the first electronic device may update the simulated vehicle trajectory information of the target vehicle in the simulated environment model based on the obtained information.

[0136] Step S205, adjusting the model parameters of the basic control model according to the deviation between the simulated vehicle trajectory information and the first expected trajectory information, to obtain a target control model of the controller in the target vehicle.

[0137] The first electronic device may first calculate the error value between the simulated vehicle trajectory information and each piece of information in the first expected trajectory information, and then calculate the average value or weighted average value between the error values ​​of each piece of information, and use the obtained average value or weighted average value as the deviation between the simulated vehicle trajectory information and the first expected trajectory information. The above-mentioned information may include the lateral position, longitudinal position and heading angle of the target vehicle. For the above-mentioned information, the first electronic device may respectively calculate the difference between the information in the simulated vehicle trajectory information and the information in the first expected trajectory information as the error value, and then calculate the average value or weighted average value between the error values ​​of each piece of information, and use the obtained average value or weighted average value as the deviation between the simulated vehicle trajectory information and the first expected trajectory information.

[0138] After obtaining the deviation between the simulated vehicle trajectory information and the first expected trajectory information, the first electronic device can adjust the model parameters of the basic control model according to the deviation between the simulated vehicle trajectory information and the first expected trajectory information to minimize the deviation. When the deviation between the simulated vehicle trajectory information and the first expected trajectory information is less than a set threshold, the first electronic device can use the adjusted basic control model as the target control model of the controller in the target vehicle.

[0139] In the technical solution provided by the embodiment of the present application, the basic control model of the controller with a relatively mature control strategy in the base vehicle is first migrated to the target vehicle as the basic control model of the controller in the target vehicle; then the target execution model reflecting the execution effect of the actuator in the target vehicle is used to predict the execution result according to the first control quantity output by the basic control model; then the model parameters of the basic control model are adjusted according to the predicted execution result, so that the adjusted basic control model matches the execution effect of the target vehicle; then the adjusted basic control model is used as the target control model of the target vehicle controller, so that the controller of the target vehicle is matched with the actuator. In this way, there is no need to train the control model of the target vehicle controller from scratch, which speeds up the convergence speed of the target control model training, thereby improving the efficiency of matching the controller with the actuator.

[0140] In addition, the above scheme uses simulated vehicle state information and target execution model to perform simulation training of the vehicle control model. Compared with real vehicle training, simulation training is more efficient in obtaining training data, the training process is safer, and no human intervention is required, which reduces the cost and safety cost required for vehicle control model training and further improves the efficiency of model training, thereby further improving the efficiency of matching the controller and the actuator.

[0141] For the current simulated vehicle status information in the above step S201, the embodiment of the present application also provides two methods of obtaining it: method 1 and method 2.

[0142] Method 1: Based on the last execution amount, the vehicle posture at the last moment is updated to obtain the vehicle posture, and the simulated vehicle state of the target vehicle after the actuator executes the last execution amount predicted by the target execution model is obtained.

[0143] Among them, the last execution amount is: the execution amount predicted by the target execution model based on the first control amount at the last moment, the historical vehicle state information and the historical execution amount. The vehicle posture includes the position, heading angle and other information of the target vehicle. The first electronic device can update the position, heading angle and other information of the target vehicle at the last moment based on the last execution amount to obtain the position, heading angle and other information of the target vehicle at the current moment. For example, the heading angle at the last moment is 0 degrees, and the last execution amount is "the heading angle turns right 5 degrees", then the updated heading angle of the target vehicle is 5 degrees.

[0144] The first electronic device may also obtain the speed, acceleration, etc. of the target vehicle at the current moment that are further predicted by the target execution model based on the previous execution amount.

[0145] Method 2: The simulated vehicle state information is obtained by updating the simulated vehicle state information at the previous moment based on the previous execution amount.

[0146] The first electronic device can also directly calculate the speed, acceleration, position, heading angle and other information of the target vehicle at the current moment based on the last execution amount. For example, the last execution amount includes the acceleration "-6 meters per second squared", and the speed of the target vehicle at the last moment, that is, 1 second ago, was 50 kilometers per hour. Then the first electronic device can calculate the speed of the target vehicle at the current moment based on the predicted last execution amount "-6 meters per second squared" as 50 kilometers per hour - 6 meters per second squared × 1 second = 21.6 kilometers per hour.

[0147] In the above-mentioned method 1 or method 2, the current simulated vehicle state information is obtained based on the last execution amount, and the real-time changes of the target vehicle in the simulation environment are taken into consideration, so that the current simulated vehicle state information obtained is more accurate.

[0148] For the above step S205, the present application embodiment also provides the following implementation method, such as Figure 3 A detailed flow chart of step S205 is shown, and the implementation method may include the following steps S301-S303.

[0149] Step S301, calculating the deviation between the simulated vehicle trajectory information and the first expected trajectory information.

[0150] Specifically, the calculation may be performed in the manner of calculating the deviation between the simulated vehicle trajectory information and the first expected trajectory information in step S205. For details, refer to the relevant description of step S205, which will not be repeated here.

[0151] Step S302: obtaining a first value return value corresponding to the first control amount based on the deviation.

[0152] The first value return value represents: the accuracy of using the first control amount to control the actuator in the target vehicle at the next moment. The larger the first value return value, the higher the accuracy of using the first control amount to control the actuator in the target vehicle at the next moment; the smaller the first value return value, the lower the accuracy of using the first control amount to control the actuator in the target vehicle at the next moment.

[0153] The first electronic device can calculate the deviation rate based on any error value included in the deviation, and then obtain the first value return value. The above error values ​​can include the lateral error value of the target vehicle position, the longitudinal error value of the position, the error value of the heading angle, etc. For any of the above error values, the first electronic device can calculate the ratio of the error value to the expected change value of the information corresponding to the error value, use the ratio as the deviation rate of the information corresponding to the error value, and then calculate the value of 1-deviation rate as the first value return value.

[0154] Specifically, the above error value can be the error value of the heading angle. The first electronic device can calculate the ratio of the error value of the heading angle to the expected change value of the heading angle, use the ratio as the deviation rate of the heading angle, and then calculate the value of 1-deviation rate as the first value return value.

[0155] For example, the heading angle of the target vehicle at the current moment is 0 degrees, and the heading angle at the next moment in the first expected trajectory information, that is, 1 second after the current moment, is 5 degrees, that is, the expected change value of the heading angle is 5 degrees; the heading angle of the target vehicle 1 second after the current moment in the simulated vehicle trajectory information is 4.75 degrees, that is, the error value of the heading angle is 5 degrees-4.75 degrees=0.25 degrees. Then the first electronic device can calculate the ratio of the error value of the heading angle to the expected change value of the heading angle, and obtain the deviation rate of the heading angle corresponding to the first control amount as 0.25 degrees / 5 degrees=5%, and then calculate the first value return value as 1-5%=95%.

[0156] The first electronic device can also calculate the comprehensive deviation rate based on the multiple error values ​​included in the deviation, and then obtain the first value return value. The above error values ​​may include the lateral error value of the target vehicle position, the longitudinal error value of the position, the error value of the heading angle, etc. The first electronic device can select multiple error values ​​from them, first calculate the ratio of the selected error values ​​to the expected change value of the information corresponding to the error value, and use the ratio corresponding to each error value as the deviation rate of the information corresponding to the error value, and then calculate the average value or weighted average value between the deviation rates corresponding to each piece of information as the comprehensive deviation rate, and then calculate the value of 1-comprehensive deviation rate as the first value return value.

[0157] Specifically, the first electronic device may select the lateral error value and the longitudinal error value of the target vehicle position to calculate the first value return value. The first electronic device may first calculate the ratio of the lateral error value of the target vehicle position to the lateral expected change value of the position and the ratio of the longitudinal error value to the longitudinal expected change value of the position, respectively, to obtain the lateral deviation rate of the position and the longitudinal deviation rate of the position; then calculate the average or weighted average value between the two deviation rates as the comprehensive deviation rate, and then calculate the value of 1-comprehensive deviation rate as the first value return value.

[0158] For example, the lateral distance between the position of the target vehicle at the next moment in the first expected trajectory information and the position of the target vehicle at the current moment is 0.5 meters, that is, the lateral expected change value of the position is 0.5 meters, and the longitudinal distance between the position of the target vehicle at the next moment in the first expected trajectory information and the position of the target vehicle at the current moment is 10 meters, that is, the longitudinal expected change value of the position is 10 meters; the lateral distance between the position of the target vehicle at the next moment in the simulated vehicle trajectory information and the position of the target vehicle at the current moment is 0.49 meters, and the longitudinal distance between the position of the target vehicle at the current moment is 9.45 meters, that is, the lateral error value of the position is 0.5 meters-0.49 meters=0.01 meters, and the longitudinal error value is 10 meters-9.45 meters=0.55 meters. Then the first electronic device can calculate the ratio of the lateral error value to the lateral expected change value of the position, and obtain the lateral deviation rate of the position is 0.01 meters / 0.5 meters=2%. The first electronic device can also calculate the ratio of the longitudinal error value to the longitudinal expected change value of the position, and obtain the longitudinal deviation rate of the position is 0.55 meters / 10 meters=5.5%. The first electronic device can calculate the average of the lateral deviation rate and the longitudinal deviation rate of the position, and obtain a comprehensive deviation rate of (2%+5.5%) / 2=3.75%. The first electronic device can then calculate a first value return value of 1-3.75%=96.25%.

[0159] The first electronic device can also calculate the first value return value using a preset value function. The value function can be set according to actual needs and is not limited here.

[0160] Step S303: adjusting the model parameters of the basic control model based on the first value return value to obtain a target control model of the controller in the target vehicle.

[0161] The first electronic device can adjust the model parameters of the basic control model based on the first value return value to maximize the first value return value. When the first value return value is greater than a set threshold, the first electronic device can use the adjusted basic control model as the target control model of the controller in the target vehicle.

[0162] In the technical solution provided by the embodiment of the present application, a first value return value is obtained based on the simulated vehicle trajectory information and the first expected trajectory information, and then the model parameters of the basic control model are adjusted based on the first value return value, which can more accurately measure the quality of the control quantity output by the basic control model, and then update the control strategy of the basic control model. In this way, the matching degree between the updated target control model and the target vehicle actuator is improved.

[0163] In a possible implementation, the simulated vehicle trajectory information may include: trajectory information of the target vehicle at each moment in a first time period, wherein the first time period is: a time period of a preset time period from the current moment backward. The preset time period may be set in combination with the characteristics of the actual control task so that the basic control model can converge better.

[0164] In this case, the first electronic device can Figure 4 In the implementation shown, the model parameters of the basic control model are adjusted to obtain the target control model of the controller in the target vehicle, that is, the above step S303 may include the following steps S401-S403.

[0165] Step S401, obtaining a second value return value corresponding to the first control amount based on the deviation.

[0166] Among them, the second value return value represents: the degree of matching of the first vehicle trajectory information relative to the first expected trajectory information when the first control amount is used to control the actuator in the target vehicle, such as the accuracy of the vehicle position at each moment in the first vehicle trajectory information relative to the vehicle position in the first expected trajectory information, and the stability of the vehicle driving trajectory in the first vehicle trajectory information. The first vehicle trajectory information is: the simulated vehicle trajectory information of the target vehicle at each moment except the current moment and the next moment in the first time period. The larger the second value return value, the higher the degree of matching of the first vehicle trajectory information relative to the first expected trajectory information when the first control amount is used to control the actuator in the target vehicle; the smaller the second value return value, the lower the degree of matching of the first vehicle trajectory information relative to the first expected trajectory information when the first control amount is used to control the actuator in the target vehicle.

[0167] Similar to the method of obtaining the first value return value, for each moment in the first time period except the current moment and the next moment, the first electronic device can calculate the deviation rate based on any error value included in the deviation between the simulated vehicle trajectory information at that moment and the first expected trajectory information, and then obtain the second value return value corresponding to that moment; or, the first electronic device can calculate the comprehensive deviation rate based on multiple error values ​​included in the deviation at that moment, and then obtain the second value return value at that moment. For details, please refer to the above description of step S202, which will not be repeated here.

[0168] Step S402, generating a cumulative value return value of the first value return value and the second value return value.

[0169] The first electronic device may calculate a sum or a weighted sum of the first value return value and each second value return value as a cumulative value return value.

[0170] In a possible implementation manner, the first electronic device may also generate a cumulative value return value according to the following expression:

[0171] R t =r t+1 +γr t+2 +…+γ n-1 r t+n ;

[0172] Among them, R t is the cumulative value return value, t is the current time, γ is the set attenuation coefficient, n is the preset time length, r t+1 is the first value return value, r t+2 ,……,r t+n is the second value return value. γ is between 0 and 1 and can be preset according to actual conditions.

[0173] In this way, the accuracy of controlling the actuator in the target vehicle based on the first control amount over a period of time can be observed, and the meaningless return value introduced over a long period of time is prevented from affecting the accuracy of the cumulative value return value.

[0174] Step S403, adjusting the model parameters of the basic control model based on the accumulated value return value to obtain a target control model of the controller in the target vehicle.

[0175] The first electronic device can adjust the model parameters of the basic control model based on the cumulative value return value to maximize the cumulative value return value. When the cumulative value return value is greater than a set threshold, the first electronic device can use the adjusted basic control model as the target control model of the controller in the target vehicle.

[0176] In the technical solution provided by the embodiment of the present application, the cumulative value return value is obtained based on the deviation between the simulated vehicle trajectory information and the first expected trajectory information within a period of time, and then the model parameters of the basic control model are adjusted based on the cumulative value return value, taking into account the influence of the first control amount on the trajectory of the target vehicle within a period of time, and the quality of the control amount output by the basic control model can be more accurately measured, and then the control strategy of the basic control model can be updated. In this way, the matching degree between the updated target control model and the actuator of the target vehicle can be further improved.

[0177] The present application embodiment also provides a training method for the basic control model used in the above step S202, such as Figure 5 As shown, the above basic control model can be trained according to the following steps S501-S503.

[0178] The training method of the basic control model provided in the embodiment of the present application can be applied to electronic devices such as desktop computers, laptop computers or servers. For ease of description, the execution subject of the training method of the basic control model provided in the embodiment of the present application is collectively referred to as the second electronic device. The second electronic device and the first electronic device can be the same electronic device or different electronic devices.

[0179] Step S501, for each sample moment, obtain sample expected trajectory information of the basic vehicle, sample vehicle state information and the actual control amount output by the controller in the basic vehicle.

[0180] Here, the sample time for training the basic control model can be the time of the historical driving of the basic vehicle. The sample expected trajectory information is the information of the trajectory expected to be driven by the basic vehicle after the sample time. The sample expected trajectory information includes the curvature of the expected driving trajectory of the basic vehicle and the speed, acceleration, position and heading angle of the basic vehicle. The sample vehicle state information includes the speed, acceleration, position and heading angle of the basic vehicle at the sample time.

[0181] The second electronic device can obtain sample expected trajectory information at historical driving moments, sample vehicle state information, and actual control amounts output by a controller in the base vehicle at historical driving moments from the base vehicle.

[0182] Step S502: input the sample expected trajectory information and the sample vehicle state information at each sample time into the basic control model respectively, and obtain the second control amount of the actuator corresponding to each sample time output by the basic control model.

[0183] The second electronic device can input the sample expected trajectory information and sample vehicle state information of the basic vehicle at each sample moment into the basic control model to obtain the second control amount of the actuator corresponding to the moment output by the basic control model.

[0184] Step S503: adjusting the model parameters of the basic control model according to the difference between the second control amount corresponding to each sample moment and the actual control amount.

[0185] The difference between the second control amount and the actual control amount may be a difference between the second control amount and the actual control amount. The second electronic device may adjust the model parameters of the basic control model based on the difference between the second control amount and the actual control amount corresponding to each sample moment, so as to minimize the difference between the second control amount and the actual control amount at each sample moment output by the basic control model, thereby enabling the basic control model to learn the control strategy of the controller in the basic vehicle.

[0186] In a possible implementation manner, the second electronic device may calculate the model parameters after the basic control model is adjusted according to the following expression:

[0187] θ * =argmin θ {E (T,S,a) [Loss(π θ (T,S),a)]};

[0188] Among them, θ * is the adjusted model parameter, θ is the model parameter of the basic control model, T is the sample expected trajectory information, S is the sample vehicle state information, a is the actual control amount at the sample time; argmin{} represents the value of θ when the expression about θ in the brackets takes the minimum value, E (T,S,a) is the mean square error between the second control quantity and the actual control quantity, Loss() is the loss function, π θ represents the control strategy of the basic control model, π θ (T, S) is the second control quantity output by the basic control model based on the input T and S, Loss (π θ (T, S), a) represents the difference between the second control quantity and the actual control quantity. The loss function can be selected according to the characteristics of the actual actuator interface. When the second control quantity is a discrete value, the cross entropy function or the likelihood function can be used. When the second control quantity is a continuous value, the mean square error can be used.

[0189] In this way, by combining the mean square error and loss function between the second control quantity and the actual control quantity, the difference between the second control quantity and the actual control quantity can be measured more accurately, and then the adjusted model parameters of the basic control model can be calculated more accurately, thereby accelerating the training process of the basic control model.

[0190] In order to better explain the above Figure 5 The training method of the basic control model provided, the embodiment of the present application also provides the following Figure 6 A framework diagram for basic control model training is shown.

[0191] Figure 6In the example, the second electronic device may first obtain a basic vehicle data set from the basic vehicle, and the basic vehicle data set includes: a trajectory posture sequence, a target motion state, an ego vehicle state sequence, and an actual control amount of the vehicle. The trajectory posture information and the target motion state are sample expected trajectory information, and the ego vehicle state sequence is sample vehicle state information. The above-mentioned trajectory posture information, target motion state, and ego vehicle state sequence may be collectively referred to as a trajectory and state sequence fragment.

[0192] The sample expected trajectory information, sample vehicle state information and actual control amount at each sample time can form a triple. The sample expected trajectory information and sample vehicle state information in the triple are used as the input of the basic control model, and the actual control amount is used as the supervision value of the basic control model. The triplets of multiple sample times can form a data set D base :

[0193] D base ={(T 1 ,S 1 ,a 1 ),(T 2 ,S 2 ,a 2 ),……,(T i ,S i ,a i ),……,(T n ,S n ,a n );

[0194] Among them, T i represents the expected trajectory information of the sample at time i, S i represents the sample vehicle state at time i, a i Represents the actual control quantity at time i.

[0195] The second electronic device inputs the trajectory posture sequence, target motion state and self-vehicle state sequence of the vehicle into the basic control model, and then obtains the second control quantity such as the steering wheel angle request value, the longitudinal acceleration request value and the power torque request value output by the basic control model. The second electronic device uses the actual control quantity as the supervision value, and updates the model parameters of the basic control model according to the difference between the second control quantity and the actual control quantity.

[0196] In the technical solution provided in the embodiment of the present application, the basic control model is trained according to the sample data of the actual driving of the basic vehicle, so that the obtained basic control model can better learn the control strategy of the controller in the basic vehicle.

[0197] The present application embodiment also provides a training method for the target execution model used in the above step S203, such as Figure 7As shown, the above target execution model can be trained according to the following steps S701-S703.

[0198] The training method of the target execution model provided in the embodiment of the present application can be applied to electronic devices such as desktop computers, laptop computers or servers. For ease of description, the execution subject of the training method of the target execution model provided in the embodiment of the present application is collectively referred to as the third electronic device. The first electronic device, the second electronic device and the third electronic device can be the same electronic device or different electronic devices.

[0199] Step S701, for each sample moment, obtain sample vehicle state information of the target vehicle, actual control amount and actual execution amount of the actuator input in the target vehicle.

[0200] Here, the sample time for training the target execution model can be the time when the target vehicle has traveled in history. The sample vehicle state information includes the speed, acceleration, position, and heading angle of the target vehicle at the sample time. The actual execution amount is the actual state change of the target vehicle after the actuator in the target vehicle executes according to the input actual control amount at the sample time.

[0201] The third electronic device may obtain the actual control amount and the actual execution amount of the actuator input at the historical driving time from the target vehicle.

[0202] Step S702, input the sample vehicle state information, actual control amount, actual execution amount and actual control amount at each sample moment corresponding to the historical moment of each sample moment into the target execution model, and obtain the first vehicle state information and first execution amount corresponding to each sample moment output by the target execution model.

[0203] The historical moments of the sample moment are multiple moments before the sample moment. The first vehicle state information is the vehicle state information of the target vehicle at the next moment of the sample moment predicted by the target execution model. The first execution amount is the state change of the target vehicle from the sample moment to the next moment of the sample moment after the actuator in the target vehicle executes the actual control amount at the sample moment predicted by the target execution model.

[0204] The third electronic device can input the sample vehicle state information, actual control amount, actual execution amount and actual control amount of the target vehicle at each sample moment into the target execution model to obtain the first vehicle state information and first execution amount corresponding to the moment output by the target execution model.

[0205] Step S703, adjust the model parameters of the target execution model according to the difference between the first vehicle state information corresponding to each sample moment and the sample vehicle state information corresponding to each sample moment, and the difference between the first execution amount corresponding to each sample moment and the actual execution amount corresponding to each sample moment.

[0206] The third electronic device may first calculate the error value between the first vehicle state information corresponding to each sample moment and each piece of information in the sample vehicle state information, and then calculate the average value or weighted average value between the error values ​​of each piece of information, and use the obtained average value or weighted average value as the difference between the first vehicle state information and the sample vehicle state information. The above-mentioned pieces of information may include speed, acceleration, lateral position, or longitudinal position, heading angle, etc. For the above-mentioned pieces of information, the third electronic device may calculate the difference between the information in the first vehicle state information and the information in the sample vehicle state information as the error value, and then calculate the average value or weighted average value between the error values ​​of each piece of information, and use the obtained average value or weighted average value as the difference value between the first vehicle state information and the sample vehicle state information.

[0207] The third electronic device may further calculate a difference between the first execution amount corresponding to each sample moment and the actual execution amount corresponding to the sample moment as a difference value between the first execution amount and the actual execution amount.

[0208] Based on the above two difference values, the third electronic device can calculate the difference value between the first vehicle state information and the sample vehicle state information, and the sum or weighted sum of the difference values ​​between the first execution amount and the actual execution amount corresponding to the sample moment, and use the obtained sum or weighted sum as the total difference value, and adjust the model parameters of the target execution model according to the total difference value to minimize the obtained total difference value, so that the target execution model learns the execution effect of the actuator in the target vehicle.

[0209] In order to better explain the above Figure 7 The training method of the target execution model provided, the embodiment of the present application also provides the following Figure 8 A framework diagram of a target execution model training is shown.

[0210] Figure 8In the process, the third electronic device can first obtain a target vehicle time series data set from the target vehicle, and the target vehicle time series data set includes: vehicle motion state (i.e., sample vehicle state information), execution feedback (i.e., actual execution amount), and control amount (i.e., actual control amount). The historical vehicle motion state sequence, control amount, historical execution feedback sequence, and actual control amount at each sample moment corresponding to the historical moment of the sample moment are used as inputs of the actuator response prediction model (i.e., target execution model), wherein the control amount corresponding to the historical moment and the control amount corresponding to the sample moment can be input as a control amount sequence, and the sample vehicle state information at the sample moment and the actual execution amount are used as supervision values ​​of the target execution model.

[0211] For each sample time t, the input and output of the target execution model can form a multi-tuple in, Represents the sample vehicle status information from time tm to time t-1, including V t-m ,……,V t-3 、V t-2 、V t-1 , Indicates the actual execution amount from time tm to time t-1, including F t-m ,……,F t-3 、F t-2 、F t-1 , Indicates the actual control quantity from time tm to time t, including C t-m ,……,C t-3 , C t-2 , C t-1 , C t , V t ′ represents the first vehicle status information at time t, F t ′ represents the first execution amount at time t, is the input of the target execution model, V t ′,F t ′ is the output of the target execution model. In addition, corresponding to the input and output in the above multi-tuple at time t, the supervision value at time t can be recorded as V t ,F t , where V t is the sample vehicle status information at time t, F t is the actual execution amount at time t. The actual control amount at time t can be recorded as C t .

[0212] The above multi-tuples corresponding to each sample moment can constitute a data set D response :

[0213]

[0214] in, Indicates the actual execution amount from time t-m+n to time t-1+n. represents the actual control quantity from time t-m+n to time t+n, V′ t+n represents the first vehicle status information at time t+n, F′ t+n Indicates the first execution amount at time t+n.

[0215] The third electronic device inputs the historical vehicle motion state sequence, historical execution feedback sequence, and control amount sequence at the sample time into the actuator response prediction model, that is, the target execution model, and then obtains the real-time execution feedback output by the actuator response prediction model, that is, the first execution amount, and the real-time vehicle motion state, that is, the first vehicle state information. The third electronic device uses the actual execution amount and the sample vehicle state information at the sample time as supervision values, and updates the model parameters of the actuator response prediction model according to the difference between the real-time vehicle motion state and the sample vehicle state information, and the difference between the first execution amount and the actual execution amount.

[0216] In the technical solution provided by the embodiment of the present application, a small amount of sample data of the actual driving of the target vehicle is used to train the target execution model, so that the obtained target execution model can better characterize the execution effect of the actuator in the target vehicle. When the basic control model is subsequently adjusted to obtain the target control model, the target execution model can be used to replace the target vehicle for simulation training, which reduces the cost of real vehicle training to a certain extent and improves the safety of subsequent real vehicle training.

[0217] Above Figure 2-Figure 4 The vehicle control model training method provided is based on simulation training. In order to better illustrate the above vehicle control model training method, the embodiment of the present application also provides the following Fig. 9 A diagram of the interactions in one example is shown.

[0218] Fig. 9 In the process, the first electronic device first obtains the vehicle state from the simulated environment model, that is, the current simulated vehicle state information, and obtains the expected trajectory from the decision planning model, that is, the first expected trajectory information, as in the above step S201. The decision planning model also relies on the perception information in the environment model to generate the expected trajectory.

[0219] The first electronic device then inputs the acquired vehicle state and expected trajectory into the target control model to obtain a first control variable output by the target control model, as described in step S202 above.

[0220] The first electronic device inputs the first control amount, the historical vehicle state information obtained from the environment model, and the historical execution amount output by the target execution model into the target execution model to obtain the current execution amount predicted by the target execution model, as in step S203 above. The first electronic device can also obtain an updated vehicle state based on the current execution amount predicted by the target execution model, and synchronize it to the target control model.

[0221] Then, the first electronic device can update the positioning of the target vehicle based on the current execution amount, that is, update the simulated vehicle trajectory information such as the position and posture information of the target vehicle, and synchronize the simulated vehicle trajectory information such as the position and posture information to the environment model, as described in step S204. The first electronic device can also convert the position and posture information into a vehicle state and synchronize it to the target control model.

[0222] Then, the first electronic device can perform error evaluation according to the deviation between the simulated vehicle trajectory information and the first expected trajectory information, including lateral error, longitudinal error and heading angle error, etc., and then calculate the value return value based on these errors, and adjust the model parameters of the basic control model based on the value return value to obtain the target control model of the controller in the target vehicle, such as the above steps S205, steps S301-S303, and steps S401-S403.

[0223] In accordance with the above Figure 2-Figure 4 After the target control model is obtained by simulation training using the vehicle control model training method, the embodiment of the present application further provides a vehicle control model training method based on a real vehicle, and the target control model obtained above is fine-tuned on the target vehicle, such as Fig.10 As shown, the method includes the following steps S1001-S1004.

[0224] Step S1001, obtaining the second expected trajectory information and the current actual vehicle state information of the target vehicle at the current moment.

[0225] The second expected trajectory information is the trajectory that the target vehicle is expected to travel after the current moment. The second expected trajectory information includes the curvature of the target vehicle's travel trajectory and information such as the speed, acceleration, position, and heading angle of the target vehicle. The current actual vehicle state information is the actual vehicle state information of the target vehicle at the current moment. The current actual vehicle state information includes information such as the speed, acceleration, position, and heading angle of the target vehicle at the current moment. The first electronic device can obtain the current actual vehicle state information of the target vehicle from the target vehicle.

[0226] Step S1002: input the second expected trajectory information and the current actual vehicle state information into the target control model to obtain a third control variable of the actuator output by the target control model.

[0227] The first electronic device may input the second expected trajectory information and the current actual vehicle state information into a target control model of a controller in the target vehicle to obtain a third control amount of the actuator output by the target control model.

[0228] Step S1003, obtaining actual vehicle trajectory information of the target vehicle after the actuator of the target vehicle performs an operation under the third control amount.

[0229] The actual vehicle trajectory information includes the curvature of the actual driving trajectory of the target vehicle after the actuator of the target vehicle performs an operation under the third control amount, and information such as the speed, acceleration, position and heading angle of the target vehicle.

[0230] After the actuator of the target vehicle executes the third control quantity, the state and position of the target vehicle change, and the first electronic device can obtain information such as the speed, acceleration, position, and heading angle of the target vehicle from the target vehicle, and then determine the actual driving trajectory of the target vehicle and the curvature of the trajectory based on this information.

[0231] Step S1004: adjusting the model parameters of the target control model according to the deviation between the actual vehicle trajectory information and the second expected trajectory information.

[0232] The first electronic device can calculate the error value of each piece of information in the actual vehicle trajectory information and the second expected trajectory information, such as the lateral error, longitudinal error, heading angle error, etc. of the position, and obtain the deviation between the actual vehicle trajectory information and the second expected trajectory information by combining the error values ​​of each piece of information. Then, the model parameters of the target control model are adjusted according to the deviation between the actual vehicle trajectory information and the second expected trajectory information, so that the deviation between the actual vehicle trajectory information and the second expected trajectory information is less than a set threshold, and the adjusted target control model can be used as the working model of the controller in the target vehicle. The first electronic device can also calculate value return values ​​based on the above deviations, such as the above first value return value, second value return value and cumulative value return value. For details, please refer to the above Figure 3 and Figure 4 The relevant description in will not be repeated here.

[0233] In order to better illustrate the above-mentioned vehicle control model training method based on a real vehicle, the present application embodiment also provides the following Fig.11 A diagram of the interactions in one example is shown.

[0234] Fig.11 In step S1001, the first electronic device obtains the vehicle state, that is, the current actual vehicle state information, from the target vehicle, and obtains the expected trajectory, that is, the second expected trajectory information, from the decision planning model, as described in step S1001.

[0235] The decision planning model generates the expected trajectory according to the perception information obtained from the actual environment. The first electronic device then inputs the acquired vehicle state and the expected trajectory into the target control model to obtain the third control amount output by the target control model, as described in step S1002 above.

[0236] After the actuator of the target vehicle performs an operation under the third control amount, the first electronic device can obtain actual vehicle trajectory information such as the position and posture information of the target vehicle after the actuator of the target vehicle performs an operation under the third control amount based on the detected positioning update of the target vehicle, as described in the above step S1003.

[0237] Then, the first electronic device can perform error evaluation based on the deviation between the actual vehicle trajectory information of the target vehicle and the second expected trajectory information, including lateral error, longitudinal error and heading angle error, etc., and then calculate the value return value based on these error values, and adjust the model parameters of the target control model based on the value return value to obtain the target control model of the controller in the target vehicle, such as the above step S1004.

[0238] In the technical solution provided in the embodiment of the present application, after obtaining the target control model, the target control model is fine-tuned based on the actual vehicle trajectory information after the target vehicle executes the third control amount output by the target control model, thereby further improving the matching degree between the target control model and the target vehicle actuator.

[0239] In combination with the above-mentioned simulation training and real vehicle training, the present application embodiment also provides the following Fig.12 A flowchart of a complete example is shown.

[0240] Fig.12 In the paper, the vehicle control model training is divided into three stages, namely stage 1, basic control model training; stage 2, target vehicle actuator modeling; stage 3, target control model training.

[0241] In the first stage, the second electronic device collects control data from the base vehicle, uses the control data to train the base control model, and imitates the control strategy of the controller in the base vehicle, as described above. Figure 5 The training method of the basic control model described in the present invention includes the sample expected trajectory information at the sample time, the sample vehicle state information and the actual control quantity output by the controller in the basic vehicle.

[0242] In the second stage, the third electronic device collects the actuator response data in the target vehicle, and uses the actuator response data to establish an actuator response characteristic prediction model for the actuator of the target vehicle, that is, the above-mentioned target execution model, as shown in the above Figure 7The training method of the target execution model described herein, wherein the actuator response data includes sample vehicle state information of the target vehicle, actual control amount and actual execution amount of the actuator input in the target vehicle, etc.

[0243] In the third stage, the first electronic device first uses the actuator response prediction model for simulation training, and performs reinforcement learning based on the basic control model to obtain the target control model, as shown above. Figure 2 The vehicle control model training method described above. Then, the first electronic device performs real vehicle closed-loop online training on the target vehicle to further strengthen the learning and obtain the adjusted target control model. Fig.10 The described vehicle control model training method based on a real vehicle.

[0244] After obtaining the target control model according to any of the above-mentioned vehicle control model training methods, the target control model can be deployed to the controller of the target vehicle as a working model of the target vehicle controller to control the vehicle to achieve autonomous driving.

[0245] Specifically, a decision planning model and a target control model may be deployed in the controller of the target vehicle.

[0246] When the target vehicle is autonomously driving, the controller can use the decision planning model deployed therein to plan the vehicle's driving trajectory. Specifically, the decision planning model can plan the vehicle's driving trajectory in combination with the current vehicle state information of the target vehicle and the actual road condition information. The actual road condition information can be obtained by scanning the sensors, radars, cameras, etc. deployed on the target vehicle. The decision planning model plans the expected trajectory information of the target vehicle according to the above information, and outputs the expected trajectory information.

[0247] After obtaining the expected trajectory information output by the decision planning model, the controller inputs the obtained expected trajectory information and the current vehicle state information of the target vehicle into the target control model to obtain the corresponding control amount required to be executed by one or more actuators output by the target control model.

[0248] The controller then sends a control instruction carrying the corresponding control quantity to the corresponding actuator in the target vehicle according to the corresponding control quantity that one or more actuators need to execute output by the target control model, and controls the actuator to execute according to the control quantity in the instruction.

[0249] After the actuator executes the command issued by the controller, the vehicle state of the target vehicle changes and travels a certain trajectory.

[0250] The higher the matching degree between the target control model and the target vehicle actuator, the higher the matching degree between the trajectory traveled by the target vehicle after executing the instruction issued by the controller and the trajectory indicated in the expected trajectory information output by the decision planning model.

[0251] Corresponding to the above-mentioned vehicle control model training method, the embodiment of the present application also provides a vehicle control model training device, such as Fig.13 As shown, the device comprises:

[0252] The first obtaining module 1301 is used to obtain the first expected trajectory information of the target vehicle at the current moment and the current simulated vehicle state information;

[0253] The second obtaining module 1302 is used to input the first expected trajectory information and the current simulated vehicle state information into a basic control model of a controller in a basic vehicle, and obtain a first control amount of an actuator output by the basic control model, wherein the basic vehicle is: a vehicle in which the controller matches the actuator, the input data of the basic control model includes: the expected trajectory information and the vehicle state information of the basic vehicle, and the output data of the basic control model includes: the control amount that the actuator of the basic vehicle needs to execute when the basic vehicle travels according to the trajectory indicated in the expected trajectory information;

[0254] The prediction module 1303 is used to input the first control amount, the historical vehicle state information of the target vehicle and the historical execution amount output by the target execution model into the target execution model, and predict the current execution amount of the actuator in the target vehicle, wherein the target execution model is: a working model of the actuator in the target vehicle, and the target execution model outputs the execution amount of the target vehicle after executing the control amount according to the input control amount;

[0255] An updating module 1304, used to update the simulated vehicle trajectory information of the target vehicle according to the current execution amount;

[0256] The first adjustment module 1305 is used to adjust the model parameters of the basic control model according to the deviation between the simulated vehicle trajectory information and the first expected trajectory information to obtain a target control model of the controller in the target vehicle.

[0257] In the technical solution provided by the embodiment of the present application, the basic control model of the controller with a relatively mature control strategy in the base vehicle is first migrated to the target vehicle as the basic control model of the controller in the target vehicle; then the target execution model reflecting the execution effect of the actuator in the target vehicle is used to predict the execution result according to the first control quantity output by the basic control model; then the model parameters of the basic control model are adjusted according to the predicted execution result, so that the adjusted basic control model matches the execution effect of the target vehicle; then the adjusted basic control model is used as the target control model of the target vehicle controller, so that the controller of the target vehicle is matched with the actuator. In this way, there is no need to train the control model of the target vehicle controller from scratch, which speeds up the convergence speed of the target control model training, thereby improving the efficiency of matching the controller with the actuator.

[0258] In addition, the above scheme uses simulated vehicle state information and target execution model to perform simulation training of the vehicle control model. Compared with real vehicle training, simulation training is more efficient in obtaining training data, the training process is safer, and no human intervention is required, which reduces the cost and safety cost required for vehicle control model training and further improves the efficiency of model training, thereby further improving the efficiency of matching the controller and the actuator.

[0259] In a possible implementation manner, the first adjustment module 1305 may be specifically configured to:

[0260] Calculating a deviation between the simulated vehicle trajectory information and the first expected trajectory information;

[0261] Obtaining a first value return value corresponding to the first control amount based on the deviation, wherein the first value return value represents: the accuracy of controlling the actuator in the target vehicle using the first control amount at the next moment;

[0262] The model parameters of the basic control model are adjusted based on the first value return value to obtain a target control model of the controller in the target vehicle.

[0263] In a possible implementation, the simulated vehicle trajectory information may include: trajectory information of the target vehicle at each moment in a first time period, wherein the first time period is: a time period of a preset time period from the current moment backward;

[0264] In this case, the first adjustment module 1305 may be specifically used to:

[0265] A second value return value corresponding to the first control amount is obtained based on the deviation, wherein the second value return value represents: a matching degree of the first vehicle trajectory information relative to the first expected trajectory information when the first control amount is used to control the actuator in the target vehicle, and the first vehicle trajectory information is: simulated vehicle trajectory information of the target vehicle at each moment except the current moment and the next moment in the first time period;

[0266] Generate a cumulative value return value of the first value return value and the second value return value;

[0267] The model parameters of the basic control model are adjusted based on the accumulated value return value to obtain the target control model of the controller in the target vehicle.

[0268] In a possible implementation manner, the first adjustment module 1305 may be specifically configured to generate a cumulative value return value according to the following expression:

[0269] R t =r t+1 +γr t+2 +…+γ n-1 r t+n ;

[0270] Among them, R t is the cumulative value return value, t is the current time, γ is the set attenuation coefficient, n is the preset time length, r t+1 is the first value return value, r t+2 ,……,r t+n Return value for the second value.

[0271] In a possible implementation manner, the current simulated vehicle state information may include:

[0272] The vehicle posture obtained by updating the vehicle posture at the previous moment based on the previous execution amount, and the simulated vehicle state of the target vehicle after the actuator executes the previous execution amount, which is predicted by the target execution model, wherein the previous execution amount is: the execution amount predicted by the target execution model based on the first control amount at the previous moment, the historical vehicle state information and the historical execution amount; or

[0273] The current simulated vehicle state information may include: simulated vehicle state information obtained by updating the simulated vehicle state information at the last moment based on the last execution amount.

[0274] In a possible implementation manner, the above basic control model can be trained in the following manner:

[0275] For each sample moment, obtain sample expected trajectory information of the base vehicle, sample vehicle state information and actual control quantity output by the controller in the base vehicle;

[0276] Inputting sample expected trajectory information and sample vehicle state information at each sample time into the basic control model respectively, and obtaining a second control quantity of the actuator corresponding to each sample time output by the basic control model;

[0277] According to the difference between the second control amount corresponding to each sample moment and the actual control amount, the model parameters of the basic control model are adjusted.

[0278] In a possible implementation, the target execution model can be trained as follows:

[0279] For each sample moment, sample vehicle state information of the target vehicle, actual control amount and actual execution amount of the actuator input in the target vehicle are obtained;

[0280] Inputting the sample vehicle state information, the actual control amount, the actual execution amount, and the actual control amount at each sample moment corresponding to the historical moment of each sample moment into the target execution model respectively, and obtaining the first vehicle state information and the first execution amount corresponding to each sample moment output by the target execution model;

[0281] The model parameters of the target execution model are adjusted according to the difference between the first vehicle state information corresponding to each sample moment and the sample vehicle state information corresponding to each sample moment, and the difference between the first execution amount corresponding to each sample moment and the actual execution amount corresponding to each sample moment.

[0282] In a possible implementation manner, the vehicle control model training device may further include a second adjustment module, which is used to:

[0283] After obtaining the target control model, obtaining the second desired trajectory information and the current actual vehicle state information of the target vehicle at the current moment;

[0284] Inputting the second desired trajectory information and the current actual vehicle state information into the target control model to obtain a third control variable of the actuator output by the target control model;

[0285] obtaining actual vehicle trajectory information of the target vehicle after the actuator of the target vehicle performs the operation under the third control amount;

[0286] According to the deviation between the actual vehicle trajectory information and the second expected trajectory information, the model parameters of the target control model are adjusted.

[0287] In a possible implementation, the actuator may include at least one of the following actuators: a steering actuator, a power actuator, and a braking actuator; the target execution model may include at least one of the following models: a steering execution model, a power execution model, and a braking execution model.

[0288] The present application also provides an electronic device, such as the first electronic device described above. Fig.14 As shown, including:

[0289] Memory 1401, used for storing computer programs;

[0290] The processor 1402 is used to implement the steps of any of the above-mentioned vehicle control model training methods when executing the program stored in the memory 1401.

[0291] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 1402, the communication interface, and the memory 1401 communicate with each other via the communication bus.

[0292] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0293] The communication interface is used for communication between the above electronic device and other devices.

[0294] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0295] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0296] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned vehicle control model training methods are implemented.

[0297] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the steps of any vehicle control model training method in the above-mentioned embodiments.

[0298] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0299] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0300] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0301] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A vehicle control model training method, characterized in that: The method comprises: Obtaining first expected trajectory information of the target vehicle at the current moment and current simulated vehicle state information; Inputting the first expected trajectory information and the current simulated vehicle state information into a basic control model of a controller in a base vehicle, and obtaining a first control amount of an actuator output by the basic control model, wherein the base vehicle is: a vehicle in which a controller matches an actuator, the input data of the basic control model includes: the expected trajectory information and vehicle state information of the base vehicle, and the output data of the basic control model includes: the control amount that the actuator of the base vehicle needs to execute when the base vehicle travels according to the trajectory indicated in the expected trajectory information; The first control amount, the historical vehicle state information of the target vehicle and the historical execution amount output by the target execution model are input into the target execution model to predict the current execution amount of the actuator in the target vehicle, wherein the target execution model is: a working model of the actuator in the target vehicle, and the target execution model outputs the execution amount of the target vehicle after executing the control amount according to the input control amount; According to the current execution amount, updating the simulated vehicle trajectory information of the target vehicle; According to the deviation between the simulated vehicle trajectory information and the first expected trajectory information, the model parameters of the basic control model are adjusted to obtain a target control model of the controller in the target vehicle.

2. The method according to claim 1, characterized in that The step of adjusting the model parameters of the basic control model according to the deviation between the simulated vehicle trajectory information and the first expected trajectory information to obtain the target control model of the controller in the target vehicle includes: Calculating a deviation between the simulated vehicle trajectory information and the first expected trajectory information; Obtaining a first value return value corresponding to the first control amount based on the deviation, wherein the first value return value represents: the accuracy of using the first control amount to control the actuator in the target vehicle at the next moment; The model parameters of the basic control model are adjusted based on the first value return value to obtain a target control model of the controller in the target vehicle.

3. The method according to claim 2, characterized in that The simulated vehicle trajectory information includes: the trajectory information of the target vehicle at each moment in a first time period, wherein the first time period is: a time period of a preset time period from the current moment backward; The step of adjusting the model parameters of the basic control model based on the first value return value to obtain a target control model of the controller in the target vehicle includes: A second value return value corresponding to the first control amount is obtained based on the deviation, wherein the second value return value represents: a matching degree of the first vehicle trajectory information relative to the first expected trajectory information when the first control amount is used to control the actuator in the target vehicle, and the first vehicle trajectory information is: simulated vehicle trajectory information of the target vehicle at each moment except the current moment and the next moment within the first time period; Generate a cumulative value return value of the first value return value and the second value return value; The model parameters of the basic control model are adjusted based on the cumulative value return value to obtain a target control model of the controller in the target vehicle.

4. The method according to claim 3, characterized in that: The generating the cumulative value return value of the first value return value and the second value return value comprises: The cumulative value return value is generated according to the following expression: R t =r t+1 +γr t+2 +…+γ n-1 r t+n ; Among them, R t is the cumulative value return value, t is the current time, γ is the set attenuation coefficient, n is the preset time length, r t+1 is the first value return value, r t+2 ,……,r t+n Return value for the second value.

5. The method according to any one of claims 1 to 4, characterized in that The current simulated vehicle state information includes: a vehicle posture obtained by updating the vehicle posture at the previous moment based on the previous execution amount, and a simulated vehicle state of the target vehicle after the actuator executes the previous execution amount, which is predicted by the target execution model, wherein the previous execution amount is: an execution amount predicted by the target execution model based on the first control amount at the previous moment, historical vehicle state information, and historical execution amount; or The current simulated vehicle state information includes: simulated vehicle state information obtained by updating the simulated vehicle state information at the last moment based on the last execution amount.

6. The method according to any one of claims 1 to 4, characterized in that The basic control model is trained in the following manner: for each sample moment, sample expected trajectory information of the basic vehicle, sample vehicle state information and actual control amount output by a controller in the basic vehicle are obtained; Inputting sample expected trajectory information and sample vehicle state information at each sample moment into the basic control model respectively, obtaining a second control amount of the actuator corresponding to each sample moment output by the basic control model; adjusting the model parameters of the basic control model according to the difference between the second control amount corresponding to each sample moment and the actual control amount; and / or The target execution model is trained in the following manner: for each sample moment, the sample vehicle state information of the target vehicle, the actual control amount and the actual execution amount input by the actuator in the target vehicle are obtained; the sample vehicle state information, the actual control amount, the actual execution amount and the actual control amount at each sample moment corresponding to the historical moment of each sample moment are respectively input into the target execution model to obtain the first vehicle state information and the first execution amount corresponding to each sample moment output by the target execution model; according to the difference between the first vehicle state information corresponding to each sample moment and the sample vehicle state information corresponding to each sample moment, and the difference between the first execution amount corresponding to each sample moment and the actual execution amount corresponding to each sample moment, the model parameters of the target execution model are adjusted; and / or After obtaining the target control model, the method further includes: obtaining second desired trajectory information and current actual vehicle state information of the target vehicle at the current moment; Inputting the second expected trajectory information and the current actual vehicle state information into the target control model to obtain a third control amount of the actuator output by the target control model; obtaining the actual vehicle trajectory information of the target vehicle after the actuator of the target vehicle performs an operation under the third control amount; adjusting the model parameters of the target control model according to the deviation between the actual vehicle trajectory information and the second expected trajectory information; and / or The actuator includes at least one of the following actuators: a steering actuator, a power actuator, and a braking actuator; and / or the target execution model includes at least one of the following models: a steering execution model, a power execution model, and a braking execution model.

7. A vehicle control model training device, characterized in that: The device comprises: A first acquisition module is used to obtain the first expected trajectory information of the target vehicle at the current moment and the current simulated vehicle state information; a second obtaining module, for inputting the first expected trajectory information and the current simulated vehicle state information into a basic control model of a controller in a basic vehicle, and obtaining a first control amount of an actuator output by the basic control model, wherein the basic vehicle is a vehicle in which a controller matches an actuator, the input data of the basic control model includes the expected trajectory information and vehicle state information of the basic vehicle, and the output data of the basic control model includes the control amount that the actuator of the basic vehicle needs to execute when the basic vehicle travels along the trajectory indicated in the expected trajectory information; a prediction module, configured to input the first control amount, the historical vehicle state information of the target vehicle and the historical execution amount output by the target execution model into the target execution model, and predict the current execution amount of the actuator in the target vehicle, wherein the target execution model is: a working model of the actuator in the target vehicle, and the target execution model outputs the execution amount of the target vehicle after executing the control amount according to the input control amount; An updating module, used for updating the simulated vehicle trajectory information of the target vehicle according to the current execution amount; The first adjustment module is used to adjust the model parameters of the basic control model according to the deviation between the simulated vehicle trajectory information and the first expected trajectory information to obtain the target control model of the controller in the target vehicle.

8. The device according to claim 7, characterized in that The first adjustment module is specifically used to: calculate the deviation between the simulated vehicle trajectory information and the first expected trajectory information; obtain a first value return value corresponding to the first control amount based on the deviation, wherein the first value return value represents: the accuracy of using the first control amount to control the actuator in the target vehicle at the next moment; adjust the model parameters of the basic control model based on the first value return value to obtain a target control model of the controller in the target vehicle; and / or The simulated vehicle trajectory information includes: the trajectory information of the target vehicle at each moment in a first time period, wherein the first time period is: a time period of a preset time period from the current moment backward; the first adjustment module is specifically used to: obtain a second value return value corresponding to the first control amount based on the deviation, wherein the second value return value represents: the degree of matching of the first vehicle trajectory information relative to the first expected trajectory information when the first control amount is used to control the actuator in the target vehicle, and the first vehicle trajectory information is: the simulated vehicle trajectory information of the target vehicle at each moment except the current moment and the next moment in the first time period; generate a cumulative value return value of the first value return value and the second value return value; adjust the model parameters of the basic control model based on the cumulative value return value to obtain the target control model of the controller in the target vehicle; and / or The first adjustment module is specifically used to generate a cumulative value return value according to the following expression: R t =r t+1 +γr t+2 +…+γ n-1 r t+n ; Among them, R t is the cumulative value return value, t is the current time, γ is the set attenuation coefficient, n is the preset time length, r t+1 is the first value return value, r t+2 ,……,r t+n The second value return value; and / or The current simulated vehicle state information includes: the vehicle posture obtained by updating the vehicle posture at the previous moment based on the previous execution amount, and the simulated vehicle state of the target vehicle after the actuator executes the previous execution amount, which is predicted by the target execution model, wherein the previous execution amount is: the execution amount predicted by the target execution model based on the first control amount at the previous moment, historical vehicle state information and historical execution amount; or, the current simulated vehicle state information includes: the simulated vehicle state information obtained by updating the simulated vehicle state information at the previous moment based on the previous execution amount; and / or The basic control model is trained in the following manner: for each sample moment, the sample expected trajectory information, the sample vehicle state information and the actual control amount output by the controller in the basic vehicle are obtained; the sample expected trajectory information and the sample vehicle state information at each sample moment are respectively input into the basic control model to obtain the second control amount of the actuator corresponding to each sample moment output by the basic control model; according to the difference between the second control amount corresponding to each sample moment and the actual control amount, the model parameters of the basic control model are adjusted; and / or The target execution model is trained in the following manner: for each sample moment, the sample vehicle state information of the target vehicle, the actual control amount and the actual execution amount of the actuator input in the target vehicle are obtained; the sample vehicle state information, the actual control amount, the actual execution amount corresponding to the historical moments of each sample moment, and the actual control amount at each sample moment are respectively input into the target execution model to obtain the first vehicle state information and the first execution amount corresponding to each sample moment output by the target execution model; according to the difference between the first vehicle state information corresponding to each sample moment and the sample vehicle state information corresponding to each sample moment, and the difference between the first execution amount corresponding to each sample moment and the actual execution amount corresponding to each sample moment, the model parameters of the target execution model are adjusted; and / or The device also includes a second adjustment module, which is used to: obtain second expected trajectory information and current actual vehicle state information of the target vehicle at the current moment; input the second expected trajectory information and the current actual vehicle state information into the target control model to obtain a third control amount of the actuator output by the target control model; obtain actual vehicle trajectory information of the target vehicle after the actuator of the target vehicle performs an operation under the third control amount; adjust the model parameters of the target control model according to the deviation between the actual vehicle trajectory information and the second expected trajectory information; and / or The actuator includes at least one of the following actuators: a steering actuator, a power actuator, and a braking actuator; and / or the target execution model includes at least one of the following models: a steering execution model, a power execution model, and a braking execution model.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the steps of the method described in any one of claims 1 to 6 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images