Decision model determination method and device, equipment, medium and product

Abstract

The embodiment of the invention discloses a decision model determination method and device, equipment, a medium and a product. The method comprises the steps that a first decision model under a first task type is acquired; for at least one piece of sample data under multiple second task types, processing a loss value of the at least one piece of sample data processed by the first decision model based on a loss value constraint function; and taking the first decision model obtained when the processing result meets a preset condition as a target decision model, and processing the vehicle associated data under the plurality of second task types or the first task type based on the target decision model to obtain a corresponding target decision. According to the technical scheme, the target decision model suitable for different application scenes can be obtained, the vehicle associated data of the application scenes can be processed based on the target decision model, the target decision matched with the vehicle associated data is obtained, and the use comprehensiveness of the target decision model and the accuracy of the target decision result are improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of unmanned driving technology, and in particular, to a decision model determination method, device, equipment, medium and product. Background Art

[0002] With the development of science and technology, autonomous driving has become the focus of current attention. Autonomous driving algorithms mainly include modules such as mapping, perception, prediction, decision-making, planning, and control. Among them, the decision-making module is like the user's brain, which can provide guidance information and constraints for downstream planning.

[0003] At present, the traditional decision-making module mainly determines the final decision information based on pre-set rules. In other words, it is possible to introduce human prior knowledge to construct related rules for constraints, such as determining and outputting the decision made for the target vehicle based on the related information between the target vehicle and other vehicles.

[0004] When implementing the technical solution based on the above method, the inventor found the following problems:

[0005] When determining the decision data of the target vehicle based on rules constructed based on prior knowledge, it mainly relies on prior experience, and the actions of each vehicle affect each other, which is highly complex. That is, in complex scenarios, when controlling the driving of the target vehicle based on prior experience, there is a problem of difficulty in meeting the safety, traffic efficiency and comfort of the target vehicle. Summary of the invention

[0006] The present invention provides a decision model determination method, device, equipment, medium and product to obtain a target decision model that can be applied to different application scenarios, thereby improving the comprehensiveness of the decision model.

[0007] In a first aspect, an embodiment of the present invention provides a decision model determination method, the method comprising:

[0008] Acquire a first decision model under a first task type, wherein the first decision model is used to process vehicle-related data under the first task type to obtain decision data to be used corresponding to the vehicle-related data, and the decision data to be used is used to control vehicle driving;

[0009] For at least one sample data under a plurality of second task types, processing the loss value of at least one sample data processed by the first decision model based on a loss value constraint function; wherein the plurality of second task types include the first task type;

[0010] The first decision model obtained when the processing result meets the preset conditions is used as the target decision model, so as to process the vehicle-related data under the second task type or the first task type based on the target decision model to obtain the corresponding target decision.

[0011] Furthermore, the first decision model is determined based on the following method:

[0012] Determining a task scenario corresponding to the first task type;

[0013] Collecting a plurality of training data corresponding to the task scenario, wherein the training data includes training driving data of the vehicle and training element data of traffic participation elements associated with the vehicle;

[0014] For the plurality of training data, the driving data and the element data in the training data are input into the first decision model to be trained to obtain predicted decision data; wherein the predicted decision data includes the predicted driving data of the vehicle;

[0015] Based on the predicted decision data of the training data and the corresponding actual decision data, the model parameters in the decision model to be trained are adjusted to obtain the first decision model.

[0016] Furthermore, the loss value of at least one sample data processed by the first decision model is processed based on the loss value constraint function, including:

[0017] For at least one sample data under multiple second task types,

[0018] Processing the current sample data based on the first decision model to obtain a current loss value corresponding to the current sample data;

[0019] Based on the current loss value, the historical loss value corresponding to the previous sample data and the loss value constraint function, the current loss value of the current sample data is updated to update the model parameters in the first decision model based on the current loss value.

[0020] Furthermore, the preset condition is a condition that the loss value after processing based on the loss value constraint function is the minimum, and the first decision model obtained when the processing result meets the preset condition is used as the target decision model, including:

[0021] The first decision model when the current loss value is minimized is used as the target decision model, wherein the model parameters in the target decision model correspond to the model parameters determined when the current loss value is minimized.

[0022] Furthermore, the first task type and multiple second task types are constructed based on field contents under at least one field, and the at least one field includes a road type field, a traffic participant element type field, an intersection type field, and an intersection passable type field.

[0023] Furthermore, after obtaining the target decision model, the method further includes: determining a plurality of test samples according to the first task type and a plurality of the second task types, wherein the test samples include test driving data of the vehicle and test element data of traffic participation elements associated with the vehicle;

[0024] For the plurality of test samples, processing the loss values ​​of the test sample data processed by the target decision model according to the loss value constraint function;

[0025] When it is determined that the loss value does not meet the preset condition, at least one sample data under multiple second task types is obtained again, and the step of processing the loss value of at least one sample data processed by the first decision model based on the loss value constraint function is repeated to obtain the target decision model.

[0026] Furthermore, after obtaining the target decision model, the method further includes:

[0027] The target decision model is deployed in a control system corresponding to a target vehicle, so as to process vehicle driving data of the target vehicle and element data of traffic participation elements associated with the target vehicle based on the target decision model to obtain a target decision for the target vehicle;

[0028] Wherein, the target decision includes the driving speed and / or heading angle data of the target vehicle.

[0029] In a second aspect, an embodiment of the present invention further provides a decision model determination device, the device comprising:

[0030] A model acquisition module, used to acquire a first decision model under a first task type, wherein the first decision model is used to process vehicle-related data under the first task type to obtain decision data to be used corresponding to the vehicle-related data, and the decision data to be used is used to control vehicle driving;

[0031] a loss processing module, configured to process, for at least one sample data under a plurality of second task types, a loss value of at least one sample data processed by the first decision model based on a loss value constraint function; wherein the plurality of second task types include the first task type;

[0032] The target model determination module is used to use the first decision model obtained when the processing result meets the preset conditions as the target decision model, so as to process the vehicle-related data under the second task type or the first task type based on the target decision model to obtain the corresponding target decision.

[0033] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0034] one or more processors;

[0035] a storage device for storing one or more programs,

[0036] When the one or more programs are executed by the one or more processors, the one or more processors implement the decision model determination method as described in any one of the embodiments of the present invention.

[0037] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute a decision model determination method as described in any one of the embodiments of the present invention.

[0038] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, is used to execute the decision model determination method as described in any one of the embodiments of the present invention.

[0039] The technical solution of the embodiment of the present invention can first obtain a first decision model obtained by pre-training. At this time, the first decision model is determined based on the vehicle-related data under the first task type. Sample data under multiple second task types are obtained, and based on the sample data and the first decision model, a target decision model that can process multiple task types is determined. This solves the problem that when determining decision information based on rules in the prior art, relatively rich prior knowledge is required, and there is a problem of low security for complex scenes. The solution provided by the embodiment of the present invention can first obtain a decision model through pre-training, and then obtain behavioral data under different scenes (using vehicle driving data under different scenes as specific task types) to train a target decision model that can be applied to any scene. At this time, the target decision model is based on the model obtained after meta-learning. Therefore, the target decision model can be trained by using a small number of training samples under different task types, which not only improves the low cost and convenience of determining the target decision model, but also improves the universality of the target decision model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A flowchart of a decision model determination method provided by an embodiment of the present invention;

[0042] Figure 2 A flowchart of a decision model determination method provided by an embodiment of the present invention;

[0043] Figure 3 A flowchart of a decision model determination method provided by an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of a decision model provided by an embodiment of the present invention;

[0045] Figure 5 A schematic diagram of the structure of a decision model determination device provided by an embodiment of the present invention;

[0046] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] Before introducing the technical solution provided by the embodiment of the present invention, an exemplary description of the application scenario and meta-learning may be given first.

[0048] The target decision model provided by the embodiment of the present invention can be applied to any unmanned or assisted driving vehicle. That is, the target decision model deployed in the target vehicle or the vehicle requiring assisted driving can be used to determine the decision result corresponding to each moment. Based on this, the target decision model corresponding to different application scenarios can be trained first.

[0049] Meta-learning means "learning how to learn". In the autonomous driving decision-making algorithm based on robot learning, the most labor-intensive task is to adjust the model parameters in the model. Usually, each task type needs to be adjusted from scratch, which takes a lot of time to train and test the training effect. At the same time, there is no guarantee that the model can work well.

[0050] Figure 1A flow chart of a decision model determination method provided in an embodiment of the present invention is applicable to situations in which one wants to obtain a target decision model that can handle various task types. The method can be executed by a decision model determination device, which can be deployed on a target vehicle or in a vehicle requiring assisted driving, so as to output a target decision for controlling vehicle driving based on the target decision model.

[0051] like Figure 1 The method of this embodiment includes:

[0052] S110: Obtain a first decision model under a first task type.

[0053] It should be noted that the unmanned vehicle can be driven in a variety of different scenarios. Different scenario types can be defined as a task type. That is, the first task type corresponds to the scenario type corresponding to the target vehicle driving the vehicle during the driving process.

[0054] The first task type corresponds to a certain scenario type. The first decision model is a model trained based on vehicle-related data corresponding to the first task type.

[0055] It can be understood that: multiple sample data associated with the first task type can be obtained, and the sample data may include vehicle driving data and element data of multiple traffic participating elements associated with the vehicle. The first decision model can be trained based on the sample data. In other words, the first decision model is only a model that can process the vehicle-related data under one of the task types. The function of the first decision model is mainly to process the vehicle-related data under the first task type (vehicle driving data and element data of other traffic participating elements associated with the vehicle, the element data at this time may include relative driving speed, relative distance, etc.) to obtain the decision data to be used corresponding to the vehicle-related data, and the decision data to be used is used to control the vehicle driving.

[0056] S120. For at least one sample data under multiple second task types, process the loss value of at least one sample data processed by the first decision model based on a loss value constraint function.

[0057] In order to enable the target decision model to process vehicle-related data in different scenarios, sample data in all other scenarios can be obtained to update the model parameters in the first decision model based on the sample data, so as to obtain a target decision model suitable for all scenarios.

[0058] It should be noted that the scenarios to which the target decision model can be applied are associated with the sample data under the second task type. In other words, if the number of the second task types includes multiple, then the applicable scenarios of the target decision model also include multiple.

[0059] Among them, the number of the plurality of second task types may include a plurality, and the specific number may be related to the scope of use of the target decision model. Based on the above, it can be known that the task type is adapted to the scene in which the unmanned vehicle is located, therefore, the task type can be defined based on the scene in which the target vehicle is located. At least one sample data under the plurality of second task types may be obtained. Of course, in order to improve the accuracy of the target decision model, multiple sample data under each second task type may be obtained as much as possible.

[0060] The loss value constraint function can be understood as a constraint function on how to deal with the loss value of each sample data. The loss value constraint function is mainly used to determine the final loss value corresponding to each sample data, and then adjust the model parameters in the first decision model based on the final loss value to obtain the target decision model.

[0061] Specifically, at least one sample data under multiple second task types is obtained, and the sample data is input into the first decision model. A loss value corresponding to each sample data can be obtained, and the loss value corresponding to the corresponding sample is adjusted based on the loss value and the loss value constraint function to determine the final loss of the sample data.

[0062] In this embodiment, the first task type and the second task type corresponding to each application scenario may be introduced in detail. Optionally, the first task type and the plurality of the second task types are composed based on the field content under at least one field, and the at least one field includes a road type field, a traffic participation element type field, an intersection type field, and an intersection passable type field.

[0063] Among them, the road type field includes road types such as urban main roads, urban auxiliary roads, provincial main roads, provincial auxiliary roads, rural auxiliary roads, rural main roads, and expressways. The traffic participation element type field can include: traffic light type, vehicle type, and obstacle type. The content under the obstacle type can include family cars, medium-sized trucks, large trucks, trailers, electric vehicles, motorcycles, bicycles, pedestrians, wheelchairs, buses, various pets, urban work vehicles, special vehicles, etc. The content under the intersection type includes crossroads, T-junctions, and one-way intersections. The content of the intersection passable type field includes passable or impassable.

[0064] The field contents under the above different types of fields can be combined and processed to obtain each scenario. This scenario can be used as a task type. Multiple vehicle-related data under different scenarios (vehicle driving data of unmanned vehicles and element data of other traffic participating elements) are obtained, and this vehicle-related data can be used as sample data under the corresponding scenario, that is, sample data under the corresponding task type.

[0065] S130. The first decision model obtained when the processing result meets the preset conditions is used as the target decision model, and the vehicle-related data of multiple second task types or first task types are processed based on the target decision model to obtain corresponding target decisions.

[0066] Among them, the target decision model is the final model that can process the vehicle-related data in any scenario and output the target decision result. The target decision can be understood as the control information that controls how the unmanned vehicle should drive.

[0067] Specifically, the first decision model obtained when the processing result meets the preset conditions is used as the target decision model. After the target decision model is obtained, the vehicle-related data under the plurality of second task types or the first task type can be processed based on the target decision model to output a target decision corresponding to the vehicle-related data.

[0068] After obtaining the target decision model, the method further includes: deploying the target decision model in a control system corresponding to the target vehicle, so as to process the vehicle driving data of the target vehicle and the element data of the traffic participation elements associated with the target vehicle based on the target decision model to obtain the target decision of the target vehicle; wherein the target decision includes the driving speed and / or heading angle data of the target vehicle.

[0069] It can be understood that after obtaining the target decision model, the target decision model can be deployed in the control system of the target vehicle. During the unmanned driving process, the vehicle driving information of the target vehicle and the element association information of at least one traffic participating element associated with the target vehicle can be collected based on at least one sensor associated with the target vehicle. At this time, the element association information can be information about other vehicles, users and intersections relative to the target vehicle. The above-mentioned collected information can be input into the deployed target decision model to obtain the target decision at the current moment. That is, the driving speed and / or heading angle data of the target vehicle are controlled so that the control system of the target vehicle can control the driving of the target vehicle according to the target decision.

[0070] The technical solution of the embodiment of the present invention can first obtain a first decision model obtained by pre-training. At this time, the first decision model is determined based on the vehicle-related data under the first task type. Sample data under multiple second task types are obtained, and based on the sample data and the first decision model, a target decision model that can process multiple task types is determined. This solves the problem that when determining decision information based on rules in the prior art, relatively rich prior knowledge is required, and there is a problem of low security for complex scenes. The solution provided by the embodiment of the present invention can first obtain a decision model through pre-training, and then obtain behavioral data under different scenes (using vehicle driving data under different scenes as specific task types) to train a target decision model that can be applied to any scene. At this time, the target decision model is based on the model obtained after meta-learning. Therefore, the target decision model can be trained by using a small number of training samples under different task types, which not only improves the low cost and convenience of determining the target decision model, but also improves the universality of the target decision model.

[0071] Figure 2 A flowchart of a method for determining a decision model provided in an embodiment of the present invention. Based on the above embodiment, "how to determine the first decision model" can be further refined, and its specific method can be found in the detailed description of the embodiment of the present invention. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here.

[0072] like Figure 2 As shown, the method includes:

[0073] S210: Determine a task scenario corresponding to the first task type.

[0074] It can be understood that one or more data contents can be selected from the data contents under each field type. After the selection is completed, all the selected data contents can be combined to obtain one of the scenarios. This task scenario is used as the first task type.

[0075] S220. Collect a plurality of training data corresponding to the task scenario, wherein the training data includes training driving data of the vehicle and training element data of traffic participation elements associated with the vehicle.

[0076] After determining the mission scenario, sample data under the mission scenario can be collected. The sample data at this time is used as training data. In other words, the training data includes training driving data and training element data of traffic participation elements associated with the vehicle.

[0077] It should be noted that the training data obtained at this time is mainly training data that is adapted to the task scenario.

[0078] S230. For the plurality of training data, input the training driving data and the training element data in the training data into a first decision model to be trained to obtain predicted decision data; wherein the predicted decision data includes the predicted driving data of the vehicle.

[0079] The first decision model to be trained is a neural network model, and the model parameters in the neural network model are default values. Based on this, a first decision model whose model parameters meet corresponding conditions can be trained. Since the model parameters in the first decision model to be trained are default values, the output result of the first decision model to be trained may have a certain deviation from the actual result, and the output result of the first decision model to be trained at this time can be used as the predicted decision data. The predicted driving data may include the predicted driving speed and / or heading angle of the target vehicle.

[0080] Specifically, for each training data, the training driving data and the training element data in the training data can be input into the first decision model to be trained in turn. The first decision model to be trained can output predicted decision data corresponding to the training data. The predicted decision data at this time includes the predicted driving data of the vehicle, and optionally, the predicted driving data includes the heading angle and / or the vehicle driving speed. S240. Based on the predicted decision data of the training data and the corresponding actual decision data, adjust the model parameters in the decision model to be trained to obtain the first decision model.

[0081] For each training data, after obtaining the predicted decision data, the loss value can be determined according to the actual decision data of the training sample and the corresponding predicted decision data. The model parameters in the first decision model to be trained are corrected based on the loss value. The convergence of the loss function in the first decision model to be trained is used as the training goal to obtain the first decision model.

[0082] It should be noted that the training data can be road sampling data or simulation data, and all of them are data generated under normal and safe driving conditions. Therefore, the actual decision data has a certain reference value. The actual decision data in the training data can be used as a reference standard to determine the loss value between the actual decision data and the predicted decision data, and then adjust the model parameters in the first decision model.

[0083] The technical solution provided in the embodiment of the present invention can train the first decision model to be trained by acquiring training data under the first task type to obtain a usable first decision model, which is convenient for subsequent training of multiple target decision models that can process data under different task types based on the first decision model, which not only improves the convenience of determining the target decision model, but also improves the comprehensiveness of the use of the target decision model.

[0084] Figure 3 A flowchart of a method for determining a decision model provided in an embodiment of the present invention. Based on the above embodiment, further refinement can be made to "how to determine a target decision model", and the specific method can be found in the detailed description of the embodiment of the present invention. Technical terms that are the same as or corresponding to the above embodiment are not repeated here.

[0085] like Figure 3 As shown, the method includes:

[0086] S310: Obtain a first decision model under a first task type.

[0087] Before or after obtaining the first decision model, multiple second task types, that is, multiple task scenarios, can be determined. Optionally, the second task type is determined by obtaining corresponding content from different field types. In order to improve the comprehensiveness of the target decision model for data processing, as many scenarios as possible can be determined, and sample data under the corresponding scenarios can be obtained.

[0088] S320. Process the current sample data based on the first decision model to obtain a current loss value corresponding to the current sample data.

[0089] Multiple sample data under multiple second task types can be obtained. The multiple sample data can be processed according to the first decision model. The processing method for each sample data is the same. In this embodiment, the processing of one of the sample data is used as an example to illustrate, and the sample data currently introduced is used as the current sample data.

[0090] Specifically, the current sample data may be input into the first decision model. The first decision model may output a model output result corresponding to the current sample data. The model output result may be a model prediction decision corresponding to the current sample data. Based on the model prediction result and the actual decision data in the current sample data, a current loss value corresponding to the current sample data may be determined.

[0091] S330. Based on the current loss value, the historical loss value corresponding to the previous sample data, and the loss value constraint function, update the current loss value of the current sample data to update the model parameters in the first decision model based on the current loss value.

[0092] Specifically, after obtaining the current loss value corresponding to the current sample data, the loss value corresponding to other sample data determined by the first decision model before the current sample data can be obtained. The loss value obtained after the first decision model previously processed other sample data is used as the historical loss value. After obtaining the current loss, the total loss value corresponding to the current sample data can be determined by combining the current loss value, the historical loss value, and the loss value constraint function. The current loss value is updated based on the total loss value. After updating the current loss value, the model parameters in the first decision model can be updated based on the updated current loss value.

[0093] It should be noted that, for the next sample data, the current loss value corresponding to the current sample data can be used as the historical loss value corresponding to the next sample data.

[0094] In this embodiment, the loss value constraint function can be understood as a constraint function that controls the loss value to be minimum.

[0095] S340. The first decision model obtained when the processing result meets the preset conditions is used as the target decision model, and the vehicle-related data of multiple second task types or first task types are processed based on the target decision model to obtain corresponding target decisions.

[0096] Among them, the preset condition is the condition that the loss value after processing based on the loss value constraint function is minimized, and the first decision model obtained when the processing result meets the preset condition is used as the target decision model, including: using the first decision model when the current loss value is minimized as the target decision model, wherein the model parameters in the target decision model correspond to the model parameters determined when the current loss is minimized.

[0097] It can be understood that: according to the minimum loss value constraint function, the historical loss value and the current loss value can be processed to update the current loss value of the current sample data. If the current loss value is the minimum, the model parameters determined according to the current loss value can be used as the model parameters in the target decision model. In other words, the model parameters determined when the current loss value is the minimum can be used as the model parameters in the target decision model.

[0098] After obtaining the target decision model, in order to determine whether the target decision model can be used as a model for final use, a test sample can be obtained to verify the target decision model based on the test sample to obtain a usable target decision model.

[0099] On the basis of the above technical solution, the method further includes: determining a plurality of test samples according to the first task type and a plurality of the second task types, wherein the test samples include test driving data of the vehicle and test element data of traffic participating elements associated with the vehicle; for the plurality of the test samples, processing the loss value of the test sample data processed by the target decision model according to the loss value constraint function;

[0100] It can be understood as follows: for the test sample, the current test sample can be input into the target decision model. After the target decision model outputs the predicted decision, the loss value can be determined based on the predicted decision and the actual decision in the test sample. The current loss value is updated based on the loss value constraint function, the loss value, and the historical loss value determined before the current one. If the current loss value is the smallest, it means that the target decision model can be used.

[0101] Of course, when it is determined that the loss value does not meet the preset condition, at least one sample number under multiple second task types is obtained again, and the loss value of at least one sample data processed by the first decision model is processed based on the loss value constraint function, and the step of obtaining the target decision model is repeated.

[0102] The technical solution of the embodiment of the present invention can first obtain a first decision model obtained by pre-training. At this time, the first decision model is determined based on the vehicle-related data under the first task type. Sample data under multiple second task types are obtained, and based on the sample data and the first decision model, a target decision model that can process multiple task types is determined. This solves the problem that when determining decision information based on rules in the prior art, relatively rich prior knowledge is required, and there is a problem of low security for complex scenes. The solution provided by the embodiment of the present invention can first obtain a decision model through pre-training, and then obtain behavioral data under different scenes (using vehicle driving data under different scenes as specific task types) to train a target decision model that can be applied to any scene. At this time, the target decision model is based on the model obtained after meta-learning. Therefore, the target decision model can be trained by using a small number of training samples under different task types, which not only improves the low cost and convenience of determining the target decision model, but also improves the universality of the target decision model.

[0103] As an optional embodiment of the above embodiment, an autonomous driving vehicle decision algorithm based on Meta Learning can be used, with the goal of allowing the machine to learn to adjust parameters by itself, so that it can draw inferences and draw inferences when encountering similar tasks. When entering different scenarios or facing different types of obstacles, there is no need to adjust parameters from scratch, nor is there a need for re-training with a large amount of labeled data.

[0104] The goal of the autonomous driving decision algorithm based on Meta Learning in this invention is to determine Function F φ The algorithm model is allowed to automatically learn hyperparameters instead of manually adjusting them according to the scene and obstacle type. Such hyperparameters include initialization parameter Theta 0, learning rate α, and network architecture. The problem to be solved by the present invention is to enable the algorithm model to output behavioral decisions of different types of obstacles in different road scenes, guide the target vehicle (autonomous driving vehicle) to make corresponding decisions, avoid collision accidents, and improve the traffic efficiency of the target vehicle. Therefore, the hyperparameters automatically learned by the algorithm proposed in the present invention mainly revolve around the initialization parameters. After learning the initialization parameters, when calling the algorithm model in different scenes or with different obstacles, only a small amount of data is needed to automatically fine-tune the parameters.

[0105] Specifically, it can be:

[0106] The training data under the first task type is obtained, and the training data may include field contents under different fields. That is, the training data may include vehicle driving data and element data of other traffic participating elements that are compatible with the field contents. Multiple task types may be determined based on the field contents under different fields.

[0107] The task types may include: the behavior decision of the main vehicle (target vehicle) on urban roads without traffic lights, the behavior decision of urban roads at traffic lights, the behavior decision of urban roads without intersections, the behavior decision of urban road auxiliary roads, the behavior decision of township roads without traffic lights, the behavior decision of township roads at traffic lights, the behavior decision of township roads without intersections, the behavior decision of highways, the behavior decision of rural roads, etc. The obstacle types include family cars, medium trucks, large trucks, trailers, electric vehicles, motorcycles, bicycles, pedestrians, wheelchairs, buses, dogs, cats, urban work vehicles, special vehicles (police cars, fire trucks, ambulances, etc.), and the main vehicle's decision on different types of obstacles can be divided into different task types.

[0108] Based on the training data corresponding to one of the task types, a first decision model is trained, which can be: Figure 5 , the training driving data and the training element data in the training data under the first task type (TASK1) are input into the first decision model, so that F in the first decision model φ Learn and output the predicted decision data. Determine the loss value based on the predicted decision data and the actual decision data in the training data. By superimposing the loss values ​​of each sample data, a usable first decision model is obtained. After obtaining the first decision model, the test sample in the test data set can be input into the first decision model to obtain the loss value l corresponding to the test sample data.1 . Further, when the first decision model is obtained, in order to obtain a target decision model that can process vehicle driving data of any task type, sample data under multiple second task types can be obtained. For all sample data under the second task type, the sample data can be processed based on the first decision model to obtain the loss value corresponding to the sample data. Based on the loss value, the historical loss value obtained after the first decision model previously processed the sample data and the loss value constraint function, the loss value of the sample data is updated, and the model parameters of the first decision model are corrected based on the loss value. The model parameters obtained when the loss value is minimized are used as the model parameters of the target decision model.

[0109] For example, see Figure 5 , train the first decision model based on the sample data of the second task type (TASK2) to update the model parameters in the first decision model. Next, obtain the test sample corresponding to the second task type. Input the test sample into the target decision model to obtain the loss value of the target decision model, that is, obtain the loss value l corresponding to each second task type. i , the goal of the behavior decision algorithm based on Meta Learning is to minimize L(φ). L(φ) is the superposition function of the loss values ​​corresponding to all second task types.

[0110] It should be noted that the behavior decision method based on meta-learning is not aimed at a specific task, but to find a good hyperparameter for these similar behavior decision tasks, so that when encountering similar tasks, the learned hyperparameters can be used directly without manual adjustment. Therefore, the loss function of the autonomous driving behavior decision algorithm based on Meta Learning is defined as: Where N is the number of training tasks. L(φ) represents the loss constraint function, l n is the loss value corresponding to the sample data under each task type, n represents the total number of task types, and φ represents the model parameters of the target decision model.

[0111] After obtaining the optimal parameters of the target decision model based on the above method, the trained model can be tested with a test data set (test samples of the first task type and multiple second task types). The loss value obtained by the test is l 1 Similarly, the loss value l can be determined based on the above function 1 If so, the target decision model can be used, otherwise, sample data under the first task type and / or multiple second task types are obtained to continue training the target decision model to obtain a target decision model that satisfies the loss constraint function.

[0112] Figure 5A schematic diagram of the structure of a decision model determination device provided in an embodiment of the present invention, the device includes: a model acquisition module 410, a loss processing module 420 and a target model determination module 430.

[0113] Among them, the model acquisition module 410 is used to obtain a first decision model under a first task type, wherein the first decision model is used to process vehicle-related data under the first task type to obtain decision data to be used corresponding to the vehicle-related data, and the decision data to be used is used to control vehicle driving; the loss processing module 420 is used to process the loss value of at least one sample data processed by the first decision model for at least one sample data under multiple second task types based on a loss value constraint function; wherein the multiple second task types include the first task type; the target model determination module 430 is used to use the first decision model obtained when the processing result meets the preset conditions as the target decision model, so as to process the vehicle-related data under the second task type or the first task type based on the target decision model to obtain the corresponding target decision.

[0114] On the basis of the above technical solution, the device further comprises: a scene determination module, used to determine a task scene corresponding to the first task type;

[0115] A training data collection module, used to collect a plurality of training data corresponding to the task scenario, wherein the training data includes training driving data of the vehicle and training element data of traffic participation elements associated with the vehicle;

[0116] A decision data prediction model, for the plurality of training data, inputting the driving data and the element data in the training data into a first decision model to be trained to obtain predicted decision data; wherein the predicted decision data includes the predicted driving data of the vehicle;

[0117] The decision model determination module is used to adjust the model parameters in the decision model to be trained based on the predicted decision data of the training data and the corresponding actual decision data to obtain the first decision model.

[0118] On the basis of the above technical solution, the loss value determination module includes:

[0119] For at least one sample data under multiple second task types,

[0120] a current loss value determining unit, configured to process the current sample data based on the first decision model to obtain a current loss value corresponding to the current sample data;

[0121] A model parameter updating unit is used to update the current loss value of the current sample data based on the current loss value, the historical loss value corresponding to the previous sample data, and the loss value constraint function, so as to update the model parameters in the first decision model based on the current loss value.

[0122] On the basis of the above technical solutions, the decision model determination module includes:

[0123] A decision model determination unit is used to use the first decision model when the current loss value is minimized as the target decision model, wherein the model parameters in the target decision model correspond to the model parameters determined when the current loss is minimized.

[0124] Based on the above technical solutions, the first task type and multiple second task types are composed based on the field content under at least one field, and the at least one field includes a road type field, a traffic participant element type field, an intersection type field, and an intersection passable type field.

[0125] On the basis of the above technical solutions, the device further includes: a test sample acquisition module, which is used to determine a plurality of test samples according to the first task type and a plurality of the second task types, wherein the test samples include test driving data of the vehicle and test element data of traffic participation elements associated with the vehicle;

[0126] A loss processing module, used for processing the loss value of the test sample data processed by the target decision model according to the loss value constraint function for the plurality of test samples;

[0127] A repeated execution module is used to obtain at least one sample number under multiple second task types again when it is determined that the loss value does not meet the preset condition, and repeatedly execute the step of processing the loss value of at least one sample data processed by the first decision model based on the loss value constraint function to obtain the target decision model.

[0128] On the basis of the above-mentioned technical solutions, the device also includes: a target decision model deployment module, which is used to deploy the target decision model in a control system corresponding to the target vehicle, so as to process the vehicle driving data of the target vehicle and the element data of the traffic participation elements associated with the target vehicle based on the target decision model to obtain the target decision of the target vehicle; wherein the target decision includes the driving speed and / or heading angle data of the target vehicle.

[0129] The technical solution of the embodiment of the present invention can first obtain a first decision model obtained by pre-training. At this time, the first decision model is determined based on the vehicle-related data under the first task type. Sample data under multiple second task types are obtained, and based on the sample data and the first decision model, a target decision model that can process multiple task types is determined. This solves the problem that when determining decision information based on rules in the prior art, relatively rich prior knowledge is required, and there is a problem of low security for complex scenes. The solution provided by the embodiment of the present invention can first obtain a decision model through pre-training, and then obtain behavioral data under different scenes (using vehicle driving data under different scenes as specific task types) to train a target decision model that can be applied to any scene. At this time, the target decision model is based on the model obtained after meta-learning. Therefore, the target decision model can be trained by using a small number of training samples under different task types, which not only improves the low cost and convenience of determining the target decision model, but also improves the universality of the target decision model.

[0130] The decision model determination device provided in the embodiment of the present invention can execute the decision model determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0131] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0132] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 6 A block diagram of an exemplary electronic device 50 suitable for implementing exemplary embodiments of the present invention is shown. Figure 6 The electronic device 50 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0133] like Figure 6 As shown, the electronic device 50 is in the form of a general computing device. The components of the electronic device 50 may include but are not limited to: one or more processors or processing units 501, a system memory 502, and a bus 503 connecting different system components (including the system memory 502 and the processing unit 501).

[0134] Bus 503 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0135] The electronic device 50 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 50, including volatile and non-volatile media, removable and non-removable media.

[0136] System memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 504 and / or cache memory 505. Electronic device 50 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 506 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 503 via one or more data medium interfaces. The memory 502 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the embodiments of the present invention.

[0137] A program / utility 508 having a set (at least one) of program modules 507 may be stored, for example, in the memory 502, such program modules 507 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 507 generally perform the functions and / or methods of the embodiments described herein.

[0138] The electronic device 50 may also communicate with one or more external electronic devices 509 (e.g., keyboards, pointing devices, displays 410, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 50, and / or communicate with any device that enables the electronic device 50 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 511. Furthermore, the electronic device 50 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 512. As shown, the network adapter 512 communicates with other modules of the electronic device 50 through the bus 503. It should be understood that although Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0139] The processing unit 501 executes various functional applications and data processing by running the programs stored in the system memory 502, such as implementing the decision model determination method provided in the embodiment of the present invention.

[0140] An embodiment of the present invention also provides a storage medium containing computer executable instructions, and the computer executable instructions are used to execute the decision model determination method when executed by a computer processor.

[0141] The method includes:

[0142] Acquire a first decision model under a first task type, wherein the first decision model is used to process vehicle-related data under the first task type to obtain decision data to be used corresponding to the vehicle-related data, and the decision data to be used is used to control vehicle driving;

[0143] For at least one sample data under a plurality of second task types, processing the loss value of at least one sample data processed by the first decision model based on a loss value constraint function; wherein the plurality of second task types include the first task type;

[0144] The first decision model obtained when the processing result meets the preset conditions is used as the target decision model, so as to process the vehicle-related data under the second task type or the first task type based on the target decision model to obtain the corresponding target decision.

[0145] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0146] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0147] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0148] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0149] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A decision model determination method, characterized in that: include: Acquire a first decision model under a first task type, wherein the first decision model is used to process vehicle-related data under the first task type to obtain decision data to be used corresponding to the vehicle-related data, and the decision data to be used is used to control vehicle driving; For at least one sample data under a plurality of second task types, processing the loss value of at least one sample data processed by the first decision model based on a loss value constraint function; wherein the plurality of second task types at least include the first task type; The first decision model obtained when the processing result meets the preset conditions is used as the target decision model, and the vehicle-related data of multiple second task types are processed based on the target decision model to obtain corresponding target decisions.

2. The method according to claim 1, characterized in that The first decision model is determined based on the following method: Determining a task scenario corresponding to the first task type; Collecting a plurality of training data corresponding to the task scenario, wherein the training data includes training driving data of the vehicle and training element data of traffic participation elements associated with the vehicle; For the plurality of training data, input the training driving data and the training element data in the training data into the first decision model to be trained to obtain predicted decision data; wherein the predicted decision data includes the predicted driving data of the vehicle; Based on the predicted decision data of the training data and the corresponding actual decision data, the model parameters in the first decision model to be trained are adjusted to obtain the first decision model.

3. The method according to claim 1, characterized in that The processing of the loss value of at least one sample data processed by the first decision model based on the loss value constraint function includes: For at least one sample data under multiple second task types, Processing the current sample data based on the first decision model to obtain a current loss value corresponding to the current sample data; Based on the current loss value, the historical loss value corresponding to the previous sample data and the loss value constraint function, the current loss value of the current sample data is updated to update the model parameters in the first decision model based on the current loss value.

4. The method according to claim 3, characterized in that The preset condition is a condition that the loss value after processing based on the loss value constraint function is the minimum, and the first decision model obtained when the processing result meets the preset condition is used as the target decision model, including: The first decision model obtained when the current loss value is minimized after updating is used as the target decision model, wherein the model parameters in the target decision model correspond to the model parameters determined when the current loss value is minimized.

5. The method according to claim 1, characterized in that The first task type and the plurality of the second task types are constructed based on the field content under at least one field, and the at least one field includes a road type field, a traffic participant element type field, an intersection type field, and an intersection passable type field.

6. The method according to claim 1, characterized in that After obtaining the target decision model, the method further includes: Determining a plurality of test samples according to the first task type and a plurality of the second task types, wherein the test samples include test driving data of the vehicle and test element data of traffic participation elements associated with the vehicle; For the plurality of test samples, processing the loss values ​​of the test sample data processed by the target decision model according to the loss value constraint function; When the loss value does not meet the preset condition, at least one sample data under multiple second task types is obtained again, and the step of processing the loss value of at least one sample data processed by the first decision model based on the loss value constraint function is repeated to obtain the target decision model.

7. The method according to claim 1, characterized in that After obtaining the target decision model, the method further includes: The target decision model is deployed in a control system corresponding to a target vehicle, so as to process vehicle driving data of the target vehicle and element data of traffic participation elements associated with the target vehicle based on the target decision model to obtain a target decision for the target vehicle; Wherein, the target decision includes the driving speed and / or heading angle data of the target vehicle.

8. A decision model determination device, characterized in that: The device comprises: A model acquisition module, used to acquire a first decision model under a first task type, wherein the first decision model is used to process vehicle-related data under the first task type to obtain decision data to be used corresponding to the vehicle-related data, and the decision data to be used is used to control vehicle driving; a loss processing module, configured to process, for at least one sample data under a plurality of second task types, a loss value of at least one sample data processed by the first decision model based on a loss value constraint function; wherein the plurality of second task types at least include the first task type; The target model determination module is used to use the first decision model obtained when the processing result meets the preset conditions as the target decision model, so as to process the vehicle-related data under the second task type based on the target decision model to obtain the corresponding target decision.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the decision model determination method as described in any one of claims 1-7.

10. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the decision model determination method according to any one of claims 1 to 7 when executed by a computer processor.

11. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program is used to execute the decision model determination method according to any one of claims 1 to 7.

Images