A biased imitation learning method and system supported by a dynamics model

By introducing dynamic models and discriminators in the training of autonomous vehicles, using dynamic models to amplify expert data and imitate strategy models through discriminator weighting training, the expert data in the existing technology is solved with limited expert data, high training cost and low efficiency in the existing technology. The problem has achieved efficient and robust imitation learning effect.

CN115099037BActive Publication Date: 2024-11-29TSINGHUA UNIVERSITY
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210744890.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-27
Publication Date
2024-11-29
Estimated Expiration
2042-06-27

AI Technical Summary

Technical Problem

The existing imitation learning methods have problems such as limited expert data, high training costs and low efficiency in the training of autonomous driving vehicles.

Method used

By introducing dynamic models and discriminators, we obtain the training sample set required for autonomous driving training, including offline expert data sets and dynamic model data sets. The driving imitation strategy model is trained using the discriminator output weighted values, and the trained model is simulated and trained.

Benefits of technology

It realizes that under the case of limited expert data sets, the efficiency and effectiveness of imitation learning are improved, the cost of obtaining expert data is reduced, and the robustness and performance of driving strategies are enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115099037B_ABST
    Figure CN115099037B_ABST
Patent Text Reader

Abstract

The present invention provides a biased imitation learning method and system supported by a dynamic model, including: obtaining a training sample set required for autonomous driving training, wherein the training sample includes an offline expert data set and a dynamic model data set; training a driving imitation strategy model by outputting a preset discriminator weighted value; and performing imitation training on an autonomous driving vehicle by using the trained driving imitation strategy model. The present invention solves the defects of existing imitation learning, such as limited available expert data, high training cost, and low efficiency, so as to realize fast and efficient biased imitation learning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of machine imitation, and in particular to a biased imitation learning method and system supported by a dynamics model. Background Art

[0002] Through imitation learning, the characteristics of the target learning object can be obtained, and a model with basically the same functions as the target object can be generated, playing the same role. Existing imitation learning can adopt online imitation training or offline training. In the online training method, the commonly used GAN structure is difficult to train and the training results are difficult to converge; in the offline training method, the BCND structure uses the output of the previous moment to weight the next moment, which has high requirements on the quality of expert data set samples and is difficult to achieve ideal results; the ORIL structure trains the discriminator as a reward, which takes a long time, and the discriminator and strategy training are separated, resulting in low training efficiency.

[0003] And when using offline imitation learning, it is necessary to obtain learning data from expert datasets, but expert datasets have a small amount of data, high acquisition costs, difficult to guarantee quality, and limited coverage of scenarios, resulting in poor overall training results and low training efficiency. Summary of the invention

[0004] The present invention provides a biased imitation learning method and system supported by a dynamic model, which are used to solve the defects of existing imitation learning, such as limited available expert data, high training cost and low efficiency, so as to realize fast and efficient biased imitation learning.

[0005] The present invention provides a biased imitation learning method supported by a dynamics model, comprising:

[0006] Acquire a training sample set required for autonomous driving training, wherein the training sample includes an offline expert data set and a dynamic model data set;

[0007] The driving imitation strategy model is trained by using the preset discriminator output weighted values;

[0008] Through the trained driving imitation strategy model, the autonomous driving vehicle is trained by imitation

[0009] According to a biased imitation learning method supported by a dynamics model provided by the present invention,

[0010] Acquire a training sample of a driving simulation strategy model, wherein the training sample includes an offline expert data set and a dynamic model data set;

[0011] The driving imitation strategy model is trained by using the preset discriminator output weighted values, including:

[0012] The offline expert data set and the dynamic model data set are input into the discriminator, and the authenticity of the judged samples is output;

[0013] The driving imitation strategy model is trained using the weighted values ​​output by the discriminator to learn expert data;

[0014] The dynamic model data set is generated by learning the offline expert data set through the dynamic model.

[0015] According to a biased imitation learning method supported by a dynamics model provided by the present invention, the dynamics model data set is generated by learning an offline expert data set through a dynamics model, specifically comprising:

[0016] Inputting offline expert datasets into the kinetic model;

[0017] The kinetic model is learned based on the expert data in the offline expert data set, and data amplification is performed to generate a kinetic model data set;

[0018] The dynamics model data set includes real expert data and pseudo expert data.

[0019] According to a biased imitation learning method supported by a dynamics model provided by the present invention, the offline expert data set and the dynamics model data set are input into a discriminator, and the authenticity of the judged sample is output, specifically including:

[0020] The discriminator discriminates the input offline expert data set and the dynamic model data set to discriminate the real expert data;

[0021] The real expert data is a true value sample, and the weight of the real expert data is increased.

[0022] According to a biased imitation learning method supported by a dynamics model provided by the present invention, the real expert data is a true value sample, and the weight of the real expert data is increased, specifically including:

[0023] The data judged by the discriminator includes real expert data and pseudo-expert data;

[0024] Increase the weight for true value samples and reduce the weight coefficient for pseudo-expert data;

[0025] The discriminator outputs a weighted value based on the weight system.

[0026] According to a biased imitation learning method supported by a dynamic model provided by the present invention, the discriminator and the dynamic model are trained in adversarial coupling, and the result of the discriminator's judgment is fed back to the dynamic model, so that the dynamic model adjusts its learning strategy and learns more real data.

[0027] According to a biased imitation learning method supported by a dynamics model provided by the present invention, the imitation training of the autonomous driving vehicle is performed through the trained driving imitation strategy model, specifically comprising:

[0028] The driving imitation strategy model is trained by outputting weighted values ​​of the discriminator, the imitation learning strategy is corrected, and a corrected driving imitation strategy model is generated;

[0029] The modified driving imitation strategy model performs imitation learning on real expert data to imitate the functions corresponding to the expert data set.

[0030] The present invention also provides a biased imitation learning system supported by a dynamics model, the system comprising:

[0031] A data acquisition module, which acquires a training sample set required for autonomous driving training, wherein the training sample includes an offline expert data set and a dynamic model data set;

[0032] The imitation learning module trains the driving imitation strategy model through the preset discriminator output weighted values;

[0033] The training module performs imitation training on the autonomous driving vehicle through the trained driving imitation strategy model.

[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a biased imitation learning method supported by a dynamic model as described above is implemented.

[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a biased imitation learning method supported by a dynamic model as described in any one of the above.

[0036] The present invention provides a biased imitation learning method and system supported by a dynamics model. By introducing a system dynamics model and using the data generated by it to train the imitation strategy, the insufficient sample size of the expert data set is compensated, and to a certain extent, the compound error that may be caused by traditional imitation learning when the limited expert data set cannot cover the entire state space is alleviated. Biased imitation learning is achieved by introducing a discriminator, and different weights are calculated for samples from different data sets, so that the imitation strategy is more sensitive to samples without reward labels or samples generated by the dynamics model that are close to the expert data set, thereby making the driving strategy more robust and having better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is one of the flow charts of a biased imitation learning method supported by a kinetic model provided by the present invention;

[0039] Figure 2 This is the second flow chart of a biased imitation learning method supported by a dynamics model provided by the present invention;

[0040] Figure 3 This is the third flow chart of a biased imitation learning method supported by a dynamics model provided by the present invention;

[0041] Figure 4 This is the fourth flow chart of a biased imitation learning method supported by a dynamics model provided by the present invention;

[0042] Figure 5 This is the fifth flow chart of a biased imitation learning method supported by a dynamics model provided by the present invention;

[0043] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention.

[0044] Figure 7 It is a schematic diagram of module connection of a biased imitation learning system supported by a dynamics model provided by the present invention;

[0045] Figure 8 It is an architecture diagram of a biased imitation learning method supported by a dynamics model provided by the present invention;

[0046] Fig. 9 It is a learning curve diagram of an original imitation learning method of biased imitation learning supported by a dynamic model provided by the present invention in a Hopper-expert environment;

[0047] Fig.10 It is a learning curve diagram of a preferred imitation learning method supported by a dynamic model in a Hopper-expert environment provided by the present invention;

[0048] Fig.11 It is a learning curve diagram of an original imitation learning method of biased imitation learning supported by a dynamic model provided by the present invention in a Halfcheetah-exper environment;

[0049] Fig.12 It is a learning curve diagram of a preferred imitation learning method supported by a kinetic model in a Halfcheetah-exper environment provided by the present invention;

[0050] Fig.13 It is a learning curve diagram of an original imitation learning method of biased imitation learning supported by a dynamic model in a walker2d-expert environment provided by the present invention;

[0051] Fig.14 It is a learning curve diagram of a preferred imitation learning method supported by a dynamic model in a walker2d-expert environment provided by the present invention.

[0052] Reference numerals:

[0053] 110: data acquisition module; 120: imitation learning module; 130: training module;

[0054] 610: processor; 620: communication interface; 630: memory; 640: communication bus. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Combine the following Figure 1-Figure 5 Describe the preferred imitation method supported by a dynamics model of the present invention to obtain a training sample set required for autonomous driving training;

[0057] S100, obtaining a training sample set required for autonomous driving training, wherein the training sample includes an offline expert data set and a dynamic model data set;

[0058] S200: training a driving imitation strategy model by outputting a weighted value of a preset discriminator.

[0059] S300: Performing simulation training on the autonomous driving vehicle through the trained driving simulation strategy model.

[0060] The driving imitation strategy model is trained by outputting the preset discriminator weighted values, including

[0061] S201, the offline expert data set and the dynamic model data set are input into the discriminator, and the authenticity of the judged samples is output;

[0062] S202, using the discriminator output weighted value to train the driving imitation strategy model to learn expert data;

[0063] The dynamic model data set is generated by learning the offline expert data set through the dynamic model.

[0064] Since a large amount of expert data is required for imitation learning, learning training can ensure that the learning effect reaches an ideal state, but the amount of expert data is small and the acquisition cost is high. Therefore, the present invention learns the offline expert data set through the dynamic model to achieve data amplification, and discriminates the data in the offline expert data set and the dynamic model data set through the discriminator, selects the real expert data, and assigns a higher weight, thereby achieving the amplification of expert data, covering the entire state space, and making the imitation strategy more sensitive to the unrewarded label samples or dynamic model generated samples close to the expert data set, so that the imitation strategy is more robust and has better performance.

[0065] refer to Figure 8 ,The kinetic model dataset is generated by learning the offline expert dataset through the kinetic model, including:

[0066] Inputting offline expert datasets into the kinetic model;

[0067] The kinetic model is learned based on the expert data in the offline expert data set, and data amplification is performed to generate a kinetic model data set;

[0068] The dynamics model data set includes real expert data and pseudo expert data.

[0069] The offline expert data set includes expert data. The offline expert data set is learned through the dynamic model, and the data in the offline expert data set is learned and amplified. However, during the learning and data amplification process, pseudo-expert data exists and cannot participate in the training of the driving imitation strategy model. Therefore, it is necessary to distinguish the data in the dynamic model data set. Learning through the dynamic model increases the amount of expert data, reduces the cost of data acquisition, and helps to make the subsequent training of the driving imitation strategy model more accurate.

[0070] The offline expert dataset and the dynamic model dataset are input into the discriminator, and the output is used to judge the authenticity of the sample, including:

[0071] S2011, the discriminator discriminates the input offline expert data set and the dynamic model data set to discriminate the real expert data;

[0072] S2012: The real expert data is a true value sample, and the weight of the real expert data is increased.

[0073] The real expert data is the true value sample, and the weight of the real expert data is increased, including:

[0074] S20121. The data judged by the discriminator includes real expert data and pseudo-expert data;

[0075] S20122. Increase the weight of true value samples and reduce the weight coefficient for pseudo-expert data;

[0076] S20123. The discriminator outputs a weighted value according to different weight systems.

[0077] The discriminator and the dynamics model are trained in adversarial coupling, and the results of the discriminator's judgment are fed back to the dynamics model, so that the dynamics model can adjust its learning strategy and learn more real data.

[0078] The discriminator and the dynamics model are trained in adversarial coupling. The discriminator selects real data with higher weights, allowing the dynamics model to learn more real data. The dynamics model also provides feedback to the discriminator, allowing the subsequent discriminator to more accurately discriminate real data.

[0079] In the present invention, a discriminator is set to perform comprehensive judgment on the data in the offline expert data set and the dynamic model data set, and the output weighted value selects the real expert data, thereby increasing the expert data and reducing the cost of obtaining the expert data.

[0080] Through the trained driving imitation strategy model, the autonomous driving vehicle is trained by imitation, including:

[0081] S2021, training the driving imitation strategy model by outputting weighted values ​​of the discriminator, correcting the imitation learning strategy, and generating a corrected driving imitation strategy model;

[0082] S2022. The modified driving imitation strategy model performs imitation learning on real expert data to imitate the functions corresponding to the expert data set.

[0083] The expert data is expanded, the cost of acquiring expert data is reduced, and the driving imitation strategy model imitates the expert data more closely.

[0084] The imitation strategy training process is derived in the form of min max formula:

[0085]

[0086]

[0087]

[0088] in:

[0089] α1: imitation strategy loss weight; α2: dynamics model loss weight

[0090] e: Expert dataset; f: Dataset generated by the dynamics model

[0091] d: discriminator; π: imitation strategy; F: dynamics model

[0092] L f : Dynamics model loss; Loss function of the discriminator, in cross entropy form.

[0093] Through specific experiments, reference Fig. 9 and Fig.10 , in the Hopper-expert environment, Fig. 9 is the learning curve of the original imitation learning method, Fig.10 The preferred imitation learning method learning curve disclosed in the present invention; Fig.11 and Fig.12 , in the Halfcheetah-expert environment, Fig.11 is the learning curve of the original imitation learning method, Fig.12 The preferred imitation learning method learning curve disclosed in the present invention; Fig.13 and Fig.14 , in the walker2d-expert environment, Fig.13 is the learning curve of the original imitation learning method, Fig.14 This is the learning curve of the biased imitation learning method disclosed in the present invention.

[0094] The present invention provides a biased imitation learning method supported by a dynamics model. By introducing a system dynamics model and using the data generated by the model to train the imitation strategy, the insufficient sample size of the expert data set is compensated, and to a certain extent, the compound error that may be caused by traditional imitation learning when the limited expert data set cannot cover the entire state space is alleviated. Biased imitation learning is achieved by introducing a discriminator, and different weights are calculated for samples from different data sets, so that the imitation strategy is more sensitive to samples without reward labels or samples generated by the dynamics model that are close to the expert data set, thereby making the driving strategy more robust and having better performance.

[0095] refer to Figure 7 The present invention also discloses a biased imitation learning system supported by a dynamics model, the system comprising:

[0096] A data acquisition module, used to acquire training samples of a driving simulation strategy model, wherein the training samples include an offline expert data set and a dynamic model data set;

[0097] The imitation learning module trains the driving imitation strategy model through the preset discriminator output weighted values;

[0098] The training module performs imitation training on the autonomous driving vehicle through the trained driving imitation strategy model.

[0099] Since a large amount of expert data is required for imitation learning, learning training can ensure that the learning effect reaches an ideal state, but the amount of expert data is small and the acquisition cost is high. Therefore, the present invention learns the offline expert data set through the dynamic model to achieve data amplification, and discriminates the data in the offline expert data set and the dynamic model data set through the discriminator, selects the real expert data, and assigns a higher weight, thereby achieving the amplification of expert data, covering the entire state space, and making the imitation strategy more sensitive to the unrewarded label samples or dynamic model generated samples close to the expert data set, so that the imitation strategy is more robust and has better performance.

[0100] The offline expert data set acquired by the data acquisition module includes expert data. The offline expert data set is learned through the dynamic model, and the data in the offline expert data set is learned and amplified. However, during the learning and data amplification process, pseudo expert data exists and cannot participate in the training of the driving imitation strategy model. Therefore, it is necessary to distinguish the data in the dynamic model data set. Learning through the dynamic model increases the amount of expert data, reduces the data acquisition cost, and helps to make the subsequent training of the driving imitation strategy model more accurate.

[0101] In the imitation learning module, the offline expert dataset and the dynamic model dataset are input into the discriminator, and the output is used to judge the authenticity of the sample, including:

[0102] The discriminator judges the input offline expert data set and the dynamic model data set to discriminate the real expert data;

[0103] The real expert data is a true value sample, and the weight of the real expert data is increased.

[0104] The data judged by the discriminator includes real expert data and pseudo-expert data;

[0105] Increase the weight for true value samples and reduce the weight coefficient for pseudo-expert data;

[0106] The discriminator outputs a weighted value based on the weight system.

[0107] The discriminator and the dynamics model are trained in adversarial coupling, and the results of the discriminator's judgment are fed back to the dynamics model, so that the dynamics model can adjust its learning strategy and learn more real data.

[0108] The discriminator and the dynamics model are trained in adversarial coupling. The discriminator selects real data with higher weights, allowing the dynamics model to learn more real data. The dynamics model also provides feedback to the discriminator, allowing the subsequent discriminator to more accurately discriminate real data.

[0109] In the present invention, a discriminator is set to perform comprehensive judgment on the data in the offline expert data set and the dynamic model data set, and the output weighted value selects the real expert data, thereby increasing the expert data and reducing the cost of obtaining the expert data.

[0110] Through the trained driving imitation strategy model, the autonomous driving vehicle is trained by imitation, including:

[0111] The driving imitation strategy model is trained by outputting weighted values ​​of the discriminator, the imitation learning strategy is corrected, and a corrected driving imitation strategy model is generated;

[0112] The modified driving imitation strategy model performs imitation learning on real expert data to imitate the functions corresponding to the expert data set.

[0113] The present invention provides a biased imitation learning system supported by a dynamics model. By introducing a system dynamics model and using the data generated by it to train the imitation strategy, the insufficient sample size of the expert data set is compensated, and to a certain extent, the compound error that may be caused by traditional imitation learning when the limited expert data set cannot cover the entire state space is alleviated. Biased imitation learning is achieved by introducing a discriminator, and different weights are calculated for samples from different data sets, so that the imitation strategy is more sensitive to samples without reward labels or samples generated by the dynamics model that are close to the expert data set, thereby making the driving strategy more robust and having better performance.

[0114] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute a biased imitation learning method supported by a dynamic model, the method comprising: obtaining a training sample set required for autonomous driving training, wherein the training sample comprises an offline expert data set and a dynamic model data set;

[0115] The driving imitation strategy model is trained by using the preset discriminator output weighted values;

[0116] The trained driving imitation strategy model is used to perform imitation training on the autonomous driving vehicle.

[0117] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0118] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute a biased imitation learning method supported by a dynamics model provided by the above methods, the method comprising: obtaining a training sample set required for autonomous driving training, wherein the training sample includes an offline expert data set and a dynamics model data set;

[0119] The driving imitation strategy model is trained by using the preset discriminator output weighted values;

[0120] The trained driving imitation strategy model is used to perform imitation training on the autonomous driving vehicle.

[0121] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program is implemented to execute a biased imitation learning method supported by a dynamics model provided by the above methods, the method comprising: obtaining a training sample set required for autonomous driving training, wherein the training sample comprises an offline expert data set and a dynamics model data set;

[0122] The driving imitation strategy model is trained by using the preset discriminator output weighted values;

[0123] The trained driving imitation strategy model is used to perform imitation training on the autonomous driving vehicle.

[0124] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A biased imitation learning method supported by a dynamics model, characterized in that: include: Acquire a training sample set required for autonomous driving training, wherein the training sample includes an offline expert data set and a dynamic model data set; The driving imitation strategy model is trained by using the preset discriminator output weighted values; Through the trained driving imitation strategy model, the autonomous driving vehicle is trained by imitation; Among them, the driving imitation strategy model is trained by outputting weighted values ​​of the preset discriminator, specifically including: The offline expert data set and the dynamic model data set are input into the discriminator, and the authenticity of the judged samples is output; The driving imitation strategy model is trained using the weighted values ​​output by the discriminator to learn expert data; Wherein, the dynamic model data set is generated by learning the offline expert data set through the dynamic model; The dynamic model data set is generated by learning the offline expert data set through the dynamic model, and specifically includes: Inputting offline expert datasets into the kinetic model; The kinetic model is learned based on the expert data in the offline expert data set, and data amplification is performed to generate a kinetic model data set; The dynamics model data set includes real expert data and pseudo expert data.

2. The biased imitation learning method supported by a dynamic model according to claim 1, characterized in that: The offline expert data set and the dynamic model data set are input into the discriminator, and the output judges the authenticity of the sample, specifically including: The discriminator judges the input offline expert data set and the dynamic model data set to discriminate the real expert data; The real expert data is a true value sample, and the weight of the real expert data is increased.

3. The biased imitation learning method supported by a dynamic model according to claim 2, characterized in that: The real expert data is a true value sample, and the weight of the real expert data is increased, specifically including: The data judged by the discriminator includes real expert data and pseudo-expert data; Increase the weight for true value samples and reduce the weight coefficient for pseudo-expert data; The discriminator outputs a weighted value based on the weight system.

4. The biased imitation learning method supported by a dynamic model according to claim 3, characterized in that: The discriminator is trained in adversarial coupling with the dynamics model, and the result of the discriminator's judgment is fed back to the dynamics model, so that the dynamics model adjusts its learning strategy and learns more real data.

5. The biased imitation learning method supported by a dynamic model according to claim 1, characterized in that: The imitation training of the autonomous driving vehicle by the trained driving imitation strategy model specifically includes: The driving imitation strategy model is trained by outputting weighted values ​​of the discriminator, the imitation learning strategy is corrected, and a corrected driving imitation strategy model is generated; The modified driving imitation strategy model performs imitation learning on real expert data to imitate the functions corresponding to the expert data set.

6. A biased imitation learning system supported by a dynamics model, characterized in that: The system comprises: A data acquisition module, which acquires a training sample set required for autonomous driving training, wherein the training sample includes an offline expert data set and a dynamic model data set; The imitation learning module trains the driving imitation strategy model through the preset discriminator output weighted values; The training module uses the trained driving imitation strategy model to perform imitation training on the autonomous driving vehicle; Among them, the driving imitation strategy model is trained by outputting weighted values ​​of the preset discriminator, specifically including: The offline expert data set and the dynamic model data set are input into the discriminator, and the authenticity of the judged samples is output; The driving imitation strategy model is trained using the weighted values ​​output by the discriminator to learn expert data; Wherein, the dynamic model data set is generated by learning the offline expert data set through the dynamic model; The dynamic model data set is generated by learning the offline expert data set through the dynamic model, and specifically includes: Inputting offline expert datasets into the kinetic model; The kinetic model is learned based on the expert data in the offline expert data set, and data amplification is performed to generate a kinetic model data set; The dynamics model data set includes real expert data and pseudo expert data.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, a biased imitation learning method supported by a dynamics model as described in any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements a biased imitation learning method supported by a dynamics model as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • Automatic driving training method and device, equipment, storage medium and program product

    CN114372501A

  • Planning in mobile robots

    WO2021152047A1