Vehicle autonomous driving operation control using planning model and cognitive learning model

Abstract

A method for controlling autonomous driving operations includes establishing respective independent models for performing planning and learning in autonomous driving operations, the independent models including a vehicle planning model and a cognitive learning model. A semantic layer is generated to act as a bridge between the cognitive learning model and the vehicle planning model, the semantic layer including a first data adapter and a second data adapter. The method includes converting, via the first data adapter, real traffic data into a respective equivalent abstract representation, such that the cognitive learning model can use it to generate a humanized reward model. The method includes determining, via the vehicle planning model, a trajectory plan based in part on the humanized reward model. A vehicle has a controller that performs autonomous driving operations based in part on the trajectory plan.

Description

Technical Field

[0001] This disclosure generally relates to a method and system for controlling the autonomous driving operation of a vehicle. Background Technology

[0002] Advanced driver assistance systems (ADAS) and autonomous vehicles typically incorporate various systems for effective operation, such as blind spot information systems, lane departure warning systems, and adaptive cruise control systems. The availability of human driving data assisting these systems is expected to increase in quantity, diversity, and richness. However, due to the complexity of planning models and the various representations used by planners, drivers, and other factors, using this data to control the autonomous operation of the vehicle is a significant and challenging process. Summary of the Invention

[0003] This paper discloses a method for controlling the autonomous driving operation of a vehicle having a controller with a processor and tangible non-transitory memory. The method includes establishing corresponding independent models for performing planning and learning during autonomous driving operations, the corresponding independent models including a vehicle planning model and a cognitive learning model. A semantic layer is developed or generated to act as a bridge between the cognitive learning model and the vehicle planning model, the semantic layer including a first data adapter and a second data adapter.

[0004] The method includes converting real-world traffic data into a corresponding equivalent abstract representation via a first data adapter, enabling a cognitive learning model to generate a human-centered reward model. The method also includes determining trajectory planning, partially based on the human-centered reward model, via a vehicle planning model. The controller then performs autonomous driving operations, partially based on the trajectory planning.

[0005] In some embodiments, a Markov decision process model is employed in the semantic layer. The method may include obtaining pairs of state and action variables of the vehicle via a vehicle planning model. The pairs of state and action variables are then converted into corresponding equivalent abstract representations in the semantic layer via a second data adapter. These equivalent abstract representations are then transferred to a humanized reward model. The method may also include incorporating a third data adapter in the semantic layer and sending relevant data from the humanized reward model to the vehicle planning model via the third data adapter.

[0006] In some embodiments, a dual-alternative forced selection model or a multi-alternative forced selection model is incorporated into the cognitive learning model. In other embodiments, an inverse reinforcement learning model is incorporated into the cognitive learning model. The cognitive learning model and the semantic layer may be hosted on off-premises cloud computing services.

[0007] This method may include using quantile binning in a cognitive learning model to discretize the state features of a predetermined group of states into action variables. This method may also include incorporating a neural network into the cognitive learning model, wherein the neural network has an activation function in its output layer that predicts a multinomial probability distribution.

[0008] This paper discloses a system for controlling autonomous driving operations in a vehicle. The system includes a controller operatively connected to the vehicle, the controller having a processor and a tangible, non-transitory memory on which instructions are recorded. A computing unit communicates with the controller and has corresponding independent models for performing planning and learning during autonomous driving operations, the corresponding independent models including a vehicle planning model and a cognitive learning model.

[0009] The computational unit includes a semantic layer adapted to act as a bridge between a cognitive learning model and a vehicle planning model. The semantic layer includes a first data adapter and a second data adapter. The first data adapter is configured to convert real-world traffic data into a corresponding equivalent abstract representation, which the cognitive learning model can use to generate a human-centered reward model. The vehicle planning model is adapted to determine trajectory planning in part based on the human-centered reward model. The controller is adapted to perform autonomous driving operations in part based on the trajectory planning.

[0010] The foregoing features and advantages of this disclosure, as well as other features and advantages, will become apparent from the following detailed embodiments of the best specification for carrying out this disclosure when taken in conjunction with the accompanying drawings. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of a system used to control the autonomous driving operation in a vehicle;

[0012] Figure 2 It is used in operating systems Figure 1 A flowchart of the system's method; and

[0013] Figure 3 It is shown Figure 1 A schematic partial diagram of an example scenario faced by the vehicle.

[0014] Representative embodiments of this disclosure are illustrated by way of non-limiting example in the accompanying drawings and are described in further detail below. However, it should be understood that novel aspects of the invention are not limited to the specific forms illustrated in the foregoing figures. Rather, this disclosure will cover modifications, equivalents, combinations, sub-combinations, arrangements, groupings, and alternatives that fall within the scope of this disclosure, for example, as covered by the appended claims. Detailed Implementation

[0015] Referring to the accompanying drawings, the same reference numerals denote the same parts. Figure 1A system 10 for controlling autonomous driving operations D in vehicle 12 is schematically illustrated. Vehicle 12 may include, but is not limited to, passenger cars, SUVs, light trucks, heavy vehicles, minivans, buses, transport vehicles, bicycles, mobile robots, agricultural implements (e.g., tractors), sports-related equipment (e.g., golf carts), boats, airplanes, and trains. Vehicle 12 may be an electric vehicle, which may be fully electric or partially electric. It should be understood that vehicle 12 may take many different forms and have additional components.

[0016] refer to Figure 1 The vehicle 12 includes a controller C, which has at least one processor P and at least one memory M (or a non-transitory tangible computer-readable storage medium) on which instructions can be recorded. A computing unit 14 communicates with the controller C and includes a learning phase 16 and a planning phase 18.

[0017] System 10 employs independent models for performing vehicle planning and learning in autonomous driving operation D. Computational unit 14 includes a cognitive-based learning model 20 in learning phase 16 and a vehicle planning model 22 in planning phase 18; the two models are independent of each other. The cognitive-based model is an estimate of human cognitive processes for the purpose of understanding and prediction. An example flowchart of the method 100 for operating system 10 is shown in... Figure 2 It is shown in the figure and described below.

[0018] Current methods use the same problem representation or model for both the learning and planning processes. System 10 achieves effective learning based on human driving data by separating the model used for learning from the model used for planning. The vehicle planning model 22 operates in a general state / action space, supports a wide range of scenarios, and changes frequently during the development lifecycle. In some embodiments, the vehicle planning model 22 is non-cognitive. In contrast, the cognitive-based learning model 20 (hereinafter referred to as the cognitive learning model 20) uses fewer state and action representations and is simpler. Maintaining two independent representations for the learning and planning processes allows the vehicle planning model 22 and reward learning to progress in parallel throughout the development lifecycle.

[0019] System 10 provides a framework for the interface between cognitive learning model 20 and vehicle planning model 22, and allows for offline learning of the cognitive process. Computational unit 14 includes a semantic layer 24 adapted to act as a bridge between cognitive learning model 20 and vehicle planning model 22. As described below, semantic layer 24 includes a first data adapter A1 configured to convert real traffic / vehicle data (e.g., from database 26) into a corresponding equivalent abstract representation, which the cognitive learning model 20 can use to generate a humanized reward model 30.

[0020] refer to Figure 1 Semantic layer 24 includes a second data adapter A2 for transforming pairs of state variables and action variables (paired state variables and action variables) into corresponding equivalent abstract representations in semantic layer 24, wherein the corresponding abstract representations are transmitted to humanized reward model 30. Semantic layer 24 translates the reward function learned by cognitive learning model 20 via a third data adapter A3 for transmission to vehicle planning model 22.

[0021] refer to Figure 1 The vehicle planning model 22 is adapted to determine trajectory planning 32 in part based on the humanized reward model 30. The controller C is adapted to perform autonomous driving operations D in part based on trajectory planning 32. Trajectory planning 32 determines appropriate steps for navigating traffic signs, intersections, anticipated and unexpected road and traffic conditions. Scenarios evaluated by system 10 include roundabouts, lane changes, unprotected left turns, overtaking and merging scenarios, pedestrian behavior, etc. Vehicle 12 may include multiple sensors (not shown) for perceiving the surrounding environment. Data from the sensors is transmitted to the controller C for use by the vehicle planning model, etc. Vehicle sensors may include various types of technologies available to those skilled in the art, such as radar, cameras, acoustic or lidar units, and inertial measurement units.

[0022] In some embodiments, computing unit 14 is remotely located as part of an off-board cloud computing service, referred to herein as cloud unit 34. Cloud unit 34 may include one or more servers hosted on the Internet to store, manage, and process data, and is maintained by an organization such as, for example, a research institute or company. Cloud unit 34 may be managed at least in part by personnel at various locations. Cloud unit 34 may be a private or public source of information maintained by an organization such as a research institute, company, university, and / or hospital.

[0023] In other embodiments, computing unit 14 may run on mobile application 36 embedded in a smart device belonging to a user of vehicle 12. Mobile application 36 may be linked to vehicle 12, for example, as part of a vehicle infotainment system. Circuitry and components of mobile application 36 (“app”) available to those skilled in the art may be employed.

[0024] refer to Figure 1 The controller C can be configured to communicate with the computing unit 14 via the wireless network 40. Figure 1 The wireless network 40 can be a short-range network or a long-range network. The wireless network 40 can be a communication bus (BUS), which can be in the form of a Serial Controller Area Network (CAN-BUS). The wireless network 40 may include Bluetooth. TMWireless local area networks (LANs), wireless metropolitan area networks (MANs), or wireless wide area networks (WANs) can connect and link multiple devices using wireless distribution methods. Other types of connections are also possible.

[0025] Now for reference Figure 2 An example flowchart of a method 100 for controlling autonomous driving operations is shown. Method 100 can be embodied in a file stored in... Figure 1 The method 100 contains computer-readable code or instructions that are executed, at least in part, on and / or by the controller C and / or computing unit 14. Method 100 does not need to be applied in the specific order described herein. Furthermore, it should be understood that some boxes or steps may be omitted. Method 100 may be performed in real time, continuously, systematically, sporadically, and / or at regular intervals (e.g., every 10 milliseconds) during normal and ongoing operation of the vehicle 12.

[0026] according to Figure 2 In box 110, controller C is programmed to set up cognitive learning model 20. The cognitive model is an estimate of human cognitive processes for the purpose of understanding and prediction. Cognitive learning model 20 uses small representations of states and actions and is designed to capture human behavior and the problems that influence it. Cognitive learning model 20 may employ quantile-binning to discretize state features (from a predetermined set of states) into action variables.

[0027] The specific type of model used by cognitive learning model 20 can vary depending on the application at hand. In one embodiment, cognitive learning model 20 includes an automatic abstraction method. In another embodiment, cognitive learning model 20 includes a dimensionality reduction method. Cognitive learning model 20 may include an inverse reinforcement learning model. In another embodiment, cognitive learning model 20 may include a dual-alternative forced selection model or a multi-alternative forced selection model.

[0028] In another embodiment, the cognitive learning model 20 incorporates a neural network with an activation function that predicts a multinomial probability distribution in the output layer. As those skilled in the art will understand, the neural network is designed to recognize patterns from real-world data (e.g., images, sounds, text, time series, etc.), translate them or convert them into numerical form, and embed them in vectors or matrices. The neural network can employ a deep learning graph to match the input vector x with the output vector y. In other words, the neural network learns an activation function f such that f(x) maps to y. The training process enables the neural network to associate the appropriate activation function f(x) used to transform the input vector x with the output vector y. In the case of a simple linear regression model, two parameters are learned: bias and slope. The bias is the level of the output vector y when the input vector x is 0, and the slope is the rate by which the predicted increase or decrease in the output vector y is achieved for each unit increase in the input vector x.

[0029] Progressing to Figure 2 In block 120, controller C is programmed to establish semantic layer 24, including first, second, and third data adapters A1, A2, and A3. Block 120 includes sub-blocks 122, 124, and 126. Semantic layer 24 can be configured to employ a Markov Decision Process (MDP). As understood by those skilled in the art, a Markov Decision Process is a discrete-time stochastic control process that provides a mathematical framework for decision modeling. Markov Decision Process (MDP) models are used to learn rewards from human drivers. Semantic layer 24 will store the original Markov Decision Process variables (e.g., ...<S,A,T,R> ) is converted into an abstract representation (e.g., ).

[0030] According to subframe 122, method 100 includes converting real-world traffic data (e.g., from database 26) into a corresponding equivalent abstract representation via a first data adapter A1, enabling cognitive learning model 20 to use it to generate a humanized reward model 30. Cognitive learning model 20 creates abstract concepts of tasks, states, and actions, which facilitates learning from available data. The first adapter A1 translates temporal data of human behavior into a semantic space. The real-world data can be raw visual or tracked, and the first adapter A1, based on a model selected for cognitive learning model 20, converts sequential data of states and human behavior into abstract concepts defined by the cognitive semantic space.

[0031] According to subframe 124, method 100 includes obtaining state and action variable pairs of vehicle 12 via vehicle planning model 22. Via second data adapter A2, the state and action variable pairs are converted into corresponding equivalent abstract representations in semantic layer 24. These equivalent abstract representations are then sent to humanized reward model 30. The reward function R(s,a)=∑βiFi(s,a) can be approximated as… Where s is the state vector and a is the action variable. Semantic MDP aims to learn human behavior simply and more effectively. Rewards can be represented as numerical scores for each potential action.

[0032] According to subframe 126, method 100 includes sending relevant data from the humanized reward model 30 to the vehicle planning model 22 via a third data adapter A3. In other words, the learned knowledge (e.g., reward function, probability distribution, etc.) is sent to the vehicle planning model 22.

[0033] Advance to Figure 2 In box 130, controller C is programmed to control the autonomous driving operation D of vehicle 12 in part based on trajectory planning 32, generating optimized trajectory planning based on vehicle state and dynamics, data related to neighboring vehicles and road conditions, sensor data, and training provided by cognitive learning model 20. The recommended travel path of vehicle 12 can be updated based on modeling and real-time data.

[0034] Now for reference Figure 3 This illustrates a lane merging scenario faced by vehicle 12. Figure 3 Vehicle 12 is located in lane 214, near lane 216. Lane 214 and lane 216 merge into a single road 218 at a merging trajectory location 220. Merging trajectory location 220 can occur when two lanes physically merge into one or in unstructured traffic scenarios. (Reference) Figure 3 Vehicle 12 is near one or more adjacent vehicles 224. Adjacent vehicles 224 may be in the same lane as vehicle 12 or in adjacent or nearby lanes, such as car 226 in the first lane 214 and cars 228, 230, 232 and 234 in the second lane 216.

[0035] Merging trajectory position 220 defines the gap sequence between vehicle pairs for each time step. The default gap (t) is the pair of vehicles (leading / lagging) that vehicle 12 is expected to merge. Features tracked by vehicle sensors may include the corresponding gap length between adjacent vehicles 224 and vehicle 12, the variation in the corresponding gap length between each adjacent vehicle 224, the acceleration of adjacent vehicles 224 in the gap, the corresponding collision time, and the relative speed of the leading vehicle (e.g., car 226) in the same lane. Under a set of assumptions such as constant speed or acceleration, the action space can be categorized using the following options: deceleration, maintaining the status quo, and acceleration.

[0036] In one example, cognitive learning model 20 includes a softmax function, which can be used as an activation function in the output layer of a neural network model predicting a multinomial probability distribution. The output of the softmax function is a vector with the probability of each possible outcome. A first adapter A1 transforms a real-world recorded trajectory (e.g., from NGSIM i80 data) into a sequence of gap-ordered and take / skip actions. A second adapter A2 transforms the planner scene state representation into sequences classified as gap-ordered and take / skip. A third adapter A3 transforms the distribution of the normalized probability of actions (take|state) into a single reward number, such as taking the mean or the 25th percentile of the distribution.

[0037] Another example scenario faced by vehicle 12 is lane changing. Here, the cognitive learning model 20 may include inverse reinforcement learning using a maximum entropy algorithm, as understood by those skilled in the art. The first adapter A1 of the semantic layer 24 transforms the human driving record and state representation of the lane change by the human driver (as a set of relative speed and distance sequences). The second adapter A2 transforms the planner's scenario state representation into a state code indicating whether a lane change occurred. The third adapter A3 transforms the approximate reward (state, action) into the planner's reward.

[0038] The first feature tracked by vehicle sensors includes the relative speeds of the leading vehicle and potential leading vehicles, and the second feature may include the relative speeds of vehicle 12 and adjacent vehicles. The third feature may include the distances of vehicle 12 from potential following and potential leading vehicles. The first possible driving action includes lane keeping, and the second possible driving action is lane changing (action / space). Here, selecting a trajectory or possible driving action may include implementing a Markov decision. The reward function outputs a corresponding negative or positive reward for each driving action, incorporating learning from human behavior.

[0039] In summary, System 10 (via the execution of Method 100) adds a “human factor” to vehicle planning. System 10 decouples the vehicle planning representation from the learned / trained representation when learning the reward function from human data using Semantic Layer 24. This allows for the independent design of the cognitive learning model and the planning model. Semantic Layer 24 performs transformation / bridging between the two representations by implementing three adapters that convert (state, action) pairs from traffic data or vehicle planning data into equivalent abstract representations that can be used and evaluated by the cognitive learning model 20. System 10 enables the learning from human behavior by leveraging the compactness and semantics of the cognitive model, while also enabling the integration of the learned knowledge into a general model used by the vehicle planning model 22.

[0040] Figure 1 The controller C can be an integrated part of other controllers of vehicle 12, or a separate module operatively connected to other controllers of vehicle 12. In one example, the controller C can be an electronic control unit (ECU) of vehicle 12. The memory M can store the controller executable instruction set, and the processor P can execute the controller executable instruction set stored in the memory M.

[0041] Figure 1 The controller C includes computer-readable media (also known as processor-readable media), including non-transitory (e.g., tangible) media that participate in providing data (e.g., instructions) that can be read by a computer (e.g., by the computer's processor). Such media can take many forms, including but not limited to non-volatile and volatile media. Non-volatile media can include, for example, optical discs or magnetic disks, and other persistent storage. Volatile media can include, for example, dynamic random access memory (DRAM) that can constitute main memory. Such instructions can be transmitted by one or more transmission media, including coaxial cables, copper wires, and optical fibers, including wires containing a system bus coupled to the computer's processor. Some forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, other magnetic media, CD-ROMs, DVDs, other optical media, physical media with perforated patterns, RAM, PROMs, EPROMs, FLASH-EEPROMs, other memory chips or cassette tapes, or other media from which a computer can read.

[0042] The lookup tables, databases, data warehouses, or other data storage devices described herein can include various mechanisms for storing, accessing, and retrieving various types of data, including hierarchical databases, a set of files in a file system, application databases in proprietary formats, relational database energy management systems (RDBMS), etc. Each such data storage device can be included within a computing device employing a computer operating system such as one of the aforementioned operating systems and can be accessed via a network in one or more of various ways. File systems can be accessed from the computer operating system and can include files stored in various formats. In addition to languages ​​used for creating, storing, editing, and executing stored procedures (such as PL / SQL as described above), RDBMS can also employ Structured Query Language (SQL).

[0043] Figure 2 The flowcharts illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, including one or more executable instructions for implementing a specified logical function. It should also be noted that each block in a block diagram and / or flowchart illustration, and combinations of blocks in block diagrams and / or flowchart illustrations, may be implemented by a system based on dedicated hardware or a combination of dedicated hardware and computer instructions that performs the specified function or action. These computer program instructions may also be stored in a computer-readable medium that may instruct a controller or other programmable data processing means to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of art including instructions for implementing the functions / actions specified in the flowchart and / or block diagram blocks.

[0044] The numerical values ​​of parameters (e.g., quantities or conditions) in this specification (including the appended claims) should be understood to be modified by the term “about” in each respective instance, regardless of whether “about” actually appears before the numerical value. “About” indicates that the numerical value allows for some slight imprecision (having some degree of closeness to precision in that value; about or reasonably close to that value; close). If the imprecision provided by “about” is not otherwise understood in this ordinary sense in the art, then “about” as used herein at least indicates variations that may arise from common methods of measuring and using these parameters. Furthermore, the disclosure of ranges includes the disclosure of each value throughout the range and further subdivisions of the range. Each value within the range and the endpoints of the range are disclosed herein as separate embodiments.

[0045] A detailed specification and accompanying drawings or figures support and describe this disclosure, but the scope of this disclosure is defined only by the claims. While some best modes and other embodiments for carrying out the claimed disclosure have been described in detail, various alternative designs and embodiments exist for practicing the disclosure as defined in the appended claims. Furthermore, features of the embodiments shown in the drawings or the various embodiments mentioned in this specification are not necessarily to be construed as independent embodiments. Rather, each feature described in one example of an embodiment may be combined with one or more other desired features from other embodiments to produce other embodiments not described in words or by reference to the drawings. Therefore, such other embodiments fall within the framework of the appended claims.

Claims

1. A method for controlling the autonomous driving operation of a vehicle, the vehicle having a controller, the controller having a processor and a tangible non-transitory memory, the method comprising: Establish corresponding independent models for performing planning and learning in the autonomous driving operation, the independent models including a vehicle planning model and a cognitive learning model; A semantic layer is generated to act as a bridge between the cognitive learning model and the vehicle planning model, the semantic layer including a first data adapter and a second data adapter; Obtain real traffic data; The real traffic data is converted into a corresponding equivalent abstract representation via the first data adapter, enabling the cognitive learning model to use the equivalent abstract representation to generate a humanized reward model. Trajectory planning is determined in part based on the humanized reward model, using the vehicle planning model. as well as The autonomous driving operation is performed in part based on the trajectory planning via the controller.

2. The method according to claim 1, further comprising: The vehicle's state variables and action variables are obtained through the vehicle planning model; and through the second data adapter, the state variables and action variables are converted into corresponding equivalent abstract representations in the semantic layer, and the corresponding equivalent abstract representations are transmitted to the humanized reward model.

3. The method according to claim 2, further comprising: Incorporate the third data adapter into the semantic layer; as well as The corresponding data is transmitted from the humanized reward model to the vehicle planning model via the third data adapter.

4. The method according to claim 2, further comprising: A Markov decision process model is used at the semantic layer.

5. The method according to claim 2, further comprising: The dual-alternative forced selection model or the multi-alternative forced selection model is incorporated into the cognitive learning model.

6. The method according to claim 2, further comprising: The inverse reinforcement learning model is incorporated into the cognitive learning model.

7. The method according to claim 2, further comprising: In the cognitive learning model, quantile binning is used to discretize the state features of a predetermined group of states into action variables.

8. The method according to claim 2, further comprising: The neural network is incorporated into the cognitive learning model, wherein the neural network has an activation function in its output layer that predicts a multinomial probability distribution.

9. The method according to claim 2, further comprising: The cognitive learning model and the semantic layer are hosted in an off-board cloud computing service.

10. A system for controlling automated driving operations in a vehicle, the system comprising: A controller operatively connected to the vehicle, the controller having a processor and a tangible, non-transitory memory thereon on which instructions are recorded; A computing unit that communicates with the controller and has corresponding independent models for performing planning and learning in the autonomous driving operation, the independent models including a vehicle planning model and a cognitive learning model; The computing unit includes a semantic layer, which is adapted to act as a bridge between the cognitive learning model and the vehicle planning model. The semantic layer includes a first data adapter and a second data adapter. The first data adapter is configured to convert real traffic data into a corresponding equivalent abstract representation, so that the cognitive learning model can use the equivalent abstract representation to generate a humanized reward model. The vehicle planning model is adapted to determine trajectory planning in part based on the humanized reward model; and The controller is adapted to perform the autonomous driving operation in part based on the trajectory planning.

Images