A migration-based autonomous lane-changing method and system

Abstract

The present invention relates to a migration-based autonomous lane-changing method and system, belonging to the field of vehicle intelligent driving technology. The present invention discretizes the vehicle position coordinates to obtain a state grid space, assigns values to the state grid space, then uses a random walk strategy to extract the traffic state in the assigned state grid space. A temporal difference learning algorithm is used to determine the transition relationship between different traffic states to obtain a state transition matrix. Then, a reward grid space based on scene migration knowledge and an observation area is established. Next, a lane-changing preference evaluation value is determined after generating a reward matrix based on the reward grid space. Finally, the lane-changing preference evaluation value is used as the planning basis for the vehicle's driving trajectory, and a vehicle control signal from a lane-changing motion primitive library is selected to achieve safe and efficient autonomous lane changing for the vehicle, thereby solving the problems of poor versatility and complex calculations in existing vehicle lane-changing technologies.

Description

Technical Field

[0001] The present invention relates to the field of vehicle intelligent driving technology, and in particular to a migration-based autonomous lane changing method and system. Background Art

[0002] With the continuous increase in car ownership and the continuous advancement of intelligent driving technology, intelligent vehicles are gradually entering the public eye. Among them, autonomous lane-changing systems are increasingly attracting the attention of researchers around the world. However, current global research on autonomous lane-changing systems still has certain shortcomings.

[0003] For intelligent driving systems, autonomous lane changing is one of the most dangerous and complex driving scenarios. However, existing autonomous lane changing systems for intelligent vehicles, based on traditional planning methods such as the A* algorithm and Rapid Random Tree (RRT), generally suffer from poor versatility and high algorithmic complexity. Autonomous lane changing systems based on traditional reinforcement learning methods also have poor generalization capabilities and lack the ability to adapt to changing scenarios, making them difficult to apply in real-world driving environments. Summary of the Invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a migration-based autonomous lane changing method and system.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A migration-based autonomous lane changing method, comprising:

[0007] Get the vehicle position coordinates in the original scene;

[0008] Discretizing the vehicle position coordinates to construct a state grid space;

[0009] Performing value assignment processing on the state grid space;

[0010] A random walk strategy is used to extract the traffic state in the state grid space after the assignment;

[0011] The temporal difference learning algorithm is used to determine the transition relationship between different traffic states and obtain the state transition matrix;

[0012] Establishing scene migration knowledge based on the state transition matrix;

[0013] Establish reward grid space for the observation area;

[0014] generating a reward matrix based on the reward grid space;

[0015] Obtaining a lane change preference evaluation value according to the state transition matrix and the reward matrix;

[0016] Build a lane-changing motion primitive library based on vehicle control signal data;

[0017] The lane-changing preference evaluation value is used as a basis for planning a vehicle driving trajectory, and a vehicle control signal in the lane-changing motion primitive library is selected to achieve autonomous lane changing of the vehicle.

[0018] Preferably, the assigning processing to the state grid space specifically includes:

[0019] The grids occupied by the obstacles in the state grid space are assigned a first preset value, and the grids other than the grids occupied by the obstacles in the state grid space are assigned a second preset value.

[0020] Preferably, when the first preset value is 1, the second preset value is 0.

[0021] Preferably, the temporal difference learning algorithm is used to determine the transition relationship between different traffic states to obtain a state transition matrix, which specifically includes:

[0022] The temporal difference learning algorithm is used to update the transfer relationship between different traffic states and obtain the updated transfer relationship;

[0023] A state transition matrix is formed based on the updated transition relationship.

[0024] Preferably, the size of the reward grid space is the same as the size of the state grid space.

[0025] Preferably, generating a reward matrix based on the reward grid space specifically includes:

[0026] The reward value of the grid where the lane change target point is located in the reward grid space is set to a first preset value, the reward value of the grid occupied by other vehicles in the reward grid space except the target vehicle is set to a second preset value, and the reward value of the grid in the reward grid space except the grid occupied by the obstacle is set to a third preset value.

[0027] Preferably, the third preset value is -1.

[0028] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0029] The migration-based autonomous lane-changing method provided by the present invention realizes the selection of motion primitives and automatic lane-changing control of the vehicle through modeling, extraction and migration of scene migration knowledge, thereby achieving safe and efficient autonomous lane-changing of the vehicle, and solving the problems of poor versatility and complex calculation in existing vehicle lane-changing technologies.

[0030] In addition, the present invention also provides a migration-based autonomous lane-changing system, which includes:

[0031] A memory for storing a computer software program; the computer software program is used to implement the above-mentioned migration-based autonomous lane changing method;

[0032] A processor is connected to the memory and is used to execute the computer software program.

[0033] Since the technical effects achieved by the migration-based autonomous lane changing system provided by the present invention are the same as the technical effects achieved by the migration-based autonomous lane changing method provided above, they will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A flow chart of the migration-based autonomous lane changing method provided by the present invention;

[0036] Figure 2 A schematic diagram of a state grid space provided by an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of the state grid space after assignment provided by an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of a lane-changing scenario provided by an embodiment of the present invention;

[0039] Figure 5 A logic flow chart for implementing the migration-based autonomous lane changing method provided by an embodiment of the present invention;

[0040] Figure 6 This is a schematic structural diagram of the migration-based autonomous lane changing system provided by the present invention. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] The purpose of the present invention is to provide a migration-based autonomous lane-changing method and system, which can control the vehicle through the learning and migration of scene knowledge and the selection of motion primitives to complete safe and efficient autonomous lane changing.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] like Figure 1 As shown, the present invention provides a migration-based autonomous lane changing method, comprising:

[0045] Step 100: Obtain the vehicle position coordinates in the original scene.

[0046] Step 101: Discretize the vehicle position coordinates to construct a state grid space. In order to avoid the infinity of the state space caused by continuous states, the present invention uses a discretized state space to model the vehicle's observation area. Therefore, it is necessary to use grid coordinates to mark the position in the state space, and then use a migration algorithm (Successor Representation, SR) to represent the state transition relationship in the state space. Based on this, in this step, for the problem of autonomous lane changing on a straight road, the state grid space in the scene involved is defined and constructed, and the state grid space constructed for the front vehicle's field of view is as follows: Figure 2 In this state grid space, the position coordinates of the vehicle in front are discretized into grid coordinates, which will serve as the source scene for the knowledge to be transferred in the present invention.

[0047] Step 102: Assign values to the state grid space. For example, assign the values of the grids occupied by obstacles (such as other vehicles) in the state grid space to -1, and divide the remaining areas in the state grid space into passable areas and assign them to 0. The state grid space after assignment is as follows: Figure 3 shown.

[0048] Step 103: A random walk strategy is used to extract the traversable state in the assigned state grid space. In this invention, the random walk strategy means that, in this state grid space, upon each state transition (i.e., agent movement), the agent (vehicle) randomly selects an equally likely adjacent traversable grid, and upon completing the state transition, moves to the next adjacent grid. Traversable grids are all traversable areas excluding the walls at the edge of the state grid space and the obstacles marked in the state grid space, i.e., grids assigned a value of 0.

[0049] Step 104: Use the time difference learning algorithm to determine the transition relationship between different traffic states and obtain the state transfer matrix. Specifically, to illustrate the specific training process of the state transfer matrix M, assume that a line consisting of position p The random walk trajectory from the start is used to train the state transfer matrix M. p In the trajectory of departure, the position p’ The number of visits after discount is recorded as M ( p , p’ ), stored in the state transfer matrix M. The values of each element in the state transfer matrix M shown in Formula 1 are updated through training until convergence.

[0050] (1)

[0051] in, p 1, p 2,... p N Represents each state in the state grid space, each element value in the matrix M ( p , p’ ) represents the transition relationship between various states, and its specific definition is shown in formula (2).

[0052] (2)

[0053] Among them, E refers to the mathematical expectation, Express p t and p’ Is it the same state to judge, when p t = p’ hour, ,otherwise . t Refers to the time step. Starting from the initial position, the agent completes a state transfer at each time step. γ is the decay rate. In mathematical terms, the transfer relationship M ( p , p’ ) represents the initial position p Start by looking at each subsequent point on the trajectory p’ The sum of the number of visits after exponential decay.

[0054] Using the temporal difference learning algorithm, transfer relations M ( p , p’ ) can be iteratively updated by the iterative formula shown in formula (3).

[0055] (3)

[0056] in, η is the learning rate, is the Successor Prediction Error, which represents the deviation between the predicted reward and the actual reward in differential learning and is defined by the following formula (4).

[0057] (4)

[0058] After sufficient training, all transition states in the state grid space M ( p , p’ ) will converge to a fixed value to form the state transfer matrix M.

[0059] Step 105: Establish scene migration knowledge based on the state transition matrix. The state transition matrix M obtained in step 104 records the state transition relationship between all grids in the state grid space. For grids that cannot perform state transition, the transition relationship M ( p , p’ ) is 0, that is, the state transfer matrix M records the non-transferable relationship between non-transferable states. For grids that can perform state transfer (such as between passable areas), the transfer relationship M ( p , p’ ) is greater than 0, and the transfer relationship between adjacent grids M ( p , p’ ) is larger, and the transfer relationship between grids that are farther apart is M ( p , p’ ) is small, meaning the state transfer matrix M records the transferable relationships between transferable states, and the size of the state transfer matrix M records the ease of transfer. Therefore, the state transfer matrix M can serve as a representation of the source scene, becoming the understanding of the source scene gained after learning, and thus becoming scene transfer knowledge.

[0060] Step 106: Establish a reward grid space for the observed area. This reward grid space is established to transfer the scene understanding from step 105 to the actual traffic scene, define the actual driving task, and complete the autonomous lane change. In the actual traffic scene, a reward grid space is established for the vehicle's observable area. To better transfer the knowledge from the scene transfer knowledge established in step 105, the size of the reward grid space needs to be the same as the source scene state grid space from step 101.

[0061] Step 107: Generate a reward matrix based on the reward grid space. For example, set the reward value for cells in the reward grid space occupied by other vehicles to -1, indicating that they are impassable or will trigger a collision penalty. Set the reward value for cells in the reward grid space where the lane change target is located to 1, indicating that reaching the target will result in a reward. Set the reward value for the remaining traversable areas in the reward grid space to 0. Store these reward values in a matrix of the same dimensions as the reward grid space, thus generating the reward matrix R.

[0062] Step 108: Obtain the lane change preference evaluation value based on the state transition matrix and the reward matrix. Specifically, based on the state transition matrix M generated in step 104 and the reward matrix R generated in step 107, this step 109 uses formula (5) to superimpose the two and generate the lane change preference evaluation value for the target scene. Specifically, in order to achieve knowledge transfer from the source scene to the target scene, it is necessary to try to combine the scene representation learned in the source scene with the reward space of the target scene. Therefore, formula (6) is used to calculate the grid position in the target scene. p lane-changing preference evaluation value.

[0063] (5)

[0064] (6)

[0065] Where, R ( p’ ) represents the position in the reward grid space p’ The reward value. V ( p ) represents the calculated grid position p lane-changing preference evaluation value.

[0066] By calculating the lane change preference evaluation values for all grid locations in the target scene, we form the evaluation matrix V for the target scene. For all grid locations, a larger lane change preference evaluation value means that the vehicle at that location will experience a stronger attraction from the target location, while a smaller lane change preference evaluation value corresponds to a smaller attraction from the target or a repulsive force from obstacles (such as the vehicle ahead). This matrix V can be used as a lane change preference evaluation for the target scene.

[0067] Step 109: Construct a lane-changing motion primitive library based on the vehicle control signal data. In this step, a motion primitive is a control sequence or trajectory that meets kinematic requirements and connects a pair of starting and ending targets. In previous autonomous lane-changing methods for intelligent vehicles, the planned trajectory and the trajectory tracking control system were difficult to coordinate with each other. The introduction of motion primitives has solved this problem well. In the present invention, after extracting and processing the trajectory data and vehicle control data of the human driver when changing lanes, a motion primitive library is constructed, which contains lane-changing trajectories of different speeds and trajectory shapes and corresponding vehicle control signals such as throttle, brake and steering wheel angle. Therefore, the motion primitive can integrate the existing planning module and control module into one. It only needs to select and execute the motion primitives in the motion primitive library according to certain rules in a specific scenario to complete the autonomous lane change of the intelligent vehicle.

[0068] Step 110: Use the lane change preference evaluation value as the planning basis for the vehicle's driving trajectory, and select the vehicle control signal in the lane change motion primitive library to achieve autonomous lane changing of the vehicle. Specifically, based on the lane change preference evaluation of the target scene obtained in step 108 and the motion primitive library generated in step 109, appropriate motion primitives are selected in a specific scene to complete autonomous lane changing. For example, after migrating the scene representation obtained by offline learning in the initial scene to the target scene that requires lane changing, the generated lane change preference evaluation matrix V is used as the basis for trajectory a planning, and the pre-generated motion primitives are selected to complete a safe and efficient autonomous lane change. In a specific lane changing scenario, such as Figure 4 As shown, HV is the host vehicle and OV is the vehicle being overtaken.

[0069] For example, the specific rules used when selecting motion primitives in the motion primitive library using the lane-changing preference evaluation matrix V are shown in formula (7).

[0070] (7)

[0071] Where, S ( p ) refers to the selected motion primitive, m is the total number of selectable motion primitives in the motion primitive library, n m It is m The total number of grids occupied by the motion primitive trajectories, v m It is m The sum of the lane change preference evaluation values of all grids occupied by the strip motion primitive.

[0072] That is, formula (7) selects the trajectory with the largest average lane-changing preference evaluation value among all optional motion primitives, and uses it as the optimal motion primitive that best suits the current scenario. The control system of the intelligent vehicle can control the vehicle to complete autonomous lane changing according to the control signal sequence of the optimal motion primitive.

[0073] Based on the above description, the overall implementation framework process of the autonomous lane changing method based on migration provided by the present invention is as follows: Figure 5 shown.

[0074] Based on the above description, the present invention can extract state transition relationships in training scenarios and migrate them to a variety of target lane-changing scenarios. By combining them with the reward space in the target lane-changing scenarios, the optimal lane-changing motion primitives can be selected from the motion primitive library to achieve real-time, efficient, and safe autonomous lane changing. Compared with existing autonomous lane-changing systems that use traditional planning methods based on A* and RRT, hierarchical reinforcement learning methods, and artificial potential field methods, the autonomous lane-changing system proposed in the present invention has advantages in terms of training efficiency, versatility, robustness, and response speed. At the same time, the present invention also ensures the consistency of motion planning. Therefore, the system designed by the present invention can utilize knowledge learned offline, better meet the lane-changing needs during actual road driving, and has greater novelty and practicality.

[0075] In addition, the present invention also provides an autonomous lane-changing system based on migration, such as Figure 6 As shown, the system includes: a memory 1 and a processor 2.

[0076] The memory 1 is used to store a computer software program for implementing the above-mentioned autonomous lane-changing method based on migration.

[0077] The processor 2 is connected to the memory 1 and is used to execute computer software programs.

[0078] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0079] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A migration-based autonomous lane changing method, characterized in that: include: Get the vehicle position coordinates in the original scene; Discretizing the vehicle position coordinates to construct a state grid space; Performing value assignment processing on the state grid space, including: assigning a value of -1 to a grid occupied by an obstacle in the state grid space, and assigning a value of 0 to a grid other than the grid occupied by the obstacle in the state grid space; A random walk strategy is used to extract the traffic state in the state grid space after the assignment; The temporal difference learning algorithm is used to determine the transition relationship between different traffic states and obtain the state transition matrix; Establishing scene migration knowledge based on the state transition matrix; Establish reward grid space for the observation area; Generating a reward matrix based on the reward grid space includes: setting a reward value of a grid where a lane change target point is located in the reward grid space to 1, setting a reward value of a grid occupied by other vehicles in the reward grid space except the target vehicle to -1, and setting a reward value of the remaining passable area in the reward grid space to 0; Obtaining a lane change preference evaluation value according to the state transition matrix and the reward matrix; Build a lane-changing motion primitive library based on vehicle control signal data; The lane-changing preference evaluation value is used as a basis for planning a vehicle driving trajectory, and a vehicle control signal in the lane-changing motion primitive library is selected to achieve autonomous lane changing of the vehicle.

2. The autonomous lane changing method based on migration according to claim 1, characterized in that: The temporal difference learning algorithm is used to determine the transition relationship between different traffic states to obtain a state transition matrix, which specifically includes: The temporal difference learning algorithm is used to update the transfer relationship between different traffic states and obtain the updated transfer relationship; A state transition matrix is formed based on the updated transition relationship.

3. The autonomous lane changing method based on migration according to claim 1, characterized in that: The size of the reward grid space is the same as the size of the state grid space.

4. A migration-based autonomous lane-changing system, characterized in that: include: A memory for storing a computer software program; the computer software program is used to implement the migration-based autonomous lane changing method according to any one of claims 1 to 3; A processor is connected to the memory and is used to execute the computer software program.

Images