Vehicle travel joint planning method and system, and storage medium

Through scene mirroring and deep Q network technology, the problem of inaccurate arrival time prediction in existing vehicle travel planning is solved, and the joint planning of departure time and travel route is realized, which improves the accuracy of the planning and the ability to meet actual needs.

CN118913303APending Publication Date: 2024-11-08HUAZHONG UNIV OF SCI & TECH
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410983679.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing vehicle travel planning methods are difficult to accurately predict the arrival time, and most of them are predicted with a fixed time interval as the departure time, resulting in a large difference between the actual arrival time and the driver's expected time, which cannot meet the actual needs of arrival on time.

Method used

Through scene mirroring technology, the traffic road network and background traffic flow are mirrored, and the agent is trained based on the deep Q network to output the optimal path and departure time, so as to realize the joint planning of departure time and travel route, ensuring that the target vehicle arrives at the goal on time and reduce driving time.

Benefits of technology

The joint planning of departure time and travel route is realized, the accuracy of vehicle travel planning is improved, and repeated iterative calculations are avoided. It can provide continuous departure time plans in real dynamic traffic scenarios to meet the actual needs of drivers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118913303A_ABST
    Figure CN118913303A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of vehicle travel planning, and discloses a vehicle travel joint planning method and system and a storage medium, and the method comprises the steps: extracting background traffic flow information of each road section of a traffic road network in different time slices in collected GPS point location data; overturning the traffic network and the background traffic flow information to obtain a mirrored traffic scene; in the mirrored traffic scene, exchanging a starting road section and a target road section of a target vehicle, inputting state information at the current moment into the intelligent agent, taking a driving road section at the next moment as an action of the intelligent agent, training the intelligent agent based on reinforcement learning, and outputting an optimal path and time for reaching the target road section when the intelligent agent converges; turning over the optimal path sequence to obtain a planned travel route; and advancing the time of arriving at the target road section by the travel time of an optimal path, namely the planned departure time. According to the invention, joint planning of the departure time and the travel route is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle travel planning, and more specifically, relates to a vehicle travel joint planning method, system and storage medium. Background Art

[0002] Vehicle travel planning methods are widely used in various scenarios in the field of transportation, such as travel navigation, autonomous driving, logistics distribution, etc. Vehicle travel planning requires a comprehensive assessment of the driver's travel needs and the dynamic road conditions, fully tapping into the road condition characteristics information, and planning a better travel plan to meet the actual needs of the driver.

[0003] Most of the travel planning methods in existing navigation systems use the estimated departure time and destination as input, and then plan the route with the shortest travel time as the goal. This may cause the actual arrival time to differ greatly from the driver's expected arrival time. In real-world scenarios, the actual demand is to plan the departure time and travel route based on the arrival time and destination. The most common scenario is that commuters arrive at their destinations according to working hours, or residents need to arrive at the airport or train station on time. The driver's early arrival or late arrival will affect the travel experience. In addition, most of the current vehicle travel planning methods predict the travel time based on multiple fixed time intervals as the departure time. For example, if the destination is Railway Station A, the estimated arrival time is predicted by selecting 8:30, 9:30, ..., or other time point departure times. Repeated iterations are required to obtain a departure time that is consistent with the expected arrival time. Summary of the invention

[0004] In view of the above defects or improvement needs of the prior art, the present invention provides a vehicle travel joint planning method, system and storage medium, which aims to achieve joint planning of departure time and travel route.

[0005] To achieve the above object, according to a first aspect of the present invention, a vehicle travel joint planning method is provided, comprising:

[0006] S1, extracting the background traffic flow information of each section of the traffic network in different time slices from the collected GPS point data;

[0007] S2, flipping the starting and ending points of each road section in the traffic network to obtain a mirrored traffic network; taking the arrival time of the target vehicle as the time symmetry axis, mirroring the departure time of the background vehicles in different road sections in the background traffic flow information to achieve the mirroring of the background traffic flow information;

[0008] S3. In the traffic scene after mirroring, the starting section and the target section of the target vehicle are exchanged, and the arrival time of the target vehicle is set to the departure time t n; Get the current status information Input to the agent, take the driving section at the next moment as the action of the agent, train the agent based on reinforcement learning, and when the agent converges, output the optimal path consisting of the driving section sequence corresponding to different moments, and the time t′ when the target vehicle reaches the target section on the optimal path n ; Where Te is the expected travel time of the target vehicle on each road section of the mirrored traffic network, Ne is the number of background vehicles on each road section, curpos is the current location coordinate of the target vehicle, and tarpos is the destination coordinate; t′ n With t n The difference is the optimal travel time;

[0009] S4, flipping the driving section sequence to obtain the planned travel route; and reversing the time t′ to reach the target section. n Move forward one optimal travel time, which is the planned departure time.

[0010] Furthermore, the arrival time of the target vehicle is used as the time symmetry axis, and the departure time of the background vehicles in the background traffic flow information in different road sections is mirrored, including:

[0011] Divide the length l of the current road section by the speed V of the background vehicle to obtain the travel time t of the background vehicle on the current road section, and calculate the time t1 when the background vehicle arrives at the end of the current road section before the mirroring; where t1=t0+t, t0 is the original departure time of the background vehicle;

[0012] Then the departure time of the background vehicle after mirroring with the target arrival time as the axis of symmetry is t n +(t n -t1).

[0013] Furthermore, the agent is trained based on reinforcement learning. When the agent converges, it outputs the optimal path consisting of the driving section sequence corresponding to different times, including:

[0014] S31, the current state information The state information is input to the agent, and the agent evaluates the score of each action based on the state information, and selects an action according to the set action decision strategy based on the evaluation score to obtain the driving section at the next moment;

[0015] S32. Take the next moment as the current moment, and obtain the status information of the current moment to determine whether the target vehicle has reached the boundary of the traffic network or entered an infinite loop; if so, end the current round; if not, determine whether the target vehicle has reached the target section, if so, end the current round, if not, jump to S31 until the target vehicle reaches the target section; when the preset training round is reached, the intelligent agent is deemed to have converged, and the intelligent agent outputs the optimal path consisting of the driving section sequence corresponding to different moments.

[0016] Furthermore, the agent adopts a deep Q network;

[0017] S31 also includes calculating the environmental reward value of the corresponding action decision; and saving the environmental reward value to the experience pool of the deep Q network. When the number of experience items in the experience pool reaches a minimum threshold, batch extraction is performed to update the parameters in the deep Q network; wherein the environmental reward value is set with the goal of the target vehicle arriving on time and staying in the traffic network for the shortest time.

[0018] Furthermore, the background traffic flow information of each section of the traffic network in different time slices in the collected GPS point data is extracted, including:

[0019] Extracting the speed data of background vehicles in different time slices of each road section in the traffic network from the GPS point data, and averaging the speed data as the average speed of each road section in different time slices;

[0020] Based on the road attribute information contained in the traffic road network, the average speed is mapped into background traffic flow information using a traffic flow model; wherein the road attribute information includes the ID, starting and ending points, number of lanes and length information of each road section.

[0021] Furthermore, it also includes:

[0022] Extracting actual driving trajectory data of background vehicles from the GPS point data;

[0023] The optimal travel time is compared with the travel time in the driving trajectory data to determine the quality of the planned path; wherein, the smaller the optimal travel time is than the travel time in the driving trajectory data, the better the planned travel route is.

[0024] Furthermore, SUMO simulation software is used to simulate the traffic scene after the mirroring.

[0025] According to a second aspect of the present invention, there is provided a vehicle travel joint planning system, comprising a computer-readable storage medium and a processor;

[0026] The computer-readable storage medium is used to store executable instructions;

[0027] The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the vehicle travel joint planning method described in any one of Example 1.

[0028] According to a second aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the vehicle travel joint planning method as described in any one of the first aspects is implemented.

[0029] According to a third aspect of the present invention, a computer program product is provided. When the computer program product is run on a computer, the computer executes the vehicle travel joint planning method described in any one of the first aspects.

[0030] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0031] (1) The vehicle travel planning method of the present invention performs travel planning by means of scene mirroring, wherein the mirrored traffic network and the mirrored background traffic volume constitute the mirrored real traffic scene, the starting driving section of the target vehicle is set as the target section, and the target arrival time is set as the departure time of the target vehicle. In this way, the departure time and travel route of the target vehicle can be inferred based on the mirrored traffic scene; specifically, the expected driving time Te of the target vehicle on each section in the mirrored traffic network, the mirrored background traffic volume Ne, the current position coordinates curpos of the target vehicle and the destination coordinates tarpos are used as the status information at different times, and the target vehicle is selected to obtain the target vehicle's starting point. An intelligent agent is used to make road section decisions, where Te and Ne together reflect the road condition information of the mirrored traffic network. This state information can reflect multiple aspects of environmental feature information that affect the intelligent agent's road section decision-making, so that the intelligent agent can make the optimal path planning; the optimal path output by the intelligent agent, that is, the sequence of driving sections corresponding to different times, is flipped to obtain the final planned travel route, and the time for the target vehicle to arrive at the target section on the optimal path is moved forward by a travel time, which is the final planned departure time. In this way, the joint planning of the departure time and the travel route is realized, which is more in line with actual needs and improves the accuracy of vehicle travel planning.

[0032] Furthermore, compared with the existing navigation technology that predicts travel time based on multiple fixed time intervals as departure time, the scene mirroring in the present invention can ensure that the travel time of the target vehicle is the same before and after the mirroring. When planning the departure time, the time for the target vehicle to arrive at the target section on the optimal path can be directly moved forward by the travel time of the optimal path, thereby realizing continuous departure time planning without repeated iterations.

[0033] (2) As a preferred method, the agent adopts a deep Q network for strategy learning. Based on the designed state information and decision actions, the travel plan of departure time and travel path is jointly planned. Compared with the existing method of considering only one factor (departure time or travel path), the joint impact of departure time and travel path on travel experience is considered at the same time, which is more in line with actual travel needs. In addition, the planning objectives of departure time and travel path are unified on the shortest travel time, making the optimization objectives of the two compatible, further improving the accuracy of vehicle travel planning.

[0034] (3) Furthermore, the present invention provides a specific method for extracting dynamic background traffic flow. Compared with the random traffic flow data in the prior art, the background traffic flow data in the present invention is extracted based on actual traffic road condition information, which can better simulate real traffic scenarios and achieve better path planning based on actual dynamic background traffic flow, thereby shortening the travel time, being sensitive to dynamic traffic flow, and further improving the accuracy of vehicle travel planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a vehicle travel joint planning method in an embodiment of the present invention.

[0036] Figure 2 It is an overall flow chart of the vehicle travel joint planning method in an embodiment of the present invention.

[0037] Figure 3 Schematic diagram of mirror processing of a road section in a traffic road network in an embodiment of the present invention.

[0038] Figure 4 Schematic diagram of the departure time mirroring processing of background vehicles in an embodiment of the present invention.

[0039] Figure 5 This is a mirrored example diagram of the traffic network and background traffic flow in an embodiment of the present invention.

[0040] Figure 6 This is a schematic diagram of a path planning method based on real dynamic background traffic using deep reinforcement learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0042] Example 1

[0043] like Figure 1 and Figure 2 As shown, the vehicle travel joint planning method in the embodiment of the present invention mainly includes:

[0044] S1. Obtain GPS point data collected by vehicle-mounted equipment, and extract background traffic flow information of each road section of the traffic network in different time slices after preprocessing; wherein the GPS point data collected by the vehicle-mounted equipment includes the position, speed, ID of the driving road, driving attribute information, and time information of each data upload of the background vehicle; the ID of the driving road is used to reflect the information of each road section in the traffic network; the background traffic flow information includes the driving direction, departure time and number of background vehicles on each road section of the background vehicle;

[0045] S2, flipping the starting and ending points of each road section in the traffic network to obtain a mirrored traffic network; based on the mirrored traffic network, mirroring the departure time of background vehicles in each road section to achieve mirroring of background traffic flow information, and the mirrored traffic network and the mirrored background traffic flow information constitute a mirrored real traffic scene;

[0046] S3. In the real traffic scenario after the mirroring, the starting driving section of the target vehicle is set as the target section, and the target arrival time (i.e., the driver's expected arrival time) is set as the departure time t of the target vehicle. n ; Get the current status information of the mirrored real traffic scene Among them, Te represents the expected travel time of the target vehicle on each road section in the mirrored traffic network, which reflects the speed limit of the road section and the current level of traffic flow; Ne represents the number of background vehicles on each road section, which reflects the load of the current road section at the current moment. Te and Ne together reflect the road condition information of the mirrored traffic network; curpos represents the current location coordinates of the target vehicle, and tarpos represents the destination coordinates, that is, the actual departure coordinates of the target vehicle;

[0047] The state information s of the current moment is input to the agent, and the driving section at the next moment is used as the action of the agent. The agent is trained based on reinforcement learning. After reaching the preset training rounds, the agent outputs the optimal path planned after the mirror, that is, the sequence of driving sections corresponding to different moments, and at the same time, outputs the time t′ when the target vehicle reaches the target section on the optimal path. n ; In the embodiment of the present invention, the intelligent agent adopts a Deep Q-Network (DQN) model;

[0048] S4, flip the driving section sequence output by S3 to obtain the final planned travel route; set the time t′ to reach the target section nThe travel time of the optimal path obtained in S3 is moved forward by one, which is the final planned departure time; the travel time of the optimal path obtained in S3 is t′ n -t n .

[0049] Specifically, in S1, based on GPS point data, the background traffic volume of each section of the traffic network in different time slices is extracted, including:

[0050] Extract the speed data of background vehicles in different time slices of each road section in the road network from the GPS point data, and average the speed data to obtain the average speed of each road section in different time slices; wherein the average speed is used to reflect the current road conditions; in the embodiment of the present invention, a day is divided into 144 time slices, each time slice is 10 minutes long, and the average speed of the road is calculated in this way; in other embodiments, the length of each time slice can be selected according to actual needs;

[0051] The average speed of each road section in different time slices is mapped to the road background traffic volume. In the embodiment of the present invention, the traffic road network and the road attribute information contained in the traffic road network are obtained from the OpenStreetMap platform; wherein the road attribute information includes the ID, starting and ending points, number of lanes, length and other information of each road section; based on the road attribute information, the average speed of each road section in different time slices is mapped to the road background traffic volume using a traffic flow model.

[0052] In the embodiment of the present invention, SUMO simulation software is used to simulate the real traffic scene. Therefore, the background traffic flow is converted into the traffic flow routing file of the SUMO simulation platform. Figure 2 The background traffic .ROU.XML file is used to convert the traffic network into the real road network .NET.XML file of the SUMO simulation platform.

[0053] Correspondingly, in S2, the traffic network and dynamic background traffic flow in the SUMO simulation are flipped to obtain the mirrored traffic network and dynamic background traffic flow in SUMO.

[0054] Specifically, in S2, for the traffic network, the direction of each road section, that is, the node elements corresponding to the starting node and the ending node are flipped to obtain the mirrored traffic network, such as Figure 3 As shown, Figure 3 Node 0 and node 1 in represent the starting node and the ending node of a road segment respectively. For the background traffic flow, since the road segment has been flipped, the background vehicle driving segment is still set to the original segment to complete the mirroring of the driving direction, but the departure time needs to be calculated and modified according to the target arrival time, including:

[0055] Taking the target arrival time as the time symmetry axis, the departure time of the background vehicles in different sections and time slices is mirrored to obtain the departure time of the mirrored background vehicles in different sections and time slices. Figure 4 As shown, specifically including:

[0056] Divide the current road section length l by the background vehicle speed V to get the background vehicle travel time t on the current road section;

[0057] Assume that the original departure time of the background vehicle is t0, and the time when the background vehicle arrives at the end of the road section before the mirroring is t1=t0+t;

[0058] If the target arrival time of the target vehicle in this experiment is t n , then the departure time of the background vehicle after mirroring with the target arrival time as the axis of symmetry is t n +(t n -t1).

[0059] By mirroring the road network and background traffic according to the above operation, it is possible to ensure that the test vehicle encounters the same road conditions before and after the mirroring, thereby ensuring that the vehicle's travel time is the same before and after the start and end points are swapped. Therefore, this method can be used to simplify the travel time planning problem and realize the planning of continuous departure time, not limited to departure at fixed intervals.

[0060] Since mirroring is performed before the path planning is implemented, after obtaining the path planning and the corresponding travel time, reverse mirroring is required to obtain the true forward travel path and departure time. To reverse mirror the path, you only need to flip the segment ID sequence of the path, which is the forward optimized travel path; reverse mirroring of the departure time is to reverse the optimal travel time based on the target arrival time to obtain the planned departure time.

[0061] like Figure 5 To further explain the mirroring of the traffic network and the background traffic flow, a specific example is given below. Figure 5 Shown is a simple road network with six background vehicles. Figure 5 The two road network diagrams and dynamic flow tables on the left represent the changes in road conditions and the distribution of dynamic flow tables before the mirroring process. The two road network diagrams and dynamic flow tables on the right represent the changes in road conditions and the distribution of dynamic flow tables after the mirroring process. v1-v4 represent nodes in the traffic network, such as intersections, and e1-e4 represent sections in the network. (e i , t j ) means e i The road section is t iThe number of background vehicles in the time slice, for example, (e1, t1) = 3 means that the number of background vehicles in the e1 section in the t1 time slice is 3. By comparing the road conditions and dynamic flow table before and after the mirroring, it can be found that the dynamic flow table is mirrored with the target arrival time as the symmetry axis, and the entire traffic scene is also flipped, but it is moved backward in time.

[0062] In the embodiment of the present invention, as a preferred embodiment, Figure 6 As shown in the figure, the agent in S3 adopts the DQN model. The DQN model uses the mirrored traffic scene constructed by SUMO as the environment, constantly interacts and learns with the environment, and finally converges to a better path that makes the reward value larger. Figure 6 The DRL in the above code represents deep reinforcement learning, and DQN is a type of DRL. The specific process includes:

[0063] Initialize the DQN model and SUMO simulation environment. Specifically, the SUMO simulation platform loads the mirrored traffic network file, configures the dynamic background traffic flow file of the corresponding time period, sets the starting section of the target vehicle to the ending section, and sets the target arrival time to the departure time t of the target vehicle. n .

[0064] Obtain the initial state information of the simulated traffic environment through the interactive control module Input to the DQN model, the DQN model evaluates the score of each action based on the state information, where each action of the DQN model represents the driving section at the next moment; according to the set action decision strategy, the action is selected according to the evaluation score of each action, that is, the driving section at the next moment is output; and the environmental reward value corresponding to the action decision is calculated. The environmental reward is set with the target vehicle arriving on time and staying in the road network for the shortest time; at the same time, the environmental reward value is saved to the experience pool of the DQN model. When the number of experience items in the experience pool reaches the minimum threshold, batch extraction is performed to complete the update of the neural network parameters in the DQN model. The neural network is used to evaluate the score of each action based on the state information;

[0065] The interactive control module also sets the next driving section for the target vehicle according to the action selected by the DQN model, and drives the target vehicle to the next section in the simulation environment; and obtains the new state information at the current moment to determine whether the current state has reached the map boundary or entered an infinite loop. If so, the vehicle cannot reach the destination no matter how it drives in the given road network, so a penalty item is added to end the current round;

[0066] If not, determine whether the target road section has been reached at the current moment. If so, end the current round immediately, record the current time, reinitialize the simulation environment, and the target vehicle performs the next round of training. If not, the current round has not ended yet, input the new state information into the DQN model and repeat the above steps until the target road section is reached.

[0067] When the preset training rounds are reached, the DQN model outputs the optimal path planned after mirroring, that is, the sequence of driving sections corresponding to different times, and the time t′ when the target vehicle reaches the target section on the optimal path n .

[0068] Furthermore, the method of the present invention also includes: cleaning and screening the GPS point data, and matching the data with the topological information in the traffic network to obtain the actual driving trajectory data of the background vehicle; and filtering out unreasonable data in the driving trajectory data according to the time information of each data upload in the GPS point data to obtain the required actual driving trajectory data of the background vehicle.

[0069] By comparing the travel time corresponding to the optimal path with the corresponding travel time in the actual driving trajectory data of the background vehicle, it can be found through simulation experiments that the travel time corresponding to the optimal path obtained by the present invention is shorter than the corresponding travel time in the actual driving trajectory data of the background vehicle.

[0070] The vehicle travel planning method of the present invention performs travel planning in a scene mirroring manner, and uses the mirrored traffic network and the mirrored background traffic flow to form a mirrored real traffic scene, sets the starting driving section of the target vehicle as the target section, and sets the target arrival time as the departure time of the target vehicle. In this way, the departure time and travel route of the target vehicle can be inferred based on the mirrored traffic scene; specifically, the expected driving time Te of the target vehicle on each section in the mirrored traffic network, the mirrored background traffic flow Ne, the current position coordinates curpos of the target vehicle and the destination coordinates tarpos are used as state information at different times, and the vehicle travel planning method is implemented. The intelligent agent makes a road section decision, wherein Te and Ne together reflect the road condition information of the traffic road network after mirroring. This state information can reflect multiple aspects of environmental feature information that affect the intelligent agent's road section decision, so that the intelligent agent can make the optimal path planning; the optimal path output by the intelligent agent, that is, the driving section sequence corresponding to different times, is flipped to obtain the final planned travel route, and the time when the target vehicle arrives at the target section on the optimal path is shifted forward by one travel time, which is the final planned departure time. In this way, the joint planning of the departure time and the travel route is realized, which is more in line with actual needs and improves the accuracy of vehicle travel planning. In addition, compared with the existing navigation technology that predicts the travel time based on multiple fixed time intervals as the departure time, the scene mirroring in the present invention can realize continuous departure time planning, that is, only one forward shift is required to obtain the final planned departure time without repeated iterations.

[0071] The present invention provides a specific method for extracting dynamic background traffic flow. Compared with the random traffic flow data in the prior art, the background traffic flow data in the present invention is extracted based on actual traffic road condition information, which can better simulate real traffic scenes and achieve better path planning based on actual dynamic background traffic flow, so that the travel time is shorter, it is sensitive to dynamic traffic flow, and can further improve the accuracy of vehicle travel planning.

[0072] In the present invention, the intelligent agent adopts a deep Q network for strategy learning, and realizes the joint planning of the departure time and the travel path based on the designed state information and decision actions. Compared with the existing method of considering only one factor (departure time or travel path), the joint impact of the departure time and the travel path on the travel experience is considered at the same time, and the planning goals of the two are unified on the shortest travel time, so that the optimization goals of the two are compatible, and the accuracy of vehicle travel planning is further improved.

[0073] Example 2

[0074] An embodiment of the present invention provides a vehicle travel joint planning system, including a computer-readable storage medium and a processor;

[0075] Computer-readable storage media for storing executable instructions;

[0076] The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the vehicle travel joint planning method in Example 1. The relevant technical solutions are described in Example 1 and will not be repeated here.

[0077] Example 3

[0078] The embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the vehicle travel joint planning method in embodiment 1 is implemented. The relevant technical solutions are described in embodiment 1 and will not be repeated here.

[0079] Example 4

[0080] The embodiment of the present invention provides a computer program product, which, when executed on a computer, enables the computer to execute the vehicle travel joint planning method in Embodiment 1. For the related technical solutions, please refer to the description in Embodiment 1, which will not be repeated here.

[0081] In summary, the present invention provides a vehicle travel joint planning method based on scene mirroring and DQN model, which takes into account the actual dynamic traffic flow. On the one hand, a scene mirroring method is designed to mirror the starting and ending locations, road network and actual road traffic flow to achieve continuous planning of departure time and avoid repeated iterative calculations. On the other hand, the DQN model is combined with traffic simulation for path planning, which fully explores the characteristic information of the environment, with the goal of arriving at the destination on time and minimizing the travel time. By learning, the path selection is continuously optimized to achieve path planning under the background of actual dynamic traffic flow.

[0082] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle travel joint planning method, characterized in that: include: S1, extracting the background traffic flow information of each section of the traffic network in different time slices from the collected GPS point data; S2, flipping the starting and ending points of each road section in the traffic network to obtain a mirrored traffic network; taking the arrival time of the target vehicle as the time symmetry axis, mirroring the departure time of the background vehicles in different road sections in the background traffic flow information to achieve the mirroring of the background traffic flow information; S3. In the traffic scene after mirroring, the starting section and the target section of the target vehicle are exchanged, and the arrival time of the target vehicle is set to the departure time t n ; Get the current status information Input to the agent, take the driving section at the next moment as the action of the agent, train the agent based on reinforcement learning, and when the agent converges, output the optimal path consisting of the driving section sequence corresponding to different moments, and the time t′ when the target vehicle reaches the target section on the optimal path n ; Where Te is the expected travel time of the target vehicle on each road section of the mirrored traffic network, Ne is the number of background vehicles on each road section, curpos is the current location coordinate of the target vehicle, and tarpos is the destination coordinate; t′ n With t n The difference is the optimal travel time; S4, flipping the driving section sequence to obtain the planned travel route; and reversing the time t′ to reach the target section. n Move forward one optimal travel time, which is the planned departure time.

2. The vehicle travel joint planning method according to claim 1, characterized in that: Taking the arrival time of the target vehicle as the time symmetry axis, the departure time of the background vehicles in different road sections in the background traffic flow information is mirrored, including: Divide the length l of the current road section by the speed V of the background vehicle to obtain the travel time t of the background vehicle on the current road section, and calculate the time t1 when the background vehicle arrives at the end of the current road section before the mirroring; where t1=t0+t, t0 is the original departure time of the background vehicle; Then the departure time of the background vehicle after mirroring with the target arrival time as the axis of symmetry is t n +(t n -t1).

3. The vehicle travel joint planning method according to claim 1, characterized in that: The agent is trained based on reinforcement learning. When the agent converges, it outputs the optimal path consisting of the driving section sequence corresponding to different times, including: S31, the current state information The state information is input to the agent, and the agent evaluates the score of each action based on the state information, and selects an action according to the set action decision strategy based on the evaluation score to obtain the driving section at the next moment; S32. Take the next moment as the current moment, and obtain the status information of the current moment to determine whether the target vehicle has reached the boundary of the traffic network or entered an infinite loop; if so, end the current round; if not, determine whether the target vehicle has reached the target section, if so, end the current round, if not, jump to S31 until the target vehicle reaches the target section; when the preset training round is reached, the intelligent agent is deemed to have converged, and the intelligent agent outputs the optimal path consisting of the driving section sequence corresponding to different moments.

4. The vehicle travel joint planning method according to claim 3 is characterized in that: The intelligent agent adopts a deep Q network; S31 also includes calculating the environmental reward value of the corresponding action decision; and saving the environmental reward value to the experience pool of the deep Q network. When the number of experience items in the experience pool reaches a minimum threshold, batch extraction is performed to update the parameters in the deep Q network; wherein the environmental reward value is set with the goal of the target vehicle arriving on time and staying in the traffic network for the shortest time.

5. The vehicle travel joint planning method according to claim 1, characterized in that: Extract the background traffic flow information of each section of the traffic network in different time slices from the collected GPS point data, including: Extracting the speed data of background vehicles in different time slices of each road section in the traffic network from the GPS point data, and averaging the speed data as the average speed of each road section in different time slices; Based on the road attribute information contained in the traffic road network, the average speed is mapped into background traffic flow information using a traffic flow model; wherein the road attribute information includes the ID, starting and ending points, number of lanes and length information of each road section.

6. The vehicle travel joint planning method according to claim 1, characterized in that: Also includes: Extracting actual driving trajectory data of background vehicles from the GPS point data; The optimal travel time is compared with the travel time in the driving trajectory data to determine the quality of the planned path; wherein, the smaller the optimal travel time is than the travel time in the driving trajectory data, the better the planned travel route is.

7. The vehicle travel joint planning method according to claim 1, characterized in that: The SUMO simulation software is used to simulate the traffic scene after the mirroring.

8. A vehicle travel joint planning system, characterized in that: comprising a computer readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the vehicle travel joint planning method described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the vehicle travel joint planning method as described in any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the vehicle travel joint planning method according to any one of claims 1 to 7.

Citation Information

Cited By

  • Gold mine efficient detecting and positioning method based on volcanic hidden explosion breccia characteristics

    CN121302229A