A vehicle path planning method based on graph structure and reinforcement learning

Abstract

This invention provides a vehicle path planning method based on graph structure and reinforcement learning. The method includes traffic map construction and reinforcement learning model training. First, a real-world map is reconstructed, incorporating factors such as traffic light waiting time and peak hours, and traffic lights are selected as nodes to construct a real-time traffic map. Then, a reinforcement learning model is trained, using Dijkstra's algorithm to construct suboptimal paths within the graph. During training, the suboptimal paths guide the learning process to select the optimal path, accelerating the training speed. Finally, the trained reinforcement learning model is used for real-time vehicle path planning. This invention introduces reinforcement learning into path planning, reducing computational costs and time. Furthermore, the trained reinforcement learning model can provide optimal paths at different times based on varying road conditions, considering factors such as distance and traffic conditions, resulting in more accurate and optimized planned paths.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a vehicle path planning method based on graph structure and reinforcement learning. Background Technology

[0002] With the rapid development of the national economy, the number of cars owned by residents has been rising sharply. Therefore, researching how to rationally plan vehicle routes to avoid congestion has become increasingly important. Applying reinforcement learning methods to vehicle route planning is a major research direction in this field. Urban vehicle route planning faces several challenges. Rush hours and sudden traffic accidents can significantly impact existing urban congestion, thus affecting real-time vehicle routes. Furthermore, the sheer number of roads and urban areas makes the planning problem massive and difficult to solve.

[0003] Reinforcement learning, as a type of machine learning, can learn how to solve various complex tasks. With the increasing number of vehicles in cities, reasonable route planning is a key part of solving urban congestion. Currently, commonly used vehicle route planning algorithms include shrinking hierarchy algorithms and model prediction-based route planning algorithms. However, these methods all have some drawbacks: firstly, they involve a large amount of computation and are computationally complex; secondly, they do not take into account real-time urban road traffic conditions. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a vehicle path planning method based on graph structure and reinforcement learning. By training a reinforcement learning model using traffic road maps and enhanced real-time data, an efficient model capable of obtaining the optimal path is obtained for real-time vehicle path planning.

[0005] A vehicle path planning method based on graph structure and reinforcement learning includes the following steps:

[0006] Step S1: Map data processing, including road data processing and road segment division, standardizing road information, dividing road segments, marking traffic lights on the map, and then connecting these traffic lights sequentially according to the road, with the road between two traffic lights being considered a separate road segment.

[0007] Step S2: Using road information and traffic lights as nodes, and incorporating time factors, construct multiple time-sensitive traffic maps. First, construct a baseline traffic map, using traffic lights as nodes to create a node matrix storage graph. The distance between traffic lights is used as the edge weights of the graph. Then, analyze building locations and perform clustering centered on traffic lights. Next, construct the time-sensitive traffic map by statistically analyzing real-world data to obtain the average transit time for different road segments at different times, calculating the actual transit length of road segments at different times, updating the edges of the graph, and obtaining the time-sensitive traffic map.

[0008] Step S3: Train the reinforcement learning model. Randomly select the start and end points in the time-based traffic map to simulate real vehicle driving. Assign the start and end points to the nearest traffic lights based on distance. Then, use Dijkstra's algorithm to find the shortest path between the various time-based traffic maps as the suboptimal path. Train the reinforcement learning model to find the optimal path between the start and end points and use the suboptimal path to speed up the training. Repeat this process continuously until a reinforcement learning model that can plan the optimal path is obtained.

[0009] Step S4: Use the trained reinforcement learning model for path planning. Convert the user input data into the model input to obtain the optimal path through the reinforcement learning model, and update the optimal path according to the user's real-time location.

[0010] Further, step S1 includes:

[0011] Road data processing involves standardizing the storage formats of different road information, including road labeling. For different road map formats, a reference road map is constructed, and the unit length of the reference road map is set. Roads are identified and recorded in the reference road map through manual or intelligent recognition methods. If roads overlap, they are distinguished according to their road numbers.

[0012] Road segment division involves dividing the roads in the baseline road map into segments based on the road segment length data required for subsequent traffic map construction. This includes traffic light location marking and road segment construction and recording. Traffic light location marking involves marking the location of traffic lights as points on the baseline road map using their latitude and longitude addresses. Road segment construction and recording involves sequentially connecting the traffic lights in the road, recording the road segment between two traffic lights as a separate segment, and recording the segment's length, starting traffic light, minimum speed limit, maximum speed limit, and segment type. The minimum speed limit, maximum speed limit, and segment type are consistent with the road in which the segment is located. Adjacent traffic lights that are not on the same road are not connected.

[0013] Further, step S2 includes:

[0014] First, a baseline traffic map node is constructed. A node in the map represents a set, which is specifically a set centered on a traffic light and including multiple nearby buildings. To obtain such a set, the Dis clustering algorithm is used to cluster based on differences in latitude and longitude. The clustering results ensure that the traffic light closest to the building in different sets is the traffic light represented by the cluster center.

[0015] Next, a baseline traffic map is constructed. Based on the Cartesian product of the clustered sets obtained above, the result is used as a matrix. The initial value of all points in the matrix is ​​set to -1. According to the road segment information, when there is a direct path to the corresponding traffic light, the value of the corresponding position in the matrix is ​​set to the road segment length.

[0016] Finally, a time-sensitive traffic map is constructed. Based on actual statistical data, the city is divided into morning and evening peak hours, normal hours, and late-night hours. Then, the average passage time of different road segments at different times is obtained by statistical analysis of real-world data. Finally, the average passage time is multiplied by the maximum speed limit of the corresponding road segment, and the result is added as an additional distance to the edge weight of the corresponding road segment. This gives the traffic map for different time periods, i.e., the time-sensitive traffic map.

[0017] Furthermore, the Dis clustering algorithm is used for clustering, including:

[0018] Initialization: Select K initial centroids, which are all the traffic lights, with each traffic light serving as a centroid;

[0019] Calculate the interval distance: For each point in the traffic lights, calculate the distance between it and each directly connected traffic light. Let the latitude and longitude of two traffic lights be represented as Traffic(lat1, lon1) and Traffic(lat2, lon2), respectively. The formula for calculating the interval distance Diff is as follows:

[0020]

[0021] Delineating the building area: For each traffic light, calculate the delineation range using it as the midpoint. Let the latitude and longitude of the traffic light be represented as Traffic(lat1,lon1), then the delineation range is calculated as follows:

[0022] Latitude = lat1 + sin(θ) * Diff

[0023] Longtitude=lon1+sin(θ)*Diff

[0024] Where θ represents the angle, which ranges from 0° to 360°;

[0025] For all buildings, if their latitude and longitude are within the range of (Latitude, Longitude), they are within the distance range centered on this traffic light.

[0026] For the remaining buildings not within the division range, calculate their distances to all traffic lights, and then assign them to the clusters corresponding to the nearest traffic lights in turn.

[0027] Further, step S3 includes:

[0028] First, simulating real-world vehicle navigation, a building is selected as the starting point to construct a path planning task and a suboptimal path, including:

[0029] Building mapping requires that the selected building belongs to a non-adjacent cluster to avoid excessively short distances. The starting point is then processed and mapped to the nodes of the constructed baseline traffic map. Specifically, the input building latitude and longitude are replaced with the latitude and longitude of the traffic lights at the corresponding cluster centers.

[0030] For suboptimal path construction, Dijkstra's algorithm is used to calculate the shortest path in the time-sensitive traffic maps for each time period, which is then recorded as the suboptimal path.

[0031] P = {node start ,node1,...,node end {Time period T}}

[0032] Here, node represents the traffic light in the path, specifically the traffic light number, and time period T indicates which time-based traffic map the path was obtained from, which is an enumeration type value;

[0033] Then, reinforcement learning model training is performed. During the training of the reinforcement learning model, the constructed suboptimal path will be used as an aid to accelerate the model training. First, the model input is constructed. The model input is a set that includes the latitude and longitude of the starting and ending buildings of the path and the number of the current time period.

[0034] Furthermore, the model training is guided by a constructed suboptimal path, including:

[0035] The model input transformation first processes the model input by classifying the input building latitude and longitude using a clustering model to find the corresponding distance. Then, it is replaced with the traffic light number corresponding to the cluster center to meet the conditions for using the suboptimal path. After replacement, the model input is represented as: {traffic light number 1, traffic light number 2, time period number}, where traffic light number 1 corresponds to the starting point and traffic light number 2 corresponds to the ending point.

[0036] The model is trained using a suboptimal path. During training, the model uses the suboptimal path with a probability of 1-α to provide the next expected traffic light number, and uses the model's predicted output with a probability of α as the next expected traffic light number. α gradually increases with the number of training rounds. The specific calculation formula is as follows:

[0037] α=α+α*β

[0038] in, β is the growth parameter that controls the rate at which α increases. In each training round, the size of α is increased by this formula.

[0039] The output path is constructed by the model after it receives the input. The model will provide the next expected traffic light number and record it as a relay point in the path. Then, the model input is reconstructed with the given expected traffic light number. Specifically, the first item of the input is replaced, the model is input again, the next expected traffic light number is calculated, and so on until the destination is reached. All the expected traffic lights are connected on the map to obtain the path output by the model.

[0040] Further, step S4 includes:

[0041] The system acquires input, uses a pre-trained clustering model to find the corresponding traffic lights based on the user's selected start and end points, and then selects the corresponding time period based on the current time to construct the initial input for the model.

[0042] The optimal path is updated in real time. The system will obtain the latitude and longitude of the user's current location in real time, and then use a trained clustering model to find the corresponding traffic light. At the same time, the time is re-acquired, a new input is constructed, and then input into the model to obtain the latest optimal path given by the model.

[0043] Compared with the prior art, the beneficial technical effects of the present invention are as follows:

[0044] This invention processes map information to construct the required graph structure, ensuring it meets the input requirements of subsequent steps. While vehicle starting points can be any building, a large number of building locations increases the difficulty of problem-solving. Therefore, this invention employs a clustering method, using traffic lights to represent multiple buildings, effectively reducing training difficulty. Then, by combining time periods, it analyzes the specific traffic conditions at different times, constructing different real-time traffic maps that better reflect actual vehicle driving conditions. During model training, by constructing suboptimal paths within the constructed real-time traffic maps, it guides training in the early stages, effectively avoiding numerous output errors due to insufficient early model training, thus reducing training efficiency and model capability. In practical use, this invention periodically acquires the latest location information to update the current model output, making output construction more flexible and consistent with real-world driving conditions.

[0045] This invention introduces reinforcement learning into path planning, thereby avoiding the problems of high computational cost and long computation time faced by traditional path planning. In practical use, the trained neural network model can output the required path in a timely manner, with lower computational load and cost. At the same time, the solution of this invention incorporates the time factor, which can better reflect the current situation of urban roads. The trained reinforcement learning model can provide the optimal path at different times based on different road conditions, while taking into account factors such as distance and road traffic conditions, making the planned path more accurate and optimized. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating the overall process of a vehicle path planning method based on graph structure and reinforcement learning, as provided in an embodiment of the present invention.

[0048] Figure 2 This is a flowchart of map data processing provided in an embodiment of the present invention;

[0049] Figure 3 This is a flowchart of the time-sensitive traffic map construction provided in an embodiment of the present invention;

[0050] Figure 4 This is a flowchart of the reinforcement learning model training provided in an embodiment of the present invention;

[0051] Figure 5 This is a flowchart of the model usage provided in the embodiments of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] The present invention will be further described below with reference to the accompanying drawings:

[0054] The data source for this invention is detailed map data of a certain location, including traffic lights, roads, building information, etc. Specific data analysis includes: traffic light related data: traffic light location, traffic light number, traffic light type, and signal setting time; building related data: building location, building number, and building type; and road related data: road number, road type, road location, road length, maximum speed limit, and minimum speed limit. A detailed explanation of the map data is as follows:

[0055]

[0056]

[0057] Figure 1 This invention provides an overall flowchart of a vehicle path planning method based on graph structure and reinforcement learning. The method includes map data processing, real-time traffic map construction, model training, and model usage. Map data processing includes road data processing and road segment division. Real-time traffic map construction incorporates factors such as time into the model's considerations, making the planned path more consistent with the current situation. This includes baseline traffic map construction and real-time traffic map construction. Model training involves training the reinforcement learning model to learn how to select the optimal path by randomly selecting starting points on the real-time traffic map. The training speed is also accelerated by constructing suboptimal paths. Model usage involves planning the optimal path in real time using the input data.

[0058] Figure 2 This is a flowchart illustrating the specific process of map data processing. First, the source data is processed, including traffic light data and road data. Then, this data is used to construct the required environmental map, as detailed below:

[0059] Road data processing involves standardizing the storage formats of different road information. Specifically, this includes: road labeling; constructing required baseline road maps for different map formats, with each baseline road map unit being 0.01 kilometers in length; identifying roads using manual or intelligent methods and recording them in the baseline road map; and, if overlapping roads occur, distinguishing them based on their road numbers and then recording these different roads by constructing new baseline maps.

[0060] Road segment division involves dividing the roads in the baseline road map into segments based on the road segment length data required for subsequent environmental map construction. This includes traffic light location marking and road segment construction and recording. Traffic light location marking involves marking the location of traffic lights on the baseline road map using their latitude and longitude addresses, specifically by marking traffic lights as dots on the baseline road map. Road segment construction and recording involves sequentially connecting the traffic lights in the road, recording the road segment between two traffic lights as a separate segment, and recording the segment's length, starting traffic light, minimum speed limit, maximum speed limit, and segment type. The minimum speed limit, maximum speed limit, and segment type are consistent with the road in which the segment is located. Adjacent traffic lights that are not on the same road are not connected.

[0061] Through the above processing, the necessary data for constructing the baseline traffic map, namely traffic light and road segment information, are obtained, and then all the required fields are converted into numerical data.

[0062] Figure 3 This invention describes the process of constructing a real-time traffic map. The map consists of nodes and edges. First, the nodes of a baseline traffic map are constructed using traffic lights. Then, the relationships between the nodes and the edges of the map are constructed using road segment information. The specific operations are as follows:

[0063] First, a baseline traffic map node is constructed. Each node in the map represents a set, specifically a set centered on a traffic light and including multiple nearby buildings. To obtain such a set, the Dis clustering algorithm is used to cluster based on differences in latitude and longitude. The clustering results ensure that the traffic light closest to a building in each set is the traffic light represented by the cluster center.

[0064] The implementation steps of the Dis clustering algorithm are as follows:

[0065] Initialization: Select K initial centroids, which are all the traffic lights, with each traffic light serving as a centroid;

[0066] Calculate the interval distance: For each point in the traffic lights, calculate the distance between it and each directly connected traffic light. Assuming the latitude and longitude of two traffic lights are represented as Traffic(lat1, lon1) and Traffic(lat2, lon2), the interval distance Diff is calculated using the following formula:

[0067]

[0068] This distance applies to both traffic lights.

[0069] Delineating the area around each traffic light: Calculate the area defined by using the traffic light as its midpoint. Assuming the traffic light's latitude and longitude are represented as Traffic(lat1, lon1), the formula for calculating the area is as follows:

[0070] Latitude = lat1 + sin(θ) * Diff

[0071] Longtitude=lon1+sin(θ)*Diff

[0072] Where θ represents the angle, which ranges from 0° to 360°;

[0073] For all buildings, if their latitude and longitude are within the range of (Latitude, Longitude), they are within the distance range centered on this traffic light.

[0074] For the remaining buildings not within the division range, calculate their distances to all traffic lights, and then assign them to the clusters corresponding to the nearest traffic lights in turn.

[0075] Then, a baseline traffic map is constructed. Based on the Cartesian product of the cluster set obtained above, the result is used as a matrix. The initial value of all points in the matrix is ​​set to -1. According to the road segment information, when there is a direct path to the corresponding traffic light, the value of the corresponding position in the matrix is ​​set to the road segment length.

[0076] Finally, a time-sensitive traffic map is constructed. First, based on actual statistical data, the city is divided into morning and evening peak hours, normal hours, and late-night hours. Then, statistical analysis is conducted using real-world data to obtain the average transit time for different road segments at different times. Finally, the average transit time is multiplied by the maximum speed limit of the corresponding road segment, and the result of the multiplication is added as an additional distance to the edge weight of the corresponding road segment. This yields the traffic map for different time periods, i.e., the time-sensitive traffic map.

[0077] Figure 4 This is the training process for a reinforcement learning model. First, a building is randomly selected as the starting point to simulate real-world vehicle planning. By constructing suboptimal paths, model training is accelerated, and the model's performance is enhanced. The specific steps are as follows:

[0078] First, simulating real-world vehicle navigation, a building is selected as the starting point to construct a path planning task and a suboptimal path, including:

[0079] Building mapping requires that the selected building belongs to a non-adjacent cluster to avoid excessively short distances. The starting point is then processed and mapped to the nodes of the constructed baseline traffic map. Specifically, the input building latitude and longitude are replaced with the latitude and longitude of the traffic lights at the corresponding cluster centers.

[0080] For suboptimal path construction, Dijkstra's algorithm is used to calculate the shortest path in the time-sensitive traffic maps for each time period, which is then recorded as the suboptimal path.

[0081] P = {node start node1, ..., node end {Time period T}}

[0082] Here, `node` represents the traffic light in the path, specifically its number. `time period T` indicates which traffic map the path was obtained from; it is an enumeration value, specifically represented as follows:

[0083] Time period number Corresponding meaning 1 Morning rush hour 2 Evening rush hour 3 Normal times 4 Late night

[0084] Then, reinforcement learning model training is performed. During the training of the reinforcement learning model, a constructed suboptimal path will be used to assist in the training process and accelerate the model training. First, the model input is constructed. The model input is a set, specifically represented as follows:

[0085] Enter number Meaning Example 1 Path starting point building latitude and longitude [104.92193082,30.75117200] 2 Latitude and longitude of the building at the end of the route [103.88046799,30.82336421] 3 Current time period number 1

[0086] In the early stages of model training, a suboptimal path is constructed to guide the training process, as follows:

[0087] The model input transformation first processes the model input by classifying the input building latitude and longitude using a clustering model to find the corresponding distance. Then, it replaces the signal light number with the corresponding cluster center to meet the conditions for using the suboptimal path. After replacement, the model input is represented as: {signal light number 1, signal light number 2, time period number}, where signal light number 1 corresponds to the starting point and signal light number 2 corresponds to the ending point.

[0088] The model is trained using a suboptimal path. During training, the model uses the suboptimal path with a probability of 1-α to provide the next expected traffic light number, and uses the model's predicted output with a probability of α as the next expected traffic light number. α gradually increases with the number of training rounds. The specific calculation formula is as follows:

[0089] α=α+α*β

[0090] in, β is the growth parameter that controls the rate at which α increases. In each training round, the size of α is increased using this formula.

[0091] The output path is constructed as follows: After the model receives the input, it will provide the next expected traffic light number. This expected traffic light number is recorded as a relay point on the path. Then, the model input is reconstructed using the given expected traffic light number. Specifically, the first item of the input is replaced, and the model is input again to calculate the next expected traffic light number until the destination is reached. At this point, all the expected traffic lights are connected on the map to obtain the path given by the model.

[0092] Figure 5 This describes the actual usage process of the model of this invention. In actual use, the trained reinforcement learning model is periodically invoked to update the path information in real time, so as to better meet the actual needs. The specific operation is as follows:

[0093] To obtain input, in actual use, the user selects the starting point and the destination. Then, the pre-trained clustering model is used to find the corresponding traffic lights. Based on the current time, the corresponding time period is selected to construct the initial input of the model.

[0094] The optimal path is updated in real time. In actual use, every minute, the system obtains the user's current location latitude and longitude, then uses a trained clustering model to find the corresponding traffic light, simultaneously re-acquires the time, constructs new input, inputs it into the model, and obtains the latest optimal path provided by the model. For a trained model, this process is very fast.

[0095] This invention is based on reinforcement learning and incorporates graph-related ideas (path, clustering, etc.) into traditional path planning methods, replacing the original algorithms such as shrinking levels that cannot be updated in real time and have a huge computational load. This makes the results more timely. Experiments have shown that this method has a good evaluation effect, a wide range of applications, and a short running time.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle path planning method based on graph structure and reinforcement learning, characterized in that, The method includes: Step S1: Map data processing, including road data processing and road segment division, standardizing road information, dividing road segments, marking traffic lights on the map, and then connecting these traffic lights sequentially according to the road, with the road between two traffic lights being considered a separate road segment. Step S2: Using road information and traffic lights as nodes, and incorporating time factors, construct multiple time-sensitive traffic maps. First, construct a baseline traffic map, using traffic lights as nodes to create a node matrix storage graph. The distance between traffic lights is used as the edge weights of the graph. Then, analyze building locations and perform clustering centered on traffic lights. Next, construct the time-sensitive traffic map by statistically analyzing real-world data to obtain the average transit time for different road segments at different times, calculating the actual transit length of road segments at different times, updating the edges of the graph, and obtaining the time-sensitive traffic map. Step S3: Train the reinforcement learning model. Randomly select the start and end points in the time-based traffic map to simulate real vehicle driving. Assign the start and end points to the nearest traffic lights based on distance. Then, use Dijkstra's algorithm to find the shortest path between the various time-based traffic maps as the suboptimal path. Train the reinforcement learning model to find the optimal path between the start and end points and use the suboptimal path to speed up the training. Repeat this process continuously until a reinforcement learning model that can plan the optimal path is obtained. Step S4: Use the trained reinforcement learning model for path planning. Convert the user input data into the model input to obtain the optimal path through the reinforcement learning model, and update the optimal path according to the user's real-time location.

2. The method according to claim 1, characterized in that, Step S1 further includes: Road data processing involves standardizing the storage formats of different road information, including road labeling. For different road map formats, a reference road map is constructed, and the unit length of the reference road map is set. Roads are identified and recorded in the reference road map through manual or intelligent recognition methods. If roads overlap, they are distinguished according to their road numbers. Road segment division involves dividing the roads in the baseline road map into segments based on the road segment length data required for subsequent traffic map construction. This includes traffic light location marking and road segment construction and recording. Traffic light location marking involves marking the location of traffic lights as points on the baseline road map using their latitude and longitude addresses. Road segment construction and recording involves sequentially connecting the traffic lights in the road, recording the road segment between two traffic lights as a separate segment, and recording the segment's length, starting traffic light, minimum speed limit, maximum speed limit, and segment type. The minimum speed limit, maximum speed limit, and segment type are consistent with the road in which the segment is located. Adjacent traffic lights that are not on the same road are not connected.

3. The method according to claim 1, characterized in that, Step S2 further includes: First, a baseline traffic map node is constructed. A node in the map represents a set, which is specifically a set centered on a traffic light and including multiple nearby buildings. To obtain such a set, the Dis clustering algorithm is used to cluster based on differences in latitude and longitude. The clustering results ensure that the traffic light closest to the building in different sets is the traffic light represented by the cluster center. Next, a baseline traffic map is constructed. Based on the Cartesian product of the clustered sets obtained above, the result is used as a matrix. The initial value of all points in the matrix is ​​set to -1. According to the road segment information, when there is a direct path to the corresponding traffic light, the value of the corresponding position in the matrix is ​​set to the road segment length. Finally, a time-sensitive traffic map is constructed. Based on actual statistical data, the city is divided into morning and evening peak hours, normal hours, and late-night hours. Then, the average passage time of different road segments at different times is obtained by statistical analysis of real-world data. Finally, the average passage time is multiplied by the maximum speed limit of the corresponding road segment, and the result is added as an additional distance to the edge weight of the corresponding road segment. This gives the traffic map for different time periods, i.e., the time-sensitive traffic map.

4. The method according to claim 3, characterized in that, Clustering is performed using the Dis clustering algorithm, including: Initialization: Select K initial centroids, which are all the traffic lights, with each traffic light serving as a centroid; Calculate the interval distance: For each point in the traffic lights, calculate the distance between it and each directly connected traffic light. Let the latitude and longitude of two traffic lights be represented as Traffic(lat1, lon1) and Traffic(lat2, lon2), respectively. The formula for calculating the interval distance Diff is as follows: Delineating the building area: For each traffic light, calculate the delineation range using it as the midpoint. Let the latitude and longitude of the traffic light be represented as Traffic(lat1, lon1), then the delineation range is calculated as follows: Latitude = lat1 + sin(θ) * Diff Longtitude=lon1+sin(θ)*Diff Where θ represents the angle, which ranges from 0° to 360°; For all buildings, if their latitude and longitude are within the range of (Latitude, Longitude), they are within the distance range centered on this traffic light. For the remaining buildings not within the division range, calculate their distances to all traffic lights, and then assign them to the clusters corresponding to the nearest traffic lights in turn.

5. The method according to claim 1, characterized in that, Step S3 further includes: First, simulating real-world vehicle navigation, a building is selected as the starting point to construct a path planning task and a suboptimal path, including: Building mapping requires that the selected building belongs to a non-adjacent cluster to avoid excessively short distances. The starting point is then processed and mapped to the nodes of the constructed baseline traffic map. Specifically, the input building latitude and longitude are replaced with the latitude and longitude of the traffic lights at the corresponding cluster centers. For suboptimal path construction, Dijkstra's algorithm is used to calculate the shortest path in the time-sensitive traffic maps for each time period, which is then recorded as the suboptimal path. P = {node start node1, ..., node end {Time period T}} Here, node represents the traffic light in the path, specifically the traffic light number, and time period T indicates which time-based traffic map the path was obtained from, which is an enumeration type value; Then, reinforcement learning model training is performed. During the training of the reinforcement learning model, the constructed suboptimal path will be used as an aid to accelerate the model training. First, the model input is constructed. The model input is a set that includes the latitude and longitude of the starting and ending buildings of the path and the number of the current time period.

6. The method according to claim 5, characterized in that, Model training is guided by a constructed suboptimal path, including: The model input transformation first processes the model input by classifying the input building latitude and longitude using a clustering model to find the corresponding distance. Then, it is replaced with the traffic light number corresponding to the cluster center to meet the conditions for using the suboptimal path. After replacement, the model input is represented as: {traffic light number 1, traffic light number 2, time period number}, where traffic light number 1 corresponds to the starting point and traffic light number 2 corresponds to the ending point. The model is trained using a suboptimal path. During training, the model uses the suboptimal path with a probability of 1-α to provide the next expected traffic light number, and uses the model's predicted output with a probability of α as the next expected traffic light number. α gradually increases with the number of training rounds. The specific calculation formula is as follows: α＝α+α*β in, β is the growth parameter that controls the rate at which α increases. In each training round, the size of α is increased by this formula. The output path is constructed by the model after it receives the input. The model will provide the next expected traffic light number and record it as a relay point in the path. Then, the model input is reconstructed with the given expected traffic light number. Specifically, the first item of the input is replaced, the model is input again, the next expected traffic light number is calculated, and so on until the destination is reached. All the expected traffic lights are connected on the map to obtain the path output by the model.

7. The method according to claim 1, characterized in that, Step S4 further includes: The system acquires input, uses a pre-trained clustering model to find the corresponding traffic lights based on the user's selected start and end points, and then selects the corresponding time period based on the current time to construct the initial input for the model. The optimal path is updated in real time. The system will obtain the latitude and longitude of the user's current location in real time, and then use a trained clustering model to find the corresponding traffic light. At the same time, the time is re-acquired, a new input is constructed, and then input into the model to obtain the latest optimal path given by the model.

Images