An AI large model-based urban driving strategy generation method and device

Abstract

The application provides an urban driving strategy generation method and device based on an AI large model, relates to the technical field of transportation, and is used to generate a driving plan that balances passenger flow demand and vehicle resources. The method generates an initial first state array and an initial second state array based on initial driving information; iteratively executes the following process until the first value reaches a preset threshold based on the initial first state array as input, generates a first driving plan based on the first state array: updates the initial first state array based on each standard departure time through a first model; adjusts the first model based on the updated first state array; iteratively executes the following process until the second value is within a preset threshold interval based on the initial second state array as input, generates a second driving plan based on a prediction result: generates a prediction result based on a second state array through a second model, adjusts the second model based on the prediction result; and determines a target driving plan based on the first driving plan and the second driving plan.

Description

Technical Field

[0001] The present application relates to the field of transportation technology, and in particular to a method and device for generating urban driving strategies based on an AI large model. Background Art

[0002] With the increasing complexity of urban transportation systems, efficient driving strategy generation has become the key to improving the efficiency of public transportation operations. Traditional driving plan generation methods are mainly based on static schedules and heuristic algorithms, which have the following shortcomings:

[0003] (1) Existing methods usually rely on historical passenger flow to generate timetables. Although they can reflect basic demand, they lack dynamic coordination of transport resources. This leads to an excessive number of vehicles and low utilization of some vehicles (such as double-shift vehicles running a single-shift mission), resulting in wasted transport capacity and increased operating costs.

[0004] (2) Manual adjustments or rule-based algorithms, such as genetic algorithms, can only perform local optimization for a fixed time period or a single vehicle, and cannot globally coordinate multiple objective constraints such as departure intervals, station stop times, and vehicle trip frequency. For example, during peak hours, it is difficult to simultaneously optimize departure intervals and vehicle turnover efficiency, which can easily lead to longer passenger waiting times or idle vehicles.

[0005] (3) Urban traffic scenarios are highly dynamic, such as sudden passenger flows or road network congestion. However, traditional methods rely on preset rules and lack the ability to autonomously learn from real-time data, resulting in rigid generation strategies that are difficult to respond to dynamic changes.

[0006] Timetables are typically generated based on passenger flow. While this approach effectively reflects passenger demand, it lacks consideration for transport capacity. Consequently, these timetables often utilize a high number of vehicles overall, with some vehicles running less frequently. Consequently, this approach fails to effectively balance passenger flow and transport capacity, leading to wasted transport capacity and high operating costs for businesses. Summary of the Invention

[0007] The purpose of this application is to provide a method and device for generating urban driving strategies based on an AI large model, so as to generate a driving plan that balances passenger flow demand and vehicle resources, and reduces operating costs while meeting service levels.

[0008] In a first aspect, an embodiment of the present application provides a method for generating urban driving strategies based on an AI large model, comprising:

[0009] Obtaining initial driving information, and generating an initial first state array and an initial second state array based on the initial driving information; the initial first state array includes candidate departure times configured for each vehicle in the initial driving information, and the initial second state array includes multiple candidate departure time periods determined based on the earliest standard departure time and the latest vehicle arrival time in the initial driving information;

[0010] The initial first state array is input into the first model, and the following process is iteratively performed until the first value reaches a preset threshold: the candidate departure times in the initial first state array are updated based on the standard departure times in the initial driving information using the first model; a first value corresponding to the updated first state array is determined, and parameters of the first model are adjusted based on the first value;

[0011] using the first state array corresponding to the first value reaching the preset threshold as the target first state array, and generating a first driving plan based on the target first state array;

[0012] Inputting the initial second state array into the second model, iteratively performing the following process until the second value is within a preset threshold range: generating a prediction result based on the second state array by the second model, the prediction result including a predicted vehicle number for each candidate departure time period; determining a second value corresponding to the prediction result, and adjusting parameters of the second model based on the second value;

[0013] using the prediction result corresponding to the second value within the preset threshold range as the target prediction result, and generating a second driving plan based on the target prediction result;

[0014] A target driving plan is determined based on the first driving plan and the second driving plan.

[0015] In a second aspect, an embodiment of the present application provides an urban driving strategy generation device based on an AI large model, comprising at least one processor and at least one memory; wherein the memory stores program code, and when the program code is executed by the processor, the processor performs the following process:

[0016] Obtaining initial driving information, and generating an initial first state array and an initial second state array based on the initial driving information; the initial first state array includes candidate departure times configured for each vehicle in the initial driving information, and the initial second state array includes multiple candidate departure time periods determined based on the earliest standard departure time and the latest vehicle arrival time in the initial driving information;

[0017] inputting an initial first state array into the first model, iteratively performing the following process until a first value reaches a preset threshold: updating candidate departure times in the initial first state array based on the standard departure times in the initial driving information by the first model; determining a first value corresponding to the updated first state array, and adjusting parameters of the first model based on the first value;

[0018] taking the first state array corresponding to the first value reaching the preset threshold as a target first state array, and generating a first driving plan based on the target first state array;

[0019] inputting an initial second state array into the second model, iteratively performing the following process until a second value is within a preset threshold interval: generating a prediction result based on the second state array by the second model, the prediction result including vehicle numbers predicted for each candidate departure time interval; determining a second value corresponding to the prediction result, and adjusting parameters of the second model based on the second value;

[0020] taking the prediction result corresponding to the second value within the preset threshold interval as a target prediction result, and generating a second driving plan based on the target prediction result;

[0021] determining a target driving plan based on the first driving plan and the second driving plan.

[0022] The technical scheme provided by the embodiments of the present application at least brings the following beneficial effects:

[0023] The first model updates the departure times of each vehicle based on the standard departure times, and reversely adjusts the first model based on the updated departure times, so that the first model learns the change rule of the departure times and the vehicle selection to form an internal expression function, and finally forms a better combination of vehicles and times, i.e., the first driving plan; the second model predicts the corresponding vehicle numbers based on the candidate departure time intervals, and reversely adjusts the second model based on the output result, so that the second model learns the correlation between the departure times, and finally forms a better combination, i.e., the second driving plan; two driving plans are generated by the first model and the second model, and the final driving plan is generated based on the two driving plans, which ensures the uniformity of the times in the driving plan and the rationality of the vehicle selection. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical scheme of the embodiments of the present application, the drawings needed in the embodiments of the present application will be briefly introduced as follows. Obviously, the drawings introduced below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0025] Figure 1A schematic diagram of the principle of reinforcement learning provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram of an action selection provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0028] Figure 4 A flowchart of a method for generating urban driving strategies based on an AI large model provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of a possible first model structure provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of a working process of a first model provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of a candidate departure time update process provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of a standard departure time adjustment process provided in an embodiment of the present application;

[0033] Figure 9 A schematic diagram of a pointer network provided in an embodiment of the present application;

[0034] Figure 10 A schematic diagram of a working process of a second model provided in an embodiment of the present application;

[0035] Figure 11 A schematic diagram of a prediction result generation process provided in an embodiment of the present application;

[0036] Figure 12 A schematic diagram of a second value determination process provided in an embodiment of the present application;

[0037] Figure 13 A schematic diagram of an urban driving strategy generation device based on an AI large model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] To make the purpose, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. The described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0039] Moreover, in the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0040] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0041] In order to facilitate understanding of the interactive medical sequence image segmentation method and device provided in the embodiments of the present application, some terms in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0042] (1) Reinforcement learning: Figure 1 As shown in Figure 1, reinforcement learning consists of an agent and an environment. The agent interacts with the environment state through actions and receives rewards or penalties (i.e., R) from the environment based on the changes in the environment. The agent then takes better actions through rewards, thereby continuously optimizing actions to obtain better rewards. Actions in reinforcement learning can be selected based on strategy (i.e., probability p) or value (i.e., score V), as shown in Figure 1. Figure 2 As shown, Figure 2 The left image in the middle shows a value-based approach (i.e., score), while the right image shows a policy-based approach (i.e., probability). Reinforcement learning uses state input into a neural network, which then predicts actions based on that state, gradually optimizing the problem. Through massive iterative training, a more optimal network model is obtained, and the predictions from that model are used to solve similar problems.

[0043] (2) Up and down: Usually a vehicle has two terminal stations, namely the main station and the secondary station. Going from the main station to the secondary station is down, and going from the secondary station to the main station is up.

[0044] (3) Circular: The starting point and the end point of the vehicle are the same station.

[0045] Traditional driving plan generation schemes based on heuristic algorithms typically generate timetables based on passenger flow. While this approach effectively reflects passenger demand, the input timetable or time interval remains constant and lacks consideration for capacity demand. Consequently, driving plans compiled based on this timetable suffer from issues such as a high overall number of vehicles and infrequent trips for some vehicles. This approach fails to effectively balance passenger flow and capacity, leading to wasted capacity and high operating costs for businesses. Furthermore, manual optimization cannot determine how to adjust time intervals and other information.

[0046] Based on the above problems, this application provides a method and device for generating urban driving strategies based on an AI large model, which is used to generate a driving plan that balances passenger flow demand and vehicle resources, and reduces operating costs while meeting service levels.

[0047] The following describes an application scenario of an optional urban driving strategy generation method based on an AI large model provided by an embodiment of the present application in conjunction with the accompanying drawings. Figure 3 As shown, the application scenario includes at least one terminal 301 and a server 302, wherein:

[0048] The terminal 301 is used to send initial driving information to the server 301 .

[0049] Specifically, the terminal 301 may provide a user operation interface, through which the user uploads initial driving information and issues a driving plan generation instruction. The terminal 301 responds to the user's instruction and sends the initial driving information to the server 302.

[0050] The server 302 is used to obtain initial driving information and generate an initial first state array and an initial second state array based on the initial driving information; the initial first state array includes candidate departure times configured for each vehicle in the initial driving information, and the initial second state array includes multiple candidate departure time periods determined based on the earliest standard departure time and the latest vehicle arrival time in the initial driving information; the initial first state array is input into the first model, and the following process is iteratively performed until the first value reaches a preset threshold: the candidate departure times in the initial first state array are updated based on the standard departure times in the initial driving information through the first model; the first value corresponding to the updated first state array is determined, and the parameters of the first model are adjusted based on the first value. Adjustment; taking the first state array corresponding to the first value reaching the preset threshold as the target first state array, and generating a first driving plan based on the target first state array; inputting the initial second state array into the second model, and iteratively executing the following process until the second value is within the preset threshold range: through the second model, generating a prediction result based on the second state array, the prediction result includes the vehicle serial number predicted for each candidate departure time period; determining the second value corresponding to the prediction result, and adjusting the parameters of the second model based on the second value; taking the prediction result corresponding to the second value within the preset threshold range as the target prediction result, and generating a second driving plan based on the target prediction result; determining the target driving plan based on the first driving plan and the second driving plan.

[0051] In some embodiments, after generating the target driving plan, the server 302 sends the target driving plan to the terminal 301, and the terminal 301 displays the target generation plan on the display interface for the user to view.

[0052] Of course, the method provided in the embodiment of the present application is not limited to Figure 3 The application scenarios shown can also be used in other possible application scenarios, which are not limited in the embodiments of the present application.

[0053] like Figure 4 As shown, a flowchart of a driving plan generation embodiment of the present application is shown, and the specific steps are as follows:

[0054] Step S401: acquiring initial driving information, and generating an initial first state array and an initial second state array based on the initial driving information;

[0055] The initial first-state array includes candidate departure times assigned to each vehicle in the initial driving information, and the initial second-state array includes multiple candidate departure time periods determined based on the earliest standard departure time and the latest vehicle arrival time in the initial driving information. It should be noted that in this embodiment of the present application, the number of candidate departure times assigned to each vehicle may be one or more, and in implementation, this may be determined based on the maximum number of trips corresponding to each vehicle.

[0056] In step S402, the initial first state array is input into the first model, and the following process is iteratively performed until the first value reaches a preset threshold: the candidate departure time in the initial first state array is updated based on each standard departure time in the initial train operation information by the first model; the first value corresponding to the updated first state array is determined, and the parameters of the first model are adjusted based on the first value.

[0057] Optionally, the determination manner of the first value is not limited in the embodiments of the present application, and in some embodiments, the first value can be the sum of a third value (i.e., the third value corresponding to the last standard departure time) and a fourth value, where the determination manner of the third value is described below, and the fourth value is the overall score of the updated first state array. The determination manner of the fourth value is not limited in the embodiments of the present application, and in some embodiments, the fourth value can also be composed of three parts of the time arrangement rationality value, the train type and train distribution value, and the time interval compliance value (see the description of the determination manner of the third value below), and the score value of the rule compliance such as the first departure and the first arrival is added. Through the sum of the score (i.e., the third value) and the overall score (i.e., the fourth value), the overall evaluation rate of the plan can be improved, and the problem of insufficient local evaluation can be reduced.

[0058] In step S403, the first state array corresponding to the first value reaching the preset threshold is taken as a target first state array, and a first train operation plan is generated based on the target first state array.

[0059] In some embodiments, when the first value reaches the preset threshold, it is determined that the first model reaches the expected target, and the first state array at this time is taken as the target first state array. The target first state array includes the final departure time corresponding to each vehicle in the initial train operation information. The information in the target first state array is arranged, and the first train operation plan is generated.

[0060] In step S404, the initial second state array is input into the second model, and the following process is iteratively performed until the second value is within the preset threshold interval: the prediction result is generated based on the second state array by the second model, the prediction result including the vehicle serial number predicted for each candidate departure time; the second value corresponding to the prediction result is determined, and the parameters of the second model are adjusted based on the second value.

[0061] Step S405 uses the prediction result corresponding to the second value within the preset threshold range as the target prediction result, and generates a second driving plan based on the target prediction result. In some embodiments, when the second value is within the preset threshold range, it is determined that the second model has achieved the expected goal, and the prediction result at this time is used as the target prediction result. The target prediction result includes the multiple candidate departure times finally selected and their corresponding vehicle numbers. The information in the target prediction result is sorted to generate the second driving plan.

[0062] Step S406: Determine a target driving plan based on the first driving plan and the second driving plan.

[0063] In some embodiments, after generating a first driving plan and a second driving plan, a first evaluation value corresponding to the first driving plan and a second evaluation value corresponding to the second driving plan may be determined based on preset rules. Based on the magnitude relationship between the first evaluation value and the second evaluation value, one of the first and second driving plans may be selected as the target driving plan. In specific implementations, the aforementioned preset rules may be set based on business needs and are not limited in the embodiments of this application. For example, the corresponding evaluation value may be determined based on parameters such as the departure interval and the number of vehicles used in the driving plan.

[0064] The above method updates the departure time of each vehicle based on the standard departure time through the first model, and reversely adjusts the first model based on the updated departure time, so that the first model learns the changing rules of the departure time and vehicle selection to form an internal expression function, and finally forms a better combination of vehicles and times, that is, the first driving plan; predicts the corresponding vehicle serial number based on the candidate departure time period through the second model, and reversely adjusts the second model based on the output result, so that the second model learns the correlation between the departure times, and finally forms a better combination, that is, the second driving plan; generates two plans through the first model and the second model, and generates the final driving plan based on the two driving plans, thereby ensuring the uniformity of the time in the driving plan and the rationality of the vehicle selection.

[0065] Optionally, the initial driving information in the above step S401 is introduced below with reference to specific examples.

[0066] In some embodiments, the above-mentioned initial driving information includes but is not limited to the total number of available vehicles, a standard timetable generated based on the running direction, departure time, arrival time, standard departure interval and standard running time of each time period, the maximum number of trips corresponding to different types of vehicles (such as the number of trips run by double-shift vehicles per day and the number of trips run by single-shift vehicles), the maximum and minimum stop time and other information.

[0067] Tables 1 to 3 below are specific examples of standard departure intervals, standard operating times, and standard timetables in the initial driving information.

[0068] Table 1. Standard departure intervals

[0069]

[0070] As shown in Table 1, the standard departure interval may include the running direction corresponding to each time period (which can be divided into up and down directions, or circular, Table 1 only gives an example of up direction), the start time and end time of the time period, and the departure interval.

[0071] Table 2. Standard operating time

[0072]

[0073] As shown in Table 2, the standard operating time may include the operating direction corresponding to each time period (which can be divided into up and down directions, or circular travel, and Table 2 only gives an example of up travel), the start time and end time of the time period, and the operating time (that is, the one-way duration of a vehicle departing at a certain time within the time period).

[0074] Table 3. Standard timetable

[0075]

[0076] As shown in Table 3, the standard timetable may include the direction of travel, departure time (i.e. the above-mentioned standard departure time), arrival time, departure interval and running time.

[0077] In some embodiments, the departure time can be expressed in hours (such as Departure Time 1 in the table) or converted to minutes (such as Departure Time 2 in Table 3). The minutes can be expressed as the number of minutes from midnight to the current departure time, such as 6:00 converted to 360. Similarly, the arrival time can be expressed in hours (such as Arrival Time 1 in the table) or converted to minutes (such as Arrival Time 2 in Table 3).

[0078] In some embodiments, the maximum and minimum vehicle counts can be further determined based on the number of standard schedules, facilitating the subsequent calculation of the second value. The maximum number of vehicles is calculated by dividing the number of departure times by the number of single-shift trips (adding 1 if there is a remainder), while the minimum number of vehicles is calculated by dividing the number of departure times by the number of double-shift trips. In some embodiments, the algorithm can be iterated from the maximum to the minimum number of vehicles, calculating the required times for each vehicle count. Multiple combinations can be generated and solved using reinforcement learning, or the user can select a combination for solution.

[0079] Optionally, the initial first state array and the initial second state array in the above step S401 are introduced below with reference to specific examples.

[0080] In some embodiments, the initial first state array includes at least one candidate departure time configured for each vehicle in the initial driving information. Optionally, to facilitate subsequent parameter adjustment of the first model based on the initial first state array, the above-mentioned initial first state array also includes relevant information of the currently selected departure time and vehicle.

[0081] The following example provides a possible structure for the initial first state array. In this example, assume the route mode is up and down, the total number of vehicles is Cn, and the maximum number of vehicle trips is Tn (i.e., the number of candidate departure times assigned to each vehicle is Tn). Since Tn is an even number for up and down routes, the sub-state array for each vehicle (i.e., the set of candidate departure times) is: [time placeholder (i.e., candidate departure time) 1, time placeholder 2, ..., time placeholder Tn+1, redundant time placeholders].

[0082] In some embodiments, bits 1 to Tn÷2+1 are placeholders for uplink times, and bits Tn÷2+2 to Tn+2 are placeholders for downlink times, i.e., the length of the sub-state array for each vehicle is Tn+2. It should be noted that in the embodiments of the present application, to separate uplink and downlink times and prevent time overflow, time placeholder redundancy is provided for both uplink and downlink times. i.e., bit Tn÷2+1 is a redundant placeholder for the uplink time, and bit Tn+2 is a redundant placeholder for the downlink time.

[0083] Combine all vehicle sub-state arrays into a one-dimensional array by vehicle serial number, and add 4 bits of data at the end to store the final selected state, which facilitates the neural network's ability to identify feature changes. The final selected state is the last selected vehicle serial number, departure time serial number, departure time, and arrival time, forming the initial first state array. The length of the initial first state array is Cn × (Tn + 2) + 4.

[0084] It should be noted that, since the initial first state array is generated, the selection of the standard departure time and the selection of the vehicle serial number have not yet been performed, so the final selected state mean is 0.

[0085] In some embodiments, when the driving route mode is circular, since there are no uplink and downlink signals, only one bit of redundant time slot is required, meaning that the length of the sub-state array corresponding to each vehicle is Tn+1. Therefore, the length of the resulting initial first state array is Cn×(Tn+1)+4.

[0086] In some embodiments, when generating the initial first state array, if the driving route mode is up and down, the first up standard departure time is assigned to the first vehicle, and the first down standard departure time is assigned to the second vehicle according to the time sequence of the standard departure times in the initial driving information; that is, in the initial first state array, the first moment of the sub-state array corresponding to the first vehicle is the first up standard departure time, and the Tn÷2+2th moment of the sub-state array corresponding to the second vehicle is the first down standard departure time, and the initial values ​​of the remaining moments are 0.

[0087] If the driving route mode is circular, the first standard departure time is assigned to the first vehicle according to the time sequence of the standard departure times in the initial driving information; that is, in the initial first state array, the first time position of the sub-state array corresponding to the first vehicle is the first standard departure time, and the initial values ​​of the remaining time positions are 0.

[0088] In some embodiments, the initial second state array includes a plurality of candidate departure time periods determined according to the earliest standard departure time and the latest vehicle arrival time in the initial driving information.

[0089] It should be noted that the principles for determining each candidate departure time period can be set based on business needs. For example, it can be set based on unit duration, and the interval between the earliest standard departure time and the latest vehicle arrival time is divided into multiple candidate departure time periods. The duration corresponding to each candidate departure time period is the above-mentioned unit duration.

[0090] In some embodiments, the initial second state array includes, in addition to each candidate departure time period, the corresponding travel direction for each candidate departure time period, including up, down, or a circular movement. Specifically, in an up or down route, 1 may be used to indicate up and 2 to indicate down, and 1 may be used to indicate a circular movement in a circular route.

[0091] Optionally, the number of rows of the initial second state array in the embodiment of the present application is the number of candidate departure time periods, that is, each row corresponds to a candidate departure time period, and the number of columns is 3, which are [running direction, departure minutes (that is, the departure time corresponding to the candidate departure time period), arrival minutes (that is, the arrival time corresponding to the candidate departure time period)].

[0092] The following example provides a possible structure for the initial second-state array. In this example, it is assumed that the first departure time (i.e., the earliest standard departure time) is 360 minutes, the end time (i.e., the latest vehicle arrival time) is 1360 minutes, and the unit duration is 1 minute. In this case, the number of candidate departure time slots is 1360 - 360 + 1 = 1001, meaning the initial second-state array is a 1001 (row) × 3 (column) matrix. It should be noted that the above example is for a circular route. For an uplink or downlink route, the number of candidate departure time slots is (1360 - 360 + 1) × 2 = 2002, meaning there are 1001 candidate departure time slots for each of the uplink and downlink routes. This means the initial second-state array is a 2002 (row) × 3 (column) matrix.

[0093] Optionally, in the embodiment of the present application, the first model is designed and generated based on the actor-critic algorithm. Figure 5 A schematic diagram of a possible first model structure provided in an embodiment of the present application is shown as follows: Figure 5 As shown, after receiving the state input (i.e., the initial first state array), the first model outputs the logarithmic probability through the neural network and normalizes the above logarithmic probability (optionally, it can be implemented based on a normalized exponential function, i.e., a softmax function), selects an action based on the normalized probability, and executes the corresponding action. At the same time, the loss is calculated based on the states before and after the action, and the neural network is optimized based on the loss gradient backpropagation.

[0094] In some embodiments, the first model in this application includes two agents, the moment and the vehicle, and a critic.

[0095] In the specific implementation, in each step, the time agent first selects a time, and then the vehicle agent selects a vehicle number, and updates the environment state (i.e., the current first state array) based on the standard departure time and vehicle number.

[0096] like Figure 6 As shown, assuming that the standard departure times include 360, 370, 380, etc., and the vehicle numbers include car 1, car 2, car 3, etc. In a specific implementation, the action performed is that the time agent selects a standard departure time, such as 370, and then the vehicle agent selects a vehicle number, such as car 3. Based on the selected standard departure time and vehicle number, the sub-state array corresponding to car 3 in the first state array is updated, and the last selected state in the first state array is updated at the same time, thereby obtaining an updated first state array.

[0097] Then, this step of operation is scored according to the state of the environment and evaluated by the critic. Finally, the networks corresponding to the two agents and the critic (i.e., the first model) are optimized through backpropagation based on the evaluation results.

[0098] Among them, the networks corresponding to the moment agent, vehicle agent and critic are all fully connected neural networks (Feedforward Neural Network). The vehicle agent selects the moment based on its corresponding first fully connected neural network, the vehicle agent selects the vehicle serial number based on its corresponding second fully connected neural network, and the critic determines the evaluation result based on its corresponding third fully connected network.

[0099] A fully connected neural network consists of multiple layers, including input layer, hidden layer and output layer.

[0100] Input layer: Receives raw data and passes it to the neurons in the next layer.

[0101] Hidden layer: This layer consists of at least one neuron. Each neuron in the hidden layer receives information from neurons in the previous layer (this information is aggregated after weight adjustment by multiplying the output of each neuron in the previous layer by the corresponding weight and then summing the results). It then generates an output through an activation function and passes the output to the neurons in the next layer. The activation function introduces nonlinearity in the neurons, enabling the neural network to learn and represent complex nonlinear relationships. In this way, useful features are extracted from the raw input data and used for prediction or classification.

[0102] Output layer: This layer produces the network's final output. This output typically corresponds to the result of a prediction or classification task. For example, in a classification task, the number of neurons in the output layer is typically equal to the number of categories, and the output of each neuron represents the predicted probability of the corresponding category.

[0103] Since the fully connected neural network is an existing structure, the embodiment of the present application will not elaborate on its working principle in detail.

[0104] The above step S402 is introduced below in conjunction with the first model structure.

[0105] Optionally, in the above step S402, when the candidate departure times in the initial first state array are updated based on the standard departure times in the initial driving information, such as Figure 7 As shown, according to the time sequence of each standard departure time in the initial driving information, the standard departure time is selected in turn, and the following process is iterated to the last standard departure time.

[0106] It should be noted that when selecting the standard departure times in sequence according to the time sequence of the standard departure times in the initial driving information, if the driving route mode is up and down, the first up standard departure time and the first down standard departure time are excluded; if the driving route mode is circular, the first standard departure time is excluded.

[0107] Step S701: for the currently selected standard departure time, based on the current first state array, determine the target vehicle sequence number corresponding to the selected standard departure time.

[0108] It should be noted that the current first state array here refers to the first state array updated based on the last selected standard departure time, that is, the latest first state array.

[0109] Optionally, the initial driving information also includes the first candidate vehicle serial number corresponding to each vehicle.

[0110] The first candidate vehicle serial number corresponds to the total number of vehicles included in the initial driving information. For example, when the total number of vehicles included in the initial driving information is K, the first candidate vehicle serial numbers can be vehicle 1, vehicle 2, ..., vehicle K, with each first candidate vehicle serial number corresponding to a vehicle in the initial driving information.

[0111] In some embodiments, for the currently selected standard departure time, the selection probability of each first candidate vehicle number corresponding to the selected standard departure time is first predicted based on feature fusion of the first state array and the resulting feature vector. Specifically, using the current first state array as input, a fully connected neural network corresponding to the vehicle agent outputs a first output array with a length equal to the total number of vehicles. The first output array includes the selection probability corresponding to each first candidate vehicle number.

[0112] Then, based on the selection probabilities of each first candidate vehicle number, a target vehicle number corresponding to the selected standard departure time is determined from the first candidate vehicle numbers. Specifically, the vehicle agent, based on the selection probabilities corresponding to each first candidate vehicle number in the first output array, selects a candidate vehicle number based on a preset principle as the target vehicle number corresponding to the currently selected standard departure time.

[0113] It should be noted that the above preset principle can be set based on demand and is not limited in the embodiment of the present application. For example, the first candidate vehicle serial number with the largest probability value is selected as the target vehicle serial number.

[0114] In some embodiments, in order to avoid selecting an unselectable vehicle when selecting the target vehicle number, the unselectable vehicles include vehicles whose candidate departure times are full or vehicles that do not meet the requirements (such as the vehicle's last arrival time is earlier than the currently selected standard departure time), the embodiment of the present application uses a mask mechanism to correct the selection probability in the above-mentioned first output value to block the above-mentioned unselectable vehicles.

[0115] In a specific implementation, a first mask array is determined for the first output array. The first mask array includes mask values ​​corresponding to each first candidate vehicle number, wherein the mask value corresponding to an unselectable vehicle (the first mask value) is different from the mask value corresponding to an selectable vehicle (the second mask value). The values ​​of the first mask value and the second mask value can be set based on demand. Typically, the first mask value is set to a very small negative value, and the second mask value is set to 0. Before determining the target vehicle number corresponding to the selected standard departure time from each first candidate vehicle number based on the selection probability of each first candidate vehicle number, the selection probability of each first candidate vehicle number is corrected (e.g., added) based on the mask value corresponding to each first candidate vehicle number to obtain an updated selection probability. Based on the updated selection probability, the target vehicle number determination process is executed.

[0116] Step S702: Based on the selected standard departure time, the candidate departure time corresponding to the target vehicle number in the current first state array is updated.

[0117] In a specific implementation, the above-mentioned candidate departure time corresponding to the target vehicle number in the current first state array is updated based on the selected standard departure time, that is, the value of the unupdated candidate departure time (that is, the candidate departure time with a value of 0) corresponding to the target vehicle number in the current first state array is modified to the currently selected standard departure time.

[0118] It's important to note that when updating candidate departure times, the update is based on the order of the candidate departure times (i.e., time placeholders) in the sub-state array corresponding to the target vehicle number. For example, if the sub-state array corresponding to the target vehicle number is [time placeholder 1, time placeholder 2, time placeholder 3, time placeholder redundant], and time placeholder 1 is non-zero, while time places 2 and 3 are zero, time placeholder 2 is updated, while time placeholder 3 remains unchanged.

[0119] In some embodiments, when the candidate departure time corresponding to the target vehicle sequence number in the current first state array is updated, the last selected state in the current first state array is also updated, i.e., the vehicle sequence number in the last selected state is updated to the target vehicle sequence number, the sequence number of the departure time is updated to the sequence number corresponding to the currently selected standard departure time, the departure time is updated to the currently selected standard departure time, and the arrival time is updated to the arrival time corresponding to the currently selected standard departure time.

[0120] Optionally, after the candidate departure time corresponding to the target vehicle sequence number in the current first state array is updated, the method further includes: determining a third value corresponding to the updating operation based on the current first state array and the first state array before updating; and updating the parameters in the first model based on the third value.

[0121] At this time, the determination of the target vehicle sequence number corresponding to the selected standard departure time based on the current first state array includes: determining the target vehicle sequence number corresponding to the selected standard departure time based on the current first state array by using the updated first model after the first model is updated based on the third value corresponding to the previous standard departure time.

[0122] That is, in the embodiments of the present application, after the candidate departure time corresponding to the target vehicle sequence number in the current first state array is updated for each currently selected standard departure time, the third value (i.e., the score) corresponding to the updating is determined, the parameters in the first model are optimized based on the third value, and the step of updating the first state array based on the next selected standard departure time is executed based on the updated model, so that the network is optimized to have self-learning ability to develop in a better direction and obtain a better plan.

[0123] It should be noted that the determination method of the third value in the embodiments of the present application is not limited, and in some embodiments, the third value can be determined based on the changes in the departure time, the number of vehicles used, and the departure interval between the current first state array and the first state array before updating.

[0124] The following provides a possible determination method of the third value.

[0125] The states before and after each action are predicted and scored by the fully connected neural network corresponding to the critic to obtain the score before the action Sq and the score after the action Sh. That is, the reference value Sq corresponding to the current first state array and the reference value Sh corresponding to the first state array before the update are determined respectively. Then the third value = action value + optimization coefficient × Sh-Sq is calculated. The setting of the optimization coefficient is used to avoid the non-convergence of the network due to the same evaluation network score before and after the action. In the embodiment of the present application, there is no restriction on its value, for example, it is set to 0.99.

[0126] In specific implementation, the above action value can be composed of three parts: the time arrangement rationality value, the schedule distribution value and the time interval compliance value, among which:

[0127] The schedule rationality value is calculated based on the current stop time between the new time added by the vehicle and the previous time (that is, the difference between the candidate departure time updated this time and its previous candidate departure time among the candidate departure times corresponding to the target vehicle serial number. For example, when the candidate departure time updated this time is time slot 2, the current stop time is the difference between time slot 2 and time slot 1). Assuming the optimal stop time is Ty, if the current stop time is between the minimum stop time and the maximum stop time, the schedule rationality value is the maximum stop time - abs(optimal stop time - current stop time), where abs is the function that finds the absolute value of the expression; if the current stop time is between 0 and the minimum stop time, the schedule rationality value is the stop time. If the value is negative, the schedule rationality value is the stop time multiplied by the negative stop penalty multiplier (the negative stop penalty multiplier can be set based on demand); if the current stop time is greater than the maximum stop time, it is determined whether the preset conditions are met, such as whether the departure time is within the lunch break period, the meal period, or meets the charging rules. If the preset conditions are met and the current stop time is less than the preset threshold, the schedule rationality value is set to a positive number; otherwise, the schedule rationality value is set to a negative number.

[0128] The distribution value of the scheduled train schedule refers to the closeness between the current number of trips of the vehicle (the updated time in the candidate departure time corresponding to the target vehicle serial number, that is, the number of times with non-zero values) and the number of single or double trips. If the current number of trips is less than or equal to the number of single trips, the distribution value of the scheduled train schedule is the current number of trips - the number of single trips. If the current number of trips is greater than the number of single trips, the distribution value of the scheduled train schedule is the current number of trips - the number of double trips.

[0129] Time period departure interval compliance value: calculated based on the difference between the departure interval requirement for the time period of the currently selected departure time and the actual departure interval. If the difference is 0, 5 points will be awarded (i.e., the time period departure interval compliance value is 5 points); if the difference is less than 3, the difference points will be awarded (i.e., the time period departure interval compliance value is equal to the difference points); if the difference is greater than 3, the difference points will be penalized (i.e., the time period departure interval compliance value is equal to the negative value of the difference), etc.

[0130] In some embodiments, the above-mentioned action value can be the sum of three parts: the time scheduling rationality value, the class and number distribution value, and the time period departure interval compliance value.

[0131] Optionally, in step S702 above, before updating the candidate departure time corresponding to the target vehicle number in the current first state array based on the selected standard departure time, the method further includes:

[0132] Based on the current first state array, predict the selection probabilities corresponding to the respective adjustment values ​​corresponding to the selected standard departure time;

[0133] Based on the selection probabilities corresponding to the respective adjustment values, a target adjustment value corresponding to the selected standard departure time is determined from the respective adjustment values, and the selected standard departure time is adjusted based on the target adjustment value.

[0134] Among them, each adjustment value is determined based on the maximum departure interval in the initial driving information.

[0135] Optionally, the adjustment value is selected based on a preset rule from a value between a preset initial value and a maximum departure interval value. For example, the difference between two adjacent adjustment values ​​can be set to be the same and equal to the preset interval value.

[0136] In one possible embodiment, a parameter Jmax is determined based on the maximum departure interval in the initial driving information. If the maximum departure interval is an even number, Jmax is equal to the maximum departure interval; if the maximum departure interval is an odd number, Jmax is equal to the maximum departure interval + 1. An adjustment array is then determined based on parameter Jmax, comprising multiple adjustment values. The adjustment values ​​are selected from a range between a preset initial value and the maximum departure interval, with intervals equal to the preset interval values.

[0137] For example, if the difference between two adjacent adjustment values ​​in the plurality of adjustment values ​​is 1, the number of adjustment values ​​included in the adjustment array is Jmax. For example, when the initial value is 0 and Jmax is 10, the adjustment value array is [0, 1, 2, 3, 4, 5, 6, 7, 8, 9].

[0138] In some embodiments, the current first state array is used as input, and the selection probability corresponding to each adjustment value is determined by a fully connected neural network corresponding to the agent at that moment. The output of the fully connected neural network is an array of length Jmax, which includes the selection probability corresponding to each adjustment value.

[0139] Then, based on the selection probabilities corresponding to the adjustment values, a target adjustment value corresponding to the selected standard departure time is determined from the adjustment values. In some embodiments, the adjustment value with the highest selection probability can be used as the target adjustment value corresponding to the standard departure time. The selected standard departure time is then adjusted based on the target adjustment value.

[0140] In some embodiments, assuming that the currently selected standard departure time is Ti, the target adjustment value is determined to be S based on the selection probability (if S is 0, it means that the time is abandoned), then the correction value of the standard departure time is determined to be A=S-(Jmax÷2), and the value of the currently selected standard departure time is adjusted from Ti to Ti+A. This operation ensures that the adjustment of the standard departure time is within the preset range, such as Figure 8 As shown, if the currently selected standard departure time is Ti = 370 and Jmax = 10, this operation can ensure that the adjusted standard departure time is between 365 and 375, that is, within the range of reducing (Jmax ÷ 2) to increasing (Jmax ÷ 2) on the basis of 370.

[0141] At this time, in the above step S702, based on the selected standard departure time, the candidate departure time corresponding to the target vehicle number in the current first state array is updated, specifically including:

[0142] Based on the adjusted standard departure time, the candidate departure time corresponding to the target vehicle number in the current first state array is updated.

[0143] In some embodiments, because the adjustment value is determined based on the maximum departure interval and the output length of the fully connected neural network cannot be changed after the model is instantiated, it is possible that the difference between the previously selected departure time and the currently selected standard time is small, or the adjusted standard departure time exceeds the next standard departure time. Based on this, embodiments of the present application provide a masking mechanism for the adjustment value to block unselectable adjustment values.

[0144] Optionally, the initial driving information also includes a target departure interval corresponding to the selected standard departure time. Based on the selection probability corresponding to each adjustment value, before determining the target adjustment value corresponding to the selected standard departure time from each adjustment value, it also includes: based on the numerical relationship between each adjustment value and the target departure interval, determining the mask value corresponding to each adjustment value; based on the mask value corresponding to each adjustment value, correcting the selection probability corresponding to each adjustment value.

[0145] Specifically, based on the numerical relationship between each adjustment value and the target departure interval, each adjustment is divided into a selectable adjustment value and a non-selectable adjustment value, and the selectable adjustment value and the non-selectable adjustment value are respectively set with different mask values, the value of the mask value can be pre-set, such as setting the mask value corresponding to the non-selectable adjustment value as a very small negative number (such as -9999999), and the mask value corresponding to the selectable adjustment value as 0. Then, the selection probability corresponding to each adjustment value and the mask value corresponding thereto are added to correct the selection probability corresponding to each adjustment value. For example, [-9999999, 0, 0] (mask value corresponding to each adjustment value) + [0.2, 0.6, 0.2] (selection probability corresponding to each adjustment value) = [-9999998.8, 0.6, 0.2] (corrected selection probability). In this way, the selection probability corresponding to the non-selectable adjustment value can be reduced, thereby avoiding being sampled and avoiding the problem of out-of-bound.

[0146] In some embodiments, the division of the selectable adjustment value and the non-selectable adjustment value can be based on the following principle: if the absolute value of the correction value A calculated based on the adjustment value is greater than the target departure interval, the adjustment value is a non-selectable adjustment value, otherwise the adjustment value is a selectable adjustment value. For example, when Jmax is 10 and the target departure interval is 3, the adjustment array is [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], and the correction value -4 or 4 may cause the standard departure time after correction to be misaligned, therefore, the adjustment values 1 and 9 are non-selectable adjustment values (corresponding to correction values -4 and 4, respectively).

[0147] At this time, based on the selection probability corresponding to each adjustment value, the target adjustment value corresponding to the selected standard departure time is determined from each adjustment value, including:

[0148] Based on the corrected selection probability, the target adjustment value corresponding to the selected standard departure time is determined from each adjustment value.

[0149] The above method forms an internal expression function based on the neural network learning the change rule of the departure interval and the single trip time, and continuously optimizes the combination of vehicles and times through the selection actions of vehicles and times based on reinforcement learning, finally forms a better combination of vehicles and times, so that the time selected by the vehicle is close to the departure interval requirement and the number of vehicles and drivers used is less, the labor of the driver is balanced and the station stop time is reasonable, thereby forming a better driving plan.

[0150] In some embodiments, when adjusting parameters based on the aforementioned first model, an overall evaluation cannot be performed until all data in the first state array has been updated. Furthermore, a local evaluation cannot represent the global optimum, and the trained first model may have poor universality due to different routes and time period conditions (e.g., changes in the number of assigned vehicles and departure times). To avoid the potential for inaccurate driving plans when determining a driving plan based solely on the target first state array output by the first model, embodiments of the present application provide a second model that generates a plan once and performs an overall evaluation based on the second model. The final driving plan is then determined by combining the driving plans generated by the first and second models to further optimize the driving plan.

[0151] Optionally, the second model includes a pointer network, such as Figure 9 As shown in Figure 2, the pointer network consists of three parts: encoding network, attention network, and decryption network, where:

[0152] The encoding network usually adopts a network such as long short-term memory (LSTM) or recurrent neural network (RNN) to encode the input and then transmit the output and hidden layer output to the attention network. The attention network usually adopts a fully connected neural network to improve the context (contextual relationship of departure time) understanding ability of the entire network and transmit the output with attention weight to the decoding network. The structure of the decoding network is similar to adding a fully connected layer on the basis of the encoding network structure to control the number of outputs; it is used to encode the received features and obtain the output results. Since the pointer network is an existing structure, the embodiment of the present application will no longer elaborate on its working principle.

[0153] Optional, such as Figure 10 As shown, the overall operating principle of the second model is as follows: the initial second state array is set as an output pointer network. Based on this pointer network, an output array corresponding to each candidate departure time slot is generated. The length of the output array is equal to the total number of vehicles + N (the value of N is set based on demand and can be greater than 1 to improve convergence speed). The corresponding vehicle number (i.e., the vehicle number corresponding to the minute of each time) is determined based on the output array corresponding to the candidate departure time slot. If the vehicle number is within the range corresponding to the total number of vehicles, the candidate departure time slot is assigned to the corresponding vehicle; otherwise, the candidate departure time slot is discarded. The entire plan is evaluated based on the vehicle allocation time, and the pointer network is reversely optimized based on the evaluation results through the optimizer.

[0154] Optionally, the initial driving information also includes the first candidate vehicle serial number corresponding to each vehicle. Figure 11As shown, the step S404 of generating the prediction result based on the second state array includes the following steps:

[0155] In step S1101, the second state array is subjected to feature extraction based on the attention mechanism, and the selection probability of each second candidate vehicle sequence number corresponding to each candidate departure time period is predicted based on the extracted features.

[0156] Each second candidate vehicle sequence number includes the first candidate vehicle sequence number.

[0157] In some embodiments, the second candidate vehicle sequence number is formed by adding N vehicle sequence numbers to the first candidate vehicle sequence number. For example, assuming that the initial traffic information includes a total of K vehicles, the first candidate vehicle sequence number is vehicle No. 1, vehicle No. 2, …, and vehicle No. K, and the second candidate vehicle sequence number includes vehicle No. 1, vehicle No. 2, …, vehicle No. K, vehicle No. K+1, …, and vehicle No. K+N. Vehicle No. K+1, …, and vehicle No. K+N are not included in the first candidate vehicle sequence number.

[0158] In a specific implementation, the initial second state array is input into a pointer network, encoded by the pointer network, and then input into a decoding network for decoding based on the attention mechanism to calculate the weight and obtain an output result. The output result includes the selection probability of each second candidate vehicle sequence number corresponding to each candidate departure time period.

[0159] In step S1102, the vehicle sequence number corresponding to each candidate departure time period is determined based on the selection probability of each second candidate vehicle sequence number.

[0160] In some embodiments, for each candidate departure time period, the second candidate vehicle sequence number corresponding to the maximum selection probability in the selection probability corresponding to the second candidate vehicle sequence number is determined as the vehicle sequence number corresponding to the candidate departure time period. Figure 10 As shown, the selection probability marked by bold in the output is the final selected selection probability.

[0161] In step S1103, the target departure time is selected from each candidate departure time period, and the prediction result is generated based on each target departure time and the corresponding vehicle sequence number. The vehicle sequence number corresponding to the target departure time belongs to the first candidate vehicle sequence number.

[0162] In some embodiments, after determining the vehicle number corresponding to each candidate departure time period, if the vehicle number corresponding to the candidate departure time period does not belong to the first candidate vehicle number, the corresponding candidate departure time period and its corresponding vehicle number are discarded. The remaining candidate departure time periods are used as target departure times, and based on each target departure time and its corresponding vehicle number, a list of vehicles and their corresponding departure times (i.e., the aforementioned prediction results) is generated. After generating the prediction results, if the prediction results do not include the vehicle label corresponding to a vehicle, the vehicle is considered discarded, and the total number of used vehicles is reduced by 1.

[0163] Assume that the first candidate vehicle numbers are car 1, car 2, ..., car K, and there are K+N second candidate vehicle numbers, including car 1, car 2, ..., car K, car K+1, ..., car K+N, among which cars K+1, ..., car K+N do not belong to the first candidate vehicle numbers.

[0164] like Figure 10 As shown, when the candidate departure time period is [1,361,401], the output is [0.1,0.05,0.1,...0.6]. If the second candidate vehicle number selected is vehicle K+N corresponding to 0.6, since vehicle K+N does not belong to the first candidate vehicle number, the candidate departure time period [1,361,401] and its selected vehicle number are discarded.

[0165] After generating the prediction result, a second value corresponding to the prediction result is determined, and the parameters of the second model are adjusted based on the second value. In some embodiments, an expected value can be set, and the error between the second value and the expected value can be calculated, and the network can be adjusted inversely based on the error.

[0166] Optional, such as Figure 12 As shown, determining the second value corresponding to the prediction result in the above step S404 specifically includes the following steps:

[0167] Step S1201, comparing the candidate departure time periods included in the prediction results with the standard departure time periods in the initial driving information to determine a first intermediate quantity;

[0168] In some embodiments, the above-mentioned first intermediate quantity, i.e., the number of time points, meets the standard value. For example, if the difference between the departure time in the prediction result and the standard departure time is within the set error (such as 5%), then the number of time points that meets the standard value is a negative number closer to 0, such as -0.05×difference; if it is greater than the set error, it is a penalty value, such as -3×difference within 20% and -6×difference above 20%.

[0169] Step S1202, determining whether the first intermediate quantity is greater than a preset value, if not, executing step S1203, and if so, executing step S1204; wherein the preset value can be set based on demand.

[0170] Step S1203, determining the first intermediate value as a second value corresponding to the prediction result;

[0171] Step S1204: determine the second intermediate quantity based on the intervals between candidate departure time periods, the number of vehicle numbers, and the candidate departure time periods corresponding to each vehicle number included in the prediction result, and determine the sum of the first intermediate quantity and the second intermediate quantity as the second numerical value corresponding to the prediction result.

[0172] In some embodiments, the second intermediate quantity includes an interval compliance value, a shift compliance value, an error value, a vehicle quantity compliance value, and an average stop time value, and its value may be the sum of the components.

[0173] The interval compliance value is the difference between the departure interval of the actual departure time and the departure interval of the standard departure time. Its value is negative or 0.

[0174] The class conformance value is the difference between the actual number of departure times and the standard number of single or double shift departure times. If the actual number of departure times is close to the standard number of single shift departure times, the current class is considered a single shift class. If the actual number of departure times is close to the standard number of double shift departure times, the current class is considered a double shift class. The difference between the current number of departure times and the standard number of departure times corresponding to the current class is then calculated to determine the class conformance value. The smaller the difference, the closer the score is to 0. The larger the difference, the class conformance value is a preset multiple of the difference.

[0175] The error value is divided into the stop time being less than the minimum stop time, or the stop time being greater than the maximum stop time. If the stop time is less than the minimum stop time, the error value can be set to a preset multiple of the difference (negative value).

[0176] The vehicle quantity compliance value is the difference between the current number of vehicles in use and the theoretical minimum number of vehicles in use. A negative value indicates that the closer the value is to the theoretical minimum number of vehicles in use, the higher the value. The theoretical minimum number of vehicles in use can be calculated as the number of departure times divided by the number of double shifts.

[0177] The mean value of the station stop time is the negative value of the average square deviation of the station stop time in the prediction results and the average value of the station stop time.

[0178] Based on the same inventive concept, the embodiment of the present application provides a device for generating urban driving strategies based on an AI large model. The device can realize the function of generating driving plans discussed above. Please refer to Figure 13The device includes at least one processor 1401 and a memory 1402 connected to the at least one processor. The specific connection medium between the processor 1401 and the memory 1402 is not limited in the embodiment of the present application. Figure 13 The processor 1401 and the memory 1402 can be connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 13 The fact that only one line is used does not mean that there is only one bus or one type of bus.

[0179] Among them, the processor 1401 is the control center of the electronic device, which can use various interfaces and lines to connect various parts of the electronic device, and realize data processing by running or executing instructions stored in the memory 1402 and calling data stored in the memory 1402. Optionally, the processor 1401 may include one or more processing units. The processor 1401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes the issuance of instructions. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1401. In some embodiments, the processor 1401 and the memory 1402 may be implemented on the same chip. In some embodiments, they may also be implemented separately on independent chips.

[0180] Processor 1401 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the test data generation method embodiment can be directly implemented as a hardware processor, or can be implemented using a combination of hardware and software modules in the processor.

[0181] The memory 1402, as a non-volatile computer readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 1402 can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 1402 is any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto. The memory 1402 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used for storing program instructions and / or data.

[0182] In the embodiments of the present application, the memory 1402 stores a computer program, which, when executed by the processor 1401, implements the method of the above driving plan generation.

[0183] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0184] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (system) and computer program product of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowcharts and / or block diagrams. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0185] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0187] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for generating urban driving strategies based on an AI large model, characterized in that: include: Obtaining initial driving information, and generating an initial first state array and an initial second state array based on the initial driving information; the initial first state array includes candidate departure times configured for each vehicle in the initial driving information, and the initial second state array includes multiple candidate departure time periods determined based on the earliest standard departure time and the latest vehicle arrival time in the initial driving information; Inputting the initial first state array into a first model, iteratively performing the following process until the first value reaches a preset threshold: updating the candidate departure times in the initial first state array based on the standard departure times in the initial driving information using the first model; Determining a first value corresponding to the updated first state array, and adjusting parameters of the first model based on the first value; The updating of the candidate departure times in the initial first state array based on the standard departure times in the initial driving information includes: According to the time sequence of each standard departure time in the initial driving information, select the standard departure time in turn, and iterate the following process until the last standard departure time: For the currently selected standard departure time, based on the current first state array, determining the target vehicle sequence number corresponding to the selected standard departure time; Based on the selected standard departure time, the candidate departure time corresponding to the target vehicle number in the current first state array is updated; using the first state array corresponding to the first value reaching the preset threshold as the target first state array, and generating a first driving plan based on the target first state array; Inputting the initial second state array into a second model, iteratively performing the following process until the second value is within a preset threshold range: generating a prediction result based on the second state array using the second model, the prediction result including a predicted vehicle number for each candidate departure time period; determining a second value corresponding to the prediction result, and adjusting parameters of the second model based on the second value; using the prediction result corresponding to the second value within the preset threshold range as the target prediction result, and generating a second driving plan based on the target prediction result; determining a target driving plan based on the first driving plan and the second driving plan; The initial driving information also includes the first candidate vehicle serial number corresponding to each vehicle. The generating of the prediction result based on the second state array includes: Extracting features from the second state array based on an attention mechanism, and predicting, based on the extracted features, the selection probability of each second candidate vehicle number corresponding to each candidate departure time period; wherein each second candidate vehicle number includes the first candidate vehicle number; Determine the vehicle number corresponding to each candidate departure time period based on the selection probability corresponding to each second candidate vehicle number; A target departure time is selected from each candidate departure time period, and the prediction result is generated based on each target departure time and the corresponding vehicle serial number; wherein the vehicle serial number corresponding to the target departure time belongs to the first candidate vehicle serial number.

2. The method according to claim 1, wherein The initial driving information also includes the first candidate vehicle serial number corresponding to each vehicle. The determining of the target vehicle serial number corresponding to the selected standard departure time based on the current first state array includes: Performing feature fusion on the first state array and, based on the feature vector obtained after fusion, predicting the selection probability of each first candidate vehicle number corresponding to the selected standard departure time; Based on the selection probability of the first candidate vehicle numbers, a target vehicle number corresponding to the selected standard departure time is determined from each first candidate vehicle number.

3. The method according to claim 1 or 2, wherein: After the candidate departure time corresponding to the target vehicle number in the current first state array is updated, the method further includes: Determining a third value corresponding to the update operation based on the current first state array and the first state array before the update; updating parameters in the first model based on the third value; The determining of the target vehicle sequence number corresponding to the selected standard departure time based on the current first state array includes: After the first model is updated based on the third value corresponding to the previous standard departure time, the target vehicle number corresponding to the selected standard departure time is determined through the updated first model based on the current first state array.

4. The method according to claim 1 or 2, wherein: Before updating the candidate departure time corresponding to the target vehicle number in the current first state array based on the selected standard departure time, the method further includes: Based on the current first state array, predicting the selection probability corresponding to each adjustment value corresponding to the selected standard departure time; the adjustment value is determined based on the maximum departure interval in the initial driving information; determining a target adjustment value corresponding to the selected standard departure time from the adjustment values ​​based on the selection probabilities corresponding to the adjustment values, and adjusting the selected standard departure time based on the target adjustment value; The updating of the candidate departure time corresponding to the target vehicle number in the current first state array based on the selected standard departure time includes: Based on the adjusted standard departure time, the candidate departure time corresponding to the target vehicle number in the current first state array is updated.

5. The method according to claim 3, wherein The initial driving information also includes a target departure interval corresponding to the selected standard departure time. Before determining the target adjustment value corresponding to the selected standard departure time from each adjustment value based on the selection probability corresponding to each adjustment value, the method further includes: Based on the numerical relationship between each adjustment value and the target departure interval, respectively determine the mask value corresponding to each adjustment value; Based on the mask value corresponding to each adjustment value, the selection probability corresponding to each adjustment value is modified respectively; The step of determining a target adjustment value corresponding to the selected standard departure time from the adjustment values ​​based on the selection probabilities corresponding to the adjustment values ​​includes: Based on the corrected selection probability, a target adjustment value corresponding to the selected standard departure time is determined from among the adjustment values.

6. The method according to claim 1, wherein The determining of a second value corresponding to the prediction result includes: Comparing the candidate departure time periods included in the prediction result with the standard departure time periods in the initial driving information to determine a first intermediate quantity; If the first intermediate quantity is not greater than the preset value, determining the first intermediate quantity as a second value corresponding to the prediction result; If the first intermediate quantity is greater than a preset value, the second intermediate quantity is determined based on the intervals between the candidate departure time periods, the number of vehicle serial numbers, and the candidate departure time periods corresponding to each vehicle serial number included in the prediction result, and the sum of the first intermediate quantity and the second intermediate quantity is determined as the second value corresponding to the prediction result.

7. A device for generating urban driving strategies based on an AI large model, characterized in that: The device includes at least one processor and at least one memory; wherein the memory stores program code, and when the program code is executed by the processor, the processor performs the following process: Obtaining initial driving information, and generating an initial first state array and an initial second state array based on the initial driving information; the initial first state array includes candidate departure times configured for each vehicle in the initial driving information, and the initial second state array includes multiple candidate departure time periods determined based on the earliest standard departure time and the latest vehicle arrival time in the initial driving information; Inputting the initial first state array into a first model, iteratively performing the following process until a first value reaches a preset threshold: updating candidate departure times in the initial first state array based on each standard departure time in the initial driving information using the first model; determining a first value corresponding to the updated first state array, and adjusting parameters of the first model based on the first value; The updating of the candidate departure times in the initial first state array based on the standard departure times in the initial driving information includes: According to the time sequence of each standard departure time in the initial driving information, select the standard departure time in turn, and iterate the following process until the last standard departure time: For the currently selected standard departure time, based on the current first state array, determining the target vehicle sequence number corresponding to the selected standard departure time; Based on the selected standard departure time, the candidate departure time corresponding to the target vehicle number in the current first state array is updated; using the first state array corresponding to the first value reaching the preset threshold as the target first state array, and generating a first driving plan based on the target first state array; Inputting the initial second state array into a second model, iteratively performing the following process until the second value is within a preset threshold range: generating a prediction result based on the second state array using the second model, the prediction result including a predicted vehicle number for each candidate departure time period; determining a second value corresponding to the prediction result, and adjusting parameters of the second model based on the second value; using the prediction result corresponding to the second value within the preset threshold range as the target prediction result, and generating a second driving plan based on the target prediction result; determining a target driving plan based on the first driving plan and the second driving plan; The initial driving information also includes the first candidate vehicle serial number corresponding to each vehicle. The generating of the prediction result based on the second state array includes: Extracting features from the second state array based on an attention mechanism, and predicting, based on the extracted features, the selection probability of each second candidate vehicle number corresponding to each candidate departure time period; wherein each second candidate vehicle number includes the first candidate vehicle number; Determine the vehicle number corresponding to each candidate departure time period based on the selection probability corresponding to each second candidate vehicle number; A target departure time is selected from each candidate departure time period, and the prediction result is generated based on each target departure time and the corresponding vehicle serial number; wherein the vehicle serial number corresponding to the target departure time belongs to the first candidate vehicle serial number.

Images