Station type shared car system dynamic management and control method and system considering platform game

Abstract

The invention belongs to the technical field of shared bicycles, and particularly relates to a station type shared bicycle system dynamic management and control method and system considering platform game, and the method comprises the steps: obtaining the related data of a target operation range, and initializing a simulation module; defining the state, action, income and state transition probability of the intelligent agent of the platform in a Markov state transition equation according to the operation target and management and control means of each platform, and initializing a neural network used for evaluating the action quality of the intelligent agent in a specific state to realize initialization of an optimization control module; supply and demand deduction and strategy optimization are carried out in the simulation module; judging whether the income of each platform in the simulation module tends to be stable or not; according to the shared bicycle-electric bicycle dynamic management and control method, a traditional shared bicycle static management and control normal form is broken through, and dynamic management and control of a shared bicycle-electric bicycle system are achieved in a high-dimensional random scene through the Markov decision process model and the deep reinforcement learning algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of shared vehicles, and more specifically, discloses a method and system for dynamically controlling a station-type shared vehicle system taking platform game into consideration. Background Art

[0002] With the continuous improvement of urbanization, the importance of green travel in alleviating global climate deterioration has become increasingly prominent. As a healthy and time- and cost-effective travel solution, shared bicycles (and shared electric motorcycles) provide more flexible first- and last-mile travel services for other large-capacity public transportation modes (subways, buses), effectively alleviating traffic congestion and environmental pollution.

[0003] The introduction of shared electric bikes has brought challenges to the original shared bike operations. First, the shared electric bike-bicycle hybrid system has a wider user audience, which enhances the heterogeneity of demand. The travel time, distance, and purpose are more diverse, which enhances the temporal and spatial imbalance of supply and demand and increases the difficulty of refined vehicle configuration. Secondly, the rebalancing and charging and battery replacement issues of shared electric bikes significantly increase the decision-making dimension.

[0004] However, most current research focuses on the operation of shared bicycle / shared electric motorcycle systems in a monopoly market, and few studies focus on the management and control strategies of shared (electric) bicycle systems under multi-platform competition. There is also a lack of pricing and scheduling methods for shared bicycle and electric motorcycle hybrid systems in the market. Summary of the invention

[0005] The present invention provides a method and system for dynamic control of a shared car system with a station considering platform game. The technical solution mainly includes: a simulation module based on multi-agents and an optimization control module based on deep reinforcement learning. The specific technical solution is as follows:

[0006] In one aspect of the present invention, a method for dynamic control of a shared car system with a station considering platform game is provided, comprising the following steps:

[0007] S1: Obtain relevant data of the target operation range and initialize the simulation module;

[0008] S2: According to the operation objectives and control methods of each platform, define the state, action, benefit and state transition probability of the large platform and small platform agents in the Markov state transition equation; and initialize the neural network used to evaluate the action quality of the agent in a specific state to realize the initialization optimization control module;

[0009] S3: Supply and demand deduction and strategy optimization in the simulation module;

[0010] S4: Determine whether the income of each platform in the simulation module tends to be stable;

[0011] S5: Output control plan.

[0012] Preferably, the target operating scope-related data in S1 include the number and capacity of dispatched trucks, the number of shared bicycles on each platform, location data, the number of shared electric motorcycles, power, location data and order data.

[0013] Preferably, S1 comprises the following steps:

[0014] S11: Divide the operation scope into several operation sub-areas to realize the scheduling task;

[0015] S12: define the states of each type of intelligent agent and the interaction mode in the simulation module, and determine the charging and replacement power and driving power consumption rate of the shared electric motorcycle at the station;

[0016] S13: Calibrate the user travel choice model.

[0017] S14: Generate a large platform Shared bicycles, Shared electric bikes, small platforms Shared bicycles, The spatial position of the shared electric motorcycles and the power β of the electric motorcycles on each platform are initialized;

[0018] S15: Use List B 1 and B 2 Real-time recording of available vehicles on both platforms.

[0019] Preferably, S2 comprises the following steps:

[0020] S21: Define the status, actions, and benefits of the large and small platforms; the status of the large platform is defined as

[0021] The status includes the current motorcycle pricing of the platform and the operating status of the trucks belonging to the platform, written as t represents the time period information, represents the current motorcycle pricing, represents the state of the mth truck;

[0022] It consists of the truck's mission type and mission start and end points. Indicates the task type, which includes transporting all shared bicycles, transporting all shared motorcycles, transporting half bicycles and half motorcycles, and not transporting.

[0023] in, The start and end points of the truck mission were coded separately;

[0024] in, is the external state of the large platform, including the demand d during this period t , the location information of its shared bicycles and motorcycles, and

[0025] Among them, demand can be predicted by deducing historical data;

[0026] is the external state of the small platform, including the demand d during this period t , location information of small platform shared bicycles and motorcycles and It also contains the vehicle location information of the large platform and And its motorcycle pricing

[0027] Among them, the platform action represents the control plan that the platform should complete in the next period t+1, l∈{1,2} represents any platform; among them, the platform action is defined as Includes pricing for shared electric bikes M trucks’ mission types and work areas

[0028] The state transition rule of the platform can be expressed as follows:

[0029]

[0030] The platform's revenue is the sum of the revenue from shared bicycles and electric motorcycle rentals. Minus the trucking costs The income from shared bikes and electric motorcycles is a function of distance;

[0031] S22: Initialize the deep reinforcement learning algorithm, including using the D3QN algorithm (Double Dueling Deep Q-Network), determine the number of neural network layers, the number of hidden units, and the learning rate parameters, and obtain the initial strategy evaluation network and experience pool for the large platform and the small platform respectively.

[0032] Preferably, S3 includes generating riding demands according to order data in each sub-region within t time periods of a day, and passengers select a certain type of vehicle on a certain platform in the sub-region according to the random utility theory, and the platform decides the action in the current time period. Since multiple platforms are involved here, a modified nested Logit model is used to describe the selection behavior among multiple platforms and multiple models; it includes:

[0033] Passengers first determine the type of vehicle to be used, whether it is a shared bicycle, shared electric motorcycle, or other transportation mode, and then choose a large or small platform;

[0034] S31: For passengers whose travel destinations are in areas o and o', the probability of choosing a shared bike can be expressed as:

[0035]

[0036] According to the nested Logit model, the relative utility value of shared bicycles can be obtained. represents the number of shared bikes available on platform l in area o, θ 1 is the model parameter, and the relative utility value of a shared bicycle on a certain platform is:

[0037]

[0038] where d o,o' represents the driving distance between two areas. Indicates the cost, represents the time cost of traveling, I O is the utility value of choosing other modes, ∈ is the random term caused by individual selection differences, p b Indicates the inertia of using shared bikes. Indicates the discomfort caused by physical exertion while the utility of shared electric bikes does not contain p b and ε(d o,o' );

[0039] If a passenger chooses to use a shared electric motorcycle, the platform will give priority to recommending high-capacity bicycles at the station.

[0040] After the passenger completes the vehicle selection, he / she will reach the destination according to the set shared bicycle / motorcycle running speed, and then update the location of the shared bicycle, motorbike, and the battery level of the shared motorbike.

[0041] S32: During the supply and demand simulation, when the time period t ends, the optimization control module first reads the state of the large platform from the simulation module, and then determines the action through the greedy strategy:

[0042] Generate a random number between 0 and 1. When the value is less than δ, randomly generate an action for the large platform.

[0043] If it is greater than δ, the neural network outputs the optimal action in the current state and passes it to the simulation module. Based on the action, the big platform determines the price of motorcycles in the next period and dispatches trucks to perform the dispatching task, and the state of the big platform is updated;

[0044] Subsequently, the optimization control module reads the status of the small platform from the simulation module, and also uses the greedy strategy to obtain the action of the small platform, which is then passed to the simulation module. The platform also performs actions according to the output control plan, determines the price of the motorcycle and the task of the truck, and updates the status.

[0045] When the period t+1 ends, the two platforms calculate their own profits and transmit them to the optimization control module in combination with the status and actions in the period, and store them in the experience pool;

[0046] The D3QN algorithm collects several batches of record information from the experience pool According to its loss function, the gradient descent method is used to update the strategy to evaluate the neural network. The loss function can be expressed as:

[0047]

[0048] Where ω is the neural network parameter, R(s t ,a t ) is the return in period t, Q(s t ,a t ,ω) is the training network, For the target network, According to the training network, the optimal action for the next period is obtained. 0 Rounds, synchronize the parameters of the target network with the training network;

[0049] Preferably, the method for judging whether the income of each platform in the simulation module tends to be stable in S4 is:

[0050] When the platform's cumulative revenue after t periods of time in one day tends to be stable, or the algorithm reaches the maximum number of iterations, the optimization process can be considered to be over.

[0051] Preferably, obtaining a real-time control plan in S4 includes inputting the state in the simulation module at any time into the trained optimization control module, wherein the state includes the current period demand d t , location information of small platform shared bicycles and motorcycles and Large platform shared bicycle and motorcycle location information and The optimization control module can generate optimized dynamic vehicle rebalancing and shared electric motorcycle pricing plans for large platforms and small platforms based on the parameters in this state.

[0052] In a second aspect, the present invention provides a dynamic control system for a shared car system with a station taking into account platform game, the system comprising:

[0053] An acquisition unit, the acquisition unit is used to acquire data related to the target operation range and initialize the simulation module;

[0054] A computing unit, which is used to define the state, action, benefit and state transition probability of the platform agent in the Markov state transition equation according to the operation objectives and control means of each platform, and initialize a neural network for evaluating the action quality of the agent in a specific state, so as to realize the initialization optimization control module;

[0055] An optimization unit, the optimization unit is used for supply and demand deduction and strategy optimization in the simulation module;

[0056] A judgment unit, the judgment unit is used to judge whether the income of each platform in the simulation module tends to be stable;

[0057] An output unit, wherein the output unit is used to output a control plan.

[0058] In the third aspect, the present invention provides a dynamic control device for a station-type shared car system taking into account platform game, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the program: obtaining data related to the target operating range and initializing the simulation module; defining the state, action, benefit and state transition probability of the platform agent in the Markov state transfer equation according to the operating objectives and control measures of each platform, and initializing a neural network for evaluating the action quality of the agent under a specific state, so as to realize the initialization optimization control module; supply and demand deduction and strategy optimization in the simulation module; judging whether the benefits of each platform in the simulation module tend to be stable; and outputting the control plan.

[0059] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps: obtaining data related to the target operating range and initializing a simulation module; defining the state, action, benefit and state transition probability of the platform agent in the Markov state transfer equation according to the operating objectives and control measures of each platform, and initializing a neural network for evaluating the action quality of the agent under a specific state, thereby initializing an optimization control module; supply and demand deduction and strategy optimization in the simulation module; determining whether the benefits of each platform in the simulation module tend to be stable; and outputting a control plan.

[0060] Beneficial effects:

[0061] 1. The present invention breaks through the traditional static control paradigm of shared bicycles, and uses the Markov decision process model and deep reinforcement learning algorithm to realize the dynamic control of the shared bicycle-electric motorcycle system in high-dimensional random scenarios.

[0062] 2. Based on game theory, the present invention constructs a multi-platform Markov decision process model, which can provide decision makers with management insights in a heterogeneous multi-agent competitive market, reduce the waste of resources caused by competition, and achieve a win-win situation for users and multiple platforms.

[0063] 3. The present invention fully considers the user's selection preferences and simulates the passenger's selection behavior between multiple platforms and multiple models based on the random utility theory, which is more in line with the actual situation and improves the feasibility of problem solving. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a structural schematic diagram of the control method of the present invention;

[0065] Figure 2 It is a schematic diagram of the execution process of the simulation module of the present invention;

[0066] Figure 3 It is a schematic diagram of the execution process of the optimization control module of the present invention. DETAILED DESCRIPTION

[0067] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0068] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0069] Embodiment 1: The present invention provides a method for dynamic control of a shared car system with a station taking into account platform game, comprising the following steps:

[0070] S1: Obtain relevant data of the target operation range and initialize the simulation module;

[0071] S2: According to the operation objectives and control methods of each platform, including pricing and scheduling, define the state, action, benefit and state transition probability of the platform agent in the Markov state transition equation; and initialize the neural network used to evaluate the action quality of the agent in a specific state to realize the initialization optimization control module;

[0072] S3: Supply and demand deduction and strategy optimization in the simulation module;

[0073] S4: Determine whether the income of each platform in the simulation module tends to be stable;

[0074] S5: Output control plan.

[0075] Furthermore, the data related to the target operating scope in S1 include the number and capacity of dispatched trucks, the number of shared bicycles on each platform, location data, the number of shared electric motorcycles, power consumption, location data and order data.

[0076] Furthermore, S1 includes the following steps:

[0077] S11: Divide the operation scope into several operation sub-areas to realize the scheduling task;

[0078] S12: defining the states of various types of intelligent agents and their interaction modes in the simulation module, and determining the charging and swapping power and driving power consumption rate of the shared electric motorcycles at the station; wherein the intelligent agents include the platform, the shared bicycles, and the shared electric motorcycles;

[0079] S13: calibrate the user travel choice model, that is, use the maximum likelihood estimation method to calibrate the relevant parameters in the discrete choice model based on the user travel choice records, and determine the probability of choosing a shared electric motorcycle as exogenous variables such as distance and price change;

[0080] S14: Generate a large platform Shared bicycles, Shared electric bikes, small platforms Shared bicycles, The spatial position of the shared electric motorcycles and the power β of the electric motorcycles on each platform are initialized;

[0081] S15: Use List B 1 and B 2 Real-time record of available vehicles on the two platforms. When the power of the motorcycle is greater than the threshold, β>β 0 , marked as available;

[0082] Specifically, S2 includes the following steps:

[0083] S21: Define the status, actions, and benefits of the large and small platforms; the status of the large platform is defined as This is used as an algorithm to obtain specified input information for subsequent control;

[0084] The status includes the current motorcycle pricing of the platform and the operating status of the trucks belonging to the platform, written as t represents the time period information, represents the current motorcycle pricing, represents the state of the mth truck;

[0085] Among them, truck refers to a truck with transport loading capacity, which can be used to transport shared bicycles / motorcycles;

[0086] There are M trucks on the platform, where m is a positive integer from 1 to M. It consists of the truck's mission type and mission start and end points. Indicates the task type, which includes transporting all shared bicycles, transporting all shared motorcycles, transporting half bicycles and half motorcycles, and not transporting.

[0087] in, The start and end points of the truck mission were coded separately;

[0088] in, is the external state of the large platform, including the demand d during this period t , the location information of its shared bicycles and motorcycles, and

[0089] Demand can be predicted through historical data deduction, such as inputting platform historical data into machine learning and deep learning models to estimate current demand that has not occurred, or obtained in real time within the system, such as collecting users' real-time travel needs on the App side based on user inquiries, reservations and other records at every unit time;

[0090] is the external state of the small platform, including the demand d during this period t , location information of small platform shared bicycles and motorcycles and It also contains the vehicle location information of the large platform and And its motorcycle pricing Set as a follower of decision making;

[0091] Among them, the platform action represents the control plan that the platform should complete in the next period t+1, including the rebalancing and pricing plan of shared electric motorcycles / bikes, and l∈{1,2} represents any platform;

[0092] Among them, the platform action is defined as Includes pricing for shared electric bikes M trucks’ mission types and work areas

[0093] The state transition rule of the platform can be expressed as follows:

[0094]

[0095] The platform's revenue is the sum of the revenue from shared bicycles and electric motorcycle rentals. Minus the trucking costs The income from shared bikes and electric motorcycles is a function of distance;

[0096] Among them, the platform's revenue refers to the tiered pricing rules that can be referred to by mainstream shared travel companies in the market. Shared electric motorcycles are priced according to the price within 30 minutes. Charge, after 30 minutes, an additional charge of τ is charged for every 10 minutes e Yuan; shared bikes charge τ every 30 minutes b Yuan (less than 30 minutes will be calculated as 30 minutes);

[0097] S22: Initialize the deep reinforcement learning algorithm, including using the D3QN algorithm (Double Dueling Deep Q-Network), determine the number of neural network layers, the number of hidden units, the learning rate and other parameters, obtain the initial policy evaluation network and experience pool for the large platform and the small platform respectively, and then perform iterative optimization on the initial policy evaluation network; the initial experience pool is an empty set, which is used to store the experience learned during the training process.

[0098] Further, S3 includes generating riding demands based on order data in each sub-region within T time periods of a day. Passengers follow the random utility theory to choose a certain type of vehicle on a certain platform in the sub-region, and the platform decides the action in the current period. Since multiple platforms are involved here, the modified nested Logit model is used to characterize the selection behavior among multiple platforms and multiple models. Among them, passengers first determine the type of vehicle to be used, shared bicycles, shared electric motorcycles or other transportation modes (walking, public transportation, etc.), and then choose a large platform or a small platform, including:

[0099] S31: For passengers whose travel destinations are in areas o and o', the probability of choosing a shared bicycle can be expressed as follows (the same applies to shared electric motorcycles):

[0100]

[0101] According to the nested Logit model, the relative utility value of shared bicycles can be obtained. represents the number of shared bikes available on platform l in area o, θ 1 is the model parameter, and the relative utility value of a shared bicycle on a certain platform is:

[0102]

[0103] where d o,o' represents the driving distance between two areas. Indicates the cost, represents the time cost of traveling, I O is the utility value of choosing other modes, ∈ is the random term caused by individual selection differences, p b Indicates the inertia of using shared bikes. Indicates the discomfort caused by physical exertion while the utility of shared electric bikes does not contain p b and ε(d o,o' ).

[0104] This step calculates the probability of passengers choosing shared electric bikes and bicycles on large and small platforms, thereby simulating travel demand and implementing algorithm training;

[0105] If passengers choose to use shared electric bikes, the platform will give priority to recommending high-powered bikes at the station;

[0106] After the passenger has selected a vehicle, the passenger will reach the destination at the set shared bicycle / motorcycle running speed, and then update the location of the shared bicycle / motorcycle, as well as the power level of the shared motorcycle;

[0107] S32: During the supply and demand simulation, when the time period t ends, the optimization control module first reads the state of the large platform from the simulation module, and then determines the action through the greedy strategy:

[0108] Generate a random number between 0 and 1. When the value is less than δ, randomly generate an action for the large platform.

[0109] If it is greater than δ, the neural network outputs the optimal action in the current state and passes it to the simulation module. Based on the action, the big platform determines the price of motorcycles in the next period and dispatches trucks to perform the dispatching task, and the state of the big platform is updated;

[0110] Subsequently, the optimization control module reads the status of the small platform from the simulation module, and also uses the greedy strategy to obtain the action of the small platform, which is then passed to the simulation module. The platform also performs actions according to the output control plan, determines the price of the motorcycle and the task of the truck, and updates the status.

[0111] When the period t+1 ends, the two platforms calculate their own profits and transmit them to the optimization control module in combination with the status and actions in the period, and store them in the experience pool;

[0112] The D3QN algorithm collects several batches (usually 32 or 64) of record information from the experience pool. According to its loss function, the gradient descent method is used to update the strategy to evaluate the neural network. The loss function can be expressed as:

[0113]

[0114] Where ω is the neural network parameter, R(s t ,a t ) is the return in period t, Q(s t ,a t ,ω) is the training network, For the target network, According to the training network, the optimal action for the next period is obtained. 0 In this round, the parameters of the target network are synchronized with the training network. This step optimizes the strategy evaluation network of the large platform and the small platform to improve the quality of action decision-making.

[0115] Furthermore, the method for judging whether the income of each platform in the simulation module tends to be stable in S4 is to achieve convergence by using an algorithm, including:

[0116] When the platform's cumulative revenue after t periods of time in one day tends to be stable, or the algorithm reaches the maximum number of iterations, the optimization process can be considered to be over. Specifically, the optimization process includes the calculation of S1-S3. When the algorithm does not converge, the optimization process of S3 is repeated until convergence;

[0117] Furthermore, obtaining a real-time control plan in S4 includes inputting the state in the simulation module at any time into the trained optimization control module, wherein the state includes the current period demand d t , location information of small platform shared bicycles and motorcycles and Large platform shared bicycle and motorcycle location information and The optimization control module can generate optimized dynamic vehicle rebalancing and shared electric motorcycle pricing plans for large platforms and small platforms based on the parameters in this state.

[0118] Embodiment 2: The present invention provides a station-type shared car system dynamic management and control system considering platform game, the system comprising:

[0119] An acquisition unit, the acquisition unit is used to acquire data related to the target operation range and initialize the simulation module;

[0120] A computing unit, which is used to define the state, action, benefit and state transition probability of the platform agent in the Markov state transition equation according to the operation objectives and control means of each platform, and initialize a neural network for evaluating the action quality of the agent in a specific state, so as to realize the initialization optimization control module;

[0121] An optimization unit, the optimization unit is used for supply and demand deduction and strategy optimization in the simulation module;

[0122] A judgment unit, the judgment unit is used to judge whether the income of each platform in the simulation module tends to be stable;

[0123] An output unit, wherein the output unit is used to output a control plan.

[0124] Embodiment 3: The present invention provides a dynamic control device for a station-type shared car system taking into account platform game, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the program: obtaining data related to the target operating range and initializing the simulation module; defining the state, action, benefit and state transition probability of the platform agent in the Markov state transfer equation according to the operating objectives and control measures of each platform, and initializing a neural network for evaluating the action quality of the agent under a specific state, so as to initialize the optimization control module; performing supply and demand deduction and strategy optimization in the simulation module; determining whether the benefits of each platform in the simulation module tend to be stable; and outputting a control plan.

[0125] Embodiment 4: The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps: obtaining data related to the target operating range and initializing a simulation module; defining the state, action, benefit and state transition probability of the platform agent in the Markov state transfer equation according to the operating objectives and control measures of each platform, and initializing a neural network for evaluating the action quality of the agent under a specific state, thereby initializing an optimization control module; supply and demand deduction and strategy optimization in the simulation module; determining whether the benefits of each platform in the simulation module tend to be stable; and outputting a control plan.

[0126] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A dynamic control method for a shared car system with a station considering platform game, characterized by: The steps include: S1: Obtain relevant data of the target operation range and initialize the simulation module; S2: According to the operation objectives and control methods of each platform, define the state, action, benefit and state transition probability of the platform agent in the Markov state transition equation; and initialize the neural network used to evaluate the action quality of the agent in a specific state to realize the initialization optimization control module; S3: Supply and demand deduction and strategy optimization in the simulation module; S4: Determine whether the income of each platform in the simulation module tends to be stable; S5: Output control plan.

2. The method for dynamic control of a shared car system with a station considering platform game as claimed in claim 1, characterized in that: The data related to the target operation scope in S1 include the number and capacity of dispatch trucks, the number of shared bicycles on each platform, location data, the number of shared electric motorcycles, power consumption, location data and order data.

3. The method for dynamic control of a shared car system with a station considering platform game as claimed in claim 2, characterized in that: S1 includes the following steps: S11: Divide the operation scope into several operation sub-areas to realize the scheduling task; S12: define the states and interaction modes of various types of intelligent entities (i.e., large platforms, small platforms, shared (electric) bicycles, and passengers) in the simulation module, and determine the charging and replacement power and driving power consumption rate of shared electric bicycles at the station; S13: Calibrate user travel choice model; S14: Generate a large platform Shared bicycles, Shared electric bikes, small platforms Shared bicycles, The spatial position of the shared electric motorcycles and the power β of the electric motorcycles on each platform are initialized; S15: Use List B 1 and B 2 Real-time recording of available vehicles on both platforms.

4. The method for dynamic control of a shared car system with a station considering platform game as claimed in claim 1, characterized in that: S2 includes the following steps: S21: Define the status, actions, and benefits of large and small platforms; the status of the large platform is defined as The status includes the current motorcycle pricing of the platform and the operating status of the trucks belonging to the platform, written as t represents the time period information, represents the current motorcycle pricing, represents the state of the mth truck; It consists of the truck's mission type and mission start and end points. Indicates the task type, which includes transporting all shared bicycles, transporting all shared motorcycles, transporting half bicycles and half motorcycles, and not transporting. in, The start and end points of the truck mission were coded separately; in, is the external state of the large platform, including the demand d during this period t , the location information of its shared bicycles and motorcycles, and Among them, demand can be predicted by deducing historical data; is the external state of the small platform, including the demand d during this period t , location information of small platform shared bicycles and motorcycles and It also contains the vehicle location information of the large platform and And its motorcycle pricing Among them, the platform action represents the control plan that the platform should complete in the next period t+1, l∈{1,2} represents any platform; among them, the platform action is defined as Includes pricing for shared electric bikes M trucks’ mission types and work areas The state transition rule of the platform can be expressed as follows: The platform's revenue is the sum of the revenue from shared bicycles and electric motorcycle rentals. Minus the trucking costs The income from shared bikes and electric motorcycles is a function of distance; S22: Initialize the deep reinforcement learning algorithm, including using the D3QN algorithm to determine the number of neural network layers, the number of hidden units, and the learning rate parameters, and obtain the initial strategy evaluation network and experience pool for the large platform and the small platform respectively.

5. The method for dynamic control of a shared car system with a station considering platform game as claimed in claim 1, characterized in that: S3 includes generating riding demands based on order data in each sub-region within T time periods of a day. Passengers follow the random utility theory to choose a certain type of vehicle on a certain platform in the sub-region, and the platform decides the action in the current period. Since multiple platforms are involved here, the modified nested logit model is used to describe the selection behavior among multiple platforms and multiple models. These include: Passengers first determine the type of vehicle to be used, whether it is a shared bicycle, shared electric motorcycle, or other transportation mode, and then choose a large or small platform; S31: For passengers whose travel destinations are in areas o and o', the probability of choosing a shared bike can be expressed as: According to the nested Logit model, the relative utility value of shared bicycles can be obtained. represents the number of shared bicycles available on platform l in area o, θ1 is the model parameter, and the relative utility value of a shared bicycle on a certain platform is: where d o,o' represents the driving distance between two areas. Indicates the cost, represents the time cost of traveling, I O is the utility value of choosing other modes, ∈ is the random term caused by individual selection differences, p b Indicates the inertia of using shared bikes. Indicates the discomfort caused by physical exertion while the utility of shared electric bikes does not contain p b and ε(d o,o' ); If passengers choose to use shared electric bikes, the platform will give priority to recommending high-powered bikes at the station; After the passenger has selected a vehicle, the passenger will reach the destination at the set shared bicycle / motorcycle running speed, and then update the location of the shared bicycle / motorcycle, as well as the power level of the shared motorcycle; S32: During the supply and demand simulation, when the time period t ends, the optimization control module first reads the state of the large platform from the simulation module, and then determines the action through the greedy strategy: Generate a random number between 0 and 1. When the value is less than δ, randomly generate an action for the large platform. If it is greater than δ, the neural network outputs the optimal action in the current state and passes it to the simulation module. Based on the action, the big platform determines the price of motorcycles in the next period and dispatches trucks to perform the dispatching task, and the state of the big platform is updated; Subsequently, the optimization control module reads the status of the small platform from the simulation module, and also uses the greedy strategy to obtain the action of the small platform, which is then passed to the simulation module. The platform also performs actions according to the output control plan, determines the price of the motorcycle and the task of the truck, and updates the status. When the period t+1 ends, the two platforms calculate their own profits and transmit them to the optimization control module in combination with the status and actions in the period, and store them in the experience pool; The D3QN algorithm collects several batches of record information from the experience pool According to its loss function, the gradient descent method is used to update the strategy to evaluate the neural network. The loss function can be expressed as: Where ω is the neural network parameter, R(s t ,a t ) is the return in period t, Q(s t ,a t ,ω) is the training network, For the target network, The optimal action for the next period is obtained based on the training network. Every n0 rounds, the parameters of the target network are synchronized with the training network.

6. The method for dynamic control of a shared car system with a station considering platform game as claimed in claim 1, characterized in that: The method for judging whether the income of each platform in the simulation module tends to be stable in S4 is: When the platform's cumulative revenue after t periods of time in one day tends to be stable, or the algorithm reaches the maximum number of iterations, the optimization process can be considered to be over.

7. The method for dynamic control of a shared car system with a station considering platform game as claimed in claim 6, characterized in that: The real-time control scheme obtained in S4 includes inputting the state in the simulation module at any time into the trained optimization control module, wherein the state includes the current period demand d t , location information of small platform shared bicycles and motorcycles n 2 e and n b 2 ,,Large platform shared bicycle and motorcycle location informationn 1 e and n b 1 ,The optimization control module can generate optimized dynamic vehicle rebalancing and shared electric motorcycle pricing schemes for large and small platforms based on the parameters in this state.

8. A dynamic control system for a shared car system with a station considering platform game, characterized by: The system comprises: An acquisition unit, the acquisition unit is used to acquire data related to the target operation range and initialize the simulation module; A computing unit, which is used to define the state, action, benefit and state transition probability of the platform agent in the Markov state transition equation according to the operation objectives and control means of each platform, and initialize a neural network for evaluating the action quality of the agent in a specific state, so as to realize the initialization optimization control module; An optimization unit, the optimization unit is used for supply and demand deduction and strategy optimization in the simulation module; A judgment unit, the judgment unit is used to judge whether the income of each platform in the simulation module tends to be stable; An output unit, wherein the output unit is used to output a control plan.

9. A dynamic control device for a shared car system with a station considering platform game, characterized in that: It includes a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: obtaining data related to the target operation range and initializing the simulation module; defining the state, action, benefit and state transition probability of the platform agent in the Markov state transition equation according to the operation objectives and control measures of each platform, and initializing the neural network used to evaluate the action quality of the agent under a specific state, so as to realize the initialization optimization control module; supply and demand deduction and strategy optimization in the simulation module; judging whether the benefits of each platform in the simulation module tend to be stable; and outputting the control plan.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps: obtaining data related to the target operating range and initializing a simulation module; defining the state, action, benefit and state transition probability of the platform agent in the Markov state transition equation according to the operating objectives and control measures of each platform, and initializing a neural network for evaluating the action quality of the agent under a specific state, thereby initializing an optimization control module; supply and demand deduction and strategy optimization in the simulation module; determining whether the benefits of each platform in the simulation module tend to be stable; and outputting a control plan.

Images