A spatio-temporal resource dynamic allocation method based on markov tree search

Abstract

In the field of large-scale operation control, especially in the field of traffic hub management and urban operation control, space-time resource allocation is a common and key problem. Such problems require allocating limited space resources and time use rights to multiple consumers within a given total time interval and space range. The demand for resources fluctuates dynamically over time. Decision makers need to develop an allocation plan that meets resource capacity constraints, adapts to dynamic environments, and achieves global optimization. This paper provides a space-time resource allocation scheme that can balance computational efficiency, adapt to time-varying demand, and achieve near-global optimization. This method should inherit the accuracy of mixed integer programming in generating high-quality solutions under strong constraints, and have the ability to make long-term deductions and global trade-offs to overcome the inherent shortcomings of traditional rolling optimization. By introducing Markov tree search (MCTS) as the upper planning and modeling framework, mixed integer programming is regarded as an action generator for each decision, and a simulation method is used to search for the global optimal path in the candidate allocation sequence, and to build a model to drive the search, thereby efficiently solving the space-time resource allocation problem for time-varying demand.

Description

Technical Field

[0001] This invention relates to the field of resource scheduling and operations optimization technology, and in particular to a dynamic allocation method, apparatus, electronic device and storage medium for time and space resources. Background Technology

[0002] In scenarios such as cloud computing, logistics and delivery, wireless spectrum allocation, and shared mobility, spatiotemporal resource allocation problems are widespread. These problems typically involve a total time interval T and a spatial range S, with multiple objects C consuming spatiotemporal resources. Each object's resource requirements change dynamically over time. The goal is to allocate limited spatial resources and time usage rights to each object to optimize a certain global objective O (such as maximizing total revenue, minimizing total cost, or maximizing resource utilization).

[0003] Traditional solution methods include static mixed integer programming (MIP) [2] and rolling time domain optimization. Static MIP treats the entire time interval as a one-time decision problem, establishes a large-scale mixed integer programming model and solves it. However, this type of method has significant shortcomings: First, as the time slice is subdivided and the spatial scale expands, the model variables and constraints explode, and the solution time is so long that it is even impossible to obtain a feasible solution within an acceptable time. Second, the dynamic change of demand over time requires the model to have online adjustment capabilities, while static MIP is difficult to adapt to environmental changes. Rolling time domain optimization divides the time interval into several short time windows, solves the local MIP problem in each window and only executes the current window decision, and then rolls forward. Although this method can cope with time-varying demand, each decision only considers a limited look-ahead window, which is essentially a short-sighted greedy strategy and is prone to falling into global suboptimal. Another type of method is represented by deep reinforcement learning. Although it can learn long-term strategies, it requires a large amount of training data, the reward design is complex and the decision process lacks interpretability. In strongly constrained combinatorial optimization problems such as resource allocation, it is difficult to guarantee the quality and feasibility of the solution.

[0004] Therefore, a new spatiotemporal resource allocation scheme is urgently needed, which can effectively cope with the time-varying characteristics of object requirements while taking into account computational efficiency, and achieve near-global optimal allocation planning. This invention proposes a spatiotemporal resource allocation scheme that can take into account computational efficiency, adapt to time-varying requirements and achieve near-global optimality. This method should inherit the accuracy of generating high-quality solutions under strong constraints of mixed integer programming, and also have the ability to perform long-term deduction and global trade-offs, so as to overcome the inherent defects of short-sightedness in traditional rolling optimization. By introducing Markov Tree Search (MCTS) [1] as the upper-level planning and modeling framework, mixed integer programming is regarded as the action generator for each step of decision, and the global optimal path is found in the candidate allocation sequence by means of search simulation driven by machine learning model (see Appendix). Figure 1 This allows for efficient solutions to spatiotemporal resource allocation problems oriented towards time-varying needs. Summary of the Invention

[0005] The technical problem to be solved by this invention is: within a time interval T and a spatial range S, how to allocate resources to a set of objects C that consume spatiotemporal resources, where each object has a demand C. C As time t changes, how can we efficiently decide on resource allocation for each time slice to maximize the global optimization objective O? To address this problem, this invention provides a spatiotemporal resource allocation method based on Markov Tree Search (MCTS, also known as Monte Carlo Tree Search). This method models the sequential allocation process as a Markov decision process, uses mixed-integer programming to generate high-quality candidate actions, and performs global evaluation and optimization of the candidate action sequence through Markov Tree Search, combining the accuracy of operations research with the long-term planning capabilities of tree search. The method of this invention includes two parts: model training and inference (application). The model training process (see attached diagram)... Figure 2 This includes the following steps: Step S1: Obtain allocation parameters, including the total time interval T, spatial range S, optimization objective O, the set of objects consuming spatiotemporal resources C, and the demand information C of each object as it changes over time. C ; Step S1: Obtain allocation parameters, including the total time interval T, spatial range S, optimization objective O, the set of objects consuming spatiotemporal resources C, and the demand information C of each object as it changes over time. C ; Step S3: Initialize the current time slice index i=1, and the system state reflecting information such as the allocated resources and object status; Step S4: In the current time slice t i Based on the system state, a mixed-integer programming model is constructed. The spatial range of this model is S, and the time range is from t... i The time interval from the start point to the end of the entire time interval T (i.e., covering all remaining time slices). The objective function is to optimize objective O, and the resource capacity R is... t Object requirement C Ci (C corresponding to the i-th time slice) C Using constraints such as ) and ) to solve for K feasible allocation schemes (K≥2); Step S5: Construct a search tree with time slices as the decision level, with the root node corresponding to time slice t. i Given the current system state, the child nodes of the root node correspond to the K candidate solutions. A Markov tree search algorithm is applied to iteratively select, expand, simulate, and backtrack the search tree to find the solution from t. i To t N The cumulative optimization target value is used as the evaluation criterion to determine the time slice t. i The optimal allocation scheme p i ; Step S6: Execute the optimal allocation scheme p i In time slice t i Complete resource allocation and update the system state within the specified time slice. Let i = i + 1. If i ≤ N, return to step S4 to continue the decision for the next time slice; otherwise, the allocation ends. Step S7: Collect data from the resource states (S, R) t Demand State C CI Allocation scheme p i The sequence D constitutes j = [(S,R0, C C0 (, p0), (S, R1, C C1 , p1) … (S, R N C CN , p N Collect M scenes (different S, R, Cc) to form a training dataset D=[D0, D1, … D]; N Based on D, a recommendation model M is constructed: M: p i = f(S, R i C Ci S: spatial range, R i In the i-th time slice of time range R, C Ci Resource requirements corresponding to the i-th time slice In step S5, the Markov tree search algorithm (see attached) Figure 3 Specifically, it includes: Selection: Starting from the root node, recursively select child nodes according to the confidence upper bound (UCB) formula until a node that has not yet been fully expanded is reached; Extension: If not all candidate solutions for the current node have been generated, then call the mixed integer programming model to generate K candidate solutions for the node and create corresponding child nodes, and randomly select one of the new child nodes; Simulation: Starting from the new child node, a fast simulation strategy is used to sequentially determine the allocation scheme for each subsequent time slice, up to time slice t. N Calculate the overall cumulative value of the sequence under the optimization objective O; Backtracking: Propagate the accumulated value backward along the search path to update the statistical information of each node, including the average accumulated value and the number of visits; Repeat the above process until the preset number of iterations or convergence condition is met, and select the candidate solution corresponding to the child node with the largest average cumulative value under the root node as t. i The optimal allocation scheme.

[0006] Preferably, the method for generating K candidate solutions in step S4 is as follows: Utilizing the solution pool function of a mixed-integer programming solver to obtain the optimal solution and several differentiated suboptimal solutions at once; or generating solutions with different preferences by adjusting the weights of different terms in the objective function, thus ensuring the diversity of candidate solutions' behaviors. Furthermore, the requirement C... c It changes dynamically over time, in each time slice t i When reconstructing a mixed-integer programming model, the latest demand information obtained or predicted using this time slice is used to adapt to changes in the environment.

[0007] The inference process is a dynamic optimization process driven by the trained model M, that is, in each time slice t i The mixed integer programming approach selects the current solution from the K solutions provided and ultimately achieves global optimization. The specific steps are as follows: Step I1: Obtain allocation parameters, including the total time interval T, spatial range S, optimization objective O, the set of objects consuming spatiotemporal resources C, and the demand information C of each object as it changes over time. C ; Step I2: Divide the total time interval T evenly into N time slices, denoted as t1, t2, …, t N ; Step I3: Initialize the current time slice index i=1, and the system state reflecting information such as the allocated resources and object status; Step I4: In the current time slice t i Based on the system state, a mixed-integer programming model is constructed. The spatial range of this model is S, and the time range is from t... i The time interval from the start point to the end of the entire time interval T (i.e., covering all remaining time slices). The objective function is to optimize objective O, and the resource capacity R is... t Object requirement C Ci (C corresponding to the i-th time slice) C Using constraints such as ) and ) to solve for K feasible allocation schemes (K≥2); Step I5: Provide model M to determine time slice t i The optimal allocation scheme p i ; Step I6: Execute the optimal allocation scheme p i In time slice t i Complete resource allocation and update the system state within the specified time slice. Let i = i + 1. If i ≤ N, return to step S4 to continue the decision for the next time slice; otherwise, the allocation ends.

[0008] Compared with the prior art, the beneficial effects of the present invention are: 1. The MIP model for each time slice covers the complete remaining time domain from the current time to the end of the task, and the generated candidate solutions are naturally forward-looking, overcoming the shortcomings of traditional rolling optimization that only focuses on short time windows; 2. MCTS uses extensive random simulations to extrapolate the long-term benefits of different candidate sequences, which can balance exploration and utilization, select the current solution from a global perspective, and significantly improve the overall optimization objective. 3. By using the MIP solver to generate a limited number of high-quality candidate actions, the huge continuous or discrete decision space is compressed into a small set of discrete actions, which greatly reduces the width of the search tree and enables MCTS to converge with fewer simulations. 4. The entire method is executed in a rolling manner, introducing the latest requirement information in each time slice, which can adapt to the time-varying and uncertain nature of requirements and has good robustness.

[0009] 5. Training data is obtained based on the MCTS process, and model M is built to drive dynamic search, achieving a balance between efficiency and optimization. Attached Figure Description

[0010] Figure 1 Spatiotemporal resource allocation Figure 2 Flowchart of Spatiotemporal Resource Allocation Method Figure 3 Schematic diagram of the structure of a Markov tree search tree

[0011] [1] A Survey of Monte Carlo Tree Search Methods. By Browne C., PowleyE., Whitehouse D., Lucas SM, Cowling PI, Rohlfshagen P., Tavener S.,Perez D., Samothrakis S., Colton S. 2012.

[0012] [2] Integer and Combinatorial Optimization. By Nemhauser GL, WolseyL.A. 1988.

Claims

1. A method for dynamic allocation of spatiotemporal resources based on Markov tree search, characterized in that, It is divided into two parts: search model modeling and inference (applying the model to search). The search model modeling process requires Markov tree search to achieve global optimization and obtain the policy selection of each step extracted in the optimization process. Based on the state and optimal policy of each step, a training dataset is constructed to train model M.

2. A method for dynamic allocation of spatiotemporal resources based on Markov tree search, characterized in that, The modeling process includes: parameter acquisition: used to obtain the total time interval T, spatial range S, optimization objective O, object set C, and requirement information; Time partitioning: Divide T into N time slices; Allocation decision: In each time slice, based on the current state, call the mixed integer programming algorithm to generate K candidate solutions, run the MCTS algorithm to select the optimal solution, and output the allocation instruction; State update: Execute the allocation instruction and update the system state.

3. A method for dynamic allocation of spatiotemporal resources based on Markov tree search, characterized in that, The solution yields K candidate allocation schemes, including obtaining the optimal solution and multiple suboptimal solutions through the solution pooling function of the mixed integer programming solver.

4. A method for dynamic allocation of spatiotemporal resources based on Markov tree search, characterized in that, The MCTS algorithm includes: Selection steps: Starting from the root node, select child nodes according to the confidence upper bound formula until a node that has not been fully expanded is reached; Extension Steps: If the candidate solutions for the current node have not yet been fully generated, call the mixed-integer programming model to generate K candidate solutions for that node and create child nodes, randomly selecting one new child node; Simulation Steps: Starting from the extended child nodes, use a fast simulation strategy to sequentially determine the allocation schemes for each subsequent time slice, until t N The cumulative optimization objective value is calculated; backtracking step: the cumulative optimization objective value is propagated backward along the path, updating the statistical values ​​of the nodes traversed; the above steps are iteratively executed until a predetermined number of iterations are reached, then the candidate child node scheme corresponding to the optimal statistical value under the root node is selected as t. i The optimal allocation scheme.

5. A method for dynamic allocation of spatiotemporal resources based on Markov tree search, characterized in that, The time interval is divided into slices, and each time slice t i When constructing a mixed-integer programming model, the latest demand Cc value obtained for that time slice is used.

6. A method for dynamic allocation of spatiotemporal resources based on Markov tree search, characterized in that, The system state includes the resource allocation records completed within t1 to t_{i-1}, and the current resource occupancy status of object C.

7. A method for dynamic allocation of spatiotemporal resources based on Markov tree search, characterized in that... The optimal solution selection for mixed integer solutions under different time slices and spatiotemporal resource states is searched and collected through MCTS, and this is used as training data to train the machine learning model M.

8. A method for dynamic allocation of spatiotemporal resources based on Markov tree search, characterized in that... The reasoning process includes the modeling process. Includes: Parameter acquisition: total time interval T, spatial range S, optimization objective O, object set C, and requirement information; Time division: dividing T into N time slices; Allocation decision: used to generate K candidate solutions based on the current state in each time slice, run model M to select the optimal solution, and output allocation instructions; State update: executing allocation instructions and updating the system state.

Images