A smart city AI computing resource dynamic scheduling optimization decision method

Abstract

The present disclosure provides a smart city AI computing resource dynamic scheduling optimization decision method. It includes four steps S1-S4: S1, obtain the observable system state of the smart city AI computing platform in each scheduling period, and construct an observation space dataset; S2, model the smart city AI computing resource scheduling problem as a standard seven-tuple partially observable Markov decision process, perform online Bayesian estimation, and output the belief state and belief state feature vector at the current time; S3, input the belief state feature vector into the pre-trained robust belief state Q-learning strategy network, and output the optimal computing resource allocation joint action in the current scheduling period; S4, execute the resource allocation joint action and interact with the smart city AI computing platform environment. In an incomplete information environment, the smart city AI computing resource utilization rate and scheduling robustness are improved, and the core business SLA requirements are effectively guaranteed.

Description

Technical Field

[0001] This disclosure relates to the field of computer science, and more specifically, to a dynamic scheduling and optimization decision-making method for AI computing resources in smart cities. Background Technology

[0002] With the deepening of smart city construction, core scenarios such as urban traffic management, public safety video analysis, real-time ecological environment monitoring, and intelligent government approval have been fully empowered by AI. Massive AI inference and training tasks are deployed and run in a unified manner through the smart city cloud-edge collaborative computing platform. The aforementioned AI services are mostly deployed in the form of lightweight model interfaces and distributed inference instances in a three-tier computing architecture consisting of urban cloud computing centers, regional edge computing nodes, and front-end sensing terminals. Global scheduling optimization is required through shared GPU / CPU computing power pools, video memory / memory resources, inference nodes, and network links.

[0003] However, in practical engineering implementation, smart city AI computing platforms face core technical challenges that existing technologies cannot effectively address: The assumption of complete information is severely out of touch with real-world scenarios and cannot adapt to scheduling environments with incomplete information. Smart city AI computing resources are deployed heterogeneously across cloud, edge, and endpoint. Key information such as the resource occupancy status of edge nodes in different regions, the task queue length of non-cooperative tenants, the jitter latency of cross-node network links, and the queuing time of shared computing pools in the cloud are all hidden states that cannot be obtained in real time or without bias. Most existing mainstream scheduling schemes are based on fully observable Markov decision processes (MDPs), which forcibly assume that the entire system state can be observed in real time. This assumption is completely invalid in the real-world computing environment of smart cities, directly leading to systemic biases in scheduling decisions and causing problems such as resource allocation imbalances and task congestion.

[0004] The latent state estimation methods suffer from insufficient accuracy, poor robustness, and inability to adapt to multi-source heterogeneous observation noise. Existing particle filter-based latent state estimation methods face two major drawbacks in smart city computing power scheduling scenarios: First, the fixed particle number design cannot achieve a dynamic balance between estimation accuracy and computational overhead. The limited computing resources of edge nodes cannot support high-density particle computation, while insufficient particle number leads to a sharp drop in estimation accuracy. Second, the use of single observation likelihood to update particle weights cannot adapt to the multi-source heterogeneous observation signals (node ​​load, task requests, network latency, etc.) in smart cities. Observation noise easily causes weight degradation, ultimately resulting in the latent state estimation results deviating significantly from the true value.

[0005] The scheduling strategy lacks risk constraints and scenario adaptability, failing to guarantee the SLA of core smart city operations. AI tasks in core smart city scenarios such as transportation and security have strong real-time requirements, and sudden traffic surges (morning rush hour traffic, video analysis of sudden events) can easily cause drastic fluctuations in system load. Existing reinforcement learning-based scheduling schemes only aim to maximize resource utilization, without considering decision-making risks in extreme scenarios. When faced with sudden changes in hidden states and observation noise disturbances, they are prone to policy oscillations and scheduling failures, leading to AI task inference timeouts and a sharp drop in SLA compliance rates, failing to meet the high reliability requirements of smart city operations.

[0006] Lacking closed-loop online iteration capabilities, it cannot adapt to the highly time-varying characteristics of computing power demands in smart cities. The computing needs of AI tasks in different scenarios of smart cities exhibit significant spatiotemporal dynamic characteristics. For example, the difference in traffic AI task load between weekdays and holidays can be more than 5 times. Existing static scheduling strategies or offline-trained reinforcement learning models cannot adapt to the dynamic changes in task requirements and system states online, resulting in weak generalization ability and poor engineering feasibility.

[0007] Therefore, how to achieve robust, efficient, and highly reliable dynamic scheduling and optimization of AI computing resources for smart cities in an environment with incomplete information is a core technical problem that urgently needs to be solved in the field. Summary of the Invention

[0008] The purpose of this disclosure is to provide a dynamic scheduling optimization decision-making method for AI computing resources in smart cities, aiming to solve problems such as insufficient scheduling accuracy, poor robustness, and weak business SLA guarantee capability in scenarios with incomplete information in the prior art.

[0009] In general, this paper provides a dynamic scheduling and optimization decision-making method for AI computing resources in smart cities, comprising four steps: S1-S4. S1. Obtain the observable system status of the smart city AI computing platform in each scheduling cycle and construct an observation space dataset; S2. Through the hidden state online estimation module, the AI ​​computing resource scheduling problem of smart cities is modeled as a standard seven-tuple partially observable Markov decision process. The adaptive multi-observation fusion particle filter algorithm is used to perform online Bayesian estimation of the unobservable hidden state of the system, and output the belief state and belief state feature vector at the current moment. S3. Through the robust joint decision-making module, the belief state feature vector is input into the pre-trained robust belief state Q-learning policy network, and combined with the SLA constraint action pruning mechanism, the optimal computing resource allocation joint action for the current scheduling cycle is output. S4. Execute the resource allocation joint action, interact with the smart city AI computing platform environment, obtain system feedback data and real-time rewards for the current scheduling cycle, complete the belief state update and policy network parameter iterative optimization based on the new observation data, and enter the next scheduling cycle to form a closed-loop decision-making process.

[0010] The specific expression for the standard seven-tuple is: Among them, the state space From the observable state subset and hidden state subset constitute, The observable state subset The local node and system public states that the scheduling entity can obtain without bias include: local node GPU / CPU real-time utilization rate, video memory / memory usage rate, length of the AI ​​task queue to be processed, SLA level requirements of the connected tasks, uplink and downlink bandwidth usage rate of the local node network, and public quota information of the cloud computing power pool; the hidden state subset The hidden states in the system that cannot be directly observed in real time include: the remaining available computing power of adjacent edge nodes, the task request arrival rate of non-cooperative tenants, the task queuing time of the cloud shared computing power pool, the jitter latency of cross-node network links, and the performance loss coefficient of heterogeneous computing power nodes. Action space The joint action space for allocating AI computing resources in smart cities consists of a subset of computing power scheduling actions. and task scheduling action subset Composition, that is The subset of computing power scheduling actions This includes: GPU memory allocation, CPU core binding count, local computing power to cloud offload ratio, and edge node start / stop control; the subset of task scheduling actions. This includes: AI task priority adjustment, inference instance replica scaling, task execution node mapping and allocation; observation space. The set of multi-source heterogeneous observation signals that can be obtained for each scheduling cycle includes: local node load monitoring data, actual execution time of AI tasks, number of task requests arriving, cross-node network latency detection values, and cloud resource pool quota update information. State transition probability function Used to describe the current state Execute scheduling actions Then, the system transitions to the state of the next time step. The probability distribution includes deterministic transition rules for observable states and stochastic transition models for hidden states; Observation probability function Used to describe system state transitions to Execution of actions Afterwards, the observation signal was obtained. The probability mapping relationship, i.e., the observation likelihood function; reward function The single-step reward is the comprehensive benefit of the scheduling strategy within a single scheduling cycle, used to measure the performance of resource scheduling actions. Discount factor The range of values ​​is This is used to weigh the immediate rewards against the long-term cumulative rewards.

[0011] The specific steps for performing online Bayesian estimation and outputting the current belief state and its feature vector are as follows: S21. Based on prior knowledge of smart city computing power scheduling, set initial prior distributions that conform to the characteristics of the scenario for each type of hidden state, and sample from the initial prior distributions. Initialize one particle, and initialize all particle weights to equal values. ,in The particles are numbered, and the initial particles are discrete random samples of the unobservable hidden states of the system. Each particle corresponds to a set of possible values ​​of the hidden states, which are used to approximate the posterior probability distribution of the hidden states. The specific method for setting the initial prior distribution is as follows: a Beta distribution is used to model the remaining available computing power of adjacent edge nodes and the performance loss coefficient of heterogeneous computing power nodes; a Gamma distribution is used to model the task request arrival rate of non-cooperative tenants, the task queuing time of the cloud shared computing power pool, and the jitter latency of cross-node network links; the hyperparameters of each distribution are determined based on historical statistical data of the smart city computing power scheduling industry. S22. Perform the prediction step, for each particle, according to the state transition probability function. Sample the particle state at the next time step to complete the prior prediction of the particle state; S23. Perform the multi-observation fusion weight update step. After obtaining the multi-source observation signals at the current time, construct the multi-observation joint likelihood function to complete the update and normalization of particle weights. The specific weight update formula is as follows: in, For system load observation signals, The mission requests the observation of signals. For network delay observation signals; The adaptive fusion weights for each observed signal, and satisfying the following conditions: ; S24. Perform adaptive particle number adjustment and regularized resampling, and calculate the current effective particle number. ,according to The total number of particles is dynamically adjusted. When the number of effective particles is lower than the preset threshold, regularized resampling is performed, and the particle weights are reset to equal values. The formula for calculating the effective number of particles is: The adaptive particle number adjustment formula is as follows: in, The total number of particles at the current moment. These are the upper and lower limits of the particle number, respectively. For the effective particle number, a double threshold is used. S25. Perform belief state update and feature extraction. Based on the updated particle set and weights, complete the Bayesian update of belief state and extract core statistical features from the belief state particle set to form a fixed-dimensional belief state feature vector.

[0012] The specific method for Bayesian updating of the belief state is to introduce an adaptive confidence adjustment mechanism for observation noise, and the specific update formula is as follows: in, for The state of belief at any given moment, i.e., the state of the system. The posterior probability distribution; The belief state after confidence adjustment; This is the adaptive confidence adjustment coefficient, whose value is dynamically adjusted according to the observed signal-to-noise ratio. Its expression is: in, To observe the noise variance, Let V be the variance of the state signal.

[0013] The construction and reasoning process of the robust belief state Q-learning policy network specifically includes: S31. Construct a risk-sensitive reward function constrained by SLA, with resource utilization, task SLA achievement rate, and system energy consumption as optimization objectives, and introduce conditional value of risk (CVaR) to implement risk constraints: in, This represents the current cycle's system computing resource utilization rate. To improve the SLA compliance rate for AI tasks, The system's energy consumption per unit of computing power. Loss due to task timeout; Let be the weight coefficient, and satisfy... ; As a risk penalty coefficient, Confidence level Conditional value of risk for task timeout; among which Losses due to task timeout Quantities represent the lower bound of risk for returns; the aforementioned The calculation is performed based on the particle set corresponding to the belief state. After sorting the task timeout loss of each particle in ascending order, the top-ranked particles are selected. The mean of the K minimum values ​​is obtained, where K is the current total number of particles; S32. Introduce CVaR to replace the expectation operator in the standard Bellman equation, construct a robust Bellman optimal equation adapted to the computing power scheduling scenario of smart cities, and define a robust action value function: in, This is the robust optimal action value function. For the current state of belief, For the current scheduling action, For the next moment of scheduling action, For the immediate reward expectation in the belief state, For the belief state update function, For the next moment of observation; S33. For a continuous infinite-dimensional belief state space, a linear function is used to approximate the robust action value function, with the belief state feature vector as the function input: in, For action The corresponding weight vector, For the feature vector of belief state, This is the set of parameters to be optimized for the policy network; S34. Design a robust temporal difference update rule, and iteratively optimize the weight vector of the strategy network based on system interaction data until the network converges. S35. Based on the converged policy network and combined with the SLA-constrained action pruning mechanism, output the optimal resource allocation joint action for the current scheduling period; the expression of the SLA-constrained action pruning mechanism is: in, For a set of legal actions, For action The corresponding task's estimated execution time, This represents the upper limit of SLA latency for AI tasks.

[0014] The weight update formula for the robust temporal difference update rule in step S34 is as follows: in, For adaptive learning rate, For robust timing differential error, the expression is: The adaptive learning rate The update formula is: in, The initial learning rate, The maximum number of iteration cycles. This is the attenuation coefficient.

[0015] The technical effects to be achieved by the embodiments of the present invention are as follows: The method employs a two-layer closed-loop architecture consisting of an online hidden state estimation module and a robust joint decision-making module. First, the AI ​​computing resource scheduling problem in smart cities is modeled as a partially observable Markov decision process. An adaptive multi-observation fusion particle filter algorithm is used to perform online Bayesian estimation of the unobservable hidden states of the system, outputting a belief state feature vector. Then, based on a robust belief state Q-learning policy network and combined with an SLA constraint action pruning mechanism, the optimal resource allocation joint action is output, achieving closed-loop online optimization. This application can significantly improve the utilization rate and scheduling robustness of AI computing resources in smart cities under incomplete information environments, effectively ensuring the SLA requirements of core businesses, and is applicable to resource management and task scheduling in smart city cloud-edge collaborative AI computing platforms. Attached Figure Description

[0016] The above and other objects and features of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings.

[0017] Figure 1 This is a schematic diagram illustrating the architecture of a smart city AI computing resource dynamic scheduling optimization decision-making method according to an embodiment of the present disclosure; Figure 2 This is a schematic diagram illustrating the complete algorithmic decision-making closed-loop flowchart according to an embodiment of the present disclosure. Detailed Implementation

[0018] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0019] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become clear upon understanding the disclosure of this application.

[0020] As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more.

[0021] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0022] In the specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, the element may be directly "on" another element, directly "connected to," or "bonded to" the other element, or one or more other elements may be present in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, no other elements may be present in between.

[0023] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0024] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.

[0025] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.

[0026] Figure 1 This is a schematic diagram illustrating a smart city AI computing resource dynamic scheduling optimization decision-making method according to an embodiment of the present disclosure.

[0027] To achieve the aforementioned objectives, the present invention employs the following technical framework: Figure 1 As shown.

[0028] To achieve the above objectives, this application adopts a two-layer closed-loop decision architecture including a hidden state online estimation module and a robust joint decision module, and provides a dynamic scheduling and optimization decision-making method for AI computing resources in smart cities, specifically including the following steps: S1. Obtain the observable system status of the smart city AI computing platform in each scheduling cycle and construct an observation space dataset; S2. Through the hidden state online estimation module, the AI ​​computing resource scheduling problem of smart cities is modeled as a standard seven-tuple partially observable Markov decision process (POMDP). The adaptive multi-observation fusion particle filter algorithm is used to perform online Bayesian estimation of the unobservable hidden state of the system, and output the belief state and belief state feature vector at the current moment. S3. Through the robust joint decision-making module, the belief state feature vector is input into the pre-trained robust belief state Q-learning policy network, and combined with the SLA constraint action pruning mechanism, the optimal computing resource allocation joint action for the current scheduling cycle is output. S4. Execute the resource allocation joint action, interact with the smart city AI computing platform environment, obtain system feedback data and real-time rewards for the current scheduling cycle, complete the belief state update and policy network parameter iterative optimization based on the new observation data, and enter the next scheduling cycle to form a closed-loop decision-making process.

[0029] Furthermore, the standard seven-tuple POMDP model described in step S2 is specifically expressed as follows: The contextual definition of each tuple is as follows: state space , consisting of a subset of observable states and hidden state subset Composition, that is ; The subset of observable states The local node and system public status that the scheduling entity can obtain without bias includes: local node GPU / CPU real-time utilization rate, video memory / memory usage rate, length of the AI ​​task queue to be processed, SLA level requirements of the connected tasks, uplink and downlink bandwidth usage rate of the local node network, and public quota information of the cloud computing power pool. The hidden state subset The hidden states in the system that cannot be directly observed in real time include: the remaining available computing power of adjacent edge nodes, the task request arrival rate of non-cooperative tenants, the task queuing time of the cloud shared computing power pool, the jitter latency of cross-node network links, and the performance loss coefficient of heterogeneous computing power nodes. Action space The joint action space for allocating AI computing resources in smart cities consists of a subset of computing power scheduling actions. and task scheduling action subset Composition, that is ; The subset of computing power scheduling actions This includes: GPU memory allocation, CPU core binding count, local computing power to cloud offload ratio, and edge node start / stop control; the subset of task scheduling actions. This includes: adjusting AI task priorities, scaling up or down inference instance replicas, and mapping and allocating task execution nodes; Observation space The set of multi-source heterogeneous observation signals that can be obtained for each scheduling cycle includes: local node load monitoring data, actual execution time of AI tasks, number of task requests arriving, cross-node network latency detection values, and cloud resource pool quota update information. State transition probability function Used to describe the current state Execute scheduling actions Then, the system transitions to the state of the next time step. The probability distribution includes deterministic transition rules for observable states and stochastic transition models for hidden states; Observation probability function Used to describe system state transitions to Execution of actions Afterwards, the observation signal was obtained. The probability mapping relationship, i.e., the observation likelihood function; reward function The single-step reward is the comprehensive benefit of the scheduling strategy within a single scheduling cycle, used to measure the performance of resource scheduling actions. Discount factor The range of values ​​is This is used to weigh the immediate rewards against the long-term cumulative rewards.

[0030] Furthermore, the online Bayesian estimation of the unobservable hidden states of the system using the adaptive multi-observation fusion particle filter algorithm described in step S2 specifically includes the following steps: S21. Based on prior knowledge of smart city computing power scheduling, set initial prior distributions that conform to the characteristics of the scenario for each type of hidden state, and sample from the initial prior distributions. Initialize one particle, and initialize all particle weights to equal values. ,in Number the particles; S22. Perform the prediction step, for each particle, according to the state transition probability function. Sample the particle state at the next time step to complete the prior prediction of the particle state; S23. Perform the multi-observation fusion weight update step. After obtaining the multi-source observation signals at the current time, construct the multi-observation joint likelihood function to complete the update and normalization of particle weights. The specific weight update formula is as follows: in, For system load observation signals, The mission requests the observation of signals. For network delay observation signals; The adaptive fusion weights for each observed signal, and satisfying the following conditions: ; S24. Perform adaptive particle number adjustment and regularized resampling, and calculate the current effective particle number. ,according to The total number of particles is dynamically adjusted. When the number of effective particles is lower than the preset threshold, regularized resampling is performed, and the particle weights are reset to equal values. The formula for calculating the effective number of particles is: The adaptive particle number adjustment formula is as follows: in, The total number of particles at the current moment. These are the upper and lower limits of the particle number, respectively. For the effective particle number, a double threshold is used. S25. Perform belief state update and feature extraction. Based on the updated particle set and weights, complete the Bayesian update of belief state and extract core statistical features from the belief state particle set to form a fixed-dimensional belief state feature vector.

[0031] Furthermore, the Bayesian update of the belief state described in step S25 introduces an adaptive confidence adjustment mechanism for observation noise, with the specific update formula as follows: in, for The state of belief at any given moment, i.e., the state of the system. The posterior probability distribution; The belief state after confidence adjustment; This is the adaptive confidence adjustment coefficient, whose value is dynamically adjusted according to the observed signal-to-noise ratio. Its expression is: in, To observe the noise variance, Let V be the variance of the state signal.

[0032] Furthermore, the robust belief state Q-learning policy network described in step S3 specifically includes the following construction and reasoning process: S31. Construct a risk-sensitive reward function with SLA constraints, using resource utilization, task SLA achievement rate, and system energy consumption as optimization objectives. Conditional Value at Risk (CVaR) is introduced to implement risk constraints. The specific expression is: in, This represents the current cycle's system computing resource utilization rate. To improve the SLA compliance rate for AI tasks, The system's energy consumption per unit of computing power. Loss due to task timeout; Let be the weight coefficient, and satisfy... ; As a risk penalty coefficient, Confidence level Conditional value of risk for task timeout; S32. Introduce CVaR to replace the expectation operator in the standard Bellman equation, construct a robust Bellman optimal equation adapted to the computing power scheduling scenario of smart cities, and define a robust action value function, the specific expression of which is: in, This is the robust optimal action value function. For the current state of belief, For the current scheduling action, For the next moment of scheduling action, For the immediate reward expectation in the belief state, For the belief state update function, For the next moment of observation; S33. For a continuous infinite-dimensional belief state space, a linear function is used to approximate the robust action value function, with the belief state feature vector as the function input. The expression is: in, For action The corresponding weight vector, For the feature vector of belief state, This is the set of parameters to be optimized for the policy network; S34. Design a robust temporal difference update rule, and iteratively optimize the weight vector of the strategy network based on system interaction data until the network converges. S35. Based on the converged policy network and combined with the SLA-constrained action pruning mechanism, output the optimal resource allocation joint action for the current scheduling period; the expression of the SLA-constrained action pruning mechanism is: in, For a set of legal actions, For action The corresponding task's estimated execution time, This represents the upper limit of SLA latency for AI tasks.

[0033] Furthermore, the robust temporal differential update rule described in step S34 has the following weight update formula: in, For adaptive learning rate, For robust timing differential error, the expression is: The adaptive learning rate The update formula is: in, The initial learning rate, The maximum number of iteration cycles. This is the attenuation coefficient.

[0034] Furthermore, the complete closed-loop execution flow of the method is as follows: Step 1: Initialize the POMDP model parameters, particle filter parameters, and reinforcement learning policy network parameters to complete the system cold start; Step 2: Obtain the observable system status and multi-source observation signals for the current scheduling period; Step 3: Execute the adaptive multi-observation fusion particle filter algorithm to complete the online Bayesian estimation of the hidden state and output the belief state and belief state feature vector after confidence adjustment; Step 4: Input the belief state feature vector into the robust belief state Q-learning policy network, filter legal actions through the SLA-constrained action pruning mechanism, and output the optimal resource allocation joint action that maximizes the robust action value function; Step 5: Execute the selected resource allocation joint action, interact with the smart city AI computing platform environment, obtain the system feedback data for the current period, and calculate the robust reward value; Step 6: Based on the new observation signal, complete the belief state update, calculate the robust temporal difference error, and complete the iterative optimization of the policy network weight vector; Step 7: Update the relevant parameters of the particle filter and policy network, enter the next scheduling cycle, and repeat steps 2-7 to achieve continuous online optimization of the policy.

[0035] Specific application scenario 1: AI scenario for smart city traffic management, cloud-edge collaborative scheduling This embodiment applies to a cloud-edge collaborative scheduling scenario for AI services in smart city traffic management. It covers one cloud computing center and five intersection edge computing nodes in the core urban area, deploying three core AI inference tasks: traffic flow prediction, intelligent traffic light optimization, and traffic violation identification. The single scheduling cycle is 100ms, with a total of 1000 scheduling cycles. The specific implementation steps are as follows: Step 1: Environment and Parameter Initialization System basic parameter settings: Total cloud computing power is 8 A100 GPUs, single edge node computing power is 1 Jetson AGX Orin GPU, maximum latency requirement (SLA limit) for single-frame inference of AI tasks is 50ms, discount factor. Reward function weights , , Risk penalty coefficient CVaR confidence level ; Particle filter parameter initialization: initial number of particles upper and lower limits of particle number , Effective particle number threshold , Adaptive fusion weight initial values , , ; Reinforcement learning parameter initialization: Initial learning rate Minimum learning rate Initial exploration rate Minimum exploration rate Cold start exploration phase duration Initialize all weight vectors of the policy network to 0.

[0036] Step 2: Cold Start Exploration Phase (First 100 Scheduling Cycles) Within each scheduling cycle, a resource allocation joint action is randomly selected from the set of legal actions constrained by the SLA. It interacts with the traffic management AI computing platform environment to obtain observation data, and simultaneously performs sequential updates of adaptive particle filtering to accumulate system interaction data and complete the initial convergence of the hidden state estimation model.

[0037] Step 3: Strategy Learning and Online Optimization Phase (100-1000 scheduling cycles) Within each scheduling cycle, the complete closed-loop decision-making process of this application must be strictly implemented: Step 3.1: Obtain observable signals such as edge node load, number of task requests, and network latency in the current period, perform adaptive multi-observation fusion particle filter update, complete hidden state Bayesian estimation, and output the belief state and feature vector after confidence adjustment; Step 3.2: Based on attenuation - Greedy strategy: select the joint action that maximizes the value function of the robust action, or randomly select legal actions to explore; Step 3.3: Execute the selected joint actions, including GPU resource allocation, adjustment of cloud-edge computing power offloading ratio, and task priority scheduling. Interact with the system environment to obtain feedback data such as resource utilization, task SLA achievement rate, and task execution latency for the current period, and calculate the robust reward value. Step 3.4: Based on the new observation signal, complete the belief state update, calculate the robust temporal difference error, and complete the gradient descent update of the policy network weight vector; Step 3.5: Update relevant parameters such as exploration rate, learning rate, and particle number, add the current action and observation to the historical sequence, and enter the next scheduling cycle.

[0038] In this embodiment, the average computing power resource utilization rate of the system in this application reaches 89.7%, which is 42.6% higher than the 62.9% of the existing complete information MDP method; the AI ​​task SLA compliance rate reaches 98.2%, which is 37.5 percentage points higher than the existing method; and the average inference latency of the task is 22.6ms, which is 58.3% lower than the existing method, achieving a significant performance improvement in an incomplete information environment.

[0039] Specific application scenario 2: Multi-node scheduling of video analytics AI scenarios for smart city public safety This embodiment applies to a multi-node scheduling scenario for AI services in smart city public safety video analytics. It covers one city-wide video cloud computing center and 12 regional edge computing nodes, accessing 2000 channels of high-definition video surveillance. It deploys three types of AI analysis tasks: personnel gathering identification, abnormal behavior detection, and key target tracking. The environment is characterized by heavily incomplete information, with a single scheduling cycle of 200ms and a total of 800 scheduling cycles. The specific implementation steps are as follows: Step 1: Parameter Adaptation and Adjustment The hidden state dimension is expanded to The dimension corresponds to the 5 types of core hidden states of the 12 edge nodes (remaining available computing power, task queuing time, network jitter latency, task request arrival rate, and performance loss coefficient). The initial particle count for particle filtering is adjusted to 350, and the upper and lower limits of the particle count are adjusted to... , The observation noise variance was adjusted to adapt to the video analysis scenario, while the remaining system basic parameters and reinforcement learning parameters remained the same as in Example 1. The action space is expanded to include a joint action space for video stream distribution scheduling, GPU memory sharding allocation, and inference instance scaling of 12 edge nodes, with an SLA latency limit set to 100ms.

[0040] Step 2: Algorithm Execution Flow The execution flow is completely consistent with that of Example 1. Only in the state transition and likelihood calculation stages of particle filtering, the hidden states of 12 edge nodes are processed simultaneously. In the belief state feature extraction stage, the statistical features of the hidden states of all edge nodes are merged. In the policy reasoning stage, global multi-node joint scheduling optimization is completed.

[0041] In this embodiment, the cumulative scheduling benefit of this application reaches 9674, which is 3.47 times that of the existing complete information MDP method; the system computing resource utilization rate reaches 87.2%, which is 45.8% higher than the existing method; in a heavily incomplete information environment, the average error of hidden state estimation is only 4.15%, which is far lower than the 21.36% of the existing method; the average processing latency of video analysis AI tasks is reduced by 61.2%, and excellent scheduling performance and stability are maintained in the highly competitive and dynamic smart city real-world scenario.

[0042] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

Claims

1. A method for dynamic scheduling optimization decision of AI computing resources in smart city, characterized in that, It includes four steps: S1-S4. S1. Obtain the observable system status of the smart city AI computing platform in each scheduling cycle and construct an observation space dataset; S2. Through the hidden state online estimation module, the AI ​​computing resource scheduling problem of smart cities is modeled as a standard seven-tuple partially observable Markov decision process. The adaptive multi-observation fusion particle filter algorithm is used to perform online Bayesian estimation of the unobservable hidden state of the system, and output the belief state and belief state feature vector at the current moment. S3. Through the robust joint decision-making module, the belief state feature vector is input into the pre-trained robust belief state Q-learning policy network, and combined with the SLA constraint action pruning mechanism, the optimal computing resource allocation joint action for the current scheduling cycle is output. S4. Execute the resource allocation joint action, interact with the smart city AI computing platform environment, obtain system feedback data and real-time rewards for the current scheduling cycle, complete the belief state update and policy network parameter iterative optimization based on the new observation data, and enter the next scheduling cycle to form a closed-loop decision-making process. 2.The smart city AI computing resource dynamic scheduling optimization decision method of claim 1, wherein, The specific expression for the standard seven-tuple is: Among them, the state space From the observable state subset and hidden state subset constitute, The observable state subset The local node and system public states that the scheduling entity can obtain without bias include: local node GPU / CPU real-time utilization rate, video memory / memory usage rate, length of the AI ​​task queue to be processed, SLA level requirements of the connected tasks, uplink and downlink bandwidth usage rate of the local node network, and public quota information of the cloud computing power pool; the hidden state subset The hidden states in the system that cannot be directly observed in real time include: the remaining available computing power of adjacent edge nodes, the task request arrival rate of non-cooperative tenants, the task queuing time of the cloud shared computing power pool, the jitter latency of cross-node network links, and the performance loss coefficient of heterogeneous computing power nodes. Action space The joint action space for allocating AI computing resources in smart cities consists of a subset of computing power scheduling actions. and task scheduling action subset Composition, that is The subset of computing power scheduling actions This includes: GPU memory allocation, CPU core binding count, local computing power to cloud offload ratio, and edge node start / stop control; the subset of task scheduling actions. This includes: AI task priority adjustment, inference instance replica scaling, task execution node mapping and allocation; observation space. The set of multi-source heterogeneous observation signals that can be obtained for each scheduling cycle includes: local node load monitoring data, actual execution time of AI tasks, number of task requests arriving, cross-node network latency detection values, and cloud resource pool quota update information. State transition probability function Used to describe the current state Execute scheduling actions Then, the system transitions to the state of the next time step. The probability distribution includes deterministic transition rules for observable states and stochastic transition models for hidden states; Observation probability function Used to describe system state transitions to Execution of actions Afterwards, the observation signal was obtained. The probability mapping relationship, i.e., the observation likelihood function; reward function The single-step reward is the comprehensive benefit of the scheduling strategy within a single scheduling cycle, used to measure the performance of resource scheduling actions. Discount factor The range of values ​​is This is used to weigh the immediate rewards against the long-term cumulative rewards.

3. The method for dynamic scheduling and optimization decision-making of AI computing resources in smart cities as described in claim 2, characterized in that, The specific steps for performing online Bayesian estimation and outputting the current belief state and its feature vector are as follows: S21. Based on prior knowledge of smart city computing power scheduling, set initial prior distributions that conform to the characteristics of the scenario for each type of hidden state, and sample from the initial prior distributions. Initialize one particle, and initialize all particle weights to equal values. ,in The particles are numbered; the initial particles are the result of a discrete random sampling of the unobservable hidden state vector of the system. Each particle corresponds to a set of possible true values ​​in the hidden state space, which are used to form a discrete approximate representation of the posterior probability distribution of the hidden state. The hidden state vector includes at least the remaining available computing power of adjacent edge nodes, the task request arrival rate of non-cooperative tenants, the task queuing time of the cloud shared computing power pool, the jitter latency of cross-node network links, and the performance loss coefficient of heterogeneous computing power nodes. The specific method for setting the initial prior distribution includes: modeling and estimating the statistical characteristics corresponding to various hidden states based on the historical scheduling and operation data of the smart city AI computing platform. Specifically, for the remaining available computing power of adjacent edge nodes and the performance loss coefficient of heterogeneous computing power nodes with values ​​limited to the interval [0,1], a Beta distribution is used for modeling, and its distribution parameters are estimated and determined by the mean and variance of historical samples. For non-negative continuous variables such as the arrival rate of non-cooperative tenant task requests, the queuing time of the cloud shared computing power pool, and the jitter latency of cross-node network links, a Gamma distribution is used for modeling, and its shape and scale parameters are determined by the moment estimation or maximum likelihood estimation method of historical observation data. After modeling the probability distribution of each hidden state variable, based on the assumption that each variable is independent in the initial stage, samples are taken from the corresponding probability distribution to construct multidimensional hidden state particle samples. The sampled particles are then subjected to scenario constraint correction to meet the physical feasibility constraints of the smart city computing power scheduling system, including upper limit constraints on computing power resources, matching constraints between task arrival and service capabilities, and outlier truncation or resampling. This results in an initial particle set that meets the requirements of the actual engineering scenario and is used for the belief state initialization of subsequent particle filtering. S22. Perform the prediction step, for each particle, according to the state transition probability function. Sample the particle state at the next time step to complete the prior prediction of the particle state; S23. Perform the multi-observation fusion weight update step. After obtaining the multi-source observation signals at the current time, construct the multi-observation joint likelihood function to complete the update and normalization of particle weights. The specific weight update formula is as follows: in, For system load observation signals, The mission requests the observation of signals. For network delay observation signals; The adaptive fusion weights for each observed signal, and satisfying the following conditions: ; S24. Perform adaptive particle number adjustment and regularized resampling, and calculate the current effective particle number. ,according to The total number of particles is dynamically adjusted. When the number of effective particles is lower than the preset threshold, regularized resampling is performed, and the particle weights are reset to equal values. The formula for calculating the effective number of particles is: The adaptive particle number adjustment formula is as follows: in, The total number of particles at the current moment. These are the upper and lower limits of the particle number, respectively. For the effective particle number, a double threshold is used. S25. Perform belief state update and feature extraction. Based on the updated particle set and weights, complete the Bayesian update of belief state and extract core statistical features from the belief state particle set to form a fixed-dimensional belief state feature vector.

4. The method for dynamic scheduling and optimization decision-making of AI computing resources in smart cities as described in claim 3, characterized in that, The specific method for Bayesian updating of the belief state is to introduce an adaptive confidence adjustment mechanism for observation noise, and the specific update formula is as follows: in, for The state of belief at any given moment, i.e., the state of the system. The posterior probability distribution; The belief state after confidence adjustment; This is the adaptive confidence adjustment coefficient, whose value is dynamically adjusted according to the observed signal-to-noise ratio. Its expression is: in, To observe the noise variance, Let V be the variance of the state signal.

5. The method for dynamic scheduling and optimization decision-making of AI computing resources in smart cities as described in claim 4, characterized in that, The construction and reasoning process of the robust belief state Q-learning policy network specifically includes: S31. Construct a risk-sensitive reward function constrained by SLA, with resource utilization, task SLA achievement rate, and system energy consumption as optimization objectives, and introduce conditional value of risk (CVaR) to implement risk constraints: in, This represents the current cycle's system computing resource utilization rate. To improve the SLA compliance rate for AI tasks The system's energy consumption per unit of computing power. Loss due to task timeout; Let be the weight coefficient, and satisfy... ; As a risk penalty coefficient, Confidence level The conditional value of risk for task timeout, of which Losses due to task timeout Quantities represent the lower bound of risk for returns; the aforementioned The conditional value of risk is calculated based on the particle set corresponding to the belief state, and the task timeout loss corresponding to each particle is sorted in ascending order. The specific calculation method is as follows: Based on the particle set corresponding to the belief state, the task timeout loss corresponding to each particle is calculated, and a weighted empirical loss distribution is constructed by combining the particle weights. All particle losses are sorted in ascending order, and the confidence level is determined based on the cumulative distribution of particle weights. The conditional value of risk (VQS) is obtained by weighting the tail loss exceeding the quantile threshold and then calculating a weighted average of the tail losses. When the particle weights are equal, the VQS is equivalent to the VQS calculated by weighting the tail losses at a ratio of (1-...). The average particle loss is taken to characterize the risk level under extremely unfavorable scenarios, and it is introduced into the reward function to constrain the risk of the scheduling strategy. S32. Introduce CVaR to replace the expectation operator in the standard Bellman equation, construct a robust Bellman optimal equation adapted to the computing power scheduling scenario of smart cities, and define a robust action value function: in, This is the robust optimal action value function. For the current state of belief, For the current scheduling action, For the next moment of scheduling action, For the immediate reward expectation in the belief state, For the belief state update function, For the next moment of observation; S33. For a continuous infinite-dimensional belief state space, a linear function is used to approximate the robust action value function, with the belief state feature vector as the function input: in, For action The corresponding weight vector, For the feature vector of belief state, This is the set of parameters to be optimized for the policy network; S34. Design a robust temporal difference update rule, and iteratively optimize the weight vector of the strategy network based on system interaction data until the network converges. S35. Based on the converged policy network and combined with the SLA-constrained action pruning mechanism, output the optimal resource allocation joint action for the current scheduling period; the expression of the SLA-constrained action pruning mechanism is: in, For a set of legal actions, For action The corresponding task's estimated execution time, This represents the upper limit of SLA latency for AI tasks.

6. The method for dynamic scheduling and optimization decision-making of AI computing resources in smart cities as described in claim 5, characterized in that, The weight update formula for the robust temporal difference update rule in step S34 is as follows: in, For adaptive learning rate, For robust timing differential error, the expression is: The adaptive learning rate The update formula is: in, The initial learning rate, The maximum number of iteration cycles. This is the attenuation coefficient.

Images