A streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning

Abstract

This invention relates to a streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning, belonging to the field of task scheduling in big data processing technology. First, two-stage clustering is performed on moving objects at the initial moment. Second, incremental clustering is used to filter moving objects to be switched, generating a candidate cluster set and constructing a search space. Then, the competition intensity of moving objects is calculated to construct a moving object cluster decision tree. Next, the spatial distribution of moving objects at the next moment is predicted, multi-agent reinforcement learning is executed, and the moving object cluster decision tree is updated using a comprehensive evaluation index that linearly weights partition locality, partition offset, and switching cost to obtain the optimal scheduling strategy. Finally, the one-to-one mapping between moving objects, clusters, partitions, and computing cores is re-executed according to the optimal scheduling strategy. This invention solves the problem of real-time task scheduling in large-scale traffic scenarios, significantly saving computing resources and reducing time latency.

Description

Technical Field

[0001] This invention relates to a streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning, belonging to the field of task scheduling in big data processing technology. Background Technology

[0002] Streaming computing engines, with their efficient real-time data processing capabilities, have been widely applied in fields such as the Internet and Location-Based Services (LBS). In LBS scenarios, streaming computing engines need to perform computationally intensive tasks (such as trajectory deviation detection, synchronous pattern mining, dual-stream spatial connectivity, map matching, prediction, and clustering) on ​​GPS data generated by hundreds of thousands of moving objects (such as buses and ride-hailing vehicles). Given the dynamic changes in the spatial location of moving objects, partitioning these moving objects and mapping them to different computing cores requires scheduling methods to simultaneously satisfy uniform partitioning, maintain partition locality, reduce switching costs to save computing resources, and reduce latency. Most existing scheduling methods schedule moving objects based on resource-level or data-level characteristics. While these methods meet the system or business metrics in their respective scenarios, they still have the following shortcomings:

[0003] (1) Multi-objective optimization cannot be achieved. The scheduling strategy needs to simultaneously satisfy the requirements of uniform partitioning, maintaining partition locality, and reducing switching costs, but existing scheduling methods based on static clustering or dynamic clustering cannot simultaneously satisfy the above three constraints.

[0004] (2) Insufficient real-time performance. The scheduling method based on reinforcement learning can satisfy the three constraints at the same time, but the scheduling of moving objects will generate a huge search space. Reinforcement learning searches for the optimal scheduling strategy in this space in an iterative manner, which takes too long and is difficult to support the real-time scheduling of large-scale moving objects.

[0005] (3) It is difficult to balance spatiotemporal dynamics and scheduling stability. In continuous time series, real-time scheduling of large-scale moving objects requires updating the scheduling strategy in real time according to the dynamically changing spatiotemporal distribution. However, the scheduling strategy of a single time window is often difficult to maximize the cumulative value over multiple time steps.

[0006] The clustering method proposed in the patent "CN117828388A, an improved heuristic K-means clustering method for vehicle network nodes" makes full use of the dynamic changes in the spatiotemporal attributes of moving objects and improves the K-means clustering algorithm, thereby increasing the clustering speed. However, it ignores two key indicators, data locality and data offset, resulting in poor clustering quality of the obtained cluster sets.

[0007] The clustering method proposed in patent "CN117591904A, a truck clustering method based on density clustering" accelerates the search for co-cluster points and enhances data locality by converting truck GPS into OD points and calculating the similarity between GeoHash values ​​and word vectors, compared to the DBSCAN clustering algorithm. However, it ignores the two factors of data offset and excessive code execution time. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide a streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning, in order to solve the problems of high operating costs and low resource utilization in streaming task scheduling in traffic scenarios.

[0009] The technical solution of this invention is as follows: Based on the above analysis, a streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning is proposed. First, two-stage clustering is performed on moving objects at the initial moment to generate an initial scheduling strategy that satisfies both partition locality and partition uniformity throughout the entire scheduling process. Second, incremental clustering is used to filter moving objects to be switched, reducing switching costs and selecting candidate clusters to construct a search space. Then, by calculating the competition intensity of moving objects, the search space is transformed into a moving object cluster decision tree. Next, the spatial distribution of moving objects at the next moment is predicted, and multi-agent reinforcement learning is iteratively executed to explore different scheduling strategies. The scheduling strategy is evaluated using a comprehensive evaluation index that linearly weights partition locality, partition offset, and switching cost, and the moving object cluster decision tree is updated to obtain the optimal scheduling strategy. Finally, the one-to-one mapping between moving objects, clusters, partitions, and computing cores is re-executed according to the optimal scheduling strategy. This invention solves the problem of real-time task scheduling in large-scale traffic scenarios and avoids the problems of high computational resource consumption and high time latency in existing solutions.

[0010] Specifically, the steps of a streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning are as follows:

[0011] Step 1: Obtain the spatial distribution of the initial time window of the moving object set on the road network, and perform two-stage clustering to obtain a cluster partition set as the initial scheduling strategy. In the first stage of clustering, a preset number of sub-intervals are generated by cutting longitude intervals, and the number of moving objects in different sub-intervals remains the same. In the second stage of clustering, the moving object with the largest longitude and the smallest latitude among the moving objects in each sub-interval is selected as the cluster center, and a preset number of moving objects closest to the cluster center are selected as co-cluster points, so that the cluster center and co-cluster points form a complete cluster.

[0012] Step 2: Using the spatial distribution and scheduling strategy of moving objects in the previous time window and the spatial distribution of moving objects in the current time window as input, set a time window threshold, a displacement threshold, and a radius threshold. Divide the time window according to the time window threshold, calculate the displacement of each moving object in the adjacent time window, and form a set of moving objects to be switched for moving objects whose displacement exceeds the displacement threshold and exceeds the bounding polygon range of the cluster to which the previous time window belongs. The scheduling strategy is the initial scheduling strategy in the first iteration, and the scheduling strategy of the previous window is adopted from the second iteration onwards.

[0013] Step 3: Using each moving object in the set of moving objects to be switched as the center and the radius threshold as the radius, construct several search areas, and form a candidate cluster set of the current moving object by the clusters that appear in the search areas;

[0014] Step 4: Construct the search space using the set of objects to be switched and the set of candidate clusters;

[0015] Step 5: Calculate the competition intensity of each moving object in the set of moving objects to be switched;

[0016] Step 6: Convert the search space into a Moving Object Cluster Decision Tree (MOCDT). According to the competition intensity from small to large, insert the moving objects into each level of the MOCDT from the root node downwards, and the branch of the node corresponds to the candidate cluster of the current moving object to obtain the initialized MOCDT.

[0017] Step 7: Predict the spatial distribution of all moving objects in the next time window;

[0018] Step 8: Multi-agent Q-learning combines the spatial distribution of the next time window, starting from the root node of MOCDT, to sequentially switch clusters of moving objects, and update the local score of the local scheduling strategy corresponding to each branch in MOCDT. After all moving objects have switched clusters, the global score of the global scheduling strategy corresponding to each path in MOCDT is updated by combining the linearly weighted comprehensive evaluation index.

[0019] Step 9: Iterate through Step 8 until the iteration stopping condition is met, traverse MOCDT, obtain the path with the highest global score, and generate the globally optimal scheduling strategy.

[0020] Step 10: Based on the global optimal scheduling strategy, map the moving objects, clusters, partitions and computing cores one by one, and execute tasks on the moving objects on the computing cores.

[0021] Step 11: Iterate through Steps 2 to 10 until the time series reaches the preset maximum time window number, thus completing the task scheduling for the moving object.

[0022] Optionally, Step 4 specifically includes:

[0023] Step 4.1: Obtain the set of moving objects to be switched through incremental clustering. With the set of candidate clusters of all moving objects to be switched ,in, represent Time of the first A moving object to be switched. represent The set of candidate clusters;

[0024] Step 4.2: Search Space It is a by and The constructed two-dimensional matrix stores all scheduling strategies, where rows represent moving objects to be switched. Cluster, expressed as:

[0025] (1)

[0026] in, This represents the number of objects to be switched or moved. Represents the total number of clusters. elements in The value can be 1 or 0. 1 means that the current cluster is a candidate cluster of the current moving object, and 0 means that the current cluster is not a candidate cluster of the current moving object.

[0027] Optionally, Step 5 specifically includes:

[0028] Step 5.1: Calculate the cluster's capacity, representing the number of movable objects that can still be allocated in the current partition. The number of movable objects is expressed as... The ideal number of moving objects within each partition is expressed as , No. The number of existing moving objects in each partition is represented as follows: The number of places available is expressed as The expression is:

[0029] (2)

[0030] in, This means that the number of spaces a cluster can accommodate is limited;

[0031] Step 5.2: Calculate the competition intensity of each moving object in the set of moving objects to be switched. The expression is:

[0032] (3)

[0033] (4)

[0034] in, express In cluster Intensity of competition within the country express Competition intensity across all clusters express The number of moving objects to be switched corresponding to the candidate clusters is used to determine the clustering certainty. The lower the competition intensity of the moving objects, the higher the certainty of the clustering.

[0035] Optionally, Step 6 specifically includes:

[0036] In order of increasing competition intensity, The moving objects are inserted into each layer of the MOCDT node in sequence, and the clusters of the moving objects are switched in the order from the root node to the leaf node.

[0037] The depth of MOCDT is equal to the number of moving objects to be switched. Each layer has only one node, representing the moving object to be switched. The number of branches of each node is equal to the number of candidate clusters corresponding to that node. The branches represent dividing the moving objects into different candidate clusters. Starting from the root node, traversing MOCDT downwards to the leaf node yields different paths, representing different global scheduling strategies and corresponding to global scores. The combination of each node and branch in the path represents a local scheduling strategy and corresponds to a local score.

[0038] Optionally, Step 8 specifically includes:

[0039] Will Each moving object to be switched in the process is considered an independent intelligent agent, represented as... ;

[0040] Represent the state as ,in, express state, express time Local scheduling strategy; express time The spatial coordinates are used to obtain the state space. Represented as ,in, express exist Time allocation to cluster ;

[0041] If the action is defined as assigning a moving object to a candidate cluster, then... Classified to the Cluster This action is represented as Action space All actions of an agent are stored and represented as ;

[0042] In multi-agent Q-learning, all agents adopt... - A greedy decay strategy is used to select actions, thereby generating a local scheduling strategy, represented as... ,in, express exist Local scheduling strategy at any given moment;

[0043] Local scheduling strategy Will The state is changed by Mapped to Receive feedback from the environment as an immediate reward. , For reward function The output value, where the reward matrix stores the reward matrix. All variables in the expression are:

[0044] (5)

[0045] in, Represents the reward matrix, express arrive The vertical Euclidean distance of the cluster centers, The larger, the more it means Switch to After that, the smaller the locality, the greater the locality; conversely, the larger the locality, when When it is greater than 0, it means Switch to , The offset decreases; conversely, the offset increases.

[0046] reward function The value of a local scheduling strategy can be measured from the perspective of a single agent, expressed as:

[0047] (6)

[0048] in, It consists of two parts: the normalized vertical Euclidean distance of the agent from the cluster center and the cluster's capacity. and These represent the weights of the two parts, and the sum of the weights is 1. When the cluster's capacity is greater than 0, the closer the agent is to the cluster center, the higher the locality of a certain local scheduling strategy, the lower the data offset, and the greater the immediate reward value.

[0049] The local Q-function is used to measure the value of local scheduling strategies from a global perspective, generating and updating the local score for each branch of MOCDT. Its expression is:

[0050] (7)

[0051] in, Represents the local Q-function. Represents a discount factor used to discount MOCDT. The cumulative discount sum of the maximum local scores of the multi-layered agents is introduced into the current... In the local Q function;

[0052] The value matrix stores the different local scheduling strategies and local scores of all agents, and its expression is:

[0053] (8)

[0054] in, Representing the value matrix, The representative will Classified to , Saved Local scheduling strategies and local scores;

[0055] The scheduler passes through All local scheduling policies are aggregated into a global scheduling policy, represented as: ;

[0056] Global Q function Used to evaluate the value of the global scheduling strategy, generating and updating the global score for each path in MOCDT, the expression is:

[0057] (9)

[0058] in, This represents the comprehensive evaluation value output by inputting a global scheduling strategy. It is a comprehensive evaluation index that is linearly weighted by partition offset, partition locality, and handover cost.

[0059] Optionally, there are three iteration stopping conditions, specifically:

[0060] Iteration stopping condition one: ;

[0061] Iteration stopping condition two: ;

[0062] Iteration stopping condition three: ;

[0063] in, The threshold for the global score. This represents the current iteration number. E The maximum number of iterations is, C It is a constant; when the global scheduling policy MOCDT stops updating when any of the iteration stopping conditions are met, and outputs the path with the highest global score as the globally optimal scheduling strategy. .

[0064] The beneficial effects of this invention are:

[0065] (1) This invention achieves multi-objective optimization by designing a comprehensive evaluation index with linear weighting of three constraints. This enables the scheduling strategy updated in this invention to meet the requirements of uniform partitioning, maintaining partition locality, and reducing switching costs in scenarios where the spatial distribution of moving objects changes dynamically in a continuous time series.

[0066] (2) Under the premise of satisfying the three constraints, the present invention significantly reduces the search space by incremental clustering, further plans the order of switching scheduling of mobile objects by competition intensity, transforms the search space into MOCDT, and finally designs three iteration stopping conditions. Through reinforcement learning, different scheduling strategies are explored in an iterative manner, the corresponding scores of MOCDT are updated, the search for the optimal scheduling strategy is accelerated, the real-time performance is improved, and it can support the real-time scheduling of large-scale mobile objects.

[0067] (3) This invention uses reinforcement learning to update the optimal scheduling strategy in real time and maximizes the cumulative value of the scheduling strategy in a single time window over multiple time steps, thus balancing spatiotemporal dynamics and scheduling stability. Attached Figure Description

[0068] Figure 1 This is a flowchart of the steps of the present invention;

[0069] Figure 2 This is the incremental clustering diagram of the present invention;

[0070] Figure 3 The moving object cluster decision tree initialized for this invention;

[0071] Figure 4 This is the updated moving object cluster determination tree of the present invention. Detailed Implementation

[0072] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0073] Example 1: As Figure 1 The diagram shows a flowchart of a streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning. In Step 1, a two-stage clustering process is performed only once in the initial time window to obtain the initial scheduling strategy. Subsequently, the scheduling process shown in the diagram is iteratively executed in each time window to obtain the optimal scheduling strategy (Steps 2-9). Next, in each time window, the one-to-one mapping of moving objects, clusters, partitions, and computing cores is completed according to the optimal scheduling strategy, and various computationally intensive tasks are executed (Step 10). Finally, the scheduling process stops when the predetermined maximum number of time windows is reached (Step 11). The four dashed boxes in the diagram, from bottom to top and left to right, represent: incremental clustering initialization of MOCDT, obtaining the spatial distribution of the current time window, and predicting the spatial distribution and MAQL of the next time window. The area outside the dashed boxes corresponds to iteratively updating MOCDT and obtaining the optimal scheduling strategy.

[0074] Specifically, the streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning includes the following steps:

[0075] Step 1: Obtain the spatial distribution of the initial time window of the moving object set on the road network, and perform two-stage clustering to obtain a cluster partition set as the initial scheduling strategy. In the first stage of clustering, a preset number of sub-intervals are generated by cutting longitude intervals, and the number of moving objects in different sub-intervals remains the same. In the second stage of clustering, the moving object with the largest longitude and the smallest latitude among the moving objects in each sub-interval is selected as the cluster center, and a preset number of moving objects closest to the cluster center are selected as co-cluster points, so that the cluster center and co-cluster points form a complete cluster.

[0076] Understandably, the results of the two-stage clustering serve as the initial input for the entire scheduling process, satisfying the requirements of partition locality and uniformity.

[0077] Step 2: Using the spatial distribution and scheduling strategy of moving objects in the previous time window and the spatial distribution of moving objects in the current time window as input, set a time window threshold, a displacement threshold, and a radius threshold. Divide the time window according to the time window threshold, calculate the displacement of each moving object in the adjacent time window, and form a set of moving objects to be switched for moving objects whose displacement exceeds the displacement threshold and exceeds the bounding polygon range of the cluster to which the previous time window belongs. The scheduling strategy is the initial scheduling strategy in the first iteration, and the scheduling strategy of the previous window is adopted from the second iteration onwards.

[0078] Specifically, the first time window uses two-stage clustering to generate an initial scheduling strategy; the second time window uses the initial scheduling strategy and spatial distribution of the first window, and the spatial distribution of the current window, to generate a scheduling strategy through incremental clustering and reinforcement learning; the third time window uses the scheduling strategy and spatial distribution of the second window, and the spatial distribution of the current window, to generate a scheduling strategy through incremental clustering and reinforcement learning, and so on, until the last time window generates a scheduling strategy.

[0079] Step 3: Using each moving object in the set of moving objects to be switched as the center and the radius threshold as the radius, construct several search areas, and form a candidate cluster set of the current moving object by the clusters that appear in the search areas;

[0080] Step 4: Construct the search space using the set of objects to be switched and the set of candidate clusters;

[0081] Step 4.1: Obtain the set of moving objects to be switched through incremental clustering. With the set of candidate clusters of all moving objects to be switched ,in, represent Time of the first A moving object to be switched. represent The set of candidate clusters;

[0082] Step 4.2: Search Space It is a by and The constructed two-dimensional matrix stores all scheduling strategies, where rows represent moving objects to be switched. Cluster, expressed as:

[0083] (1)

[0084] in, This represents the number of objects to be switched or moved. Represents the total number of clusters. elements in The value can be 1 or 0, where 1 represents that the current cluster is a candidate cluster for the currently moving object, and 0 represents that the current cluster is not a candidate cluster for the currently moving object. For example, when element When, it indicates the first m Cluster yes Candidate clusters.

[0085] Step 5: Calculate the competition intensity of each moving object in the set of moving objects to be switched;

[0086] Step 5.1: Calculate the cluster's capacity. The cluster's capacity refers to the difference between the ideal number of mobile objects within a partition and the current number of mobile objects within that partition. This capacity represents the number of mobile objects that can still be allocated to the current partition. The number of mobile objects is expressed as... The ideal number of moving objects within each partition is expressed as , No. The number of existing moving objects in each partition is represented as follows: The number of places available is expressed as The expression is:

[0087] (2)

[0088] in, This means that the number of spaces a cluster can accommodate is limited;

[0089] Step 5.2: Calculate the competition intensity of each moving object in the set of moving objects to be switched. The expression is:

[0090] (3)

[0091] (4)

[0092] in, express In cluster Intensity of competition within the country express Competition intensity across all clusters express The number of moving objects to be switched corresponding to the candidate clusters is used to determine the clustering certainty. The lower the competition intensity of the moving objects, the higher the certainty of the clustering.

[0093] It is important to understand that a cluster may be a candidate cluster for multiple moving objects, and a moving object may have multiple candidate clusters. Competition intensity refers to the intensity of competition among moving objects for the limited capacity within a cluster when selecting a candidate cluster.

[0094] Step 6: Convert the search space into a Moving Object Cluster Decision Tree (MOCDT). According to the competition intensity from small to large, insert the moving objects into each level of the MOCDT from the root node downwards, and the branch of the node corresponds to the candidate cluster of the current moving object to obtain the initialized MOCDT.

[0095] It is important to understand that the Moving Object Cluster Decision Tree (MOCDT) is derived from the search space and also stores all possible scheduling strategies. However, the difference is that MOCDT plans the order of moving objects switching clusters based on the contention intensity, making the mutual influence between adjacent moving objects when switching clusters more obvious, thereby accelerating the search for the optimal scheduling strategy.

[0096] Optionally, such as Figure 3 As shown, since the search space consists of randomly switching clusters of moving objects, the search for the optimal scheduling strategy takes too long. Therefore, in this embodiment, the optimal scheduling strategy is determined by ordering the competition intensity from lowest to highest. The moving objects are inserted into each layer of the MOCDT node in sequence, and the clusters of the moving objects are switched in the order from the root node to the leaf node.

[0097] The depth of MOCDT is equal to the number of moving objects to be switched. Each layer has only one node, representing the moving object to be switched. The number of branches of each node is equal to the number of candidate clusters corresponding to that node. The branches represent dividing the moving objects into different candidate clusters. Starting from the root node, traversing MOCDT downwards to the leaf node yields different paths, representing different global scheduling strategies and corresponding to global scores. The combination of each node and branch in the path represents a local scheduling strategy and corresponds to a local score.

[0098] Step 7: Predict the spatial distribution of all moving objects in the next time window;

[0099] Alternatively, in this embodiment, a trained linear model can be used for prediction;

[0100] Step 8: Multi-agent Q-learning combines the spatial distribution of the next time window, starting from the root node of MOCDT, to sequentially switch clusters of moving objects, and update the local score of the local scheduling strategy corresponding to each branch in MOCDT. After all moving objects have switched clusters, the global score of the global scheduling strategy corresponding to each path in MOCDT is updated by combining the linearly weighted comprehensive evaluation index.

[0101] Optionally, Each moving object to be switched in the process is considered an independent intelligent agent, represented as... ;

[0102] Represent the state as ,in, express state, express time Local scheduling strategy; express time The spatial coordinates are used to obtain the state space. Represented as ,in, express exist Time allocation to cluster ;

[0103] If the action is defined as assigning a moving object to a candidate cluster, then... Classified to the Cluster This action is represented as Action space All actions of an agent are stored and represented as ;

[0104] To balance the exploration and utilization of the action space, all agents in multi-agent Q-learning adopt... - A greedy decay strategy is used to select actions, thereby generating a local scheduling strategy, represented as... ,in, express exist Local scheduling strategy at any given moment;

[0105] Local scheduling strategy Will The state is changed by Mapped to Receive feedback from the environment as an immediate reward. , For reward function The output value, where the reward matrix stores the reward matrix. All variables in the expression are:

[0106] (5)

[0107] in, Represents the reward matrix, express arrive The vertical Euclidean distance of the cluster centers, The larger, the more it means Switch to After that, the smaller the locality, the greater the locality; conversely, the larger the locality, when When it is greater than 0, it means Switch to , The offset decreases; conversely, the offset increases.

[0108] reward function The value of a local scheduling strategy can be measured from the perspective of a single agent, expressed as:

[0109] (6)

[0110] in, It consists of two parts: the normalized vertical Euclidean distance of the agent from the cluster center and the cluster's capacity. and These represent the weights of the two parts, and the sum of the weights is 1. When the cluster's capacity is greater than 0, the closer the agent is to the cluster center, the higher the locality of a certain local scheduling strategy, the lower the data offset, and the greater the immediate reward value.

[0111] The local Q-function is used to measure the value of local scheduling strategies from a global perspective, generating and updating the local score for each branch of MOCDT. Its expression is:

[0112] (7)

[0113] in, Represents the local Q-function. Represents a discount factor used to discount MOCDT. The cumulative discount sum of the maximum local scores of the multi-layered agents is introduced into the current... In the local Q-function, the scheduler can measure from a global perspective. Local scheduling strategy Its value to the whole.

[0114] It's important to understand that designing the local Q-function presents two challenges: first, the local optimum trap, where a single agent, viewing the local situation from a local perspective, uses immediate reward values ​​to evaluate the value of its local scheduling strategy, easily falling into local optima; second, competition between agents, where the candidate cluster sets of different agents overlap, but the capacity of candidate clusters is limited, leading to competition between agents. These two points constrain and affect the search for the globally optimal scheduling strategy.

[0115] To address the two challenges mentioned above, this embodiment introduces a global-perspective competitive balancing mechanism into the local Q-function. This mechanism consists of two parts: first, an immediate reward value, which measures the performance from the local perspective of a single agent. for The first is the value of the agent, the greater the better; the second is the maximum local score of the other agents, which is determined by introducing MOCDT. The following is a sum of discounts on the maximum local scores of the multi-layered agents, measured from a global perspective. To maximize local scores and avoid getting trapped in local optima. Furthermore, the higher the local scores of the other agents, the better. The higher the local score, the less invalid competition for limited slots in a given area is avoided.

[0116] The value matrix stores the different local scheduling strategies and local scores of all agents, and its expression is:

[0117] (8)

[0118] in, Representing the value matrix, The representative will Classified to , Saved Local scheduling strategies and local scores;

[0119] The scheduler passes through All local scheduling policies are aggregated into a global scheduling policy, represented as: ;

[0120] Global Q function Used to evaluate the value of the global scheduling strategy, generating and updating the global score for each path in MOCDT, the expression is:

[0121] (9)

[0122] in, This represents the comprehensive evaluation value output by inputting a global scheduling strategy. It is a comprehensive evaluation index that is linearly weighted by partition offset, partition locality, and handover cost.

[0123] It's important to understand that designing the global Q-function presents a challenge: real-time scheduling of moving objects is a continuous scheduling process over a continuous time series. The scheduler is prone to short-sightedness when evaluating the global scheduling strategy, meaning the current global scheduling strategy is only optimal for the current moment, and its long-term value is not maximized. To address this issue, this embodiment introduces a temporal difference mechanism into the global Q-function, consisting of two parts: the global scheduling strategy... The sum of the global score and the discounted global scores of the global scheduling strategies at subsequent time steps is used to search for the globally optimal scheduling strategy with long-term value in the MOCDT from a long-term perspective.

[0124] Step 9: Iterate through Step 8 until the iteration stopping condition is met, traverse MOCDT, obtain the path with the highest global score, and generate the globally optimal scheduling strategy.

[0125] Optionally, there are three iteration stopping conditions, specifically:

[0126] Iteration stopping condition one: ;

[0127] Iteration stopping condition two: ;

[0128] Iteration stopping condition three: ;

[0129] in, The threshold for the global score. This represents the current iteration number. E The maximum number of iterations is, C It is a constant; when the global scheduling policy MOCDT stops updating when any of the iteration stopping conditions are met, and outputs the path with the highest global score as the globally optimal scheduling strategy. .

[0130] It is important to understand that updating the cluster decision tree refers to MAQL iteratively updating the local scores of all branches and the global scores of all paths in the MOCDT.

[0131] Furthermore, Figure 4 This is the updated MOCDT. By predicting the spatial distribution of all moving objects in the next time window, MAQL iteratively divides the moving objects into clusters from the root node to the leaf node, and updates the local score of each branch and the global score of all paths. The iteration is stopped by three stopping conditions, and the path with the largest global score generates the optimal scheduling strategy, corresponding to Steps 7 to 9.

[0132] Step 10: Based on the global optimal scheduling strategy, map the moving objects, clusters, partitions and computing cores one by one, and execute tasks on the moving objects on the computing cores.

[0133] Optionally, the implementation task can be to perform a variety of computationally intensive tasks, such as GPS map matching of moving objects, trajectory clustering, synchronous movement pattern mining, line anomaly detection, and dual-stream spatial connection.

[0134] Step 11: Iterate through Steps 2 to 10 until the time series reaches the preset maximum time window number, thus completing the task scheduling for the moving object.

[0135] Furthermore, the technical solution of the present invention will be further illustrated below through a specific implementation example.

[0136] Step 1: Obtain the spatial distribution of 10,000 buses in Shenzhen on the road network at 09:00:00 on January 1, 2019 (initial time window), and perform two-stage clustering to obtain the initial input for the scheduling process. The first stage of clustering involves dividing the longitude into 5 sub-intervals, with each sub-interval containing moving objects. The second stage of clustering is performed iteratively within each subinterval until each subinterval is divided into 5 clusters, and all subintervals are divided into 5 clusters. There are clusters, and the moving objects within each cluster are... The construction process for each cluster is as follows: Select the moving object with the largest longitude and the smallest latitude among the unassigned moving objects in each sub-interval as the cluster center, and then select the moving object closest to the cluster center. Let each moving object be a cluster of points, and let the cluster center and the cluster of points form a complete cluster.

[0137] Step 2: Incremental Clustering (Filtering Stage). This assumes... There are 6 moving objects on the timeline network. With 3 clusters ,like Figure 2 As shown in (a), the threshold combination (time window threshold, displacement threshold, radius threshold) takes values ​​of (5 minutes, 3000 meters, 4000 meters) to obtain... exist Spatial distribution at time, calculated from Time's up The displacement at time, where The displacement is greater than 3000 meters, and at the same time... The GPS location at that time was outside the range. Given the spatial extent of each cluster at any given time, the set of moving objects to be switched is... .

[0138] Step 3: Incremental Clustering (Search Phase). (The remaining text appears to be incomplete and requires further context.) A circular search area is constructed with a center of 4000 meters and a radius of 4000 meters. Clusters appearing within this search area form a candidate cluster set for the corresponding moving object. The candidate cluster sets are respectively , , ,like Figure 2 As shown in (b);

[0139] Specifically, the filtering stage, such as Figure 2 (a) Generate a set of objects to be switched using a displacement threshold combined with the scheduling strategy of the previous time window. During the search phase, such as... Figure 2 (b) A candidate cluster set is generated for the set of moving objects to be switched using a radius threshold. The two sets together construct the search space, storing all possible scheduling strategies, as shown in formula (1). Figure 2 (a) The dashed arrow represents the moving object from arrive The displacement. The colored dashed hollow circle represents the displacement of the moving object. The position of the moving object. A solid colored circle represents the location of the moving object. The location. Figure 2 (b) The dashed circle represents the search area of ​​the moving object, and the dashed line represents the radius of the search area.

[0140] Step 4: Construct the search space. For row index, Using column indices, a two-dimensional matrix is ​​constructed as the search space to store all possible scheduling strategies. Each element in the search space takes a value of either 0 or 1; for example, a row index is... The column index is The element takes a value of 1, representing yes A candidate cluster, otherwise not;

[0141] Step 5: According to formulas (3) and (4) The competitive intensities were 8, 6, and 8, respectively.

[0142] Step 6: Arrange the items in order of increasing competition intensity. (or The nodes at levels 1, 2, and 3, with branches for each level as follows: , and MOCDT initialization is complete, as follows: Figure 3 As shown;

[0143] Step 7: Prediction exist Spatial distribution of time;

[0144] Step 8: Multi-Agent Q-Learning (MAQL) updates the MOCDT. MAQL combines all moving objects in... The spatial distribution of time points is as follows: Switch clusters and update the corresponding local scores in MOCDT according to formula (7). After all moving objects switch clusters, update the corresponding global scores according to formula (9).

[0145] Step 9: Iterate through Step 8 to update MOCDT. The process is shown in Tables 1 and 2, and the meanings of the variables in the tables are shown in Table 3. After the update, the highest global score of MOCDT is 1.46, and the corresponding globally optimal scheduling strategy is ( , ), ( , ),( , );

[0146] Step 10: Perform a one-to-one mapping between moving objects, clusters, partitions, and computing cores based on the globally optimal scheduling strategy: , Mapping to clusters To partition To the computing core , ( , Mapped to arrive arrive , ( , Mapped to arrive arrive On the computing core, various computationally intensive tasks are performed on moving objects, such as GPS map matching of moving objects, trajectory clustering, synchronous movement pattern mining, line anomaly detection, and dual-stream spatial connection.

[0147] Step 11: Iterate through Steps 2 to 10 until the time series reaches the maximum number of time windows, thus completing the task scheduling for the moving object.

[0148] Table 1 MOCDT Local Scores

[0149]

[0150] Table 2 MOCDT Global Score

[0151]

[0152] Table 3. Variables and their meanings in Tables 1 and 2.

[0153]

[0154] In summary, compared with existing scheduling schemes, this invention can schedule large-scale moving objects with dynamic spatiotemporal distribution in real time by designing two-stage clustering, incremental clustering, MOCDT and prediction-based MAQL. At the same time, each update of the scheduling strategy can maintain partition locality, reduce partition offset, reduce switching costs, thereby improving resource utilization and reducing time latency between tasks.

[0155] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning, characterized in that, The method includes the following steps: Step 1: Obtain the spatial distribution of the initial time window of the moving object set on the road network, and perform two-stage clustering to obtain a cluster partition set as the initial scheduling strategy. In the first stage of clustering, a preset number of sub-intervals are generated by cutting longitude intervals, and the number of moving objects in different sub-intervals remains the same. In the second stage of clustering, the moving object with the largest longitude and the smallest latitude among the moving objects in each sub-interval is selected as the cluster center, and a preset number of moving objects closest to the cluster center are selected as co-cluster points, so that the cluster center and co-cluster points form a complete cluster. Step 2: Using the spatial distribution and scheduling strategy of moving objects in the previous time window and the spatial distribution of moving objects in the current time window as input, set a time window threshold, a displacement threshold, and a radius threshold. Divide the time window according to the time window threshold, calculate the displacement of each moving object in the adjacent time window, and form a set of moving objects to be switched for moving objects whose displacement exceeds the displacement threshold and exceeds the bounding polygon range of the cluster to which the previous time window belongs. The scheduling strategy is the initial scheduling strategy in the first iteration, and the scheduling strategy of the previous window is adopted from the second iteration onwards. Step 3: Using each moving object in the set of moving objects to be switched as the center and the radius threshold as the radius, construct several search areas, and form a candidate cluster set of the current moving object by the clusters that appear in the search areas; Step 4: Construct the search space using the set of objects to be switched and the set of candidate clusters; Step 5: Calculate the competition intensity of each moving object in the set of moving objects to be switched; Step 6: Convert the search space into a Moving Object Cluster Decision Tree (MOCDT). According to the competition intensity from small to large, insert the moving objects into each level of the MOCDT from the root node downwards, and the branch of the node corresponds to the candidate cluster of the current moving object to obtain the initialized MOCDT. Step 7: Predict the spatial distribution of all moving objects in the next time window; Step 8: Multi-agent Q-learning combines the spatial distribution of the next time window, starting from the root node of MOCDT, to sequentially switch clusters of moving objects, and update the local score of the local scheduling strategy corresponding to each branch in MOCDT. After all moving objects have switched clusters, the global score of the global scheduling strategy corresponding to each path in MOCDT is updated by combining the linearly weighted comprehensive evaluation index. Step 9: Iterate through Step 8 until the iteration stopping condition is met, traverse MOCDT, obtain the path with the highest global score, and generate the globally optimal scheduling strategy. Step 10: Based on the global optimal scheduling strategy, map the moving objects, clusters, partitions and computing cores one by one, and execute tasks on the moving objects on the computing cores. Step 11: Iterate through Steps 2 to 10 until the time series reaches the preset maximum time window number, thus completing the task scheduling for the moving object.

2. The streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning according to claim 1, characterized in that, Step 4 specifically refers to: Step 4.1: Obtain the set of moving objects to be switched through incremental clustering. With the set of candidate clusters of all moving objects to be switched ,in, represent Time of the first A moving object to be switched. represent The set of candidate clusters; Step 4.2: Search Space It is a by and The constructed two-dimensional matrix stores all scheduling strategies, where rows represent moving objects to be switched. Cluster, expressed as: （1）； in, This represents the number of objects to be switched or moved. Represents the total number of clusters. elements in The value can be 1 or 0. 1 means that the current cluster is a candidate cluster of the current moving object, and 0 means that the current cluster is not a candidate cluster of the current moving object.

3. The streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning according to claim 2, characterized in that, Step 5 specifically includes: Step 5.1: Calculate the cluster's capacity, representing the number of movable objects that can still be allocated in the current partition. The number of movable objects is expressed as... The ideal number of moving objects within each partition is expressed as , No. The number of existing moving objects in each partition is represented as follows: The number of places available is expressed as The expression is: （2）； in, This means that the number of spaces a cluster can accommodate is limited; Step 5.2: Calculate the competition intensity of each moving object in the set of moving objects to be switched. The expression is: （3）； （4）； in, express In cluster Intensity of competition within the country express Competition intensity across all clusters express The number of moving objects to be switched corresponding to the candidate clusters is used to determine the clustering certainty. The lower the competition intensity of the moving objects, the higher the certainty of the clustering.

4. The streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning according to claim 3, characterized in that, Step 6 specifically refers to: In order of increasing competition intensity, The moving objects are inserted into each layer of the MOCDT node in sequence, and the clusters of the moving objects are switched in the order from the root node to the leaf node. The depth of MOCDT is equal to the number of moving objects to be switched. Each layer has only one node, representing the moving object to be switched. The number of branches of each node is equal to the number of candidate clusters corresponding to that node. The branches represent dividing the moving objects into different candidate clusters. Starting from the root node, traversing MOCDT downwards to the leaf node yields different paths, representing different global scheduling strategies and corresponding to global scores. The combination of each node and branch in the path represents a local scheduling strategy and corresponds to a local score.

5. The streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning according to claim 4, characterized in that, Step 8 specifically refers to: Will Each moving object to be switched in the process is considered an independent intelligent agent, represented as... ; Represent the state as ,in, express state, express time Local scheduling strategy; express time The spatial coordinates are used to obtain the state space. Represented as ,in, express exist Time allocation to cluster ; If the action is defined as assigning a moving object to a candidate cluster, then... Classified to the Cluster This action is represented as Action space All actions of an agent are stored and represented as ; In multi-agent Q-learning, all agents adopt... - A greedy decay strategy is used to select actions, thereby generating a local scheduling strategy, represented as... ,in, express exist Local scheduling strategy at any given moment; Local scheduling strategy Will The state is changed by Mapped to Receive feedback from the environment as an immediate reward. , For reward function The output value, where the reward matrix stores the reward matrix. All variables in the expression are: （5）； in, Represents the reward matrix, express arrive The vertical Euclidean distance of the cluster centers, The larger, the more Switch to After that, the smaller the locality, the greater the locality; conversely, the larger the locality, when When it is greater than 0, it means Switch to , The offset decreases; conversely, the offset increases. reward function The value of a local scheduling strategy can be measured from the perspective of a single agent, expressed as: （6）； in, It consists of two parts: the normalized vertical Euclidean distance of the agent from the cluster center and the cluster's capacity. and These represent the weights of the two parts, and the sum of the weights is 1. When the cluster's capacity is greater than 0, the closer the agent is to the cluster center, the higher the locality of a certain local scheduling strategy, the lower the data offset, and the greater the immediate reward value. The local Q-function is used to measure the value of local scheduling strategies from a global perspective, generating and updating the local score for each branch of MOCDT. Its expression is: （7）； in, Represents the local Q-function. Represents the discount factor used in MOCDT. The cumulative discount sum of the maximum local scores of the multi-layered agents is introduced into the current... In the local Q function; The value matrix stores the different local scheduling strategies and local scores of all agents, and its expression is: （8）； in, Representing the value matrix, The representative will Classified to , Saved Local scheduling strategies and local scores; The scheduler passes through All local scheduling policies are aggregated into a global scheduling policy, represented as follows: ; Global Q function Used to evaluate the value of the global scheduling strategy, generating and updating the global score for each path in MOCDT, the expression is: （9）； in, This represents the comprehensive evaluation value output by inputting a global scheduling strategy. It is a comprehensive evaluation index that is linearly weighted by partition offset, partition locality, and handover cost.

6. The streaming computing task scheduling method based on spatiotemporal awareness and multi-agent reinforcement learning according to claim 5, characterized in that, There are three iteration stopping conditions, specifically: Iteration stopping condition one: ; Iteration stopping condition two: ; Iteration stopping condition three: ; in, The threshold for the global score. This represents the current iteration number. E The maximum number of iterations, C It is a constant; when the global scheduling policy MOCDT stops updating when any of the iteration stopping conditions are met, and outputs the path with the highest global score as the globally optimal scheduling strategy. .

Images