Task scheduling method and system based on big data analysis
The scheduling system, which utilizes big data analytics and combines graph convolutional networks and multi-scale attention mechanisms, achieves accurate prediction and dynamic scheduling of resource demands. This solves the problems of resource prediction bias, inefficient human-machine collaboration, and lagging rule updates in existing technologies, thereby improving the system's resource utilization efficiency and robustness.
Patent Information
- Application Number
- CN202511013997.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-23
- Publication Date
- 2025-11-07
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing task scheduling systems suffer from insufficient accuracy in resource prediction, low efficiency in human-machine collaboration, passive fault handling, and lagging rule updates, resulting in low resource utilization efficiency and poor system robustness.
A task scheduling system based on big data analysis is adopted, including a spatiotemporal prediction module, a human-machine decision coordination module, an active resource scheduling module, and a knowledge management module. Through technologies such as graph convolutional networks, multi-scale attention mechanisms, fuzzy logic algorithms, health assessment matrices, knowledge graphs, and reinforcement learning, dynamic resource allocation and decision optimization are achieved.
It improves resource utilization efficiency, enhances system robustness and decision consistency, reduces the risk of failure propagation, and ensures service continuity and flexibility.
Smart Images

Figure CN120909722A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of big data analysis and intelligent scheduling technology, and in particular to a scheduling task method and system based on big data analysis. Background Technology
[0002] With the development of distributed computing and IoT technologies, task scheduling systems need to handle massive amounts of heterogeneous data, dynamic resource requirements, and complex environmental interference. Existing technologies have the following limitations:
[0003] Insufficient accuracy in resource forecasting: Traditional methods rely on static historical data or single time series analysis, which makes it difficult to capture dynamic changes in topology and spatiotemporal correlations, leading to deviations in resource demand forecasting and causing resource overload or idleness.
[0004] Low efficiency of human-machine collaboration: Most systems use AI decision-making with fixed weights or rely entirely on human intervention, which cannot dynamically balance AI suggestions and human instructions. In particular, they are prone to scheduling conflicts or response delays when the environment changes suddenly.
[0005] Passive fault handling: Existing solutions typically activate emergency mechanisms after a fault occurs, lacking real-time assessment of node health status and proactive suppression of fault propagation paths, leading to the spread of service interruption risks.
[0006] Delayed rule updates: Scheduling rules are mostly based on fixed policy libraries, making it difficult to integrate real-time operational experience and algorithm logs. Rule conflicts and knowledge fragmentation are prominent issues, affecting the consistency and credibility of decisions.
[0007] To address the aforementioned issues, there is an urgent need for an intelligent scheduling method that integrates spatiotemporal prediction, dynamic weight allocation, proactive fault tolerance, and knowledge-driven approaches to improve resource utilization efficiency and system robustness in complex environments. Summary of the Invention
[0008] The main objective of this invention is to provide a scheduling task method and system based on big data analysis, which can effectively solve the problems in the background art.
[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0010] A task scheduling system based on big data analytics includes the following modules:
[0011] Spatiotemporal prediction module: Real-time collection of node historical logs and topology data, combined with graph convolutional networks and time series analysis to extract spatiotemporal features, and using multi-scale attention mechanism to predict resource demand curves and node vulnerability scores, providing a forward-looking decision-making basis for the system;
[0012] Human-machine decision coordination module: based on fuzzy logic dynamic balance algorithm to suggest and artificial instruction weight, using semantic analysis and constraint optimization solver to generate multi-objective scheduling scheme, and through the visual rule engine output with decision logic marked instruction set, realize the human-computer bidirectional collaborative optimization;
[0013] Active resource scheduling module: based on health degree evaluation matrix to construct resource portrait, using propagation tree model to simulate the fault diffusion path in topology, automatically trigger software defined network isolation strategy when the risk is over threshold, and dynamically allocate backup resources to ensure service continuity;
[0014] Knowledge management module: fusion artificial operation, equipment procedure and algorithm log to construct knowledge graph, through formal verification to ensure the consistency of new rule logic, combined with reinforcement learning dynamic iteration strategy library, form a multi-layer intelligent decision rule system with credibility grading.
[0015] Preferably, the spatio-temporal prediction module specifically comprises:
[0016] Data acquisition unit: Apache Kafka is used to acquire historical task logs, resource consumption time series data and device topology connection relationship of distributed nodes in real time;
[0017] Spatio-temporal feature extraction unit: based on graph convolution network to construct topology space feature matrix, coupled with time series analysis to extract resource fluctuation periodicity characteristics;
[0018] Prediction model operation unit: multi-scale attention mechanism is used to fuse spatio-temporal features, output resource demand prediction curve and node vulnerability score.
[0019] Preferably, the spatio-temporal feature extraction unit performs asymmetric graph convolution operation according to the following formula:
[0020] H topo =-∑p i logp i +α(1-e -βΔt )
[0021] Where, p i =deg(i) is the contact weight of node i, which is calculated by the degree centrality (i.e. the number of connections) of the node; Δt is the time interval (unit: second) of the change of topology structure; α, β are dynamic state parameters, the value range is (0, 1), which is used to adjust the weight and decay rate of dynamic change term; log is the natural logarithm, which is used to calculate information entropy.
[0022] Preferably, the human-machine decision coordination module specifically comprises:
[0023] Instruction analysis unit: integrate BERT model to realize operation semantic analysis, receive prediction results and manual instructions, and analyze operation intention through semantic understanding model;
[0024] Weight allocation unit: dynamically adjust the decision weight of AI suggestion and manual intervention based on fuzzy logic algorithm;
[0025] Specifically, the weight allocation unit calculates the dynamic weight according to the following formula:
[0026]
[0027] Wherein, μ AI ,μ H are the initial preset baseline weights of AI suggestion and manual decision; k is the rate coefficient of weight growth with time; T is the system running time; Var(H) is the variance of historical records of manual decision, used to reflect the volatility of manual decision; ε is the adjustment coefficient, used to balance the influence of variance on weight;
[0028] Strategy generation unit: adopt constraint satisfaction problem solver to generate multi-objective scheduling scheme.
[0029] Preferably, the active resource scheduling module specifically comprises:
[0030] Health degree evaluation unit: performs double-channel health degree calculation according to the following formula:
[0031]
[0032] Wherein, ω1 is the initial weight coefficient of health degree; λ is the time attenuation coefficient, controlling the growth rate of weight with time; t is the continuous running time of system; SLA i is the actual value of the i th service level agreement (SLA) of node j; SLA max is the theoretical maximum value of SLA; R(j) ready ,R(j) total is the prepared resource amount and total resource amount of node j; d is the contribution coefficient of resource preparation degree to health degree; ⊙ is Hadamard product (element-wise multiplication), indicating the fusion of two parts after independent calculation;
[0033] Fault path analysis unit: construct probability back propagation tree:
[0034]
[0035] Wherein, R i ,R j are the resource vectors of nodes i and j; ||R i ||,||R j || are the lengths of resource vectors, indicating the total amount of resources; Hop i,jis the topological hop number from node i to j; h j is the health degree of node j; and α, β, and γ are weight parameters for adjusting the contribution proportion of different factors.
[0036] Preferably, the knowledge management module specifically comprises:
[0037] a heterogeneous knowledge fusion unit: a knowledge graph containing artificial operation records, equipment procedures, and algorithm logs is constructed;
[0038] a rule verification unit: formal verification methods are used to detect the logical consistency of decision rules.
[0039] The application also discloses a scheduling task method based on big data analysis, and the specific steps are as follows:
[0040] S1. Spatiotemporal prediction and resource demand modeling:
[0041] Historical task logs, resource consumption data, and topological connection states of nodes are collected in real time through Apache Kafka to construct a spatiotemporal feature matrix; topological structure features are extracted using a graph convolution network, periodic rules are captured by combining time series analysis, spatiotemporal features are fused through a multi-scale attention mechanism, and resource demand prediction results are output;
[0042] S2. Human-machine collaborative decision scheduling:
[0043] User operation instructions and scheduling suggestions generated by AI are analyzed, and the weights of artificial decision and AI suggestions are balanced based on a dynamic weight distribution model; a multi-objective scheduling scheme is generated using a constraint optimization solver, and decision options are fed back through a visual interface to receive artificial correction instructions;
[0044] S3. Active resource dynamic allocation:
[0045] A resource scheduling graph is constructed according to the node health degree evaluation results, the health degree is fused by a double-channel calculation model to integrate service level agreement (SLA) compliance rate and resource preparation degree; when the health degree is lower than a threshold value, a beacon architecture model is used to reversely block the failure propagation path to trigger a resource reallocation strategy to ensure service continuity;
[0046] S4. Knowledge-driven decision optimization:
[0047] A knowledge graph is constructed by fusing artificial operation records, equipment procedures, and algorithm logs, and the consistency of formal constraints is verified; a policy library is dynamically updated based on reinforcement learning to generate multi-layer decision rules with credibility classification, which are fed back to the weight distribution model in step S2 for optimization
[0048] Preferably, the construction of the resource scheduling graph comprises the following sub-steps:
[0049] S31. Based on the node health assessment results, mark the nodes with a health degree lower than a preset threshold as vulnerable nodes, and generate a vulnerability heat map;
[0050] S32. Simulate a fault diffusion path through a beacon architecture model, insert a virtual isolation barrier between the vulnerable nodes and their adjacent nodes, and block the fault propagation link;
[0051] S33. According to the residual resource capacity of each node in the resource scheduling map and the task queue load, dynamically allocate redundant resources to the vulnerable nodes, and preferentially meet the continuity requirements of critical tasks.
[0052] Preferably, in step S4, the construction and rule optimization of the knowledge graph include the following sub-steps:
[0053] S41. Perform semantic annotation on the manual operation records, extract operation intent and execution context features, and align them with the standard operation steps in the device procedure;
[0054] S42. Detect rule conflicts, including logical contradictions, resource overallocation, and timing violations, through a formal verification method, generate conflict resolution suggestions, and feed them back to the policy library;
[0055] S43. Based on the reinforcement learning model, evaluate the historical execution effect of the rules in the policy library, dynamically eliminate inefficient rules combined with the credibility classification labels, and inject new rules to cover unforeseen scheduling scenarios.
[0056] Compared with the prior art, the present application has the following beneficial effects:
[0057] 1. The present application captures the spatial correlation of the topological structure through the graph convolution network, extracts periodic features combined with time series analysis, and introduces a multi-scale attention mechanism to fuse long-term, medium-term and short-term spatio-temporal dependencies, solving the prediction bias problem caused by ignoring dynamic topological changes in traditional methods.
[0058] 2. The present application adjusts the weight of AI suggestions and manual decisions in real time through a dynamic weight distribution model, generates a multi-objective scheduling scheme combined with a constraint optimization solver, effectively solving the decision rigidity problem of fixed weight strategies in environmental mutations.
[0059] 3. The present application is based on a dual-channel health assessment model, which constructs a knowledge graph by fusing manual operation records, device procedures and algorithm logs, and eliminates rule conflicts through formal verification, solving the inconsistent decision problem caused by knowledge fragmentation in traditional rule libraries; through a beacon architecture model to simulate the fault propagation path and insert a virtual isolation barrier to block the fault diffusion link. BRIEF DESCRIPTION OF DRAWINGS
[0060] Figure 1 is a block diagram of the present application. DETAILED DESCRIPTION
[0061] In order to make the technical means, creative features, purposes and effects of the present application easy to understand, the present application is further described below in conjunction with specific embodiments.
[0062] The application discloses a scheduling task system based on big data analysis, which can be realized through software programming (such as Python and Java) and a big data platform (such as Apache Kafka and Spark) and deployed in a distributed computing environment.
[0063] The system specifically comprises:
[0064] The space-time prediction module: real-time collection of node historical logs and topology data, extraction of space-time features by combining graph convolution network and time series analysis, prediction of resource demand curve and node vulnerability score by using multi-scale attention mechanism, and provision of prospective decision basis for the system;
[0065] The man-machine decision coordination module: suggestion and artificial instruction weight based on fuzzy logic dynamic balance algorithm, generation of multi-objective scheduling scheme by using semantic analysis and constraint optimization solver, output of instruction set with decision logic annotation by a visual rule engine, and realization of man-machine bidirectional collaborative optimization;
[0066] The active resource scheduling module: construction of resource portrait based on health degree evaluation matrix, simulation of fault diffusion path in topology by using a propagation tree model, automatic triggering of software-defined network isolation strategy when the risk is over the threshold, and synchronous dynamic allocation of backup resources to guarantee service continuity;
[0067] The knowledge management module: construction of a knowledge graph by fusing artificial operation, equipment procedure and algorithm log, ensuring of the consistency of newly added rule logic by formal verification, combination of a dynamic iteration strategy library based on reinforcement learning, and formation of a multi-layer intelligent decision rule system with credibility grading.
[0068] The application also discloses a scheduling task method based on big data analysis, which is suitable for the above system and specifically comprises the following steps:
[0069] S1. Space-time prediction and resource demand modeling:
[0070] The historical task logs, resource consumption data and topology connection state of nodes are collected in real time through Apache Kafka, a space-time feature matrix is constructed, topology structure features are extracted by using a graph convolution network, periodic rules are captured by combining time series analysis, space-time features are fused by using a multi-scale attention mechanism, and a resource demand prediction result is output;
[0071] S2. Man-machine collaborative decision scheduling:
[0072] The system analyzes user operation commands and AI-generated scheduling suggestions, balances the weights of human decision-making and AI suggestions based on a dynamic weight allocation model, generates multi-objective scheduling schemes using a constraint optimization solver, and provides feedback on decision options through a visual interface while receiving human correction commands.
[0073] S3. Active dynamic resource allocation:
[0074] A resource scheduling graph is constructed based on the node health assessment results. The health score is calculated by a dual-channel model that integrates the Service Level Agreement (SLA) compliance rate and resource readiness. When the health score is below the threshold, the fault propagation path is blocked in reverse based on the beacon architecture model, triggering a resource reallocation strategy to ensure service continuity.
[0075] S4. Knowledge-driven decision optimization:
[0076] A knowledge graph is constructed by integrating manual operation records, equipment procedures, and algorithm logs. The consistency of rules is verified through formal constraints. The policy library is dynamically updated based on reinforcement learning to generate multi-level decision rules with credibility levels, which are fed back to step S2 to optimize the weight allocation model.
[0077] It should be noted that the construction of the resource scheduling graph includes the following sub-steps:
[0078] S31. Based on the node health assessment results, nodes with health below a preset threshold are marked as vulnerable nodes, and a vulnerability heatmap is generated;
[0079] S32. Simulate the fault propagation path using a beacon architecture model, insert virtual isolation barriers between vulnerable nodes and their adjacent nodes to block the fault propagation link;
[0080] S33. Based on the remaining resource capacity of each node in the resource scheduling diagram and the task queue load, dynamically allocate redundant resources to vulnerable nodes, prioritizing the continuity requirements of critical tasks.
[0081] The construction and rule optimization of knowledge graphs include the following sub-steps:
[0082] S41. Perform semantic annotation on manual operation records, extract operation intent and execution context features, and align them with standard operating procedures in equipment specifications;
[0083] S42. Detect rule conflicts through formal verification methods, including logical contradictions, resource over-provisioning, and timing violations, generate conflict resolution suggestions, and feed them back to the strategy library;
[0084] S43. Based on the reinforcement learning model, evaluate the historical execution effect of rules in the policy library, combine the credibility level label to dynamically eliminate inefficient rules, and inject new rules to cover unforeseen scheduling scenarios.
[0085] The present application is further disclosed in connection with specific examples: the present example is cloud computing task scheduling:
[0086] Scenario: A certain cloud computing platform needs to handle the sudden surge of order processing tasks during the "Double 11" shopping festival, and requires dynamic scheduling of 100 node resources to ensure service continuity,
[0087] Implementation process
[0088] 1. Spatiotemporal prediction and feature extraction
[0089] Data collection: Through Kafka, real-time collection of CPU utilization, memory occupancy and topology connection state of each node, generation of adjacency matrix A and resource feature matrix X.
[0090] Topology complexity calculation:
[0091] Using the formula H topo =-∑p i logp i +α(1-e -βΔt ):
[0092] Where, p i =deg(i) is the contact weight of node i, calculated from the degree centrality (i.e. the number of connections) of the node; Δt is the time interval (unit: seconds) at which the topology structure changes; α, β are dynamic state parameters, with a value range of (0, 1), used to adjust the weight and decay rate of the dynamic change term; log is the natural logarithm, used to calculate the information entropy;
[0093] p i =deg(i) / ∑deg(i), calculate the degree centrality weight (for example, the degree centrality of node N1 is 0.12).
[0094] Δt = 5 seconds (topology structure change interval), set α = 0.7, β = 0.3, H topo = 2.35, indicating that the topology dynamics is high.
[0095] Multi-scale attention prediction:
[0096] After fusing the spatiotemporal features, it is predicted that the CPU demand peak value will reach 82% and the memory demand peak value will reach 76% in the next 1 hour.
[0097] 2. Human-machine collaborative dynamic weight allocation
[0098] Weight calculation:
[0099] According to the formula Where, μ AI , μ HThe initial preset AI suggestion and the benchmark weight of manual decision; k is the rate coefficient of weight growth over time; T is the system running time; Var(H) is the variance of the historical record of manual decision, used to reflect the volatility of manual decision; ε is the adjustment coefficient, used to balance the influence of variance on weight.
[0100] Set μ AI = 0.8, μ H = 0.2, k = 0.1, ε = 0.05, system running time T = 6 hours, and manual decision variance Var(H) = 0.12.
[0101] The calculation result is that is, the weight of AI suggestion accounts for 73%.
[0102] Scheme generation:
[0103] AI suggestion adds 15 nodes, and manual adjustment according to cost limitation adjusts to 12 nodes, finally generating a scheduling scheme that takes into account delay and cost.
[0104] 3. Active health assessment and resource scheduling
[0105] Health calculation:
[0106] The double-channel formula
[0107] is used, where ω1 is the initial weight coefficient of health; λ is the time decay coefficient, controlling the growth rate of weight over time; t is the continuous running time of the system; SLA i is the actual value of the i-th service level agreement (SLA) of node j; SLA max is the theoretical maximum value of SLA; R(j) ready , R(j) total is the prepared resource amount and total resource amount of node j; d is the contribution coefficient of resource preparation degree to health; ⊙ is Hadamard product (element-wise multiplication), indicating the fusion of two parts after independent calculation;
[0108] The SLA compliance rate of node N5 is 0.92, R ready / R total = 0.15, set ω1 = 0.6, λ j = 0.2, d = 0.4, and the calculation result is H j = 0.55 (lower than the threshold value 0.6), which is marked as a fragile node.
[0109] Fault isolation and migration:
[0110] According to the beacon model, the communication link of node N5 is blocked, and its tasks are migrated to healthy node N20 (H j = 0.85) according to priority. The migration process takes 8 seconds, and the service is uninterrupted.
[0111] 4. Knowledge graph association rule generation
[0112] Node association probability calculation:
[0113] Using the formula
[0114] Where, R i , R j is the resource vector of node i and j; ||R i ||, ||R j || is the length of the resource vector, indicating the total amount of resources; Hop i,j is the topological hop number of node i to j; h j is the health degree of node j; α, β, γ are weight parameters, adjusting the contribution proportion of different factors;
[0115] The Hop i,j of node N20 and N5 is 2, the resource similarity H j = 0.85, set α = 0.5, β = 0.3, γ = 0.2, calculate P i,j = 0.91, indicating that they are highly associated.
[0116] In the high concurrency scene of the "Double 11" cloud computing platform, the application accurately predicts resource demand through the space-time prediction formula (H topo ), combines the dynamic weight model (ω AI = 0.73) to realize the efficient cooperation of AI and artificial, reduces the task delay; based on the health degree formula (H j = 0.55) to actively isolate fragile nodes and migrate tasks, at the same time, through the association probability formula (P i,j = 0.91) to optimize the strategy library, the key order processing SLA compliance rate is improved, which significantly guarantees the service continuity and resource utilization.
[0117] The above shows and describes the basic principles and main features of the application and the advantages of the application. Those skilled in the art should understand that the application is not limited to the above examples, and the above examples and descriptions in the specification are only to illustrate the principles of the application. Without departing from the spirit and scope of the application, various changes and improvements can be made to the application, and these changes and improvements all fall within the scope of the claimed application. The scope of protection of the application is defined by the appended claims and their equivalents.
Claims
1. A dispatch task system based on big data analysis, characterized in that, The method comprises the following modules: A space-time prediction module: real-time collection of node historical logs and topology data, combination of graph convolution network and time series analysis to extract space-time features, use of a multi-scale attention mechanism to predict resource demand curves and node vulnerability scores to provide prospective decision-making basis for the system; A human-machine decision coordination module: based on a fuzzy logic dynamic balance algorithm to suggest and manually instruct weight, use of semantic analysis and constraint optimization solver to generate a multi-objective scheduling scheme, and through a visual rule engine to output a decision logic annotated instruction set to realize human-machine collaborative optimization; An active resource scheduling module: based on a health assessment matrix to construct a resource portrait, use of a propagation tree model to simulate the fault diffusion path in the topology, and when the risk exceeds the threshold, automatically trigger the software-defined network isolation strategy to dynamically allocate backup resources to ensure service continuity; A knowledge management module: fusion of artificial operation, equipment procedure and algorithm log to construct a knowledge graph, through formal verification to ensure the consistency of new rule logic, combination of reinforcement learning dynamic iteration strategy library to form a multi-layer intelligent decision rule system with credibility classification.
2. The system of claim 1, wherein, The space-time prediction module specifically comprises: A data acquisition unit: Apache Kafka is used to acquire historical task logs, resource consumption time series data and device topology connection relationships of distributed nodes in real time; A space-time feature extraction unit: based on a graph convolution network to construct a topology space feature matrix, coupled with time series analysis to extract resource fluctuation periodicity features; A prediction model operation unit: multi-scale attention mechanism is used to fuse space-time features to output resource demand prediction curves and node vulnerability scores.
3. The system of claim 2, wherein, The space-time feature extraction unit performs asymmetric graph convolution operation according to the following formula: H topo = -∑p i logp i + α(1 - e -βΔt ) where p i = deg(i) is the contact weight of node i, which is calculated by the degree centrality (i.e., the number of connections) of the node; Δt is the time interval (in seconds) during which the topology changes; α, β are dynamic state parameters, whose values range from (0, 1), and are used to adjust the weight and decay rate of the dynamic change term; log is the natural logarithm, which is used to calculate the information entropy.
4. The system of claim 1, wherein, The human-machine decision coordination module specifically comprises: An instruction analysis unit: BERT model is integrated to realize operation semantic analysis, receives prediction results and manual instructions, and analyzes operation intent through a semantic understanding model; A weight distribution unit: based on a fuzzy logic algorithm to dynamically adjust the decision weight of AI suggestions and manual intervention; Specifically, the weight distribution unit calculates the dynamic weight according to the following formula: wherein μ AI , μ H are the initial preset reference weights of the AI recommendation and the manual decision; k is a rate coefficient of the weight growth over time; T is the system running time; Var(H) is the variance of the historical record of the manual decision, used to reflect the volatility of the manual decision; and ε is an adjustment coefficient, used to balance the influence of the variance on the weight. A strategy generation unit: a constraint satisfaction problem solver is used to generate a multi-objective scheduling scheme.
5. The system of claim 1, wherein, The active resource scheduling module specifically comprises: A health assessment unit: performs double-channel health calculation according to the following formula: Where ω1 is the initial weight coefficient for health; λ is the time decay coefficient, controlling the rate at which the weight increases over time; t is the continuous running time of the system; SLA i Let $i$ be the actual value of the Service Level Agreement (SLA) for node $j$. max R(j) represents the theoretical maximum value of SLA. ready ,R(j) total represents the prepared resources and total resources of node j; d represents the contribution coefficient of resource preparedness to health; ⊙ represents the Hadamard product (element-by-element multiplication), indicating that the two parts are calculated independently and then merged. A fault path analysis unit: constructs a probabilistic back propagation tree: where R i is the resource vector of node i and j; ||R j is the resource vector of node i and j; ||R i is the resource vector of node i and j; ||R j is the resource vector of node i and j; Hop i,j is the topology hop number from node i to j; h j is the health degree of node j; α, β, γ are weight parameters, adjusting the contribution proportion of different factors.
6. The system of claim 1, wherein, The knowledge management module specifically comprises: A heterogeneous knowledge fusion unit: a knowledge graph containing artificial operation records, equipment procedures and algorithm logs is constructed; A rule verification unit: formal verification method is used to detect the consistency of decision rule logic.
7. A method for scheduling tasks based on big data analysis, applied to the system of any one of claims 1-6, characterized in that, The specific steps are as follows: S1. Space-time prediction and resource demand modeling: Through Apache Kafka, real-time collection of node historical task logs, resource consumption data and topology connection states to construct a space-time feature matrix; use of a graph convolution network to extract topology structure features, combination of time series analysis to capture periodicity, and through a multi-scale attention mechanism to fuse space-time features to output resource demand prediction results; S2. Human-machine collaborative decision scheduling: The user operation instruction and the AI generated scheduling suggestion are analyzed, the dynamic weight distribution model is used to balance the weight of manual decision and AI suggestion, the multi-objective scheduling scheme is generated by using the constraint optimization solver, and the decision options are fed back through the visual interface, and the manual correction instruction is received; S3. Active resource dynamic allocation: According to the node health degree evaluation result, the resource scheduling graph is constructed, the health degree is fused by the double channel calculation model, the service level agreement (SLA) compliance rate and the resource preparation degree, when the health degree is lower than the threshold value, the resource reallocation strategy is triggered to guarantee the service continuity based on the beacon architecture model to block the fault propagation path in reverse; S4. Knowledge-driven decision optimization: The knowledge graph is constructed by combining artificial operation records, equipment procedures and algorithm logs, and the consistency of formal constraint verification rules is verified; Based on reinforcement learning, the strategy library is dynamically updated, multi-layer decision rules with credibility classification are generated, and are fed back to step S2 to optimize the weight distribution model.
8. The method of claim 7, wherein, In step S3, the construction of the resource scheduling graph includes the following sub-steps: S31. Based on the node health degree evaluation result, the nodes with health degree lower than the preset threshold value are marked as fragile nodes, and the fragility heat map is generated; S32. The fault diffusion path is simulated by the beacon architecture model, the virtual isolation barrier is inserted between the fragile node and its adjacent node, and the fault propagation link is blocked; S33. According to the residual resource capacity of each node in the resource scheduling graph and the task queue load, the redundant resources are dynamically allocated to the fragile nodes, and the continuity demand of the key task is preferentially met.
9. The method of claim 7, wherein, In step S4, the construction and rule optimization of the knowledge graph include the following sub-steps: S41. The artificial operation record is semantically annotated, the operation intention and execution context feature are extracted, and the standard operation steps in the equipment procedure are aligned; S42. The rule conflict is detected by the formal verification method, including logical contradiction, resource overallocation and timing violation, the conflict resolution suggestion is generated and fed back to the strategy library; S43. Based on the reinforcement learning model, the historical execution effect of the rules in the strategy library is evaluated, combined with the credibility classification label, the inefficient rules are dynamically eliminated, and the new rules are injected to cover the unforeseen scheduling scenarios.
Citation Information
Cited By
Man-machine fusion scheduling method and system applied to DCS operator station of thermal power plant
CN121209454A
Multi-role cooperation-oriented dynamic evaluation system for managing procurement process
CN121599592A
Cloud native resource dynamic scheduling security isolation method based on cloud computing
CN121984747A