Resource scheduling risk identification system based on artificial intelligence distributed computing

Abstract

The invention discloses a resource scheduling risk identification system based on artificial intelligence distributed computing, which relates to the field of intelligent resource scheduling and comprises a management module serving as a scheduling master control end and used for editing and issuing a control instruction; the resource analysis unit is used for registering current available resources, performing classification marking and analyzing supply demand indexes of the current scheduling task; the correlation calculation module is used for calculating a correlation coefficient between the marked resource and the current scheduling task supply index; the resource extraction module is used for extracting the marked resources of which the correlation coefficients are higher than a preset threshold value according to the calculation result of the correlation calculation module; according to the method, resource characteristics and environment variables are subjected to fusion analysis through loss risk coefficient calculation, potential fault points are pre-judged, a scheduling path is actively optimized, the robustness of resource allocation in a complex distributed environment is improved, the task interruption rate is reduced, and a scientific basis with real-time performance and sustainability is provided for scheduling decisions.

Description

Technical Field

[0001] This invention relates to the field of intelligent resource scheduling technology, specifically a resource scheduling risk identification system based on artificial intelligence distributed computing. Background Technology

[0002] With the development of cloud computing, edge computing, and high-performance computing, the number of data centers and computing nodes has increased significantly, making management and scheduling more complex. Various applications have significantly different requirements for computing resources. For example, tasks such as machine learning, data mining, and large-scale simulation have different requirements for computing, storage, and network bandwidth. In a distributed computing environment, factors such as network latency, node failure, and resource contention can cause unpredictable resource availability.

[0003] With the advancement of machine learning and artificial intelligence technologies, traditional scheduling algorithms are gradually failing to meet the needs of modern distributed computing. They lack real-time monitoring mechanisms for resource availability fluctuations, leading to changes in resource status, such as sudden node overload or bandwidth drops, which are difficult to respond to in a timely manner. Traditional static resource pool database updates are delayed, which can easily cause scheduling decisions to become disconnected from the actual resource status, leading to the risk of task timeouts or failures. Conventional technologies only focus on basic resource indicators, ignoring dynamic interferences such as network latency, node load peaks and valleys, and hardware operating environment, and lack a coefficient-based evaluation of task execution stability. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In view of the above-mentioned shortcomings of the existing technology, the present invention provides a resource scheduling risk identification system based on artificial intelligence distributed computing, which can effectively solve the problems of the existing technology.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] This invention discloses a resource scheduling risk identification system based on artificial intelligence distributed computing, comprising:

[0009] The management module, acting as the main control terminal for scheduling, is responsible for editing and issuing control commands.

[0010] The resource analysis unit is used to register and classify currently available resources, and analyze the supply and demand indicators of the current scheduling task.

[0011] The correlation calculation module is used to calculate the correlation coefficient between the marked resource and the supply index of the current scheduled task;

[0012] The resource extraction module is used to extract marked resources with a correlation coefficient higher than a preset threshold based on the calculation results of the correlation calculation module.

[0013] The environment acquisition module is used to acquire current network latency, node load, and ambient temperature parameters;

[0014] The feature extraction module is used to extract several key environmental impact features based on the data acquired by the environment acquisition module.

[0015] The impact quantification module is used to quantify the impact coefficients of various environmental impact characteristics on scheduling tasks;

[0016] The risk calculation module is used to calculate the loss risk coefficient under the current scheduling task based on the characteristics of environmental impact and its impact coefficient, combined with the characteristics of the candidate resource set.

[0017] Furthermore, the resource analysis unit is equipped with sub-modules, including a resource registration module, a resource tagging module, and an indicator analysis module. The resource tagging module is interconnected with the resource registration module and the indicator analysis module via a wireless network.

[0018] The resource registration module is used to collect the CPU utilization, memory utilization, storage capacity and network bandwidth status of computing nodes in real time. It monitors changes in resource status through a sliding window mechanism and automatically updates the resource pool database when resource availability fluctuations exceed the preset tolerance threshold.

[0019] The resource tagging module is used to classify resources, which are divided into performance tags, risk tags, and task compatibility tags.

[0020] The metrics analysis module is used to extract the core supply metrics of the current scheduled task, and obtain the task's computational complexity, data dependency strength, and deadline constraints.

[0021] Furthermore, the correlation calculation module constructs a demand vector for the current scheduled task supply indicators, including quantified values ​​of task computational complexity, data dependency strength, and deadline constraints. It extracts performance tags, risk tags, and task adaptation tags from the marked resources and calculates the correlation coefficient between the demand vector and the characteristic vector using a cosine similarity algorithm. The calculation formula is as follows:

[0022]

[0023] In the formula, R i T represents the i-th dimension index value of the demand vector. i The i-th dimension of the resource characteristic vector represents the index value, and n represents the total number of vectors.

[0024] Furthermore, when the resource characteristic vector changes dynamically, the correlation calculation module collects real-time data of resource characteristics in units of time windows, calculates the mean and standard deviation of resource indicators within the window, dynamically corrects the characteristic vector, substitutes the corrected characteristic vector into the correlation coefficient calculation, and generates an interference-resistant correlation result.

[0025] Furthermore, the resource extraction module sets a preset threshold based on the correlation coefficient between the marked resources and the scheduling task supply indicators calculated by the correlation calculation module, and extracts marked resources with correlation coefficients higher than the preset threshold through comparison logic to form a candidate resource set; wherein, the preset threshold is dynamically adjusted based on historical scheduling data.

[0026] Furthermore, the quantization process of the influence quantization module includes:

[0027] Collect historical scheduling task data, including task execution time, failure rate, and resource consumption indicators;

[0028] For each environmental impact characteristic, analyze the correlation strength between its characteristic value changes and task performance indicators;

[0029] Based on feature data and task requirement parameters, the relative contribution of this feature to the execution risk of the scheduled task is obtained.

[0030] Furthermore, the risk calculation module comprehensively evaluates the impact weight of each feature based on the key features extracted from the environmental impact features and their impact coefficients. The environmental impact features include network latency, node load, and ambient temperature parameters. The contribution value of the corresponding feature to the overall environmental risk of the task is calculated through the quantified impact coefficients.

[0031] Furthermore, when calculating the loss risk coefficient, the risk calculation module introduces a risk assessment mechanism, which divides the environmental impact characteristics into two categories: short-term impact and long-term impact. The short-term impact mainly considers the immediate interference of current network latency and node load on task execution, while the long-term impact focuses on the continuous effect of factors such as ambient temperature on resource stability. By assigning different weights to these two types of impacts and combining the dynamic adjustment capability of the candidate resource set, the potential loss risk coefficient of the current scheduling task at different time scales is comprehensively calculated.

[0032] Furthermore, the risk calculation module is interconnected with a risk warning module via a wireless network. The risk warning module determines whether the potential risk of the current scheduled task exceeds a safety threshold based on the calculation results of the risk calculation module, and triggers a warning mechanism when the threshold is exceeded to notify the scheduling system or administrator to take intervention measures.

[0033] Furthermore, the resource analysis unit is interconnected with the management module and the correlation calculation module via a wireless network; the correlation calculation module is interconnected with the resource extraction module via a wireless network; the environment acquisition module is interconnected with the resource extraction module, the feature extraction module, and the impact quantification module via a wireless network; and the risk calculation module is interconnected with the feature extraction module and the impact quantification module via a wireless network.

[0034] (III) Beneficial Effects

[0035] Compared with known prior art, the technical solution provided by this invention has the following beneficial effects:

[0036] 1. Achieve precise and forward-looking resource scheduling through a multi-level linkage mechanism. Utilize a sliding window to monitor node performance fluctuations in real time, dynamically refresh the resource pool status to ensure data timeliness, and combine artificial intelligence algorithms to calculate the correlation coefficient between resources and tasks. Automatically filter high-matching resources to avoid performance insufficiency risks. Quantify the impact weight of network latency, load, temperature, and other characteristics on tasks to form a multi-dimensional interference assessment model. Through loss risk coefficient calculation, integrate resource characteristics with environmental variables for analysis, predict potential failure points, proactively optimize scheduling paths, improve the robustness of resource allocation in complex distributed environments, and reduce task interruption rates.

[0037] 2. By integrating resource adaptability and environmental disturbance intensity, and adopting a hierarchical weighted strategy, short-term risks focus on the stability of immediate tasks, while long-term risks focus on the degradation of hardware reliability. This provides a scientific basis for scheduling decisions that is both real-time and sustainable, maximizing resource utilization efficiency and extending infrastructure lifespan. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0039] Figure 1 This is a schematic diagram of the framework of the present invention.

[0040] The numbers in the diagram represent: 1. Management module; 2. Resource analysis unit; 21. Resource registration module; 22. Resource marking module; 23. Indicator analysis module; 3. Correlation calculation module; 4. Resource extraction module; 5. Environment acquisition module; 6. Feature extraction module; 7. Impact quantification module; 8. Risk calculation module; 9. Risk warning module. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0042] The present invention will be further described below with reference to embodiments.

[0043] ① Example 1

[0044] This embodiment presents a resource scheduling risk identification system based on artificial intelligence distributed computing, such as... Figure 1 As shown, it includes:

[0045] Management module 1, as the main control terminal for scheduling, is responsible for editing and issuing control commands;

[0046] Resource analysis unit 2 is used to register and classify currently available resources, and analyze the supply and demand indicators of the current scheduling task. Resource analysis unit 2 has sub-modules, including a resource registration module 21, a resource marking module 22, and an indicator analysis module 23. The resource marking module 22 is interconnected with the resource registration module 21 and the indicator analysis module 23 via a wireless network.

[0047] The resource registration module 21 is used to collect the CPU utilization, memory utilization, storage capacity and network bandwidth status of computing nodes in real time. It monitors resource status changes through a sliding window mechanism and automatically updates the resource pool database when resource availability fluctuations exceed the preset tolerance threshold.

[0048] Resource tagging module 22 is used to classify resources into performance tags, risk tags, and task adaptation tags;

[0049] The indicator analysis module 23 is used to extract the core supply indicators of the current scheduled task and obtain the task's computational complexity, data dependency strength, and deadline constraints.

[0050] The correlation calculation module 3 is used to calculate the correlation coefficient between the marked resource and the supply index of the current scheduling task. When the resource characteristic vector changes dynamically, the correlation calculation module 3 collects real-time data of resource characteristics in units of time windows, calculates the mean and standard deviation of resource indicators within the window, dynamically corrects the characteristic vector, substitutes the corrected characteristic vector into the correlation coefficient calculation, and generates an anti-interference correlation result.

[0051] Resource extraction module 4 is used to extract marked resources with a correlation coefficient higher than a preset threshold based on the calculation results of correlation calculation module 3. Resource extraction module 4 sets a preset threshold based on the correlation coefficient between the marked resources and the scheduling task supply index calculated by correlation calculation module 3, and extracts marked resources with a correlation coefficient higher than the preset threshold through comparison logic to form a candidate resource set. The preset threshold is dynamically adjusted based on historical scheduling data.

[0052] Environment acquisition module 5 is used to acquire current network latency, node load, and ambient temperature parameters;

[0053] Feature extraction module 6 is used to extract several key environmental impact features based on data acquired by environment acquisition module 5.

[0054] Impact Quantification Module 7 is used to quantify the impact coefficients of various environmental impact characteristics on scheduling tasks; the quantification process of Impact Quantification Module 7 includes:

[0055] Collect historical scheduling task data, including task execution time, failure rate, and resource consumption indicators; for each environmental impact feature, analyze the correlation strength between its feature value changes and task performance indicators; based on the feature data and task requirement parameters, obtain the relative contribution of the feature to the execution risk of the scheduling task.

[0056] Risk calculation module 8 is used to calculate the loss risk coefficient under the current scheduling task based on the environmental impact characteristics and their impact coefficients, combined with the characteristics of the candidate resource set. Risk calculation module 8 comprehensively evaluates the impact weight of each feature based on the key features extracted from the environmental impact characteristics and their impact coefficients. The environmental impact characteristics include network latency, node load and ambient temperature parameters. The contribution value of the corresponding feature to the overall environmental risk of the task is calculated through the quantified impact coefficient.

[0057] The risk calculation module 8 is connected to the risk warning module 9 via a wireless network. Based on the calculation results of the risk calculation module 8, the risk warning module 9 determines whether the potential risk of the current scheduling task exceeds the safety threshold, and triggers the warning mechanism when the threshold is exceeded, notifying the scheduling system or administrator to take intervention measures.

[0058] Resource analysis unit 2 is connected to management module 1 and correlation calculation module 3 via a wireless network. Correlation calculation module 3 is connected to resource extraction module 4 via a wireless network. Environment acquisition module 5 is connected to resource extraction module 4, feature extraction module 6 and impact quantification module 7 via a wireless network. Risk calculation module 8 is connected to feature extraction module 6 and impact quantification module 7 via a wireless network.

[0059] Compared with existing technologies, this technology uses a dynamic sliding window mechanism to perceive resource fluctuations in real time and automatically update the resource pool. It combines performance, risk, and task adaptation triple tags with multi-dimensional task requirement indicators to achieve anti-interference dynamic calculation of the resource-task correlation coefficient. Based on historical data, it adaptively adjusts thresholds to accurately screen candidate resources. It introduces multi-dimensional parameters such as network latency, node load, and ambient temperature, and quantifies the impact coefficient of environmental characteristics on task failure rate and loss through historical data. Finally, it integrates resource characteristics and environmental risks to generate a dynamic loss risk coefficient and triggers an early warning mechanism. This reduces the cost of manual intervention while improving the robustness and success rate of resource scheduling in a distributed environment.

[0060] ② Example 2

[0061] At other levels, in this embodiment, the association calculation module 3 constructs a demand vector for the current scheduled task supply indicators, including quantified values ​​of task computational complexity, data dependency strength, and deadline constraints. It extracts performance tags, risk tags, and task adaptation tags for the marked resources, and calculates the association coefficient between the demand vector and the characteristic vector using a cosine similarity algorithm. The calculation formula is as follows:

[0062]

[0063] In the formula, R i T represents the i-th dimension index value of the demand vector. i The i-th dimension of the resource characteristic vector represents the index value, and n represents the total number of vectors.

[0064] ③ Example 3

[0065] In this embodiment, when calculating the loss risk coefficient, the risk calculation module 8 introduces a risk assessment mechanism, which divides the environmental impact characteristics into two categories: short-term impact and long-term impact. The short-term impact mainly considers the immediate interference of the current network latency and node load on task execution, while the long-term impact focuses on the continuous effect of factors such as ambient temperature on resource stability. By assigning different weights to these two types of impacts and combining the dynamic adjustment capability of the candidate resource set, the potential loss risk coefficient of the current scheduling task at different time scales is comprehensively calculated.

[0066] Compared with existing technologies, by assigning dynamic weights to short-term influencing factors such as network latency and node load, and long-term influencing factors such as ambient temperature, and combining the dynamic adjustment capability of the candidate resource set, the comprehensive loss coefficient of the task under instantaneous interference and continuous stability risk is accurately quantified. This solves the prediction bias problem caused by the single-factor treatment of environmental factors in traditional methods, thereby improving the risk resistance of distributed scheduling in complex environments.

[0067] In summary, this invention combines resource scheduling, risk assessment, and environmental impact analysis, enabling accurate identification of the matching degree between resources and tasks and dynamic adjustment of the candidate resource set to improve scheduling efficiency. The system fully considers multi-dimensional environmental factors such as network latency, node load, and ambient temperature, and achieves comprehensive prediction and early warning of potential risks of scheduling tasks through quantitative impact coefficients and risk assessment mechanisms.

[0068] By introducing dynamic correction and historical data-driven threshold adjustment mechanisms, the system can adapt to real-time changes in resource characteristics and task requirements, enhancing its anti-interference capabilities and adaptability. The system performs excellently in resource utilization optimization, risk control, and task execution stability, providing technical support for efficient scheduling in a distributed computing environment.

[0069] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A resource scheduling risk identification system based on artificial intelligence distributed computing, characterized in that, include: The management module (1) serves as the main control terminal for scheduling, and is responsible for editing and issuing control commands. The resource analysis unit (2) is used to register the currently available resources, classify and mark them, and analyze the supply and demand indicators of the current scheduling task. The correlation calculation module (3) is used to calculate the correlation coefficient between the marked resource and the supply index of the current scheduling task; The resource extraction module (4) is used to extract marked resources with a correlation coefficient higher than a preset threshold based on the calculation results of the correlation calculation module (3); The environment acquisition module (5) is used to acquire current network latency, node load and ambient temperature parameters; The feature extraction module (6) is used to extract several key environmental impact features based on the data obtained by the environment acquisition module (5). The impact quantification module (7) is used to quantify the impact coefficients of various environmental impact characteristics on scheduling tasks; The risk calculation module (8) is used to calculate the loss risk coefficient under the current scheduling task based on the environmental impact characteristics and their impact coefficients, combined with the characteristics of the candidate resource set.

2. The resource scheduling risk identification system based on artificial intelligence distributed computing according to claim 1, characterized in that, The resource analysis unit (2) has sub-modules deployed below it. The sub-modules include a resource registration module (21), a resource tagging module (22), and an indicator analysis module (23). The resource tagging module (22) is interconnected with the resource registration module (21) and the indicator analysis module (23) via a wireless network. The resource registration module (21) is used to collect the CPU utilization, memory utilization, storage capacity and network bandwidth status of computing nodes in real time. It monitors resource status changes through a sliding window mechanism and automatically updates the resource pool database when the resource availability fluctuation exceeds the preset tolerance threshold. The resource tagging module (22) is used to classify resources into performance tags, risk tags and task adaptation tags; The indicator analysis module (23) is used to extract the core supply indicators of the current scheduling task and obtain the task computational complexity, data dependency strength and deadline constraints.

3. The resource scheduling risk identification system based on artificial intelligence distributed computing according to claim 1, characterized in that, The association calculation module (3) constructs a demand vector for the supply indicators of the current scheduled task, including the quantified values ​​of task computational complexity, data dependency strength, and deadline constraints. It extracts the performance labels, risk labels, and task adaptation labels of the marked resources, and calculates the association coefficient between the demand vector and the characteristic vector using the cosine similarity algorithm. The calculation formula is as follows: In the formula, R i T represents the i-th dimension index value of the demand vector. i The i-th dimension of the resource characteristic vector represents the index value, and n represents the total number of vectors.

4. The resource scheduling risk identification system based on artificial intelligence distributed computing according to claim 1, characterized in that, When the resource characteristic vector changes dynamically, the correlation calculation module (3) collects real-time data of resource characteristics in units of time windows, calculates the mean and standard deviation of resource indicators within the window, dynamically corrects the characteristic vector, substitutes the corrected characteristic vector into the correlation coefficient calculation, and generates an anti-interference correlation result.

5. The resource scheduling risk identification system based on artificial intelligence distributed computing according to claim 1, characterized in that, The resource extraction module (4) sets a preset threshold based on the correlation coefficient between the marked resources and the scheduling task supply index calculated by the correlation calculation module (3), and extracts marked resources with a correlation coefficient higher than the preset threshold through comparison logic to form a candidate resource set; wherein, the preset threshold is dynamically adjusted based on historical scheduling data.

6. The resource scheduling risk identification system based on artificial intelligence distributed computing according to claim 1, characterized in that, The quantification process of the influence quantification module (7) includes: Collect historical scheduling task data, including task execution time, failure rate, and resource consumption indicators; For each environmental impact characteristic, analyze the correlation strength between its characteristic value changes and task performance indicators; Based on feature data and task requirement parameters, the relative contribution of this feature to the execution risk of the scheduled task is obtained.

7. The resource scheduling risk identification system based on artificial intelligence distributed computing according to claim 1, characterized in that, The risk calculation module (8) comprehensively evaluates the impact weight of each feature based on the key features extracted from the environmental impact features and their impact coefficients. The environmental impact features include network latency, node load and environmental temperature parameters. The contribution value of the corresponding feature to the overall environmental risk of the task is calculated by the quantified impact coefficient.

8. The resource scheduling risk identification system based on artificial intelligence distributed computing according to claim 1, characterized in that, When calculating the loss risk coefficient, the risk calculation module (8) introduces a risk assessment mechanism, which divides the environmental impact characteristics into two categories: short-term impact and long-term impact. The short-term impact mainly considers the immediate interference of the current network latency and node load on task execution, while the long-term impact focuses on the continuous effect of factors such as ambient temperature on resource stability. By assigning different weights to these two types of impacts and combining the dynamic adjustment capability of the candidate resource set, the potential loss risk coefficient of the current scheduling task at different time scales is calculated comprehensively.

9. The resource scheduling risk identification system based on artificial intelligence distributed computing according to claim 1, characterized in that, The risk calculation module (8) is connected to the risk warning module (9) via a wireless network. The risk warning module (9) determines whether the potential risk of the current scheduling task exceeds the safety threshold based on the result calculated by the risk calculation module (8), and triggers the warning mechanism when the threshold is exceeded to notify the scheduling system or administrator to take intervention measures.

10. The resource scheduling risk identification system based on artificial intelligence distributed computing according to claim 1, characterized in that, The resource analysis unit (2) is interconnected with the management module (1) and the association calculation module (3) via a wireless network. The association calculation module (3) is interconnected with the resource extraction module (4) via a wireless network. The environment acquisition module (5) is interconnected with the resource extraction module (4), the feature extraction module (6), and the impact quantification module (7) via a wireless network. The risk calculation module (8) is interconnected with the feature extraction module (6) and the impact quantification module (7) via a wireless network.

Images