Calculation method, system and equipment for computing power scheduling factor of deterministic network

Abstract

The invention relates to the technical field of deterministic network application, and discloses a computing power scheduling factor calculation method, system and device for a deterministic network, and the method comprises the steps: filtering original data through employing a dynamic threshold value, obtaining association rule model training data, and training an association rule model in combination with preset support degree and confidence; scoring data are extracted in the association rule model training process; scoring the association rule model trained each time based on the scoring data and outputting a scoring result; and performing data mining on the scoring result through an association rule generated by an FP-Tree algorithm and an adjacency service integration strategy, and forming a final computing power service scheduling factor in combination with an actual computing power management rule. According to the invention, based on the ultra-low network forwarding delay capability of the deterministic network, the scheduling factor of the multi-association service maintained by the system is used, so that the computing power scheduling has a basis, the association service always keeps lower delay, and the service scheduling process is accelerated.

Description

Technical Field

[0001] The present invention relates to the field of deterministic network application technology, and in particular to a method, system and device for calculating a computing power scheduling factor for a deterministic network. Background Art

[0002] In the current information age, with the rapid development of technologies such as cloud computing, big data, and the Internet of Things, the scale and complexity of network services and applications have increased dramatically, and the demand for computing resources has also increased. Traditional computing scheduling methods often rely on static rules or simple threshold settings, which are difficult to adapt to rapidly changing business scenarios and service needs, resulting in uneven resource allocation, low utilization, slow response, and other problems, which seriously affect service quality and user experience. The main problems currently faced by computing management include: (1) High data complexity: The amount of data generated by massive service requests and computing resource status is huge, and the data types are diverse. How to quickly and accurately extract useful information from them has become a major problem. (2) Poor dynamic adaptability: The uncertainty of network environment and business load requires the computing scheduling system to have high flexibility and rapid response capabilities, but traditional methods are unable to do so. (3) Unoptimized resource allocation: There is a lack of effective data analysis methods to accurately predict service needs and optimize resource allocation, resulting in frequent resource waste and over-allocation. (4) Lag in model update: The model training and update cycle is long, and it cannot reflect the latest network status and user behavior patterns in real time, which reduces the timeliness and accuracy of scheduling decisions.

[0003] Therefore, a computing power scheduling method that combines association rule learning and big data analysis came into being. Association rule analysis can dig out the intrinsic connections and laws between service requests from historical data, predict future service demand trends, and thus provide a scientific basis for the intelligent scheduling of computing power. At the same time, the use of big data technology for continuous data cleaning, model training, and result evaluation can ensure the real-time and effectiveness of the scheduling strategy, and improve the response speed and resource utilization efficiency of the overall system. In this context, the method for calculating the computing power scheduling factor that uses association rules to serve a deterministic network proposed in the present invention aims to solve the challenges faced by the above-mentioned computing power scheduling through advanced data analysis methods, and to achieve a more efficient and intelligent resource management solution. Summary of the invention

[0004] In view of the above-mentioned technical deficiencies, the technical problem to be solved by the present invention is to provide a method, system and device for calculating the computing power scheduling factor for a deterministic network. By using association rule calculation combined with big data processing, an intelligent framework is constructed for computing power scheduling in a deterministic network environment, aiming to overcome the technical problems of unbalanced resource allocation, slow response and poor dynamic adaptability.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solution: The present invention provides a method for calculating a computing power scheduling factor for a deterministic network, comprising the following steps:

[0006] S1. Use dynamic thresholds to filter the original data, remove data that does not conform to the scheduling rules, and obtain association rule model training data;

[0007] S2, using the training data, combined with preset support and confidence, to train the association rule model;

[0008] S3, extracting the scoring data of the association rule model from the association rule model training process;

[0009] S4, scoring each trained association rule model based on the scoring data and outputting the scoring result;

[0010] S5, performing data mining on the scoring results by using the association rules generated by the FP-Tree algorithm and the adjacency service integration strategy;

[0011] S6. Based on the scoring results after data mining and the actual computing power management rules, the final computing power service scheduling factor is formed to schedule all computing power in cloud management.

[0012] Furthermore, the step S1 includes the following steps:

[0013] S11, divide all scheduled services into a group of data according to their names within every m minutes, and perform association rule model training once every h hours. The number of input item sets during association rule model training is

[0014] S12, respectively calculate the average value of all service scheduling time A=AVERAGE(S1:Sn) and the standard deviation of a group of service scheduling time S=sqrt(S1:Sn) to obtain the service running time scheduling timeout threshold θ=A±3S; wherein S1:Sn is the first to nth data;

[0015] S13, filter out all sample data whose scheduling time is less than the service running time scheduling timeout threshold θ and save them into Table 1 dataset1.

[0016] Furthermore, the step S2 comprises the following steps:

[0017] S21. Summarize the CPU resource utilization of all computing nodes in the previous scheduling cycle, and calculate the average CPU utilization of all computing nodes;

[0018] S22. Filter out computing power nodes that are greater than the average value, and calculate the proportion of the number of computing power nodes to the total number of nodes.

[0019] Furthermore, step S3 includes the following steps:

[0020] S31, filtering out the data whose scheduling time is greater than the service running time scheduling timeout threshold θ in all sample data, and extracting them several times using the bag method and saving them in Table 2 dataset2; wherein, each time extracting a set percentage of the data whose scheduling time is greater than the service running time scheduling timeout threshold θ in all sample data;

[0021] S32, calculate the absolute value of the time difference between the called services in Table 2 dataset2, and save it to Table 3 dataset3;

[0022] S33, performing desensitization processing on all service names in the second table dataset2 and table three dataset3, including: replacing each service name with a unique serial number, assigning an initial serial number to the service that appears for the first time, and when new services are added subsequently, the serial numbers are incremented in sequence;

[0023] S34. Group all service running time data according to the namespace and save them as scoring data in Table 4 dataset4.

[0024] Furthermore, the step S5 comprises:

[0025] Analyze the pmml file output by the association rule standard, and analyze the scheduling service object data with associated relationship contained in the recommendation and exclusive recommendation data items in the scoring results of the pmml file;

[0026] If there are multiple associations or recommended objects, then the multiple associations or recommended objects are added to the scheduling list d_service_schedule at the same time;

[0027] If there is only one recommended object A, then locate the previous service of the recommended object A in table 3 dataset3, record it as A-1, and the next service as A+1, and retrieve their detailed service data in the second table dataset2 according to the identification information of A-1 and A+1, and add the three service entities of A-1, recommended object A and A+1 to the scheduling table d_service_schedule;

[0028] The insertion rule of the scheduling table d_service_schedule is that the more associated rule items generated by scoring, the higher the position in the scheduling table d_service_schedule. When the rule items are the same, they are added in the order of data generation.

[0029] Furthermore, the scoring result is also used as part of the input data for the next association rule model training data, and the output data is filtered to filter out the data whose forward support and lift values ​​reach the set threshold, and the data is recorded in the scheduling table d_service_schedule.

[0030] Furthermore, after the association rule model is trained once, the next input parameter is adjusted according to the result of the previous training, including:

[0031] If the proportion of the number of computing power nodes to the total number of nodes is greater than a set threshold, the minimum confidence value for association rule model training is reduced to increase the number of frequent item sets selected through association rule model training and the final computing power service scheduling factor; wherein the frequent item set is one of the data items output from the association rule model training result;

[0032] If the proportion of the number of computing power nodes to the total number of nodes is not greater than the set threshold, no correction is made to the input parameters.

[0033] Furthermore, safety thresholds for evaluating resource utilization are set to continuously track server resource usage;

[0034] If the resource usage is greater than its safety threshold, the service running in the resource and the associated objects of the service in the scheduling factor are scheduled to run on the server resource whose resource usage is less than the safety threshold.

[0035] A computing system for computing power scheduling factors of deterministic networks, including:

[0036] The data preprocessing module is used to filter the original data using a dynamic threshold, remove the data that does not conform to the scheduling rules, and obtain the association rule model training data;

[0037] A model training module, used to train the association rule model using the training data in combination with preset support and confidence;

[0038] The scoring extraction module is used to extract the scoring data of the association rule model from the association rule model training process;

[0039] A model evaluation module scores each trained association rule model based on the scoring data and outputs the scoring result;

[0040] A deep mining module, used for performing data mining on the scoring results through association rules generated by FP-Tree algorithm and adjacency service integration strategy;

[0041] The scheduling strategy generation module forms the final computing power service scheduling factor based on the scoring results after data mining and the actual computing power management rules, and schedules all computing power in cloud management.

[0042] A computing device comprising:

[0043] Memory and processor;

[0044] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for calculating the computing power scheduling factor for a deterministic network are implemented.

[0045] The beneficial effects of the present invention are:

[0046] 1. The present invention dynamically monitors and analyzes the usage of multiple computing resources in cross-regional networks, and based on the low-latency characteristics of deterministic networks, realizes more sophisticated computing power scheduling. Scheduling is no longer triggered only when resources are saturated or service is degraded. Instead, resource allocation is proactively and preventively optimized to ensure that high-load resources are effectively alleviated and low-load resources are fully utilized, avoiding local overload caused by excessive concentration of resources and improving the stability and efficiency of the overall system.

[0047] 2. The present invention realizes more intelligent computing power scheduling decisions through pre-maintained multi-association service scheduling factors, ensuring that the network impact is minimized as much as possible during service migration; this scheduling based on association rules not only responds quickly, but also predictably reduces the risk of business interruption, ensuring business continuity and user experience.

[0048] 3. The present invention combines the low latency advantage of deterministic networks with advanced data-driven scheduling strategies to make service scheduling decisions more accurate and efficient. By analyzing service call patterns and resource occupancy in real time, the system can quickly identify and execute the optimal computing power allocation, shorten service response time, and accelerate business processing. This mechanism is particularly effective in business scenarios with fast dynamic changes and rapid response requirements, improving overall operational efficiency and resource flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flowchart of a method for calculating a computing power scheduling factor for a deterministic network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Example 1

[0053] like Figure 1 As shown, the present invention provides a method for calculating a computing power scheduling factor for a deterministic network, comprising the following steps:

[0054] S1: Use dynamic thresholds to filter the original data, remove data that does not conform to the scheduling rules, and obtain the association rule model training data. It should be noted that:

[0055] S11. Divide all scheduled services into a group of data according to their names within every m minutes, and perform association rule model training every h hours. The number of input item sets during association rule model training is

[0056] S12, respectively calculate the average value of all service scheduling time A=AVERAGE(S1:Sn) and the standard deviation of a group of service scheduling time S=sqrt(S1:Sn) to obtain the service running time scheduling timeout threshold θ=A±3S; wherein S1:Sn is the first to nth data;

[0057] S13, filter all sample data whose scheduling time is less than the service running time scheduling timeout threshold θ and save them into Table 1 dataset1; the data structure in Table 1 dataset1 includes the service belonging namespace, service name, scheduling time, and the vector value of the time interval with the previous service scheduling.

[0058] S2: Use the training data to train the association rule model in combination with the preset support and confidence. For example, set the initial confidence to 0.618 and the support to 0.5, and then adjust it based on the training results with 0.618 as the benchmark. It should be noted that:

[0059] S21. Summarize the CPU resource utilization of all computing nodes in the previous scheduling cycle, and calculate the average CPU utilization of all computing nodes;

[0060] S22. Filter out the computing power nodes that are greater than the average value, and calculate the proportion of the number of computing power nodes to the total number of nodes.

[0061] S3: Extracting the scoring data of the association rule model from the association rule model training process. It should be noted that:

[0062] S31, filtering out the data whose scheduling time is greater than the service running time scheduling timeout threshold θ in all sample data, and extracting them several times using the bag method and saving them to table 2 dataset2; wherein, each time extracting a set percentage of the data whose scheduling time is greater than the service running time scheduling timeout threshold θ in all sample data;

[0063] S32, calculate the absolute value of the time difference between the called services in Table 2 dataset2, and save it to Table 3 dataset3;

[0064] S33, performing desensitization processing on all service names in the second table dataset2 and table three dataset3, including: replacing each service name with a unique serial number, assigning an initial serial number to the service that appears for the first time, and when new services are added subsequently, the serial numbers are incremented in sequence;

[0065] S34. Group all service running time data according to the namespace and save them as scoring data in Table 4 dataset4.

[0066] S4. Score each trained association rule model based on the scoring data and output the scoring result.

[0067] S5. Data mining is performed on the scoring results through the association rules generated by the FP-Tree algorithm and the adjacent service integration strategy. It should be noted that:

[0068] The algorithm referenced by the association rule model involved in this embodiment is the mature big data mining algorithm FP-Tree, and the specific deduction process of the algorithm is not involved in this embodiment.

[0069] Analyze the pmml file output by the association rule standard, and analyze the scheduling service object data with association relationship contained in the recommendation and exclusive Recommendation data items in the scoring results of the pmml file; if multiple association relationships or recommended objects appear, then add multiple association relationships or recommended objects to the scheduling list d_service_schedule at the same time; if there is only one recommended object A, then after locating the previous service of the recommended object A in Table 3 dataset3 and recording it as A-1 and the next service as A+1, retrieve their detailed service data in the second table dataset2 according to the identification information of A-1 and A+1, and add the three service entities of A-1, recommended object A and A+1 found in the query to the scheduling table d_service_schedule; wherein, the insertion rule of the scheduling table d_service_schedule is that the more association rule items generated by the scoring, the earlier the position in the scheduling table, and when the rule items are the same, they are appended in the order of data generation.

[0070] The scoring results are also used as part of the input data for the next association rule model training data, and the output data is filtered to filter out the data whose forward support and lift values ​​reach the set thresholds, and the data is recorded in the scheduling table d_service_schedule.

[0071] S6. Based on the scoring results after data mining and the actual computing power management rules, the final computing power service scheduling factor is formed to schedule all computing power in cloud management.

[0072] Set a safety threshold for evaluating resource utilization, such as setting the CPU utilization rate to 65%, and continuously track server resource usage;

[0073] If the resource usage is greater than its safety threshold, the service running in the resource and the associated objects of the service in the scheduling factor are scheduled to run on the server resource whose resource usage is less than the safety threshold.

[0074] Because the scheduling process is completed in a deterministic network, with the help of the excellent forwarding delay performance of the deterministic network, users have zero perception at the business layer.

[0075] Furthermore, after completing one training of the association rule model, when training again, the input parameters for the next training are adjusted according to the result of the previous training, including:

[0076] If the proportion of computing power nodes to the total number of nodes is greater than the set threshold, the minimum confidence value used for association rule model training is reduced to increase the number of frequent item sets selected through association rule model training and the final computing power service scheduling factors; where the frequent item set is one of the data items output from the association rule model training result;

[0077] If the proportion of computing power nodes to the total number of nodes is not greater than the set threshold, no correction will be made to the input parameters.

[0078] Example 2

[0079] Based on Example 1, this embodiment further provides a calculation system for a computing power scheduling factor of a deterministic network, including:

[0080] The data preprocessing module is used to filter the original data using a dynamic threshold, remove the data that does not conform to the scheduling rules, and obtain the association rule model training data;

[0081] A model training module, used to train the association rule model using the training data in combination with preset support and confidence;

[0082] The scoring extraction module is used to extract the scoring data of the association rule model from the association rule model training process;

[0083] A model evaluation module scores each trained association rule model based on the scoring data and outputs the scoring result;

[0084] A deep mining module, used for performing data mining on the scoring results through association rules generated by FP-Tree algorithm and adjacency service integration strategy;

[0085] The scheduling strategy generation module forms the final computing power service scheduling factor based on the scoring results after data mining and the actual computing power management rules, and schedules all computing power in cloud management.

[0086] Example 3

[0087] Based on Embodiment 1, this embodiment further provides a computing device, including:

[0088] Memory and processor;

[0089] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for calculating the computing power scheduling factor for a deterministic network are implemented.

[0090] Example 4

[0091] Based on Example 1, this embodiment also provides the steps for scheduling computing power in the specific implementation scenario center of the present invention: scheduling the computing power of the cloud native service running in Hangzhou, Nanjing, and Suzhou through the management node in Yangzhou.

[0092] A1. The M server resources in the computer rooms in Nanjing, Hangzhou, and Suzhou are each formed into a sub-computing node, and then these three sub-computing nodes are mounted to the Yangzhou master node to form a computing cloud service;

[0093] A2: Multiple copies of a platform business are deployed in the cloud, so that more than 1,000 services are deployed in the cloud service, and these services belong to 30 different namespaces;

[0094] A3. Perform stress tests of different degrees on services in 10 namespaces for 4 days according to the business process to generate original training data.

[0095] A4. Divide the data of these four days into 10-minute steps, and generate 576 sets of data to generate the model training data table 1 dataset1;

[0096] A6. Extract 3 times from 576 groups of data in a bag method, extract 30% each time, and finally obtain 584 training data. Generate Table 2 dataset2 and calculate the absolute value of the time difference between the called services in Table 2 dataset2, and save it in Table 3 dataset3 to quantify the timing interval characteristics of service calls and provide intuitive difference data in time series for subsequent association rule model training.

[0097] A7. Aggregate the data based on the namespace dimension and calculate the service time difference between the namespace service and the service time before and after the service is used to generate Table 4 dataset4.

[0098] A8. Train the association rule model. The input parameters are dataset1, confidence, and support parameters.

[0099] A9. Score the association rule model. The input parameters are the output data of the trained association rule model itself and the data in Table 4 dataset4.

[0100] A10. Process the scoring results to form the final computing power service scheduling factor;

[0101] A11. Monitor the overall resource utilization of resource nodes in the three locations during service scheduling;

[0102] A12. Adjust the confidence value and support parameter of the next association rule model training according to resource usage;

[0103] A13. Randomly select 10 command spaces and execute A3 to A12.

[0104] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for calculating a computing power scheduling factor for a deterministic network, characterized in that: The following steps are involved: S1. Use dynamic thresholds to filter the original data, remove data that does not conform to the scheduling rules, and obtain association rule model training data; S2, using the training data, combined with preset support and confidence, to train the association rule model; S3, extracting the scoring data of the association rule model from the association rule model training process; S4, scoring each trained association rule model based on the scoring data and outputting the scoring result; S5, performing data mining on the scoring results by using the association rules generated by the FP-Tree algorithm and the adjacency service integration strategy; S6. Based on the scoring results after data mining and the actual computing power management rules, the final computing power service scheduling factor is formed to schedule all computing power in cloud management.

2. The method for calculating the computing power scheduling factor for a deterministic network according to claim 1, characterized in that: The step S1 comprises the following steps: S11, divide all scheduled services into a group of data according to their names within every m minutes, and perform association rule model training once every h hours. The number of input item sets during association rule model training is S12, respectively calculate the average value of all service scheduling time A=AVERAGE(S1:Sn) and the standard deviation of a group of service scheduling time S=sqrt(S1:Sn) to obtain the service running time scheduling timeout threshold θ=A±3S; wherein S1:Sn is the first to nth data; S13, filter out all sample data whose scheduling time is less than the service running time scheduling timeout threshold θ and save them into Table 1 dataset1.

3. The method for calculating the computing power scheduling factor for a deterministic network according to claim 1, characterized in that: The step S2 comprises the following steps: S21. Summarize the CPU resource utilization of all computing nodes in the previous scheduling cycle, and calculate the average CPU utilization of all computing nodes; S22. Filter out computing power nodes that are greater than the average value, and calculate the proportion of the number of computing power nodes to the total number of nodes.

4. The method for calculating the computing power scheduling factor for a deterministic network according to claim 1, characterized in that: The step S3 comprises the following steps: S31, filtering out the data whose scheduling time is greater than the service running time scheduling timeout threshold θ in all sample data, and extracting them several times using the bag method and saving them in Table 2 dataset2; wherein, each time extracting a set percentage of the data whose scheduling time is greater than the service running time scheduling timeout threshold θ in all sample data; S32, calculate the absolute value of the time difference between the called services in Table 2 dataset2, and save it to Table 3 dataset3; S33, performing desensitization processing on all service names in the second table dataset2 and table three dataset3, including: replacing each service name with a unique serial number, assigning an initial serial number to the service that appears for the first time, and when new services are added subsequently, the serial numbers are incremented in sequence; S34. Group all service running time data according to the namespace and save them as scoring data in Table 4 dataset4.

5. The method for calculating the computing power scheduling factor for a deterministic network according to claim 1, characterized in that: The step S5 comprises: Analyze the pmml file output by the association rule standard, and analyze the scheduling service object data with associated relationship contained in the recommendation and exclusive recommendation data items in the scoring results of the pmml file; If there are multiple associations or recommended objects, then the multiple associations or recommended objects are added to the scheduling list d_service_schedule at the same time; If there is only one recommended object A, then locate the previous service of the recommended object A in table 3 dataset3, record it as A-1, and the next service as A+1, and retrieve their detailed service data in the second table dataset2 according to the identification information of A-1 and A+1, and add the three service entities of A-1, recommended object A and A+1 to the scheduling table d_service_schedule; The insertion rule of the scheduling table d_service_schedule is that the more associated rule items generated by scoring, the higher the position in the scheduling table d_service_schedule. When the rule items are the same, they are added in the order of data generation.

6. The method for calculating the computing power scheduling factor for a deterministic network according to claim 1, characterized in that: The scoring results are also used as part of the input data for the next association rule model training data, and the output data is filtered to filter out the data whose forward support and lift values ​​reach the set thresholds, and the data is recorded in the scheduling table d_service_schedule.

7. The method for calculating the computing power scheduling factor for a deterministic network according to claim 3, characterized in that: After completing one training of the association rule model, when training again, the input parameters for the next training are adjusted according to the result of the previous training, including: If the proportion of the number of computing power nodes to the total number of nodes is greater than a set threshold, the minimum confidence value for association rule model training is reduced to increase the number of frequent item sets selected through association rule model training and the final computing power service scheduling factor; wherein the frequent item set is one of the data items output from the association rule model training result; If the proportion of the number of computing power nodes to the total number of nodes is not greater than the set threshold, no correction is made to the input parameters.

8. The method for calculating the computing power scheduling factor for a deterministic network according to claim 1, characterized in that: Set safety thresholds for evaluating resource usage and continuously track server resource usage; If the resource usage is greater than its safety threshold, the service running in the resource and the associated objects of the service in the scheduling factor are scheduled to run on the server resource whose resource usage is less than the safety threshold.

9. A system based on the method for calculating the computing power scheduling factor for a deterministic network according to any one of claims 1 to 8, characterized in that: The data preprocessing module is used to filter the original data using a dynamic threshold, remove the data that does not conform to the scheduling rules, and obtain the association rule model training data; A model training module, used to train the association rule model using the training data in combination with preset support and confidence; The scoring extraction module is used to extract the scoring data of the association rule model from the association rule model training process; A model evaluation module scores each trained association rule model based on the scoring data and outputs the scoring result; A deep mining module, used for performing data mining on the scoring results through association rules generated by FP-Tree algorithm and adjacency service integration strategy; The scheduling strategy generation module forms the final computing power service scheduling factor based on the scoring results after data mining and the actual computing power management rules, and schedules all computing power in cloud management.

10. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for calculating the computing power scheduling factor for a deterministic network described in any one of claims 1 to 8 are implemented.

Images