A method and system for adaptively scheduling computing hardware resources in a cloud-edge collaborative system

Abstract

The present invention provides a computing power hardware resource adaptive scheduling method and system under a cloud-edge collaborative system, by acquiring hardware resource status data; constructing a first combination feature, transmitting the first combination feature to multiple first-domain adaptive algorithms respectively, and obtaining a first algorithm embedded output feature; determining the information coverage of the hardware resource status data in each data group deployed in advance, and constructing a first vacancy indication feature based on the information coverage of the hardware resource status data in each data group and the quantitative features set for each data group; determining the importance parameters of each first-domain adaptive algorithm according to the first vacancy indication feature, performing feature weighted fusion on each first algorithm embedded output feature according to the importance parameter to obtain a first vacancy perception feature, and obtaining a scheduling strategy based on the feature combination result of the first vacancy perception feature and each first algorithm embedded output feature. The present invention can adaptively formulate an accurate scheduling strategy.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for adaptively scheduling computing hardware resources in a cloud-edge collaborative system. Background Art

[0002] With the rapid development of cloud computing and edge computing, cloud-edge collaborative systems have become an important means to improve computing efficiency and resource utilization. In cloud-edge collaborative systems, the scheduling of computing hardware resources is a key link to ensure system performance and stability. However, traditional computing hardware resource scheduling methods often rely on static rules or heuristic algorithms. These methods are difficult to accurately reflect the real-time status and resource requirements of the system when faced with complex and changeable cloud-edge collaborative environments, resulting in low scheduling efficiency and resource waste.

[0003] In order to overcome the limitations of traditional methods, researchers have begun to explore computing hardware resource scheduling methods based on machine learning in recent years. These methods collect and analyze hardware resource status data and use machine learning algorithms to predict and decide scheduling strategies to improve the accuracy and flexibility of scheduling. However, existing scheduling methods based on machine learning still have some shortcomings. Existing methods often ignore the loss of data item values ​​when processing hardware resource status data. In practical applications, due to sensor failures, network delays, data processing errors, etc., there may be problems with the loss of some data item values ​​in the hardware resource status data. The loss of these data item values ​​may cause the machine learning algorithm to be unable to accurately understand the true state of the system, thereby affecting the accuracy and reliability of the scheduling strategy. Summary of the invention

[0004] In view of this, the present invention provides a method and system for adaptively scheduling computing hardware resources in a cloud-edge collaborative system. The technical solution of the present invention is implemented as follows:

[0005] On the one hand, the present invention provides a method for adaptive scheduling of computing power hardware resources under a cloud-edge collaborative system, the method comprising: obtaining hardware resource status data of a target cloud-edge collaborative system, the hardware resource status data comprising data item values ​​of the target cloud-edge collaborative system corresponding to a plurality of reference data respectively, the plurality of reference data comprising a plurality of hardware resource basic data, a plurality of past hardware resource scheduling statistical data and a plurality of past hardware resource scheduling node flow data, wherein the past hardware resource scheduling statistical data comprises scheduling parameters of the target cloud-edge collaborative system in a plurality of computing power hardware resources, the past hardware resource scheduling node flow data being used to represent a computing power hardware resource scheduling event stream composed of scheduling events of a preset number of times of the target cloud-edge collaborative system in the plurality of computing power hardware resources before a current time node; quantizing the data item values ​​corresponding to each hardware resource basic data and the past hardware resource scheduling statistical data to obtain a first quantitative feature, quantizing the sequence data item values ​​corresponding to each past hardware resource scheduling node flow data to obtain a second quantitative feature, and based on each first quantitative feature and Each second quantitative feature obtains a first combination feature; the first combination feature is transmitted to multiple first domain adaptive algorithms respectively, and each first domain adaptive algorithm is used to perform feature embedding to obtain the first algorithm embedded output feature output by each first domain adaptive algorithm; the information coverage of the hardware resource status data in each data group deployed in advance is determined, and based on the information coverage of the hardware resource status data in each data group and the quantitative features set for each data group, a first vacancy indication feature of the hardware resource status data is constructed, and the first vacancy indication feature is used to mark the missing data of the data item value corresponding to the control data in the hardware resource status data; the importance parameter of each first domain adaptive algorithm is determined according to the first vacancy indication feature, and the first algorithm embedded output feature is weightedly fused according to the importance parameter to obtain the first vacancy perception feature, and based on the feature combination result of the first vacancy perception feature and the first algorithm embedded output feature, the inferred scheduling strategy for the target computing power hardware resources under the target cloud-edge collaborative system is obtained.

[0006] In a second aspect, the present invention provides a computer system, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps in the above-described method when executing the program.

[0007] The present invention obtains the hardware resource status data of the target cloud-edge collaborative system, quantifies the data item values ​​corresponding to each hardware resource basic data and the past hardware resource scheduling statistical data to obtain the first quantitative feature, and quantifies the sequence data item values ​​corresponding to each past hardware resource scheduling node flow data to obtain the second quantitative feature; then, a first combination feature is constructed based on each first quantitative feature and each second quantitative feature; then, each first-domain adaptive algorithm is used to perform feature embedding to obtain the first algorithm embedded output feature corresponding to the first combination feature, and based on the information coverage of the hardware resource status data in each data group and the quantitative features set for each data group, a first vacancy indication feature of the hardware resource status data is constructed, so that the loss information of the data item value is directly presented to the algorithm, and the importance parameters of each first-domain adaptive algorithm are determined according to the first vacancy indication feature, and the first algorithm embedded output features are weightedly fused according to the importance parameters to obtain the first vacancy perception feature, and at the same time, based on the feature combination result of the first vacancy perception feature and each algorithm embedded output feature, a scheduling strategy for the target computing power hardware resources under the target cloud-edge collaborative system is obtained, and the algorithm can have adaptive perception of lost information, which is convenient for formulating more accurate scheduling strategies.

[0008] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present invention and, together with the specification, are used to explain the technical solutions of the present invention.

[0010] Figure 1 A schematic diagram of the implementation flow of a method for adaptively scheduling computing hardware resources in a cloud-edge collaborative system provided in an embodiment of the present invention.

[0011] Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0013] The embodiment of the present invention provides a method for adaptively scheduling computing hardware resources in a cloud-edge collaborative system, which can be executed by a processor of a computer system, wherein the computer system may refer to a device with data processing capabilities such as a server, a laptop, a tablet computer, and a desktop computer.

[0014] Figure 1 A schematic diagram of the implementation flow of a method for adaptively scheduling computing hardware resources in a cloud-edge collaborative system provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0015] Step S110: Obtain the hardware resource status data of the target cloud-edge collaborative system, the hardware resource status data including data item values ​​of the target cloud-edge collaborative system corresponding to multiple reference data respectively, the multiple reference data including multiple hardware resource basic data, multiple past hardware resource scheduling statistical data and multiple past hardware resource scheduling node flow data, wherein the past hardware resource scheduling statistical data include the scheduling parameters of the target cloud-edge collaborative system in multiple computing power hardware resources, and the past hardware resource scheduling node flow data is used to represent the computing power hardware resource scheduling event flow composed of a preset number of scheduling events of the target cloud-edge collaborative system in multiple computing power hardware resources before the current time node.

[0016] In the cloud-edge collaborative system, hardware resources may be distributed in the cloud and the edge, including servers, storage devices, network devices, etc. The status data of these resources changes dynamically, reflecting information such as resource availability, load conditions, and performance. In order to accurately obtain this status data, the computer system needs to design a data collection mechanism that can collect relevant information from each resource node in real time or on a scheduled basis.

[0017] Specifically, the hardware resource status data includes data item values ​​of multiple reference data. These reference data can be divided into three categories: hardware resource basic data, past hardware resource scheduling statistics, and past hardware resource scheduling node flow data. Each type of data provides a different perspective on the hardware resource status, and together constitutes a comprehensive description of the current state of the system.

[0018] Hardware resource basic data is the most basic information, which describes the basic properties and current status of hardware resources. For example, for a server, basic data may include CPU model, memory capacity, current CPU usage, memory occupancy, disk usage, etc. These data item values ​​directly reflect the hardware configuration and current operating status of the server, and are an important basis for scheduling decisions.

[0019] Past hardware resource scheduling statistics record the system's scheduling of hardware resources over the past period of time. These data include the scheduling parameters of the target cloud-edge collaborative system in multiple computing hardware resources, such as scheduling time, scheduling frequency, scheduling task type, resource allocation, etc. By analyzing this data, the computer system can understand the system's past scheduling behavior patterns and discover potential scheduling rules and problems. For example, if a server is frequently assigned a large number of computing tasks, causing its CPU usage to be high for a long time, then the system may consider reducing the task allocation to the server in future scheduling to balance the load.

[0020] Past hardware resource scheduling node stream data provides more detailed time series information. It represents a computing power hardware resource scheduling event stream composed of a preset number of scheduling events in multiple computing power hardware resources based on the target cloud-edge collaborative system before the current time node. Simply put, it is to arrange past scheduling events in chronological order to form an event stream. This event stream not only contains the specific information of each scheduling event (such as event type, occurrence time, involved resources, scheduling parameters, etc.), but also implies the temporal relationship and causal relationship between events. By analyzing this event stream, the computer system can dig out the correlation and trend between scheduling events, and provide deeper insights for future scheduling decisions.

[0021] For example, suppose the system has recorded 100 scheduling events in the past week, involving 5 different servers. The system can sort these events by the time they occurred to form an event stream. By analyzing this event stream, the system may find that some servers are often assigned compute-intensive tasks during a specific time period, while other servers are relatively idle. Based on this discovery, the system can try to assign more computing tasks to idle servers in future scheduling to improve resource utilization and overall system performance.

[0022] In order to obtain the data item values ​​of these control data, the computer system designs and implements a set of data collection and processing processes. First, the system needs to determine which data items need to be collected and define clear data formats and units for them. Secondly, the system needs to develop a data acquisition module that can collect these data from various resource nodes in real time or on a scheduled basis. The acquisition module may need to be integrated with the management system or monitoring tool on the resource node to ensure the accuracy and real-time nature of the data. Finally, the system also needs to design a data storage and processing module to store the collected data and perform necessary data cleaning, conversion, and aggregation operations for subsequent algorithm use.

[0023] During the data collection process, data security and privacy protection need to be considered. Since hardware resource status data may contain sensitive information (such as user data, business data, etc.), the system needs to take appropriate security measures (such as encrypted transmission, access control, etc.) to protect this data from unauthorized access and leakage.

[0024] Step S120: quantify the data item values ​​corresponding to each hardware resource basic data and past hardware resource scheduling statistical data to obtain a first quantitative feature, quantify the sequence data item values ​​corresponding to each past hardware resource scheduling node flow data to obtain a second quantitative feature, and obtain a first combined feature based on each first quantitative feature and each second quantitative feature.

[0025] In step S120, the computer system first processes the hardware resource basic data and the past hardware resource scheduling statistics. These two types of data contain important information about the basic attributes of hardware resources and historical scheduling behaviors, but their original forms may not be suitable for direct use in algorithm analysis. Therefore, the system needs to quantify these data item values ​​and convert them into first quantitative features.

[0026] The specific method of quantization depends on the nature of the data and the requirements of the algorithm. For hardware resource basic data, the system may need to use different quantization methods according to different data items. For example, for the data item of CPU utilization, the system can directly use its value as a quantitative feature, because the utilization itself is a numerical value between 0 and 1, which has a clear mathematical meaning. However, for some non-numeric data items (such as CPU model), the system may need to convert them into numerical features first. This can be achieved by using encoding techniques (such as one-hot encoding, label encoding, etc.) to map each different model to a unique numerical value.

[0027] For past hardware resource scheduling statistics, quantification may be more complicated. These data usually contain multiple scheduling parameters, such as scheduling time, scheduling frequency, resource allocation, etc., each of which may have an impact on scheduling decisions. The system needs to select appropriate quantification methods to extract useful information based on the needs of the algorithm and the nature of the data. For example, for the scheduling time parameter, the system can convert it into a timestamp or relative time (such as the number of days, hours, etc. from the current time) for time series analysis. For the resource allocation parameter, the system may need to normalize according to the type of resource (such as CPU, memory, storage, etc.) and unit (such as the number of cores, number of bytes, etc.) to ensure comparability between different resources.

[0028] After obtaining the first quantitative feature, the computer system then processes the past hardware resource scheduling node flow data. This type of data is time series data, which records the system's scheduling events for hardware resources over a period of time in the past. In order to convert these events into feature representations that can be used for algorithm analysis, the system needs to quantify the sequence data item values ​​to generate a second quantitative feature. Quantitative processing of sequence data usually involves two steps: feature extraction and feature representation. Feature extraction refers to extracting information from the original sequence data that is useful for scheduling decisions, such as the type of event, time of occurrence, resources involved, etc. Feature representation refers to converting this information into quantitative features with clear mathematical meaning.

[0029] In the feature extraction stage, the system may need to use various techniques to parse the sequence data. For example, for dispatch logs in text form, the system can use natural language processing techniques (such as word segmentation, part-of-speech tagging, named entity recognition, etc.) to extract event information. For structured data (such as database records), the system can directly query the relevant fields to obtain the required information.

[0030] In the feature representation stage, the system needs to select an appropriate quantification method based on the extracted feature type. For the event type feature, the system can use one-hot encoding or label encoding to convert it into a numerical feature. For the occurrence time feature, the system can use timestamps or relative time to represent it. For the resource-related feature, the system may need to normalize it based on the resource type and unit.

[0031] In addition, since sequence data has temporal order, the system also needs to consider how to represent this order. A common method is to convert sequence data into a fixed-length vector or matrix form, such as using a sliding window technique to split the sequence data into multiple subsequences, and extract and represent features for each subsequence. Another method is to use time series models such as recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) to process sequence data. These models can automatically learn the temporal dependencies of sequence data.

[0032] After obtaining the first quantitative feature and the second quantitative feature, the computer system needs to combine them to generate a first combined feature. The way to combine features depends on the requirements of the algorithm and the nature of the data. One method is to directly concatenate the two types of features into a longer vector or matrix. However, this method may not fully utilize the correlation and complementarity between the features. Therefore, the system may need to use more complex feature fusion technology to generate combined features.

[0033] Feature fusion technology usually includes two steps: feature selection and feature transformation. Feature selection refers to selecting the most useful feature subset for scheduling decisions from the original feature set. This can be achieved by using various feature selection algorithms (such as filtering, wrapping, embedded, etc.). Feature transformation refers to transforming the selected features to generate new combined features. This can be achieved by using linear transformations (such as principal component analysis PCA), nonlinear transformations (such as kernel methods, neural networks, etc.) or combined transformations (such as feature crossover, feature splicing, etc.). For example, in a cloud-edge collaborative system, the system may need to consider the problem of collaborative scheduling between cloud and edge resources. In order to generate effective combined features, the system can first extract quantitative features about cloud and edge resources (such as cloud CPU usage, edge memory occupancy, etc.) from the hardware resource basic data and past hardware resource scheduling statistics, and then use feature selection algorithms to select the most useful feature subset for collaborative scheduling. Then, the system can use feature transformation technology to fuse these features, such as using neural networks to nonlinearly combine cloud and edge resource features to generate combined features that can reflect the collaborative relationship between the two.

[0034] Step S130: transmitting the first combined features to a plurality of first domain adaptive algorithms respectively, and using each first domain adaptive algorithm to perform feature embedding to obtain first algorithm embedded output features output by each first domain adaptive algorithm.

[0035] In step S130, the computer system transmits the first combined features to multiple first domain adaptive algorithms respectively. These domain adaptive algorithms are machine learning models specially designed for processing data in specific fields. They are expert models that can use professional knowledge and experience in the field to optimize model performance. In the cloud-edge collaborative system, these algorithms may include algorithms for cloud resource scheduling, algorithms for edge resource scheduling, algorithms for resource collaborative scheduling, etc. Each algorithm has its own unique network structure and parameter settings to adapt to the characteristics and requirements of data in different fields.

[0036] Once the first combined features are input into the domain adaptation algorithm, the algorithm begins to perform the feature embedding process. Feature embedding is a technique that maps the original feature space to a new feature space. It aims to retain the key information between the original features while removing redundancy and noise, making the new feature representation more compact and effective. In domain adaptation algorithms, feature embedding is usually implemented through deep learning models such as neural networks.

[0037] Taking neural networks as an example, domain adaptation algorithms may contain one or more hidden layers, each of which contains a certain number of neurons. These neurons are connected to input features through weights and biases, and are transformed nonlinearly through activation functions. During the training process, the algorithm will continuously adjust these weights and biases to minimize the loss function, thereby learning the mapping relationship from input features to output features.

[0038] In the feature embedding process, the domain adaptation algorithm uses its own network structure and parameter settings to perform complex transformations and processing on the first combination features. These transformations may include linear transformations, nonlinear transformations, convolution operations, pooling operations, etc., depending on the type and structure of the algorithm. Through these transformations, the algorithm can extract the key information in the first combination features and represent them as the first algorithm embedding output features with more representation and discrimination.

[0039] For example, in a cloud-edge collaborative system, suppose there is a domain adaptive algorithm for cloud resource scheduling. The algorithm may include a multi-layer perceptron (MLP) as a feature embedding module. When the first combined feature is input into the algorithm, the MLP transforms and processes it layer by layer. In the first hidden layer, the algorithm may map the input features to a new feature space through linear transformation and ReLU activation function. In the second hidden layer, the algorithm may further extract key information from the features through nonlinear transformation and pooling operations. Finally, in the output layer, the algorithm generates a fixed-length vector as the first algorithm embedded output feature, which contains the key information of the input feature and is more suitable for subsequent scheduling decision analysis.

[0040] Step S140: Determine the information coverage of the hardware resource status data in each data group deployed in advance, and based on the information coverage of the hardware resource status data in each data group and the quantitative features set for each data group, construct a first vacancy indication feature of the hardware resource status data, the first vacancy indication feature is used to mark the missing data of the data item value corresponding to the control data in the hardware resource status data.

[0041] In step S140, the computer system performs a detailed analysis of the hardware resource status data. These data usually cover data item values ​​of multiple control data, including hardware resource basic data, past hardware resource scheduling statistics, and past hardware resource scheduling node flow data. In order to process these data more effectively, the computer system usually pre-processes them during the data collection stage, such as data cleaning, data conversion, etc., to ensure the accuracy and consistency of the data. Next, the computer system determines the information coverage of the hardware resource status data in each data group deployed in advance. Data grouping is to divide the data into different subsets in advance according to certain rules or standards (such as data type, time range, resource category, etc.). Each data group contains data item values ​​of a specific type or range, and these data item values ​​together constitute a description of the hardware resource status in the group.

[0042] Information coverage refers to the ratio of the number of data item values ​​actually contained in a data group to the number of data item values ​​that should be contained in theory. For example, if a data group should theoretically contain 100 data item values, but in fact only 90 are collected, then the information coverage of the data group is 90%. The level of information coverage directly reflects the integrity and reliability of the data group, which is of great significance for subsequent data analysis and scheduling decisions.

[0043] To calculate information coverage, computer systems typically use the following formula:

[0044] Information coverage = (actual number of data item values / theoretical number of data item values) × 100%. After determining the information coverage of each data group, the computer system also needs to consider the quantitative characteristics of each data group. Quantitative characteristics are the result of quantifying the data item values ​​in the data group, which reflects the statistical characteristics or distribution law of the data in the data group. For example, for a data group containing CPU usage, its quantitative characteristics may include average value, standard deviation, maximum value, minimum value, etc.

[0045] Based on the information coverage and quantitative characteristics of each data group, the computer system can construct a first vacancy indication feature. The first vacancy indication feature is a feature used to mark the loss of data item values ​​corresponding to the control data in the hardware resource status data. It identifies data groups that may have data item value loss by comprehensively analyzing and evaluating the information coverage and quantitative characteristics of each data group, and gives corresponding indications.

[0046] Specifically, the process of constructing the first vacancy indication feature may include: for each data group, calculating its information coverage and comparing it with a preset threshold. If the information coverage is lower than the threshold, it is considered that the data group has a data item value missing. For the data group with a data item value missing, further analyze its quantitative characteristics. By comparing the difference between the actual value of the quantitative feature and the theoretical value (or expected value), evaluate the impact of the data item value loss on the hardware resource status description. According to the situation and impact of the data item value loss, assign a vacancy indication value to each data group. The vacancy indication value can be a numerical value (such as 0 for no missing, 1 for missing), or a vector or matrix (used to indicate the specific situation of the missing). The vacancy indication values ​​of all data groups are summarized and integrated to construct the first vacancy indication feature. This feature can be a vector, matrix or other form of data structure, which is used to comprehensively reflect the data item value loss corresponding to the control data in the hardware resource status data.

[0047] For example, in a cloud-edge collaborative system, suppose there is a data group dedicated to recording the CPU usage of a cloud server. If the computer system finds that the information coverage of the data group is lower than a preset threshold (such as 90%), and its quantitative characteristics (such as mean value, standard deviation, etc.) are significantly different from the theoretical values, then it can be considered that the data group has missing data item values. At this time, the computer system can assign a vacancy indication value (such as 1) to the data group and include it in the first vacancy indication feature.

[0048] By constructing the first missing indication feature, the computer system can more accurately identify missing conditions in the hardware resource status data, thereby providing more comprehensive and accurate information support for subsequent scheduling decisions. This helps to improve the accuracy and efficiency of scheduling decisions and reduce misjudgments and losses caused by missing data.

[0049] Step S150: Determine the importance parameters of each first-domain adaptive algorithm based on the first vacancy indication feature, perform feature weighted fusion on the embedded output features of each first algorithm based on the importance parameters to obtain the first vacancy perception feature, and obtain the inferred scheduling strategy for the target computing power hardware resources under the target cloud-edge collaborative system based on the feature combination result of the first vacancy perception feature and the embedded output features of each first algorithm.

[0050] In step S150, the first vacancy indication feature is one of the core inputs, which marks the loss of data item values ​​corresponding to the control data in the hardware resource status data. The loss of these data item values ​​may have a negative impact on the performance of the algorithm and the accuracy of the scheduling decision. Therefore, the computer system needs to use the first vacancy indication feature to evaluate the importance and reliability of each first domain adaptive algorithm under the current data state.

[0051] Specifically, the computer system will first analyze the first vacancy indication feature to determine which data groups have missing data item values, as well as the degree and scope of the loss. Based on this information, the system can assign an importance parameter to each first-domain adaptive algorithm. This parameter reflects the importance of the algorithm under the current data state and its potential contribution to scheduling decisions. The determination of the importance parameter can be based on a variety of factors, including but not limited to the degree of information coverage of the data group, the change of quantitative characteristics, the historical performance of the algorithm, etc. For example, if the information coverage of a data group is very low, and the quantitative characteristics within the group are significantly different from the theoretical values, then the importance parameter of the algorithm that relies on the group data may be reduced accordingly. On the contrary, if an algorithm has performed well in its past historical performance and the data groups it processes have a high degree of information coverage, then the importance parameter of the algorithm may be increased accordingly.

[0052] Once the importance parameters of each algorithm are determined, the computer system can perform feature weighted fusion on the algorithm embedded output features. Feature weighted fusion is a technique that linearly combines multiple feature vectors or matrices according to certain weights to generate a new feature representation that integrates multiple feature information. In this step, the system assigns corresponding weights to the algorithm embedded output features corresponding to each algorithm according to its importance parameters, and fuses these weighted features to generate the first vacancy perception feature.

[0053] The first vacancy-aware feature is a new feature representation that integrates the embedded output feature information of multiple algorithms. It not only contains the information of the original hardware resource status data, but also incorporates the unique understanding and processing results of each algorithm on the current data status. More importantly, because the system takes into account the situation of missing data item values ​​and adjusts the importance parameters of each algorithm accordingly, the first vacancy-aware feature can better reflect the real state of the data and the reliability of the algorithm, providing more accurate and comprehensive information support for subsequent scheduling decisions.

[0054] After generating the first vacancy-aware feature, the computer system combines it with the algorithm embedding output features. Feature combination is a technique that combines or concatenates multiple feature representations to generate a richer and more comprehensive feature set. In this step, the system combines or concatenates the first vacancy-aware feature with the algorithm embedding output features to generate a new feature set that contains all relevant information.

[0055] Finally, based on this new feature set, the computer system can use various machine learning algorithms or models to infer the scheduling strategy for the target computing power hardware resources under the target cloud-edge collaborative system. These algorithms or models may include decision trees, support vector machines, neural networks, etc., which can automatically learn the mapping relationship from features to scheduling strategies based on the input feature set. During the reasoning process, the system will comprehensively consider multiple factors such as the current state of hardware resources, historical scheduling data, and data item value loss to generate the optimal or near-optimal scheduling strategy.

[0056] For example, in a cloud-edge collaborative system, suppose there is a data group that records the CPU usage of a cloud server, but some data item values ​​are missing. At this point, the computer system may reduce the importance parameters of the algorithms that depend on the data group and increase the importance parameters of other algorithms accordingly. The system then performs weighted fusion of the embedded output features of these algorithms to generate a first vacancy-aware feature. Finally, based on this feature and other relevant information, the system may infer a more balanced and efficient scheduling strategy, such as allocating more computing tasks to other servers or edge devices to reduce the load on the cloud server and improve the performance of the overall system.

[0057] In summary, step S150 determines the importance parameters of each algorithm by using the first vacancy indication feature, and performs weighted fusion and feature combination on the algorithm embedded output features, and finally infers the scheduling strategy for the target computing power hardware resources under the target cloud-edge collaborative system. This step not only improves the accuracy and efficiency of scheduling decisions, but also enhances the system's adaptive perception of data item value loss.

[0058] In one implementation, in step S150, a scheduling strategy for target computing power hardware resources in the target cloud-edge collaborative system is inferred based on a feature combination result of the first vacancy perception feature and each first algorithm embedded output feature, including:

[0059] Step S151: fusing the first vacancy perception feature and each first algorithm embedding output feature to obtain a second combined feature;

[0060] Step S152: transmitting the second combined feature or the third combined feature generated based on the second combined feature to the multi-layer perceptron module to obtain an execution result of the multi-layer perceptron module;

[0061] Step S153: The execution results are transmitted to the logic activation operator, the linear activation operator, and the logarithmic unit activation operator respectively, and the possibility coefficient of the target cloud-edge collaborative system for scheduling, the mean of the scheduling parameters of the target cloud-edge collaborative system, and the discreteness of the scheduling parameters of the target cloud-edge collaborative system are estimated respectively;

[0062] Step S154: Determine the inferred user scheduling strategy for the target computing power hardware resources based on the probability coefficient, mean value and dispersion.

[0063] In step S151, the computer system uses the first vacancy perception feature and each first algorithm embedding output feature as input to perform feature fusion processing. Feature fusion is a technology that combines multiple feature vectors or matrices into a new feature representation, aiming to extract more comprehensive and comprehensive features while retaining the original feature information.

[0064] Specifically, assuming that the first vacancy perception feature is an n-dimensional vector F_vacancy, the first algorithm embedding output features are m k-dimensional vectors F_algo_1, F_algo_2, ..., F_algo_m. In order to perform feature fusion, the computer system can use a variety of methods, such as concatenation, weighted sum, etc.

[0065] Taking the concatenation method as an example, the computer system will concatenate the first vacancy perception feature vector F_vacancy with each algorithm embedding output feature vector F_algo_i (i=1,2,...,m) to form a new (n+k)-dimensional vector. If the weighted summation method is used, it is first necessary to assign a weight w_i (i=1,2,...,m) to each algorithm embedding output feature. These weights can be determined based on the importance parameters of the algorithm. Then, the weighted algorithm embedding output feature vectors are summed and concatenated or added with the first vacancy perception feature vector to obtain the second combined feature.

[0066] For example, suppose F_vacancy = [v1, v2, ..., vn], F_algo_1 = [a11, a12, ...,a1k], F_algo_2 = [a21, a22, ..., a2k], ..., F_algo_m = [am1, am2, ..., amk], and the corresponding weights are w_1, w_2, ..., w_m. Using the weighted summation method, we can get a new vector F_weighted_sum = [w1*a11 + w2*a21 + ... + wm*am1, w1*a12 + w2*a22 + ... + wm*am2, ..., w1*a1k + w2*a2k + ... + wm*amk], and then concatenate or add F_weighted_sum and F_vacancy to get the second combined feature F_combined = [v1, v2, ..., vn, w1*a11 + w2*a21 +... + wm*am1, w1*a12 + w2*a22 + ... + wm*am2, ..., w1*a1k + w2*a2k + ... +wm*amk].

[0067] In step S152, the computer system transmits the second combined feature as input to a multi-layer perceptron (MLP) module. A multi-layer perceptron is a feedforward neural network consisting of multiple layers, including an input layer, a hidden layer, and an output layer. Each layer contains a certain number of neurons, and the neurons are connected by weights and biases. A multi-layer perceptron can classify or regress input data by learning the nonlinear relationship between input and output.

[0068] In the multi-layer perceptron module, the second combined feature is first input to the input layer, and then transmitted and processed layer by layer through the hidden layer. Each layer of neurons receives the output of the previous layer of neurons as input and performs nonlinear transformation through the activation function. Finally, the neurons in the output layer generate the execution results of the multi-layer perceptron.

[0069] In addition to directly using the second combined feature as input, the computer system can further process the second combined feature as needed to generate a third combined feature. For example, the second combined feature can be subjected to dimensionality reduction processing (such as PCA dimensionality reduction), feature selection (such as feature selection based on information gain), or feature transformation (such as feature transformation based on kernel method) to extract more critical and effective feature information. Then, the processed third combined feature is transmitted as input to the multilayer perceptron module.

[0070] The execution result of the multilayer perceptron module is a vector or scalar that reflects the relationship between the input features and the output target. In the cloud-edge collaborative system, this execution result may represent the predicted value or probability distribution of the system's scheduling of the target computing power hardware resources.

[0071] In step S153, the computer system transmits the execution results of the multilayer perceptron module to three different activation operators respectively to estimate the possibility coefficient of scheduling of the target cloud-edge collaborative system, the mean of the scheduling parameters and the discreteness of the scheduling parameters.

[0072] First, the execution result is transmitted to the logical activation operator (such as the logic function). The logic function is a commonly used activation function that maps the input value to the interval (0,1), and the output value can be interpreted as a probability or possibility coefficient. In the cloud-edge collaborative system, this output value can represent the possibility coefficient of the system scheduling the target computing power hardware resources, that is, the probability that the system believes that scheduling will be performed under the current state.

[0073] Secondly, the execution result is transferred to the linear activation operator (such as the identity function), which directly outputs the input value itself. In this scenario, the output can be interpreted as the mean of the scheduling parameters, that is, the average value of the scheduling parameters of the target computing power hardware resources estimated by the system.

[0074] Finally, the execution result is transmitted to the logarithmic unit activation operator (such as a soft positive function), which can map the input value into the range of positive real numbers. In this scenario, the output can be interpreted as the dispersion of the scheduling parameters, such as the standard deviation, which reflects the uncertainty or volatility of the scheduling parameters estimated by the system.

[0075] Through these three activation operators, the computer system can estimate the possibility coefficient of scheduling the target cloud-edge collaborative system, the mean of the scheduling parameters, and the dispersion of the scheduling parameters. These indicators together constitute a comprehensive evaluation of the scheduling strategy.

[0076] In step S154, the computer system determines the inferred user's scheduling strategy for the target computing power hardware resources based on the probability coefficient, mean and dispersion estimated in step S153. This process involves comprehensive analysis and decision-making of the estimation results.

[0077] First, the computer system will evaluate the size of the probability coefficient. If the probability coefficient is high (close to 1), it means that the system believes that the probability of scheduling is high in the current state, so it may tend to formulate a more aggressive scheduling strategy, such as increasing the allocation of computing resources or adjusting the priority of scheduling tasks. On the contrary, if the probability coefficient is low (close to 0), it means that the system believes that the probability of scheduling is low in the current state, so it may tend to formulate a more conservative scheduling strategy, such as maintaining the status quo or reducing the allocation of computing resources.

[0078] Secondly, the computer system will consider the combined effects of mean and dispersion. The mean reflects the average level of the scheduling parameters estimated by the system, while the dispersion reflects the uncertainty or volatility of these estimates. If the mean is high and the dispersion is small, it means that the scheduling parameters estimated by the system are relatively stable and tend to be at a higher level, so a more aggressive scheduling strategy may be formulated. On the contrary, if the mean is low and the dispersion is large, it means that the scheduling parameters estimated by the system are relatively unstable and tend to be at a lower level, so a more conservative scheduling strategy may be formulated.

[0079] Finally, the computer system will consider multiple indicators such as the probability coefficient, mean and dispersion, as well as other factors such as the current hardware resource status, historical scheduling data, user needs, etc., to formulate the final scheduling strategy. This strategy may include the allocation of computing resources, the priority sorting of scheduling tasks, the selection of scheduling time, etc., aiming to maximize the overall performance and resource utilization of the system while meeting user needs.

[0080] In one implementation, before the second combined feature or the third combined feature generated based on the second combined feature is transmitted to the multi-layer perceptron module in step S152, the method further includes:

[0081] Step S152A: transmitting the second combined features to a plurality of second domain adaptive algorithms respectively, and using each second domain adaptive algorithm to perform feature embedding to obtain second algorithm embedded output features output by each second domain adaptive algorithm;

[0082] Step S152B: constructing a second missing indication feature of the hardware resource status data according to the information coverage of the hardware resource status data in each data group and the quantitative feature set for each data group, the second missing indication feature being used to mark missing data about the interaction relationship between the control data in the hardware resource status data;

[0083] Step S152C: determining importance parameters of each second domain adaptive algorithm according to the second vacancy indication feature, and performing feature weighted fusion on each second algorithm embedded output feature according to the importance parameter to obtain a second vacancy perception feature;

[0084] Step S152D: Fusing the second vacancy perception feature and the first vacancy perception feature to obtain a third combined feature.

[0085] In step S152A, the computer system uses the second combined features as input and transmits them to multiple second domain adaptive algorithms respectively. These second domain adaptive algorithms are machine learning models specially designed to process data in a specific field or solve specific problems. They may have different network structures, parameter settings and training strategies. By applying these algorithms to the second combined features, the computer system aims to extract more in-depth and professional feature representations to capture potential patterns and associations in the data.

[0086] Feature embedding is one of the core operations of these second domain adaptation algorithms, which extracts useful information for subsequent tasks by mapping input features to a new feature space. In this process, the algorithm learns a mapping function from the original feature space to the embedded space, which can retain the key information between the original features while removing redundancy and noise. Taking neural networks as an example, each second domain adaptation algorithm may contain one or more hidden layers, each of which contains a certain number of neurons. These neurons are connected to the input features through weights and biases, and are nonlinearly transformed through activation functions. During the training process, the algorithm continuously adjusts these weights and biases to minimize the loss function, thereby learning the mapping relationship from input features to embedded features. Ultimately, each algorithm outputs a second algorithm embedding output feature, which may have different dimensions, ranges, and distribution characteristics, but they all reflect specific aspects or attributes of the input data.

[0087] For example, suppose there is a second domain adaptation algorithm that is specifically designed to process network traffic data. When the algorithm receives the second combined features as input, it may process the features layer by layer through a series of convolutional layers, pooling layers, and fully connected layers. In the convolutional layer, the algorithm uses convolution kernels to extract local patterns in the input features; in the pooling layer, the algorithm downsamples the features to reduce the amount of computation; in the fully connected layer, the algorithm maps the features to a low-dimensional space to generate embedded features. Finally, the algorithm outputs an embedded feature vector that contains key information about the network traffic data.

[0088] In step S152B, the computer system constructs a second vacancy indication feature using the information coverage and quantitative characteristics of the hardware resource status data in each data group. Unlike the first vacancy indication feature, the second vacancy indication feature is intended to mark the loss of the interaction relationship between the control data in the hardware resource status data. These interaction relationships may include dependencies between different hardware resources, time sequence relationships between scheduling tasks, etc., which are crucial for understanding the overall status of the system and formulating effective scheduling strategies.

[0089] In order to construct the second vacancy indication feature, the computer system analyzes the information coverage of the hardware resource status data in each data group. The information coverage reflects the integrity and reliability of the data group, which can be obtained by calculating the ratio of the number of actual data item values ​​to the number of theoretical data item values. Then, the system will combine the quantitative characteristics of each data group (such as average value, standard deviation, maximum value, minimum value, etc.) to evaluate the impact of the loss of data item values ​​on the interaction relationship.

[0090] Specifically, the system uses a graph-based approach to analyze the interactions between the control data. In this graph, each data item value is considered a node, and the association or dependency between data item values ​​is represented as an edge. By traversing this graph and calculating the weight of each edge (the weight can be determined based on indicators such as similarity and correlation between data item values), the system can construct an interaction matrix. The system then adjusts the element values ​​in this matrix based on the degree of information coverage and quantitative characteristics to mark the interactions that may be affected by the loss of data item values.

[0091] For example, suppose there are two data groups that record the CPU usage and memory occupancy of cloud servers respectively. If the system finds that one of the data groups has a low information coverage and the quantitative characteristics are significantly different from the theoretical values, it may believe that the loss of data item values ​​in this group has affected the interaction between CPU usage and memory occupancy. In this case, the system will reduce the weight of the edges related to the two data item values ​​in the interaction relationship matrix accordingly to mark this potential loss of interaction relationship.

[0092] In step S152C, the computer system uses the second vacancy indication feature to determine the importance parameters of each second domain adaptive algorithm, and based on these parameters, weightedly fuses the embedding output features of each algorithm to generate a second vacancy perception feature. The purpose of this step is to take into account the reliability and effectiveness differences of different algorithms in processing specific data states when fusing multiple algorithm embedding output features.

[0093] Specifically, the system will first analyze the loss of interaction relationships marked in the second vacancy indication feature, and use this information to evaluate the importance of each second domain adaptive algorithm in the current data state. If the data grouping on which an algorithm relies has more interaction relationship loss, then the importance parameter of the algorithm may be reduced accordingly; on the contrary, if the data grouping on which an algorithm relies is relatively complete and the quantitative characteristics are consistent with the theoretical value, then the importance parameter of the algorithm may be increased accordingly.

[0094] After determining the importance parameters of each algorithm, the system will perform weighted fusion on the embedded output features of these algorithms. Weighted fusion is a technique that linearly combines multiple feature vectors or matrices according to certain weights, aiming to generate a new feature representation that integrates multiple feature information. In this step, the system will assign corresponding weights to the embedded output features of the corresponding algorithms according to the importance parameters of each algorithm, and fuse these weighted features to generate the second vacancy perception feature.

[0095] For example, suppose there are two second domain adaptive algorithms A and B that output embedded feature vectors F_A and F_B respectively, and their importance parameters are w_A and w_B (w_A + w_B = 1). In this case, the system can generate the second vacancy perception feature F_vacancy_perceived = w_A * F_A + w_B * F_B through weighted fusion. This new feature vector not only contains the information of the original algorithm embedded output features, but also incorporates the system's evaluation results of the importance of each algorithm under the current data state.

[0096] In step S152D, the computer system fuses the second vacancy-aware feature with the previously generated first vacancy-aware feature to generate a third combined feature. The purpose of this step is to integrate and summarize feature information from different processing paths to generate a more comprehensive and rich feature representation.

[0097] In order to perform feature fusion, the computer system can use a variety of methods, such as concatenation, weighted sum, etc. Taking the concatenation method as an example, the system can concatenate the second vacancy-aware feature vector with the first vacancy-aware feature vector to form a new feature vector. This new feature vector not only contains the information of the original hardware resource status data and the information of the algorithm embedded output features, but also incorporates the system's evaluation results of the loss of interactive relationships under the current data state and the evaluation results of the importance of each algorithm. For example, assuming that the first vacancy-aware feature vector is F_vacancy_1 = [v1, v2, ..., vn] and the second vacancy-aware feature vector is F_vacancy_perceived = [p1, p2, ..., pm]. In this case, the system can generate a third combined feature vector F_combined = [v1, v2, ..., vn, p1, p2, ..., pm] by concatenation. This new feature vector contains all relevant information and can be used in subsequent multi-layer perceptron processing or other machine learning tasks.

[0098] In summary, the implementation scheme of step S152 further enhances the capability and richness of feature representation by introducing multiple second domain adaptive algorithms for feature embedding, constructing second vacancy indication features to mark the loss of interactive relationships, performing feature weighted fusion based on importance parameters, and fusing the second vacancy perception features with the first vacancy perception features. These steps together constitute an in-depth analysis and extended processing process of the second combined features, providing more comprehensive and effective input features for subsequent multi-layer perceptron processing or other machine learning tasks.

[0099] In one implementation, in step S120, obtaining a first combined feature based on each first quantitative feature and each second quantitative feature includes:

[0100] Step S121: fusing the first quantitative features into a fourth combined feature, and fusing the second quantitative features into a fifth combined feature;

[0101] Step S122: using a plurality of interaction operators to respectively determine the interactions between the features of the fourth combination feature, and obtaining a first hidden layer array output by each interaction operator;

[0102] Step S123: transmitting the fifth combined feature to the recurrent network operator to obtain a second hidden layer array output by the recurrent network operator;

[0103] Step S124: Merge the first hidden layer array and the second hidden layer array into a first combined feature.

[0104] In step S121, the computer system fuses the first quantitative features and the second quantitative features to generate a higher-level feature representation. The first quantitative features are usually derived from the hardware resource basic data and the past hardware resource scheduling statistics, which reflect the static properties and historical scheduling behavior of the hardware resources. The second quantitative features are derived from the past hardware resource scheduling node flow data, which record the time series information of the scheduling events.

[0105] In order to fuse these features, the computer system can use a variety of methods, such as concatenation, weighted sum, or feature transformation (such as PCA dimensionality reduction), etc. In this implementation, the system chooses to fuse each first quantitative feature into a fourth combined feature, and fuse each second quantitative feature into a fifth combined feature.

[0106] Specifically, suppose there are n first quantitative features, namely F1_1, F1_2, ..., F1_n, which may represent different hardware resource attributes or scheduling parameters. The system can concatenate these features into a long vector F1_combined = [F1_1, F1_2, ..., F1_n] as the fourth combined feature. Similarly, suppose there are m second quantitative features, namely F2_1, F2_2, ..., F2_m, which may represent the status or changes of scheduling events at different time points. The system can also concatenate these features into a long vector F2_combined = [F2_1, F2_2, ..., F2_m] as the fifth combined feature.

[0107] In step S122, the computer system uses multiple interaction operators to explore the interaction between feature items in the fourth combination feature. Interaction between features refers to the possible association, dependence or mutual influence relationship between different feature items, which is crucial to understanding the intrinsic structure and regularity of data. The interaction operator can be any mathematical model or algorithm that can capture the interaction between features, such as a polynomial kernel, a radial basis function kernel, a neural network layer, etc. In this implementation, the system uses multiple different interaction operators to process the fourth combination feature to extract various types of interaction information between features.

[0108] Taking the neural network layer as an example, assume that the system uses a fully connected layer as an interaction operator. This layer contains a certain number of neurons, each of which is connected to each feature item in the fourth combined feature. Through the adjustment of weights and biases, these neurons can learn the nonlinear relationship between feature items and output this relationship in the form of a hidden layer array. Specifically, if the dimension of the fourth combined feature F1_combined is n and the output dimension of the interaction operator is k, then the interaction operator can be represented as an n×k weight matrix W and a k-dimensional bias vector b. By calculating the matrix multiplication of F1_combined and W and adding b, the system can obtain a k-dimensional first hidden layer array H1 = W * F1_combined + b.

[0109] In step S123, the computer system transmits the fifth combined feature to a recurrent network operator to capture the time dependency in the second quantitative feature using its time series processing capability. Recurrent network operators are a type of neural network specially designed to process sequence data, which can capture the time dependency in the sequence through recurrent connections and represent this relationship in the form of hidden layer states.

[0110] Common recurrent network operators include recurrent neural network (RNN), long short-term memory network (LSTM), gated recurrent unit (GRU), etc. In this implementation, the system can select any one of them as a recurrent network operator to process the fifth combination feature.

[0111] Taking LSTM as an example, assume that the fifth combined feature F2_combined is a sequence of length m, in which each element is a feature vector. The LSTM network can be regarded as a sequence model composed of multiple LSTM units, each of which contains components such as input gate, forget gate, output gate and cell state. When processing sequence data, the LSTM network processes the elements in the sequence one by one in chronological order, and controls the flow and forgetting of information through the gating mechanism. Specifically, for the t-th element F2_t in the sequence, the LSTM unit calculates the hidden state ht at the current moment based on the hidden state ht-1 at the previous moment and the input F2_t at the current moment. This process is repeated m times until the entire sequence is processed. Finally, the system can obtain a second hidden layer array H2 = [h1, h2, ..., hm] of length m, in which each element is the hidden state at the corresponding moment.

[0112] It should be noted that since recurrent network operators such as LSTM have time-series dependencies, they need to consider the order and length of the sequence when processing sequence data. In practical applications, the system may need to preprocess the fifth combined feature (such as padding, truncation, or normalization) to ensure that it meets the input requirements of the recurrent network operator.

[0113] In step S124, the computer system fuses the first hidden layer array generated in step S122 and the second hidden layer array generated in step S123 to generate a final first combined feature. The purpose of this step is to integrate and summarize feature information from different processing paths to generate a comprehensive feature representation that includes both static attribute information and temporal dependency information.

[0114] There are many options for fusing the first hidden layer array and the second hidden layer array, such as concatenation, weighted summation, feature transformation, etc. In this implementation, the system chooses to use the concatenation method for fusion processing. Specifically, assuming that the dimension of the first hidden layer array H1 is k, the length of the second hidden layer array H2 is m and the dimension of each element is p (note that p may be different from k here), the system can reshape H2 into a k×m matrix H2_reshaped (if p≠k, a dimension transformation or padding operation is required), and then concatenate H1 and H2_reshaped to form a new (k+k×m)-dimensional vector as the first combined feature F_combined.

[0115] It should be noted that although the concatenation method is simple and direct, it may result in too high a feature dimension. In practical applications, the system may choose other fusion methods, such as weighted summation or feature transformation, according to the specific situation to balance the richness of feature representation and computational efficiency. In addition, if the dimensions of the first hidden layer array and the second hidden layer array are very different or the types of information they represent are very different, the system may also need to perform additional processing or transformation on them to ensure that the fused feature representation is consistent and comparable.

[0116] In summary, the implementation scheme of step S120 generates a first combined feature that is both comprehensive and rich in deep information by fusing the first quantitative feature and the second quantitative feature, using the interactive operator to capture the interaction between features, using the recurrent network operator to capture the temporal dependency, and fusing the feature information of different processing paths. This feature representation not only provides a rich information basis for subsequent processing and analysis, but also helps to improve the performance and effect of the entire scheduling method.

[0117] In one implementation, step S140, determining the information coverage of the hardware resource status data in each data group deployed in advance, and constructing a first vacancy indication feature of the hardware resource status data based on the information coverage of the hardware resource status data in each data group and the quantitative features set for each data group, includes:

[0118] Step S141: for any data group, determine the number of lost data items corresponding to the lost data item value in the corresponding data group, subtract the number of lost data items from the total number of data items in the corresponding data group to obtain the total number of non-lost data items, divide the total number of non-lost data items by the total number of data items to obtain the information coverage of the corresponding data group;

[0119] Step S142: for each data group, determining the corresponding information coverage degree;

[0120] Step S143: for each data group, multiply the corresponding information coverage degree by the set quantitative feature to obtain the adjusted quantitative feature corresponding to each data group, and fuse the adjusted quantitative features corresponding to each data group to obtain the first vacancy indication feature.

[0121] In step S141, the computer system first focuses on any data group, which is a subset divided from the hardware resource status data in advance according to certain rules or standards (such as data type, time range, resource category, etc.). The data group contains data item values ​​of a specific type or range, and these data item values ​​together constitute a description of the hardware resource status in the group.

[0122] In order to evaluate the information coverage of the data group, the system first needs to determine the number of missing data item values ​​in the group. The missing data item values ​​may not be correctly collected or recorded due to various reasons (such as sensor failure, network delay, data processing error, etc.). The system can calculate the number of missing data item values ​​by comparing the theoretical total number of data items (that is, the total number of data item values ​​that the group should contain) with the actual number of data items collected.

[0123] Once the number of lost data item values ​​is determined, the system can calculate the total number of data items that are not lost, that is, the total number of data items minus the number of lost data items. The system then divides the total number of data items that are not lost by the total number of data items to obtain a ratio between 0 and 1, which is the information coverage of the data group. The information coverage reflects the integrity and reliability of the data group. The closer its value is to 1, the higher the integrity of the data group and the fewer the lost data item values; conversely, the closer its value is to 0, the lower the integrity of the data group and the more the lost data item values.

[0124] For example, suppose there is a data group dedicated to recording the CPU usage of cloud servers. In theory, this group should contain 24 data item values ​​of CPU usage per hour (assuming that it is recorded once a day). However, during the actual collection process, the system found that 3 of the data item values ​​were lost due to sensor failure. Therefore, the number of lost data item values ​​is 3, the total number of non-lost data items is 21, and the total number of data items is 24. Substituting these values ​​into the formula, the information coverage is 21 / 24=0.875, or 87.5%.

[0125] Based on step S141, step S142 requires the computer system to perform the same information coverage calculation process for each data group deployed in advance, which means that the system needs to traverse all data groups and determine the information coverage of each group.

[0126] Since each data group may contain data item values ​​of different types or ranges and be affected by different factors (such as different sensor performance, different network environment, etc.), their information coverage may be different. By calculating the information coverage of each data group separately, the system can fully understand the data integrity and reliability of each group, providing more accurate information support for subsequent data analysis and scheduling decisions.

[0127] For example, in a cloud-edge collaborative system, in addition to data groups recording the CPU usage of cloud servers, there may also be multiple data groups recording the memory usage of edge devices, network bandwidth usage, etc. The system needs to calculate the information coverage of these groups one by one to fully understand the integrity and reliability of the hardware resource status data in the entire system.

[0128] In step S143, the computer system further uses the information coverage to construct a first vacancy indication feature. The first vacancy indication feature is a feature used to mark the missing data item value corresponding to the control data in the hardware resource status data, and is generated by comprehensively analyzing and evaluating the information coverage and quantitative features of each data group.

[0129] In this step, the system first sets a quantitative feature for each data group. The quantitative feature is the result of quantifying the data item values ​​in the data group, which reflects the statistical characteristics or distribution law of the data in the data group. For example, for a data group that records CPU usage, its quantitative features may include average usage, standard deviation of usage, maximum usage, etc.

[0130] Then, the system multiplies the information coverage of each data group by its corresponding quantitative feature. This multiplication operation is actually a kind of adjustment or weighting of the quantitative feature, and its purpose is to adjust the importance or influence of the quantitative feature according to the integrity of the data group. Specifically, if a data group has a higher information coverage (that is, better data integrity), then the influence of its quantitative features in subsequent analysis will be relatively large; conversely, if a data group has a lower information coverage (that is, poorer data integrity), then the influence of its quantitative features will be relatively small.

[0131] Through this multiplication operation, the system obtains the adjusted quantitative features corresponding to each data group. These adjusted quantitative features not only contain the information of the original quantitative features, but also incorporate the information coverage factor of the data group, thus more accurately reflecting the actual situation of the data in the data group.

[0132] Finally, the system fuses the adjusted quantization features corresponding to each data group to generate a first vacancy indication feature. The fusion process can be performed in a variety of ways, such as splicing, weighted summation, feature transformation, etc. In this implementation, the system may choose to splice each adjusted quantization feature into a long vector as the first vacancy indication feature. This long vector contains the adjusted quantization feature information of all data groups, and because it has been adjusted for information coverage, it can more accurately reflect the loss of data item values ​​in the hardware resource status data.

[0133] For example, suppose there are two data groups G1 and G2, which record the CPU usage of the cloud server and the memory occupancy of the edge device respectively. After calculation in steps S141 and S142, the information coverage of the two groups is 0.875 and 0.95 respectively. At the same time, quantitative features Q1 and Q2 (such as average usage and standard deviation of usage) are set for each group. In step S143, the information coverage is multiplied by the quantitative feature to obtain the adjusted quantitative features R1=0.875*Q1 and R2=0.95*Q2. Then, R1 and R2 are concatenated into a long vector [R1, R2] as the first vacancy indication feature. This feature not only contains the quantitative information of CPU usage and memory occupancy, but also incorporates their respective data integrity factors, so it can provide more accurate and comprehensive information support for subsequent scheduling decisions.

[0134] In one implementation, step S152B, based on the information coverage of the hardware resource status data in each data group and the quantitative features set for each data group, constructs a second vacancy indication feature of the hardware resource status data, including:

[0135] Step S152B1: for each pair of data groups, based on the information coverage of the hardware resource status data in each data group in the pair of data groups, determine the combined information coverage of the hardware resource status data in the pair of data groups, and based on the quantitative features set for each data group in the pair of data groups, determine the quantitative features of the pair of data groups;

[0136] Step S152B2: Determine the second vacancy indication feature of each pair of data groups based on the combined information coverage and quantitative features of each pair of data groups; fuse the second vacancy indication features of each pair of data groups to obtain the second vacancy indication feature of the hardware resource status data.

[0137] In step S152B1, the computer system focuses on each pair of data packets. Data packets are subsets divided from the hardware resource status data in advance according to certain rules or standards, and they contain data item values ​​of a specific type or range. A pair of data packets refers to a combination of two data packets, which may contain data item values ​​of different types or ranges, but there may be some potential interaction or dependency between them.

[0138] In order to evaluate the combined information coverage of this pair of data groups, the system needs to first determine the information coverage of each data group. The information coverage reflects the integrity and reliability of the data group, which can be obtained by calculating the ratio of the actual number of data item values ​​to the theoretical number of data item values. In step S140, the system has calculated the information coverage of each data group, so these results can be directly used here.

[0139] Next, the system needs to determine the combined information coverage of this pair of data groups. The combined information coverage is intended to reflect the data integrity and reliability of the two data groups in the combined state. Since there may be an interactive relationship or dependency between the two data groups, their combined information coverage may be different from their independent information coverage. In order to calculate the combined information coverage, the system can use a variety of methods, such as the product method, the weighted sum method, etc. In this implementation, the system chooses to use the product method to calculate the combined information coverage, that is, to multiply the information coverage of the two data groups.

[0140] For example, suppose there are two data groups G1 and G2, and their information coverage is C1 and C2 respectively. Then, the combined information coverage C_combined of this pair of data groups G1 and G2 can be obtained by calculating C1*C2. If C1=0.8 (indicating that 80% of the data item values ​​in G1 are correctly collected) and C2=0.9 (indicating that 90% of the data item values ​​in G2 are correctly collected), then C_combined=0.8*0.9=0.72, that is, the combined information coverage is 72%.

[0141] In addition to calculating the combined information coverage, the system also needs to determine the quantitative features of the pair of data groups. The quantitative features are the result of quantizing the data item values ​​in the data group, which reflects the statistical characteristics or distribution law of the data in the data group. In step S140, the system has set quantitative features for each data group. Here, the system needs to combine or transform these quantitative features to generate new quantitative features that reflect the characteristics of the pair of data groups.

[0142] There are many options for combining or transforming quantitative features, such as splicing, weighted summation, feature transformation, etc. In this implementation, the system chooses to use a splicing method to combine quantitative features. Specifically, if the quantitative feature of G1 is Q1 (such as a vector containing multiple statistical indicators), and the quantitative feature of G2 is Q2 (also a vector), then the quantitative feature Q_combined of this pair of data groups G1 and G2 can be obtained by splicing Q1 and Q2. It should be noted that if the dimensions of Q1 and Q2 are different, the system may need to perform additional dimensional transformation or padding operations to ensure that they can be spliced ​​together.

[0143] In step S152B2, the computer system determines the second missing indication feature of each pair of data packets based on the combined information coverage and quantitative features of each pair of data packets. The second missing indication feature is a feature used to mark the loss of data item values ​​in the hardware resource status data, which reflects the impact of potential interactions and dependencies between data packets on data integrity.

[0144] In order to determine the second vacancy indication feature, the system can adopt a variety of methods, such as rule-based methods, machine learning-based methods, etc. In this implementation, the system chooses to use a machine learning-based method to generate the second vacancy indication feature. Specifically, the system can train a machine learning model (such as a neural network, a support vector machine, etc.), using the combined information coverage and quantitative features of each pair of data groups as input features, and using the label reflecting the missing data item value as the output target. Through the training process, the model can learn the mapping relationship from the input features to the output targets, and generate the second vacancy indication feature accordingly.

[0145] For example, suppose the system trains a neural network model to generate a second vacancy indication feature. The model includes an input layer, a hidden layer, and an output layer. In the input layer, the model receives the combined information coverage and quantitative features of each pair of data groups as input features (these features may have been converted into a format suitable for model processing). In the hidden layer, the model performs nonlinear transformation and feature extraction on the input features through a series of neurons and activation functions. In the output layer, the model generates a scalar value or vector representing the missing value of the data item as the second vacancy indication feature. This value or vector may reflect the comprehensive impact of the interaction and dependency between data groups on data integrity.

[0146] It should be noted that since the second vacancy indication feature is generated based on the combined information coverage and quantitative features of each pair of data groups, it may have different dimensions and representations. In order to obtain a unified second vacancy indication feature of the hardware resource status data, the system needs to fuse the second vacancy indication features of each pair of data groups. There are many options for the fusion processing method, such as splicing, weighted summation, feature transformation, etc. In this implementation scheme, the system chooses to use a splicing method to fuse the second vacancy indication feature. Specifically, the system can splice the second vacancy indication features of all pairs of data groups into a long vector or matrix as the final second vacancy indication feature. This feature not only contains the information coverage and quantitative feature information of a single data group, but also incorporates the information of the interaction and dependency between data groups, so it can more comprehensively reflect the possible loss of data item values ​​in the hardware resource status data.

[0147] In one implementation, step S152B1, based on the information coverage of the hardware resource status data in each data group in the data group, determines the combined information coverage of the hardware resource status data in the data group, and based on the quantitative feature set for each data group in the data group, determines the quantitative feature of the data group, including:

[0148] Step S152B11: Calculate the product of the information coverage of the hardware resource status data in each data group in the data grouping, and determine it as the combined information coverage of the hardware resource status data in the data grouping;

[0149] Step S152B12: Determine the quantitative product of the quantitative features set for each data group in the data grouping, and use it as the quantitative feature of the data group.

[0150] Step S152B2, based on the combined information coverage and the quantitative characteristics of each pair of data packets, determining the second vacancy indication characteristics of each pair of data packets, comprises:

[0151] Step S152B21: Calculate the product of the combined information coverage of each pair of data groups and the quantitative feature to obtain the second vacancy indication feature of each pair of data groups.

[0152] In step S152B11, the computer system focuses on a pair of specific data groups, which may contain data item values ​​of different types or ranges, but there may be some potential interactive relationship or dependency between them. In order to evaluate the overall data integrity of this pair of data groups, the system needs to calculate their combined information coverage.

[0153] The combined information coverage is a comprehensive indicator that reflects the data integrity and reliability of the two data groups in the combined state. Since there may be an interaction or dependency between the two data groups, their combined information coverage may be different from their independent information coverage. In order to calculate the combined information coverage, the system uses the product method, that is, multiplying the information coverage of the two data groups.

[0154] Information coverage is a proportional value between 0 and 1, which represents the ratio of the actual number of data item values ​​to the theoretical number of data item values ​​in a data group. For example, suppose there are two data groups G1 and G2, and their information coverage is C1 and C2 (C1, C2 ∈ [0, 1]). Then, the combined information coverage C_combined of this pair of data groups G1 and G2 can be obtained by calculating C1*C2. If C1=0.8 (indicating that 80% of the data item values ​​in G1 are collected correctly) and C2=0.9 (indicating that 90% of the data item values ​​in G2 are collected correctly), then C_combined=0.8*0.9=0.72, that is, the combined information coverage is 72%. This means that in the combined state, only 72% of the data item values ​​are collected correctly, reflecting the possible data loss or incompleteness between data groups.

[0155] In step S152B12, after determining the coverage of the combined information, the system needs to further process the quantitative features of each data group to generate new quantitative features that reflect the characteristics of this pair of data groups. The quantitative features are the results of quantizing the data item values ​​in the data group, which reflect the statistical characteristics or distribution law of the data in the data group.

[0156] In order to generate new quantitative features, the system uses the scalar product method, that is, to perform a dot multiplication operation on the quantitative features of two data groups. Here, it is assumed that the quantitative feature of each data group is a vector, and each element of the vector represents the value of a statistical indicator in the data group. Through the scalar product operation, the system can obtain a scalar value as a new quantitative feature, which reflects the similarity or correlation between the two data groups in the quantitative feature space.

[0157] For example, suppose the quantized feature of data group G1 is Q1 = [q11, q12, ..., q1n], and the quantized feature of data group G2 is Q2 = [q21, q22, ..., q2n] (where n is the dimension of the quantized feature). Then, the quantized feature product Q_combined of the pair of data groups G1 and G2 can be obtained by calculating Q1·Q2, that is, Q_combined =Σ(q1i * q2i) (i=1, 2, ..., n). This quantized feature product value Q_combined, as a new quantized feature, reflects the similarity or correlation between G1 and G2 in the quantized feature space. If the value of Q_combined is large, it means that G1 and G2 have a high similarity or correlation in the quantized features; conversely, if the value of Q_combined is small, it means that G1 and G2 have a large difference in the quantized features.

[0158] In step S152B2, the computer system further calculates a second vacancy indication feature based on the combined information coverage and quantitative features of each pair of data packets. The second vacancy indication feature is a feature used to mark the loss of data item values ​​in the hardware resource status data, which reflects the impact of the potential relationship between data packets on data integrity and reliability.

[0159] In order to calculate the second vacancy indication feature, the system uses the product method, that is, multiplying the combined information coverage and quantitative features of each pair of data groups. This product value contains both the information of the interaction relationship and dependency between data groups (reflected by the combined information coverage), and the information of the statistical characteristics or distribution law of the data item values ​​in the data group (reflected by the quantitative features). Therefore, the second vacancy indication feature can more comprehensively reflect the impact of the potential relationship between data groups on data integrity and reliability.

[0160] For example, suppose there is a pair of data packets G1 and G2, whose combined information coverage is C_combined and the quantitative feature product is Q_combined. Then, the second vacancy indication feature V_vacancy of this pair of data packets can be obtained by calculating C_combined * Q_combined. If the value of C_combined is small (indicating low data integrity) and the value of Q_combined is also small (indicating large differences in quantitative features between data packets), then the value of V_vacancy will be small, reflecting that this pair of data packets has major problems in data integrity and relevance. On the contrary, if the values ​​of C_combined and Q_combined are both large, then the value of V_vacancy will be large, indicating that this pair of data packets performs relatively well in data integrity and relevance.

[0161] In summary, steps S152B1 and S152B2 successfully generate the second vacancy indication feature that reflects the impact of the potential relationship between data groups on data integrity and reliability by calculating the combined information coverage, the quantitative feature quantity product and their product. This process not only deeply explores the interactive relationship and dependency between data groups, but also provides more accurate and comprehensive information support for subsequent scheduling decisions.

[0162] In one implementation, the method is executed by a scheduling strategy estimation algorithm. Before obtaining hardware resource status data of the target cloud-edge collaborative system, the method further includes:

[0163] Step S10: Obtain a learning sample set and a target evaluation sample set. Both the learning sample set and the target evaluation sample set are sample sets including multiple computing power hardware learning samples and training marks corresponding to each computing power hardware learning sample. The computing power hardware learning samples include data item values ​​corresponding to multiple hardware resource basic data under the cloud-edge collaborative system, data item values ​​corresponding to multiple past hardware resource scheduling statistical data, and data item values ​​corresponding to multiple past hardware resource scheduling node flow data. The past hardware resource scheduling statistical data include scheduling parameters in multiple computing power hardware resources. The past hardware resource scheduling node flow data is used to represent a computing power hardware resource scheduling event flow composed of a preset number of scheduling events in multiple computing power hardware resources before the current time node. The training mark is the scheduling parameter in the subsequent set time range of the target computing power hardware resource.

[0164] Step S20: obtaining a scheduling strategy estimation algorithm based on training of a learning sample set;

[0165] Step S30: Evaluate the scheduling strategy prediction algorithm based on the target evaluation sample set, wherein obtaining the hardware resource status data of the target cloud-edge collaborative system is performed during the evaluation process of the scheduling strategy prediction algorithm.

[0166] In step S10, the computer system collects and organizes two sets of sample sets: a learning sample set and a target evaluation sample set. Both sets of sample sets contain multiple computing hardware learning samples, and each learning sample corresponds to a set of training tags. These learning samples and training tags together constitute the basic data for algorithm training and evaluation.

[0167] Computing hardware learning samples are the core input data for algorithm training and evaluation. They contain multiple hardware resource basic data, past hardware resource scheduling statistics, and past hardware resource scheduling node flow data under the cloud-edge collaborative system. The hardware resource basic data reflects the static properties of hardware resources, such as CPU model, memory capacity, etc.; the past hardware resource scheduling statistics record the system's scheduling behavior of hardware resources in the past period of time, such as scheduling time, scheduling frequency, scheduling task type, etc.; the past hardware resource scheduling node flow data is a time series data, which records the scheduling event sequence that occurred in the system in the past period of time, such as the computing power resource allocation at a certain point in time.

[0168] The training mark is the supervision information in the algorithm training process, which indicates the scheduling strategy or scheduling parameters that the system expects to obtain given the learning sample. In this implementation, the training mark is set as the scheduling parameters of the target computing power hardware resources in the subsequent set time range, such as CPU usage, memory occupancy, etc. These scheduling parameters reflect the system's demand for hardware resources and expected status.

[0169] The learning sample set and the target evaluation sample set differ in function and purpose. The learning sample set is mainly used in the algorithm training process. It helps the algorithm learn the mapping relationship from input features to output scheduling parameters by providing a large number of learning samples and training labels. The target evaluation sample set is used in the algorithm evaluation process. It provides a set of independent samples to test the algorithm's generalization ability and performance on unseen data.

[0170] In step S20, the computer system uses the learning sample set to train the scheduling strategy prediction algorithm. This process usually involves multiple sub-steps, including data preprocessing, model selection, parameter tuning, and model training.

[0171] Data preprocessing is a necessary step before algorithm training. It aims to clean, transform and normalize the original learning samples to improve the quality and consistency of the data. For example, for basic hardware resource data, the system may need to perform operations such as missing value filling, outlier detection and processing; for past hardware resource scheduling statistics and scheduling node flow data, the system may need to perform time series analysis, feature extraction and dimensionality reduction.

[0172] Model selection is a key step in algorithm training, which involves selecting a suitable machine learning model or neural network structure to fit the relationship between input features and output scheduling parameters. In this implementation, the system may select a model that can process time series data and capture complex relationships between features, such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or gated recurrent units (GRUs). These models capture the temporal dependencies in time series data by introducing recurrent connections or gating mechanisms, and extract and fuse useful information from input features through multi-layer network structures.

[0173] Parameter tuning is another important part of algorithm training, which aims to optimize the performance of the algorithm by adjusting the model's hyperparameters. Hyperparameters are parameters that need to be set in advance before model training, such as learning rate, batch size, number of iterations, etc. The system can find the optimal hyperparameter combination through methods such as cross-validation, grid search, or random search.

[0174] Model training is the core step of algorithm training. It iteratively updates model parameters to minimize the loss function, so that the model can accurately predict the output scheduling parameters. During the training process, the system inputs the input features and training tags in the learning sample set into the model, and updates the model parameters such as weights and biases through the back propagation algorithm. As the training progresses, the prediction performance of the model will gradually improve until it converges to a stable state.

[0175] In step S30, the computer system uses the target evaluation sample set to evaluate the performance of the scheduling strategy prediction algorithm. This process usually involves multiple evaluation indicators and evaluation methods to comprehensively measure the generalization ability and performance of the algorithm on unseen data.

[0176] Evaluation indicators are important criteria for measuring algorithm performance, which usually include accuracy, recall, F1 score, mean square error (MSE), root mean square error (RMSE), etc. In this implementation, the system may select indicators such as accuracy or mean square error to evaluate the performance of the algorithm in predicting output scheduling parameters. These indicators can intuitively reflect the accuracy and stability of the algorithm prediction results. Evaluation methods are specific means for algorithm evaluation, which usually include cross-validation, retention method, bootstrap method, etc. In this implementation, the system may select retention method as the evaluation method, that is, randomly divide the target evaluation sample set into a training set and a test set, where the training set is used for the algorithm training process (although the training has been completed in step S20), and the test set is used for the algorithm evaluation process. By calculating the evaluation indicator value on the test set, the system can objectively evaluate the generalization ability and performance of the algorithm on unseen data.

[0177] It should be noted that in step S30, the hardware resource status data of the target cloud-edge collaborative system is obtained during the evaluation process of the scheduling strategy prediction algorithm. This means that in the algorithm evaluation phase, the system needs to collect hardware resource status data from the cloud-edge collaborative system in real time or periodically, and input it as an input feature into the algorithm for prediction. The system then compares the prediction results with the actual scheduling parameters, and calculates the evaluation index value to evaluate the performance of the algorithm. This process can not only help the system understand the adaptability and accuracy of the algorithm in the current environment, but also provide strong support for subsequent algorithm improvement and optimization.

[0178] In one implementation, before evaluating the scheduling strategy prediction algorithm according to the target evaluation sample set in step S20, the method further includes:

[0179] Step S1: obtaining a candidate evaluation sample set including computing power hardware learning samples under multiple cloud-edge collaborative systems and training labels corresponding to each computing power hardware learning sample;

[0180] Step S2: The algorithm is calibrated in a repeated training manner according to the learning sample set. After each round of calibration, the scheduling strategy estimation algorithm in the calibration is evaluated based on the candidate evaluation sample set to obtain the algorithm loss.

[0181] Step S3: After each round of algorithm adjustment, determine whether the algorithm loss continues to decrease, wherein the scheduling strategy prediction algorithm is evaluated based on the target evaluation sample set when it is determined that the algorithm loss does not continue to decrease.

[0182] In step S1, the computer system collects and organizes a set of additional sample data, namely, the candidate evaluation sample set. This set of sample data contains multiple computing hardware learning samples under the cloud-edge collaborative system, and each learning sample corresponds to a training label. These learning samples and training labels are similar to the learning sample set obtained in step S10, and are used to describe the hardware resource status, past scheduling behavior, and expected scheduling parameters under the cloud-edge collaborative system. However, the purpose of the candidate evaluation sample set is not to be used directly for algorithm training, but to serve as reference data in the algorithm tuning and evaluation process.

[0183] Specifically, each computing power hardware learning sample contains multiple hardware resource basic data, past hardware resource scheduling statistics, and past hardware resource scheduling node flow data under the cloud-edge collaborative system. The hardware resource basic data describes the static properties of the hardware resources, such as CPU model, memory capacity, etc.; the past hardware resource scheduling statistics record the system's scheduling behavior of hardware resources in the past period of time, such as scheduling time, scheduling frequency, etc.; the past hardware resource scheduling node flow data is a time series data that records the sequence of scheduling events that occurred in the system in the past period of time. The training mark corresponding to each learning sample represents the scheduling parameters that the system expects to obtain under the hardware resource state described by the learning sample, such as CPU utilization, memory occupancy, etc.

[0184] The candidate evaluation sample set may be obtained in a similar way to the learning sample set, both from the actual operation data or simulation data of the cloud-edge collaborative system. However, in order to ensure the independence and effectiveness of algorithm tuning and evaluation, the candidate evaluation sample set should maintain a certain difference from the learning sample set, that is, they should not completely overlap or contain the same samples. This can be achieved by collecting data from different time periods, different hardware resources, or different scheduling scenarios.

[0185] In step S2, the computer system begins to calibrate the scheduling strategy prediction algorithm using the learning sample set. Algorithm calibration is an iterative process that aims to optimize the performance of the algorithm by continuously adjusting the algorithm parameters. In each iteration, the system uses the sample data in the learning sample set to train the algorithm and tries to find a set of optimal parameter settings so that the algorithm performs best on the training data. However, simply performing well on the training data is not enough to prove the effectiveness of the algorithm. Therefore, after each round of algorithm calibration, the system also needs to evaluate the algorithm in calibration using the candidate evaluation sample set. The purpose of this step is to test the generalization ability of the algorithm on unseen data, that is, whether the algorithm can accurately predict the scheduling parameters of unknown samples. During the evaluation process, the system calculates the loss function value of the algorithm on the candidate evaluation sample set, that is, the algorithm loss. The loss function is an indicator that measures the difference between the algorithm's prediction results and the actual results, and it is usually a non-negative real number. The smaller the loss function value, the closer the algorithm's prediction results are to the actual results, and the better the algorithm's performance is. In machine learning, commonly used loss functions include mean square error (MSE), root mean square error (RMSE), cross entropy loss, etc.

[0186] In order to obtain the algorithm loss, the system needs to input the sample data in the candidate evaluation sample set into the algorithm being tuned and obtain the algorithm's prediction results. The system then compares the prediction results with the actual training labels in the sample data and calculates the loss function value. This process is repeated multiple times, each time using different sample data to calculate the loss function value, and finally an average loss function value is obtained as the evaluation result of this round of algorithm tuning.

[0187] In step S3, the computer system needs to analyze and judge the evaluation results of each round of algorithm calibration. Specifically, the system will compare the loss function values ​​of two adjacent rounds of algorithm calibration to determine whether the algorithm loss continues to decrease. If the loss function value of the current round is less than the loss function value of the previous round, it means that the performance of the algorithm continues to improve, and the system will continue to the next round of algorithm calibration. On the contrary, if the loss function value of the current round is greater than or equal to the loss function value of the previous round, it means that the performance of the algorithm has reached a bottleneck or has begun to overfit. At this time, the system should stop the algorithm calibration and prepare for the final evaluation of the algorithm.

[0188] When the algorithm's loss is judged not to continue to decrease, the system will proceed to the next step: evaluate the scheduling strategy estimation algorithm based on the target evaluation sample set. The purpose of this step is to test the performance of the algorithm on the final evaluation data and determine whether it can meet the needs of actual applications.

[0189] Similar to the evaluation process of the target evaluation sample set, the system will input the sample data in the target evaluation sample set into the algorithm and obtain the algorithm's prediction results. The system will then compare the prediction results with the actual training labels in the sample data and calculate the loss function value and other related evaluation indicators (such as accuracy, recall, F1 score, etc.). These evaluation indicators will comprehensively reflect the performance of the algorithm on the target evaluation sample set and provide important reference for subsequent algorithm deployment and application.

[0190] It should be noted that although the evaluation process of the target evaluation sample set is similar to that of the candidate evaluation sample set, they differ in purpose and use. The candidate evaluation sample set is mainly used for performance monitoring and early stopping decisions during algorithm tuning, while the target evaluation sample set is used for the final evaluation and performance verification of the algorithm. Therefore, when collecting and processing these two sets of sample data, the system needs to ensure their independence and differences to avoid problems such as data leakage and overfitting.

[0191] In summary, steps S1 to S3 constitute a complete algorithm tuning and evaluation process. By continuously adjusting algorithm parameters, monitoring algorithm performance, and conducting final evaluation, the computer system can gradually optimize the performance of the scheduling strategy prediction algorithm and provide strong support for its deployment and use in practical applications.

[0192] In one implementation, step S20, training a scheduling strategy prediction algorithm based on a learning sample set, includes:

[0193] Step S21: inputting the computing power hardware learning samples in the learning sample set into the basic scheduling strategy estimation algorithm to obtain the probability coefficient, mean value and dispersion output by the basic scheduling strategy estimation algorithm;

[0194] Step S22: determining a first generation value of a first cost function according to the likelihood coefficient and the training mark corresponding to the computing hardware learning sample;

[0195] Step S23: determining the second generation value of the second cost function according to the training mark, mean and dispersion corresponding to the computing hardware learning sample;

[0196] Step S24: Determine the total generation value according to the first generation value and the second generation value, and update the algorithm parameters of the basic scheduling strategy estimation algorithm based on the total generation value to obtain the scheduling strategy estimation algorithm.

[0197] In step S21, the computer system transmits the computing power hardware learning samples in the learning sample set as input data to the basic scheduling strategy prediction algorithm. The basic scheduling strategy prediction algorithm is a pre-designed machine learning model that can predict scheduling strategies or scheduling parameters based on the input learning samples. In this implementation, the algorithm is designed to output three key indicators: probability coefficient, mean, and dispersion.

[0198] The probability coefficient is a value between 0 and 1, which indicates the possibility of a specific scheduling strategy under the current hardware resource status. This indicator helps the system understand the applicability of different scheduling strategies in the current environment. For example, if the probability coefficient is close to 1, the system may be more inclined to choose this scheduling strategy; on the contrary, if the probability coefficient is close to 0, the system may be more inclined to avoid this scheduling strategy.

[0199] The mean is a numerical value that represents the expected value or average level of the scheduling parameters. In the cloud-edge collaborative system, the scheduling parameters may include CPU usage, memory occupancy, network bandwidth, etc. The mean indicator helps the system understand the expected load of hardware resources and make more reasonable scheduling decisions.

[0200] Dispersion is an indicator that measures the volatility or uncertainty of scheduling parameters, which is usually expressed through statistics such as standard deviation or variance. The dispersion indicator helps the system understand the range of changes and potential risks of scheduling parameters, thereby formulating more robust scheduling strategies.

[0201] The output of the basic scheduling strategy estimation algorithm is a vector or matrix containing the probability coefficient, mean and dispersion. This output will be used as input data for calculating the cost function value in the subsequent steps.

[0202] In step S22, the computer system calculates the first generation value of the first cost function using the likelihood coefficient and the training mark corresponding to the computing hardware learning sample. The first cost function is an indicator that measures the difference between the algorithm prediction result and the actual training mark, which is usually constructed based on the likelihood coefficient.

[0203] In this implementation, the first cost function may be in the form of a cross entropy loss function or a logarithmic loss function. These loss functions can effectively measure the difference between the probability distribution predicted by the algorithm and the actual training mark. Specifically, if the probability coefficient predicted by the algorithm matches the actual training mark (that is, the algorithm correctly predicts the probability of the scheduling policy), then the value of the first cost function will be small; conversely, if the probability coefficient predicted by the algorithm does not match the actual training mark, then the value of the first cost function will be large.

[0204] To calculate the first generation value, the system needs to compare the probability coefficient of each computing hardware learning example with the corresponding training mark and calculate the loss value according to the definition of the first cost function. This process will iteratively calculate each example in the learning example set and finally obtain an average loss value as the first generation value.

[0205] In step S23, the computer system uses the computing hardware to learn the training labels, mean and dispersion corresponding to the sample to calculate the second generation value of the second cost function. The second cost function is an indicator that measures the numerical difference between the algorithm prediction result and the actual training label, which is usually constructed based on the mean and dispersion.

[0206] In this implementation, the second cost function may take the form of a mean square error (MSE) loss function, a root mean square error (RMSE) loss function, or a mean absolute error (MAE) loss function. These loss functions can effectively measure the difference between the scheduling parameter values ​​predicted by the algorithm and the actual training marks. Specifically, if the mean and dispersion predicted by the algorithm match the actual training marks (that is, the algorithm accurately predicts the expected value and volatility of the scheduling parameters), then the value of the second cost function will be small; conversely, if the mean and dispersion predicted by the algorithm do not match the actual training marks, then the value of the second cost function will be large.

[0207] In order to calculate the second-generation value, the system needs to compare the mean and dispersion of each computing hardware learning sample with the corresponding training mark, and calculate the loss value according to the definition of the second cost function. This process will also iterate each sample in the learning sample set and finally obtain an average loss value as the second-generation value.

[0208] In step S24, the computer system performs a weighted summation (or other combination) of the first generation value and the second generation value to obtain a total generation value. The total generation value is an indicator that comprehensively measures the difference between the algorithm prediction result and the actual training mark, which combines information from multiple aspects such as the likelihood coefficient, mean, and dispersion.

[0209] In order to optimize the performance of the basic scheduling strategy estimation algorithm, the system needs to update the algorithm parameters based on the total cost value. This process usually involves optimization methods such as gradient descent algorithm, stochastic gradient descent algorithm, Adam optimization algorithm, etc. In each iteration, the system calculates the total cost value under the current parameters and updates the parameter values ​​according to the optimization algorithm to reduce the total cost value and improve the performance of the algorithm.

[0210] As the number of iterations increases, the parameters of the basic scheduling strategy prediction algorithm will gradually converge to an optimal solution or a near-optimal solution. At this point, the algorithm can already predict scheduling strategies or scheduling parameters more accurately and can be used in subsequent scheduling decisions. Ultimately, the trained scheduling strategy prediction algorithm will be used in the scheduling of computing hardware resources in the cloud-edge collaborative system to improve the overall performance and resource utilization of the system.

[0211] Figure 2 A hardware entity diagram of a computer system provided by an embodiment of the present invention is as follows Figure 2 As shown, the hardware entity of the computer system 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can be run on the processor 1001, and the processor 1001 implements the steps in the method of any of the above embodiments when executing the program.

[0212] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0213] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0214] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0215] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0216] The above description is only an implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for adaptively scheduling computing hardware resources in a cloud-edge collaborative system, characterized in that: The method comprises: Obtain the hardware resource status data of the target cloud-edge collaborative system, the hardware resource status data including data item values ​​of the target cloud-edge collaborative system corresponding to multiple reference data respectively, the multiple reference data including multiple hardware resource basic data, multiple past hardware resource scheduling statistical data and multiple past hardware resource scheduling node flow data, wherein the past hardware resource scheduling statistical data include the scheduling parameters of the target cloud-edge collaborative system in multiple computing power hardware resources, and the past hardware resource scheduling node flow data is used to represent a computing power hardware resource scheduling event flow composed of a preset number of scheduling events of the target cloud-edge collaborative system in the multiple computing power hardware resources before the current time node; Quantifying the data item values ​​corresponding to each hardware resource basic data and the past hardware resource scheduling statistical data to obtain a first quantitative feature, quantifying the sequence data item values ​​corresponding to each past hardware resource scheduling node flow data to obtain a second quantitative feature, and obtaining a first combined feature based on each first quantitative feature and each second quantitative feature; The first combined features are transmitted to a plurality of first domain adaptive algorithms respectively, and each first domain adaptive algorithm is used to perform feature embedding to obtain first algorithm embedded output features output by each first domain adaptive algorithm; Determine the information coverage of the hardware resource status data in each data group deployed in advance, and construct a first vacancy indication feature of the hardware resource status data based on the information coverage of the hardware resource status data in each data group and the quantitative feature set for each data group, wherein the first vacancy indication feature is used to mark missing data of a data item value corresponding to control data in the hardware resource status data; Determine the importance parameters of each first-domain adaptive algorithm based on the first vacancy indication feature, perform feature weighted fusion on each first algorithm embedded output feature based on the importance parameter to obtain a first vacancy perception feature, and obtain an inferred scheduling strategy for target computing power hardware resources under the target cloud-edge collaborative system based on a feature combination result of the first vacancy perception feature and each first algorithm embedded output feature; The method of obtaining the first combination feature based on each first quantitative feature and each second quantitative feature includes: fusing each first quantitative feature into a fourth combination feature, and fusing each second quantitative feature into a fifth combination feature; using multiple interaction operators to respectively determine the interaction between the features of the fourth combination feature, and obtaining the first hidden layer array output by each interaction operator; transmitting the fifth combination feature to a recurrent network operator, and obtaining the second hidden layer array output by the recurrent network operator; fusing the first hidden layer array and the second hidden layer array into the first combination feature; The determining of the information coverage of the hardware resource status data in each data group deployed in advance, and constructing the first vacancy indication feature of the hardware resource status data based on the information coverage of the hardware resource status data in each data group and the quantitative features set for each data group, includes: for any data group, determining the number of lost data items corresponding to the lost data item value in the corresponding data group, subtracting the number of lost data items from the total number of data items in the corresponding data group to obtain the total number of non-lost data items, dividing the total number of non-lost data items by the total number of data items to obtain the information coverage of the corresponding data group; for each data group, determining the corresponding information coverage; for each data group, multiplying the corresponding information coverage by the quantitative features set for each data group to obtain the adjustment quantitative features corresponding to each data group, and fusing the adjustment quantitative features corresponding to each data group to obtain the first vacancy indication feature; The scheduling strategy for target computing power hardware resources under the inferred target cloud-edge collaborative system based on the feature combination result of the first vacancy perception feature and the first algorithm embedded output features includes: fusing the first vacancy perception feature and the first algorithm embedded output features to obtain a second combination feature; transmitting the second combination feature or the third combination feature generated based on the second combination feature to a multi-layer perceptron module to obtain the execution result of the multi-layer perceptron module; transmitting the execution result to a logical activation operator, a linear activation operator and a logarithmic unit activation operator respectively, and estimating the possibility coefficient of scheduling the target cloud-edge collaborative system, the mean of the scheduling parameters of the target cloud-edge collaborative system and the discreteness of the scheduling parameters of the target cloud-edge collaborative system respectively; determining the inferred user's scheduling strategy for target computing power hardware resources based on the possibility coefficient, the mean and the discreteness.

2. The method according to claim 1, characterized in that Before transmitting the second combined feature or the third combined feature generated according to the second combined feature to the multi-layer perceptron module, the method further includes: The second combined features are transmitted to a plurality of second domain adaptive algorithms respectively, and each second domain adaptive algorithm is used to perform feature embedding to obtain a second algorithm embedded output feature output by each second domain adaptive algorithm; Constructing a second missing indication feature of the hardware resource status data according to the information coverage of the hardware resource status data in each data group and the quantitative feature set for each data group, wherein the second missing indication feature is used to mark missing data about the interaction relationship between the control data in the hardware resource status data; Determining importance parameters of each second domain adaptive algorithm according to the second vacancy indication feature, and performing feature weighted fusion on each second algorithm embedded output feature according to the importance parameter to obtain a second vacancy perception feature; The second vacancy perception feature and the first vacancy perception feature are fused to obtain a third combined feature.

3. The method according to claim 2, characterized in that The constructing the second vacancy indication feature of the hardware resource status data according to the information coverage of the hardware resource status data in each data group and the quantitative feature set for each data group includes: For each pair of data groups, determining the combined information coverage of the hardware resource status data in the pair of data groups based on the information coverage of each data group in the pair of data groups, and determining the quantitative features of the pair of data groups based on the quantitative features set for each data group in the pair of data groups; Based on the combined information coverage and quantitative characteristics of each pair of data groups, second vacancy indication characteristics of each pair of data groups are determined; the second vacancy indication characteristics of each pair of data groups are merged to obtain the second vacancy indication characteristics of the hardware resource status data.

4. The method according to claim 3, characterized in that The determining, based on the information coverage degree of each data group of the hardware resource status data in the pair of data groups, of the combined information coverage degree of the hardware resource status data in the pair of data groups, and determining, based on the quantitative feature set for each data group in the pair of data groups, the quantitative feature of the pair of data groups, comprises: Calculate the product of the information coverage of the hardware resource status data in each data group in the pair of data groups to determine the combined information coverage of the hardware resource status data in the pair of data groups; Determine a quantitative product of the quantitative features set for each data group in the pair of data groups, and use the product as the quantitative feature of the pair of data groups; The determining, based on the combined information coverage and the quantitative characteristics of each pair of data packets, the second vacancy indication characteristics of each pair of data packets comprises: The product of the combined information coverage of each pair of data groups and the quantitative feature is calculated to obtain the second vacancy indication feature of each pair of data groups.

5. The method according to claim 1, characterized in that The method is executed by a scheduling strategy estimation algorithm, and before obtaining the hardware resource status data of the target cloud-edge collaborative system, the method further includes: Obtain a learning sample set and a target evaluation sample set, wherein the learning sample set and the target evaluation sample set are both sample sets including multiple computing power hardware learning samples and training marks corresponding to each computing power hardware learning sample, wherein the computing power hardware learning samples include data item values ​​corresponding to multiple hardware resource basic data under the cloud-edge collaborative system, data item values ​​corresponding to multiple past hardware resource scheduling statistical data, and data item values ​​corresponding to multiple past hardware resource scheduling node flow data, wherein the past hardware resource scheduling statistical data include scheduling parameters in multiple computing power hardware resources, and the past hardware resource scheduling node flow data is used to represent a computing power hardware resource scheduling event flow composed of a preset number of scheduling events in the multiple computing power hardware resources before the current time node, and the training mark is a scheduling parameter in a subsequent set time range for the target computing power hardware resource; Obtaining a scheduling strategy estimation algorithm based on the training of the learning sample set; The scheduling strategy prediction algorithm is evaluated based on the target evaluation sample set, wherein the hardware resource status data of the target cloud-edge collaborative system is obtained during the evaluation process of the scheduling strategy prediction algorithm.

6. The method according to claim 5, characterized in that Before evaluating the scheduling strategy prediction algorithm according to the target evaluation sample set, the method further includes: Obtain a candidate evaluation sample set including computing power hardware learning samples under multiple cloud-edge collaborative systems and training labels corresponding to each computing power hardware learning sample; The algorithm is calibrated in a repeated training manner according to the learning sample set, and after each round of calibration, the scheduling strategy estimation algorithm in the calibration is evaluated based on the candidate evaluation sample set to obtain the algorithm loss; After each round of algorithm adjustment, it is determined whether the loss of the algorithm continues to decrease, wherein the scheduling strategy prediction algorithm is evaluated according to the target evaluation sample set when it is determined that the loss of the algorithm does not continue to decrease.

7. The method according to claim 5, characterized in that The scheduling strategy prediction algorithm obtained by training the learning sample set includes: Inputting the computing power hardware learning samples in the learning sample set into the basic scheduling strategy prediction algorithm to obtain the probability coefficient, mean value and dispersion output by the basic scheduling strategy prediction algorithm; Determining a first generation value of a first cost function according to the likelihood coefficient and a training tag corresponding to the computing hardware learning sample; Determining a second generation value of a second cost function according to the training mark corresponding to the computing hardware learning sample, the mean value and the dispersion; A total cost value is determined according to the first generation value and the second generation value, and algorithm parameters of the basic scheduling strategy estimation algorithm are updated based on the total cost value to obtain a scheduling strategy estimation algorithm.

8. A computer system comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, wherein: When the processor executes the program, the steps in the method according to any one of claims 1 to 7 are implemented.

Images