Cloud platform resource scheduling optimization method and system of multi-chip architecture

Abstract

The invention discloses a cloud platform resource scheduling optimization method and system for a multi-chip architecture, and relates to the technical field of resource scheduling, and the method comprises the steps: carrying out the real-time monitoring of the multi-chip architecture of a cloud platform, and obtaining the monitoring data of each chip; performing fault risk inspection to obtain a chip cluster; task processing resource scheduling decision making is carried out, and a chip resource scheduling first decision space is established; performing initial optimization, and constructing a second decision space for chip resource scheduling; performing deep optimization to generate a chip resource scheduling optimization strategy; and task processing resource scheduling optimization is carried out. The technical problems of low resource utilization rate and poor task processing efficiency caused by lack of comprehensive management and control in the cloud platform multi-chip architecture in the prior art are solved, the full-process management and control of the cloud platform multi-chip architecture from monitoring, fault risk inspection to task processing resource scheduling optimization are realized, and the task processing efficiency is improved. And the resource utilization rate and the task processing efficiency of the cloud platform are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and in particular to a cloud platform resource scheduling optimization method and system with a multi-chip architecture. Background Art

[0002] In today's digital age, cloud platforms have become key infrastructure to support all kinds of businesses. With the explosive growth of data volume and the increasing demand for computing performance, cloud platform multi-chip architecture has emerged. This architecture significantly improves computing power by integrating multiple chips to work together. However, in actual operation, it faces many severe challenges. On the one hand, when multiple chips work together, it is difficult to accurately monitor the operating status of each chip in real time. Once a chip fails, it will have a serious impact on the overall operation of the cloud platform, but it cannot be discovered and handled in time. On the other hand, due to the lack of an efficient task processing resource scheduling mechanism, the chip resource utilization rate is low, the task processing efficiency is low, some chips are overloaded, and some chips are idle, resulting in a huge waste of resources.

[0003] The existing technology has technical problems such as low resource utilization and poor task processing efficiency due to the lack of comprehensive management and control in the multi-chip architecture of the cloud platform. Summary of the invention

[0004] The present application provides a cloud platform resource scheduling optimization method and system for a multi-chip architecture, which is used to solve the technical problems in the prior art of cloud platform multi-chip architecture, such as low resource utilization and poor task processing efficiency due to lack of comprehensive management and control.

[0005] In view of the above problems, the present application provides a cloud platform resource scheduling optimization method and system with a multi-chip architecture.

[0006] In a first aspect of the present application, a method for optimizing resource scheduling of a cloud platform with a multi-chip architecture is provided, the method comprising: The multi-chip architecture of the cloud platform is monitored in real time to obtain monitoring data of each chip, wherein the multi-chip architecture includes H chips, and H is a positive integer greater than 1; the H chips are tested for fault risks according to the monitoring data of each chip to obtain a first chip cluster less than a chip fault risk threshold, and a second chip cluster greater than or equal to the chip fault risk threshold; task processing resource scheduling decisions are made according to the first chip cluster and the second chip cluster to establish a first decision space for chip resource scheduling; a chip resource scheduling evaluation channel is introduced to perform initial optimization on the first chip resource scheduling decision space to construct a second decision space for chip resource scheduling; the second chip resource scheduling decision space is deeply optimized according to the task processing loss prediction channel to generate a chip resource scheduling optimization strategy; task processing resource scheduling optimization is performed on the first chip cluster and the second chip cluster according to the chip resource scheduling optimization strategy.

[0007] A second aspect of the present application provides a cloud platform resource scheduling optimization system with a multi-chip architecture, the system comprising: A chip monitoring data acquisition module is used to perform real-time monitoring on the multi-chip architecture of the cloud platform and obtain monitoring data of each chip, wherein the multi-chip architecture includes H chips, and H is a positive integer greater than 1; a risk inspection module is used to perform a fault risk inspection on the H chips according to the monitoring data of each chip, and obtain a first chip cluster less than a chip fault risk threshold, and a second chip cluster greater than or equal to the chip fault risk threshold; a first decision space establishment module is used to make a task processing resource scheduling decision according to the first chip cluster and the second chip cluster, and establish a first decision space for chip resource scheduling; a second decision space establishment module is used to introduce a chip resource scheduling evaluation channel to perform initial optimization on the first decision space for chip resource scheduling, and construct a second decision space for chip resource scheduling; a resource scheduling optimization strategy generation module is used to perform deep optimization on the second decision space for chip resource scheduling according to the task processing loss prediction channel, and generate a chip resource scheduling optimization strategy; a resource scheduling optimization module is used to perform task processing resource scheduling optimization on the first chip cluster and the second chip cluster according to the chip resource scheduling optimization strategy.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages: The multi-chip architecture of the cloud platform is monitored in real time to obtain monitoring data of each chip; fault risk inspection is performed to obtain a first chip cluster with a chip fault risk threshold value less than the chip fault risk threshold value, and a second chip cluster with a chip fault risk threshold value greater than or equal to the chip fault risk threshold value; task processing resource scheduling decisions are made to establish a first decision space for chip resource scheduling; a chip resource scheduling evaluation channel is introduced to perform initial optimization on the first decision space for chip resource scheduling to construct a second decision space for chip resource scheduling; a deep optimization is performed on the second decision space for chip resource scheduling according to the task processing loss prediction channel to generate a chip resource scheduling optimization strategy; task processing resource scheduling optimization is performed on the first chip cluster and the second chip cluster according to the chip resource scheduling optimization strategy. The full process control of the multi-chip architecture of the cloud platform from monitoring, fault risk inspection to task processing resource scheduling optimization is achieved, and the technical effect of improving the resource utilization rate and task processing efficiency of the cloud platform is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic diagram of a method for optimizing resource scheduling on a cloud platform using a multi-chip architecture according to an embodiment of the present application; Figure 2 A schematic diagram of the structure of a cloud platform resource scheduling and optimization system with a multi-chip architecture provided in an embodiment of the present application.

[0011] Explanation of the accompanying drawings: chip monitoring data acquisition module 10, risk verification module 20, first decision space establishment module 30, second decision space establishment module 40, resource scheduling optimization strategy generation module 50, resource scheduling optimization module 60. DETAILED DESCRIPTION

[0012] The present application provides a cloud platform resource scheduling optimization method and system with a multi-chip architecture, which is used to solve the technical problems in the prior art of cloud platform multi-chip architecture, such as low resource utilization and poor task processing efficiency due to the lack of comprehensive management and control.

[0013] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] Embodiment 1, as Figure 1 As shown, the present application provides a cloud platform resource scheduling optimization method for a multi-chip architecture, the method comprising: Step S100: Real-time monitoring of a multi-chip architecture of a cloud platform is performed to obtain monitoring data of each chip, wherein the multi-chip architecture includes H chips, and H is a positive integer greater than 1.

[0015] Specifically, in order to ensure that the multi-chip architecture works stably and efficiently in the complex operating environment of the cloud platform, it is necessary to monitor it in real time in an all-round and uninterrupted manner. This multi-chip architecture consists of H chips (H is greater than 1). For these chips, a high-precision sensor network is deployed at the hardware level and a special data acquisition program is run at the software level to monitor the chips from multiple dimensions. For example, the temperature sensor in the sensor network can accurately measure the operating temperature of each chip and capture small changes in temperature in real time; the power consumption monitor closely monitors the power consumption level of the chip to detect abnormal energy consumption fluctuations in time; and the performance counter records the key performance indicators of the chip, such as the computing speed, data processing volume, and instruction execution frequency. The data acquisition program integrates, preprocesses, and formats the raw data collected by these sensors, and converts them into monitoring data of each chip for subsequent analysis and processing. These data not only reflect the current operating status of the chip, but also provide a key basis for predicting problems with the chip and optimizing resource scheduling, ensuring that the cloud platform maintains reliable operation in the multi-chip collaborative mode.

[0016] Step S200: performing a failure risk check on the H chips according to the chip monitoring data to obtain a first chip cluster with a chip failure risk less than a threshold value and a second chip cluster with a chip failure risk greater than or equal to the threshold value.

[0017] Specifically, after obtaining the monitoring data of each chip in the multi-chip architecture of the cloud platform, the fault risk inspection phase is immediately entered. For each chip in the H chips (set as the hth chip, 1≤h≤H), its monitoring data is first carefully extracted and sorted to ensure the integrity and accuracy of the data. Then, the pre-built and fully trained P base learners (P is greater than 1) are used. These base learners are trained based on a large amount of historical chip operation data and failure cases, and have strong data analysis and pattern recognition capabilities. Through the integrated fusion learning technology, the analysis results of the P base learners are comprehensively processed to construct a fault risk prediction channel dedicated to the hth chip. The monitoring data of the hth chip is input into the prediction channel to obtain the fault risk prediction coefficient of the hth chip. This coefficient accurately quantifies the possibility of failure of the chip in the current operating state. Finally, the prediction coefficient is strictly compared with the preset chip failure risk threshold. If the prediction coefficient is less than the threshold, it indicates that the chip is in a relatively stable operating state and is classified into the first chip cluster; conversely, if the prediction coefficient is greater than or equal to the threshold, it means that the chip has a high failure risk and is classified into the second chip cluster. Through such a rigorous analysis process, accurate classification of the failure risks of all chips in the multi-chip architecture is achieved.

[0018] Step S300: performing task processing resource scheduling decisions according to the first chip cluster and the second chip cluster, and establishing a first decision space for chip resource scheduling.

[0019] Specifically, when making task processing resource scheduling decisions based on the chip cluster and establishing the first decision space for chip resource scheduling, for the collection of unfinished task information of the second chip cluster, comprehensive scanning and real-time data capture technology are used to ensure that detailed parameters of the task are obtained, such as task priority, task execution progress, required resource type and quantity, etc. Then, the cluster analysis algorithm is used to process these task information to obtain the distribution of unfinished tasks. For the identification of the resource demand distribution of unfinished tasks, the resource demand prediction model is used. The model is trained based on historical task data and resource usage, and the required resources are estimated by inputting task feature parameters. When collecting the available resource information of the first chip cluster, the built-in resource monitoring tool is used to obtain the real-time resource status of the chip, including CPU usage, memory idleness, bandwidth occupancy, etc., and the data normalization algorithm is used to unify these different types of resource data into the same dimension to form the chip available resource distribution. When making task processing resource scheduling decisions, a combination of greedy algorithm and dynamic programming algorithm is adopted. The greedy algorithm is used to initially allocate tasks. According to the priority and resource requirements of the tasks, the tasks are preferentially allocated to the most suitable chips with the most resources at present, so as to quickly build the initial scheduling plan. On this basis, the dynamic programming algorithm globally optimizes the entire scheduling process, considers the dependencies between tasks and the dynamic changes of chip resources, and continuously adjusts the task allocation plan by recursively solving the optimal substructure to achieve the goal of maximizing the overall resource utilization efficiency. Ultimately, these different scheduling scheme combinations constitute the first decision space for chip resource scheduling. The policy space provides a rich selection basis for further optimizing resource scheduling in the future.

[0020] Step S400: introducing a chip resource scheduling evaluation channel to perform initial optimization on the first chip resource scheduling decision space to construct a second chip resource scheduling decision space.

[0021] Specifically, the chip resource scheduling evaluation channel is introduced to perform initial optimization on the first decision space of chip resource scheduling. The chip resource scheduling evaluation channel includes two core parts: the chip load balancing evaluation model and the chip resource utilization evaluation model. For the chip load balancing evaluation model, the variance analysis algorithm is used to calculate the variance of each chip load under each scheduling scheme. The smaller the variance, the more balanced the load; the chip resource utilization evaluation model uses the proportion calculation method to calculate the ratio of the actual usage of each chip resource to the total resource amount to measure the resource utilization. A chip resource scheduling scheme is randomly selected from the first decision space (recorded as the nth decision of chip resource scheduling, n>1), and it is input into the two evaluation models at the same time. The model evaluates the scheme according to its own algorithm, obtains the chip load balancing index and the chip resource utilization index, and generates the nth decision resource scheduling evaluation result by combining the two. Then, this evaluation result is compared with the preset chip resource scheduling evaluation expectation. The chip load balancing constraint in the evaluation expectation can be set as the upper limit of the load variance, and the chip resource utilization constraint is set as the lower limit of the resource utilization. If the evaluation result of the nth decision resource scheduling meets these constraints, it indicates that the solution performs well in terms of load balancing and resource utilization, and it is added to the second decision space of chip resource scheduling; if not, the solution is eliminated. By repeating this random extraction, evaluation and screening process, better scheduling solutions are gradually screened out, thus constructing the second decision space of chip resource scheduling, providing a more potential solution set for subsequent deep optimization.

[0022] Step S500: performing deep optimization on the second decision space for chip resource scheduling according to the task processing loss prediction channel to generate a chip resource scheduling optimization strategy.

[0023] Specifically, when performing deep optimization on the second decision space of chip resource scheduling according to the task processing loss prediction channel to generate a chip resource scheduling optimization strategy, the task processing loss optimization operator must be clarified first. This operator is defined by comprehensively considering the task processing quality loss prediction coefficient and the task processing delay loss prediction coefficient, wherein the task processing quality loss prediction coefficient can be obtained based on the data accuracy evaluation model, which analyzes the degree of deviation between the task processing result and the expected result. The greater the deviation, the greater the coefficient; the task processing delay loss prediction coefficient is calculated based on the task execution time monitoring data, and the coefficient increases when it exceeds the predetermined time threshold. Based on the task processing loss prediction channel, it includes a task processing quality loss prediction model and a task processing delay loss prediction model. Each resource scheduling scheme in the second decision space is sequentially input into the task processing quality loss prediction model, which uses a regression analysis algorithm to predict the task processing quality loss according to factors such as task characteristics and chip performance, and obtains the task processing quality loss prediction coefficient; then input into the task processing delay loss prediction model, which is based on the queuing theory algorithm, considering factors such as task queuing waiting time and chip processing speed, predicts the task processing delay loss, and obtains the task processing delay loss prediction coefficient. The solutions that simultaneously meet the requirements that the predicted coefficient of task processing quality loss is less than the task processing quality loss threshold and the predicted coefficient of task processing delay loss is less than the task processing delay loss threshold are selected to construct the third decision space for chip resource scheduling. Then, a chip resource scheduling merit evaluation function is constructed, and the chip resource scheduling merit value of each solution in the third decision space is calculated based on this evaluation function. The solution with the largest value is selected as the chip resource scheduling optimization strategy, thereby achieving deep optimization of resource scheduling.

[0024] Step S600: performing task processing resource scheduling optimization on the first chip cluster and the second chip cluster according to the chip resource scheduling optimization strategy.

[0025] Specifically, according to the generated chip resource scheduling optimization strategy, the task processing resource scheduling optimization is performed on the first chip cluster and the second chip cluster. For the first chip cluster, the optimization strategy will accurately guide the task allocation of each chip. According to the performance characteristics, resource availability and task requirements of the chip, the appropriate tasks are accurately allocated to the corresponding chips to ensure the best match between tasks and chip resources. For example, for computing-intensive tasks, chips with strong computing power are given priority; for tasks with large data storage and transmission requirements, chips with rich storage and communication resources are arranged. At the same time, the task volume of each chip is reasonably controlled to avoid overload or idle resources, so as to maximize the overall task processing efficiency and resource utilization of the first chip cluster.

[0026] For the second chip cluster, since its chips have a higher risk of failure, the optimization strategy will handle its task allocation more carefully. On the one hand, the task volume of high-risk chips is reduced, and some critical tasks or tasks with high real-time requirements are transferred to reliable chips in the first chip cluster to ensure the stability and timeliness of task execution. On the other hand, for tasks that still need to be processed locally in the second chip cluster, the optimization strategy will closely monitor their execution process and dynamically adjust task parameters according to the real-time status of the chip, such as reducing the task processing accuracy requirements to reduce resource consumption, or appropriately extending the task processing time to reduce chip pressure. In addition, combined with the early warning mechanism, when the failure risk of the chip in the second chip cluster reaches a higher warning level, its unfinished tasks are promptly suspended or migrated to ensure the continuity and reliability of task processing of the entire cloud platform, and ultimately achieve the optimal scheduling of task processing resources and overall performance improvement under the multi-chip architecture.

[0027] In a possible implementation, step S300 further includes: Step S310: Collecting the unfinished task information of the second chip cluster to obtain the unfinished task distribution.

[0028] Step S320: Identify resource requirements based on the unfinished task distribution to obtain the unfinished task resource requirement distribution.

[0029] Step S330: Collect the available resource information of the first chip cluster to obtain the chip available resource distribution.

[0030] Step S340: Taking the unfinished task distribution as the resource scheduling target, making task processing resource scheduling decisions for the first chip cluster according to the unfinished task resource demand distribution and the chip available resource distribution, and generating the first decision space for chip resource scheduling.

[0031] Specifically, in order to gain an in-depth understanding of the task status of the second chip cluster, its unfinished task information is collected through a variety of technical means. In the task management system of the cloud platform, a special data collection interface is set up to establish a close data connection with each chip in the second chip cluster. Using these interfaces, detailed data of tasks that the chip is processing but has not yet ended is obtained in real time. These data include the name of the task, the start time of the task, the duration of the task execution, the amount of data involved in the task, and the current execution stage of the task. For example, for a data processing task, information such as the number of data blocks that have been processed and the amount of data remaining to be processed will be obtained. At the same time, in order to grasp the characteristics of the task more accurately, the software environment information that the task depends on is also collected, such as the specific library file version required to run the task, the operating system configuration, etc. By classifying and summarizing these rich task information and conducting statistical analysis, a detailed map of the distribution of unfinished tasks is produced with dimensions such as task type, task execution time range, and task data size. For example, it may be found that a certain type of specific data processing task occupies a large proportion of the unfinished tasks in the second chip cluster, and the execution time of these tasks is generally long.

[0032] For each type of task in the distribution of unfinished tasks, a resource demand estimation model based on task feature analysis is used. The model establishes a mapping relationship between task features and resource requirements by extracting and learning features from a large amount of historical task data. For example, for data processing tasks, the model will analyze the data volume, data format, processing algorithm and other features of the task. If the task involves large-scale data processing and uses complex algorithms, the model will calculate the required computing resources (such as the number of CPU cores and computing time) based on pre-trained weight parameters. At the same time, a rule-based resource demand matching algorithm is used to perform preliminary matching based on task types and common resource demand patterns. For example, image processing tasks usually require a large amount of memory space for image caching. The algorithm will allocate corresponding memory resource requirements for image processing tasks based on this rule. Then, the results of the estimation model and the matching algorithm are comprehensively weighed. For the parts with differences, an expert system is introduced for manual intervention and adjustment. The expert system further optimizes the resource requirements based on domain knowledge and empirical rules. Finally, the resource requirements of all unfinished tasks are summarized and counted according to resource types (computing resources, storage resources, communication resources, etc.) to form the distribution of unfinished task resource requirements. For example, the computing resource demand distribution may show that during a certain period of time, computing-intensive tasks have a peak demand for the number of CPU cores, while the storage resource demand distribution indicates that certain data storage tasks occupy a large amount of hard disk space, thereby providing an accurate basis for resource demand for subsequent resource scheduling decisions.

[0033] In order to obtain the callable resource information of the first chip cluster and generate the chip available resource distribution, at the hardware level, the key resource status data of the chip is collected in real time through the resource monitoring unit integrated in each chip. These data include the current utilization rate of the CPU, the number of idle cores, the cache occupancy, the used space, the remaining available space, the read and write speed of the memory, and the used capacity, remaining capacity, read and write bandwidth of the storage device (such as a hard disk or solid-state drive). At the same time, for the communication resources between the chip and the external device, the current network connection status, data transmission rate, port occupancy and other information are obtained. At the software level, the resource management program is run, which works closely with the chip's operating system to obtain system-level resource usage information, such as the number of running processes, the amount of resources occupied by each process, and the resource occupancy of system services. The resource information collected at the hardware and software levels is summarized and integrated, and the data of different types and units is uniformly converted into a standard format using a data standardization algorithm for subsequent analysis and processing. Finally, by statistically analyzing these standardized resource information, a chip available resource distribution chart is produced according to the chip numbering sequence or resource type classification. This distribution clearly shows the availability of various types of resources for each chip in the first chip cluster. For example, chip 1 has a large number of idle CPU cores but less available memory space, while chip 2 has more abundant storage resources, etc., providing an accurate data basis for the reasonable allocation of tasks.

[0034] Guided by the distribution of unfinished tasks, the resource demand distribution of unfinished tasks and the distribution of available resources of chips are fully considered to make task processing resource scheduling decisions for the first chip cluster, and then construct the first decision space for chip resource scheduling. First, analyze the key factors such as task priority, task type and task volume in the distribution of unfinished tasks to determine the urgency of tasks and resource demand characteristics. Combined with the distribution of resource demand of unfinished tasks, clarify the specific quantity and type of computing resources, storage resources and communication resources required for various tasks. Then, compare the distribution of available resources of chips to evaluate the resource capacity and performance that each chip in the first chip cluster can provide. Use intelligent matching algorithms, such as resource matching algorithms based on greedy strategies, to preferentially allocate tasks with high priority and urgent resource demand to chips that can meet their resource demand and have low resource utilization. In the allocation process, the load balancing of chips is comprehensively considered to avoid excessive load on a chip while other chips are idle. For example, for computing-intensive tasks, chips with strong computing power and low current computing load are preferred; for tasks with large data storage requirements, chips with abundant storage resources are matched. At the same time, according to the dependencies between tasks and data interaction requirements, the distribution of tasks between chips is reasonably arranged to reduce data transmission delay. By continuously adjusting the task allocation plan, a variety of feasible resource scheduling combinations are formed. These combinations constitute the first decision space for chip resource scheduling, providing a rich set of options for further optimizing resource scheduling.

[0035] In a possible implementation, step S400 further includes: Step S410: randomly extracting the nth decision for chip resource scheduling according to the first decision space for chip resource scheduling, where n is a positive integer greater than 1.

[0036] Step S420: inputting the nth decision of the chip resource scheduling into the chip resource scheduling evaluation channel to obtain the nth decision resource scheduling evaluation result, wherein the chip resource scheduling evaluation channel includes a chip load balancing evaluation model and a chip resource utilization evaluation model.

[0037] Step S430: Determine whether the nth decision resource scheduling evaluation result meets chip resource scheduling evaluation expectations, wherein the chip resource scheduling evaluation expectations include chip load balancing constraints and chip resource utilization constraints.

[0038] Step S440: If the evaluation result of the nth decision resource scheduling meets the chip resource scheduling evaluation expectation, the nth decision of chip resource scheduling is added to the second decision space of chip resource scheduling.

[0039] Step S450: If the resource scheduling evaluation result of the nth decision does not meet the chip resource scheduling evaluation expectation, the nth decision of chip resource scheduling is eliminated.

[0040] Specifically, in the first decision space of chip resource scheduling, since there are multiple chip resource scheduling decisions, in order to comprehensively evaluate the pros and cons of these decisions, it is necessary to randomly select one of the decisions for analysis, which is recorded as the nth decision of chip resource scheduling (n>1). The random extraction here is achieved by using a random number generation algorithm, which randomly generates an integer as the decision number within the index range of the decision space, thereby selecting the corresponding nth decision of chip resource scheduling. This random extraction method can ensure that in the subsequent evaluation process, each decision has the same probability of being selected, thereby achieving an unbiased evaluation of the entire decision space.

[0041] After extracting the nth decision of chip resource scheduling, it is input into the chip resource scheduling evaluation channel. This evaluation channel consists of two important parts: chip load balancing evaluation model and chip resource utilization evaluation model. The chip load balancing evaluation model analyzes the load of tasks assigned to each chip and measures the load balance by calculating the variance of the task volume of each chip. If the variance or standard deviation is small, it means that the load between chips is relatively balanced; otherwise, the load is unbalanced. The chip resource utilization evaluation model focuses on the proportional relationship between the actual use of chip resources and the total resource volume. For computing resources, it will count indicators such as the CPU usage rate and memory occupancy rate of each chip; for storage resources, it will consider the usage ratio of hard disk space, etc. Through the collaborative work of these two models, the performance of the nth decision of chip resource scheduling in terms of load balancing and resource utilization can be comprehensively evaluated, thereby obtaining the resource scheduling evaluation result of the nth decision, that is, the chip load balancing coefficient and chip resource utilization coefficient corresponding to the nth decision of chip resource scheduling.

[0042] After obtaining the resource scheduling evaluation result of the nth decision, it is necessary to compare it with the pre-set chip resource scheduling evaluation expectation. The chip resource scheduling evaluation expectation includes two aspects: chip load balancing constraint and chip resource utilization constraint. The chip load balancing constraint can be set as a threshold range of a load balancing coefficient. For example, it is required that the variance of the chip task volume is less than a certain value to ensure that there is not too much difference in the load between the chips, and to avoid the situation where some chips are too busy while other chips are idle. The chip resource utilization constraint stipulates the lower limit of the resource utilization coefficient to ensure that chip resources can be fully utilized and avoid resource waste. By comparing the chip load balancing coefficient and chip resource utilization coefficient of the nth decision with these constraints, it is determined whether the decision meets expectations.

[0043] If the evaluation result of the nth decision resource scheduling meets the chip resource scheduling evaluation expectations, it means that the decision performs well in terms of load balancing and resource utilization, and has high feasibility and optimization potential. At this time, the nth decision of chip resource scheduling is added to the second decision space of chip resource scheduling. The second decision space of chip resource scheduling is gradually constructed during the screening and optimization of the first decision space, which contains those decision solutions that performed well in the initial optimization process. Adding the decisions that meet expectations to the second decision space provides a better decision solution basis for subsequent deep optimization, which helps to further improve the efficiency and quality of resource scheduling.

[0044] If the resource scheduling evaluation result of the nth decision does not meet the chip resource scheduling evaluation expectations, it indicates that the decision is insufficient in load balancing or resource utilization and does not meet the optimization goal of resource scheduling. In this case, the nth decision of chip resource scheduling is eliminated and no longer considered in the future. By eliminating these decisions that do not meet expectations, the amount of calculation in the subsequent optimization process can be reduced, and more potential decision-making solutions can be optimized, thereby improving the efficiency of the entire resource scheduling optimization process and gradually screening out the optimal resource scheduling strategy.

[0045] In a possible implementation, step S500 further includes: Step S510: Obtain a task processing loss optimization operator, wherein the task processing loss optimization operator includes a task processing quality loss prediction coefficient being less than a task processing quality loss threshold, and a task processing delay loss prediction coefficient being less than a task processing delay loss threshold.

[0046] Step S520: Based on the task processing loss prediction channel, taking the task processing loss optimization operator as the task processing loss optimization target, perform task processing loss optimization on the second decision space of chip resource scheduling, and establish a third decision space for chip resource scheduling, wherein the task processing loss prediction channel includes a task processing quality loss prediction model and a task processing delay loss prediction model.

[0047] Step S530: construct a chip resource scheduling optimality evaluation function.

[0048] Step S540: maximizing the chip resource scheduling optimality of the third decision space for chip resource scheduling according to the chip resource scheduling optimality evaluation function, and generating the chip resource scheduling optimization strategy.

[0049] Specifically, in order to obtain the task processing loss optimization operator, a calculation model based on multi-dimensional data evaluation is established for the task processing quality loss prediction coefficient. Key data in the task execution process are collected, such as data accuracy in data processing tasks (by comparing the consistency of data before and after processing, error rate and other indicators), image clarity in image processing tasks (using image recognition algorithms to evaluate the difference between processed images and original images), and data transmission integrity in communication tasks (checking the loss rate of transmitted data, error verification results, etc.). These data are normalized to be in the same dimensional range, and then the task processing quality loss prediction coefficient is calculated using a weighted summation algorithm based on the pre-set weight allocation (based on the importance of different data on task quality). At the same time, for the task processing delay loss prediction coefficient, the task execution time series analysis algorithm is used to calculate the waiting time from submission to the start of execution, the time spent in each stage of the execution process, and the final completion time, and compared with the preset task standard execution time. According to the size of the time difference and the priority of the task (tasks with high priority are more sensitive to delays and have higher weights), the task processing delay loss prediction coefficient is obtained through a time weighted algorithm. When determining the task processing quality loss threshold and the task processing delay loss threshold, the actual needs of the business should be taken into consideration. For example, for financial transaction businesses, the task processing quality loss threshold is set to an extremely low level to ensure the accuracy of transaction data; for real-time video streaming services, the task processing delay loss threshold is set according to the video frame rate and user experience requirements to ensure the smoothness of video playback. In this way, a task processing loss optimization operator that meets the conditions is obtained, providing an accurate basis for the subsequent screening of high-quality resource scheduling solutions.

[0050] When optimizing the second decision space of chip resource scheduling based on the task processing loss prediction channel to establish the third decision space, the neural network regression algorithm in machine learning is used to construct the task processing quality loss prediction model. The task characteristics (including task type, task data volume, allocated chip resource parameters, etc.) are used as input variables, and the quality loss evaluation value after the execution of historical tasks (such as data error rate in data processing tasks, image distortion in image processing tasks, etc.) are used as output variables to train the model. After the training is completed, the task characteristic data of each scheduling scheme in the second decision space of chip resource scheduling is input into the model, and the model predicts the task processing quality loss prediction coefficient under the scheme based on the learned rules. For the task processing delay loss prediction model, an algorithm combining queuing theory and time series analysis is used. According to the queuing rules of tasks in the chip queue, the rate of chip processing tasks, and the historical task execution time data, a prediction model for task processing delay time is constructed. Considering the differences in delay tolerance of different task priorities, higher weights are assigned to high-priority tasks so that the model can more accurately predict their delay losses. When the scheduling schemes in the second decision space are input into the model, the model will calculate the task processing delay loss prediction coefficient corresponding to each scheme. Then, with the task processing loss optimization operator as the target, each scheduling scheme in the second decision space is evaluated one by one. If the task processing quality loss prediction coefficient of a scheme is less than the quality loss threshold and the task processing delay loss prediction coefficient is less than the delay loss threshold, the scheme is included in the third decision space for chip resource scheduling. Through such a screening process, the schemes in the third decision space have better performance in terms of task processing quality and delay, providing a more potential candidate set for the subsequent generation of the final optimization strategy.

[0051] For the chip load balancing coefficient, the standard deviation of the task load of each chip is calculated and normalized so that its value range is between 0 and 1, thereby accurately reflecting the degree of load balance between chips. The chip resource utilization coefficient is obtained by statistically analyzing the ratio of the actual resource usage of each chip to the total resource usage of the chip, and is obtained after normalization to measure the utilization efficiency of chip resources. The task processing quality loss prediction coefficient and the task processing delay loss prediction coefficient are derived from the output results of the task processing loss prediction model respectively. These coefficients are also normalized to ensure comprehensive evaluation under the same dimension. In order to reflect the importance of different factors in the evaluation, weight factors are introduced. The chip load balancing weight, chip resource utilization weight, task quality loss weight and task delay loss weight are set according to specific business needs and optimization goals, and finally the chip resource scheduling optimal evaluation function is constructed.

[0052] According to the constructed chip resource scheduling optimal evaluation function, each scheme in the third decision space of chip resource scheduling is calculated to obtain the optimal value of chip resource scheduling for each scheme. Then, by comparing these optimal values, the maximum value is found. The genetic algorithm is used to iteratively optimize the schemes in the third decision space, and the task allocation and resource configuration in the scheme are continuously adjusted to improve the optimal value. In the iterative process, according to the feedback of the evaluation function, the search is gradually carried out in the direction of maximizing the optimal value. After multiple iterations, the scheme with the largest optimal value is finally determined, which is the chip resource scheduling optimization strategy. This strategy can achieve a good balance in many aspects such as chip load balancing, resource utilization, task processing quality and latency, and realize the optimal scheduling of task processing resources under the multi-chip architecture of the cloud platform.

[0053] In a possible implementation, step S530 further includes: ; Among them, SCR represents the optimality of chip resource scheduling, exp represents the exponential function with the natural constant e as the base, G(LBC) represents the normalized chip load balancing coefficient, LBCW represents the chip load balancing weight, G(CRE) represents the normalized chip resource utilization coefficient, CREW represents the chip resource utilization weight, G(QLP) represents the normalized task processing quality loss prediction coefficient, QLPW represents the task quality loss weight, G(DLP) represents the normalized task processing delay loss prediction coefficient, and DLPW represents the task delay loss weight.

[0054] Specifically, the chip load balancing coefficient G (LBC) is a quantitative representation of the degree of load balancing between chips. By calculating the standard deviation of the task load of each chip under resource scheduling, and then normalizing it, its value is mapped to the range of 0 to 1. The chip resource utilization coefficient G (CRE) reflects the utilization efficiency of chip resources. It is obtained by counting the ratio of the actual amount of resources used by the chip to the total amount of resources and normalizing it. The task processing quality loss prediction coefficient G (QLP) is derived from the output of the task processing quality loss prediction model. It is used to measure the degree of loss of task processing quality under a specific resource scheduling scheme. After normalization, it can be comprehensively evaluated with other coefficients under a unified standard. The task processing delay loss prediction coefficient G (DLP) also comes from the task processing delay loss prediction model and is used to evaluate the potential loss of task processing delay. The chip load balancing weight LBCW determines the importance of chip load balancing in the overall evaluation. If LBCW is large, it means that in the resource scheduling decision, more attention is paid to the balanced distribution of loads between chips to avoid excessive load on some chips and affect the overall performance and stability. The chip resource utilization weight CREW reflects the influence of chip resource utilization efficiency in the evaluation. A larger CREW means giving priority to making full use of chip resources, reducing resource waste, and improving the resource input-output ratio. Task quality loss weight QLPW This weight reflects the key position of task processing quality in the evaluation system. When QLPW is higher, it means that there are higher requirements for the accuracy, completeness and other quality aspects of task processing results, and resource scheduling should tend to ensure task quality. The task delay loss weight DLPW determines the degree of influence of task processing delay on the overall evaluation. A higher DLPW indicates that special attention should be paid to the real-time nature of tasks during resource scheduling, and task execution delays should be minimized to meet time-sensitive business needs.

[0055] The application of the exponential function exp performs a nonlinear transformation on the evaluation results. This transformation makes the function more sensitive to the slight differences between the coefficients and can more finely distinguish the advantages and disadvantages of different resource scheduling schemes. When the numerator (the part related to chip load balancing and resource utilization) is large and the denominator (the part related to task processing quality and delay loss) is small, the SCR value will increase significantly, indicating that the scheme performs well in many aspects and has a high degree of superiority; conversely, if the numerator is small or the denominator is large, the SCR value will become smaller and the superiority of the scheme will decrease. Through this function form, the optimal chip resource scheduling scheme can be accurately screened out based on comprehensive consideration of multiple factors and their interrelationships, and the optimal decision of resource scheduling under the multi-chip architecture of the cloud platform can be achieved.

[0056] In a possible implementation, step S200 further includes: Step S210: extracting the h-th chip monitoring data corresponding to the h-th chip according to the monitoring data of each chip, wherein h is a positive integer, 1≤h≤H.

[0057] Step S220: performing integrated fusion learning on the h-th chip according to P basis learners to establish a fault risk prediction channel for the h-th chip, where P is a positive integer greater than 1.

[0058] Step S230: inputting the h-th chip monitoring data into the h-th chip failure risk prediction channel to obtain the h-th chip failure risk prediction coefficient.

[0059] Step S240: determining whether the hth chip failure risk prediction coefficient is less than the chip failure risk threshold.

[0060] Step S250: If the h-th chip failure risk prediction coefficient is less than the chip failure risk threshold, add the h-th chip to the first chip cluster.

[0061] Step S260: If the hth chip failure risk prediction coefficient is greater than or equal to the chip failure risk threshold, add the hth chip to the second chip cluster.

[0062] Specifically, when extracting the monitoring data corresponding to the hth chip from the monitoring data of each chip, a unique identifier is assigned to the monitoring data of each chip using data indexing technology, and the identifier corresponds to the chip number. When storing the monitoring data, it is organized into a structured data table, in which each row corresponds to a monitoring record at a time point, and each column corresponds to a different monitoring indicator, such as temperature, power consumption, operating frequency, etc. By traversing the row records of the data table, based on the key column of the chip number, a hash search algorithm is used to quickly locate all records matching the hth chip number. The hash search algorithm calculates a hash function on the chip number to obtain a hash value, and uses the hash value to directly search for the corresponding data record in the hash table, greatly improving the search efficiency. These found records are aggregated to form a subset of monitoring data corresponding to the hth chip, ensuring that the extracted data is complete and accurate, and providing a reliable data basis for the subsequent analysis and evaluation of the hth chip.

[0063] To construct the fault risk prediction channel for the hth chip, we first need to select P different types of machine learning models as base learners, such as decision tree, naive Bayes, K nearest neighbor algorithm, neural network, and support vector machine. In the data preparation stage, the historical monitoring data of the hth chip is divided into training set, validation set, and test set at a ratio of 70%, 20%, and 10%. These data cover multi-dimensional features such as chip temperature, power consumption, operation frequency, and error rate. For each base learner, the training set is used as input for model training. The decision tree learner recursively divides the features of the training data to construct a tree structure so that each leaf node corresponds to a fault risk category. The naive Bayes learner is based on the Bayesian theorem, assuming that the features are independent of each other, and calculates the probability of chip failure or no failure under given monitoring data features. The K nearest neighbor algorithm stores all training samples in the training stage. When predicting new monitoring data, it selects the K samples with the closest distance by calculating the distance between them and the training samples, and determines the fault risk category of the hth chip according to the categories of these samples. The neural network learner uses the back propagation algorithm to continuously adjust the connection weights between neurons to minimize the error between the predicted value and the true label, thereby learning the complex relationship between the monitoring data and the fault risk. The support vector machine learner divides the faulty and fault-free states of the chip in the feature space by finding an optimal classification hyperplane. After all base learners are trained, the Stacking ensemble learning algorithm is used for fusion. First, each base learner is evaluated using the validation set to obtain the prediction results of each base learner on the validation set. Then, these prediction results are input as new features into a meta-learner (such as a logistic regression model) for training. The meta-learner learns the prediction performance of each base learner on the validation set and comprehensively obtains a final prediction result. Finally, the test set is used to evaluate the entire integrated model to ensure the accuracy and generalization ability of the model. After such an integrated fusion learning process, the h-th chip fault risk prediction channel is successfully established.

[0064] The extracted h-th chip monitoring data is input into the h-th chip failure risk prediction channel that has just been constructed. Each base learner in the prediction channel will analyze the monitoring data according to its own learned pattern and output its own prediction results. Then, the outputs of these base learners are fused through the prediction fusion layer, and a comprehensive h-th chip failure risk prediction coefficient is obtained by weighted average. This coefficient is a quantitative value, which represents the possibility of failure of the h-th chip under the current monitoring state. Its numerical range and specific meaning can be determined according to the design and training method of the prediction model, which provides an intuitive basis for the subsequent judgment of chip failure risk.

[0065] After obtaining the prediction coefficient of the h-th chip failure risk, compare it with the preset chip failure risk threshold. The chip failure risk threshold is a critical value determined based on the cloud platform's requirements for chip reliability and a large amount of historical data statistical analysis. It divides the chip failure risk into two ranges: acceptable and unacceptable. If the prediction coefficient is less than the threshold, it means that the h-th chip has a low risk of failure in the current state and is in a relatively safe operating state; conversely, if the prediction coefficient is greater than or equal to the threshold, it indicates that the h-th chip has a high risk of failure, which may affect the normal operation of the cloud platform. This judgment step is a key link in determining the subsequent classification of the chip, which directly affects the formulation of resource scheduling strategies.

[0066] If the h-th chip failure risk prediction coefficient is less than the chip failure risk threshold, it is added to the first chip cluster according to the risk status of the chip. The chips in the first chip cluster are considered healthy and reliable, can normally undertake task processing work, and will be given priority in resource scheduling decisions to assign more tasks to make full use of their stable performance. If the h-th chip failure risk prediction coefficient is greater than or equal to the chip failure risk threshold, it is added to the second chip cluster. Due to the high failure risk of the chips in the second chip cluster, special processing measures need to be taken, such as reducing their task allocation, strengthening monitoring frequency, arranging maintenance or replacement plans, etc., to reduce their potential impact on the overall performance of the cloud platform, while ensuring the stability and reliability of the cloud platform under the multi-chip architecture. Through such chip cluster division, the foundation is laid for subsequent targeted resource scheduling decisions, which can better adapt to the complex and changing operating environment of the cloud platform.

[0067] In a possible implementation, step S220 further includes: Step S221: performing a fault risk record search according to the h-th chip to obtain an h-th chip monitoring sample set and an h-th chip fault risk sample set.

[0068] Step S222: performing supervised learning on the P base learners according to the h-th chip monitoring sample set and the h-th chip failure risk sample set to obtain P chip failure risk prediction models.

[0069] Step S223: using the output data sets of the P chip failure risk prediction models as input information and the h-th chip failure risk sample set as output information to train the h-th chip failure risk prediction fusion layer.

[0070] Step S224: merging the P chip failure risk prediction models as parallel independent nodes to generate an h-th chip failure risk prediction processing layer.

[0071] Step S225: taking the h-th chip fault risk prediction processing layer as a fault risk prediction primary node, and taking the h-th chip fault risk prediction fusion layer as a fault risk prediction secondary node.

[0072] Step S226: connecting the fault risk prediction level 1 node and the fault risk prediction level 2 node to generate the h-th chip fault risk prediction channel.

[0073] Specifically, by comprehensively searching the fault risk records of the hth chip, the hth chip monitoring sample set and the hth chip fault risk sample set are carefully selected. These sample sets are the key data basis for subsequent model training. The monitoring sample set contains various monitoring data during the chip operation process, such as temperature, voltage, operation frequency, etc., while the fault risk sample set clearly marks whether the chip has a fault risk under the corresponding monitoring data.

[0074] Based on the above two sample sets, supervised learning is performed on P base learners. Each base learner continuously adjusts its own parameters and strives to learn the intrinsic connection between the monitoring sample set and the failure risk sample set, and finally generates P chip failure risk prediction models. These models each have unique prediction capabilities and evaluate chip failure risks from different angles.

[0075] The output data sets of P chip failure risk prediction models are used as input information, and the h-th chip failure risk sample set is used as output information to train the h-th chip failure risk prediction fusion layer. The fusion layer aims to integrate the prediction results of each model, explore the commonalities and differences, and thus improve the accuracy of the prediction.

[0076] The P chip failure risk prediction models are merged as parallel independent nodes to generate the h-th chip failure risk prediction processing layer. This process involves the efficient integration of different models. First, the functions and input and output interfaces of each chip failure risk prediction model are clarified. Although these models are built based on different algorithm principles, they are all aimed at evaluating the failure risk of the h-th chip. For example, the decision tree-based model hierarchically divides the chip monitoring data features to obtain prediction results, while the neural network-based model uses complex neuron connections and weight calculations to achieve predictions. To achieve parallel processing, a unified input data distribution mechanism is established. When new chip monitoring data is input, the mechanism can simultaneously distribute the data to the P parallel chip failure risk prediction models. After receiving the data, each model independently analyzes and processes the data according to the rules and parameters obtained by its own training. For example, some models focus on analyzing the impact of the chip temperature change trend on the failure risk, while others focus on the stability of the operating frequency. After each model completes the data processing, a result collection mechanism is established, which collects the output results of the P models to form an output data set containing multi-dimensional information. Although these results may be in different forms, they are all related to the chip failure risk. Through this merging method of parallel independent nodes, the generated h-th chip fault risk prediction processing layer can efficiently perform multi-faceted analysis of the input data, providing a rich and comprehensive information basis for further processing of the subsequent h-th chip fault risk prediction fusion layer, significantly improving the accuracy and efficiency of the h-th chip fault risk prediction.

[0077] In order to establish the h-th chip fault risk prediction processing layer as the first-level node for fault risk prediction, and the h-th chip fault risk prediction fusion layer as the second-level node for fault risk prediction, first, configure the h-th chip fault risk prediction processing layer and clarify its input and output interface specifications in the entire system. By writing specific interface functions, it is ensured that it can receive chip monitoring data from the outside and accurately output the results of parallel processing of P chip fault risk prediction models. At the hardware level, independent computing resources are allocated to the processing layer, such as dedicated CPU cores or GPU computing units, to ensure its efficient operation. For the h-th chip fault risk prediction fusion layer, the operation logic of its internal algorithm is optimized. Using a distributed computing framework, after receiving data from the first-level node, the data can be quickly distributed to different computing nodes for parallel processing. At the same time, by establishing a data caching mechanism, frequently accessed data is cached to reduce data reading time and improve fusion efficiency. In terms of data interaction, a unified data format standard is set for the two-level nodes. Whether it is the data output by the processing layer or the data received by the fusion layer, it follows this standard to avoid compatibility problems caused by inconsistent data formats.

[0078] By connecting these two nodes, the h-th chip failure risk prediction channel is successfully generated. This channel integrates the results of the previous steps to form a complete and efficient chip failure risk prediction system, which can accurately predict the failure risk of the h-th chip.

[0079] Embodiment 2, based on the same inventive concept as the cloud platform resource scheduling optimization method of the multi-chip architecture in the previous embodiment, Figure 2 As shown, the present application provides a cloud platform resource scheduling optimization system with a multi-chip architecture, and the system and method embodiments in the present application are based on the same inventive concept. The system includes: The chip monitoring data acquisition module 10 is used to monitor the multi-chip architecture of the cloud platform in real time and obtain monitoring data of each chip, wherein the multi-chip architecture includes H chips, and H is a positive integer greater than 1.

[0080] The risk verification module 20 is used to perform a failure risk verification on the H chips according to the chip monitoring data to obtain a first chip cluster with a chip failure risk less than a threshold value and a second chip cluster with a chip failure risk greater than or equal to the threshold value.

[0081] The first decision space establishing module 30 is used to make a task processing resource scheduling decision according to the first chip cluster and the second chip cluster, and establish a first decision space for chip resource scheduling.

[0082] The second decision space establishment module 40 is used to introduce a chip resource scheduling evaluation channel to perform initial optimization on the first chip resource scheduling decision space to construct a second chip resource scheduling decision space.

[0083] The resource scheduling optimization strategy generation module 50 is used to perform deep optimization on the second decision space of chip resource scheduling according to the task processing loss prediction channel to generate a chip resource scheduling optimization strategy.

[0084] The resource scheduling optimization module 60 is used to perform task processing resource scheduling optimization on the first chip cluster and the second chip cluster according to the chip resource scheduling optimization strategy.

[0085] Furthermore, the first decision space establishing module 30 further includes: The unfinished task distribution acquiring unit is used to collect the unfinished task information of the second chip cluster to obtain the unfinished task distribution.

[0086] The unfinished task resource requirement distribution acquisition unit is used to identify resource requirements according to the unfinished task distribution and obtain the unfinished task resource requirement distribution.

[0087] The chip available resource distribution acquisition unit is used to collect the available resource information of the first chip cluster to obtain the chip available resource distribution.

[0088] The task processing resource scheduling decision unit is used to take the unfinished task distribution as the resource scheduling target, make task processing resource scheduling decisions for the first chip cluster according to the unfinished task resource demand distribution and the chip available resource distribution, and generate the first decision space for chip resource scheduling.

[0089] Furthermore, the second decision space establishing module 40 further includes: The nth decision extraction unit is used to randomly extract the nth decision of chip resource scheduling according to the first decision space of chip resource scheduling, wherein n is a positive integer greater than 1.

[0090] The nth decision resource scheduling evaluation result acquisition unit is used to input the chip resource scheduling nth decision into the chip resource scheduling evaluation channel to obtain the nth decision resource scheduling evaluation result, wherein the chip resource scheduling evaluation channel includes a chip load balancing evaluation model and a chip resource utilization evaluation model.

[0091] The chip resource scheduling evaluation expectation judgment unit is used to judge whether the nth decision resource scheduling evaluation result meets the chip resource scheduling evaluation expectation, wherein the chip resource scheduling evaluation expectation includes chip load balancing constraints and chip resource utilization constraints.

[0092] The nth decision adding unit is used to add the nth decision of chip resource scheduling to the second decision space of chip resource scheduling if the evaluation result of the nth decision resource scheduling meets the chip resource scheduling evaluation expectation.

[0093] The nth decision elimination unit is used to eliminate the nth decision of chip resource scheduling if the evaluation result of the nth decision resource scheduling does not meet the chip resource scheduling evaluation expectation.

[0094] Furthermore, the resource scheduling optimization strategy generation module 50 also includes: A task processing loss optimization operator acquisition unit is used to obtain a task processing loss optimization operator, wherein the task processing loss optimization operator includes a task processing quality loss prediction coefficient that is less than a task processing quality loss threshold, and a task processing delay loss prediction coefficient that is less than a task processing delay loss threshold.

[0095] The third decision space unit for chip resource scheduling is used to optimize the task processing loss of the second decision space of chip resource scheduling based on the task processing loss prediction channel and taking the task processing loss optimization operator as the task processing loss optimization target, so as to establish the third decision space of chip resource scheduling, wherein the task processing loss prediction channel includes a task processing quality loss prediction model and a task processing delay loss prediction model.

[0096] The optimality evaluation function construction unit is used to construct the optimality evaluation function for chip resource scheduling.

[0097] The optimality maximization optimization unit is used to perform optimality maximization optimization of the chip resource scheduling in the third decision space of the chip resource scheduling according to the chip resource scheduling optimality evaluation function, and generate the chip resource scheduling optimization strategy.

[0098] Furthermore, the optimality evaluation function construction unit also includes: ; Among them, SCR represents the optimality of chip resource scheduling, exp represents the exponential function with the natural constant e as the base, G(LBC) represents the normalized chip load balancing coefficient, LBCW represents the chip load balancing weight, G(CRE) represents the normalized chip resource utilization coefficient, CREW represents the chip resource utilization weight, G(QLP) represents the normalized task processing quality loss prediction coefficient, QLPW represents the task quality loss weight, G(DLP) represents the normalized task processing delay loss prediction coefficient, and DLPW represents the task delay loss weight.

[0099] Furthermore, the risk verification module 20 also includes: The hth chip monitoring data extraction unit is used to extract the hth chip monitoring data corresponding to the hth chip according to the monitoring data of each chip, wherein h is a positive integer, 1≤h≤H.

[0100] The h-th chip failure risk prediction channel establishing unit is used to perform integrated fusion learning on the h-th chip according to P basis learners to establish the h-th chip failure risk prediction channel, wherein P is a positive integer greater than 1.

[0101] The h-th chip failure risk prediction coefficient acquisition unit is used to input the h-th chip monitoring data into the h-th chip failure risk prediction channel to obtain the h-th chip failure risk prediction coefficient.

[0102] The chip failure risk threshold judgment unit is used to judge whether the h-th chip failure risk prediction coefficient is less than the chip failure risk threshold.

[0103] The first chip cluster adding unit is configured to add the hth chip to the first chip cluster if the hth chip failure risk prediction coefficient is less than the chip failure risk threshold.

[0104] The second chip cluster adding unit is configured to add the hth chip to the second chip cluster if the hth chip failure risk prediction coefficient is greater than or equal to the chip failure risk threshold.

[0105] Furthermore, the hth chip failure risk prediction channel establishing unit further includes: The sample set acquisition unit is used to retrieve the fault risk record of the hth chip to obtain the hth chip monitoring sample set and the hth chip fault risk sample set.

[0106] A supervised learning unit is used to perform supervised learning on the P base learners according to the h-th chip monitoring sample set and the h-th chip failure risk sample set to obtain P chip failure risk prediction models.

[0107] The fault risk prediction fusion layer training unit is used to train the hth chip fault risk prediction fusion layer using the output data sets of the P chip fault risk prediction models as input information and the hth chip fault risk sample set as output information.

[0108] The fault risk prediction processing layer generation unit is used to merge the P chip fault risk prediction models as parallel independent nodes to generate the hth chip fault risk prediction processing layer.

[0109] The fault risk prediction node acquisition unit is used to use the h-th chip fault risk prediction processing layer as a fault risk prediction primary node and the h-th chip fault risk prediction fusion layer as a fault risk prediction secondary node.

[0110] A fault risk prediction channel generating unit is used to connect the fault risk prediction primary node and the fault risk prediction secondary node to generate the hth chip fault risk prediction channel.

[0111] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0113] This specification and the drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A multi-chip architecture cloud platform resource scheduling optimization method, characterized in that: The method comprises: Performing real-time monitoring on a multi-chip architecture of a cloud platform to obtain monitoring data of each chip, wherein the multi-chip architecture includes H chips, and H is a positive integer greater than 1; Performing a failure risk test on the H chips according to the chip monitoring data to obtain a first chip cluster with a chip failure risk less than a chip failure risk threshold and a second chip cluster with a chip failure risk greater than or equal to the chip failure risk threshold; Performing task processing resource scheduling decisions according to the first chip cluster and the second chip cluster, and establishing a first decision space for chip resource scheduling; Introducing a chip resource scheduling evaluation channel to perform initial optimization on the first decision space for chip resource scheduling, and constructing a second decision space for chip resource scheduling; Performing deep optimization on the second decision space for chip resource scheduling according to the task processing loss prediction channel to generate a chip resource scheduling optimization strategy; The task processing resource scheduling of the first chip cluster and the second chip cluster is optimized according to the chip resource scheduling optimization strategy.

2. The method according to claim 1, characterized in that Performing task processing resource scheduling decisions according to the first chip cluster and the second chip cluster to establish a first decision space for chip resource scheduling includes: Collecting unfinished task information of the second chip cluster to obtain unfinished task distribution; Identify resource requirements based on the unfinished task distribution to obtain the unfinished task resource requirement distribution; Collecting available resource information of the first chip cluster to obtain chip available resource distribution; Taking the unfinished task distribution as the resource scheduling target, performing task processing resource scheduling decision on the first chip cluster according to the unfinished task resource demand distribution and the chip available resource distribution, and generating the first decision space for chip resource scheduling.

3. The method according to claim 1, characterized in that Introducing a chip resource scheduling evaluation channel to perform initial optimization on the first decision space for chip resource scheduling, and constructing a second decision space for chip resource scheduling, including: According to the first decision space of chip resource scheduling, randomly extracting the nth decision of chip resource scheduling, where n is a positive integer greater than 1; Inputting the chip resource scheduling nth decision into the chip resource scheduling evaluation channel to obtain the nth decision resource scheduling evaluation result, wherein the chip resource scheduling evaluation channel includes a chip load balancing evaluation model and a chip resource utilization evaluation model; Determine whether the nth decision resource scheduling evaluation result meets the chip resource scheduling evaluation expectation, wherein the chip resource scheduling evaluation expectation includes chip load balancing constraint and chip resource utilization constraint; If the nth decision resource scheduling evaluation result meets the chip resource scheduling evaluation expectation, adding the nth chip resource scheduling decision to the chip resource scheduling second decision space; If the resource scheduling evaluation result of the nth decision does not meet the chip resource scheduling evaluation expectation, the nth decision of chip resource scheduling is eliminated.

4. The method according to claim 1, characterized in that The second decision space for chip resource scheduling is deeply optimized according to the task processing loss prediction channel to generate a chip resource scheduling optimization strategy, including: Obtaining a task processing loss optimization operator, wherein the task processing loss optimization operator includes a task processing quality loss prediction coefficient being less than a task processing quality loss threshold, and a task processing delay loss prediction coefficient being less than a task processing delay loss threshold; Based on the task processing loss prediction channel, taking the task processing loss optimization operator as the task processing loss optimization target, the second decision space of chip resource scheduling is optimized for task processing loss, and a third decision space of chip resource scheduling is established, wherein the task processing loss prediction channel includes a task processing quality loss prediction model and a task processing delay loss prediction model; Construct an optimal evaluation function for chip resource scheduling; The chip resource scheduling optimality is maximized in the third decision space of the chip resource scheduling according to the chip resource scheduling optimality evaluation function, and the chip resource scheduling optimization strategy is generated.

5. The method according to claim 4, characterized in that The chip resource scheduling optimality evaluation function is: ； Among them, SCR represents the optimality of chip resource scheduling, exp represents the exponential function with the natural constant e as the base, G(LBC) represents the normalized chip load balancing coefficient, LBCW represents the chip load balancing weight, G(CRE) represents the normalized chip resource utilization coefficient, CREW represents the chip resource utilization weight, G(QLP) represents the normalized task processing quality loss prediction coefficient, QLPW represents the task quality loss weight, G(DLP) represents the normalized task processing delay loss prediction coefficient, and DLPW represents the task delay loss weight.

6. The method according to claim 1, characterized in that Performing a fault risk test on the H chips according to the chip monitoring data to obtain a first chip cluster with a chip fault risk less than a threshold value and a second chip cluster with a chip fault risk greater than or equal to the threshold value, including: Extracting the hth chip monitoring data corresponding to the hth chip according to the monitoring data of each chip, wherein h is a positive integer, 1≤h≤H; Performing integrated fusion learning on the h-th chip according to P basis learners to establish a fault risk prediction channel for the h-th chip, wherein P is a positive integer greater than 1; Inputting the h-th chip monitoring data into the h-th chip failure risk prediction channel to obtain the h-th chip failure risk prediction coefficient; Determining whether the h-th chip failure risk prediction coefficient is less than the chip failure risk threshold; If the h-th chip failure risk prediction coefficient is less than the chip failure risk threshold, adding the h-th chip to the first chip cluster; If the h-th chip failure risk prediction coefficient is greater than or equal to the chip failure risk threshold, the h-th chip is added to the second chip cluster.

7. The method according to claim 6, characterized in that Performing integrated fusion learning on the h-th chip according to P basis learners to establish a fault risk prediction channel for the h-th chip, including: Perform a fault risk record search according to the h-th chip to obtain an h-th chip monitoring sample set and an h-th chip fault risk sample set; Performing supervised learning on the P basis learners according to the h-th chip monitoring sample set and the h-th chip failure risk sample set to obtain P chip failure risk prediction models; Taking the output data sets of the P chip failure risk prediction models as input information and the h-th chip failure risk sample set as output information, training the h-th chip failure risk prediction fusion layer; Merging the P chip failure risk prediction models as parallel independent nodes to generate an h-th chip failure risk prediction processing layer; The h-th chip fault risk prediction processing layer is used as a fault risk prediction primary node, and the h-th chip fault risk prediction fusion layer is used as a fault risk prediction secondary node; The fault risk prediction level 1 node and the fault risk prediction level 2 node are connected to generate the hth chip fault risk prediction channel.

8. A cloud platform resource scheduling and optimization system with a multi-chip architecture, characterized in that: The system is used to implement the cloud platform resource scheduling optimization method of the multi-chip architecture according to any one of claims 1 to 7, and the system includes: A chip monitoring data acquisition module is used to monitor the multi-chip architecture of the cloud platform in real time and obtain monitoring data of each chip, wherein the multi-chip architecture includes H chips, and H is a positive integer greater than 1; a risk inspection module, configured to perform a fault risk inspection on the H chips according to the monitoring data of each chip, and obtain a first chip cluster with a chip fault risk less than a threshold value, and a second chip cluster with a chip fault risk greater than or equal to the threshold value; A first decision space establishment module, configured to make a task processing resource scheduling decision based on the first chip cluster and the second chip cluster, and establish a first decision space for chip resource scheduling; A second decision space establishment module is used to introduce a chip resource scheduling evaluation channel to perform initial optimization on the first decision space for chip resource scheduling, and to construct a second decision space for chip resource scheduling; A resource scheduling optimization strategy generation module, used to perform deep optimization on the second decision space of chip resource scheduling according to the task processing loss prediction channel, and generate a chip resource scheduling optimization strategy; The resource scheduling optimization module is used to perform task processing resource scheduling optimization on the first chip cluster and the second chip cluster according to the chip resource scheduling optimization strategy.

Images