Adaptive computing power distribution control method and device and storage medium

Abstract

The invention discloses a self-adaptive computing power distribution control method and device and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: extracting the computing complexity, the video memory demand and the communication overhead corresponding to a computing unit; setting a threshold parameter according to the calculation complexity, the video memory demand and the communication overhead, and clustering and integrating the calculation units to generate a plurality of sub-model modules; performing matching deployment on the sub-model modules and heterogeneous hardware nodes based on a deployment strategy, and recording state information of each node after deployment is completed; state indexes in the state information and the historical performance data are predicted and processed through exponential smoothing, and the dynamic weight value of each node is calculated in combination with the bottleneck sensitivity coefficient; and generating a node priority list according to the dynamic weight value, and allocating the hardware type and the task proportion of the sub-model module according to the node priority list and a task allocation optimization mechanism so as to complete the self-adaptive scheduling of the sub-model module. The problem of low efficiency is solved, and the resource utilization rate is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a control method, device and storage medium for adaptive computing power allocation. BACKGROUND

[0002] In the existing heterogeneous large model inference system, computing power allocation is the core resource scheduling link and is also a key step to ensure efficient operation of each computing node. In related technical solutions, the target large model is usually split into several sub-modules by a static segmentation method, and then these sub-modules are allocated to heterogeneous computing nodes one by one by using simple scheduling strategies such as polling and hashing. This method will limit the overall performance of the system to the local bottleneck node, resulting in low computing efficiency.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a control method, device and storage medium for adaptive computing power allocation, aiming to solve the technical problem that the overall performance of the system is limited by the local bottleneck node, resulting in low computing efficiency.

[0005] To achieve the above purpose, the present application provides a control method for adaptive computing power allocation, which comprises: statically analyzing the computing units of the target large model, extracting the computing complexity, memory demand and communication overhead corresponding to the computing units; setting threshold parameters according to the computing complexity, memory demand and communication overhead, clustering and integrating the computing units to generate a plurality of sub-model modules; matching and deploying the sub-model modules with heterogeneous hardware nodes based on a deployment strategy, and recording the state information of each node after deployment; processing the state indicators in the state information and historical performance data by exponential smoothing prediction, combining a bottleneck sensitivity coefficient to calculate the dynamic weight value of each node; generating a node priority list according to the dynamic weight value, and allocating the hardware type and task proportion of the sub-model modules according to the node priority list and a task allocation optimization mechanism to complete the adaptive scheduling of the sub-model modules.

[0006] In an embodiment, the computing units of the target large model are traversed, and the structure information of the computing units is parsed; based on the structure information, the computing complexity, memory demand peak value and communication transmission total amount of each computing unit are calculated according to a preset index calculation method; The calculation complexity, memory requirement peak value, and communication transmission total amount of the computing unit are analyzed to generate the calculation complexity, memory requirement, and communication overhead corresponding to the computing unit.

[0007] In an embodiment, a static analysis dataset containing the calculation complexity, memory requirement, and communication overhead of each computing unit is called, and the threshold parameters of the three types of indicators are set according to the threshold processing method, respectively. Based on the threshold parameters, each computing unit in the static analysis dataset is analyzed one by one, and the computing unit is associated with the corresponding type label to obtain a computing unit classification set. The computing unit classification set is traversed according to the execution order of the computing unit, the target computing unit with adjacent and consistent type labels is identified, and two types of preliminary sub-model modules are generated. The resource requirement of the preliminary sub-model module is verified to see whether it meets the basic adaptation range of the heterogeneous hardware node, and the preliminary sub-model module that exceeds the adaptation range is secondarily split and adjusted to obtain a plurality of sub-model modules.

[0008] In an embodiment, each sub-model module is preferentially matched with a recommended node according to the deployment strategy. If the target node has deployed a module of the same type and the task queue length does not exceed a preset threshold, the target node is marked as a reuse object, and a preliminary deployment matching list is output. Based on the preliminary deployment matching list, a node with low load and sufficient resources is selected from an alternative node pool and assigned to a module that is not matched with a reuse object, and a deployment result of the sub-model module and the heterogeneous hardware node is output. The running data of each binding node is collected, and the running data is stored in a monitoring system according to node classification to obtain state information of each node.

[0009] In an embodiment, the state indicators corresponding to the nodes are parsed from the state information and historical performance data of the nodes. The state indicators corresponding to the nodes are adaptively exponentially smoothed, the smoothing coefficient is dynamically adjusted according to the variance of the state indicators within a recent time window, and the smoothed prediction values of the state indicators are output. Based on the smoothed prediction values of the state indicators, in combination with the average values of the state indicators in the past plurality of monitoring periods, and a preset bottleneck amplification factor, the bottleneck sensitivity coefficients corresponding to the state indicators are calculated. The smoothed prediction values of the state indicators and the corresponding bottleneck sensitivity coefficients are weighted and combined to obtain weight coefficients. The weight coefficients are automatically matched according to the node type and the sub-model module characteristics, and the dynamic weight values of each node are calculated and adjusted.

[0010] In an embodiment, according to the dynamic weight values of each node and the matching degree of the characteristics of each heterogeneous hardware node and the type of the sub-model module, the node priority list containing the node priority and the corresponding adaptive module type is generated according to the dynamic weight values of each node; According to the node priority list, the sub-model module is preferentially allocated to the node with high weight and matching hardware type, and an initial task allocation scheme of the sub-model module is generated; Based on the initial task allocation scheme, the task proportion of the sub-model module allocation is fine-tuned to complete the adaptive scheduling of the sub-model module.

[0011] In an embodiment, after completing the adaptive scheduling of the sub-model module, the running state of each heterogeneous hardware node is collected, and the running state is compared with the initial reference state of the scheduled node to generate a state comparison result; Based on the state comparison result, the time window sliding mean filtering processing is adopted for the key indicators of each node to generate a bottleneck determination result; For the node triggering the bottleneck in the bottleneck determination result, the local sub-model module re-allocation or node topology optimization is started, and a bottleneck processing result is output; Based on the bottleneck processing result, the variance of the resource utilization rate corresponding to all nodes is calculated, and the global task delay average and the global video memory occupation total are counted. If both conditions are met, the current scheduling state is maintained, and a determination result of global load balancing and resource efficient utilization is output.

[0012] In an embodiment, the running feedback data set is obtained by collecting the response delay of weight adjustment to the bottleneck, the node load balancing degree after task allocation, and the resource utilization deviation; The running feedback data set is associated and analyzed to identify abnormal problems in dynamic weight adjustment, quantify the influence degree of the task allocation optimization mechanism on performance, and output a problem analysis report; According to the problem analysis report, the dynamic weight value and the task allocation optimization mechanism are corrected, the optimized parameters are updated to the adaptive scheduling decision module, and an optimized dynamic weight adjustment strategy and task allocation optimization mechanism parameter set are generated.

[0013] In addition, in order to achieve the above-mentioned purpose, the present application also provides a computing power allocation device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the adaptive computing power allocation control method as described above.

[0014] In addition, to achieve the above-mentioned purpose, the application further provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program is executed by a processor to implement the steps of the adaptive computing power allocation control method as described above.

[0015] The application provides an adaptive computing power allocation control method, which comprises the following steps: firstly, static analysis is performed on a computing unit of a target large model to extract its corresponding computing complexity, memory requirement and communication overhead; then, a plurality of sub-model modules are generated by clustering and integrating the computing unit according to threshold parameters set according to the above indexes; subsequently, the sub-model modules are matched and deployed to heterogeneous hardware nodes based on a deployment strategy, and state information of each node is recorded; then, dynamic weight values of each node are calculated by exponentially smoothing the state indexes in the state information and historical performance data, and combining a bottleneck sensitivity coefficient; finally, a node priority list is generated according to the dynamic weight values, and the hardware types and task proportions of the sub-model modules are allocated according to the list and a task allocation optimization mechanism to complete adaptive scheduling. The application solves the technical problems of local bottleneck, resource mismatch and low overall computing efficiency caused by the static allocation strategy in the existing heterogeneous large model inference system, improves the utilization rate of heterogeneous hardware resources, the load balancing degree of the system and the inference performance of the large model, and realizes precise adaptation of hardware characteristics and sub-model modules.

[0016] In summary, the application extracts the computing complexity, memory requirement and communication overhead by static analysis of the computing unit of the large model, sets threshold values to cluster and integrate to generate sub-model modules, matches and deploys to heterogeneous nodes and records the states, calculates the dynamic weights by exponentially smoothing and the bottleneck sensitivity coefficient, generates a priority list to allocate tasks to complete adaptive scheduling, solves the problems of local bottleneck and low overall efficiency caused by the existing static allocation, and improves the resource utilization rate, load balancing degree and inference performance. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the application and, together with the specification, serve to explain the principles of the application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, brief introductions will be given to the drawings needed to be used in the embodiments or prior art descriptions. Obviously, for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0019] Figure 1 Flowchart of the first embodiment of the adaptive computing power allocation control method of the application; Figure 2 Timing diagram of adaptive scheduling of heterogeneous resources of the application; Figure 3 This is a schematic diagram of the model segmentation for heterogeneous hardware awareness in this application; Figure 4 This application provides a deployment strategy diagram for matching heterogeneous nodes. Figure 5 This is a schematic diagram of the computing power allocation device of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] In related technical solutions, the target large model is usually decomposed into several sub-modules through static partitioning, and then these sub-modules are allocated to heterogeneous computing nodes one by one using simple scheduling strategies such as round-robin and hashing. This approach will cause the overall system performance to be limited by local bottleneck nodes, ultimately resulting in low computational efficiency.

[0023] This application provides a solution: First, statically analyze the computational units of the target large model, extract the computational complexity, memory requirements, and communication overhead corresponding to the computational units, then set threshold parameters based on the computational complexity, memory requirements, and communication overhead, cluster and integrate the computational units to generate multiple sub-model modules, then match and deploy the sub-model modules with heterogeneous hardware nodes based on a deployment strategy, record the status information of each node after deployment, then use exponential smoothing prediction to process the status information and status indicators in historical performance data, and combine the bottleneck sensitivity coefficient to calculate the dynamic weight value of each node, finally generate a node priority list based on the dynamic weight value, and allocate the hardware type and task ratio of the sub-model modules according to the node priority list and task allocation optimization mechanism to complete the adaptive scheduling of the sub-model modules.

[0024] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or computing power allocation device capable of performing the above functions. The following description uses a computing power allocation device as an example to illustrate this embodiment and the subsequent embodiments.

[0025] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0026] This application provides an adaptive computing power allocation control method, referring to... Figure 1 , Figure 1This is a flowchart illustrating the first embodiment of the adaptive computing power allocation control method of this application.

[0027] In this embodiment, the adaptive computing power allocation control method includes steps S10~S50: Step S10: Staticly analyze the computational units of the target large model and extract the computational complexity, memory requirements, and communication overhead corresponding to the computational units.

[0028] In this embodiment, the target large model refers to the large-scale artificial intelligence model to be allocated computing power for inference, and is the object of subsequent static analysis and computing power allocation. A computing unit refers to the basic components that constitute the target large model and execute specific computational logic, including layers or operators. Static analysis refers to the non-real-time analysis performed on the computing units before model inference; it does not change dynamically with the inference process and is based solely on the inherent properties of the model. Computational complexity is an indicator that measures the amount of computation required for a computing unit to perform a computational task, reflecting the difficulty of completing the unit's computational task. Memory requirements refer to the memory resource requirements of the computing unit during execution, including the memory needed to store weight parameters, activation values, and temporary data. Communication overhead refers to the resource consumption incurred when a computing unit transmits data with other computing units, reflecting the cost of data interaction.

[0029] As an optional implementation method, the overall structure of the target large model is traversed, and the basic information of each computing unit is extracted sequentially according to the logical association order of the computing units. Based on the extracted basic information, the computational complexity is initially estimated by statistically analyzing the cumulative computational steps of all computational operations within each computing unit. The estimated results are then corrected, and the values ​​are adjusted by combining historical computational data of similar computing units. For video memory requirements, the scale of weight parameters, the amount of activation value data, and the amount of temporary cache data generated during execution are statistically analyzed separately for each computing unit. These three are then summed to obtain the preliminary video memory requirements. Subsequently, video memory fragmentation is considered, and the preliminary results are supplemented and adjusted. In calculating communication overhead, the connection relationship between the current computing unit and its adjacent computing units is first determined, and the total transmission volume between these connections is statistically analyzed. Then, combined with the path characteristics of data transmission, the additional resource consumption that may be generated during transmission is corrected to obtain the final communication overhead. After extracting the three indicators for a single computing unit, the results are compared and verified with the indicators of similar computing units that have already been analyzed. If the deviation exceeds a reasonable range, the analysis process is re-examined, and the calculation method is adjusted until the indicator extraction results of all computing units meet the consistency requirements. This method employs a rigorous analysis process and effectively reduces the error in indicator extraction through hierarchical progression and cross-validation.

[0030] Step S20: Based on the computational complexity, the memory requirements, and the communication overhead, a threshold parameter is set to cluster and integrate the computing units to generate multiple sub-model modules.

[0031] In this embodiment, the threshold parameter refers to a baseline value set based on characteristic data such as the computational complexity, memory requirements, and communication overhead of the computing unit, serving as a core reference standard for determining the type of the computing unit. A sub-model module refers to a model fragment formed through clustering and integration, possessing independent computational logic and adapted to specific hardware resources; it is the basic unit for subsequent deployment and scheduling.

[0032] As an optional implementation, the computational complexity, memory requirements, and communication overhead data of all computing units are aggregated, and the mean or median of each of the three metrics are calculated. These statistical values ​​are then set as fixed threshold parameters. Each computing unit is then verified one by one: if the computational complexity is higher than the corresponding threshold and the memory requirement is lower than the corresponding threshold, it is determined to be computationally intensive. If the memory requirement is higher than the corresponding threshold or the communication overhead is higher than the corresponding threshold, it is determined to be memory-communication intensive. After classification, the computing units are traversed according to their execution order in the model, and adjacent computing units of the same type are merged to form continuous unit combinations. If the resource requirements of a combination exceed the basic adaptation range of heterogeneous hardware nodes, it is split according to the smallest functional unit, ultimately generating multiple sub-model modules with clearly defined types and compliant resource requirements. This method has simple threshold setting, efficient calculation process, and is suitable for quickly completing clustering and integration.

[0033] As an alternative implementation, the computational complexity, memory requirements, and communication overhead of the computing units are standardized to eliminate differences in the magnitude of the indicators before being input into a dynamic clustering algorithm. The algorithm automatically identifies cluster centers in the data, using the distance between the computing unit and the cluster center as an adaptive threshold parameter, without the need for a preset fixed value. The computing unit is assigned based on distance: those closer to computationally intensive cluster centers are classified as computationally intensive, and those closer to memory- and communication-intensive cluster centers are classified as the corresponding type. The execution order of similar computing units is then checked, prioritizing the continuity of resource requirements when merging adjacent units. If a certain type of unit combination is too large, it is split into multiple sub-combinations according to the resource requirement gradient. Finally, the adaptability of each sub-combination to the hardware characteristics is verified, the resource configuration of the super-boundary sub-combinations is adjusted, and a sub-model module that meets the adaptability criteria is generated. This method adapts to different indicator distributions through dynamic thresholds, resulting in more accurate type classification and strong module adaptability.

[0034] Step S30: Based on the deployment strategy, match and deploy the sub-model module with heterogeneous hardware nodes, and record the status information of each node after deployment.

[0035] In this embodiment, the deployment strategy refers to the set of rules guiding the matching of sub-model modules with heterogeneous hardware nodes, including sub-model module type adaptation logic, node reuse conditions, and exception handling mechanisms. Heterogeneous hardware nodes refer to computing nodes composed of different types of computing hardware, each with differentiated computing capabilities, storage capacity, and communication efficiency. Status information refers to the operational data of each heterogeneous hardware node after deployment, covering computing resource utilization, storage resource usage, communication resource usage, and task queue status.

[0036] As an optional implementation, based on the preset adaptation rules of module type and hardware type in the deployment strategy, computationally intensive sub-model modules are targeted to hardware nodes with high parallel computing capabilities, and memory-communication intensive sub-model modules are targeted to hardware nodes with memory management or communication optimization capabilities. For each module to be deployed, it is first checked whether there are already nodes with the same module deployed under the target hardware type. If the task queue length of the node does not exceed the preset reuse threshold, the current module is directly deployed to that node to achieve resource reuse. If there are no nodes with the same module under the target hardware type or the node queue exceeds the threshold, an idle node is randomly selected from the candidate nodes of that hardware type to complete the deployment. If insufficient node resources or communication abnormalities are encountered during the deployment process, the system directly switches to the next candidate node of the same type until the deployment is completed. After all deployments are completed, the computational utilization, video memory / memory usage, and task queue length of each node are collected, and status information is generated by classifying nodes. This method has a simple deployment logic, does not require complex load analysis, has a fast deployment speed, and can quickly complete the initial matching of modules and nodes.

[0037] Step S40: The state information and the state indicators in the historical performance data are processed by exponential smoothing prediction, and the dynamic weight value of each node is calculated by combining the bottleneck sensitivity coefficient.

[0038] In this embodiment, exponential smoothing prediction refers to a method of smoothing time-series state indicators to reduce short-term fluctuations and capture trends. It highlights the latest changes in indicators by assigning higher weights to recent data. Historical performance data refers to the accumulated state indicator records of heterogeneous hardware nodes over a period of time, reflecting the long-term operating patterns and performance characteristics of the nodes. State indicators are specific parameters describing the node's operating state, including computational utilization, memory usage, network bandwidth usage, and task processing latency. The bottleneck sensitivity coefficient is a parameter used to amplify the impact of potential resource bottlenecks on weights, making weight calculation more focused on node states that may constrain performance. The dynamic weight value is a value calculated based on the node's real-time state and historical performance, representing the priority of the node's adaptation sub-model module; a higher value indicates a higher adaptation priority.

[0039] As an optional implementation, core status indicators such as computational utilization, memory usage, and network bandwidth usage are first extracted from status information and historical performance data, and time-series data are organized by node. Each indicator is then subjected to exponential smoothing prediction using a fixed smoothing coefficient. By weighting recent and historical indicator values ​​with a fixed ratio, short-term fluctuations are eliminated to obtain stable indicator trend values. Next, a bottleneck sensitivity coefficient is calculated based on a preset bottleneck factor: if the trend value of an indicator approaches a preset hardware bottleneck threshold, the coefficient increases by a fixed ratio; otherwise, it remains at its base value. Subsequently, the smoothed indicator trend value is multiplied by the corresponding bottleneck sensitivity coefficient. The weight ratios of computational utilization, memory usage, and network bandwidth usage are preset according to node type. Finally, the results of each multiplication are summed to obtain the dynamic weight value of each node, ensuring that the weights can initially reflect the node's adaptability and bottleneck risk. This method has a simple calculation logic, does not require real-time analysis of indicator fluctuations or statistical calculation of historical averages, and has a fast processing speed.

[0040] Step S50: Generate a node priority list based on the dynamic weight value, and allocate the hardware type and task ratio of the sub-model module according to the node priority list and the task allocation optimization mechanism to complete the adaptive scheduling of the sub-model module.

[0041] In this embodiment, the node priority list refers to a list sorted from high to low according to the dynamic weight values ​​of each node, clearly defining the order in which nodes are allocated to sub-model modules. The task allocation optimization mechanism refers to a set of rules guiding the allocation of sub-model modules, including hardware type adaptation logic, task ratio adjustment strategies, and load balancing measures. Hardware type refers to the category of heterogeneous hardware nodes; different types of hardware differ in computing, storage, and communication capabilities. Task ratio refers to the proportion of computational tasks assigned to each node for processing sub-model modules, determining the workload undertaken by each node. Adaptive scheduling refers to the process of dynamically adjusting the hardware allocation and task ratio of sub-model modules based on node priority and real-time load to ensure efficient system operation.

[0042] As an optional implementation, the dynamic weight values ​​of each node are sorted from high to low to generate a node priority list, which also indicates the hardware type of each node. Based on the hardware adaptation rules in the task allocation optimization mechanism, computationally intensive sub-model modules are preferentially allocated to high-priority GPU nodes in the list, and memory communication intensive modules are preferentially allocated to high-priority TPU or CPU nodes. For nodes with allocated modules, a task ratio is set according to the proportion of dynamic weight values; the higher the weight value, the higher the task ratio allocated, and this ratio remains fixed within the current scheduling cycle. If the task queue length of a node exceeds a preset threshold after allocation, the excess tasks are migrated to the next high-priority node of the same type according to the node priority list order, until all sub-model modules have completed hardware type matching and task ratio allocation, achieving adaptive scheduling. This method has simple scheduling logic and a quick process for node priority sorting and task ratio setting.

[0043] For example, refer to Figure 2 , Figure 2 This is the timing diagram for the heterogeneous resource adaptive scheduling in this application. In the adaptive scheduling scenario, the computing power monitoring module collects the status information (computing utilization, memory, etc.) of the computing node cluster according to time windows and returns the real-time status data. Simultaneously, it sends real-time status data and historical performance data to the adaptive scheduling decision module, which calculates dynamic weights (including smoothing prediction and bottleneck coefficients) and outputs node weight values. The scheduler generates a priority list based on the node weight values ​​and allocates tasks to the computing node cluster according to priority. Afterwards, the computing power monitoring module continuously pushes node status to the scheduler. The scheduler compares the status with the baseline value; if a bottleneck is triggered, it executes scheduling to reallocate tasks; otherwise, it maintains the current task allocation.

[0044] By employing a process that involves real-time data collection by the computing power monitoring module, precise calculation of dynamic weights by the adaptive scheduling decision module, and on-demand allocation and dynamic adjustment by the scheduler, the problems of uneven node load and low resource utilization in large-scale adaptive scheduling are resolved. This improves the utilization efficiency of heterogeneous hardware resources and the stability of large-scale adaptive scheduling.

[0045] Based on any of the above embodiments, in Embodiment 2 of this application, step S10 includes steps A11 to A13: Step A11: Traverse the computational units of the target large model and parse the structural information of the computational units.

[0046] In this embodiment, structural information refers to the details describing the internal composition and relationships of the computing unit, including unit type, sub-operations, input / output interfaces, connection relationships, etc.

[0047] As an optional implementation, this method starts with the top-level structure of the target large model and traverses it hierarchically from the overall to the local: first, the macroscopic structural partitions of the model are identified, then the computational unit groups within each partition are traversed, and finally, individual computational units within each group are accessed one by one. For each computational unit, the type of each computational unit is first determined, then the sub-operations contained within it and the execution order of each sub-operation are parsed, then the interface features of input and output are extracted, and finally the connection relationships between this computational unit and other computational units before and after it are recorded. This information is organized by unit number to form a complete set of structural information. This method is logically rigorous, can comprehensively cover various structural information of computational units, and has high parsing completeness, clearly reflecting the hierarchical relationships of the model.

[0048] Step A12: Based on the structural information, calculate the computational complexity, peak memory requirement, and total communication transmission volume of each computing unit according to the preset index calculation method.

[0049] In this embodiment, the preset index calculation method refers to pre-defined rules and procedures for calculating computational complexity, peak memory requirement, and total communication transmission volume. Computational complexity refers to the amount of data required for a computing unit to complete a computational task. Peak memory requirement refers to the maximum amount of memory occupied by the computing unit during runtime. Total communication transmission volume refers to the total amount of data transmitted between the computing unit and other units.

[0050] As an optional implementation, complete structural information of the computing units is extracted, covering all computing units, input / output dimensions, and connection relationships. When calculating computational complexity, the computational load of each computing unit is statistically analyzed and accumulated. When calculating peak memory requirements, the memory usage of weight parameters, activation values, and temporary caches is statistically analyzed, and the maximum value is determined by combining the memory allocation timing. When calculating total communication transmission volume, the forward and reverse data transmission volumes with preceding and following units are statistically analyzed and accumulated based on connection relationships. All calculations strictly follow preset subdivision rules to ensure that each indicator is based on full structural information, ultimately outputting the computational complexity, peak memory requirements, and total communication transmission volume for each computing unit. This method, based on full structural information, has high indicator accuracy and can accurately reflect the true resource requirements of the computing units.

[0051] Step A13: Summarize and analyze the computational complexity, peak memory requirement, and total communication transmission of the computing unit to generate the computational complexity, memory requirement, and communication overhead corresponding to the computing unit.

[0052] As an optional implementation, raw data on computational complexity, peak memory requirements, and total communication transmission for each computing unit are collected, and weights for the three indicators are assigned according to the type of computing unit. For each indicator, a deviation coefficient is calculated based on historical data of similar computing units, and the original value is corrected using the deviation coefficient to eliminate abnormal fluctuations. Subsequently, the three corrected indicators are weighted and summed according to the set weights, and then standardized to generate the computational complexity, memory requirements, and communication overhead for each computing unit, ensuring that the three values ​​can be directly used for subsequent clustering comparisons. This method, through weighted calculation and calibration with historical data, can accurately eliminate abnormal fluctuations in the original indicators, and the generated computational complexity, memory requirements, and communication overhead are highly consistent with the actual characteristics of the computing units.

[0053] For example, in an adaptive computing power allocation scenario, all computational units of the target large model (such as a multimodal dialogue large model) are traversed, and the structural information of each computational unit (including sub-operation types, input / output dimensions, and inter-unit connection relationships) is parsed one by one. Based on the parsed structural information, according to a preset index calculation method, the computational complexity (such as the total number of matrix operation steps in the self-attention computational unit), peak memory requirement (such as the maximum memory usage of weight parameters, activation values, and temporary data during the convolution computational unit), and total communication transmission (such as the total transmission of forward data and backward gradients between cross-layer computational units) of each computational unit are summarized and analyzed. The computational complexity, peak memory requirement, and total communication transmission of each computational unit are then combined with historical data of similar computational units to correct for abnormal deviations, generating standardized computational complexity, memory requirement, and communication overhead for each computational unit.

[0054] By using structured analytical computing units and precise derivation of resource requirement indicators, the problem of fuzzy and large error in the assessment of computing unit resource requirements before adaptive scheduling of large models is solved, thereby improving the accuracy of adaptive scheduling and the utilization efficiency of heterogeneous hardware resources.

[0055] Based on any of the above embodiments, in Embodiment 3 of this application, step S20 includes steps B11 to B14: Step B11: Call the static analysis dataset containing the computational complexity, memory requirements and communication overhead of each computing unit, and set the threshold parameters of the three types of indicators according to the threshold processing method.

[0056] In this embodiment, the static analysis dataset refers to the dataset generated after static analysis of the units, containing three core metrics: computational complexity, memory requirements, and communication overhead for each computing unit. The threshold processing method refers to the rules and procedures used to determine threshold parameters, including different logics such as data statistics and hardware adaptation. The three metrics—computational complexity, memory requirements, and communication overhead—are the core basis for setting threshold parameters.

[0057] As an optional implementation, a static analysis dataset is invoked to extract computational complexity, memory requirements, and communication overhead data for all computing units, and statistical characteristics of the three types of indicators are calculated respectively. For computational complexity, the mean plus one variance is used as a threshold parameter to filter out units with computational loads significantly higher than the average level. For memory requirements, the upper quartile is used as a threshold parameter to cover the upper limit of memory requirements for most units. For communication overhead, the lower quartile is used as a basic threshold, and then fine-tuned based on the dispersion of the indicator distribution to avoid thresholds that are too high or too low. After setting, a preset proportion of computing units in the dataset is randomly selected for verification. If the proportion of abnormal units in the threshold classification results of a certain type of indicator exceeds the preset proportion, the calculation logic of the statistical characteristics is readjusted until the threshold can reasonably classify the types of computing units. This method does not rely on hardware parameters, but automatically sets the thresholds based solely on the statistical characteristics of the dataset, making it highly versatile and adaptable to different deployment environments.

[0058] Step B12: Using the threshold parameter as the criterion, analyze each computational unit in the static analysis dataset one by one, associate the computational unit with the corresponding type label, and obtain a computational unit classification set.

[0059] In this embodiment, the type label refers to an identifier used to identify the type of computing unit, which matches the characteristics of the computing unit. The computing unit classification set refers to the set formed by integrating all computing units with associated type labels, which facilitates subsequent clustering and integration.

[0060] As an optional implementation, a priority order of three types of indicators is first set. The three types of indicator data for the first computational unit are extracted sequentially from the static analysis dataset. The computational complexity of the computational unit is compared with the corresponding threshold parameter first: if the computational complexity is higher than the threshold, it is directly associated with a computationally intensive type label. If it is lower than the threshold, the memory requirement is compared with the corresponding threshold; if it is higher, it is associated with a memory-sensitive type label. If the memory requirement is also lower than the threshold, the communication overhead is compared with the corresponding threshold; if it is higher, it is associated with a communication-intensive type label; if it is lower, it is associated with a general type label. After classifying the first computational unit, subsequent computational units are extracted sequentially using the same process, repeating the indicator comparison and label association operations until all computational units are processed. All labeled computational units are then integrated to obtain a computational unit classification set. This method has a simple classification logic, requires no complex calculations, only compares indicators sequentially according to a fixed priority, has a fast processing speed, and low data processing resource requirements.

[0061] Step B13: Traverse the set of computing unit classifications according to the execution order of the computing units, identify adjacent target computing units with the same type label, and generate two types of preliminary sub-model modules.

[0062] In this embodiment, the execution order of computational units refers to the logical sequence in which computational units are executed within the target large model, reflecting the runtime flow of the model. The target computational unit refers to the computational units identified as adjacent and having the same type label during the traversal process; these are the objects to be merged. The preliminary sub-model module refers to a model fragment with a preliminary structure generated by merging the target computational units, which has not yet undergone final optimization.

[0063] As an optional implementation, the set of computational unit categories is traversed according to the execution order of the computational units. Starting from the first computational unit, its type label is recorded. Then, it is checked whether the type label of the next adjacent computational unit is consistent with the current record. If they are consistent, the unit is included in the current sequence to be merged. If they are inconsistent, the merging of the current sequence is stopped, and all the included computational units are integrated into a preliminary sub-model module and classified into the corresponding type according to their type labels. Then, starting from the inconsistent unit, the above checking and merging process is repeated. After the traversal is completed, all consecutive target computational units of the same type are merged into preliminary sub-model modules, ultimately generating two types of preliminary sub-model module sets. Each module contains consecutively executed computational units of the same type. This method strictly preserves the consecutive execution sequence of computational units, and the unit logic within the module is highly coherent, conforming to the original operation flow of the model.

[0064] Step B14: Verify whether the resource requirements of the preliminary sub-model module meet the basic adaptation range of heterogeneous hardware nodes. Perform secondary splitting and adjustment on the preliminary sub-model modules that exceed the adaptation range to obtain multiple sub-model modules.

[0065] In this embodiment, the basic compatibility range of heterogeneous hardware nodes refers to the upper and lower limits of resource requirements that different types of heterogeneous hardware nodes can support, determining whether a module can be matched with the hardware. Secondary splitting and adjustment refers to the operation of further splitting the initial sub-model modules that exceed the basic hardware compatibility range to ensure their resource requirements meet the compatibility requirements.

[0066] As an optional implementation, the basic adaptation range of all heterogeneous hardware nodes is collected. The adaptation range of the corresponding hardware is matched according to the type label of the preliminary sub-model module, and the minimum adaptation resource unit of the hardware is determined. The resource requirements of each preliminary module are extracted and compared with the matched adaptation range. If all three indicators are within the range, it is directly retained as a sub-module to be confirmed. If any indicator exceeds the upper limit, it is split from the middle position of the execution order of the calculation units within the preliminary module, ensuring that the resource requirements of the first sub-module after splitting do not exceed the adaptation upper limit and are not less than the minimum adaptation unit. The resource requirements of the split sub-modules are verified again. If the remaining part still exceeds the range, the splitting continues based on the minimum adaptation unit until the resource requirements of all sub-modules meet the adaptation range. Finally, all compliant sub-modules are integrated to obtain multiple sub-model modules. This method splits based on the minimum hardware adaptation unit, allowing sub-modules to accurately match the minimum carrying capacity of the hardware, reducing the likelihood of resource shortages during subsequent deployment and ensuring high adaptation stability.

[0067] For example, refer to Figure 3 , Figure 3 This diagram illustrates the model segmentation for heterogeneous hardware awareness in this application. In an adaptive computing power allocation scenario, a static analysis dataset containing the computational complexity, memory requirements, and communication overhead of each computing unit is invoked. Threshold parameters for these three metrics—computational complexity, memory requirements, and communication overhead—are set using a thresholding method. Based on these threshold parameters, each computing unit in the static analysis dataset is analyzed one by one, and the computing units are associated with "computation-intensive" or "memory-communication-intensive" type labels to obtain a set of computing unit classifications. This set is traversed according to the execution order of the computing units, identifying adjacent target computing units with consistent type labels, and classifying them into two types: "computation-intensive preliminary sub-model module" and "memory-communication-intensive preliminary sub-model module." Verify whether the resource requirements of the initial sub-model module are compatible with the basic adaptation range of heterogeneous hardware nodes such as GPU and TPU / CPU. For the initial sub-model module that exceeds the adaptation range, perform secondary splitting and adjustment to obtain multiple sub-model modules. Subsequently, query the GPU node resources or TPU / CPU node resources to determine the node status. If the queue of similar modules has not exceeded the threshold, reuse the node; otherwise, select a low-load node for deployment. Finally, record the initial load status of the node.

[0068] By accurately classifying computing units, generating sub-model modules adapted to hardware, and dynamically verifying and splitting them, the problem of inefficient adaptive scheduling caused by ambiguous sub-model module type identification and poor resource adaptability is solved, and the module preparation and hardware adaptation of adaptive scheduling are optimized.

[0069] Based on any of the above embodiments, in Embodiment 4 of this application, step S30 includes steps C11 to C13: Step C11: Based on the deployment strategy, prioritize matching recommended nodes for each sub-model module. If the target node has already deployed similar modules and the task queue length does not exceed a preset threshold, mark the target node as a reuse object and output a preliminary deployment matching list.

[0070] In this embodiment, recommended nodes refer to heterogeneous hardware nodes that are adapted to the sub-model module type and prioritized for matching according to the deployment strategy. Target nodes refer to specific heterogeneous hardware nodes selected from recommended nodes and whose reusability needs to be verified. The task queue length refers to the number of module tasks currently being executed on the target node, reflecting the node's task saturation level. The preset threshold is a pre-set upper limit for the target node's task queue length; exceeding this limit prevents the receiving of new tasks. Reusable objects refer to target nodes that have been verified and meet the conditions to continue supporting the current sub-model module. The preliminary deployment matching list is a list recording the matching relationships between sub-model modules and corresponding reusable objects, providing a basis for subsequent formal deployment.

[0071] As an optional implementation, the mapping relationship between sub-model module types and recommended node types in the deployment strategy is first loaded. The type of the first sub-model module is extracted sequentially, and all recommended nodes corresponding to that type are filtered out to form a recommended node list. The first target node is selected from the recommended node list, and the types of modules deployed on that node are queried to determine whether there are similar modules. If so, the current task queue length of the node is obtained and compared with a preset threshold. If the queue length does not exceed the threshold, the target node is immediately marked as a reuse object, and the matching relationship between the module and the reuse object is written into the preliminary deployment matching list. Then, the next sub-model module is processed. If the target node has no similar modules or the queue exceeds the threshold, the next target node in the recommended node list is selected for repeated verification until a reuse object that meets the conditions is found. If there are no nodes that meet the conditions in the recommended node list, the module is recorded as a candidate node to be matched, and subsequent modules are processed. Finally, all records are integrated and the output list is completed. This method verifies the similar modules and queue status of the target node in real time for each module, and the matching results are accurate.

[0072] Step C12: Based on the preliminary deployment matching list, select nodes with low load and sufficient resources from the candidate node pool, allocate them to modules that do not match the reuse object, and output the deployment results of the sub-model module and the heterogeneous hardware node.

[0073] In this embodiment, the candidate node pool refers to a set of heterogeneous hardware nodes that are not included in the initial recommended node range and can be used as backups. Modules without matching reuse objects refer to sub-model modules that have not found reusable nodes in the initial deployment matching list and are awaiting allocation. The deployment result refers to the final result that clarifies the matching relationship between all sub-model modules and their corresponding heterogeneous hardware nodes, and is used for subsequent formal deployment.

[0074] As an optional implementation, all unmatched reusable object sub-model modules are extracted from the initial deployment matching list and categorized by module type. The candidate node pool is accessed, and the real-time load of each node is collected, while simultaneously recording whether the remaining resources of each node meet the resource requirements of the corresponding module type. For the first category of unmatched modules, verification begins with the candidate node with the lowest load: if the remaining resources of that node meet the module's requirements, the module is assigned to that node, and the matching relationship is recorded. If not, the next node with a lower load is verified sequentially until the module finds a suitable node. Other types of unmatched modules are processed in the same way. After all modules have been assigned, the matching relationships in the initial list and the newly assigned relationships are integrated, and the complete deployment result is output. This method has a simple filtering logic, only requiring module verification after sorting by load, without complex calculations, and has a fast allocation speed.

[0075] Step C13: Collect the running data of each bound node, and store the running data in the monitoring system according to the node classification to obtain the status information of each node.

[0076] In this embodiment, a bound node refers to a heterogeneous hardware node that has established a matching relationship with a sub-model module and carries out the module's running tasks. Node classification storage refers to the operation of classifying and storing the running data of corresponding nodes according to the type or unique identifier of the bound node. The monitoring system refers to a system used to receive, store, and manage running data, and to support the subsequent generation and querying of status information.

[0077] As an optional implementation method, the acquisition parameters are first configured, a fixed acquisition cycle is set, and the types of operational data to be acquired for each bound node are clearly defined. Data acquisition is triggered according to the acquisition cycle. For each bound node, all preset types of operational data are collected synchronously. During the acquisition process, data integrity is verified in real time. If the data is complete, it is immediately classified and marked according to the node's unique identifier. If data is missing, a supplementary acquisition mechanism is triggered, and after supplementary acquisition, it is classified and marked again. The classified and marked operational data is transmitted to the monitoring system in real time. An independent storage directory is created in the system according to the node ID, and the operational data of the corresponding node is written to the directory in real time. After each acquisition cycle, the monitoring system automatically summarizes the operational data of the nodes in that cycle and generates real-time status information for the nodes, ensuring that the status information is updated synchronously with the actual operation of the nodes. This method has a high acquisition frequency, strong data real-time performance, can capture changes in the node's operational status in real time, and the status information is highly synchronized with the actual operation.

[0078] For example, in an adaptive computing power allocation scenario, based on the deployment strategy, each sub-model module is preferentially matched with corresponding recommended nodes. For each module, it is checked whether the target node in the recommended nodes has already deployed similar modules, and whether the task queue length of the target node does not exceed a preset threshold. If both conditions are met, the target node is marked as a reuse object. After all modules are matched, a preliminary deployment matching list is output. Based on this preliminary deployment matching list, modules that do not match reuse objects are filtered out. The candidate node pool is accessed, and the load status and remaining resources of the nodes in the pool are collected. Nodes with low load and sufficient resources are selected and allocated to these unmatched modules. The matching relationships in the preliminary list and the new allocation relationships are integrated, and the deployment results of the sub-model modules and heterogeneous hardware nodes are output. The binding nodes of all allocated modules are determined, and the running data of each binding node is collected at a preset frequency. The running data is stored in the monitoring system according to the node classification. The stored data is summarized and organized to obtain the status information of each node.

[0079] By prioritizing the reuse of similar module nodes, accurately allocating resources from the alternative pool, and classifying and storing operational data, the system solves the problems of low deployment efficiency of sub-model modules, waste of node resources, and lagging status monitoring, providing real-time data support for adaptive scheduling and improving the stability of system operation.

[0080] Based on any of the above embodiments, in Embodiment 5 of this application, step S40 includes steps D11 to D15: Step D11: Parse the status indicators corresponding to each node from the status information and historical performance data of each node.

[0081] As an optional implementation, the status information and historical performance data of all nodes are loaded, and the data is split according to three dimensions: computation, storage, and communication. The computation dimension includes fields such as computation utilization and task processing rate; the storage dimension includes fields such as video memory / memory usage and storage read / write speed; and the communication dimension includes fields such as communication bandwidth utilization and data transmission latency. All data fields under each dimension are iterated through one by one, extracting the specific value of each field and marking the data source and collection time. The extracted values ​​are then validated: outliers exceeding reasonable ranges are removed, duplicate records are merged, and finally, the data is grouped by node, integrating the validated values ​​from each dimension into a complete set of status indicators for that node. This method, with its full-dimensional traversal, can cover all potential status indicators without data omission and can meet diverse subsequent analysis needs.

[0082] Step D12: Perform adaptive exponential smoothing prediction on the state indicators corresponding to the nodes, dynamically adjust the smoothing coefficient according to the variance of the state indicators in the most recent time window, and output the smoothed prediction value of each state indicator.

[0083] In this embodiment, adaptive exponential smoothing forecasting refers to a method that dynamically adjusts the smoothing coefficient based on the fluctuations of the state indicator to predict the indicator trend, balancing real-time fluctuations with historical patterns. The most recent time window refers to a selected continuous time interval used to extract recent data of the state indicator, reflecting its short-term change characteristics. The smoothing coefficient is a parameter used in exponential smoothing forecasting to adjust the weight of recent and historical data; a larger coefficient emphasizes recent data. The smoothed forecast value is the state indicator forecast result after adaptive exponential smoothing calculation, eliminating short-term random fluctuations and reflecting the indicator trend.

[0084] As an optional implementation, a short-duration recent time window is set for each status indicator, and all historical data within the window is extracted. The data variance is calculated; if the variance exceeds a preset fluctuation threshold, the smoothing coefficient is adjusted upwards to a preset high range to prioritize the retention of recent data characteristics. If the variance is below the threshold, the smoothing coefficient is adjusted downwards to a preset low range to highlight historical data patterns. After the coefficient is determined, the latest data within the window is integrated with the previous smoothing value according to exponential smoothing logic to calculate the current smoothed predicted value. After each data collection, the most recent time window data is immediately updated, the variance and smoothing coefficient are recalculated, and the smoothed predicted value is refreshed in real time to ensure synchronization with indicator fluctuations. This method can accurately capture sudden fluctuations in indicators, and the predicted value closely matches the actual trend.

[0085] Step D13: Based on the smoothed predicted value of the status indicator, combined with the average value of the status indicator over multiple past monitoring periods, and a preset bottleneck amplification factor, calculate the bottleneck sensitivity coefficient corresponding to the status indicator.

[0086] In this embodiment, the average of multiple past monitoring periods refers to the average value of all data of the status indicator within the set historical monitoring period, reflecting the long-term stability level of the indicator. The bottleneck amplification factor is a pre-set coefficient used to amplify the impact of the status indicator when it approaches a bottleneck, thereby enhancing the sensitivity to bottleneck risks.

[0087] As an optional implementation, the system categorizes status indicators into groups and sets a basic bottleneck amplification factor for each group. For each status indicator, its smoothed predicted value is extracted and its mean value over multiple past monitoring periods is calculated. The deviation between the predicted value and the mean is calculated. If the deviation is positive and the predicted value exceeds the first bottleneck threshold of the hardware capability, the amplification factor is increased by a first preset multiple of the basic factor. If the deviation is negative or the predicted value is lower than the second bottleneck threshold of the hardware capability, the amplification factor is decreased by a second preset multiple of the basic factor. If the value is in the middle range, the basic factor remains unchanged. The adjusted amplification factor is multiplied by the deviation value, and then the baseline coefficient corresponding to the mean is added to obtain the bottleneck sensitivity coefficient of the indicator. This method sets the basic factor according to the indicator type, which conforms to the bottleneck impact characteristics of different types of indicators, and the calculation logic fits the hardware operation rules.

[0088] Step D14: The smoothed predicted values ​​of each state indicator are weighted and combined with the corresponding bottleneck sensitivity coefficients to obtain the weighting coefficients.

[0089] In this embodiment, the weighting coefficient refers to a quantitative parameter obtained by weighted combination that comprehensively reflects the trend of status indicators and bottleneck risks, and is used for subsequent node priority ranking or scheduling decisions.

[0090] As an optional implementation, a fixed weighting percentage for predicted values ​​and a fixed weighting percentage for sensitivity coefficients are first assigned to each state indicator based on its impact on node operation. For each state indicator, its smoothed predicted value and bottleneck sensitivity coefficient are extracted, and multiplied by their corresponding fixed weighting percentages to calculate the local weighted result of that indicator. After the local weighted results of all indicators have been calculated, all local values ​​are summed to obtain the initial weighting coefficient of the node. The initial coefficients are then proportionally calibrated against the node's hardware capability benchmark to ensure the comparability of weighting coefficients for nodes with different hardware types, ultimately outputting standardized weighting coefficients. This method sets fixed weights based on indicator importance, has clear logic, is simple to calculate, and the weighting coefficients stably reflect the core capabilities of the node.

[0091] Step D15: The weight coefficients are automatically matched according to the node type and sub-model module characteristics, and the dynamic weight values ​​of each node are calculated and adjusted.

[0092] As an optional implementation, nodes are first categorized by type, and a basic adaptation coefficient is preset for each type of node to match the characteristics of different modules. The characteristics of the sub-model modules to be scheduled are obtained, and matching node types are selected according to the adaptation rules. The initial weight coefficients of all nodes of that type are extracted. The initial weight coefficient of each node is multiplied by the corresponding basic adaptation coefficient to obtain a preliminary adjustment value. Then, based on the detailed requirements of the module characteristics, the computing power parameters of the nodes are queried. If the node's computing power meets the detailed requirements, a compensation coefficient with a preset coefficient is added; otherwise, it is not added. Finally, the preliminary adjustment value and the compensation coefficient are integrated to obtain the dynamic weight value of the node. The weight values ​​of nodes of non-matching types are adjusted downwards according to the basic adaptation coefficient and used as alternatives. This method selects matching objects by node type, has clear logic, requires no complex calculations in the adjustment process, is highly efficient, and has strong adaptation stability.

[0093] For example, in an adaptive computing power allocation scenario, the target large model is divided into several sub-model modules at a granularity of at least layers or operators. The division of these sub-model modules is based on, but is not limited to, the computational complexity, memory requirements, and communication overhead of each layer or operator. The specific process of dividing the target large model into sub-model modules according to layers or operators includes: performing static analysis on the target large model to extract the computational complexity, memory requirements, and communication overhead of each layer or operator. Computational complexity (FLOPs) is calculated based on the number of floating-point operations per layer type. .in, Presentation layer Number of sub-operations included. It is the first The number of floating-point operations per sub-operation. For example, for a convolutional layer: ,in, , These represent the number of input and output channels, respectively. , The height and width of the convolution kernel, , To determine the output feature map size, multiply by 2 to represent a multiply-accumulate operation. Peak memory requirement: Using peak memory usage as an indicator, estimate the peak memory usage of this layer during execution. ; in, It refers to the memory size of the weight parameters. It is the video memory occupied by the activation value. It is the video memory occupied by temporary cache or intermediate data during the execution process.

[0094] Communication overhead, i.e., the amount of data transmitted between operators, is calculated by measuring the total amount of data transmitted between this layer and its adjacent layers.

[0095] in, Indicates from layer Output to layer Input feature map data volume (unit: bytes).

[0096] Based on the static analysis results, layers or operators with high computational complexity and low memory requirements are classified as computationally intensive sub-model modules, while layers or operators with high memory requirements and high communication overhead are classified as memory-communication intensive sub-model modules. Based on the extracted indicators, the following threshold parameters are set to achieve classification: computational complexity threshold. For example, the average or median of FLOPs across all layers. Memory requirement threshold. For example, a certain percentile of the peak memory usage. Communication overhead threshold. For example, the average amount of data transferred between layers. Layers or operators are divided into two types of sub-model modules according to the following rules: Computationally intensive sub-model modules: satisfying... This refers to layers with high computational demands but relatively low memory usage, suitable for devices adept at high-parallel computing. Memory-communication intensive sub-model module: satisfies... Layers with high memory requirements or large data transfer overhead are suitable for placement on nodes with stronger memory management and communication optimization capabilities. After traversing and classifying all layers based on the above conditions, adjacent layers of the same type are merged to generate continuous sub-model modules, avoiding excessive communication caused by overly fine segmentation.

[0097] Based on the type of sub-model modules, suitable heterogeneous hardware nodes are matched to provide a basis for subsequent deployment on heterogeneous hardware nodes. Computation-intensive modules are prioritized for deployment on GPU nodes with high parallel computing capabilities, making full use of their powerful CUDA cores and parallel computing resources; memory- and communication-intensive modules are prioritized for deployment on TPU nodes (optimized for matrix operations and memory access) or CPU nodes (more flexible memory management and communication scheduling capabilities). This matching method maximizes the hardware adaptability of various sub-model modules and improves overall inference performance and resource utilization. By performing systematic static analysis on the layers or operators of the large model, key computational indicators (FLOPs, peak memory requirements, communication overhead) are extracted. Sub-model module types are divided with reasonable thresholds, and precise matching is achieved in combination with hardware characteristics, forming a preliminary division of sub-model modules, providing a foundation for subsequent dynamic scheduling and heterogeneous perception monitoring. The model is divided into several computation-intensive and memory- and communication-intensive sub-model modules, and the sub-model modules are deployed in a heterogeneous hardware node cluster composed of GPUs, TPUs, and CPUs. Each computing node has at least a computing unit, a storage unit, and a communication unit, and the nodes are connected through a network to form a distributed architecture. The specific process of deploying sub-model modules to heterogeneous hardware nodes includes: establishing a deployment rule base based on the type of sub-model module and the hardware characteristics of the computing nodes. The rule base comprehensively considers the high parallel computing capabilities of GPUs, the matrix multiplication acceleration advantages of TPUs, and the general computing and memory management capabilities of CPUs. At the same time, it combines static analysis indicators such as computational complexity, memory requirements, and communication overhead extracted from S1 of the sub-model module to clarify the priority node types and resource requirements of different types of sub-model modules, ensuring that the deployment scheme has a reasonable matching basis and resource estimation.

[0098] By analyzing status indicators from multiple dimensions, dynamically optimizing prediction logic, quantifying bottleneck risks, and accurately weighting and integrating features, the system solves the problems of one-sided node status evaluation, delayed bottleneck warning, and mismatch between weights and module requirements in adaptive scheduling, thus optimizing the accuracy of adaptive scheduling and the stability of system operation.

[0099] Based on any of the above embodiments, in Embodiment Six of this application, step S50 includes steps E11 to E13: Step E11: Based on the dynamic weight values ​​of each node and the matching degree between the characteristics of each heterogeneous hardware node and the type of the sub-model module, generate a node priority list for each node according to the dynamic weight values, which includes node priority and corresponding adaptive module type.

[0100] In this embodiment, heterogeneous hardware node characteristics refer to the core capability attributes of different types of heterogeneous hardware nodes. Sub-model module type matching degree refers to a parameter that quantifies the degree of compatibility between heterogeneous hardware node characteristics and sub-model module types; a higher matching degree indicates stronger compatibility. Adaptable module type refers to the sub-model module type that has a high degree of matching with the node characteristics and is suitable for the node to carry.

[0101] As an optional implementation, the dynamic weight values ​​of all nodes are extracted, and the nodes are initially sorted from high to low based on their dynamic weight values. For nodes with the same dynamic weight value in the initial sort, the matching degree between each node and the module type of the current sub-model to be scheduled is calculated. The nodes with the same weight are then sorted a second time from high to low based on their matching degree, completing the priority fine-tuning of nodes with the same weight. After sorting, for each node, the top two module types with the highest matching degree are selected as the adapted module types and marked next to the node. Finally, according to the adjusted priority order, the nodes, their corresponding priority levels, and adapted module types are entered into a list to generate a node priority list, ensuring that nodes with high dynamic weight values ​​are always in a high-priority position, and the matching degree is only used to optimize the sorting of nodes with the same weight. This method, with dynamic weight values ​​as its core, can prioritize the selection of nodes with better overall status.

[0102] Step E12: According to the node priority list, the sub-model module is preferentially assigned to the node with high weight and matching hardware type, and the initial task allocation scheme of the sub-model module is generated.

[0103] In this embodiment, the initial task allocation scheme refers to the preliminary task deployment plan formed after matching and allocating sub-model modules and nodes according to the node priority list, which has not yet undergone final optimization.

[0104] As an optional implementation, the first module is extracted from the sub-model modules to be assigned, and its type is determined. The node priority list is traversed, and nodes are checked sequentially from highest to lowest weight. First, it is determined whether the node's hardware type matches the module type, and then it is confirmed whether the node's current remaining resources can support the module. The first node that simultaneously satisfies the highest weight, hardware matching, and sufficient resources is found, and the module is assigned to this node. The correspondence between the module and the node is recorded, and the node's remaining resources are updated. Next, the second sub-model module is extracted, and the above traversal and matching process is repeated, processing modules one by one and filtering nodes by priority, to complete the assignment of all modules sequentially. After all modules have been assigned nodes, all matching relationships are organized to generate an initial task allocation scheme, ensuring that each module is assigned to the currently optimal single node. This method ensures that each module is assigned to the node with the highest weight and strongest adaptability, resulting in high module operational stability and accurate resource utilization.

[0105] Step E13: Based on the initial task allocation scheme, fine-tune the task ratio allocated to the sub-model module to complete the adaptive scheduling of the sub-model module.

[0106] In this embodiment, the task allocation ratio of a sub-model module refers to the percentage of the total task volume of a single sub-model module on its allocated node, which determines the scale of resource consumption of the module on the node.

[0107] As an optional implementation, the real-time resource utilization rate of all allocated nodes in the initial task allocation scheme is collected, and a reasonable range for resource utilization rate is set for each node. For the sub-model modules allocated on each node, the resource consumption corresponding to the current task ratio of each module is calculated. If the node resource utilization rate exceeds the upper limit of the reasonable range, the task ratio is adjusted in reverse according to the module's resource consumption ratio. The module with the highest resource consumption ratio has its task ratio reduced by a preset step size, and the reduced ratio is preferentially allocated to the adaptation modules with low resource consumption on that node. If the node resource utilization rate is lower than the lower limit of the range, the task ratio of the adaptation modules on that node is appropriately increased. After each adjustment, the node resource utilization rate is recalculated until the resource utilization rate of all nodes is within the reasonable range, completing the fine-tuning of the sub-model module task ratio and outputting the adaptive scheduling result. This method uses resource balancing as the core for fine-tuning the ratio, which can avoid node resource overload or idle waste.

[0108] For example, refer to Figure 4 , Figure 4This application presents a deployment strategy diagram for heterogeneous nodes. During deployment, computationally intensive sub-model modules are prioritized for allocation to GPU nodes, while memory-communication intensive sub-model modules are allocated to TPU or CPU nodes. If a target node already has a similar sub-model module deployed and its task queue length does not exceed a threshold, the node's hardware features are reused. If a target node already has a heterogeneous sub-model module deployed or its task queue length exceeds the threshold, other nodes are selected for deployment. When selecting a specific node, priority is given to determining whether the target node has already deployed a similar sub-model module and whether its current task queue length does not exceed a preset threshold. If this condition is met, the node's hardware resources are reused to improve utilization; otherwise, nodes with lower loads and sufficient resources are selected from the candidate node pool based on the node's real-time load and network bandwidth. The task queue length threshold and other resource limitations can be dynamically adjusted based on data feedback from the S3 computing power monitoring module to ensure system load balancing and efficient operation. Meanwhile, the deployment strategy includes an exception handling mechanism. When a target node experiences resource shortages or communication failures, it will automatically trigger a retry or switch to another node, ensuring the robustness and continuity of the deployment. After deployment, the system records the initial load status of each computing node in real time, covering key performance indicators such as computing utilization, memory usage, network bandwidth usage, and task queue length. All monitoring data is uniformly stored in a central monitoring system, supporting historical data query and analysis, facilitating dynamic load scheduling and predictive optimization. Through this process, efficient and dynamically adaptable heterogeneous hardware node deployment based on static analysis results is achieved, providing a foundation for subsequent real-time monitoring and scheduling.

[0109] By clearly prioritizing nodes, accurately matching modules and hardware types, and dynamically fine-tuning task ratios, the problems of disordered node selection, poor module and hardware compatibility, and uneven task load in adaptive scheduling are solved, thereby improving the scheduling efficiency and business service stability of heterogeneous clusters.

[0110] Based on any of the above embodiments, in Embodiment 7 of this application, after step S50, steps F11 to F14 are further included: Step F11: After completing the adaptive scheduling of the sub-model module, collect the running status of each heterogeneous hardware node, and compare the running status with the initial baseline status of the scheduled node to generate a status comparison result.

[0111] In this embodiment, the initial baseline state of the scheduled node refers to the initial state data of the heterogeneous hardware node when the sub-model module completes adaptive scheduling and just starts running, serving as the benchmark for subsequent comparisons. The state comparison result refers to the result set containing difference values ​​and fluctuation trends generated after comparing the real-time running state of the node with the initial baseline state index by index, used to evaluate the scheduling effect.

[0112] As an optional implementation, after the sub-model module completes adaptive scheduling, it immediately collects the core metrics of each heterogeneous hardware node as the initial baseline state of that node and stores it. Short-cycle collection rules are set to trigger the collection of real-time node operating status periodically, and after each collection, the metric fields consistent with the baseline state are extracted. For each node, the real-time metric values ​​are compared field-by-field with the initial baseline values, and the absolute difference and relative difference rate are calculated. Metrics with differences exceeding a preset threshold are marked as fluctuation metrics. A single-node real-time comparison record is generated after each comparison. When the number of collections reaches a preset threshold, the real-time records of all nodes are integrated, and the frequency of fluctuation metrics and the maximum difference value are statistically analyzed to form a state comparison result containing real-time fluctuation details. This method can quickly capture instantaneous fluctuations in node status, promptly identify short-term hardware adaptation problems after scheduling, and provide a basis for emergency optimization.

[0113] Step F12: Based on the state comparison results, the key indicators of each node are processed by time window sliding mean filtering to generate bottleneck determination results.

[0114] In this embodiment, the time window sliding mean filtering process refers to setting a continuous time interval and calculating the average value of key indicators within the window sequentially according to the sliding direction of the window, thus filtering out short-term random fluctuations. The bottleneck determination result refers to the result set of "bottleneck present / no bottleneck" determination conclusions and bottleneck types generated after comparing and analyzing the filtered key indicators with the preset bottleneck threshold.

[0115] As an optional implementation, key indicator time-series data for each node are first extracted from the state comparison results, with a fixed time window length and a fixed sliding step size. Indicator data within each window are selected sequentially according to the sliding direction, the mean value for each window is calculated, and the original data within that window is replaced to complete the filtering process. The filtered indicator mean value is compared with a preset bottleneck threshold: if the mean value of a certain indicator exceeds the threshold for a consecutive preset number of windows, the node is determined to have a corresponding bottleneck. If the mean value remains below the threshold, no bottleneck is determined. The judgment results of all nodes are integrated, and the bottleneck node ID, bottleneck type, and number of consecutive windows exceeding the threshold are labeled to generate a bottleneck judgment result. This method has a simple fixed-window calculation logic, a short filtering process, and low system resource requirements.

[0116] Step F13: For the nodes that trigger the bottleneck in the bottleneck determination results, start the local sub-model module to reallocate or optimize the node topology, and output the bottleneck processing results.

[0117] In this embodiment, node topology optimization refers to hardware topology adjustments that improve the node's operating environment by modifying the communication links and resource sharing methods between the bottleneck node and other nodes. Bottleneck resolution result refers to a report recording whether the bottleneck has been resolved and the node's state has been restored after the local module reallocation or node topology optimization is completed.

[0118] As an optional implementation, all allocated sub-model modules are extracted from the bottleneck node, categorized by module business priority, and idle nodes matching the bottleneck node type are queried from the node priority list. For non-core modules, they are directly migrated to idle nodes, and the module-node mapping is updated after migration. For core modules, they are not migrated, but their task proportion on the bottleneck node is reduced, and the reduced proportion is allocated to similar modules on idle nodes. After reallocation, the status of the bottleneck node and the target idle node is collected at preset intervals. If the key indicators of the bottleneck node drop below a threshold and remain below a preset collection period, the bottleneck is considered resolved. If not resolved, the migration of secondary priority core modules continues until the bottleneck is resolved. Finally, the number of migrated modules, the task proportion adjustment value, and the status recovery data are recorded to generate the bottleneck processing result. This method only operates on local modules, does not change the node hardware topology, and has simple processing logic and short processing time.

[0119] Step F14: Based on the bottleneck processing results, calculate the variance of resource utilization for all nodes, and calculate the global task latency average and global memory usage sum. If both conditions are met, maintain the current scheduling state, and output the judgment result of global load balancing and efficient resource utilization.

[0120] In this embodiment, the variance of resource utilization refers to a parameter that measures the dispersion of resource utilization across all nodes; a smaller variance indicates a more balanced resource utilization among nodes. The global task latency mean is the average execution latency of all sub-model module tasks, reflecting the overall task execution efficiency. The global total video memory usage is the sum of the current video memory usage of all heterogeneous hardware nodes, reflecting the scale of cluster video memory resource consumption.

[0121] As an optional implementation, the real-time resource utilization of all nodes after bottleneck processing is first collected, and the variance of this utilization data is calculated. The variance is compared with a preset load balancing threshold. If the variance exceeds the threshold, it is directly determined that load balancing has not been achieved, without further verification. If the variance meets the threshold, the task latency data of all sub-model modules is extracted, the global task latency mean is calculated, and compared with a preset latency efficiency threshold. If the latency mean exceeds the threshold, efficiency is determined to be substandard. If the latency mean meets the threshold, the memory usage of all nodes is finally calculated, the total global memory usage is calculated, and compared with a preset memory capacity threshold. If all three indicators meet the preset conditions, the current scheduling state is immediately determined to meet the requirements, scheduling is maintained, and the judgment result of global load balancing and efficient resource utilization is output. If any one of them is not met, the substandard indicator is recorded to provide direction for subsequent optimization. This method does not require waiting for multi-cycle data, has a fast judgment speed, and can instantly confirm the scheduling effect after bottleneck processing, making it suitable for scenarios with high real-time requirements.

[0122] For example, after deployment, the initial load status of each node is recorded as baseline data for real-time monitoring. The heterogeneous perception monitoring module collects the status information of each computing node in real time. This status information includes computing utilization, GPU memory usage, network bandwidth usage, task processing latency, and task queue length. The collection frequency can be dynamically adjusted according to a preset threshold to reduce monitoring overhead. The specific process of the computing power monitoring module collecting status information in real time includes: collecting computing utilization by obtaining real-time running data from the computing units of each node and calculating its average value within a preset time window. Collecting GPU memory usage by obtaining the GPU / RAM usage of each node in real time, including the current used GPU memory capacity, peak GPU memory capacity, and GPU memory fragmentation level. Collecting network bandwidth usage by statistically analyzing data transmission rate and packet loss rate through communication interfaces between nodes (such as PCIe bus, RDMA network). Collecting task processing latency by recording the end-to-end latency from input to output of the sub-model module and calculating its average and maximum values ​​within a preset time window. The task queue length is collected by statistically analyzing the queue depth of tasks to be processed on each node, including task waiting time and task priority distribution. After the computing power monitoring module is started, the system will monitor computing utilization, memory usage, network bandwidth usage, task processing latency, and task queue length in real time based on the recorded initial load status of each node and according to a preset collection frequency. Computing utilization is collected by the hardware driver layer of each node directly returning core running status data, such as the CUDA core utilization of the GPU, the matrix multiplication unit utilization of the TPU, and the thread utilization of the CPU, and these instantaneous utilization values ​​are displayed within a time window. A weighted average is then applied to obtain a smoothed utilization rate sequence. This calculation uses the following formula: ; in For the first Instantaneous utilization rate of the first acquisition. The weighting coefficients for the data collection time points are dynamically adjusted based on the task queue length and network bandwidth usage ratio of the nodes in S2. The data collection of video memory usage directly obtains the current video memory usage. Peak memory capacity and the fragmentation ratio of video memory And calculate the weighted memory utilization: ; The above formula not only measures video memory usage but also corrects for video memory availability through fragmentation ratio, ensuring that nodes with high usage but low fragmentation are not misjudged as bottlenecks. Network bandwidth usage relies on the high-speed communication interface between nodes to continuously collect data on transmission rate. With packet loss rate And calculate the effective bandwidth utilization: ; in The upper limit of node physical bandwidth and the packet loss rate correction term ensure that links with high packet loss are identified as having low effective bandwidth and are therefore prioritized for avoidance during scheduling. Task processing latency and task queue length are collected by recording end-to-end processing time and counting the number of pending tasks, respectively, and combined with their fluctuation trends within a time window, providing accurate real-time data for S4's dynamic load balancing and node migration strategies; among which... , and A multi-dimensional monitoring indicator system is established, covering the three core resources of computing, storage, and communication. The status information and historical performance data are input into the adaptive scheduling decision module. The historical performance data includes the computational efficiency of each node executing sub-model modules at different time periods, trends in GPU memory usage, and network communication stability. The adaptive scheduling decision module employs a dynamic weight adjustment strategy, dynamically calculating the weight value of each node. This weight value is generated by a weighted combination of real-time status information and historical performance data, with the weight coefficient automatically matched based on the node type (GPU, TPU, or CPU) and the characteristics of the sub-model module. The core indicators collected by the dynamic weight adjustment are... , , Real-time updates are performed. To balance stability and sensitivity, adaptive exponential smoothing is first applied to each indicator for prediction, resulting in: ; in, , The smoothed predicted value. This is a smoothing coefficient that is dynamically adjusted based on the volatility of the indicator. When the variance of the indicator increases within the most recent window, Increased to improve response speed to sudden changes. To enhance the ability to proactively address potential resource bottlenecks, a bottleneck sensitivity coefficient is introduced: ; in For this indicator in the past The average of the monitoring period, This is a bottleneck amplification factor. When the current trend of a certain indicator is significantly higher than its long-term mean, this dimension... This increases the priority of the resource in the weight allocation, thus preventing it from becoming a performance bottleneck. Ultimately, the dynamic weights of each monitoring metric in the scheduling process are calculated as follows: ; The weights calculated above not only reflect the current resource utilization status but also amplify the weights of resource dimensions that are about to become bottlenecks. This allows the scheduling strategy to proactively allocate computing resources in subsequent stages, prioritizing the mitigation of potential risks. This mechanism directly transforms multi-dimensional real-time monitoring results into intelligent scheduling inputs, achieving a closed-loop connection between indicator collection, trend prediction, and weight updates, providing highly sensitive priority references for scheduling decisions.

[0123] By using state comparison to locate fluctuations, filtering to accurately identify bottlenecks, local optimization to solve bottlenecks, and multiple indicators to verify the global situation, the problems of difficult bottleneck detection, inaccurate processing, and global load imbalance after adaptive scheduling have been solved, ensuring the efficient operation of heterogeneous clusters and the stability of business services.

[0124] Based on any of the above embodiments, in Embodiment 8 of this application, after step F14, steps G11~G13 are further included: Step G11: Collect the response delay of weight adjustment to bottlenecks, the node load balancing degree after task allocation, and the deviation of resource utilization to obtain the running feedback dataset.

[0125] In this embodiment, the bottleneck response latency refers to the time interval from the start of weight adjustment to the bottleneck removal in the bottleneck determination result, reflecting the speed at which the adjustment affects the bottleneck. Node load balancing after task allocation: the degree of load balance among all nodes after task allocation; a higher balance indicates a more uniform load distribution. Resource utilization deviation refers to the difference between the actual resource utilization of a node and the preset target utilization; a smaller deviation indicates that resource utilization is more in line with expectations. The runtime feedback dataset is a collection that integrates data such as the bottleneck response latency of weight adjustment, node load balancing, and resource utilization deviation, used to evaluate scheduling effectiveness and optimize subsequent strategies.

[0126] As an optional implementation, a timing mechanism is triggered immediately upon initiation of the weight adjustment operation, synchronously recording the adjustment time and the corresponding bottleneck type. When the bottleneck determination result indicates that the bottleneck has been resolved, the timing stops, and the difference between the two times is calculated as the response delay for that adjustment, stored immediately. After task allocation is completed, load data for each node is collected in real time, and the standard deviation of the load for all nodes is calculated as load balancing data. Simultaneously, the actual resource utilization rate of each node is compared with the preset target utilization rate in real time, and the absolute difference is calculated as the resource utilization deviation, recorded once every preset time interval. The response delay associated with the same adjustment event, the load balancing rate at the corresponding time point, and the resource utilization deviation are bound together, organized by event number, forming a runtime feedback dataset containing detailed data for each event. This method, by collecting data in real time for each event, can accurately record the feedback data of a single weight adjustment, retain detailed features, and trace the scheduling effect of specific events.

[0127] Step G12: Correlate the runtime feedback dataset, identify abnormal issues in the dynamic weight adjustment, quantify the impact of the task allocation optimization mechanism on performance, and output a problem analysis report.

[0128] In this embodiment, anomalies in dynamic weight adjustment refer to unexpected situations that occur after dynamic weight adjustment. The impact of the task allocation optimization mechanism on performance refers to the quantitative assessment of the specific degree to which task allocation optimization operations improve or decrease node performance. The problem analysis report is a report that records the type of anomaly, its cause, the quantitative results of its performance impact, and optimization suggestions, providing a basis for subsequent scheduling strategy iterations.

[0129] As an optional implementation, response latency, load balancing, and resource utilization deviation data corresponding to dynamic weight adjustments are extracted from the operational feedback dataset, and a normal threshold range is set for each indicator. The indicator data for each weight adjustment event is checked one by one; if a single indicator exceeds the threshold, it is marked as a single-indicator anomaly. If multiple indicators show correlation anomalies, they are marked as correlated indicator anomalies. When quantifying the impact of task allocation optimization, the difference and percentage of the difference between the optimized and unoptimized indicators are calculated to determine the degree of impact. The types and quantities of anomalies, the quantification results of their impact, and the preliminary causes are compiled, and a problem analysis report is output. This method, based on indicator thresholds and correlation rule analysis, is logically simple and easy to operate.

[0130] Step G13: Based on the problem analysis report, correct the dynamic weight value and task allocation optimization mechanism, update the optimized parameters to the adaptive scheduling decision module, and generate the optimized dynamic weight adjustment strategy and task allocation optimization mechanism parameter set.

[0131] In this embodiment, the dynamic weight adjustment strategy refers to the set of rules that guide the calculation and updating of dynamic weight values. The task allocation optimization mechanism parameter set refers to the set of parameters that support task allocation optimization operations, including task migration thresholds, topology optimization triggering conditions, etc.

[0132] As an optional implementation, specific anomalies are extracted from the problem analysis report, and corresponding correction parameters are matched for each problem. The correction magnitude is set according to the severity of the problem. If the anomaly causes a performance degradation exceeding a first preset threshold, the corresponding parameter adjustment magnitude is set between a first and a second percentage. If it exceeds the second preset threshold, the adjustment magnitude is set between a third and a fourth percentage. After correction, single-scenario simulated scheduling is used for verification: for the high response latency problem, the same weight adjustment operation is simulated to check if the response latency has decreased to within the threshold. For the low load balancing problem, task allocation is simulated to confirm that the load balancing meets the standard. After all problems are verified, the optimized dynamic weight calculation rules and task allocation optimization parameters are updated to the adaptive scheduling decision module, generating a dynamic weight adjustment strategy and parameter set that corresponds one-to-one with the problem and the correction parameters.

[0133] For example, in dynamic weights After the update is complete, the system first generates a priority list based on the current weight values ​​of each computing node, and then... The tasks are sorted from highest to lowest weight. When selecting execution nodes for sub-model modules, the scheduler prioritizes nodes with high weights and hardware characteristics that match the module type. For example, computationally intensive modules are preferentially assigned to GPU nodes, and memory-communication intensive modules are preferentially assigned to TPU / CPU nodes. When the optimal matching node approaches the queue saturation threshold, the scheduler will choose the next highest weight node with a relatively high matching degree to achieve the optimal solution between performance and load balancing. Regarding dynamic adjustment of the task queue, the system continuously monitors the queue length of each node. ,when Exceeding the preset limit At that time, the real-time allocation ratio of the node will be calculated according to the scaling factor. The load will be reduced, and excess tasks will be redistributed according to node weight. and Higher nodes; conversely, when When the value falls below a threshold, the node's task allocation ratio is temporarily increased to maximize resource utilization while ensuring manageable latency. This dynamic allocation mechanism, driven by weight values, creates a closed-loop connection between task allocation and the previous monitoring-prediction-weight adjustment process, ensuring real-time optimization of resource allocation based on system status. After task allocation ratio optimization, the system enters a continuous monitoring phase. The scheduler acquires the latest data stream of multi-dimensional monitoring indicators in real time and compares it with the baseline state of the allocated nodes. To avoid instantaneous fluctuations when determining whether a node needs to be reassigned, the system uses a time-window moving average filter for each key indicator, calculated using the following formula: ; in, They represent the computational utilization rate, respectively. Video memory usage Network bandwidth utilization and end-to-end delay , This is the length of the sliding time window. If or Then it is determined that the node has a computing power or memory bottleneck. If or If a communication bottleneck is detected, the system initiates a local reallocation strategy when one of the above conditions is triggered. This strategy only reallocates execution nodes to sub-model modules that have not triggered a bottleneck, reducing migration overhead and maintaining overall topology stability. When the bottleneck is caused by communication factors, the scheduler adjusts the topology, prioritizing the migration of communication-intensive modules to node pairs with lower physical latency, thereby reducing cross-node data transmission costs. After reallocation and topology optimization, the system assesses the global operating status to confirm whether the heterogeneous hardware node cluster has achieved load balancing and efficient resource utilization. Load balancing is determined by calculating the variance of node utilization. Completed, when At this time, it is assumed that the computing power allocation is balanced. The determination of efficient resource utilization is based on the joint constraints of overall task latency and memory utilization. When the global latency average is... Less than the reference delay And the total global video memory usage Below the system limit When the system reaches a certain threshold, it is considered to have achieved efficient utilization. This determination process monitors key operational indicators such as computing utilization, memory usage, and network bandwidth of each computing node in real time. Combined with dynamic weight adjustment strategies and task allocation optimization mechanisms, the system can continuously and adaptively adjust based on real-time status to ensure long-term performance stability and maximize resource utilization.

[0134] By collecting feedback data in a closed loop, deeply analyzing anomalies and their impacts, and accurately correcting parameters, the system solves the problems of difficulty in identifying weight adjustment anomalies, difficulty in quantifying the impact of optimization mechanisms, and lack of basis for parameter iteration in adaptive scheduling, thereby improving the stability and iteration efficiency of the scheduling system.

[0135] This application provides a computing power allocation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the adaptive computing power allocation control method in the first embodiment described above.

[0136] The following is for reference. Figure 5 This document illustrates a structural diagram of a computing power allocation device suitable for implementing embodiments of this application. The computing power allocation device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, cloud resource allocation devices, personal digital assistants (PDAs), tablet computers (PADs), portable media players (PMPs), edge computing devices, etc., as well as fixed terminals such as edge heterogeneous computing devices, desktop computers, etc. Figure 5 The computing power allocation device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0137] like Figure 5As shown, the computing power allocation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the computing power allocation device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the computing power distribution device to communicate wirelessly or wiredly with other devices to exchange data. Although a computing power distribution device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0138] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0139] The computing power allocation device provided in this application employs the adaptive computing power allocation control method in the above embodiments, which can solve the technical problem that the overall system performance is limited by local bottleneck nodes, ultimately resulting in low computing efficiency. Compared with the prior art, the beneficial effects of the computing power allocation device provided in this application are the same as those of the adaptive computing power allocation control method provided in the above embodiments, and other technical features in this computing power allocation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0140] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0141] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0142] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the adaptive computing power allocation control method in the above embodiments.

[0143] The aforementioned computer-readable storage medium may be included in the computing power distribution device; or it may exist independently and not be assembled into the computing power distribution device.

[0144] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the computing power allocation device, the computing power allocation device performs the following actions: statically analyzes the computational units of the target large model, extracts the computational complexity, memory requirements, and communication overhead corresponding to the computational units; sets threshold parameters based on the computational complexity, memory requirements, and communication overhead, clusters and integrates the computational units to generate multiple sub-model modules; matches and deploys the sub-model modules with heterogeneous hardware nodes based on a deployment strategy, and records the status information of each node after deployment; calculates the dynamic weight value of each node by processing the status information and status indicators in historical performance data through exponential smoothing prediction, combined with a bottleneck sensitivity coefficient; generates a node priority list based on the dynamic weight value, and allocates the hardware type and task ratio of the sub-model modules according to the node priority list and a task allocation optimization mechanism to complete the adaptive scheduling of the sub-model modules.

[0145] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A control method of adaptive hash power distribution, characterized in that, The method comprises: static analysis of the computing unit of the target large model, extracting the computing complexity, memory requirement and communication overhead corresponding to the computing unit; According to the computing complexity, the memory requirement and the communication overhead, the threshold parameters are set, the computing units are clustered and integrated, and a plurality of sub-model modules are generated; Based on the deployment strategy, the sub-model modules are matched and deployed with the heterogeneous hardware nodes, and the state information of each node is recorded after the deployment is completed; Through exponential smoothing prediction, the state indicators in the state information and historical performance data are processed, combined with the bottleneck sensitive coefficient, and the dynamic weight value of each node is calculated; According to the dynamic weight value, a node priority list is generated, and the hardware type and task proportion of the sub-model module are allocated according to the node priority list and the task allocation optimization mechanism, so as to complete the adaptive scheduling of the sub-model module. 2.The method of Claim 1, wherein, The step of static analysis of the computing unit of the target large model, extracting the computing complexity, memory requirement and communication overhead of the computing unit comprises: Traverse the computing unit of the target large model, and parse the structure information of the computing unit; Based on the structure information, the computing complexity, memory requirement peak value and communication transmission total amount of each computing unit are calculated according to the preset index calculation method; The computing complexity, memory requirement peak value and communication transmission total amount of the computing unit are analyzed and summarized to generate the computing complexity, memory requirement and communication overhead corresponding to the computing unit. 3.The method of Claim 1, wherein, The step of setting threshold parameters according to the computing complexity, the memory requirement and the communication overhead, clustering and integrating the computing units, and generating a plurality of sub-model modules comprises: Call the static analysis data set containing the computing complexity, memory requirement and communication overhead of each computing unit, and set the threshold parameters of the three types of indexes according to the threshold processing method respectively; Taking the threshold parameters as the basis for judgment, each computing unit in the static analysis data set is analyzed one by one, the computing unit is associated with the corresponding type label, and a computing unit classification set is obtained; According to the execution order of the computing unit, the computing unit classification set is traversed, the target computing unit with adjacent and consistent type labels is identified, and two types of preliminary sub-model modules are generated; Verify whether the resource requirement of the preliminary sub-model module meets the basic adaptation range of the heterogeneous hardware node, and perform secondary splitting adjustment on the preliminary sub-model module that exceeds the adaptation range to obtain a plurality of sub-model modules. 4.The method of Claim 1, wherein The step of matching and deploying the sub-model modules with the heterogeneous hardware nodes based on the deployment strategy, and recording the state information of each node after the deployment is completed comprises: According to the deployment strategy, each sub-model module is preferentially matched with the recommended node, if the target node has deployed the same type of module and the task queue length does not exceed the preset threshold, the target node is marked as a reuse object, and a preliminary deployment matching list is output; Based on the preliminary deployment matching list, nodes with low load and sufficient resources are selected from the alternative node pool and allocated to the modules that are not matched with the reuse object, and the deployment result of the sub-model module and the heterogeneous hardware node is output; Collect running data of each binding node, and store the running data in the monitoring system according to node classification to obtain state information of each node. 5.The method of Claim 1, wherein The step of processing the state indicators in the state information and the historical performance data by exponential smoothing prediction, combining the bottleneck sensitivity coefficient, and calculating the dynamic weight value of each node comprises: From the state information and the historical performance data of each node, the state indicators corresponding to the nodes are parsed out; The adaptive exponential smoothing prediction is performed on the state indicators corresponding to the nodes, the smoothing coefficient is dynamically adjusted according to the variance of the state indicators in the recent time window, and the smoothed prediction value of each state indicator is outputted; Based on the smoothed prediction value of the state indicators, the average value of the state indicators in the past multiple monitoring periods, and the preset bottleneck amplification factor, the bottleneck sensitivity coefficient corresponding to the state indicators is calculated; The smoothed prediction value of each state indicator is combined with the corresponding bottleneck sensitivity coefficient to obtain a weight coefficient. The weight coefficient is automatically matched according to the node type and the sub-model module characteristics, and the dynamic weight value of each node is calculated and adjusted.

6. The control method of adaptive computational force distribution of claim 1, wherein, The step of generating a node priority list according to the dynamic weight value, distributing the hardware type and the task proportion of the sub-model module according to the node priority list and the task distribution optimization mechanism, and completing the adaptive scheduling of the sub-model module comprises: According to the dynamic weight value of each node and the matching degree of the characteristics of each heterogeneous hardware node and the type of the sub-model module, the node priority list containing the node priority and the corresponding adaptive module type is generated according to the dynamic weight value of each node; According to the node priority list, the sub-model module is preferentially distributed to the node with high weight and matched hardware type, and an initial task distribution scheme of the sub-model module is generated; Based on the initial task distribution scheme, the task proportion of the sub-model module distribution is fine-tuned to complete the adaptive scheduling of the sub-model module. 7.The method of Claim 1, wherein After the step of generating a node priority list according to the dynamic weight value, distributing the hardware type and the task proportion of the sub-model module according to the node priority list and the task distribution optimization mechanism, and completing the adaptive scheduling of the sub-model module, the following steps are further included: After completing the adaptive scheduling of the sub-model module, the running state of each heterogeneous hardware node is collected, and the running state is compared with the initial reference state of the scheduled node to generate a state comparison result; Based on the state comparison result, the time window sliding mean filter processing is performed on the key indicators of each node to generate a bottleneck determination result; For the node triggering the bottleneck in the bottleneck determination result, the local sub-model module re-distribution or node topology optimization is started, and a bottleneck processing result is outputted; Based on the bottleneck processing result, the variance of the resource utilization rate corresponding to all nodes is calculated, and the global task delay average and the global video memory occupation total are counted. If both conditions are met, the current scheduling state is maintained, and a determination result of global load balancing and resource efficient utilization is outputted.

8. The control method of adaptive computational force distribution of claim 7, wherein, The step of outputting the determination result of global load balancing and resource efficient utilization after calculating the variance of resource utilization of all node pairs corresponding to the bottleneck processing result, and counting the global task delay mean and global memory occupation sum, and maintaining the current scheduling state if both meet the conditions, further comprises: Collecting the response delay of weight adjustment to the bottleneck, the node load balancing degree after task allocation, and the resource utilization deviation to obtain a running feedback data set; Correlation analysis of the running feedback data set, identification of abnormal problems in dynamic weight adjustment, quantification of the influence degree of the task allocation optimization mechanism on performance, and output of a problem analysis report; According to the problem analysis report, the dynamic weight value and the task allocation optimization mechanism are corrected, the optimized parameters are updated to the adaptive scheduling decision module, and the optimized dynamic weight adjustment strategy and task allocation optimization mechanism parameter set are generated.

9. A computing power distribution device, characterized in that, The computing power allocation device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the adaptive computing power allocation control method according to any one of claims 1 to 8.

10. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program. The computer program is executed by the processor to implement the steps of the adaptive computing power allocation control method according to any one of claims 1 to 8.

Images