Heterogeneous computing power real-time perception and coarse-fine mixed granularity adaptive scheduling method and system

Abstract

The application provides a heterogeneous computing power real-time sensing and coarse-fine mixed granularity self-adaptive scheduling method and system. The method comprises the following steps: obtaining the running state of each heterogeneous computing power unit in an AI computing all-in-one machine to generate a real-time computing power state; determining the task granularity according to the computing characteristics and execution constraints of each AI task, and forming coarse-grained task units and fine-grained task units; distributing the coarse-grained task units to the matching candidate heterogeneous computing power units to form a main computing power occupation state; identifying the independently loadable computing fragment power according to the main computing power occupation state and the updated real-time computing power state, generating a fragment power state, selecting a target fine-grained task unit from the fine-grained task units, and filling it into the corresponding fragment power; updating the real-time computing power state during the task execution process, and determining the releasable computing power according to the task occupation state. The application can improve the computing power state sensing accuracy, improve the heterogeneous resource scheduling adaptability, and improve the utilization rate of fragment computing power.

Description

Technical Field

[0001] This application relates to the fields of heterogeneous computing power scheduling and artificial intelligence computing optimization technology, and in particular to a method and system for real-time perception and coarse-fine mixed granularity adaptive scheduling of heterogeneous computing power. Background Technology

[0002] As AI inference services continue to extend to industrial sites and edge computing, AI computing appliances typically integrate multiple heterogeneous computing units to handle different types of concurrent tasks, such as large model inference, visual inspection, state recognition, and control processing. Since different computing units differ in computing modes, resource organization methods, and task adaptability, how to rationally schedule resources based on real-time status directly affects the overall machine's task response capability and computing power utilization.

[0003] Existing heterogeneous computing power scheduling schemes typically allocate resources based on overall processor utilization, memory usage, or task queue length, using a relatively fixed task scheduling granularity to assign complete tasks or pre-divided task units to corresponding computing devices. When multiple AI tasks run concurrently, some computing units, while having a high overall load, may still have fragmented computing resources, storage resources, or short-term available execution windows. Traditional overall load metrics are insufficient to accurately reflect these resource states. Furthermore, using a uniform granularity for scheduling tasks of different sizes makes it difficult for lightweight tasks to effectively utilize the fragmented computing power generated during the execution of heavy tasks.

[0004] Furthermore, when high-priority real-time tasks arrive suddenly or local computing units experience increased load, existing solutions often adjust by queuing tasks or migrating the entire system. This makes it difficult to release suitable computing resources in a timely manner, easily leading to task waiting, local overload, and the accumulation of computing fragments. Therefore, existing technologies still suffer from insufficient heterogeneous computing power perception accuracy, mismatch between scheduling granularity and resource status, and insufficient resource utilization under dynamic load changes. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method and system for real-time heterogeneous computing power perception and coarse-fine mixed granularity adaptive scheduling to solve the problems of insufficient computing power perception accuracy, mismatch between scheduling granularity and resource status, and low utilization rate of fragmented computing power in the prior art.

[0006] A first aspect of this application provides a method for real-time perception and coarse-fine hybrid granular adaptive scheduling of heterogeneous computing power, comprising: acquiring the operating status of each heterogeneous computing power unit in an AI computing all-in-one machine; associating and representing the operating status according to resource levels to generate a real-time computing power status reflecting the computing capacity and resource continuity of each heterogeneous computing power unit; acquiring a set of AI tasks to be scheduled; determining the task granularity according to the computational characteristics and execution constraints of each AI task; and dividing each AI task according to the task granularity to form coarse-grained task units and fine-grained task units; and determining candidate heterogeneous computing power based on the adaptation relationship between the computational requirements of each task unit and the real-time computing power status. The system allocates coarse-grained task units to matching candidate heterogeneous computing power units, forming the main computing power occupancy status. Based on the main computing power occupancy status and the updated real-time computing power status, it identifies fragmented computing power that can independently carry out computation, generates fragmented computing power status, selects target fine-grained task units from the fine-grained task units whose resource requirements match the fragmented computing power status, and fills them into the corresponding fragmented computing power. During task execution, it updates the real-time computing power status. When the load status or task priority status meets the scheduling adjustment conditions, it determines the available computing power based on the task occupancy status, adjusts the execution status or resource mapping relationship of the corresponding task unit, and updates the subsequent task scheduling results.

[0007] A second aspect of this application provides a heterogeneous computing power real-time perception and coarse-fine granular adaptive scheduling system, comprising: an acquisition module, used to acquire the operating status of each heterogeneous computing power unit in an AI computing all-in-one machine, associate and characterize the operating status according to resource levels, and generate a real-time computing power status reflecting the computing capacity and resource continuity of each heterogeneous computing power unit; a partitioning module, used to acquire a set of AI tasks to be scheduled, determine the task granularity according to the computing characteristics and execution constraints of each AI task, and partition each AI task according to the task granularity to form coarse-grained task units and fine-grained task units; and an allocation module, used to determine candidate heterogeneous computing power units based on the adaptation relationship between the computing requirements of each task unit and the real-time computing power status. The system comprises four modules: a coarse-grained computing unit and an update module. The coarse-grained computing unit allocates coarse-grained task units to matching candidate heterogeneous computing units, forming the main computing power occupancy status. The identification module identifies fragmented computing power capable of independently undertaking computation based on the main computing power occupancy status and the updated real-time computing power status. It generates fragmented computing power status, selects target fine-grained task units from the fine-grained task units whose resource requirements match the fragmented computing power status, and populates the corresponding fragmented computing power. The update module updates the real-time computing power status during task execution. When the load status or task priority status meets the scheduling adjustment conditions, it determines the available computing power based on the task occupancy status, adjusts the execution status or resource mapping relationship of the corresponding task unit, and updates the subsequent task scheduling results.

[0008] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: By acquiring the operational status of each heterogeneous computing unit in the AI ​​computing appliance, and characterizing the operational status according to resource levels, a real-time computing power status reflecting the computing capacity and resource continuity of each heterogeneous computing unit is generated. A set of AI tasks to be scheduled is acquired, and the task granularity is determined based on the computational characteristics and execution constraints of each AI task. Each AI task is then divided into coarse-grained and fine-grained task units. Candidate heterogeneous computing units are determined based on the compatibility between the computational requirements of each task unit and the real-time computing power status. The coarse-grained task units are then assigned to matching candidate heterogeneous computing units. This invention constructs computing power units to establish the main computing power occupancy status. Based on the main computing power occupancy status and the updated real-time computing power status, it identifies fragmented computing power capable of independently undertaking computation, generates fragmented computing power status, selects target fine-grained task units from fine-grained task units whose resource requirements match the fragmented computing power status, and populates the corresponding fragmented computing power. During task execution, it updates the real-time computing power status. When the load status or task priority status meets the scheduling adjustment conditions, it determines the available computing power based on the task occupancy status, adjusts the execution status or resource mapping relationship of the corresponding task unit, and updates the subsequent task scheduling results. This application can improve the accuracy of computing power status perception, enhance the adaptability of heterogeneous resource scheduling, and improve the utilization rate of fragmented computing power. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating the heterogeneous computing power real-time perception and coarse-fine mixed granularity adaptive scheduling method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the heterogeneous computing power real-time perception and coarse-fine mixed granularity adaptive scheduling system provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0011] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0012] Existing AI computing appliances typically integrate multiple heterogeneous computing units for concurrent execution of AI tasks such as model inference, visual inspection, and real-time control. Current scheduling technologies usually assess computing power based on overall processor utilization, memory usage, or task queue status, and allocate tasks to different computing units at a fixed task granularity. Because these methods primarily schedule based on overall resource status, they fail to reflect localized idle computing resources, scattered storage resources, and short-term available execution resources generated during task execution. Furthermore, they struggle to simultaneously meet the different needs of continuous execution of heavy tasks and flexible scheduling of light tasks.

[0013] Therefore, existing technologies suffer from insufficient accuracy in perception of computing power, mismatch between scheduling granularity and resource status, and low utilization of fragmented computing power. When AI tasks of different scales are mixed and concurrently executed, some heterogeneous computing units, although having a high overall load, may still have surplus resources capable of independently handling lightweight computing; scheduling with uniform granularity makes it difficult to effectively utilize these resources. Furthermore, when local load increases or high-priority tasks arrive suddenly, existing scheduling methods struggle to promptly identify releaseable resources and adjust task mapping relationships, easily leading to task queuing and localized computing power congestion.

[0014] To address the aforementioned issues, this application obtains the operational status of each heterogeneous computing unit within the AI ​​computing appliance, forming a real-time computing power status reflecting computing capacity and resource continuity according to resource levels. Based on the computational characteristics and execution constraints of the AI ​​tasks to be scheduled, the task granularity is determined, dividing the AI ​​tasks into coarse-grained and fine-grained task units. Heterogeneous computing power is matched based on the computational requirements of the task units and the real-time computing power status, allocating coarse-grained task units to matching heterogeneous computing power units to form the main computing power occupancy status. Based on the real-time resource status after the main computing power occupancy, fragmented computing power capable of independently undertaking computation is identified, and fine-grained task units with matching resource requirements are dynamically filled into the corresponding fragmented computing power. Simultaneously, when the load status or task priority status meets the scheduling adjustment conditions, the available computing power is determined based on the current task occupancy status, and the execution status or resource mapping relationship of the corresponding task units is adjusted.

[0015] Through the above technical solution, this application combines real-time perception of multi-level heterogeneous computing power with coarse-fine mixed-granularity task scheduling, so that the task scheduling granularity can be adapted to the actual resource status of heterogeneous computing power units, and the fragmented computing power formed after the execution of coarse-grained tasks is filled by fine-grained tasks. When the running status changes, the resource occupation relationship is further dynamically adjusted, thereby improving the accuracy of computing power status perception and the adaptability of heterogeneous resource scheduling, improving the utilization rate of fragmented computing power, and enhancing the computing power scheduling capability in multi-task concurrency and sudden task scenarios.

[0016] The technical solution of this application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0017] Figure 1 This is a flowchart illustrating the heterogeneous computing power real-time perception and coarse-fine mixed granularity adaptive scheduling method provided in the embodiment. For example... Figure 1 As shown, the method may specifically include: S101: Obtain the operating status of each heterogeneous computing unit in the AI ​​computing all-in-one machine, associate and characterize the operating status according to the resource level, and generate real-time computing power status that reflects the computing capacity and resource continuity of each heterogeneous computing unit. S102, Obtain the set of AI tasks to be scheduled, determine the task granularity according to the computational characteristics and execution constraints of each AI task, and divide each AI task according to the task granularity to form coarse-grained task units and fine-grained task units. S103, determine candidate heterogeneous computing units based on the matching relationship between the computing requirements of each task unit and the real-time computing power status, and allocate coarse-grained task units to matching candidate heterogeneous computing units to form the main computing power occupancy status. S104. Identify fragmented computing power that can independently carry out computation based on the main computing power occupancy status and the updated real-time computing power status, generate fragmented computing power status, select target fine-grained task units whose resource requirements match the fragmented computing power status from the fine-grained task units, and fill them into the corresponding fragmented computing power. S105 updates the real-time computing power status during task execution. When the load status or task priority status meets the scheduling adjustment conditions, it determines the available computing power based on the task occupancy status, adjusts the execution status or resource mapping relationship of the corresponding task unit, and updates the subsequent task scheduling results.

[0018] In some embodiments, the operating status of each heterogeneous computing unit in the AI ​​computing all-in-one machine is obtained, and the operating status is associated and characterized according to the resource level to generate a real-time computing power status reflecting the computing capacity and resource continuity of each heterogeneous computing unit, including: The operation data of each heterogeneous computing power unit is collected according to the preset sensing cycle, and the corresponding operation status is extracted from the overall bearing layer, task execution layer and resource occupation layer of the computing power unit respectively. By associating the operational status of different resource levels with a unified time base, the task execution's occupancy relationship with the underlying computing and storage resources is determined, and a hierarchical computing power representation is generated. The remaining carrying capacity of each heterogeneous computing unit is determined based on the hierarchical computing power characterization, and the degree of resource continuity is determined in combination with the distribution status of available resources. By associating remaining carrying capacity with resource continuity, a real-time computing power status is generated that updates with the operating status.

[0019] Specifically, in an implementation scenario where an AI computing all-in-one machine is used for visual inspection, large-model assisted analysis, and equipment status monitoring in industrial production lines, the all-in-one machine integrates heterogeneous computing power units such as CPUs, GPUs, and NPUs. The scheduling end establishes independent status acquisition channels for each heterogeneous computing power unit and continuously acquires operational data according to a sensing cycle of, for example, 10ms. For the overall computing power unit load layer, the overall computing load of the chip, on-chip or external storage usage, power consumption, and temperature changes are collected to characterize whether the current computing power unit is approaching its continuous load limit. For the task execution layer, the execution time, queue length, concurrency, and stage progress status corresponding to the running model layer or task stage are read to characterize the continuous occupation of the computing power unit by existing tasks. For the resource occupation layer, the busy / idle status of computing cores or execution channels, operator-level resource consumption, and discrete occupation status of video memory or memory space are further acquired to identify local idle resources that cannot be directly reflected by the overall load indicators. Different chips can complete the acquisition through the driver interface, runtime interface, and status information returned by the task executor. The acquisition results are uniformly appended with computing power unit identifiers, task identifiers, and timestamps.

[0020] The scheduling unit uses a unified clock as a reference to align the multi-level operating states within the same perception cycle and associates task execution records with underlying resource usage records. For example, when the GPU is executing the feature extraction stage of a visual model, it determines the model layer corresponding to that stage based on the task identifier, and then matches the concurrently active computing cores and memory regions according to the execution time interval, thereby establishing a mapping between task stages and underlying computing and storage resources. For the NPU, the computation graph nodes corresponding to inference tasks can be associated with concurrently active execution channels and on-chip cache usage; for the CPU, logic processing threads can be associated with corresponding core usage and memory access status. If the same task spans multiple perception cycles, the corresponding state records are connected according to consecutive timestamps, thus forming a hierarchical computing power representation that can simultaneously reflect the overall load, task execution, and underlying resource distribution.

[0021] Furthermore, after obtaining the hierarchical computing power representation, the scheduler first determines the remaining capacity based on the rated capacity of the computing unit and the currently occupied resources. The remaining capacity is not determined solely by the inverse of the overall utilization rate, but is adjusted based on the scale of resources that the current task will continue to occupy. For example, if a GPU's overall computing utilization rate is 72%, but the currently executing large model layer will still require a portion of computing cores and memory in subsequent perception cycles, then the corresponding expected occupancy is deducted before determining the remaining computing and storage capacity available for new tasks. For computing units whose power consumption or temperature is approaching operational constraints, their short-term capacity is also reduced based on available heat margins to avoid including resources that appear idle but cannot be stably and continuously used in the effective remaining computing power.

[0022] Furthermore, the scheduler determines the resource continuity based on the spatial and temporal distribution of remaining resources. Taking GPUs as an example, it associates adjacent available computing cores, contiguous available memory regions, and execution periods that remain idle within a continuous perception cycle. The more concentrated the resource range and the longer the sustained period, the higher the resource continuity. If the remaining resources are scattered across multiple non-adjacent memory regions or only appear in fragmented execution gaps, the corresponding resource continuity is lower. For NPUs, resource continuity can be determined based on the consecutive number of idle execution channels and their sustained periods. For CPUs, resource continuity can be determined based on the number of schedulable cores and the consecutive idle time. This allows for the differentiation of remaining resource states with similar total amounts but different actual task-carrying capacities.

[0023] For example, if the GPU still has about 30% of its computing power available within a certain perception cycle, but the contiguous memory area is insufficient to support coarse-grained model layers, the scheduler will not evaluate it as highly available simply because the overall remaining computing power is high, but will instead lower the corresponding continuous capacity evaluation. If the NPU has multiple consecutive idle execution channels and its temperature and power consumption are within a stable range, then the corresponding continuous capacity evaluation will be increased. Before correlation, the operating metrics of each layer can be normalized according to the device's capacity range, enabling the states of different types of heterogeneous computing power units to be compared under a unified scheduling scale, while preserving the resource structure differences of each computing power unit.

[0024] Finally, the scheduler associates the remaining capacity and resource continuity according to a unified state structure to generate the real-time computing power status corresponding to each heterogeneous computing power unit. The real-time computing power status can at least represent the current available computing capacity, available storage capacity, sustainable capacity, and resource dispersion, and is updated on a rolling basis with each sensing cycle. When a task ends, a model layer switches, or a new operator begins execution, the task resource occupancy relationship changes synchronously, and the scheduler recalculates the remaining capacity and resource continuity, so that subsequent coarse-grained task allocation, fragmented computing power identification, and fine-grained task filling all use computing power information corresponding to the current actual operating state.

[0025] By adopting this hierarchical perception and association representation method, we can avoid the limitations of judging computing power availability solely based on the overall chip utilization rate, and provide a more refined state basis that reflects the continuity of resources for subsequent hybrid granular scheduling.

[0026] In some embodiments, a set of AI tasks to be scheduled is obtained, the task granularity is determined based on the computational characteristics and execution constraints of each AI task, and each AI task is divided according to the task granularity to form coarse-grained task units and fine-grained task units, including: Analyze the computational structure and operational requirements of each AI task, determine the computational characteristics that characterize the computational scale, task dependence, and resource requirements, and obtain the corresponding execution constraints; Based on computational characteristics and execution constraints, determine the degree of decomposability and continuous execution requirements of each AI task, and generate the corresponding task granularity; For AI tasks that meet the coarse-grained partitioning criteria, they are aggregated and partitioned along computational stages with continuous execution dependencies to form coarse-grained task units. For AI tasks that meet the conditions for fine-grained partitioning, they are split according to independently schedulable computational boundaries to form fine-grained task units that maintain the original task dependencies.

[0027] Specifically, in an implementation scenario where an AI computing all-in-one machine on an industrial production line simultaneously receives tasks such as large-scale model quality inspection and analysis, batch image verification, real-time defect detection, and equipment status monitoring, the scheduling terminal, after the tasks enter a unified waiting queue, does not directly determine the scheduling method based on the business name. Instead, it first reads the model structure, computation graph, and runtime configuration corresponding to each AI task, and analyzes the computational stages within the task and the data dependencies between stages. For large-scale model quality inspection and analysis tasks, it can identify continuous attention computation, feedforward computation, and output processing stages based on the model layer connection relationships; for visual inspection tasks, it can identify computational stages such as preprocessing, feature extraction, and target discrimination based on the computation graph, and further locate operator nodes that can be executed independently. The scheduling terminal also forms a computational scale representation based on the model parameter scale, single inference computation volume, tensor scale, parallel execution capability, and intermediate data transfer scale; forms the task dependency degree based on the strength of dependencies between computational stages and the reuse relationship of intermediate results; and forms resource requirements based on the expected computational and storage resources occupied by each stage, thereby generating the computational characteristics corresponding to each AI task.

[0028] Operational requirements can be obtained from task submission information, model runtime configuration, and historical execution records. Taking a production line scenario as an example, real-time defect detection tasks require results to be returned within a limited time after a single frame image arrives; equipment status monitoring tasks have a fixed sampling period; and large-model quality inspection analysis tasks allow for relatively long execution times but require stable data transfer between model layers. The scheduler unifies the allowed completion time, start time limit, execution priority, and task interruption restrictions into execution constraints and associates these constraints with computational characteristics. For models that have been executed in the past, the actual time consumption and peak resource requirements of each computational stage can be estimated using the most recent execution records to correct the task computational characteristics obtained from the static model structure, avoiding the need to judge task granularity solely based on the number of parameters.

[0029] Subsequently, the scheduler determines the degree of descalability based on the independent execution capability of each computation stage. If there is a large amount of intermediate tensor transfer, frequent state sharing, or strict sequential dependencies between adjacent computation stages, scheduling multiple stages separately will result in high data movement and synchronization overhead, thus reducing the corresponding degree of descalability. Conversely, if some operators have clear input-output boundaries, only need to transmit definite data results to subsequent nodes after execution, and can be started independently on different computing resources, then the corresponding degree of descalability is increased. Simultaneously, execution constraints are used to determine whether a task requires continuous computational power. For example, if large, continuous model layers are frequently migrated, it will increase the overhead of intermediate feature transfer and cache reconstruction, thus requiring high continuous execution. Conversely, some preprocessing or postprocessing operators in single-frame detection have short execution times, thus requiring relatively low continuous execution.

[0030] The scheduler jointly maps the degree of descalability to the requirement of continuous execution, translating it into task granularity. Tasks with large computational scale, tight stage dependencies, and high continuous execution requirements are designated as coarse-grained scheduling objects; tasks with small computational scale, or multiple independently schedulable computational boundaries with low resource requirements, are designated as fine-grained scheduling objects. Task granularity can also be dynamically adjusted based on the current real-time computing power status. For example, when there is a large amount of continuously available computing power, a larger task granularity can be maintained; when continuous computing power is insufficient but multiple fragmented resources exist, the granularity level corresponding to the degree of descalability can be increased without disrupting critical dependencies, enabling subsequent task units to adapt to the current resource distribution.

[0031] In some examples, for large model quality control analysis tasks that meet the coarse-grained partitioning criteria, the scheduler aggregates along the continuous execution dependencies between model layers. Several model layers or computational stages with tightly coupled data transfer and suitable for continuous execution are combined into a coarse-grained task unit. The scheduler records the input boundaries, output boundaries, estimated computational load, storage requirements, and dependencies between preceding and subsequent task units for each coarse-grained task unit. For example, several consecutive layers can be formed into a single, schedulable model stage, allowing this stage to be preferentially mapped to GPUs or NPUs with higher resource continuity, without requiring separate scheduling for each internal operator.

[0032] For real-time defect detection and equipment status monitoring tasks that meet the fine-grained partitioning criteria, the scheduler splits the task according to computational boundaries that can be independently started and completed, forming fine-grained task units from single operators or groups of short operators with complete input-output relationships. During the splitting process, the dependencies in the original computational graph are preserved; only after the preceding task unit completes and produces a valid output does the corresponding subsequent task unit enter the schedulable state. For multiple fine-grained task units that can be executed in parallel, the parallel relationships are recorded, allowing them to be matched with different fragmented computing power. Thus, heavy tasks maintain execution stability with larger, continuous computational units, while light tasks form small, flexibly mappable computational units, providing a unified task foundation for subsequent heterogeneous matching and dynamic filling of fragmented computing power based on real-time computing power status.

[0033] In some embodiments, candidate heterogeneous computing units are determined based on the compatibility between the computational requirements of each task unit and the real-time computing power status. Coarse-grained task units are then assigned to matching candidate heterogeneous computing units to form the main computing power occupancy status, including: Based on the computing requirements of each task unit, determine the corresponding computing power adaptation conditions, match the computing power adaptation conditions with the real-time computing power status of each heterogeneous computing power unit, and screen candidate heterogeneous computing power units. Based on the execution dependencies between coarse-grained task units, determine the corresponding resource consumption order and continuous execution requirements; By combining the remaining carrying capacity and resource continuity of the candidate heterogeneous computing power units, the target heterogeneous computing power units for each coarse-grained task unit are determined. Each coarse-grained task unit is allocated to the corresponding target heterogeneous computing power unit according to the order of resource occupancy, and the task resource occupancy relationship is recorded to form the main computing power occupancy status.

[0034] Specifically, in an implementation scenario where an AI computing appliance simultaneously performs large-scale model quality inspection and analysis, batch image verification, and real-time visual inspection tasks, the scheduling end, after forming coarse-grained and fine-grained task units, establishes a computational requirement description for each task unit. The computational requirement can be characterized by the expected computational load, parallel execution level, model accuracy, storage footprint, and duration of execution for the task unit, and the appropriate computing power type for executing the task unit is determined in conjunction with the corresponding computing architecture.

[0035] For example, for model stages involving highly parallel tensor computations and large intermediate feature sizes, the computational compatibility with GPUs can be improved; for model stages primarily involving quantization matrix computations and supported by the NPU runtime, the computational compatibility with NPUs can be improved; and for task units involving significant flow control and logic processing, the computational compatibility with CPUs can be improved. This adaptation process does not involve fixed chip binding based on business type, but rather generates computing power adaptation conditions based on the actual computational needs of the task unit.

[0036] The scheduler matches the computing power adaptation conditions of each task unit with the real-time computing power status generated in the current perception cycle. First, it verifies whether the heterogeneous computing power unit has the capability to execute the corresponding computational structure and determines whether the current remaining computing capacity, remaining storage capacity, and sustainable carrying capacity meet the minimum resource requirements of the task unit. For heterogeneous computing power units that meet the basic execution conditions, it then determines whether they can maintain stable carrying capacity within the expected execution period based on the degree of resource continuity. If a GPU has high remaining computing power but insufficient contiguous memory space to accommodate the model parameters and intermediate features of a coarse-grained task unit, then that GPU will not be considered a preferred candidate for the current task unit. If an NPU has slightly lower remaining computing capacity but has continuously available execution channels and can fully carry out the quantization model stage, then it can be included in the candidate heterogeneous computing power unit set.

[0037] For coarse-grained task units, the scheduler further establishes a task execution sequence along the execution dependencies of the original AI tasks. Taking the aforementioned large-scale model quality inspection analysis task as an example, if the model is divided into a pre-feature processing stage, a mid-inference stage, and a post-output stage, the resource occupancy order is determined based on the relationship that the output of the previous coarse-grained task unit serves as the input of the next coarse-grained task unit. For adjacent coarse-grained task units with large intermediate data volumes, high reloading overhead, or those requiring caching after stage switching, a higher consecutive execution requirement is marked; for task units where intermediate results can be transferred at low cost and execution boundaries are clear, mapping between different candidate heterogeneous computing power units is allowed. In this way, task dependencies are not only used to determine the execution order but also participate in the selection of subsequent computing power units.

[0038] When determining target heterogeneous computing units, the scheduler jointly evaluates the remaining capacity and resource continuity of candidate heterogeneous computing units. Taking a coarse-grained large model stage as an example, which is expected to require a large amount of continuous computing resources and GPU memory space, the scheduler prioritizes candidate computing units that can continuously provide the corresponding resources within the expected execution interval, rather than simply selecting the computing unit with the lowest instantaneous utilization. When multiple candidate computing units meet the resource requirements, further screening can be performed by combining the execution position of the preceding task units and the intermediate data transfer status, reducing intermediate data migration and runtime environment reconstruction caused by frequent switching of computing units. For large model tasks that can be jointly carried by GPUs and NPUs, target heterogeneous computing units can be selected separately according to the computational characteristics of the model stage, so that different coarse-grained task units obtain execution resources that match their own computing modes.

[0039] Subsequently, the scheduling unit establishes resource mappings between coarse-grained task units and target heterogeneous computing power units according to the predetermined resource occupancy order. Before a task begins execution, computing and storage capacity sufficient for continuous execution is reserved. After a task begins, the actual occupied computing and storage resources, the estimated release time, and the task execution progress are associated and recorded with the task identifier. If a target heterogeneous computing power unit continuously carries multiple coarse-grained task units with dependencies, the continuous resource occupancy interval is recorded according to the execution order. If coarse-grained task units are distributed across different heterogeneous computing power units, the result transfer relationships and resource switching locations between task units are recorded simultaneously.

[0040] Furthermore, after the above allocation is completed, the resource range occupied by coarse-grained tasks in each heterogeneous computing power unit, the corresponding occupied tasks, and the expected duration together constitute the main computing power occupancy status. The main computing power occupancy status is dynamically updated as the coarse-grained tasks begin, progress, and end, and corresponds to the real-time computing power status. For example, when the GPU continuously undertakes the front-end computation stage of a large model, the main computing power occupancy status can clearly identify the computing resources and video memory range that have been occupied for a long time, while retaining local resources that have not yet been used by the main tasks. Subsequent scheduling processes can use this status to further identify fragmented computing power generated during the execution of the main tasks and fill the corresponding remaining resources with appropriate fine-grained task units, thereby realizing the connection between coarse-grained main scheduling and fine-grained supplementary scheduling.

[0041] In some embodiments, fragmented computing power capable of independently undertaking computation is identified based on the main computing power occupancy status and the updated real-time computing power status, and fragmented computing power status is generated, including: The scope of occupied resources for each heterogeneous computing power unit is determined based on the main computing power occupancy status, and the remaining available resources are extracted from the updated real-time computing power status. The remaining available resources are associated with their location and available time period to determine candidate resource fragments that can independently support task units; Fragment computing power is filtered based on the carrying capacity and resource continuity of candidate resource fragments, and associated with the corresponding heterogeneous computing power units and available range to generate fragment computing power status.

[0042] Specifically, after the coarse-grained task units have completed their initial allocation and started execution, the scheduler uses the main computing power occupancy status as the resource deduction benchmark, and combines it with the real-time computing power status obtained from the latest perception cycle to identify local resources that can still be used to execute other tasks. Taking an AI computing all-in-one machine for an industrial production line as an example, a set of continuous model layers for a large model quality inspection and analysis task has been allocated to the GPU, and the quantization inference stage has been allocated to the NPU. The main computing power occupancy status records the computing cores, execution channels, video memory or on-chip storage areas occupied by the corresponding tasks, as well as the expected duration. The scheduler first determines the range of resources that have been continuously occupied or reserved by coarse-grained tasks in each heterogeneous computing power unit based on the task resource occupancy relationship, and excludes these resources from the current real-time computing power status. Then, it extracts the unallocated computing resources, storage resources, and execution time periods that are currently usable, forming a set of remaining available resources.

[0043] The remaining available resources are not all automatically considered fragmented computing power. For example, when a GPU is executing a coarse-grained vision model, approximately 25% of its computing power remains idle. However, this portion of computing power may be distributed across different computing cores and accompanied by several discontinuous idle memory regions. Similarly, while an NPU may have idle execution channels, some channels are only available within a short window formed by switching between two consecutive model phases. Therefore, the scheduler further associates the remaining available resources based on resource location and available time periods. For computing resources, spatial associations are determined according to the set of cores or execution channels capable of jointly undertaking a single independent computation; for storage resources, storage associations are determined according to the contiguously allocable memory or on-chip storage range; and for execution time, the duration for which the corresponding resource can remain available from the current moment is determined based on the continuous perception cycle.

[0044] In practical implementation, each remaining resource can be recorded with its starting position, available capacity, available start time, and expected end time. When computing and storage resources in the same heterogeneous computing unit have overlapping available intervals in time, and the combined resource size can meet the minimum independent task execution requirements, the relevant resources are merged into candidate resource fragments. For example, if four consecutive idle computing cores in a GPU and a memory region capable of accommodating a single-frame detection intermediate tensor are simultaneously available within the next 12ms, they can form a candidate resource fragment; if another memory region has a large capacity but is expected to be called by the main task after 3ms, it will not be combined with computing resources requiring a longer execution time, even if it is currently idle. Through joint time and location constraints, it is possible to avoid misidentifying scattered resources that only exist in a statistical sense but cannot jointly support a complete computing process as schedulable resources.

[0045] The scheduler then evaluates the carrying capacity and resource continuity of each candidate resource fragment. Carrying capacity is determined based on the effective computing capacity, effective storage capacity, and the scale of resources allowed for continuous use under current power consumption and temperature constraints. Resource continuity is determined jointly based on the concentration of computing resources, the continuity of storage regions, and the stability of available time periods. When the computing capacity or storage capacity of a candidate resource fragment is lower than the minimum executable boundary set by the system, the candidate resource fragment is eliminated. Similarly, when the total resource volume meets the requirements but the continuous available time period is shorter than the minimum scheduling window for fine-grained tasks, it is also not considered as currently available fragmented computing power. Multiple adjacent candidate resource fragments with highly overlapping available time periods can be merged to avoid over-splitting of continuous resources capable of jointly carrying tasks.

[0046] For example, after a large model quality inspection analysis task occupies the main computing area of ​​the GPU, the scheduler identifies two sets of remaining resources. The first set has 15% computing capacity, contiguous memory area, and approximately 20ms of stable idle time. The second set, although having 18% computing capacity, has scattered memory and frequently changing idle windows. After joint evaluation, the first set of resources is determined to be fragmented computing power capable of independently supporting real-time defect detection operators, while the second set of resources is temporarily kept as ordinary remaining resources. For the NPU, the same method can be used to identify channel combinations that are not occupied by the main quantization inference task and can be continuously maintained for several execution cycles as fragmented computing power.

[0047] Finally, the scheduler associates each selected fragment of computing power with its corresponding heterogeneous computing unit identifier, resource location, available capacity, available start time, expected duration, and resource continuity to form a fragment computing power status, which is continuously updated as the main task progresses and the real-time computing power status changes. When a coarse-grained task releases some resources, enters a new model stage, or local resources are re-occupied, the candidate resource fragments are re-divided and the fragment computing power status is updated.

[0048] Through the above processing, the subsequent fine-grained task filling does not use a simple overall idle ratio, but rather the actual available computing power that can independently complete the calculation within a defined resource range and time window, thus providing a resource basis for the precise matching of fine-grained tasks and fragmented computing power.

[0049] In some embodiments, selecting target fine-grained task units from the fine-grained task units whose resource requirements match the fragmented computing power status, and filling them into the corresponding fragmented computing power, includes: Based on the resource requirements and execution constraints of each fine-grained task unit, determine the corresponding task carrying conditions; The task carrying conditions are matched with the computing power status of each fragment, and candidate filling relationships that meet the requirements for independent execution are selected. Based on the available range of fragmented computing power and the execution timing of fine-grained task units, the target filling relationship is determined from the candidate filling relationships; Based on the target filling relationship, the target fine-grained task units are mapped to the corresponding fragment computing power, and the resource occupancy status of the corresponding heterogeneous computing power units is updated.

[0050] Specifically, after the AI ​​computing power of the industrial production line's integrated computing machine has reached a fragmented state, the scheduler selects target tasks that can utilize fragmented resources from the currently schedulable fine-grained task units. Taking a scenario where real-time defect detection, equipment status identification, and lightweight data preprocessing tasks run concurrently as an example, the scheduler first reads the resource requirements and execution constraints formed during the splitting phase of each fine-grained task unit. Resource requirements include the minimum computing capacity, storage capacity, input data scale, and estimated execution time required to complete an independent execution. Execution constraints include the completion status of prerequisites, allowed start time, latest completion time, task priority, and continuous execution requirements. For fine-grained task units that depend on the output of prerequisite operators, they are only marked as fillable if the prerequisite task unit has been completed and produced valid intermediate results.

[0051] The scheduler generates task carrying conditions for each fillable fine-grained task unit based on resource requirements and execution constraints. These conditions define the minimum effective computing capacity, available storage space, and continuous available time period required by the target fragment computing power, and record the heterogeneous computing structures the task can adapt to. For example, a lightweight convolution operator in single-frame defect detection is expected to require 8ms of execution time and a certain amount of continuous GPU memory; therefore, the candidate fragment computing power must continuously provide computing and storage resources sufficient for that computational scale within the corresponding time window. For logical processing units handling device status data, which have lower computational requirements but need to be completed before the next sampling period, the task carrying conditions focus more on the immediate start time and completion deadline. For quantization units, the operator execution format is also considered to determine whether the target heterogeneous computing power unit possesses the corresponding execution capability.

[0052] Subsequently, the scheduler matches the task carrying capacity with the current fragment computing power status one by one. During matching, it first verifies whether the heterogeneous computing power unit corresponding to the fragment computing power supports the task's computational structure, and then compares whether the effective computing capacity, available storage capacity, and continuous available time period meet the minimum carrying capacity requirements. If any critical resource is insufficient, the corresponding matching relationship is excluded; if all requirements are met, a candidate filling relationship is established between fine-grained task units and fragment computing power. For example, if a fragment computing power that can sustain for 18ms during the execution of the main GPU task can provide sufficient computing capacity and continuous video memory for a lightweight detection operator, and a fine-grained detection operator is expected to execute for 11ms, then the two can form a candidate filling relationship; if another fragment computing power can only be stably maintained for 6ms, even if the instantaneous computing capacity is sufficient, it will not be used as a candidate execution resource for that task.

[0053] After obtaining multiple candidate filling relationships, the scheduler further determines the target filling relationship by combining the specific available range of fragment computing power and the execution sequence of fine-grained task units. When the same task can be matched with multiple fragment computing powers, priority is given to fragment computing power whose available start time is close to the task's ready time, whose continuous available period can cover the expected execution time, and whose remaining resource quantity is small but sufficient to support the task, thus avoiding the occupation of continuous resources capable of supporting larger tasks. When the same fragment computing power can be matched with multiple tasks, the tasks are sorted based on priority, latest completion time, and dependencies. When a real-time defect detection task is nearing its response time limit, it can be prioritized over background status organization tasks without strict time limits and placed into the corresponding fragment computing power.

[0054] Furthermore, the scheduler's verification of target filling relationships does not affect the main computing power occupancy status. If the expected end time of fragment computing power is later than the time when the main coarse-grained task plans to use the resource in the next stage, the fillable window is shortened or the target filling relationship is canceled; if the fine-grained task is expected to complete before the main resource is reoccupied, a filling scheduling instruction is generated. For multiple fine-grained task units with sequential dependencies, consecutive fragment computing power windows can be matched separately, and the original task dependencies are preserved, so that subsequent tasks only enter the corresponding resource after the preceding task has been executed.

[0055] During the fragmentation process, the scheduler maps the execution context, input data location, and resource requirements of the target fine-grained task unit to the corresponding heterogeneous computing power unit according to the target fragmentation relationship. Computation is initiated within the resource location and available time period defined by the fragmented computing power, and the actual occupied computing resources, storage resources, start time, and expected release time are recorded. After the task starts, the corresponding fragmented computing power is adjusted from an available state to an occupied state; if the task completes ahead of schedule, the relevant resources are released immediately; if the task execution time approaches the fragment's available boundary, new dependent tasks are restricted from entering. The scheduler synchronously updates the resource occupancy status and real-time computing power status of the heterogeneous computing power units, ensuring that the next round of fragmentation identification uses the latest resource distribution.

[0056] For example, during the coarse-grained phase of a large model, which continuously occupies the majority of GPU resources, the scheduler can sequentially fill different idle windows with preprocessing operators and lightweight feature operators from single-frame defect detection, without changing the original resource mapping of the main task of the large model. Through joint matching of task carrying conditions, fragmented computing power status, and execution timing, fine-grained tasks only enter local resources that can fully support their execution process, and dynamically release and update the corresponding resources as the task is completed, providing a state basis for subsequent continuous utilization of fragmented computing power and hybrid granularity scheduling.

[0057] In some embodiments, prior to determining the target fill relationship, the method further includes: Based on the resource change status of each fragment computing power within the continuous sensing period, the sustainable carrying period of the corresponding fragment computing power is determined; Based on the computational requirements and execution schedule of each candidate fine-grained task unit, determine the corresponding expected resource consumption period. The sustainable carrying period is matched with the expected resource occupation period, and the degree of filling interference is determined in combination with the corresponding main computing power occupation status. Target filling relationships that meet preset conditions in terms of filling interference are selected from candidate filling relationships that meet the time period matching conditions.

[0058] Specifically, in scenarios where AI computing all-in-one machines on industrial production lines continuously perform large-scale model quality inspection analysis and real-time defect detection, although fragmented computing power appears idle in the current sensing cycle, its subsequent availability time may be affected by coarse-grained main task phase switching, resource reclamation, and the start of a new round of computing. Therefore, before determining the target filling relationship, the scheduler predicts the continuous availability status of fragmented computing power. The scheduler continuously records the computing resources, storage resources, and available time periods corresponding to each fragment of computing power according to a sensing cycle of, for example, 10ms, and associates the records of adjacent cycles with heterogeneous computing power unit identifiers and resource locations. When the same resource fragment remains idle in multiple consecutive sensing cycles, and the changes in available computing capacity and storage capacity do not exceed the preset fluctuation range, the continuous idle interval is extended forward; when it is detected that the main task is about to enter a computing phase that requires re-occupancy of the resource, the planned occupation time is used as the end boundary of the sustainable carrying period.

[0059] For example, during the execution of the current model phase of a large-scale coarse-grained task on the GPU, a set of local computing cores and contiguous memory regions have remained available for three consecutive sensing cycles. However, based on the main computing power occupancy status record, the next coarse-grained task unit is expected to call some of these resources in 45ms. The scheduler then determines the time range between the current moment and the expected call time as the sustainable carrying period for the corresponding fragment's computing power. If the effective computing capacity of this fragment continues to decrease within consecutive cycles, the sustainable carrying period is shortened based on the recent resource change trend. If the main task execution progress is delayed, and the expected resource reclamation time is simultaneously postponed, the corresponding carrying period is updated. Therefore, the sustainable carrying period reflects both the historical stable state and the subsequent resource occupancy plan of the main task, rather than simply using the instantaneous idle result of the current cycle.

[0060] For fine-grained task units that have already formed candidate filling relationships with fragmented computing power, the scheduler estimates the expected resource occupation period based on computational requirements and current execution progress. For single-frame defect detection operators that have not yet started, the expected execution time can be determined based on historical execution times under the same model and input scale, combined with the current computing power performance, and the earliest allowed start time is used as the starting point for expected resource occupation. For tasks that have completed part of the pre-computation, the remaining occupation time is re-estimated based on the remaining computational load, the current intermediate data status, and subsequent dependencies. For example, if the historical average execution time of a lightweight feature operator is 16ms, and the current input image resolution is high, it is expected to take 20ms after computational scale correction, then an expected resource occupation period covering the 20ms continuous execution process is generated.

[0061] Subsequently, the scheduler compares the estimated resource usage period of each candidate fine-grained task unit with the sustainable carrying period of the candidate fragment computing power. Only when the earliest start time of the task is within the fragment's available range, and the estimated completion time does not exceed the fragment's estimated resource reclamation boundary, is the time period matching condition considered met. To prevent resource conflicts during the recovery of the main task due to runtime fluctuations, a safety margin can be reserved at the end of the sustainable carrying period. For example, if a fragment's computing power is expected to last 35ms, after reserving a 5ms resource reclamation margin, only fine-grained tasks with an estimated usage time of no more than 30ms are allowed to enter.

[0062] Furthermore, after meeting the time-period matching conditions, the scheduler calculates the filling interference level based on the main computing power occupancy status. The filling interference level indicates the possibility that fine-grained tasks entering fragmented computing power will cause resource contention for the subsequent execution of the main coarse-grained tasks. It can be determined by combining the degree of overlap between the resources occupied by the filling task and the resources subsequently required by the main task, the interval between the expected release time and the main resource call time, and the amount of data movement generated during the filling process. If the fine-grained task only uses computing cores and independent video memory areas not currently needed by the main task, and can release resources before the next stage of the main task starts, the filling interference level is low; if the two share storage bandwidth, cache area, or the resource release time is close to the main task call boundary, the filling interference level is increased.

[0063] For example, candidate fragment A has a stable availability time of 28ms, candidate fragment B has a stable availability time of 40ms, and a fine-grained detection task is expected to take 22ms. Although both meet the time period conditions, the resource corresponding to fragment A will be directly called in the next stage of the coarse-grained model, while fragment B has less overlap with the main resource of the next stage. Therefore, fragment B has a lower level of filling interference, and the scheduler prioritizes establishing the target filling relationship between the task and fragment B. When multiple candidate tasks compete for the computing power of the same fragment, the task with a higher degree of fit between the expected time period and the fragment window and a lower level of filling interference is selected first, while meeting the execution time limit.

[0064] Finally, the scheduler selects candidates with filling interference levels below a preset threshold from those that meet the time-slot matching criteria as target filling relationships and reserves the corresponding resource time slots. If subsequent sensing cycles detect a shortening of the sustainable carrying time of fragmented computing power or an accelerated execution progress of the main task, the corresponding reservation is canceled and re-matched before the target fine-grained task starts. Through joint judgment of continuous resource status, expected task occupancy process, and subsequent resource requirements of the main task, fine-grained task filling not only meets instantaneous resource capacity conditions but also adapts to the dynamic changes in the time boundary of fragmented computing power as the main task executes, reducing the interference of fine-grained filling on the continuous execution of coarse-grained main tasks.

[0065] In some embodiments, when the load status or task priority status meets the scheduling adjustment conditions, the available computing power is determined based on the task occupancy status, the execution status or resource mapping relationship of the corresponding task unit is adjusted, and the subsequent task scheduling results are updated, including: Identify heterogeneous computing units with abnormal loads based on the updated real-time computing power status, or determine the target computing power requirement based on the task priority status of newly added tasks. Based on the task occupancy status, candidate task units that occupy the target computing power and meet the adjustment conditions are selected, and the corresponding releaseable computing power is determined. Based on the degree of load anomaly or target computing power requirements, identify the task units to be adjusted from the candidate task units, and adjust the corresponding resource usage or target heterogeneous computing power units accordingly. Based on the adjusted task occupancy status, unexecuted task units are rematched with available computing power, and subsequent task scheduling results are updated.

[0066] Specifically, in scenarios where AI computing all-in-one machines on industrial production lines continuously perform tasks such as large-scale model quality inspection and analysis, batch image verification, real-time defect detection, and equipment status monitoring, the scheduling terminal continuously updates the real-time computing power status of each heterogeneous computing power unit according to the perception cycle and compares the updated results with the current task occupancy status. When the actual computing load, storage occupancy, or task queuing status of a certain heterogeneous computing power unit continuously exceeds its corresponding carrying capacity, or when changes in power consumption or temperature cause a decrease in sustainable carrying capacity, the heterogeneous computing power unit is identified as a load anomaly unit. Load anomalies can manifest as the allocated task demand exceeding the current effective carrying capacity, or as local resource competition arising from the simultaneous operation of the main task and filler tasks. For example, if the GPU enters a high-computation model phase due to a large model, causing the effective remaining computing capacity to decrease from 25% to 8%, and multiple fine-grained detection tasks are still occupying local computing resources, the scheduling terminal determines that the GPU meets the load adjustment conditions based on the new real-time computing power status.

[0067] Besides load anomalies, when a new high-priority task enters the waiting queue, the scheduler determines the target computing power requirement based on the task's computational needs and execution constraints. For example, if an emergency visual re-inspection task is generated after a production line equipment triggers an alarm, requiring recognition to be completed within 80ms, the scheduler determines, based on the task's model structure, estimated computational load, GPU memory requirements, and completion time limit, that a contiguous segment of computing resources and corresponding storage space need to be obtained from the GPU, and compares the currently available resources with the target computing power requirement. When the existing idle computing power is insufficient to meet the new task, the scheduler determines the scale of resources that need to be released through scheduling adjustments.

[0068] Subsequently, the scheduler reads the task occupancy status, identifies units with abnormal current load or new tasks within the target computing power range, and filters candidate task units that can be adjusted. During the filtering process, coarse-grained task units that are currently performing uninterruptible computation, are nearing completion and have high adjustment costs, or have strict continuous execution requirements are excluded. Fine-grained task units and low-priority background tasks that can be paused, migrated, have reduced resource consumption, or have not yet started core computation are prioritized. For each candidate task unit, the available computing power is determined based on the currently occupied computing capacity, storage capacity, estimated remaining execution time, and the range of migrated resources. For example, if two fine-grained operators in background image verification occupy 6% and 5% of GPU computing resources respectively, and one can be migrated to an idle segment of another GPU while the other can be postponed to a later cycle, then both corresponding resources can be included in the available computing power.

[0069] When adjustments are triggered by load anomalies, the scheduler determines the amount of resource usage to be reduced based on the severity of the anomaly. For example, if the GPU's current effective capacity decreases by 12%, a combination of tasks that can release at least the amount of the resource shortfall is selected from the candidate task units. When adjustments are triggered by the addition of high-priority tasks, the difference between the target computing power requirement and the currently available computing power is used as the computing power requirement to be released. Task selection is based on a combination of task priority, execution progress, post-migration adaptability, and resource adjustment costs, prioritizing the adjustment of low-priority tasks with longer remaining execution time and other adaptable computing power to minimize changes to existing execution chains.

[0070] For candidate task units that can continue execution by reducing the parallel scale, the scheduler reduces their resource consumption and extends the subsequent execution interval. For task units with other candidate heterogeneous computing power units, the target heterogeneous computing power unit is re-determined based on the latest real-time computing power status, and the resource mapping relationship is updated after migrating the input data and necessary execution context. For task units that do not currently have suitable resources and allow for delay, execution is paused while the completed computation progress is retained. Taking the aforementioned emergency visual re-inspection task as an example, the scheduler can migrate the background fine-grained verification operator in the GPU to another available GPU fragment computing power, while pausing low-priority statistical tasks to release computing resources and memory space that meet the continuous execution requirements of the emergency task, and then allocate the new task to the released resource range.

[0071] Furthermore, after the adjustment is completed, the scheduler regenerates the task occupancy status and updates the real-time computing power status based on the released, migrated, and newly occupied resources. For coarse-grained and fine-grained task units that have not yet been executed, computing power matching is re-executed based on the new resource availability range, task dependencies, task priorities, and execution time limits. If the resources corresponding to a task originally scheduled to be executed are already occupied by a high-priority task, other candidate heterogeneous computing power units are selected or the startup order is adjusted; if new fragmented computing power is formed after migration, the fragment identification and fine-grained filling process is re-entered.

[0072] When the load returns to normal or high-priority tasks are completed, tasks that were paused or had their resource usage reduced are re-entered into the scheduling queue according to their retained execution progress. This results in a continuously updated scheduling result for subsequent tasks that changes with load and task priority, enabling coarse-grained main tasks, fine-grained filler tasks, and sudden high-priority tasks to be dynamically coordinated under a unified resource state.

[0073] In some embodiments, determining the task units to be adjusted from candidate task units based on the degree of load anomaly or target computing power requirements includes: Determine the computing power requirements to be released based on the target computing power requirements of the new tasks and the currently available computing power. Based on the task priority status, execution progress, and resource adjustment cost of each candidate task unit, determine the corresponding resource release status; Based on the computing power demand to be released, determine the combination of task units to be adjusted that meet the computing power release requirements from the candidate task units; Adjust the execution status or resource mapping relationship of the task unit combination to be adjusted, and resume the subsequent execution of the adjusted task unit according to the execution progress after the corresponding scheduling conditions are released.

[0074] Specifically, during the normal execution of large-model quality inspection analysis, batch image verification, and equipment status statistics tasks by the AI ​​computing all-in-one machine on the industrial production line, when a production line camera detects a suspected serious defect and submits an emergency visual re-inspection task, the scheduling terminal first generates the target computing power requirement based on the model structure, input data scale, and completion time limit of the new task. The target computing power requirement includes at least the computing capacity, storage capacity, continuous execution period, and compatible heterogeneous computing power type required to start the task. For example, if the emergency visual re-inspection task needs to be completed within 80ms, it is expected to require the GPU to provide 30% continuous computing capacity and corresponding video memory space. However, the current GPU can only directly use 12% of the continuous computing power, so 18% of the computing capacity and the corresponding storage resources are determined as the computing power requirement to be released. If there is also a shortage of currently available storage space, the corresponding storage release requirement is recorded simultaneously to avoid the inability to start the new task even after releasing computing resources.

[0075] The scheduler identifies the task units consuming target computing power based on the current task occupancy status and forms a candidate task unit set by considering whether the task allows pausing, migration, or resource reduction. For large-scale model coarse-grained task units that are currently executing and have strict continuous dependencies, their adjustable priority is reduced if interruption would lead to large-scale intermediate state reconstruction. For fine-grained task units corresponding to background image verification, log analysis, model warm-up, and statistical calculations, they are prioritized for inclusion in the candidate range if they can be paused or continue execution in other computing power units. For each candidate task unit, the scheduler reads the task priority status, current execution progress, and resource adjustment cost to form a resource release status.

[0076] The execution progress is used to determine the percentage of computation already completed, the remaining execution time, and whether the task is currently within a safe adjustment boundary. For example, if an image verification operator is 90% complete and expected to finish in 3ms, it can be allowed to complete naturally without immediate migration; if another background statistics task is only 20% complete and supports saving intermediate results, it is suitable to pause and release resources. The cost of resource adjustment can be determined based on the amount of intermediate data to be saved, the amount of data to be migrated, the time required for the target computing unit to reload the execution environment, and the amount of repetitive computation when the task resumes. Candidate task units with lower priority, execution progress suitable for interruption, and lower resource adjustment costs have a higher degree of resource release capability.

[0077] After determining the resource release status of each candidate task unit, the scheduler combines the candidate task units according to the computing power requirements to be released. Specifically, they can be sorted according to the amount of effective resources that can be released per unit adjustment cost, and the computing capacity, storage capacity, and contiguous resource range that each candidate task unit can release can be gradually accumulated until the accumulated released resources meet the target computing power requirements of the new task.

[0078] In some examples, if a single candidate task unit can release complete and continuous resources, then that single task is prioritized for adjustment. If multiple tasks need to release resources together, then it is verified whether the resource locations after the release of multiple tasks can form a continuous carrying range required by the new task, avoiding situations where the total amount of released resources meets the requirements but is still too scattered. For example, pausing two background tasks that occupy 8% and 6% of GPU computing resources respectively can only release 14% of resources, which is insufficient to make up for the 18% gap. The scheduler then selects a fine-grained task that can be migrated and occupies 7% of computing resources, forming a task unit combination to be adjusted consisting of 3 tasks.

[0079] For task unit combinations to be adjusted, the scheduler performs pause, resource reduction, or migration processing based on the resource release status. Tasks allowed to be paused stop occupying corresponding resources after saving their execution progress, intermediate result locations, and dependency completion status at the nearest recoverable computing boundary. Tasks supporting reduced parallelism reduce their currently allocated computing resources and recalculate their remaining completion time. For tasks with other adaptable heterogeneous computing units, a new execution location is selected based on the latest real-time computing power status, and resource mapping relationships are switched after necessary data migration and runtime context establishment. After adjustment, the scheduler verifies whether the actually released computing and storage resources meet the computing power requirements to be released. If the conditions are met, the emergency visual re-inspection task is mapped to the formed continuous computing power range.

[0080] During the execution of newly added high-priority tasks, the task identifier, execution position before adjustment, progress, intermediate states, and recovery conditions of the adjusted task are all recorded in the scheduling status. When the emergency visual re-inspection task is completed, or the target computing power requirement decreases to a level that the current idle resources can directly handle, the corresponding scheduling conditions are determined to be lifted. The scheduler re-examines the original execution position of the adjusted task and other candidate computing power positions based on the latest real-time computing power status, and resumes subsequent execution based on the saved execution progress. For tasks whose original resources have been restored to availability, computation continues from the interrupted position; for tasks whose original resources are still occupied by other main tasks, the completed progress is maintained and remapped to suitable available computing power.

[0081] This application links the target computing power gap, the release availability of candidate tasks, and the execution recovery status. When a sudden high-priority task arrives, it can select a combination of tasks to be adjusted that meets the resource requirements, while maintaining the effective execution progress of existing tasks, and providing a continuous status basis for task recovery after the scheduling conditions are lifted.

[0082] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.

[0083] Figure 2 This is a schematic diagram of the structure of the heterogeneous computing power real-time perception and coarse-fine mixed granularity adaptive scheduling system provided in the embodiments of this application. Figure 2 As shown, the system includes: The acquisition module 201 is used to acquire the operating status of each heterogeneous computing power unit in the AI ​​computing all-in-one machine, associate and characterize the operating status according to the resource level, and generate real-time computing power status that reflects the computing capacity and resource continuity of each heterogeneous computing power unit. The partitioning module 202 is used to obtain a set of AI tasks to be scheduled, determine the task granularity according to the computational characteristics and execution constraints of each AI task, and partition each AI task according to the task granularity to form coarse-grained task units and fine-grained task units. The allocation module 203 is used to determine candidate heterogeneous computing power units based on the matching relationship between the computing requirements of each task unit and the real-time computing power status, and to allocate coarse-grained task units to matching candidate heterogeneous computing power units to form the main computing power occupancy status. The identification module 204 is used to identify fragmented computing power that can independently carry out computing based on the main computing power occupancy status and the updated real-time computing power status, generate fragmented computing power status, select target fine-grained task units whose resource requirements match the fragmented computing power status from the fine-grained task units, and fill them into the corresponding fragmented computing power. The update module 205 is used to update the real-time computing power status during task execution. When the load status or task priority status meets the scheduling adjustment conditions, it determines the available computing power based on the task occupancy status, adjusts the execution status or resource mapping relationship of the corresponding task unit, and updates the subsequent task scheduling results.

[0084] In some embodiments, Figure 2 The acquisition module 201 collects the operating data of each heterogeneous computing power unit according to a preset sensing cycle, and extracts the corresponding operating status from the overall bearing layer, task execution layer and resource occupancy layer of the computing power unit respectively; it associates the operating status of different resource levels according to a unified time base, determines the occupancy relationship of task execution on the underlying computing resources and storage resources, and generates a hierarchical computing power representation; it determines the remaining bearing capacity of each heterogeneous computing power unit based on the hierarchical computing power representation, and determines the resource continuity in combination with the distribution status of available resources; it associates the remaining bearing capacity with the resource continuity to generate a real-time computing power status that is updated with the operating status.

[0085] In some embodiments, Figure 2 The partitioning module 202 analyzes the computational structure and operational requirements of each AI task, determines the computational characteristics representing the computational scale, task dependency, and resource requirements, and obtains the corresponding execution constraints. Based on the computational characteristics and execution constraints, it determines the decomposability and continuous execution requirements of each AI task, and generates the corresponding task granularity. For AI tasks that meet the coarse-grained partitioning conditions, they are aggregated and partitioned along the computational stages with continuous execution dependencies to form coarse-grained task units. For AI tasks that meet the fine-grained partitioning conditions, they are split according to the independently schedulable computational boundaries to form fine-grained task units that maintain the original task dependencies.

[0086] In some embodiments, Figure 2The allocation module 203 determines the corresponding computing power adaptation conditions based on the computing requirements of each task unit, matches the computing power adaptation conditions with the real-time computing power status of each heterogeneous computing power unit, and filters candidate heterogeneous computing power units; it determines the corresponding resource occupancy order and continuous execution requirements based on the execution dependency relationship between coarse-grained task units; it determines the target heterogeneous computing power unit for each coarse-grained task unit by combining the remaining carrying capacity and resource continuity of the candidate heterogeneous computing power units; it allocates each coarse-grained task unit to the corresponding target heterogeneous computing power unit according to the resource occupancy order, and records the task resource occupancy relationship to form the main computing power occupancy status.

[0087] In some embodiments, Figure 2 The identification module 204 determines the occupied resource range of each heterogeneous computing power unit based on the main computing power occupancy status, and extracts the remaining available resources from the updated real-time computing power status; it associates the remaining available resources according to the resource location and available time period to determine candidate resource fragments that can independently support task units; it filters fragmented computing power according to the carrying capacity and resource continuity of the candidate resource fragments, and associates the corresponding heterogeneous computing power units and available ranges to generate fragmented computing power status.

[0088] In some embodiments, Figure 2 The identification module 204 determines the corresponding task carrying conditions based on the resource requirements and execution constraints of each fine-grained task unit; matches the task carrying conditions with the computing power status of each fragment, and filters candidate filling relationships that meet the independent execution requirements; determines the target filling relationship from the candidate filling relationships based on the available range of fragment computing power and the execution sequence of fine-grained task units; maps the target fine-grained task unit to the corresponding fragment computing power according to the target filling relationship, and updates the resource occupancy status of the corresponding heterogeneous computing power unit.

[0089] In some embodiments, Figure 2 The identification module 204 determines the sustainable carrying period of each fragment computing power based on the resource change status of each fragment computing power within the continuous sensing period; determines the corresponding expected resource occupation period based on the computing requirements and execution progress of each candidate fine-grained task unit; matches the sustainable carrying period with the expected resource occupation period, and determines the filling interference level in combination with the corresponding main computing power occupation status; and selects the target filling relationship that meets the preset conditions for filling interference level from the candidate filling relationships that meet the time period matching conditions.

[0090] In some embodiments, Figure 2The update module 205 determines the heterogeneous computing power units with abnormal load based on the updated real-time computing power status, or determines the target computing power requirement based on the task priority status of newly added tasks; it filters candidate task units that occupy the target computing power and meet the adjustment conditions based on the task occupancy status, and determines the corresponding releaseable computing power; based on the degree of load abnormality or target computing power requirement, it determines the task units to be adjusted from the candidate task units, and adjusts the corresponding resource occupancy or target heterogeneous computing power units; based on the adjusted task occupancy status, it rematches the unexecuted task units with the available computing power, and updates the subsequent task scheduling results.

[0091] In some embodiments, Figure 2 The update module 205 determines the computing power requirement to be released based on the target computing power requirement of the new task and the current available computing power; determines the corresponding resource release status based on the task priority status, execution progress and resource adjustment cost of each candidate task unit; determines the combination of task units to be adjusted that meets the computing power release requirements from the candidate task units according to the computing power requirement to be released; adjusts the execution status or resource mapping relationship of the combination of task units to be adjusted, and resumes the subsequent execution of the adjusted task units according to the execution progress after the corresponding scheduling conditions are released.

[0092] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments.

[0093] Figure 3 This is a schematic diagram of the electronic device 3 provided in an embodiment of this application. Figure 3 As shown, the electronic device 3 of this embodiment includes: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the various method embodiments described above. Alternatively, when the processor 301 executes the computer program 303, it implements the functions of each module / unit in the various system embodiments described above.

[0094] Electronic device 3 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 3 may include, but is not limited to, processor 301 and memory 302. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or different components.

[0095] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0096] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 3. The memory 302 can also include both internal and external storage units of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0098] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which may be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0099] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for real-time perception of heterogeneous computing power and adaptive scheduling with mixed coarse and fine granularity, characterized in that, include: The system acquires the operating status of each heterogeneous computing unit in the AI ​​computing all-in-one machine, associates and represents the operating status according to the resource level, and generates a real-time computing power status that reflects the computing capacity and resource continuity of each heterogeneous computing unit. Obtain a set of AI tasks to be scheduled, determine the task granularity based on the computational characteristics and execution constraints of each AI task, and divide each AI task according to the task granularity to form coarse-grained task units and fine-grained task units. Candidate heterogeneous computing units are determined based on the matching relationship between the computing requirements of each task unit and the real-time computing power status. The coarse-grained task units are then assigned to matching candidate heterogeneous computing units to form the main computing power occupancy status. Based on the main computing power occupancy status and the updated real-time computing power status, identify fragmented computing power that can independently carry out computation, generate fragmented computing power status, select target fine-grained task units whose resource requirements match the fragmented computing power status from the fine-grained task units, and fill them into the corresponding fragmented computing power. During task execution, the real-time computing power status is updated. When the load status or task priority status meets the scheduling adjustment conditions, the available computing power is determined according to the task occupancy status, the execution status or resource mapping relationship of the corresponding task unit is adjusted, and the subsequent task scheduling results are updated.

2. The method according to claim 1, characterized in that, The process involves acquiring the operating status of each heterogeneous computing unit in the AI ​​computing all-in-one machine, associating and representing the operating status according to resource levels, and generating a real-time computing power status reflecting the computing capacity and resource continuity of each heterogeneous computing unit, including: The operation data of each heterogeneous computing power unit is collected according to the preset sensing cycle, and the corresponding operation status is extracted from the overall bearing layer, task execution layer and resource occupation layer of the computing power unit respectively. By associating the operational status of different resource levels with a unified time base, the task execution's occupancy relationship with the underlying computing and storage resources is determined, and a hierarchical computing power representation is generated. The remaining carrying capacity of each heterogeneous computing unit is determined based on the hierarchical computing power characterization, and the degree of resource continuity is determined in combination with the distribution status of available resources. The remaining carrying capacity is correlated with the resource continuity to generate a real-time computing power status that is updated with the operating status.

3. The method according to claim 1, characterized in that, The process of acquiring a set of AI tasks to be scheduled, determining the task granularity based on the computational characteristics and execution constraints of each AI task, and dividing each AI task according to the task granularity to form coarse-grained task units and fine-grained task units includes: Analyze the computational structure and operational requirements of each AI task, determine the computational characteristics that characterize the computational scale, task dependence, and resource requirements, and obtain the corresponding execution constraints; Based on the computational characteristics and execution constraints, the degree of decomposability and continuous execution requirements of each AI task are determined, and the corresponding task granularity is generated. For AI tasks that meet the coarse-grained partitioning criteria, they are aggregated and partitioned along computational stages with continuous execution dependencies to form coarse-grained task units. For AI tasks that meet the conditions for fine-grained partitioning, they are split according to independently schedulable computational boundaries to form fine-grained task units that maintain the original task dependencies.

4. The method according to claim 1, characterized in that, The process of determining candidate heterogeneous computing power units based on the matching relationship between the computing needs of each task unit and the real-time computing power status, and allocating the coarse-grained task units to matching candidate heterogeneous computing power units to form the main computing power occupancy status, includes: Based on the computing requirements of each task unit, the corresponding computing power adaptation conditions are determined, and the computing power adaptation conditions are matched with the real-time computing power status of each heterogeneous computing power unit to screen candidate heterogeneous computing power units. Based on the execution dependencies between the coarse-grained task units, the corresponding resource occupancy order and continuous execution requirements are determined; Based on the remaining carrying capacity and resource continuity of the candidate heterogeneous computing power units, the target heterogeneous computing power unit for each coarse-grained task unit is determined. According to the resource occupancy order, each coarse-grained task unit is allocated to the corresponding target heterogeneous computing power unit, and the task resource occupancy relationship is associated and recorded to form the main computing power occupancy status.

5. The method according to claim 1, characterized in that, The step of identifying fragmented computing power capable of independently undertaking computation based on the main computing power occupancy status and the updated real-time computing power status, and generating fragmented computing power status, includes: The occupied resource range of each heterogeneous computing power unit is determined based on the main computing power occupancy status, and the remaining available resources are extracted from the updated real-time computing power status. The remaining available resources are associated with their location and available time period to determine candidate resource segments that can independently support task units; Based on the carrying capacity and resource continuity of the candidate resource fragments, fragment computing power is screened, and the corresponding heterogeneous computing power units and available ranges are associated to generate fragment computing power status.

6. The method according to claim 5, characterized in that, The step of selecting target fine-grained task units from the fine-grained task units whose resource requirements match the fragment computing power status and filling them into the corresponding fragment computing power includes: Based on the resource requirements and execution constraints of each fine-grained task unit, determine the corresponding task carrying conditions; The task carrying conditions are matched with the computing power status of each fragment, and candidate filling relationships that meet the independent execution requirements are selected. Based on the available range of fragmented computing power and the execution sequence of fine-grained task units, the target filling relationship is determined from the candidate filling relationships; According to the target filling relationship, the target fine-grained task unit is mapped to the corresponding fragment computing power, and the resource occupancy status of the corresponding heterogeneous computing power unit is updated.

7. The method according to claim 6, characterized in that, Before determining the target filling relationship, the process also includes: Based on the resource change status of each fragment computing power within the continuous sensing period, the sustainable carrying period of the corresponding fragment computing power is determined; Based on the computational requirements and execution schedule of each candidate fine-grained task unit, determine the corresponding expected resource consumption period. The sustainable carrying period is matched with the expected resource occupation period, and the degree of filling interference is determined in combination with the corresponding main computing power occupation status; Target filling relationships that meet preset conditions in terms of filling interference are selected from candidate filling relationships that meet the time period matching conditions.

8. The method according to claim 1, characterized in that, When the load status or task priority status meets the scheduling adjustment conditions, the available computing power is determined based on the task occupancy status, the execution status or resource mapping relationship of the corresponding task unit is adjusted, and the subsequent task scheduling results are updated, including: Identify heterogeneous computing units with abnormal loads based on the updated real-time computing power status, or determine the target computing power requirement based on the task priority status of newly added tasks. Based on the task occupancy status, candidate task units that occupy the target computing power and meet the adjustment conditions are selected, and the corresponding releaseable computing power is determined. Based on the degree of load anomaly or the target computing power requirement, determine the task units to be adjusted from the candidate task units, and adjust the corresponding resource consumption or target heterogeneous computing power units. Based on the adjusted task occupancy status, unexecuted task units are rematched with available computing power, and subsequent task scheduling results are updated.

9. The method according to claim 8, characterized in that, The step of determining the task units to be adjusted from the candidate task units based on the degree of load anomaly or target computing power requirements includes: Determine the computing power requirements to be released based on the target computing power requirements of the new tasks and the currently available computing power. Based on the task priority status, execution progress, and resource adjustment cost of each candidate task unit, determine the corresponding resource release status; According to the computing power demand to be released, determine the combination of task units to be adjusted that meets the computing power release requirements from the candidate task units; Adjust the execution status or resource mapping relationship of the task unit combination to be adjusted, and resume the subsequent execution of the adjusted task unit according to the execution progress after the corresponding scheduling conditions are released.

10. A heterogeneous computing power real-time sensing and coarse-fine mixed granularity adaptive scheduling system, characterized in that, include: The acquisition module is used to acquire the operating status of each heterogeneous computing power unit in the AI ​​computing all-in-one machine, associate and characterize the operating status according to the resource level, and generate a real-time computing power status that reflects the computing capacity and resource continuity of each heterogeneous computing power unit. The partitioning module is used to obtain a set of AI tasks to be scheduled, determine the task granularity based on the computational characteristics and execution constraints of each AI task, and partition each AI task according to the task granularity to form coarse-grained task units and fine-grained task units. The allocation module is used to determine candidate heterogeneous computing power units based on the matching relationship between the computing requirements of each task unit and the real-time computing power status, and to allocate the coarse-grained task units to the matching candidate heterogeneous computing power units to form the main computing power occupancy status. The identification module is used to identify fragmented computing power that can independently carry out computing based on the main computing power occupancy status and the updated real-time computing power status, generate fragmented computing power status, select target fine-grained task units whose resource requirements match the fragmented computing power status from the fine-grained task units, and fill them into the corresponding fragmented computing power. The update module is used to update the real-time computing power status during task execution. When the load status or task priority status meets the scheduling adjustment conditions, the module determines the available computing power based on the task occupancy status, adjusts the execution status or resource mapping relationship of the corresponding task unit, and updates the subsequent task scheduling results.

Images