An intelligent module computing power scheduling and management system and method

Abstract

This invention discloses an intelligent module computing power scheduling and management system and method, belonging to the field of computing power resource management and distributed computing technology. The system includes a module computing power profiling module, a distributed resource discovery and registration module, a task decomposition and collaborative execution module, a decentralized scheduling decision module, a dynamic resource elastic scheduling module, a secure isolation and trusted execution module, and an intelligent operation and maintenance and self-healing module. Each intelligent module collects local multi-dimensional resource parameters to generate a standardized computing power feature vector, and achieves mutual discovery and resource registration between modules through distributed broadcasting and heartbeat mechanisms. After receiving a computing task, the task undergoes semantic parsing and computation graph decomposition, dividing it into multiple fine-grained inference units. This invention realizes distributed autonomous scheduling of intelligent module computing power resources, eliminates single points of failure in centralized scheduling, and improves the efficiency of computing power collaboration between heterogeneous modules and system reliability.

Description

Technical Field

[0001] This invention relates to the fields of computing power resource management and distributed computing technology, specifically to an intelligent module computing power scheduling and management system and method. Background Technology

[0002] In the era of booming digital economy, computing power has risen to become a core productive force, permeating all sectors of society. With the deep integration of artificial intelligence and Internet of Things technologies, the Computing Power Module (CPM), as a new type of computing power infrastructure, integrates key elements such as heterogeneous computing, edge sensing, and distributed collaboration. By unifying the abstraction, sensing, scheduling, and management of heterogeneous computing resources such as CPUs, GPUs, and NPUs deployed across various terminal devices, it can accurately schedule the computing requests of upper-layer applications to the optimal module nodes for execution. While ensuring the quality of business services, it significantly improves the utilization rate of the entire network's module computing power and achieves global optimized scheduling of distributed computing power.

[0003] In intelligent module computing networks, due to the different instruction set architectures of modules from different manufacturers, computing tasks require multi-hop node collaborative processing. Task allocation lacks awareness of the characteristics of heterogeneous computing power, resulting in low computing efficiency. Access to computing resources relies on initial static configuration, failing to adapt to the dynamic addition and removal of modules in different scenarios. Traditional centralized scheduling architectures only support decisions issued by a single control node, leading to high single-point-of-failure risk and a sharp increase in scheduling latency as the module scale increases. In industrial vision inspection, multi-module collaborative inference is required. Traditional scheduling schemes only support fixed task allocation, unable to handle sudden high-concurrency inspection demands on the production line. When multiple tenants share module computing power, they cannot effectively match heterogeneous computing resources based on task priority, resulting in a backlog of critical inspection tasks. In multimodal intelligent interaction, cross-module high-bandwidth collaborative processing is required. Existing scheduling mechanisms only support preset path forwarding, unable to dynamically adjust task allocation based on real-time module load. During multi-module collaborative inference, the transmission delay of intermediate results is uncontrollable, leading to delayed interactive responses. When performing distributed data analysis, multiple modules need to share a computing power resource pool. However, if the resource pool cannot dynamically sense the real-time computing power reserve of each module and make elastic adjustments, some modules will have long-term idle computing power and some modules will have severely insufficient computing power, which will not meet the user's real-time computing needs. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent module computing power scheduling and management system and method to solve the problems of fragmented intelligent module computing power resources, difficulty in heterogeneous collaboration, single point of failure in centralized scheduling, and insufficient dynamic adaptability in the prior art.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A method for intelligent module computing power scheduling and management, comprising: Step S1: Each intelligent module collects local multi-dimensional resource status parameters, generates a computing power feature vector for each intelligent module based on the multi-dimensional resource status parameters, and periodically updates the computing power feature vector. In step S2, each intelligent module discovers and registers resources with each other through distributed broadcast and heartbeat mechanisms, and each intelligent module maintains a dynamically updated neighbor resource directory locally. Step S3: Receive the computing task, perform semantic parsing and computation graph decomposition on the computing task, and divide the computing task into multiple fine-grained reasoning units; Step S4: A scheduling mechanism based on a distributed consensus protocol is adopted to collaboratively formulate a task allocation scheme among the intelligent modules. Step S5: Each target module executes the assigned inference unit in parallel according to the task allocation scheme, and the execution results are aggregated and returned. Step S6: Monitor the load status of each intelligent module in real time. When the load does not meet the preset balancing conditions, automatically trigger task migration. Step S7: Monitor the operating status of each intelligent module. When a module failure or performance degradation is detected, automatically perform fault isolation and task reassignment.

[0006] Further options: Step S1 specifically includes: Each intelligent module collects local multi-dimensional resource status parameters by calling the operating system kernel interface. These multi-dimensional resource status parameters include: CPU core count and current utilization, GPU model and current utilization, NPU model and current utilization, total memory capacity and remaining capacity, network interface bandwidth and current transmission latency, current power consumption and current temperature. Each intelligent module generates a standardized computing power feature vector based on the collected multi-dimensional resource status parameters: in, This is the unique identifier for the i-th smart module. Let i be the computing power level of the i-th intelligent module. Let i be the computational characteristic vector of the i-th intelligent module. This represents the current available computing power percentage of the i-th intelligent module. Let be the energy consumption coefficient of the i-th intelligent module. Let be the communication latency level of the i-th intelligent module; The computing power level The method for determining it is as follows: in, Let be the normalized value of the integer arithmetic capability of the i-th intelligent module. Let be the normalized value of the floating-point computing power of the i-th intelligent module. Let be the normalized value of the tensor computation capability of the i-th intelligent module. , , For the corresponding weight coefficients, and ; Each intelligent module uses a sliding time window mechanism to process the computing power feature vector. Perform periodic updates.

[0007] Further options: Step S2 specifically includes: When any intelligent module joins the computing network, it sends a joining announcement message through the local network interface. The joining announcement message contains the unique identifier of the intelligent module. and computing power feature vector Summary information; After receiving the join announcement message, the online module adds the new module information to the local neighbor resource directory, replies with an acknowledgment message to the new module, and synchronizes its own resource information to the new module. Each intelligent module sends a heartbeat message to its neighboring modules at a fixed period of Theartbeat. The heartbeat message contains the survival status of the intelligent module and incremental update information of the computing power feature vector. If any intelligent module does not receive a heartbeat message from a neighboring module within N consecutive heartbeat cycles, it marks that neighboring module as offline in the local neighbor resource directory.

[0008] Further options: Step S3 specifically includes: Receive computing task requests submitted by upper-layer applications. The computing task requests include task type identifier, input data, minimum computing power threshold Qmin, and maximum tolerable latency Dmax. The computation task is semantically parsed to identify the computation graph structure of the computation task, which consists of multiple operation nodes and data dependencies between operation nodes. Based on the computational complexity, data dependencies, and memory requirements of each operation node, the computational task is divided into M fine-grained inference units U1, U2, ..., UM, and the minimum computing power level required for each inference unit Uj is labeled. With maximum tolerable delay , where j=1,2,…,M.

[0009] Further options: Step S4 specifically includes: The intelligent module that initiates the scheduling obtains the list of currently available modules and the real-time computing power feature vector of each available module through the local neighbor resource directory; Each candidate module constructs a local cost matrix based on its local resource status. The element costj,k of the local cost matrix represents the expected cost of assigning the j-th inference unit to the k-th candidate module for execution, where j=1,2,…,M, k=1,2,…,K, and K is the total number of candidate modules. The expected cost, costj,k is calculated as follows: in, The estimated computation time for the j-th inference unit on the k-th candidate module is given. The estimated communication time between the j-th inference unit and the k-th candidate module. The estimated energy consumption for the j-th inference unit to execute on the k-th candidate module. To preset the maximum allowable energy consumption, , , For the corresponding weight coefficients, and ; Each intelligent module elects a temporary coordinating node through a distributed consensus protocol; The temporary coordination node collects the local cost matrices of each candidate module and merges them to generate a global cost matrix. ; The temporary coordination node operation scheduling optimization algorithm solves for the globally optimal matching scheme. The objective function of the scheduling optimization algorithm is: The constraints are: in, As decision variables, =1 indicates that the j-th inference unit is assigned to the k-th candidate module. =0 means no allocation. This represents the current available computing power level of the k-th candidate module; The temporary coordination node initiates a consensus vote by using the globally optimal matching scheme as a consensus log entry. After obtaining confirmation from more than half of the candidate modules, it broadcasts the submission. Each module atomically updates its local scheduling table after receiving the submission message. When the temporary coordinating node experiences a heartbeat timeout or a commit timeout, a re-election is automatically triggered, and the matching process is restarted based on the most recent stable snapshot.

[0010] Further solution: The scheduling optimization algorithm is the weighted Hungarian algorithm, and the solution method is as follows: Constructing an augmented cost matrix in To preset the penalty value, ; For the augmented cost matrix Perform the Hungarian algorithm to find the solution. The minimum permutation π, where The module number assigned to the j-th inference unit.

[0011] Further solution: The scheduling optimization algorithm is a genetic algorithm, and the solution method is as follows: Initialize population Each individual This represents a matching scheme, where Npop is the preset population size; Define the fitness function fitness(S) as follows: in, To match the values ​​of the decision variables in scheme S, μ is the number of times that matching scheme S violates the constraints, and μ is the preset penalty coefficient. The current population is subjected to selection, crossover, and mutation operations to generate the next generation population. This process is repeated until the preset maximum number of iterations Gmax is reached. The individual with the highest fitness is then output as the globally optimal matching scheme.

[0012] Further options: Step S5 specifically includes: After receiving the assigned inference unit, each target module executes the computation task locally; Each target module uses a task-specific session key to encrypt and transmit intermediate calculation results during execution; After each target module completes its calculation, it will return the execution result to the initiating module or the specified result aggregation node. The result aggregation node collects the execution results of all inference units, merges and post-processes them according to the data dependencies of the computation graph structure, generates the final computation result, and returns it.

[0013] Further options: Step S6 specifically includes: Dynamically monitor the real-time load rate of each intelligent module With task queue length i is the serial number of the intelligent module; The real-time load rate The calculation method is as follows: in, Let be the CPU utilization of the i-th intelligent module. Let i be the GPU utilization of the i-th intelligent module. Let NPU utilization be the value of the i-th smart module. Let i be the memory utilization rate of the i-th smart module. , , , For the corresponding weight coefficients, and ; When the real-time load rate of the i-th smart module is detected Exceeding the preset high load threshold or task queue length Exceeding the preset queue length threshold When this occurs, the load balancing process is triggered; The source module that triggers the load balancing process broadcasts a task migration request to its neighboring modules. Each neighboring module then allocates its remaining computing power accordingly. and the current task queue length Calculate migration intention score : in, The remaining computing power of the k-th neighbor module, Let this be the current task queue length of the k-th neighbor module. The communication delay between the i-th module and its k-th neighboring module is... , , For the corresponding weight coefficients, and ; The source module selects a migration intention value. The largest target module will migrate some inference units of backlogged or newly arrived tasks to the target module for execution. When the real-time load rate of the source module Reduced to a preset low load threshold Stop task migration if the following conditions are met.

[0014] Further options: Step S7 specifically includes: Detection probes are deployed along the data flow path of each intelligent module, and diagnostic tags are embedded in the message header of the detection probes; After each intelligent module identifies the diagnostic marker, it records the passing time in the locally maintained diagnostic timestamp table. The diagnostic marker is then compared with the module's own hardware counter value. Perform correlation to generate raw data for hop-by-hop latency and packet loss indication; When the i-th smart module is detected to meet any of the following abnormal conditions, the module is marked as abnormal and temporarily isolated from the neighbor resource directory: Abnormal condition 1: Continuous The response timed out. Abnormal condition 2: Continuous packet loss rate Exceeding the preset packet loss rate threshold ; Anomaly 3: Computing power feature vector The current available computing power percentage Below the preset available computing power threshold And the duration exceeds the preset time threshold. ; Automatically trigger the rescheduling of incomplete tasks on modules marked as abnormal, and assign the affected tasks to other available modules for execution; Once the module marked as abnormal returns to normal, the registration process in step S2 is executed again to rejoin the computing power network. Record abnormal feature vectors in the local performance optimization suggestion library. Repair strategy mapping relationship When the same abnormal characteristics reappear, the mapping relationship is invoked to issue a pre-repair instruction in advance.

[0015] An intelligent module computing power scheduling and management system, comprising: The module computing power profiling module is deployed locally in each intelligent module. It is used to collect local multi-dimensional resource status parameters, generate computing power feature vectors for each intelligent module, and periodically update the computing power feature vectors. The distributed resource discovery and registration module is used to realize mutual discovery and resource registration between various intelligent modules. Each intelligent module maintains a dynamically updated neighbor resource directory locally. The task decomposition and collaborative execution module is used to receive computing tasks, perform semantic parsing and computation graph decomposition on the computing tasks, and divide the computing tasks into multiple fine-grained reasoning units. The decentralized scheduling decision module is used to collaboratively formulate task allocation schemes among various intelligent modules using a scheduling mechanism based on a distributed consensus protocol. The dynamic resource elastic scheduling module is used to monitor the load status of each intelligent module in real time. When the load is detected to be not meeting the preset balancing conditions, task migration is automatically triggered. The security isolation and trusted execution module is used to assign independent session keys and execution contexts to each computing task and to perform end-to-end encryption on the data transmitted between modules. The intelligent operation and maintenance and self-healing module is used to monitor the operating status of each intelligent module. When a module failure or performance degradation is detected, it automatically performs fault isolation and task reassignment.

[0016] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the intelligent module computing power scheduling and management method described above.

[0017] A storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described intelligent module computing power scheduling and management method.

[0018] The present invention has the following beneficial effects: This invention aggregates idle computing power scattered across various intelligent modules into a unified computing resource pool through a distributed resource discovery and dynamic load balancing mechanism, enabling on-demand scheduling and full utilization of computing resources and significantly improving the overall utilization rate of module computing resources. By adopting a decentralized scheduling architecture based on distributed consensus protocols such as Raft, the failure of any module does not affect the normal operation of the entire computing network, eliminating the single point of failure risk of traditional centralized scheduling schemes, and giving the system inherent high availability and fault tolerance. At the same time, scheduling decisions are completed collaboratively among modules, eliminating the need for frequent communication with remote central nodes, effectively reducing scheduling latency and communication overhead, making it particularly suitable for latency-sensitive real-time edge computing scenarios.

[0019] This invention combines task decomposition technology to break down complex computing tasks into fine-grained inference units adapted to the computing power characteristics of different modules, realizing computing power synergy and complementary advantages among heterogeneous modules. Through a task dynamic migration mechanism based on real-time load monitoring, it effectively solves the problems of fragmentation and uneven utilization of module computing power resources. Moreover, new modules can join the computing power network through simple broadcast announcements without complex central configuration, and the system scale can grow naturally with the number of modules. In addition, through task-level session keys and end-to-end encrypted transmission, it ensures data isolation and computing security for multiple tenants in a distributed module network without a central trust anchor, ensuring that computing tasks of different tenants do not interfere with each other and data is not visible to each other when sharing module resources. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a logic block diagram of the system of the present invention. Detailed Implementation

[0021] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0022] The intelligent module computing power scheduling and management method and system disclosed in this application, through the high integration of multiple core modules such as module computing power profiling, distributed resource discovery and registration, decentralized scheduling decision-making, task decomposition and collaborative execution, dynamic resource elastic scheduling, secure isolation and trusted execution, and intelligent operation and maintenance and self-healing, aims to achieve distributed autonomous scheduling, heterogeneous computing power collaboration, and efficient utilization of intelligent module computing power resources. The system possesses strong decentralized autonomous capabilities, enabling it to accurately perceive the real-time computing power status and task requirements of each module through peer-to-peer communication and distributed consensus protocols between modules without relying on a central server. Based on real-time data, it intelligently adjusts and optimizes the allocation and deployment of computing power resources, ensuring that every computing task is met promptly and fully. Simultaneously, the system implements strict security isolation measures throughout the entire process of data transmission and computation processing, effectively protecting the security and privacy of data in multi-tenant scenarios.

[0023] The following will describe in detail the intelligent module computing power scheduling and management method involved in this application. Specific details such as particular system structures and technologies are proposed for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details.

[0024] Please see Figure 1 The diagram shows a flowchart of a method for intelligent module computing power scheduling and management in a specific embodiment. The method includes: Step S101: Each intelligent module collects local multi-dimensional resource status parameters, generates a computing power feature vector for each intelligent module based on the multi-dimensional resource status parameters, and periodically updates the computing power feature vector.

[0025] In some embodiments, after each intelligent module powers on, the module's computing power profiling module collects local multi-dimensional resource status parameters by calling operating system kernel interfaces (such as the / proc file system, system calls, etc.). These multi-dimensional resource status parameters include: the number of CPU cores and their current utilization, the GPU model and its current utilization, the NPU model and its current utilization, the total memory capacity and remaining capacity, the network interface bandwidth and current transmission latency, and the current power consumption and current temperature. Based on the collected multi-dimensional resource status parameters, each intelligent module generates a standardized computing power feature vector. in, This is the unique identifier for the i-th smart module. Let i be the computing power level of the i-th intelligent module. This is the computational characteristic vector of the i-th intelligent module (indicating the types of computation that the module excels at, including tensor operations, logic control, and parallel data stream processing). This represents the current available computing power percentage of the i-th intelligent module. Let be the energy consumption coefficient of the i-th intelligent module. Let be the communication latency level of the i-th smart module.

[0026] The computing power level The method for determining it is as follows: in, Let be the normalized value of the integer arithmetic capability of the i-th intelligent module. Let be the normalized value of the floating-point computing power of the i-th intelligent module. Let be the normalized value of the tensor computation capability of the i-th intelligent module. , , For the corresponding weight coefficients, and In this embodiment, the values ​​of each weight coefficient are preset according to the application scenario of the module. For example, for AI inference-intensive scenarios, Value greater than and Each intelligent module employs a sliding time window mechanism to process the computing power feature vector. Periodic updates are performed, with an update cycle of Tupdate every 30 seconds, to ensure the timeliness of the computing power profile.

[0027] Step S102: Each intelligent module discovers and registers resources with each other through distributed broadcast and heartbeat mechanisms. Each intelligent module maintains a dynamically updated neighbor resource directory locally.

[0028] In some embodiments, when any intelligent module joins the computing power network, it sends a joining announcement message through the local network interface. The joining announcement message contains a unique identifier for the intelligent module. and computing power feature vector The summary information. After receiving the join announcement message, the online module adds the new module information to its local neighbor resource directory, replies with an acknowledgment message to the new module, and simultaneously synchronizes its own resource information to the new module.

[0029] Each intelligent module sends heartbeat messages to its neighboring modules at a fixed interval (Theartbeat, 30 seconds). The heartbeat message contains the intelligent module's liveness status and incremental updates to its computing power feature vector. If any intelligent module does not receive a heartbeat message from a neighboring module within Nmiss heartbeat cycles, Nmiss is set to 3, and the neighboring module is marked as offline in the local neighbor resource directory. This entire process is based on point-to-point communication between modules, eliminating the need for a central registration server and enabling distributed autonomous discovery of computing resources.

[0030] Step S103: Receive the computing task, perform semantic parsing and computation graph decomposition on the computing task, and divide the computing task into multiple fine-grained reasoning units.

[0031] In some embodiments, the system receives a computing task request submitted by an upper-layer application. The computing task request includes a task type identifier, input data, a minimum computing power threshold Qmin, and a maximum tolerable latency Dmax. The task decomposition and collaborative execution module performs semantic parsing on the computing task to identify the computing graph structure of the computing task. The computing graph structure consists of multiple operation nodes and the data dependencies between the operation nodes.

[0032] Based on the computational complexity, data dependencies, and memory requirements of each operation node, the computational task is divided into M fine-grained inference units U1, U2, ..., UM, and the minimum computing power level required for each inference unit Uj is labeled. With maximum tolerable delay Where j = 1, 2, ..., M. In this embodiment, the task decomposition adopts a partitioning strategy based on computation graph topology sorting to ensure that operation nodes with dependencies are partitioned into the same inference unit or allocated to different inference units according to the dependency order, thereby avoiding data races.

[0033] Step S104: Adopt a scheduling mechanism based on a distributed consensus protocol to collaboratively formulate a task allocation scheme among the various intelligent modules.

[0034] In some embodiments, the intelligent module initiating the scheduling obtains a list of currently available modules and a real-time computing power feature vector for each available module through a local neighbor resource directory. Each candidate module constructs a local cost matrix based on its local resource status. The element costj,k of the local cost matrix represents the expected cost of assigning the j-th inference unit to the k-th candidate module for execution, where j=1,2,…,M and k=1,2,…,K is the total number of candidate modules.

[0035] The expected cost, costj,k is calculated as follows: in, The estimated computation time for the j-th inference unit on the k-th candidate module is given. The estimated communication time between the j-th inference unit and the k-th candidate module. The estimated energy consumption for the j-th inference unit to execute on the k-th candidate module. To preset the maximum allowable energy consumption, , , For the corresponding weight coefficients, and In this embodiment, the values ​​of each weight coefficient are dynamically adjusted according to the business scenario. For latency-sensitive businesses, and The value is relatively large; for energy-sensitive businesses, The value is relatively large.

[0036] Each intelligent module elects a temporary coordinating node through a distributed consensus protocol (Raft protocol is used in this embodiment). The temporary coordinating node collects the local cost matrices of each candidate module and merges them to generate a global cost matrix. .

[0037] The temporary coordination node operation scheduling optimization algorithm solves for the globally optimal matching scheme. The objective function of the scheduling optimization algorithm is: The constraints are: in, As decision variables, =1 indicates that the j-th inference unit is assigned to the k-th candidate module. =0 means no allocation. This represents the current available computing power level of the k-th candidate module.

[0038] In this embodiment, the scheduling optimization algorithm is the weighted Hungarian algorithm, and the solution method is as follows: constructing an augmented cost matrix. ; in To preset the penalty value, ; for the augmented cost matrix Perform the Hungarian algorithm to find the solution. The minimum permutation π, where the module number assigned to the j-th reasoning unit. When the number of candidate modules K is greater than or equal to the number of reasoning units M, a M×K cost matrix is directly constructed to execute the Hungarian algorithm; when K<M, virtual modules are introduced and a preset penalty value is used to fill the redundant columns.

[0039] In another embodiment of the present invention, the scheduling optimization algorithm is a genetic algorithm, and the solving manner is: initializing a population , where each individual represents a matching scheme, Npop is the preset population size, and Npop=100 in this embodiment; the fitness function fitness(S) is defined as: wherein, is the value of the decision variable in the matching scheme S, is the number of times the matching scheme S violates the constraint condition, μ is the preset penalty coefficient, and μ=100 in this embodiment; selection operation (using roulette wheel selection), crossover operation (using partially matched crossover, with a crossover probability pc=0.8) and mutation operation (using swap mutation, with a mutation probability pm=0.05) are performed on the current population to generate the next generation population, the execution is repeated until the preset maximum number of iterations Gmax=500 is reached, and the individual with the highest fitness is output as the global optimal matching scheme.

[0040] The temporary coordination node initiates a consensus vote by taking the global optimal matching scheme as a consensus log entry, broadcasts and submits after obtaining the confirmation of more than half (more than K / 2 is required in this embodiment) of the candidate modules, and each module atomically updates the local scheduling table after receiving the submission message. When the temporary coordination node experiences a heartbeat timeout or a submission timeout, re-election is automatically triggered, and the matching process is restarted based on the latest stable snapshot to ensure high availability of the scheduling service.

[0041] Step S105: Each target module executes the assigned reasoning units in parallel according to the task allocation scheme, and returns the execution results after aggregation.

[0042] In some embodiments, after receiving the assigned reasoning units, each target module executes the computing task locally. During the execution process, each target module uses a task-specific session key to encrypt and transmit intermediate computing results. After completing the computation, each target module returns the execution result to the initiating module or a designated result aggregation node. The result aggregation node collects the execution results of all reasoning units, performs merging and post-processing according to the data dependency relationship of the computation graph structure, generates a final computation result and returns it to the upper-layer application.

[0043] Step S106: Real-time monitoring the load status of each intelligent module, and when it is detected that the load does not meet the preset balance condition, task migration is automatically triggered.

[0044] In some embodiments, the dynamic resource elastic scheduling module continuously monitors the real-time load rate of each intelligent module. With task queue length , where i is the serial number of the intelligent module.

[0045] The real-time load rate The calculation method is as follows: in, Let be the CPU utilization of the i-th intelligent module. Let i be the GPU utilization of the i-th intelligent module. Let NPU utilization be the value of the i-th smart module. Let i be the memory utilization rate of the i-th smart module. , , , For the corresponding weight coefficients, and In this embodiment, the default values ​​for each weight coefficient are... =0.30、 =0.25、 =0.25、 =0.20.

[0046] When the real-time load rate of the i-th smart module is detected Exceeding the preset high load threshold =0.85, or task queue length Exceeding the preset queue length threshold When the value is 20, the load balancing process is triggered.

[0047] The source module that triggers the load balancing process broadcasts a task migration request to its neighboring modules. Each neighboring module then allocates its remaining computing power accordingly. and the current task queue length Calculate migration intention score : in, The remaining computing power of the k-th neighbor module, Let this be the current task queue length of the k-th neighbor module. The communication delay between the i-th module and its k-th neighboring module is... , , For the corresponding weight coefficients, and In this embodiment, the values ​​of each weight coefficient are... =0.50、 =0.30、 =0.20.

[0048] The source module selects a migration intention value. The largest target module will have a portion of its inference units, either from backlogged tasks or newly arriving tasks, migrated for execution. The number of inference units migrated each time is 30% of the current queue length. This migration is based on the real-time load rate of the source module. Reduced to a preset low load threshold When the value is below 0.60, task migration will stop.

[0049] Step S107: Monitor the operating status of each intelligent module. When a module failure or performance degradation is detected, automatically perform fault isolation and task reassignment.

[0050] In some embodiments, detection probes are deployed along the data flow path of each intelligent module, and these probes embed diagnostic markers in the message header. After recognizing the diagnostic markers, each intelligent module records the passage time in a locally maintained diagnostic timestamp table. The diagnostic marker is then compared with the module's own hardware counter value. Perform correlation to generate raw data for hop-by-hop latency and packet loss indication.

[0051] When the i-th smart module is detected to meet any of the following abnormal conditions, the module is marked as abnormal and temporarily isolated from the neighbor resource directory: Abnormal condition 1: Continuous =3 response timeouts; Abnormal condition 2: Continuous packet loss rate Exceeding the preset packet loss rate threshold =5%; Anomaly 3: Computing power feature vector The current available computing power percentage Below the preset available computing power threshold =20% and the duration exceeds the preset time threshold. =10 seconds.

[0052] Automatic rescheduling of unfinished tasks on modules marked as abnormal is triggered, allocating affected tasks to other available modules for execution. Once a module marked as abnormal recovers, the registration process in step S102 is re-executed to rejoin the computing network. Abnormal feature vectors are recorded in the local performance optimization suggestion library. Repair strategy mapping relationship When the same abnormal characteristics reappear, the mapping relationship is invoked to issue a pre-repair instruction in advance, forming a continuous self-learning performance optimization closed loop.

[0053] It should be understood that the sequence numbers of the steps in the above embodiments do not imply the order of execution, and the execution order of each process shall be determined according to its function and inherent logic, and shall not constitute any limitation on the implementation process of the embodiments of the present invention.

[0054] In an embodiment of the present invention, based on step S101, a possible embodiment will be given below to illustrate its specific implementation in a non-restrictive manner. Step S101 specifically includes the following steps: Step S1011: Each intelligent module collects local multi-dimensional resource state parameters by calling an operating system kernel interface.

[0055] Step S1012: Each intelligent module generates a standardized computing power feature vector based on the collected multi-dimensional resource state parameters , where is the unique identifier of the i-th intelligent module, and the unique identifier is generated by combining the module MAC address and the factory serial number. The computing power feature vector is a 32-bit binary bitmap, and each bit indicates the support status of a type of computing capability.

[0056] Step S1013: Each intelligent module periodically updates the computing power feature vector by adopting a sliding time window mechanism , the length of the update time window is 5 sampling periods, and each sampling period is Tupdate=30 seconds.

[0057] In an embodiment of the present invention, based on step S104, a possible embodiment will be given below to illustrate its specific implementation in a non-restrictive manner. Step S104 specifically includes the following steps: Step S1041: Each intelligent module elects a temporary coordination node through the Raft distributed consensus protocol. In the Raft protocol, the election timeout is randomly set between 150ms and 300ms, so as to avoid vote splitting caused by multiple modules initiating elections at the same time.

[0058] Step S1042: The temporary coordination node collects the local cost matrix of each candidate module, and merges them to generate a global cost matrix .

[0059] Step S1043: The temporary coordination node runs a scheduling optimization algorithm to solve the global optimal matching scheme. The scheduling optimization algorithm is a weighted Hungarian algorithm, and the specific implementation manner in this embodiment is as follows: first, check the relationship between the number K of candidate modules and the number M of inference units; if K≥M, directly construct an M×K cost matrix to execute the Hungarian algorithm; if K<M, construct an M×M augmented cost matrix, introduce (M-K) virtual modules, and all cost values corresponding to the virtual modules are set to preset penalty values =106. After executing the Hungarian algorithm, the inference units allocated to the virtual module are marked as scheduling failures, triggering the subsequent rescheduling process.

[0060] Step S1044: The temporary coordinating node initiates a consensus vote by using the globally optimal matching scheme as a consensus log entry, and broadcasts the submission after obtaining confirmation from more than half (≥⌊K / 2⌋+1) of the candidate modules.

[0061] Step S1045: When the temporary coordination node experiences a heartbeat timeout (no response for 3 consecutive heartbeat cycles) or a commit timeout (failure to complete the commit for more than 5 seconds), a re-election is automatically triggered, and the matching process is restarted based on the most recent stable snapshot.

[0062] In one embodiment of the present invention, based on step S106, the following is a possible embodiment and its specific implementation will be described in a non-limiting manner. Step S106 specifically includes the following steps: Step S1061: The dynamic resource elastic scheduling module continuously monitors the real-time load rate of each intelligent module. With task queue length The monitoring cycle is 5 seconds.

[0063] Step S1062: When the real-time load rate of the i-th smart module is detected... Exceeding the preset high load threshold or task queue length Exceeding the preset queue length threshold When this occurs, the load balancing process is triggered.

[0064] Step S1063: The source module that triggers the load balancing process broadcasts a task migration request to its neighboring modules. Each neighboring module then migrates its task based on its remaining computing power. The migration intention value is calculated based on the current task queue length qk. Source module selection migration intention value The largest target module performs the migration.

[0065] Step S1064: When the real-time load rate of the source module Reduced to a preset low load threshold Stop task migration if the following conditions are met.

[0066] In one embodiment of the present invention, based on step S107, the following is a possible embodiment and its specific implementation will be described in a non-limiting manner. Step S107 specifically includes the following steps: Step S1071: Deploy detection probes on the data flow path of each intelligent module. The detection probes embed diagnostic markers in optional fields of the message header. The diagnostic markers are 32 bits long, of which the first 16 bits are probe identifiers and the last 16 bits are sequence numbers. The sequence numbers are incremented according to the message sending order.

[0067] Step S1072: After each intelligent module identifies the diagnostic marker, it records the passing time in the locally maintained diagnostic timestamp table. The timestamp accuracy is in the microsecond range.

[0068] Step S1073: The intelligent operation and maintenance module maintains a health score for each module. The initial value is 100, and when a response timeout occurs... Decrease by 10 when a packet loss occurs. Subtract 5, when Task reassignment is triggered when the value falls below the threshold of 60.

[0069] Step S1074: After the module marked as abnormal returns to normal, the registration process of step S102 is executed again to rejoin the computing power network.

[0070] Step S1075: The system records the abnormal feature vector in the local performance optimization suggestion library. With repair strategy S The mapping relationship is used to issue a pre-repair instruction in advance when the same abnormal characteristics reappear.

[0071] The following are embodiments of the intelligent module computing power scheduling and management system provided in this disclosure. This system and the intelligent module computing power scheduling and management methods of the above embodiments belong to the same inventive concept. Details not described in detail in the system embodiments can be found in the embodiments of the above methods. Please refer to... Figure 2 The system includes: The module computing power profiling module is deployed locally in each intelligent module. It is used to collect local multi-dimensional resource status parameters, generate computing power feature vectors for each intelligent module, and periodically update the computing power feature vectors. The distributed resource discovery and registration module is used to realize mutual discovery and resource registration between various intelligent modules. Each intelligent module maintains a dynamically updated neighbor resource directory locally. The task decomposition and collaborative execution module is used to receive computing tasks, perform semantic parsing and computation graph decomposition on the computing tasks, and divide the computing tasks into multiple fine-grained reasoning units. The decentralized scheduling decision module is used to collaboratively formulate task allocation schemes among various intelligent modules using a scheduling mechanism based on a distributed consensus protocol. The dynamic resource elastic scheduling module is used to monitor the load status of each intelligent module in real time. When the load is detected to be not meeting the preset balancing conditions, task migration is automatically triggered. The security isolation and trusted execution module is used to assign independent session keys and execution contexts to each computing task and to perform end-to-end encryption on the data transmitted between modules. The intelligent operation and maintenance and self-healing module is used to monitor the operating status of each intelligent module. When a module failure or performance degradation is detected, it automatically performs fault isolation and task reassignment.

[0072] Among them, the module computing power profiling module can collect and manage the local heterogeneous computing power resources (CPU, GPU, NPU, etc.) of each intelligent module. Through standardized processing of resource information, it provides the scheduling decision module with a unified computing power feature vector, realizing the unified description and flexible use of heterogeneous computing power.

[0073] The distributed resource discovery and registration module is responsible for managing peer-to-peer communication between modules, including module joining announcements, heartbeat maintenance, and offline detection, ensuring that each module can independently discover neighboring modules and their resource status without relying on a central server.

[0074] The decentralized scheduling decision module generates task allocation schemes on demand based on the computing power requirements of business tasks, combined with the real-time computing power feature vectors of each module and network resource status information, and ensures the consistency of scheduling results.

[0075] The task decomposition and collaborative execution module performs semantic parsing and computation graph decomposition on the complex computing tasks submitted by the upper-layer application, breaking them down into multiple fine-grained inference units that adapt to the computing power characteristics of different modules, and then aggregates the results after the collaborative execution of each module is completed.

[0076] The dynamic resource elastic scheduling module monitors the load rate and task queue length of each module in real time, and automatically triggers task migration when uneven load is detected, thereby achieving dynamic balancing of computing resources.

[0077] The security isolation and trusted execution module achieves computing power isolation and data security in multi-tenant scenarios by building a task-level encryption and isolation mechanism.

[0078] The intelligent operation and maintenance and self-healing module deploys detection probes on the data flow path to collect hop-by-hop performance data in real time. When a module failure is detected, it automatically performs fault isolation and task reassignment to ensure the continuous and stable operation of the system.

[0079] This application also provides an electronic device, including a display module, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of an intelligent module computing power scheduling and management method.

[0080] In embodiments of the present invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, smart module embedded devices, edge computing gateways, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the embodiments described and / or claimed herein.

[0081] In this application embodiment, the processor can be implemented using at least one of an application-specific integrated circuit, a programmable logic device, a field-programmable gate array, a processor, a controller, a microcontroller, a microprocessor, or an electronic unit designed to perform the functions described herein. In some cases, such an implementation can be implemented within a controller. For software implementations, implementations such as processes or functions can be implemented with separate software modules that allow the performance of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, and the software code can be stored in memory and executed by the controller.

[0082] The display module is used to display information input by the user or information provided to the user. The display module may include a display panel, which may be configured using a liquid crystal display, an organic light-emitting diode, or other similar methods.

[0083] Memory can be used to store software programs and various types of data. Memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0084] This application also provides a storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of the intelligent module computing power scheduling and management method.

[0085] The storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent module computing power scheduling and management, characterized in that, include: Step S1: Each intelligent module collects local multi-dimensional resource status parameters, generates a computing power feature vector for each intelligent module based on the multi-dimensional resource status parameters, and periodically updates the computing power feature vector. In step S2, each intelligent module discovers and registers resources with each other through distributed broadcast and heartbeat mechanisms, and each intelligent module maintains a dynamically updated neighbor resource directory locally. Step S3: Receive the computing task, perform semantic parsing and computation graph decomposition on the computing task, and divide the computing task into multiple fine-grained reasoning units; Step S4: A scheduling mechanism based on a distributed consensus protocol is adopted to collaboratively formulate a task allocation scheme among the intelligent modules. Step S5: Each target module executes the assigned inference unit in parallel according to the task allocation scheme, and the execution results are aggregated and returned. Step S6: Monitor the load status of each intelligent module in real time. When the load does not meet the preset balancing conditions, automatically trigger task migration. Step S7: Monitor the operating status of each intelligent module. When a module failure or performance degradation is detected, automatically perform fault isolation and task reassignment.

2. The intelligent module computing power scheduling and management method according to claim 1, characterized in that, Step S1 specifically includes: Each intelligent module collects local multi-dimensional resource status parameters by calling the operating system kernel interface. These multi-dimensional resource status parameters include: CPU core count and current utilization, GPU model and current utilization, NPU model and current utilization, total memory capacity and remaining capacity, network interface bandwidth and current transmission latency, current power consumption and current temperature. Each intelligent module generates a standardized computing power feature vector based on the collected multi-dimensional resource status parameters: in, This is the unique identifier for the i-th smart module. Let i be the computing power level of the i-th intelligent module. Let i be the computational characteristic vector of the i-th intelligent module. This represents the current available computing power percentage of the i-th intelligent module. Let be the energy consumption coefficient of the i-th intelligent module. Let be the communication latency level of the i-th intelligent module; The computing power level The method for determining it is as follows: in, Let be the normalized value of the integer arithmetic capability of the i-th intelligent module. Let be the normalized value of the floating-point computing power of the i-th intelligent module. Let be the normalized value of the tensor computation capability of the i-th intelligent module. , , For the corresponding weight coefficients, and ; Each intelligent module uses a sliding time window mechanism to process the computing power feature vector. Perform periodic updates.

3. The intelligent module computing power scheduling and management method according to claim 1, characterized in that, Step S2 specifically includes: When any intelligent module joins the computing network, it sends a joining announcement message through the local network interface. The joining announcement message contains the unique identifier of the intelligent module. and computing power feature vector Summary information; After receiving the join announcement message, the online module adds the new module information to the local neighbor resource directory, replies with an acknowledgment message to the new module, and synchronizes its own resource information to the new module. Each intelligent module sends a heartbeat message to its neighboring modules at a fixed period of Theartbeat. The heartbeat message contains the survival status of the intelligent module and incremental update information of the computing power feature vector. If any intelligent module does not receive a heartbeat message from a neighboring module within N consecutive heartbeat cycles, it marks that neighboring module as offline in the local neighbor resource directory.

4. The intelligent module computing power scheduling and management method according to claim 1, characterized in that, Step S3 specifically includes: Receive computing task requests submitted by upper-layer applications. The computing task requests include task type identifier, input data, minimum computing power threshold Qmin, and maximum tolerable latency Dmax. The computation task is semantically parsed to identify the computation graph structure of the computation task, which consists of multiple operation nodes and data dependencies between operation nodes. Based on the computational complexity, data dependencies, and memory requirements of each operation node, the computational task is divided into M fine-grained inference units U1, U2, ..., UM, and the minimum computing power level required for each inference unit Uj is labeled. With maximum tolerable delay , where j=1,2,…,M.

5. The intelligent module computing power scheduling and management method according to claim 1, characterized in that, Step S4 specifically includes: The intelligent module that initiates the scheduling obtains the list of currently available modules and the real-time computing power feature vector of each available module through the local neighbor resource directory; Each candidate module constructs a local cost matrix based on its local resource status. The element costj,k of the local cost matrix represents the expected cost of assigning the j-th inference unit to the k-th candidate module for execution, where j=1,2,…,M, k=1,2,…,K, and K is the total number of candidate modules. The expected cost, costj,k is calculated as follows: in, The estimated computation time for the j-th inference unit on the k-th candidate module is given. The estimated communication time between the j-th inference unit and the k-th candidate module. The estimated energy consumption for the j-th inference unit to execute on the k-th candidate module. To preset the maximum allowable energy consumption, , , For the corresponding weight coefficients, and ; Each intelligent module elects a temporary coordinating node through a distributed consensus protocol; The temporary coordination node collects the local cost matrices of each candidate module and merges them to generate a global cost matrix. ; The temporary coordination node operation scheduling optimization algorithm solves for the globally optimal matching scheme. The objective function of the scheduling optimization algorithm is: The constraints are: in, As decision variables, =1 indicates that the j-th inference unit is assigned to the k-th candidate module. =0 means no allocation. This represents the current available computing power level of the k-th candidate module; The temporary coordination node initiates a consensus vote by using the globally optimal matching scheme as a consensus log entry. After obtaining confirmation from more than half of the candidate modules, it broadcasts the submission. Each module atomically updates its local scheduling table after receiving the submission message. When the temporary coordinating node experiences a heartbeat timeout or a commit timeout, a re-election is automatically triggered, and the matching process is restarted based on the most recent stable snapshot.

6. The intelligent module computing power scheduling and management method according to claim 5, characterized in that, The scheduling optimization algorithm is the weighted Hungarian algorithm, and the solution method is as follows: Constructing an augmented cost matrix in To preset the penalty value, ; For the augmented cost matrix Perform the Hungarian algorithm to find the solution. The minimum permutation π, where The module number assigned to the j-th inference unit.

7. The intelligent module computing power scheduling and management method according to claim 5, characterized in that, The scheduling optimization algorithm is a genetic algorithm, and the solution method is as follows: Initialize population Each individual This represents a matching scheme, where Npop is the preset population size; Define the fitness function fitness(S) as follows: in, To match the values ​​of the decision variables in scheme S, μ is the number of times that matching scheme S violates the constraints, and μ is the preset penalty coefficient. The current population is subjected to selection, crossover, and mutation operations to generate the next generation population. This process is repeated until the preset maximum number of iterations Gmax is reached. The individual with the highest fitness is then output as the globally optimal matching scheme.

8. The intelligent module computing power scheduling and management method according to claim 1, characterized in that, Step S5 specifically includes: After receiving the assigned inference unit, each target module executes the computation task locally; Each target module uses a task-specific session key to encrypt and transmit intermediate calculation results during execution; After each target module completes its calculation, it will return the execution result to the initiating module or the specified result aggregation node. The result aggregation node collects the execution results of all inference units, merges and post-processes them according to the data dependencies of the computation graph structure, generates the final computation result, and returns it.

9. The intelligent module computing power scheduling and management method according to claim 1, characterized in that, Step S6 specifically includes: Dynamically monitor the real-time load rate of each intelligent module With task queue length i is the serial number of the intelligent module; The real-time load rate The calculation method is as follows: in, Let be the CPU utilization of the i-th intelligent module. Let i be the GPU utilization of the i-th intelligent module. Let NPU utilization be the value of the i-th smart module. Let i be the memory utilization rate of the i-th smart module. , , , For the corresponding weight coefficients, and ; When the real-time load rate of the i-th smart module is detected Exceeding the preset high load threshold or task queue length Exceeding the preset queue length threshold When this occurs, the load balancing process is triggered; The source module that triggers the load balancing process broadcasts a task migration request to its neighboring modules. Each neighboring module then allocates its remaining computing power accordingly. and the current task queue length Calculate migration intention score : in, The remaining computing power of the k-th neighbor module, Let this be the current task queue length of the k-th neighbor module. The communication delay between the i-th module and its k-th neighboring module is... , , For the corresponding weight coefficients, and ; The source module selects a migration intention value. The largest target module will migrate some inference units of backlogged or newly arrived tasks to the target module for execution. When the real-time load rate of the source module Reduced to a preset low load threshold Stop task migration if the following conditions are met.

10. A system for performing the method according to any one of claims 1-9, characterized in that, include: The module computing power profiling module is deployed locally in each intelligent module. It is used to collect local multi-dimensional resource status parameters, generate computing power feature vectors for each intelligent module, and periodically update the computing power feature vectors. The distributed resource discovery and registration module is used to realize mutual discovery and resource registration between various intelligent modules. Each intelligent module maintains a dynamically updated neighbor resource directory locally. The task decomposition and collaborative execution module is used to receive computing tasks, perform semantic parsing and computation graph decomposition on the computing tasks, and divide the computing tasks into multiple fine-grained reasoning units. The decentralized scheduling decision module is used to collaboratively formulate task allocation schemes among various intelligent modules using a scheduling mechanism based on a distributed consensus protocol. The dynamic resource elastic scheduling module is used to monitor the load status of each intelligent module in real time. When the load is detected to be not meeting the preset balancing conditions, task migration is automatically triggered. The security isolation and trusted execution module is used to assign independent session keys and execution contexts to each computing task and to perform end-to-end encryption on the data transmitted between modules. The intelligent operation and maintenance and self-healing module is used to monitor the operating status of each intelligent module. When a module failure or performance degradation is detected, it automatically performs fault isolation and task reassignment.

Images