A method for AI model deployment

Abstract

The present invention relates to the field of AI model analysis technology, and specifically to a method for AI model deployment. In the present invention, environmental parameters of multiple computing nodes are collected in real time, and the environmental parameters include resource fluctuation characteristics, data flow complexity and energy consumption constraint status; the time series change rate of the resource fluctuation characteristics in a single computing node and the energy consumption threshold boundary of the energy consumption constraint status of a single computing node, as well as the coupling degree of data flow complexity between multiple computing nodes are dynamically analyzed to generate a model topology reconstruction strategy, which includes a modular subgraph set, a resource binding relationship and a data routing path; based on the dependent topology of the modular subgraph set in the model topology reconstruction strategy, the partition mapping rules of the resource binding relationship and the switching priority of the data routing path, the dynamic seamless migration of the old and new subgraphs is collaboratively performed.

Description

Technical Field

[0001] The present invention relates to the field of AI model analysis technology, and specifically to a method for AI model deployment. Background Art

[0002] Traditional AI model deployment systems typically use static resource configurations and fixed topologies, making them difficult to adapt to dynamically changing computing environments. In edge computing, cloud computing, and hybrid architecture scenarios, the resource utilization of computing nodes is significantly affected by load fluctuations, network instability, and hardware heterogeneity, leading to problems such as unbalanced GPU / CPU computing power allocation and increased memory fragmentation. Furthermore, complex AI models (such as multimodal neural networks or distributed inference pipelines) have strong inter-task data dependencies, and cross-node transmission is susceptible to network latency and bandwidth limitations, causing task queueing and spikes in end-to-end latency.

[0003] Furthermore, strict energy consumption constraints (such as battery limits for edge devices or energy efficiency targets for data centers) conflict with the rigid resource binding of traditional deployment strategies, often leading to resource mismatches where some nodes are overloaded while others are idle. Existing technologies lack the ability to jointly perceive resource fluctuations, data flow topology, and energy consumption in real time. Furthermore, reconfiguration strategies often rely on offline pre-configuration or global rescheduling, making it difficult to achieve rapid adaptive adjustments in dynamic environments. This leads to decreased service stability and increased operational costs. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for deploying an AI model to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for deploying an AI model, the method comprising the following steps:

[0006] S1. Real-time collection of environmental parameters of multiple computing nodes, including resource fluctuation characteristics, data flow complexity, and energy consumption constraint status;

[0007] S2. Dynamically analyze the time series change rate of resource fluctuation characteristics in a single computing node, the energy consumption threshold boundary of the energy consumption constraint state, and the coupling degree of data flow complexity between multiple computing nodes to generate a model topology reconstruction strategy. The model topology reconstruction strategy includes a modular subgraph set, resource binding relationships, and data routing paths;

[0008] S3. Based on the dependency topology of the modular subgraph set in the model topology reconstruction strategy, the partition mapping rules of the resource binding relationship, and the switching priority of the data routing path, the dynamic and seamless migration of the old and new subgraphs is collaboratively performed.

[0009] As a further improvement of this technical solution, the resource fluctuation characteristics include a computing power occupancy rate fluctuation value and a memory fragmentation index, wherein the computing power occupancy rate fluctuation value V is calculated by sliding window statistics, and the calculation formula is as follows:

[0010] Among them, Mmax represents the maximum value of computing power occupied in the sliding window; Mmin represents the minimum value of computing power occupied in the sliding window; Mavg represents the average value of computing power occupied in the sliding window.

[0011] As a further improvement of the present technical solution, the data flow complexity includes data dependency intensity and average transmission link delay.

[0012] As a further improvement of the present technical solution, the energy consumption constraint state includes an upper limit of energy consumption of a single computing node and a lower limit of energy consumption of a single computing node, wherein the calculation formulas of the upper limit of energy consumption of a single computing node and the lower limit of energy consumption of a single computing node are as follows:

[0013] Emax=Pthreshold·(1+γ), Emin=Pthreshold·(1-γ), where Emax represents the upper limit of energy consumption of a single computing node; Emin represents the lower limit of energy consumption of a single computing node; Pthreshold represents the rated power consumption of a single computing node; and γ represents the tolerance factor.

[0014] As a further improvement to the present technical solution, the time series change rate of the resource fluctuation characteristics in the single computing node is calculated by the computing power utilization fluctuation value in the resource fluctuation characteristics, and the calculation formula is as follows:

[0015] Where Ru represents the time series change rate of the resource fluctuation characteristics of computing node u; Vu_t and Vu_t-1 are the fluctuation values ​​of the computing power occupancy rate of the same computing node u at adjacent moments; Δt represents the time change difference.

[0016] As a further improvement of the present technical solution, the energy consumption threshold boundary of the energy consumption constraint state of the single computing node is determined by the energy consumption upper limit and the energy consumption lower limit of the single computing node in the energy consumption constraint state, and is expressed as an interval. The energy consumption threshold boundary of the energy consumption constraint state of the single computing node is (Emin, Emax).

[0017] As a further improvement to the present technical solution, the coupling degree of the data flow complexity between the multiple computing nodes is calculated by the data dependency intensity in the data flow complexity and the average delay of the transmission link, and the calculation formula is as follows:

[0018] Where Cuv represents the coupling degree of the data flow complexity between computing nodes u and v; Suv represents the data dependency strength between computing nodes u and v; Duv_avg represents the average delay of the transmission link between computing nodes u and v; Dmax represents the preset maximum tolerable delay; w1 and w2 represent the weighting coefficients of the formula.

[0019] As a further improvement to this technical solution, the dependency topology of the modular subgraph set in the model topology reconstruction strategy is calculated by the time series change rate of the resource fluctuation characteristics in a single computing node and the coupling degree of the data flow complexity between multiple computing nodes. The dependency topology is expressed by the following calculation formula:

[0020] Where Si represents the i-th modular subgraph; λ1 is the weighting coefficient of the formula, which is used to balance the influence of the time series change rate of resource fluctuation characteristics in a single computing node and the coupling degree of data flow complexity between multiple computing nodes; It represents the sum of the time series change rates of resource fluctuation characteristics within the modular subgraph and the sum of the coupling degrees of the data flow complexity between nodes within the modular subgraph, which are used to minimize the total time series change rates of resource fluctuation characteristics within the modular subgraph.

[0021] As a further improvement to the present technical solution, the partition mapping rule of the resource binding relationship is calculated by using the time series change rate of the resource fluctuation characteristics in a single computing node and the energy consumption threshold boundary of the energy consumption constraint state of a single computing node. The partition mapping rule is expressed by the following calculation formula:

[0022] Where Ni represents the set of computing nodes where the i-th modular subgraph is deployed; Eu is the current energy consumption of computing node u; λ2 is the weighting coefficient of the formula, which is used to balance the impact of the sum of the time series change rate of resource fluctuation characteristics and the energy consumption deviation; It represents minimizing the sum of the time series change rates and energy consumption deviations of the resource fluctuation characteristics of the modular subgraph on the computing nodes.

[0023] As a further improvement of the present technical solution, the switching priority of the data routing path is calculated by the coupling degree of the data flow complexity between multiple computing nodes and the average delay of the transmission link. The switching priority is expressed by the following calculation formula:

[0024] Where Puv represents the optimal data transmission path from computing node u to computing node v; Cxy represents the coupling degree of data flow complexity between computing node x and computing node y; Dxy_avg represents the average delay of the transmission link between computing node x and computing node y; It represents minimizing the sum of the coupling degrees of the data flow complexity and the sum of the average delay of the transmission link on the data transmission path; λ3 is the weighting coefficient of this formula, which is used to balance the influence of the coupling degree of the data flow complexity and the average delay of the transmission link.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] By collecting resource fluctuation characteristics, data flow complexity and energy consumption constraint status in real time, a dynamic perception and multi-dimensional analysis framework is constructed to generate a joint optimization strategy for modular sub-graph sets, resource binding relationships and data routing paths; the topology-driven atomic switching mechanism can ensure the integrity of the task dependency chain during sub-graph migration and avoid data consistency issues; the energy consumption constraint-aware partition mapping rule achieves the collaborative goals of load balancing and optimal energy efficiency, effectively suppressing local resource overload; the dynamic priority optimization of data routing significantly reduces cross-node communication overhead by minimizing the coupling and delay of the transmission path; the synergistic effect of the three enables the system to achieve simultaneous improvement in computing resource utilization, data transmission efficiency and energy consumption balance while ensuring service continuity, which is especially suitable for edge intelligence and distributed AI reasoning scenarios with high dynamics and strong real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the method steps of the present invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0029] See also Figure 1 The present invention provides a technical solution: a method for deploying an AI model, comprising the following steps:

[0030] S1. Real-time collection of environmental parameters of multiple computing nodes, including resource fluctuation characteristics, data flow complexity, and energy consumption constraint status;

[0031] S2. Dynamically analyze the time series change rate of resource fluctuation characteristics in a single computing node, the energy consumption threshold boundary of the energy consumption constraint state, and the coupling degree of data flow complexity between multiple computing nodes to generate a model topology reconstruction strategy. The model topology reconstruction strategy includes a modular subgraph set, resource binding relationships, and data routing paths;

[0032] S3. Based on the dependency topology of the modular subgraph set in the model topology reconstruction strategy, the partition mapping rules of the resource binding relationship, and the switching priority of the data routing path, the dynamic and seamless migration of the old and new subgraphs is collaboratively performed.

[0033] The process for each step specifically includes:

[0034] Step S1 is the foundational stage of the AI ​​model deployment method with dynamic topology reconstruction. Its core lies in real-time collection of key parameters of the deployment environment, including resource fluctuation characteristics, data flow complexity, and energy consumption constraint status, in order to perceive the dynamic changes in computing resources, data flow, and energy consumption. The specific implementation details are as follows:

[0035] Generation of resource fluctuation characteristics: Through the monitoring agent deployed on the computing node, two types of parameters are obtained in real time, including the computing power utilization fluctuation value and the memory fragmentation index.

[0036] The monitoring objects of the computing power utilization fluctuation value are the GPU CUDA core utilization and the CPU instruction execution load rate. The instantaneous value of computing power utilization is recorded at a fixed sampling interval (such as 1 second). The computing power utilization fluctuation value V within the sliding window is calculated through time series analysis (such as sliding window statistics). The calculation formula is as follows:

[0037] Where Mmax represents the maximum value of computing power occupied in the sliding window; Mmin represents the minimum value of computing power occupied in the sliding window; Mavg represents the average value of computing power occupied in the sliding window;

[0038] The memory fragmentation index monitors the memory allocation status of the computing node, including the number and size distribution of contiguous available memory blocks. The memory fragmentation index (e.g., 1-(maximum contiguous memory block size / total available memory size)) is calculated based on the statistical information of the memory management module. A higher index indicates more severe fragmentation.

[0039] Data flow complexity generation: By analyzing the data flow characteristics between AI model tasks, the coupling degree of data flow complexity between computing nodes is quantified to generate data flow complexity, including data dependency intensity and average transmission link latency, where:

[0040] The monitoring object of data dependency intensity is the correlation between input and output data between computing nodes (such as neural network layers or microservice modules). The data dependency intensity is calculated by counting the proportion of data transmitted from computing node u to computing node v per unit time.

[0041] The average transmission link delay monitors the end-to-end delay of data transmission across computing nodes, including network transmission delay and serialization / deserialization overhead. The time difference between sending and receiving data packets is recorded using timestamps, and the average delay is dynamically calculated (such as the exponentially weighted moving average within a sliding window) as the average transmission link delay.

[0042] Energy Constraint Status Generation: Energy consumption-related parameters are monitored in real time through hardware sensors, including the upper and lower limits of energy consumption for a single computing node. The upper and lower limits are calculated using the following formulas:

[0043] Emax=Pthreshold·(1+γ), Emin=Pthreshold·(1-γ), where Emax represents the upper limit of energy consumption of a single computing node; Emin represents the lower limit of energy consumption of a single computing node; Pthreshold represents the rated power consumption of a single computing node; and γ represents the tolerance factor.

[0044] S2. Calculate the time series change rate of the resource fluctuation characteristics of a single computing node based on the computing power utilization fluctuation value in the resource fluctuation characteristics. The calculation formula is as follows:

[0045] Where Ru represents the time series change rate of the resource fluctuation characteristics of computing node u; Vu_t and Vu_t-1 are the fluctuation values ​​of the computing power occupancy rate of the same computing node u at adjacent moments; Δt represents the time change difference.

[0046] According to the energy consumption upper limit and lower limit of a single computing node in the energy consumption constraint state, the energy consumption threshold boundary of a single computing node is determined and expressed as an interval. The energy consumption threshold boundary of the energy consumption constraint state of a single computing node is (Emin, Emax).

[0047] The coupling degree of data flow complexity between computing nodes is determined based on the data dependency strength and the average transmission link delay in the data flow complexity. The coupling degree of data flow complexity between computing nodes is generated by weighted fusion of the data dependency strength and the average transmission link delay between multiple computing nodes. The calculation formula is:

[0048] Where Cuv represents the coupling degree of the data flow complexity between computing nodes u and v; Suv represents the data dependency strength between computing nodes u and v; Duv_avg represents the average delay of the transmission link between computing nodes u and v; Dmax represents the preset maximum tolerable delay; w1 and w2 represent the weighting coefficients of the formula.

[0049] In step S2, a model topology reconstruction strategy is generated based on the time series change rate of the resource fluctuation characteristics in a single computing node, the energy consumption threshold boundary of the energy consumption constraint state of a single computing node, and the coupling degree of the data flow complexity between multiple computing nodes. The model topology reconstruction strategy includes a modular subgraph set, resource binding relationships, and data routing paths, as follows:

[0050] Modular subgraph collection: Based on the time series change rate of resource fluctuation characteristics in a single computing node and the coupling degree of data flow complexity between multiple computing nodes, the AI ​​model is decomposed into multiple modular subgraphs, and the dependency topology of the modular subgraph collection is determined. The dependency topology is expressed by the following calculation formula:

[0051] Where Si represents the i-th modular subgraph; λ1 is the weighting coefficient of the formula, which is used to balance the influence of the time series change rate of resource fluctuation characteristics in a single computing node and the coupling degree of data flow complexity between multiple computing nodes; It represents the sum of the time series change rates of resource fluctuation characteristics within the modular subgraph and the sum of the coupling degrees of data flow complexity between nodes within the subgraph;

[0052] The generation process of modular subgraph sets is:

[0053] Starting from the entire AI model, treat it as a large subgraph; use the above formula to gradually split the model, and each time choose the splitting scheme that minimizes the formula value; repeat the above steps until a certain stopping condition is met (such as the number of subgraphs reaches a predetermined value or further splitting cannot significantly reduce the formula value), and finally obtain a series of modular subgraphs Si, in which the nodes in each subgraph have a low coupling degree between the time series change rate of resource fluctuation characteristics and the data flow complexity.

[0054] Resource binding relationship: Based on the time series change rate of resource fluctuation characteristics in a single computing node and the energy consumption threshold boundary of the energy consumption constraint state of a single computing node, each modular subgraph should be deployed on which specific computing node to determine the partition mapping rule. The partition mapping rule is expressed by the following calculation formula:

[0055] Where Ni represents the set of computing nodes where the i-th modular subgraph is deployed; Eu is the current energy consumption of computing node u; λ2 is the weighting coefficient of the formula, which is used to balance the impact of the sum of the time series change rate of resource fluctuation characteristics and the energy consumption deviation; It represents minimizing the sum of the time series change rates and energy consumption deviations of the resource fluctuation characteristics of the modular subgraph on the computing nodes.

[0056] The process of generating partition mapping rules is as follows:

[0057] List all available computing nodes; for each modular subgraph Si, use the above formula to calculate its deployment cost on different computing nodes; select the computing node that minimizes the formula value as the deployment location of the modular subgraph; update the status of the computing node (such as current energy consumption, load, etc.) and continue to process the next subgraph; finally, obtain the computing node set Ni corresponding to each modular subgraph Si.

[0058] Data routing path: Based on the coupling degree of data flow complexity between multiple computing nodes and the average delay of the transmission link, the optimal data transmission path is calculated to determine the switching priority. The switching priority is expressed by the following calculation formula:

[0059] Where Puv represents the optimal data transmission path from computing node u to computing node v; Cxy represents the coupling degree of data flow complexity between computing node x and computing node y; Dxy_avg represents the average delay of the transmission link between computing node x and computing node y; It represents minimizing the sum of the coupling degrees of the data flow complexity and the sum of the average delay of the transmission link on the data transmission path; λ3 is the weighting coefficient of this formula, which is used to balance the influence of the coupling degree of the data flow complexity and the average delay of the transmission link;

[0060] The data routing path generation process is as follows:

[0061] List all possible data transmission paths; for each path, use the above formula to calculate its total cost; select the path that minimizes the formula value as the optimal data transmission path Puv; and finally obtain the optimal data transmission path Puv from computing node u to computing node v.

[0062] S3, based on the dependency topology of the modular subgraph set in the model topology reconstruction strategy in S2, the partition mapping rules of the resource binding relationship, and the switching priority of the data routing path, collaboratively executes the non-disruptive switching of the old and new subgraphs, specifically including:

[0063] A new set of modular subgraphs is generated using a dependency topology formula for the set of modular subgraphs. This formula ensures low load fluctuation and data dependency within the new subgraph by minimizing the sum of the time series change rates of resource fluctuation characteristics within the subgraph and the sum of the coupling degrees of data flow complexity between nodes within the subgraph.

[0064] The optimal deployment location for each new subgraph is determined using a partition mapping rule formula based on resource binding relationships. This formula minimizes the sum of the time series change rates of the modular subgraph's resource fluctuation characteristics on the compute nodes and the energy consumption deviation. The most suitable compute nodes are selected to deploy the new subgraph, ensuring efficient resource utilization.

[0065] The data transmission path from the old subgraph to the new subgraph is optimized through the data routing path switching priority formula. This formula ensures efficient and reliable data transmission by minimizing the sum of the coupling degree of the data flow complexity on the data transmission path and the sum of the average delay of the transmission link;

[0066] During the switchover process, traffic is gradually transferred from the old subgraph to the new subgraph. First, the new subgraph is started and begins to process some traffic while monitoring the performance and stability of the new subgraph. Gradually, the traffic proportion of the new subgraph is increased and the traffic of the old subgraph is reduced to ensure a smooth transition of the system. Once all traffic has been successfully transferred to the new subgraph and the system performance is stable, the old subgraph is stopped and its occupied resources are released.

[0067] After completing the above steps, the new subgraph completely takes over all tasks of the old subgraph, achieving uninterrupted switching. Throughout the process, through sophisticated traffic management and monitoring, users and services are ensured to be unaffected, achieving a smooth transition.

[0068] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for deploying an AI model, characterized in that: The method steps are as follows: S1. Real-time collection of environmental parameters of multiple computing nodes, including resource fluctuation characteristics, data flow complexity, and energy consumption constraint status; S2. Dynamically analyze the time series change rate of resource fluctuation characteristics in a single computing node, the energy consumption threshold boundary of the energy consumption constraint state, and the coupling degree of data flow complexity between multiple computing nodes to generate a model topology reconstruction strategy. The model topology reconstruction strategy includes a modular subgraph set, resource binding relationships, and data routing paths; wherein: Data flow complexity includes data dependency strength and average transmission link delay. The coupling degree of data flow complexity between multiple computing nodes is calculated by combining the data dependency strength in data flow complexity with the average transmission link delay. The calculation formula is as follows: ,in Represents a compute node Compute nodes The coupling degree of data flow complexity between them; Represents a compute node Compute nodes The strength of data dependence between them; Represents a compute node Compute nodes The average transmission link delay between Indicates the preset maximum tolerable delay; and represents the weighting coefficient of the formula; The modular subgraph set is a set of subgraphs generated by decomposing the AI ​​model. Each subgraph is a modular part of the AI ​​model. S3. Based on the dependency topology of the modular subgraph set in the model topology reconstruction strategy, the partition mapping rules of the resource binding relationship, and the switching priority of the data routing path, the dynamic and seamless migration of the old and new subgraphs is collaboratively performed.

2. The method for deploying an AI model according to claim 1, wherein: The resource fluctuation characteristics include computing power occupancy rate fluctuation value and memory fragmentation index, wherein the computing power occupancy rate fluctuation value is calculated by sliding window statistics. , the calculation formula is as follows: ,in Indicates the maximum value of computing power occupied within the sliding window; Indicates the minimum value of computing power occupied within the sliding window; The average value of computing power used within the sliding window.

3. The method for deploying an AI model according to claim 1, wherein: The energy consumption constraint state includes the upper limit of energy consumption of a single computing node and the lower limit of energy consumption of a single computing node, wherein the calculation formulas of the upper limit of energy consumption of a single computing node and the lower limit of energy consumption of a single computing node are as follows: , ,in Indicates the energy consumption limit of a single computing node; Indicates the lower limit of energy consumption of a single computing node; Indicates the rated power consumption of a single computing node; Represents the tolerance factor.

4. The method for deploying an AI model according to claim 2, wherein: The time series change rate of the resource fluctuation characteristics of a single computing node is calculated by the computing power utilization fluctuation value in the resource fluctuation characteristics. The calculation formula is as follows: ,in Represents a compute node The time series change rate of resource fluctuation characteristics; and The same computing node The fluctuation value of computing power utilization at adjacent moments; Indicates the time difference.

5. The method for deploying an AI model according to claim 3, wherein: The energy consumption threshold boundary of the energy consumption constraint state of the single computing node is determined by the energy consumption upper limit and the energy consumption lower limit of the single computing node in the energy consumption constraint state, and is expressed as an interval. The energy consumption threshold boundary of the energy consumption constraint state of the single computing node is .

6. The method for deploying an AI model according to claim 1, wherein: The dependency topology of the modular subgraph set in the model topology reconstruction strategy is calculated by the time series change rate of the resource fluctuation characteristics in a single computing node and the coupling degree of the data flow complexity between multiple computing nodes. The dependency topology is expressed by the following calculation formula: ,in Indicates the modular subgraphs; is the weighting coefficient of the formula, which is used to balance the influence of the time series change rate of resource fluctuation characteristics in a single computing node and the coupling degree of data flow complexity between multiple computing nodes; It represents the sum of the time series change rates of resource fluctuation characteristics in a single computing node within a modular subgraph, as well as the sum of the coupling degrees of data flow complexity between multiple computing nodes within a modular subgraph.

7. The method for deploying an AI model according to claim 1, wherein: The partition mapping rule of the resource binding relationship is calculated by the time series change rate of the resource fluctuation characteristics in a single computing node and the energy consumption threshold boundary of the energy consumption constraint state of a single computing node. The partition mapping rule is expressed by the following calculation formula: ,in Indicates the A set of computing nodes where modular subgraphs are deployed; Is a computing node Current energy consumption; is the weighting coefficient of the formula, which is used to balance the impact of the sum of the time series change rates of resource fluctuation characteristics and energy consumption deviation in a single computing node; It represents minimizing the sum of the time series change rates and energy consumption deviations of the resource fluctuation characteristics of the modular subgraph in a single computing node.

8. The method for deploying an AI model according to claim 1, wherein: The switching priority of the data routing path is calculated by the coupling degree of the data flow complexity between multiple computing nodes and the average delay of the transmission link. The switching priority is expressed by the following calculation formula: ,in Represents a compute node To the compute node The optimal data transmission path; Represents a compute node Compute nodes The coupling degree of data flow complexity between them; Represents a compute node Compute nodes The average transmission link delay between It represents minimizing the sum of coupling degrees of data flow complexity on the data transmission path and the sum of average delays of the transmission links; It is the weighting coefficient of the formula, which is used to balance the coupling degree of data flow complexity and the influence of the average delay of the transmission link.

Images