Dynamic model switching framework for AI inference optimization methods and systems for edge devices

Abstract

This invention provides a dynamic model switching framework for AI inference optimization methods and systems on edge devices, relating to the field of AI optimization technology. The methods include: correcting quantization errors by training a quantization compensation model through federated learning; constructing a hierarchical heterogeneous resource management system to achieve multi-core collaborative scheduling; integrating compensation parameters into FPGA acceleration circuits and employing intelligent power balancing technology; and utilizing graph neural networks to construct a dynamic decision engine to trigger model switching. This invention achieves a balance between high-precision AI inference and low-power consumption requirements on edge devices, improving computing resource utilization efficiency and reducing energy consumption.

Description

Technical Field

[0001] This invention relates to AI optimization technology, and more particularly to a method and system for optimizing AI inference for edge devices using a dynamic model switching framework. Background Technology

[0002] With the rapid development of artificial intelligence technology, deploying AI applications to edge devices has become an important industry trend. Edge devices, due to their limited computing resources, strict power consumption constraints, and variable operating environments, pose significant challenges to AI model inference. While traditional cloud-based AI inference models offer powerful computing capabilities, they suffer from high network latency and significant privacy and security risks. Therefore, efficiently executing AI inference tasks on edge devices has become a current research hotspot.

[0003] Current AI inference optimization on edge devices is primarily achieved through model compression techniques, including pruning, quantization, and knowledge distillation. These techniques reduce model size and computational complexity, enabling complex AI models to run on resource-constrained edge devices. Meanwhile, the development of heterogeneous computing technologies has also provided new solutions for edge AI, allowing for a better balance between performance and power consumption through the collaborative work of multiple computing units such as CPUs, GPUs, NPUs, and FPGAs.

[0004] Existing model compression methods generally adopt static quantization strategies, which cannot dynamically adjust quantization parameters according to the real-time resource status of the device. This results in inconsistent model accuracy loss under different operating environments, especially in edge scenarios with large resource fluctuations.

[0005] Current heterogeneous computing resource management systems lack effective collaborative scheduling mechanisms, making it impossible to adaptively allocate optimal computing resource combinations based on the characteristics of models with different compression levels. This results in low computing efficiency and wasted power consumption, making it difficult to meet the strict power consumption constraints of edge devices.

[0006] Existing model switching strategies are usually based on simple preset rules or threshold triggers, lacking the ability to learn from historical decision-making experience and to comprehensively analyze the state of multidimensional systems. They are unable to make optimal decisions in complex and ever-changing edge environments, resulting in inappropriate model switching timing and unnecessary performance fluctuations. Summary of the Invention

[0007] This invention provides a dynamic model switching framework for AI inference optimization methods and systems for edge devices, which can solve the problems in the prior art.

[0008] A first aspect of this invention provides a dynamic model switching framework for AI inference optimization methods on edge devices, comprising:

[0009] The system receives multiple model versions with preset compression levels and applies a federated learning mechanism to collaboratively train a quantization compensation model across an edge device cluster. The quantization compensation model is used to dynamically correct the quantization error of the multiple model versions with the compression levels and generate a set of compensated optimized model versions.

[0010] A hierarchical heterogeneous resource management system is constructed based on the optimized model version set. The hierarchical heterogeneous resource management system dynamically allocates and virtualizes computing resources based on software-defined hardware technology. It realizes the collaborative scheduling of CPU, GPU and NPU according to the computing characteristics of each model version in the optimized model version set. Based on the collaborative scheduling, reconfigurable computing technology is used to generate and adjust the FPGA dedicated inference acceleration circuit in real time.

[0011] The compensation parameters for the quantization error are integrated into the dedicated inference acceleration circuit of the FPGA, and intelligent power balancing technology is adopted to achieve precise matching between computing resources and power consumption targets through dynamic voltage and frequency adjustment; the resource status parameters of the hierarchical heterogeneous resource management system are monitored and quantitatively evaluated in real time to generate multi-dimensional status data.

[0012] Based on the multidimensional state data, a dynamic decision engine is constructed using a graph neural network. The dynamic decision engine takes the compensation parameter of the quantization error as the input of the decision feature and integrates knowledge distillation and transfer learning techniques to continuously extract optimization strategies from historical decision experience. When the dynamic decision engine identifies the optimal switching time based on the multidimensional state data, it triggers the corresponding optimization model version switching operation and uses an adaptive annealing algorithm to dynamically adjust the decision threshold.

[0013] A federated learning mechanism is applied to collaboratively train a quantization compensation model across an edge device cluster. This quantization compensation model dynamically corrects the quantization errors of multiple model versions at various compression levels, generating a set of compensated optimized model versions, including:

[0014] A hierarchical federated learning network is constructed, which divides the edge devices in the edge device cluster into core nodes and ordinary nodes according to their computing capabilities. A federated learning objective function is generated based on the local datasets of the core nodes and the ordinary nodes. The federated learning objective function is composed of the sum of the products of the contribution weights of each edge device and the local loss function. The contribution weights are dynamically adjusted according to the computing capabilities and data quality of the edge devices to obtain global model parameters.

[0015] The global model parameters are input to the quantization compensation module, which includes a weight quantization unit and an activation value quantization unit. The global model parameters are quantized based on the weight quantization unit to obtain the weight quantization error. The global model parameters are quantized based on the activation value quantization unit to obtain the activation value quantization error. The products of the weight quantization error and the activation value quantization error are added together to obtain the quantization error compensation function.

[0016] The global model parameters are used as the teacher model, and the quantized global model parameters are used as the student model. A knowledge distillation module is constructed based on the output features of the teacher model and the output features of the student model. The knowledge distillation module adjusts the distribution of the output features through a temperature parameter and balances the distillation loss using a weight factor. The distillation loss and the quantization error compensation function are input to the optimizer to obtain compensation parameters. The student model is optimized based on the compensation parameters to generate a set of optimized model versions with different compression levels.

[0017] A knowledge distillation module is constructed based on the output features of the teacher model and the student model. This module adjusts the distribution of the output features using a temperature parameter and balances the distillation loss using weighting factors.

[0018] A knowledge distillation module is constructed, and the output features of the teacher model and the student model are input into the knowledge distillation module. The knowledge distillation module calculates the basic distillation loss of the teacher model and the student model based on KL divergence, and adjusts the distribution of the output features based on temperature parameters.

[0019] The knowledge distillation module integrates a processing unit that maps the output features to a three-dimensional phase space based on the Lorenz system. The three-dimensional phase space describes the evolution of feature states through control parameters, thereby generating a feature state mapping.

[0020] The feature state mapping is input to the processing unit, which calculates the synchronization error between the output features of the teacher model and the output features of the student model based on the generalized synchronization principle, and dynamically updates the temperature parameters according to the synchronization error and the initial temperature parameters to generate temperature control parameters.

[0021] The temperature control parameters are input to the processing unit, which calculates a bifurcation metric function within a preset temperature range and determines the optimal temperature parameters based on the bifurcation metric function. The processing unit reconstructs the knowledge representation using a delayed coordinate embedding method to generate reconstructed features.

[0022] The basic distillation loss, the synchronization error, and the reconstruction features are input into the optimization unit. The optimization unit uses weighting factors to weight and balance the three losses to generate a total loss function, and optimizes the student model based on the total loss function.

[0023] A hierarchical heterogeneous resource management system is constructed based on the optimized model version set. This system dynamically allocates and virtualizes computing resources based on software-defined hardware technology. The collaborative scheduling of CPU, GPU, and NPU is implemented for each model version in the optimized model version set, taking into account their computational characteristics.

[0024] A hierarchical heterogeneous resource management system is constructed based on the set of optimized model versions, and the computational features of each model version in the set of optimized model versions are mapped to a heterogeneous resource demand vector.

[0025] A software-defined virtualization model is constructed, which maps the heterogeneous resource demand vector to the virtual resource space based on the Lorenz chaotic mapping, describes the dynamic allocation process of virtual resources through the resource state evolution function, and generates a resource allocation strategy.

[0026] A collaborative scheduling model is constructed based on the resource allocation strategy. The collaborative scheduling model uses a multi-objective genetic algorithm to optimize the scheduling scheme of CPU computing resources, GPU computing resources and NPU computing resources. The multi-objective genetic algorithm takes task completion time and resource utilization as optimization objectives, and iteratively optimizes the scheduling scheme through crossover operators and mutation operators to generate a collaborative scheduling strategy.

[0027] The collaborative scheduling strategy is input into the hierarchical heterogeneous resource management system, and the dynamic allocation and collaborative scheduling of the CPU computing resources, the GPU computing resources and the NPU computing resources are realized based on the collaborative scheduling strategy.

[0028] A collaborative scheduling model is constructed based on the resource allocation strategy. This model employs a multi-objective genetic algorithm to optimize the scheduling schemes for CPU, GPU, and NPU computing resources, including:

[0029] A resource allocation strategy is constructed based on the CPU resource scheduling objective function, the GPU resource scheduling objective function, and the NPU resource scheduling objective function. The resource allocation strategy is input into the hierarchical search space. A task scheduling sequence matrix is ​​generated based on the task scheduling order space in the hierarchical search space. A task segmentation granularity vector is generated according to the task segmentation strategy space in the hierarchical search space.

[0030] The task scheduling sequence matrix and the task segmentation granularity vector are input into the scheduling strategy feature extractor. Scheduling features are generated based on the scheduling strategy feature extractor. The scheduling features are then transformed by a first nonlinear transformation and a second nonlinear transformation to generate performance prediction results.

[0031] Based on the performance prediction results, the population is iteratively optimized. An adaptive crossover population is generated by an adaptive crossover operator, the crossover population is mutated by a dynamic mutation operator, and an optimized scheduling strategy is selected from the mutated crossover population by an elite selection strategy.

[0032] The optimized scheduling strategy is input into a joint optimization objective function, which evaluates the optimized scheduling strategy based on a weighted combination of a resource utilization loss function, a task completion time loss function, and a scheduling efficiency loss function, and generates the optimal scheduling scheme.

[0033] It employs intelligent power balancing technology to achieve precise matching between computing resources and power consumption targets through dynamic voltage and frequency adjustment; it monitors and quantifies the resource status parameters of the hierarchical heterogeneous resource management system in real time, generating multi-dimensional status data including:

[0034] The voltage vector, frequency vector, temperature vector, and utilization vector of heterogeneous computing resources are collected. Based on the voltage vector and the frequency vector, the basic power consumption value is calculated. The basic power consumption value is input into the fractal dynamics model. The fractal dynamics model describes the power consumption distribution characteristics through the fractal dimension matrix and characterizes the power consumption fluctuation characteristics through the Hurst exponent time series, thereby generating power consumption state characteristics.

[0035] The temperature vector and the utilization vector are input into the fractal dynamics model. The fractal dynamics model analyzes the temperature distribution characteristics and utilization distribution characteristics based on the fractal dimension matrix, and uses the Hurst exponent time series to characterize the temperature fluctuation characteristics and utilization fluctuation characteristics, generating temperature state characteristics and utilization state characteristics. The power consumption state characteristics, the temperature state characteristics, and the utilization state characteristics are used to construct multidimensional state data.

[0036] When the dynamic decision engine identifies the optimal switching timing based on the multidimensional state data, it triggers the corresponding optimized model version switching operation and dynamically adjusts the decision threshold using an adaptive annealing algorithm, including:

[0037] The dynamic decision engine analyzes the changing trends of the multidimensional state data through chaotic synchronization mapping, identifies the fluctuation characteristics of the multidimensional state data, and generates a switching timing assessment result.

[0038] The switching timing evaluation result is input into the annealing search module, which calculates the decision threshold based on the adaptive annealing algorithm. The adaptive annealing algorithm updates the decision threshold using a dynamic cooling strategy to generate the optimal switching timing.

[0039] Based on the optimal switching timing, the optimization model version switching operation is triggered to switch the current optimization model version to the target optimization model version.

[0040] A second aspect of this invention provides a dynamic model switching framework for an AI inference optimization system for edge devices, comprising:

[0041] The first unit is used to receive multiple model versions with preset compression levels, apply a federated learning mechanism to collaboratively train a quantization compensation model among edge device clusters, and the quantization compensation model is used to dynamically correct the quantization error of multiple model versions with the compression levels and generate a set of compensated optimized model versions.

[0042] The second unit is used to construct a hierarchical heterogeneous resource management system based on the set of optimized model versions. The hierarchical heterogeneous resource management system dynamically allocates and virtualizes computing resources based on software-defined hardware technology, and implements collaborative scheduling of CPU, GPU and NPU for the computing characteristics of each model version in the set of optimized model versions. Based on the collaborative scheduling, reconfigurable computing technology is used to generate and adjust FPGA-dedicated inference acceleration circuits in real time.

[0043] The third unit is used to integrate the compensation parameters for the quantization error into the dedicated inference acceleration circuit of the FPGA, and to use intelligent power balancing technology to achieve precise matching between computing resources and power consumption targets through dynamic voltage and frequency adjustment; to monitor and quantify the resource status parameters of the hierarchical heterogeneous resource management system in real time, and to generate multi-dimensional status data.

[0044] The fourth unit is used to construct a dynamic decision engine based on the multidimensional state data using a graph neural network. The dynamic decision engine takes the compensation parameter of the quantization error as the input of the decision feature and integrates knowledge distillation and transfer learning techniques to continuously extract optimization strategies from historical decision experience. When the dynamic decision engine identifies the optimal switching time based on the multidimensional state data, it triggers the corresponding optimization model version switching operation and uses an adaptive annealing algorithm to dynamically adjust the decision threshold.

[0045] A third aspect of the present invention provides an electronic device, comprising:

[0046] processor;

[0047] Memory used to store processor-executable instructions;

[0048] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0049] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0050] The beneficial effects of this application are as follows:

[0051] By using a federated learning mechanism to collaboratively train the quantization compensation model, the quantization error of models with different compression levels is effectively corrected, improving the inference accuracy of the model in resource-constrained environments while maintaining low computational overhead, enabling edge devices to perform complex AI tasks under limited resource conditions.

[0052] A hierarchical heterogeneous resource management system was built based on software-defined hardware technology, which realizes the collaborative scheduling of CPU, GPU and NPU and FPGA reconfigurable computing. It fully leverages the advantages of heterogeneous computing resources, improves inference performance, reduces latency, and achieves precise matching between computing resources and power consumption targets through intelligent power balancing technology, thereby extending device battery life.

[0053] By constructing a dynamic decision engine using graph neural networks and combining knowledge distillation and transfer learning techniques, the system can intelligently identify the optimal model switching timing based on real-time multidimensional state data, optimize resource allocation strategies, improve the system's adaptability to environmental changes and the accuracy of decision-making, and realize the adaptive optimization and continuous evolution of the edge AI system. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the AI ​​inference optimization method for edge devices using the dynamic model switching framework of this invention.

[0055] Figure 2 A bar chart showing the performance comparison of the federated learning quantization compensation model in embodiments of the present invention;

[0056] Figure 3 This is a flowchart of a heterogeneous resource collaborative scheduling model optimized by a multi-objective genetic algorithm according to an embodiment of the present invention.

[0057] Figure 4 This is a schematic diagram illustrating the distribution characteristics of the utilization rate of the fractal dynamics model in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0060] Figure 1 This is a flowchart illustrating the AI ​​inference optimization method for edge devices using the dynamic model switching framework of this invention, as shown in the embodiment of the invention. Figure 1 As shown, the method includes:

[0061] The system receives multiple model versions with preset compression levels and applies a federated learning mechanism to collaboratively train a quantization compensation model across an edge device cluster. The quantization compensation model is used to dynamically correct the quantization error of the multiple model versions with the compression levels and generate a set of compensated optimized model versions.

[0062] A hierarchical heterogeneous resource management system is constructed based on the optimized model version set. The hierarchical heterogeneous resource management system dynamically allocates and virtualizes computing resources based on software-defined hardware technology. It realizes the collaborative scheduling of CPU, GPU and NPU according to the computing characteristics of each model version in the optimized model version set. Based on the collaborative scheduling, reconfigurable computing technology is used to generate and adjust the FPGA dedicated inference acceleration circuit in real time.

[0063] The compensation parameters for the quantization error are integrated into the dedicated inference acceleration circuit of the FPGA, and intelligent power balancing technology is adopted to achieve precise matching between computing resources and power consumption targets through dynamic voltage and frequency adjustment; the resource status parameters of the hierarchical heterogeneous resource management system are monitored and quantitatively evaluated in real time to generate multi-dimensional status data.

[0064] Based on the multidimensional state data, a dynamic decision engine is constructed using a graph neural network. The dynamic decision engine takes the compensation parameter of the quantization error as the input of the decision feature and integrates knowledge distillation and transfer learning techniques to continuously extract optimization strategies from historical decision experience. When the dynamic decision engine identifies the optimal switching time based on the multidimensional state data, it triggers the corresponding optimization model version switching operation and uses an adaptive annealing algorithm to dynamically adjust the decision threshold.

[0065] In one optional implementation, a federated learning mechanism is applied to collaboratively train a quantization compensation model across an edge device cluster. This quantization compensation model dynamically corrects the quantization errors of multiple model versions at various compression levels, generating a set of compensated optimized model versions including:

[0066] A hierarchical federated learning network is constructed, which divides the edge devices in the edge device cluster into core nodes and ordinary nodes according to their computing capabilities. A federated learning objective function is generated based on the local datasets of the core nodes and the ordinary nodes. The federated learning objective function is composed of the sum of the products of the contribution weights of each edge device and the local loss function. The contribution weights are dynamically adjusted according to the computing capabilities and data quality of the edge devices to obtain global model parameters.

[0067] The global model parameters are input to the quantization compensation module, which includes a weight quantization unit and an activation value quantization unit. The global model parameters are quantized based on the weight quantization unit to obtain the weight quantization error. The global model parameters are quantized based on the activation value quantization unit to obtain the activation value quantization error. The products of the weight quantization error and the activation value quantization error are added together to obtain the quantization error compensation function.

[0068] The global model parameters are used as the teacher model, and the quantized global model parameters are used as the student model. A knowledge distillation module is constructed based on the output features of the teacher model and the output features of the student model. The knowledge distillation module adjusts the distribution of the output features through a temperature parameter and balances the distillation loss using a weight factor. The distillation loss and the quantization error compensation function are input to the optimizer to obtain compensation parameters. The student model is optimized based on the compensation parameters to generate a set of optimized model versions with different compression levels.

[0069] A hierarchical federated learning network is constructed to divide edge devices in the edge device cluster into core nodes and ordinary nodes based on their computing capabilities. Computing capability is assessed based on the device's processor performance, memory size, and energy consumption characteristics. For example, devices with a processor clock speed exceeding 2.5GHz, memory capacity greater than 8GB, and power consumption controlled below 10W are classified as core nodes, while the remaining devices are ordinary nodes. In a real-world application scenario, a cluster containing 100 edge devices has 20 devices identified as core nodes, and the remaining 80 devices as ordinary nodes.

[0070] A federated learning objective function is generated based on local datasets from core nodes and ordinary nodes. This objective function is the sum of the products of the contribution weights of each edge device and the local loss function. The contribution weights are dynamically adjusted based on the computing power and data quality of the edge devices. The computing power score is calculated comprehensively based on indicators such as the device's floating-point performance and memory bandwidth; for example, the computing power score ranges from 0.8 to 1.0 for core nodes and from 0.3 to 0.7 for ordinary nodes. Data quality is evaluated through dimensions such as sample diversity and annotation accuracy; for example, a high-quality dataset scores 0.9, medium-quality datasets score 0.6, and low-quality datasets score 0.3. The final contribution weight is obtained by weighted averaging of the computing power score and the data quality score, with weight coefficients of 0.6 and 0.4, respectively.

[0071] The federated learning process involves multiple iterations, each consisting of two phases: local model training and global model aggregation. During local model training, each edge device updates its model parameters using its own dataset. In the global model aggregation phase, the central server collects all local model parameters and performs a weighted average based on the contribution weights of each device to generate the global model parameters. For example, after 10 iterations, federated learning converges and yields stable global model parameters.

[0072] The global model parameters are input to the quantization compensation module, which includes a weight quantization unit and an activation value quantization unit. The weight quantization unit uses linear quantization to convert 32-bit floating-point weights into 8-bit integer representations, with a quantization step size equal to the original weight range divided by 255. For example, when the weight range is [-2.5, 2.5], the quantization step size is 0.0196. The activation value quantization unit also uses linear quantization, mapping the activation value range to integer values ​​between 0 and 255. The weight quantization error is obtained by quantizing the global model parameters using the weight quantization unit; this error is calculated as the difference between the original weights and the dequantized weights. The activation value quantization error is obtained by quantizing the global model parameters using the activation value quantization unit; this error is calculated as the difference between the original activation values ​​and the dequantized activation values. The quantization error compensation function is obtained by multiplying the weight quantization error and the activation value quantization error.

[0073] The global model parameters are used as the teacher model, and the quantized global model parameters are used as the student model. A knowledge distillation module is constructed based on the output features of the teacher and student models. The knowledge distillation module adjusts the distribution of output features using a temperature parameter of 4.0, which makes the soft label distribution smoother and facilitates the student model's learning of the teacher model's knowledge. During knowledge distillation, a weighting factor is used to balance the distillation loss. The weighting factor is set to 0.7, meaning that the distillation loss accounts for 70% of the total loss, while the hard label cross-entropy loss accounts for 30%. The distillation loss and quantization error compensation function are input into the optimizer to obtain the compensation parameters. The optimizer uses the Adam algorithm with a learning rate of 0.001, and the momentum parameters are set to 0.9 and 0.999.

[0074] The student model is optimized based on compensation parameters, generating multiple sets of optimized model versions with different compression levels. Specifically, different compression levels are generated by adjusting the quantization bit width, including 8-bit, 6-bit, and 4-bit quantization versions, corresponding to low, medium, and high compression levels, respectively. These optimized models achieve model size compression ratios of 4x, 5.3x, and 8x, respectively, while maintaining the original model's accuracy reduction by no more than 2%.

[0075] In actual deployment, the system dynamically selects the appropriate compression level model version based on the resource constraints of edge devices. For example, for core nodes with sufficient computing resources, a low-compression model with 8-bit quantization can be deployed to maintain high inference accuracy; for ordinary nodes with limited resources, a high-compression model with 4-bit quantization can be deployed to optimize execution efficiency. Through this dynamic adaptation mechanism, the system can achieve a balance between computing resources and model performance within the edge device cluster, improving overall system efficiency.

[0076] Figure 2 A bar chart showing the performance comparison of the federated learning quantization compensation model in this invention:

[0077] This figure illustrates the comparative results of three different methods (the proposed solution based on federated learning quantization, traditional quantization methods, and non-federated learning quantization methods) across five key performance indicators. In terms of quantization accuracy improvement, the proposed solution significantly outperforms the traditional method (62.4%) and the non-federated learning method (75.2%). Regarding inference speed improvement, the proposed solution achieves 76.3%, also performing well compared to the traditional method's 58.9% and the non-federated learning method's 65.1%. In terms of model size reduction, the proposed solution achieves 62.8%, surpassing the traditional method's 45.6% and the non-federated learning method's 55.7%. In terms of energy consumption reduction, the proposed solution achieves 48.5%, significantly higher than the traditional method's 32.7% and the non-federated learning method's 39.4%. Finally, in terms of response time reduction, the proposed solution achieves 72.1%, again outperforming the traditional method's 51.3% and the non-federated learning method's 61.8%. Overall, the technical solution based on federated learning quantization computation achieved the best performance across all five key performance indicators, demonstrating significant technical advantages and practical value.

[0078] In one optional implementation, a knowledge distillation module is constructed based on the output features of the teacher model and the student model. The knowledge distillation module adjusts the distribution of the output features using a temperature parameter and balances the distillation loss using a weighting factor, including:

[0079] A knowledge distillation module is constructed, and the output features of the teacher model and the student model are input into the knowledge distillation module. The knowledge distillation module calculates the basic distillation loss of the teacher model and the student model based on KL divergence, and adjusts the distribution of the output features based on temperature parameters.

[0080] The knowledge distillation module integrates a processing unit that maps the output features to a three-dimensional phase space based on the Lorenz system. The three-dimensional phase space describes the evolution of feature states through control parameters, thereby generating a feature state mapping.

[0081] The feature state mapping is input to the processing unit, which calculates the synchronization error between the output features of the teacher model and the output features of the student model based on the generalized synchronization principle, and dynamically updates the temperature parameters according to the synchronization error and the initial temperature parameters to generate temperature control parameters.

[0082] The temperature control parameters are input to the processing unit, which calculates a bifurcation metric function within a preset temperature range and determines the optimal temperature parameters based on the bifurcation metric function. The processing unit reconstructs the knowledge representation using a delayed coordinate embedding method to generate reconstructed features.

[0083] The basic distillation loss, the synchronization error, and the reconstruction features are input into the optimization unit. The optimization unit uses weighting factors to weight and balance the three losses to generate a total loss function, and optimizes the student model based on the total loss function.

[0084] The knowledge distillation module receives the output features of the teacher model and the student model as input. For the input feature vector, the probability distribution P of the teacher model's output is first calculated. t And the student model output probability distribution P s The KL divergence between them is used as the basic distillation loss L. kd During the calculation process, a temperature parameter T is introduced to adjust the probability distribution, making the knowledge in the soft tags smoother. The initial temperature parameter can be set to 8.0, and adjusting this parameter can control the degree of softness or hardness in knowledge transfer.

[0085] The knowledge distillation module integrates a processing unit that maps output features to a three-dimensional phase space based on the Lorenz system. Specifically, for the teacher model's output feature X... t And student model output features X s The mapping is performed through the following steps: Key components are extracted from the feature vectors and used as initial conditions for the Lorenz system; the control parameters of the Lorenz system are set to σ=10.0, ρ=28.0, and β=8 / 3, and the trajectory of the feature in phase space is obtained through iterative calculation; the coordinates of points on the trajectory are sampled and recorded to form the feature state mapping M. t and M s In practical applications, 1000 iteration steps can be selected, and the state can be recorded every 10 steps to finally obtain a state mapping composed of 100 sampling points.

[0086] The processing unit calculates the synchronization error of the output features of the teacher model and the student model based on the generalized synchronization principle. For the obtained feature state mapping M... t and M s Calculate their Euclidean distance sequence in phase space, and take their mean as the synchronization error E. sync When the synchronization error is large, it indicates a significant difference in knowledge representation between the student model and the teacher model, requiring strengthened knowledge transfer; when the synchronization error is small, it indicates that the student model has learned the knowledge representation of the teacher model well. Based on the synchronization error and initial temperature parameters, the temperature parameters are dynamically updated: if the synchronization error E... sync If the synchronization error is greater than the threshold of 0.5, the temperature parameter is increased to soften the knowledge transfer; if the synchronization error is less than the threshold of 0.2, the temperature parameter is decreased to harden the knowledge transfer. The update formula can be designed as: T new =T old ×(1+α×(Esync -0.35), where α is the adjustment coefficient, which can be set to 0.5.

[0087] The processing unit calculates the bifurcation metric function within a preset temperature range to determine the optimal temperature parameters. The temperature range can be set to [1.0, 20.0], with sampling in steps of 1.0. For each temperature value T... i Calculate the knowledge distillation loss L at this temperature. kd (T i ) and synchronization error E sync (T i Construct the bifurcation metric function D(T) i )=L kd (T i )×(1+γ×E sync (T i ), where γ is the balance coefficient, which can be set to 0.3. Choose D(T) to make γ the balance coefficient. i The minimum temperature value is taken as the optimal temperature parameter T. opt In practical applications, if T opt =6.0 D(T) i If the minimum value of 0.087 is obtained, then the optimal temperature parameter is determined to be 6.0.

[0088] The processing unit reconstructs the knowledge representation using a delayed coordinate embedding method. This is applied to the feature state mapping M. t and M s We selected an embedding dimension d=5 and a delay time τ=2 to construct an embedding vector sequence. By calculating the similarity matrix of these embedding vectors, we extracted principal components and generated the reconstructed feature R. t and R s The reconstructed features can capture the temporal correlation and nonlinear dynamic characteristics of the original features. Reconstruction error E rec Defined as R t and R s The cosine distance between them. In specific application scenarios, the reconstructed features can reduce the original 768-dimensional features to 128-dimensional features while preserving key information, and the reconstruction error is controlled below 0.15.

[0089] The optimization unit uses weighting factors to weight and balance the three losses, generating a total loss function. The total loss function is defined as: L total =λ1×L kd +λ2×E sync +λ3×E recλ1, λ2, and λ3 are weighting factors that control the contributions of basal distillation loss, synchronization error, and reconstruction error, respectively. The weighting factors can be adjusted according to the specific task; a typical setting is λ1=0.5, λ2=0.3, and λ3=0.2. The student model optimizes its parameters based on the total loss function using backpropagation and gradient descent algorithms. During iterative optimization, the loss value gradually decreases from the initial 1.45 to 0.38, and the model accuracy improves from 82.3% to 95.7%.

[0090] In this embodiment, by dynamically adjusting temperature parameters and employing a multi-objective optimization strategy, the student model (ResNet-18) achieved 98.2% of the test accuracy of the teacher model (ResNet-50) in image classification tasks while maintaining a model size that is only 30% of the teacher model's. In natural language processing tasks, the small BERT model (4 layers) trained using this method achieved 96.5% of the performance of the original BERT model (12 layers) in text classification tasks, while simultaneously improving inference speed by 3.2 times.

[0091] In one optional implementation, a hierarchical heterogeneous resource management system is constructed based on the optimized model version set. This system dynamically allocates and virtualizes computing resources based on software-defined hardware technology. The collaborative scheduling of CPU, GPU, and NPU is implemented for the computational characteristics of each model version in the optimized model version set, including:

[0092] A hierarchical heterogeneous resource management system is constructed based on the set of optimized model versions, and the computational features of each model version in the set of optimized model versions are mapped to a heterogeneous resource demand vector.

[0093] A software-defined virtualization model is constructed, which maps the heterogeneous resource demand vector to the virtual resource space based on the Lorenz chaotic mapping, describes the dynamic allocation process of virtual resources through the resource state evolution function, and generates a resource allocation strategy.

[0094] A collaborative scheduling model is constructed based on the resource allocation strategy. The collaborative scheduling model uses a multi-objective genetic algorithm to optimize the scheduling scheme of CPU computing resources, GPU computing resources and NPU computing resources. The multi-objective genetic algorithm takes task completion time and resource utilization as optimization objectives, and iteratively optimizes the scheduling scheme through crossover operators and mutation operators to generate a collaborative scheduling strategy.

[0095] The collaborative scheduling strategy is input into the hierarchical heterogeneous resource management system, and the dynamic allocation and collaborative scheduling of the CPU computing resources, the GPU computing resources and the NPU computing resources are realized based on the collaborative scheduling strategy.

[0096] A hierarchical heterogeneous resource management system is constructed based on an optimized model version set, mapping the computational characteristics of each model version in the set to a heterogeneous resource demand vector. Specifically, the system collects the computational characteristics of the model versions, including parameters such as computational density, memory access pattern, parallelism, and data dependency. For image recognition model version A, its computational characteristics include a floating-point operation count of 3.5 × 10⁻⁶. 9 The memory access frequency is 2.1 × 10⁻⁶. 6 The computational characteristics of the natural language processing model version B include 1.8 × 10⁻⁶ floating-point operations per second, a parallelism of 85%, and low data dependency. 9 The memory access frequency is 4.3 × 10⁻⁶. 6 The speed is 60 times per second, the parallelism is 65%, and the data dependency is medium.

[0097] These computational features are transformed into a five-dimensional heterogeneous resource demand vector through feature extraction algorithms, corresponding to the number of CPU cores, the number of GPU stream processors, the number of NPU computing units, memory capacity, and bandwidth requirements, respectively. For example, the heterogeneous resource demand vector of model version A is [4,1024,128,8GB,16GB / s], and the heterogeneous resource demand vector of model version B is [8,512,64,12GB,10GB / s].

[0098] A software-defined virtualization model is constructed. This model maps heterogeneous resource demand vectors to a virtual resource space based on Lorenz chaotic mapping. The dynamic allocation process of virtual resources is described by a resource state evolution function, generating a resource allocation strategy. In the specific implementation, the parameters used in the Lorenz chaotic mapping are σ=10, ρ=28, β=8 / 3, with initial conditions of x0=1.0, y0=1.0, z0=1.0, and 1000 iterations. For the heterogeneous resource demand vector [4,1024,128,8GB,16GB / s] of model version A, the coordinates in the virtual resource space after Lorenz chaotic mapping are [0.73,0.45,0.89]. The system maintains a virtual resource state matrix of size 100×100×100, representing a three-dimensional virtual resource space. The resource state evolution function describes the change of each element in this matrix over time, considering factors such as resource allocation, release, and migration. Based on the virtual resource space coordinates of the model version and the current system resource status, the resource status evolution function generates a resource allocation strategy. For example, model version A is allocated 4 CPU cores, 1 GPU (containing 1024 stream processors), 0 NPUs, 8GB of memory and 16GB / s bandwidth.

[0099] A collaborative scheduling model is constructed based on resource allocation strategies. This model employs a multi-objective genetic algorithm to optimize the scheduling scheme of CPU, GPU, and NPU computing resources. The multi-objective genetic algorithm optimizes task completion time and resource utilization as its objectives, iteratively optimizing the scheduling scheme through crossover and mutation operators to generate a collaborative scheduling strategy. In the specific implementation, the collaborative scheduling model uses a binary encoding method with a chromosome length equal to the number of model versions × 3, representing the execution ratio of each model version on the CPU, GPU, and NPU. The population size is set to 100, the number of iterations is 500, the crossover probability is 0.8, and the mutation probability is 0.05. The fitness function consists of two parts: task completion time and resource utilization, with weights of 0.6 and 0.4, respectively. For model versions A and B, 100 scheduling schemes are randomly generated in the initial population. Scheme 1 is [0.2, 0.7, 0.1, 0.5, 0.3, 0.2], indicating that model version A has an execution ratio of 20%, 70%, and 10% on CPU, GPU, and NPU, respectively, while model version B has an execution ratio of 50%, 30%, and 20% on CPU, GPU, and NPU, respectively. After 500 iterations of optimization, the optimal scheduling scheme [0.15, 0.8, 0.05, 0.6, 0.25, 0.15] is obtained, with a calculated task completion time of 42 milliseconds and a resource utilization rate of 87%.

[0100] The collaborative scheduling strategy is input into the hierarchical heterogeneous resource management system, enabling dynamic allocation and collaborative scheduling of CPU, GPU, and NPU computing resources. The system adopts a hierarchical architecture, including a resource abstraction layer, a virtualization layer, and a scheduling layer. The resource abstraction layer manages physical computing resources, including an 8-core CPU, 2 GPUs (each containing 2048 stream processors), and 4 NPUs (each containing 256 computing units). The virtualization layer, based on software-defined hardware technology, virtualizes the physical resource pool into allocable resource units, such as CPU cores, GPU stream processor blocks, and NPU computing unit blocks. The scheduling layer dynamically allocates virtualized resources according to the collaborative scheduling strategy, enabling parallel execution of the model version on heterogeneous hardware. For model version A, the system allocates 1.2 CPU cores (15% of total execution), 1638 GPU stream processors (80% of total execution), and 13 NPU computing units (5% of total execution); for model version B, the system allocates 4.8 CPU cores (60% of total execution), 512 GPU stream processors (25% of total execution), and 38 NPU computing units (15% of total execution). The system also implements a dynamic resource adjustment mechanism, monitoring resource utilization every 100 milliseconds. When resource utilization is detected to be below 70% or above 95%, a resource reallocation process is triggered, updating the collaborative scheduling strategy. In this way, the system achieves efficient collaborative scheduling of CPU, GPU, and NPU resources, reducing task completion time and improving resource utilization.

[0101] In one optional implementation, a collaborative scheduling model is constructed based on the resource allocation strategy. This collaborative scheduling model employs a multi-objective genetic algorithm to optimize the scheduling scheme for CPU computing resources, GPU computing resources, and NPU computing resources, including:

[0102] A resource allocation strategy is constructed based on the CPU resource scheduling objective function, the GPU resource scheduling objective function, and the NPU resource scheduling objective function. The resource allocation strategy is input into the hierarchical search space. A task scheduling sequence matrix is ​​generated based on the task scheduling order space in the hierarchical search space. A task segmentation granularity vector is generated according to the task segmentation strategy space in the hierarchical search space.

[0103] The task scheduling sequence matrix and the task segmentation granularity vector are input into the scheduling strategy feature extractor. Scheduling features are generated based on the scheduling strategy feature extractor. The scheduling features are then transformed by a first nonlinear transformation and a second nonlinear transformation to generate performance prediction results.

[0104] Based on the performance prediction results, the population is iteratively optimized. An adaptive crossover population is generated by an adaptive crossover operator, the crossover population is mutated by a dynamic mutation operator, and an optimized scheduling strategy is selected from the mutated crossover population by an elite selection strategy.

[0105] The optimized scheduling strategy is input into a joint optimization objective function, which evaluates the optimized scheduling strategy based on a weighted combination of a resource utilization loss function, a task completion time loss function, and a scheduling efficiency loss function, and generates the optimal scheduling scheme.

[0106] like Figure 3 As shown, the method further includes:

[0107] In a specific implementation, when constructing a collaborative scheduling model based on a resource allocation strategy, the model uses a multi-objective genetic algorithm to optimize the scheduling scheme of CPU, GPU and NPU computing resources, thereby achieving efficient collaborative scheduling of heterogeneous computing resources.

[0108] The construction of the cooperative scheduling model begins with the scheduling objective functions of three types of computing resources. The CPU resource scheduling objective function considers the number of CPU cores, clock speed, and cache size, and calculates the CPU's efficiency index for processing tasks through weighted averages. For example, for an 8-core 3.2GHz CPU with 16MB cache, with weights set to 0.4, 0.3, and 0.3 respectively, the CPU resource efficiency index is calculated as (0.4×8 + 0.3×3.2 + 0.3×16) = 8.76. The GPU resource scheduling objective function comprehensively considers the number of GPU cores, memory size, and computing power, setting corresponding weights to calculate the GPU's performance index for processing tasks. For example, for a GPU with 2048 cores, 16GB of memory, and 14 TFLOPS of computing power, with weights set to 0.35, 0.3, and 0.35 respectively, the GPU performance index is (0.35×2048 + 0.3×16 + 0.35×14000) = 5721.6. The NPU resource scheduling objective function evaluates the efficiency of the NPU in processing specific tasks based on its AI computing power, energy efficiency ratio, and parallel processing capability. Taking an NPU with 200 TOPS computing power, 5 TOPS / W energy efficiency ratio, and 64 parallel processing units as an example, with weights set to 0.4, 0.3, and 0.3 respectively, the NPU efficiency index is calculated as (0.4×200+0.3×5+0.3×64)=99.1.

[0109] After the resource allocation strategy described above is input into the hierarchical search space, the system generates a task scheduling sequence matrix based on the task scheduling order space. For a scenario with 5 tasks and 3 types of computing resources, the task scheduling sequence matrix can be represented as a 5×3 matrix, where the element values ​​represent the scheduling priority of a task on a specific resource. For example, the first row [1,3,2] in the matrix indicates that task 1 has a priority of 1 on the CPU, a priority of 3 on the GPU, and a priority of 2 on the NPU. Simultaneously, the system generates a task partitioning granularity vector based on the task partitioning strategy space. For the case of 5 tasks, the task partitioning granularity vector can be represented as [0.3,0.5,0.2,0.4,0.6], representing the partitioning ratio of each task, which determines the degree to which tasks can be processed in parallel.

[0110] After the task scheduling sequence matrix and task segmentation granularity vector are input into the scheduling policy feature extractor, the system extracts key features of the scheduling policy through a multilayer perceptron structure. The feature extraction network consists of an input layer, two hidden layers, and an output layer, with 64 and 32 nodes in the hidden layers, respectively. The input features undergo a first nonlinear transformation to obtain an intermediate representation, and the output of each node is calculated using the ReLU activation function. For example, for the input feature vector [0.3, 0.8, 0.5], after transformation using the weight matrix [0.2, 0.3, 0.1; 0.1, 0.4, 0.2; 0.3, 0.1, 0.4] and application of the ReLU function, the intermediate representation [0.41, 0.38, 0.43] can be obtained. The intermediate representation then undergoes a second nonlinear transformation, and performance prediction results are generated using another weight matrix and the ReLU function, including three indicators: estimated completion time, resource utilization, and scheduling efficiency.

[0111] Based on performance prediction results, the system iteratively optimizes the initial population. The initial population contains 50 scheduling strategies, each consisting of a task scheduling sequence matrix and a task partitioning granularity vector. When generating the crossover population using the adaptive crossover operator, the system dynamically adjusts the crossover probability, calculating it based on the fitness values ​​of the parent individuals. For example, when the fitness values ​​of the parent individuals are 0.7 and 0.8 respectively, the crossover probability is set to 0.5 + (0.9 - 0.5) × (1 - (0.7 + 0.8) / 2) = 0.65. The crossover operation produces two offspring individuals, inheriting some features from their parents and undergoing recombination. For the dynamic mutation operator, the system dynamically adjusts the mutation probability based on the current iteration count, decreasing as the iteration count increases. For example, in the 10th iteration, with a maximum iteration count of 100, the mutation probability can be set to 0.1 + (0.5 - 0.1) × (1 - 10 / 100) = 0.46. The mutation operation randomly changes certain gene values ​​in individuals, increasing population diversity. Through an elite selection strategy, the system retains the top 10% of individuals with the highest fitness to directly enter the next generation, while the remaining positions are selected from the mutated crossover population through a tournament selection process.

[0112] The optimized scheduling strategy is input into the joint optimization objective function for final evaluation. The resource utilization loss function calculates the difference between the average utilization of the three computing resources and the ideal utilization. For example, when the utilization rates of CPU, GPU, and NPU are 75%, 85%, and 60% respectively, and the target utilization rate is 90%, the resource utilization loss is calculated as 1 - ((75+85+60) / 3) / 90 = 0.18. The task completion time loss function evaluates the ratio of the actual completion time to the theoretical minimum completion time. For a projected completion time of 150 seconds and a theoretical minimum completion time of 120 seconds, the time loss is (150-120) / 120 = 0.25. The scheduling efficiency loss function measures the number of resource switches and load balancing. When the number of resource switches is 15 (normalized to 0.3) and the load imbalance is 0.2, the scheduling efficiency loss is 0.3 × 0.6 + 0.2 × 0.4 = 0.26. The joint optimization objective function combines the three loss functions with weights of 0.4, 0.4, and 0.2, resulting in a final evaluation score of 0.18×0.4 + 0.25×0.4 + 0.26×0.2 = 0.224. The scheduling strategy with the lowest evaluation score is selected as the optimal scheduling scheme, achieving efficient collaborative scheduling of heterogeneous computing resources.

[0113] In one optional implementation, intelligent power balancing technology is employed to achieve precise matching between computing resources and power consumption targets through dynamic voltage and frequency adjustment; the resource status parameters of the hierarchical heterogeneous resource management system are monitored and quantitatively evaluated in real time, generating multi-dimensional status data including:

[0114] The voltage vector, frequency vector, temperature vector, and utilization vector of heterogeneous computing resources are collected. Based on the voltage vector and the frequency vector, the basic power consumption value is calculated. The basic power consumption value is input into the fractal dynamics model. The fractal dynamics model describes the power consumption distribution characteristics through the fractal dimension matrix and characterizes the power consumption fluctuation characteristics through the Hurst exponent time series, thereby generating power consumption state characteristics.

[0115] The temperature vector and the utilization vector are input into the fractal dynamics model. The fractal dynamics model analyzes the temperature distribution characteristics and utilization distribution characteristics based on the fractal dimension matrix, and uses the Hurst exponent time series to characterize the temperature fluctuation characteristics and utilization fluctuation characteristics, generating temperature state characteristics and utilization state characteristics. The power consumption state characteristics, the temperature state characteristics, and the utilization state characteristics are used to construct multidimensional state data.

[0116] Voltage vectors, frequency vectors, temperature vectors, and utilization vectors of heterogeneous computing resources are collected through an embedded sensor network. The voltage vector is represented as V=(v1,v2...v...). n), where v1 represents the real-time voltage value of the first calculation unit, in volts; the frequency vector is represented as F=(f1,f2,...,f n ), where f1 represents the real-time operating frequency of the first computing unit, in Hertz; the temperature vector is represented as T=(t1,t2...t n ), where t1 represents the real-time temperature value of the first calculation unit, in degrees Celsius; the utilization vector is represented as U=(u1,u2...u n ), where u1 represents the real-time computing resource utilization rate of the first computing unit, expressed as a percentage.

[0117] Based on the acquired voltage and frequency vectors, the system calculates the base power consumption value. In practical implementation, for each computing unit i, its base power consumption P... i The voltage is squared, multiplied by the frequency, and then multiplied by a constant coefficient C related to the calculation unit. i The calculations yield the following results. For example, for a system containing four computing units, if the voltage vector is (0.9V, 1.1V, 1.0V, 1.2V), the frequency vector is (2.4GHz, 2.8GHz, 2.5GHz, 3.0GHz), and the corresponding constant coefficients are (0.5, 0.6, 0.55, 0.65), then the calculated base power consumption is (0.972W, 2.033W, 1.375W, 2.808W).

[0118] The calculated base power consumption value is input into the fractal dynamics model. This model describes the power consumption distribution characteristics using a fractal dimension matrix and characterizes power consumption fluctuations using the Hurst exponent time series. The fractal dimension matrix is ​​represented as an n×n matrix D, where each element D... ij The fractal dimension value represents the power consumption distribution between the i-th and j-th computing units, typically ranging from 1.0 to 2.0. The Hurst exponential timing representation is H = (h1, h2...h...). n ), where h1 represents the Hurst exponent of the power consumption change of the first computing unit, typically ranging from 0 to 1. An h1 value close to 1 indicates that the power consumption change is highly persistent, an h1 value close to 0.5 indicates that the power consumption change is close to a random walk, and an h1 value close to 0 indicates that the power consumption change is highly de-persistent.

[0119] The system generates power consumption state characteristics through calculations using a fractal dynamics model. In a practical example, the fractal dimension matrix of a 4-core processor is [[1.65,1.42,1.38,1.27],[1.42,1.72,1.45,1.32],[1.38,1.45,1.68,1.40],[1.27,1.32,1.40,1.70]], and its Hurst exponent timing is [0.82,0.78,0.75,0.80], indicating that the power consumption changes of each core have strong persistence and a certain correlation between them.

[0120] The system also inputs the temperature vector and utilization vector into the fractal dynamics model. The model analyzes the temperature distribution characteristics and utilization distribution characteristics based on the fractal dimension matrix, and characterizes the temperature fluctuation characteristics and utilization fluctuation characteristics through the Hurst exponent time series. For the temperature vector (45℃, 52℃, 48℃, 56℃), the fractal dimension matrix is ​​[[1.55, 1.38, 1.32, 1.25], [1.38, 1.62, 1.40, 1.30], [1.32, 1.40, 1.58, 1.35], [1.25, 1.30, 1.35, 1.60]], and the Hurst exponent time series is [0.76, 0.72, 0.70, 0.74]. For the utilization vector (65%, 78%, 70%, 85%), the fractal dimension matrix is ​​[[1.70, 1.45, 1.42, 1.30], [1.45, 1.75, 1.48, 1.35], [1.42, 1.48, 1.72, 1.38], [1.30, 1.35, 1.38, 1.68]], and the Hurst exponent time series is [0.85, 0.82, 0.80, 0.88].

[0121] Based on the analysis results of the fractal dynamics model, the system generates temperature state characteristics and utilization state characteristics. The temperature state characteristics include the fractal dimension matrix of the temperature distribution and the Hurst exponential time series of temperature fluctuations; the utilization state characteristics include the fractal dimension matrix of the utilization distribution and the Hurst exponential time series of utilization fluctuations.

[0122] The system constructs multidimensional state data based on power consumption, temperature, and utilization characteristics. This multidimensional state data is represented in tensor form and contains three main dimensions: power consumption, temperature, and utilization. Each dimension further includes two sub-dimensions: fractal dimension features and Hurst exponent features. For example, in the case of the aforementioned quad-core processor, the multidimensional state data can be represented as a 3×2×4 tensor, where 3 represents the number of main dimensions (power consumption, temperature, utilization), 2 represents the number of sub-dimensions for each main dimension (fractal dimension, Hurst exponent), and 4 represents the number of computing units.

[0123] Based on the constructed multidimensional state data, the system achieves intelligent power consumption balancing. The system analyzes the relationships between the power consumption, temperature, and utilization characteristics of each computing unit within the multidimensional state data, identifying power hotspots and utilization imbalances. For the identified problem areas, the system uses dynamic voltage and frequency adjustment technology for precise control. For example, when the power consumption of the fourth computing unit is detected to be too high (2.808W) and the temperature too high (56℃), the system reduces its voltage from 1.2V to 1.1V and its frequency from 3.0GHz to 2.8GHz, thereby reducing power consumption to 2.352W while maintaining computing performance within an acceptable range.

[0124] Through the above method, this invention achieves intelligent power consumption balancing of heterogeneous computing resources, ensuring that the system meets computing demands while achieving optimal power allocation, extending equipment lifespan, and improving energy efficiency. This technology is applicable to various high-performance computing systems, mobile computing devices, and data centers.

[0125] Figure 4 This is a schematic diagram illustrating the distribution characteristics of the utilization rate of the fractal dynamics model in an embodiment of the present invention:

[0126] This graph compares the CPU utilization over time between a smart power balancing scheme based on chaotic fractal dynamics (represented by circles in the legend) and a traditional dynamic voltage and frequency adjustment scheme (represented by diamonds in the legend). The horizontal axis represents time in minutes, from 0 to 60 minutes; the vertical axis represents the percentage of CPU utilization, ranging from 0% to 100%. The curves show that both schemes initially increase and then decrease, but their specific performance differs significantly. The traditional dynamic voltage and frequency adjustment scheme reaches a peak of approximately 94% at 20 minutes, while the smart power balancing scheme based on chaotic fractal dynamics reaches a peak of approximately 85% at 25 minutes, a decrease of about 9 percentage points. In the initial stage (0-15 minutes), the CPU utilization of this scheme increases more rapidly, quickly climbing from 25% to 82%; in contrast, the dynamic voltage and frequency adjustment scheme starts at a lower value (approximately 15%) and increases more gradually, reaching approximately 55% at 15 minutes. During the decline phase following the peak (30-60 minutes), the dynamic voltage-frequency regulation scheme exhibits a faster decline rate, dropping to approximately 18% by 60 minutes, while this scheme maintains a higher level of approximately 32%.

[0127] In one optional implementation, when the dynamic decision engine identifies the optimal switching timing based on the multidimensional state data, it triggers a corresponding optimized model version switching operation and dynamically adjusts the decision threshold using an adaptive annealing algorithm, including:

[0128] The dynamic decision engine analyzes the changing trends of the multidimensional state data through chaotic synchronization mapping, identifies the fluctuation characteristics of the multidimensional state data, and generates a switching timing assessment result.

[0129] The switching timing evaluation result is input into the annealing search module, which calculates the decision threshold based on the adaptive annealing algorithm. The adaptive annealing algorithm updates the decision threshold using a dynamic cooling strategy to generate the optimal switching timing.

[0130] Based on the optimal switching timing, the optimization model version switching operation is triggered to switch the current optimization model version to the target optimization model version.

[0131] The system receives multidimensional state data, which may include key indicators such as system operating parameters, performance metrics, and resource utilization. Specifically, the multidimensional state data can be a time-series data set D={d1(t), d2(t)...d...} containing n dimensions. n d1(t)}, where d1(t) can represent CPU utilization, d2(t) can represent memory usage, d3(t) can represent request response time, and d4(t) can represent the value of key indicators such as throughput at time t.

[0132] The dynamic decision engine analyzes the changing trends of these multidimensional state data using chaotic synchronization mapping. Chaotic synchronization mapping is a nonlinear dynamic analysis method that effectively captures and amplifies minute changes in the system state, thereby identifying inflection points. In practical implementation, for each dimension of state data d... i (t), chaotic synchronization mapping is performed as follows: First, the original data is standardized to the interval [0,1] to obtain standardized data s. i (t). Subsequently, a chaotic mapping function is applied to the standardized data to generate a mapping sequence m. i (t). The mapping function can be a Logistic mapping, where the mapping result for each time point t depends on the value of the previous time point and the control parameter r, which is typically set between 3.7 and 4 to ensure the system exhibits chaotic characteristics. By continuously applying this mapping function, a mapping sequence reflecting the changing characteristics of the original data can be obtained.

[0133] By calculating the difference Δm between mapping values ​​at adjacent time points i (t)=|m i (t)-m i (t-1)|, construct the differential sequence. A significant peak in the differential sequence indicates that the original data is undergoing a significant change. For example, when Δm iA point (t) exceeding three times the historical average can be marked as a potential turning point. By analyzing the distribution of turning points across multiple dimensions, the overall fluctuation characteristics of the system state can be comprehensively assessed.

[0134] To quantify this fluctuation characteristic, the dynamic decision engine calculates a fluctuation intensity index F, which comprehensively considers the magnitude and frequency of changes in state data across various dimensions, as well as cross-dimensional collaborative change patterns. For example, when CPU utilization suddenly rises from 45% to 90%, memory usage increases from 50% to 85%, and request response time increases from 200ms to 800ms, the fluctuation intensity index F will increase significantly, indicating that the system is at a critical moment requiring a switch to an optimization model version. In this way, the dynamic decision engine generates a switch timing assessment result E=(F, P), where F represents the fluctuation intensity and P represents the fluctuation pattern feature vector.

[0135] After the switching timing evaluation result E is input into the annealing search module, the module calculates the decision threshold based on the adaptive annealing algorithm. The adaptive annealing algorithm is a heuristic optimization method that simulates the energy change during metal annealing and finds the global optimum through a probabilistic search in the solution space. In this embodiment, the annealing search module initializes the decision threshold T0, typically set to 0.8. The initial temperature Temp0 is set to 100, indicating a higher probability that the system initially accepts a suboptimal solution.

[0136] The annealing search module uses a dynamic cooling strategy to update the decision threshold. Specifically, in each iteration i, the temperature is adjusted according to Temp. i =α×Temp i-1 The update process involves adjusting the cooling rate, where α is the cooling coefficient, typically set between 0.9 and 0.99. For example, when α = 0.95, the temperature will decrease at a relatively slow rate, giving the algorithm sufficient opportunity to explore the solution space. Simultaneously, the decision threshold update considers the difference between the current system state and historical states. When the system state fluctuates significantly, the cooling rate adaptively slows down to avoid premature convergence to a local optimum; when the system state is relatively stable, the cooling rate accelerates to improve convergence efficiency.

[0137] In practical applications, assuming the current system fluctuation intensity F = 0.75, which is lower than the initial decision threshold T0 = 0.8, no switching operation is triggered at this time. As the system load continues to increase, the fluctuation intensity rises to F = 0.85, exceeding the decision threshold, and the system calculates the switching benefit value G. If G is positive and sufficiently large, and the current temperature Temp allows for this switching operation, then this moment is marked as a potentially optimal switching opportunity. As the annealing process continues, the algorithm discovers better switching opportunities, and finally determines the final optimal switching opportunity when the temperature Temp drops to a preset termination temperature (e.g., 1.0).

[0138] Based on the determined optimal switching timing, the system triggers an optimization model version switching operation. The switching process includes the following steps: First, the system loads the target optimization model version into memory, ensuring that the loading process does not affect the current system operation; second, the system prepares the switching environment, including data state saving, cache preheating, and other operations; subsequently, during appropriate request processing intervals, the system performs the actual switching, redirecting request traffic from the current optimization model version to the target optimization model version; finally, the system performs post-switching verification to ensure that the target optimization model version is running normally.

[0139] For example, in an online recommendation system, as user traffic gradually increases from 1000 requests per second to 5000 requests per second, the system's CPU utilization rises from 40% to 85%, and memory usage increases from 50% to 90%. Through chaotic synchronization mapping analysis, the system identifies a fluctuation intensity of F=0.88, exceeding the current decision threshold T=0.82. The annealing search module calculates a switching benefit G=0.35 (a positive value indicates a beneficial switch), and the current temperature Temp=45 allows for this switch. The system then switches from the current lightweight recommendation model to a more complex but resource-efficient distributed recommendation model. After the switch, CPU utilization drops to 65%, memory usage drops to 75%, while maintaining the same recommendation quality, successfully handling the increased user load.

[0140] The method further includes:

[0141] The model pre-compression module performs multi-level compression processing on the original AI model on a cloud server. Taking the ResNet50 image classification model as an example, it can generate three model versions with different compression levels: the high-compression version uses 4-bit quantization technology, with a model size of 12MB, an accuracy of 83.2%, and an inference speed of 15ms / frame; the medium-compression version uses 8-bit quantization technology, with a model size of 45MB, an accuracy of 89.5%, and an inference speed of 28ms / frame; and the low-compression version retains 32-bit floating-point precision, with a model size of 198MB, an accuracy of 93.7%, and an inference speed of 65ms / frame. Model pre-compression is implemented using quantization tools in PyTorch, and specific quantization parameters include: weight quantization bit width setting (4-bit / 8-bit / 32-bit), activation value quantization range, and quantization method selection (symmetric / asymmetric quantization).

[0142] The resource monitoring module collects edge device status data through lightweight system calls. On Android devices, it obtains the battery percentage via the BatteryManager API, available memory via ActivityManager.MemoryInfo, and CPU usage via the / proc / stat file; on Linux devices, it obtains the CPU usage via / sys / class / power.supply It reads battery information, obtains memory information via ` / proc / meminfo`, and obtains CPU load via ` / proc / loadavg`. The monitoring frequency is configurable, typically set to collect data every 500 milliseconds. Data collection example: During a continuous video analysis task, a smartphone monitored a decrease in battery level from 85% to 19%, CPU usage fluctuating between 65% and 78%, and available memory decreasing from 1.2GB to 450MB.

[0143] Data processing employs a moving average method with a window size of 5 to smooth short-term fluctuations and avoid unnecessary model switching triggered by instantaneous peaks. Threshold settings are based on actual device performance test results: battery thresholds are set to low (<20%), medium (20%-80%), and high (>80%); CPU utilization thresholds are set to low (<30%), medium (30%-70%), and high (>70%); memory availability thresholds are set to low (<20% available), medium (20%-50% available), and high (>50% available).

[0144] The dynamic switching module selects the optimal model version based on the device's state. The switching decision uses a rule-based system implemented as a decision tree structure. For example, when the battery level is detected to be below 20% and CPU usage above 70% for 5 seconds, a switch to a high-compression model is triggered; when the battery level is above 80% and CPU usage below 30% for 10 seconds, a switch to a low-compression model is triggered. The switching frequency control mechanism includes a 5-second cooldown period and a 5% hysteresis buffer to prevent frequent switching due to state fluctuations. In actual testing, during the process of a smartphone battery dropping from 100% to 15%, the number of model switching events was controlled to within 3 times, effectively avoiding performance fluctuations.

[0145] Model priority management employs a dynamic scoring mechanism that comprehensively considers current resource status and model characteristics. Under resource-constrained conditions (e.g., battery <20%), high-compression models achieve a 50% score increase; under resource-sufficient conditions (e.g., battery >80%), low-compression models achieve a 30% score increase. At each switching decision, the score of each model is calculated, and the model version with the highest score is selected.

[0146] A seamless transition module ensures uninterrupted inference service during model switching. It implements the Model Control Protocol based on NVIDIA Triton Inference Server, supporting dynamic model loading and unloading. The switching process includes: requesting the loading of the new model, verifying the model's readiness, replacing the current model, and releasing the old model's resources. To avoid data loss, a 64-item FIFO buffer queue is set up to temporarily store data to be processed. For example, in video analytics applications, video frames during switching are stored in the buffer queue and processed sequentially after the new model becomes ready. Actual test data shows zero frame loss during the switching process.

[0147] Switching time is controlled within 80 milliseconds, and the impact on 30fps real-time video analysis is no more than 3 frames. The model loading failure rollback mechanism is implemented through status checks: if the new model does not reach the "ready" state within 10 seconds, it automatically rolls back to the old model to ensure service continuity. In actual testing, the inference service interruption time does not exceed 100 milliseconds in the event of model loading failure.

[0148] The policy management module supports user-defined switching policies, using JSON format configuration files. A typical policy configuration example: {"priority":"performance","battery"} threshold low :15, "cpu threshold high :75, "switch cooldown This configuration specifies a performance-priority strategy, with a low battery threshold of 15%, a high CPU threshold of 75%, and a switching cooldown period of 8 seconds. Strategy conflict resolution employs a priority mechanism: resource protection strategies (such as low battery strategies) have the highest priority, followed by performance optimization strategies, and accuracy optimization strategies have the lowest priority.

[0149] In specific application scenarios, taking edge video analytics as an example, actual test data shows that after adopting the dynamic model switching framework, the battery life of smartphones increased from 4.2 hours to 7.5 hours, an improvement of 78.6%; the average power consumption decreased from 3.2W to 1.8W, a reduction of 43.8%; the inference accuracy remained between 83.2% and 93.7% in different scenarios, meeting application requirements; the impact of the switching process on user experience was negligible, and the user satisfaction score increased from 7.2 to 8.9 (out of 10).

[0150] In IoT camera applications, the adoption of a dynamic model switching framework extends device battery life from 18 hours to 30 hours, a 66.7% improvement. Basic functions are maintained even during low-power periods, avoiding service interruptions caused by insufficient power in traditional solutions. Accuracy is dynamically adjusted based on the scenario, automatically switching to a high-accuracy model at critical moments (such as when abnormal activity is detected), enhancing security monitoring effectiveness.

[0151] The framework is applicable to a wide range of devices, including smartphones, tablets, smart cameras, edge servers, and IoT sensors. Testing and verification cover operating systems such as Android, iOS, and Linux, as well as processor architectures such as ARM, x86, and RISC-V, achieving compatibility of over 95%. Even on resource-constrained devices (e.g., RAM < 512MB), by configuring a lightweight monitoring strategy (reducing the monitoring frequency to 2 seconds / time) and decreasing the buffer queue size (to 16 items), the framework can still run effectively, consuming only about 5MB of memory and no more than 3% of CPU resources.

[0152] A second aspect of this invention provides a dynamic model switching framework for an AI inference optimization system for edge devices, comprising:

[0153] The first unit is used to receive multiple model versions with preset compression levels, apply a federated learning mechanism to collaboratively train a quantization compensation model among edge device clusters, and the quantization compensation model is used to dynamically correct the quantization error of multiple model versions with the compression levels and generate a set of compensated optimized model versions.

[0154] The second unit is used to construct a hierarchical heterogeneous resource management system based on the set of optimized model versions. The hierarchical heterogeneous resource management system dynamically allocates and virtualizes computing resources based on software-defined hardware technology, and implements collaborative scheduling of CPU, GPU and NPU for the computing characteristics of each model version in the set of optimized model versions. Based on the collaborative scheduling, reconfigurable computing technology is used to generate and adjust FPGA-dedicated inference acceleration circuits in real time.

[0155] The third unit is used to integrate the compensation parameters for the quantization error into the dedicated inference acceleration circuit of the FPGA, and to use intelligent power balancing technology to achieve precise matching between computing resources and power consumption targets through dynamic voltage and frequency adjustment; to monitor and quantify the resource status parameters of the hierarchical heterogeneous resource management system in real time, and to generate multi-dimensional status data.

[0156] The fourth unit is used to construct a dynamic decision engine based on the multidimensional state data using a graph neural network. The dynamic decision engine takes the compensation parameter of the quantization error as the input of the decision feature and integrates knowledge distillation and transfer learning techniques to continuously extract optimization strategies from historical decision experience. When the dynamic decision engine identifies the optimal switching time based on the multidimensional state data, it triggers the corresponding optimization model version switching operation and uses an adaptive annealing algorithm to dynamically adjust the decision threshold.

[0157] A third aspect of the present invention provides an electronic device, comprising:

[0158] processor;

[0159] Memory used to store processor-executable instructions;

[0160] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0161] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0162] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic model switching framework for AI inference optimization methods in edge devices, characterized in that, include: The system receives multiple model versions with preset compression levels and applies a federated learning mechanism to collaboratively train a quantization compensation model across an edge device cluster. The quantization compensation model is used to dynamically correct the quantization error of the multiple model versions with the compression levels and generate a set of compensated optimized model versions. A hierarchical heterogeneous resource management system is constructed based on the optimized model version set. The hierarchical heterogeneous resource management system dynamically allocates and virtualizes computing resources based on software-defined hardware technology. It realizes the collaborative scheduling of CPU, GPU and NPU according to the computing characteristics of each model version in the optimized model version set. Based on the collaborative scheduling, reconfigurable computing technology is used to generate and adjust the FPGA dedicated inference acceleration circuit in real time. The compensation parameters for the quantization error are integrated into the dedicated inference acceleration circuit of the FPGA, and intelligent power balancing technology is adopted to achieve precise matching between computing resources and power consumption targets through dynamic voltage and frequency adjustment; the resource status parameters of the hierarchical heterogeneous resource management system are monitored and quantitatively evaluated in real time to generate multi-dimensional status data. A dynamic decision engine is constructed based on the multidimensional state data. The dynamic decision engine performs trend analysis on the multidimensional state data through chaotic synchronization mapping, identifies the fluctuation characteristics of the multidimensional state data, and generates a switching timing assessment result. The switching timing evaluation result is input into the annealing search module, which calculates the decision threshold based on the adaptive annealing algorithm. The adaptive annealing algorithm updates the decision threshold using a dynamic cooling strategy to generate the optimal switching timing. Based on the optimal switching timing, the optimization model version switching operation is triggered to switch the current optimization model version to the target optimization model version.

2. The method according to claim 1, characterized in that, A federated learning mechanism is applied to collaboratively train a quantization compensation model across an edge device cluster. This quantization compensation model dynamically corrects the quantization errors of multiple model versions at various compression levels, generating a set of compensated optimized model versions, including: A hierarchical federated learning network is constructed, which divides the edge devices in the edge device cluster into core nodes and ordinary nodes according to their computing capabilities. A federated learning objective function is generated based on the local datasets of the core nodes and the ordinary nodes. The federated learning objective function is composed of the sum of the products of the contribution weights of each edge device and the local loss function. The contribution weights are dynamically adjusted according to the computing capabilities and data quality of the edge devices to obtain global model parameters. The global model parameters are input to the quantization compensation module, which includes a weight quantization unit and an activation value quantization unit. The global model parameters are quantized based on the weight quantization unit to obtain the weight quantization error. The global model parameters are quantized based on the activation value quantization unit to obtain the activation value quantization error. The products of the weight quantization error and the activation value quantization error are added together to obtain the quantization error compensation function. The global model parameters are used as the teacher model, and the quantized global model parameters are used as the student model. A knowledge distillation module is constructed based on the output features of the teacher model and the output features of the student model. The knowledge distillation module adjusts the distribution of the output features through a temperature parameter and balances the distillation loss using a weight factor. The distillation loss and the quantization error compensation function are input to the optimizer to obtain compensation parameters. The student model is optimized based on the compensation parameters to generate a set of optimized model versions with different compression levels.

3. The method according to claim 2, characterized in that, A knowledge distillation module is constructed based on the output features of the teacher model and the student model. This module adjusts the distribution of the output features using a temperature parameter and balances the distillation loss using weighting factors. A knowledge distillation module is constructed, and the output features of the teacher model and the student model are input into the knowledge distillation module. The knowledge distillation module calculates the basic distillation loss of the teacher model and the student model based on KL divergence, and adjusts the distribution of the output features based on temperature parameters. The knowledge distillation module integrates a processing unit that maps the output features to a three-dimensional phase space based on the Lorenz system. The three-dimensional phase space describes the evolution of feature states through control parameters, thereby generating a feature state mapping. The feature state mapping is input to the processing unit, which calculates the synchronization error between the output features of the teacher model and the output features of the student model based on the generalized synchronization principle, and dynamically updates the temperature parameters according to the synchronization error and the initial temperature parameters to generate temperature control parameters. The temperature control parameters are input to the processing unit, which calculates a bifurcation metric function within a preset temperature range and determines the optimal temperature parameters based on the bifurcation metric function. The processing unit reconstructs the knowledge representation using a delayed coordinate embedding method to generate reconstructed features. The basic distillation loss, the synchronization error, and the reconstruction features are input into the optimization unit. The optimization unit uses weighting factors to weight and balance the three losses to generate a total loss function, and optimizes the student model based on the total loss function.

4. The method according to claim 1, characterized in that, A hierarchical heterogeneous resource management system is constructed based on the optimized model version set. This system dynamically allocates and virtualizes computing resources based on software-defined hardware technology. The collaborative scheduling of CPU, GPU, and NPU is implemented for each model version in the optimized model version set, taking into account their computational characteristics. A hierarchical heterogeneous resource management system is constructed based on the set of optimized model versions, and the computational features of each model version in the set of optimized model versions are mapped to a heterogeneous resource demand vector. A software-defined virtualization model is constructed, which maps the heterogeneous resource demand vector to the virtual resource space based on the Lorenz chaotic mapping, describes the dynamic allocation process of virtual resources through the resource state evolution function, and generates a resource allocation strategy. A collaborative scheduling model is constructed based on the resource allocation strategy. The collaborative scheduling model uses a multi-objective genetic algorithm to optimize the scheduling scheme of CPU computing resources, GPU computing resources and NPU computing resources. The multi-objective genetic algorithm takes task completion time and resource utilization as optimization objectives, and iteratively optimizes the scheduling scheme through crossover operators and mutation operators to generate a collaborative scheduling strategy. The collaborative scheduling strategy is input into the hierarchical heterogeneous resource management system, and the dynamic allocation and collaborative scheduling of the CPU computing resources, the GPU computing resources and the NPU computing resources are realized based on the collaborative scheduling strategy.

5. The method according to claim 4, characterized in that, A collaborative scheduling model is constructed based on the resource allocation strategy. This model employs a multi-objective genetic algorithm to optimize the scheduling schemes for CPU, GPU, and NPU computing resources, including: A resource allocation strategy is constructed based on the CPU resource scheduling objective function, the GPU resource scheduling objective function, and the NPU resource scheduling objective function. The resource allocation strategy is input into the hierarchical search space. A task scheduling sequence matrix is ​​generated based on the task scheduling order space in the hierarchical search space. A task segmentation granularity vector is generated according to the task segmentation strategy space in the hierarchical search space. The task scheduling sequence matrix and the task segmentation granularity vector are input into the scheduling strategy feature extractor. Scheduling features are generated based on the scheduling strategy feature extractor. The scheduling features are then transformed by a first nonlinear transformation and a second nonlinear transformation to generate performance prediction results. Based on the performance prediction results, the population is iteratively optimized. An adaptive crossover population is generated by an adaptive crossover operator, the crossover population is mutated by a dynamic mutation operator, and an optimized scheduling strategy is selected from the mutated crossover population by an elite selection strategy. The optimized scheduling strategy is input into a joint optimization objective function, which evaluates the optimized scheduling strategy based on a weighted combination of a resource utilization loss function, a task completion time loss function, and a scheduling efficiency loss function, and generates the optimal scheduling scheme.

6. The method according to claim 1, characterized in that, It employs intelligent power balancing technology to achieve precise matching between computing resources and power consumption targets through dynamic voltage and frequency adjustment; it monitors and quantifies the resource status parameters of the hierarchical heterogeneous resource management system in real time, generating multi-dimensional status data including: The voltage vector, frequency vector, temperature vector, and utilization vector of heterogeneous computing resources are collected. Based on the voltage vector and the frequency vector, the basic power consumption value is calculated. The basic power consumption value is input into the fractal dynamics model. The fractal dynamics model describes the power consumption distribution characteristics through the fractal dimension matrix and characterizes the power consumption fluctuation characteristics through the Hurst exponent time series, thereby generating power consumption state characteristics. The temperature vector and the utilization vector are input into the fractal dynamics model. The fractal dynamics model analyzes the temperature distribution characteristics and utilization distribution characteristics based on the fractal dimension matrix, and uses the Hurst exponent time series to characterize the temperature fluctuation characteristics and utilization fluctuation characteristics, generating temperature state characteristics and utilization state characteristics. The power consumption state characteristics, the temperature state characteristics, and the utilization state characteristics are used to construct multidimensional state data.

7. A dynamic model switching framework for AI inference optimization systems of edge devices, used to implement the method of any one of claims 1-6, characterized in that, include: The first unit is used to receive multiple model versions with preset compression levels, apply a federated learning mechanism to collaboratively train a quantization compensation model among edge device clusters, and the quantization compensation model is used to dynamically correct the quantization error of multiple model versions with the compression levels and generate a set of compensated optimized model versions. The second unit is used to construct a hierarchical heterogeneous resource management system based on the set of optimized model versions. The hierarchical heterogeneous resource management system dynamically allocates and virtualizes computing resources based on software-defined hardware technology, and implements collaborative scheduling of CPU, GPU and NPU for the computing characteristics of each model version in the set of optimized model versions. Based on the collaborative scheduling, reconfigurable computing technology is used to generate and adjust FPGA-dedicated inference acceleration circuits in real time. The third unit is used to integrate the compensation parameters for the quantization error into the dedicated inference acceleration circuit of the FPGA, and to use intelligent power balancing technology to achieve precise matching between computing resources and power consumption targets through dynamic voltage and frequency adjustment; to monitor and quantify the resource status parameters of the hierarchical heterogeneous resource management system in real time, and to generate multi-dimensional status data. The fourth unit is used to construct a dynamic decision engine based on the multidimensional state data. The dynamic decision engine performs trend analysis on the multidimensional state data through chaotic synchronization mapping, identifies the fluctuation characteristics of the multidimensional state data, and generates a switching timing evaluation result. The switching timing evaluation result is input into the annealing search module, which calculates the decision threshold based on the adaptive annealing algorithm. The adaptive annealing algorithm updates the decision threshold using a dynamic cooling strategy to generate the optimal switching timing. Based on the optimal switching timing, the optimization model version switching operation is triggered to switch the current optimization model version to the target optimization model version.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images