Edge device collaborative inference method and system based on model distillation

Abstract

The application provides an edge device collaborative inference method and system based on model distillation, relates to the technical field of edge computing, and the method comprises the following steps: obtaining a pre-trained teacher model, which is divided into L continuous functional sublayers; extracting a sublayer energy vector through local distillation training; analyzing the sublayer energy vector and a device resource vector based on a matching degree function to generate a multi-copy redundant deployment distillation model; when receiving an inference task, extracting task requirement information, establishing an optimization objective function for collaborative analysis; and constructing an optimized teacher model from the L optimized functional sublayers to collaboratively execute the inference task. The application solves the technical problem that, due to the inherent resource limitation, device heterogeneity and state dynamics of the edge computing environment, the multi-dimensional targets such as the processing efficiency and model accuracy of the service conflict with each other, further affecting the inference task execution efficiency, and improves the inference task execution efficiency by constructing an edge collaborative distillation inference system.

Description

Technical Field

[0001] This application relates to the field of edge computing technology, specifically to a collaborative reasoning method and system for edge devices based on model distillation. Background Technology

[0002] With the widespread application of edge computing, an increasing number of computing tasks are being pushed to resource-constrained edge devices. The computing power, storage space, and energy consumption of these devices have become key factors limiting the efficiency of inference task execution. Traditional centralized computing solutions cannot effectively address the highly limited resources of edge devices, the heterogeneity of devices, and the dynamic changes in their states. This leads to complex allocation and coordination of inference tasks across different devices, failing to meet the balance between low latency, high accuracy, and low energy consumption. Especially in environments where the requirements for accuracy, latency, bandwidth, and energy consumption in inference tasks change dynamically, existing solutions often fail to achieve efficient and flexible resource scheduling and task optimization.

[0003] Currently, existing methods mainly rely on deploying large-scale models directly to edge devices or simplifying them through model compression. However, these methods often fail to balance the trade-off between inference task processing efficiency and model accuracy, and their robustness and adaptability are poor when device status fluctuates or task requirements change. Although some solutions attempt to distribute computing tasks across multiple devices for collaborative processing, the high heterogeneity of devices and the lack of flexibility in resource allocation make it difficult to achieve optimal collaborative inference performance in multi-tasking environments. This limits the widespread application of edge computing systems in complex application scenarios and further affects processing efficiency and reliability.

[0004] In summary, existing technologies suffer from technical problems due to the inherent resource constraints, device heterogeneity, and dynamic state of edge computing environments. These problems lead to conflicts between multiple objectives, such as service processing efficiency and model accuracy, which further affect the execution efficiency of inference tasks. Summary of the Invention

[0005] The purpose of this application is to provide a collaborative reasoning method and system for edge devices based on model distillation, in order to solve the technical problem in the prior art that the inherent resource constraints, device heterogeneity and state dynamics of the edge computing environment lead to conflicts between multiple objectives such as service processing efficiency and model accuracy, which further affect the execution efficiency of reasoning tasks.

[0006] To achieve the above objectives, this application provides a collaborative reasoning method and system for edge devices based on model distillation.

[0007] Firstly, this application provides an edge device collaborative inference method based on model distillation. This method is implemented through an edge device collaborative inference system based on model distillation. The method includes: acquiring a pre-trained teacher model and dividing it into L consecutive functional sub-layers; performing local distillation training on the L consecutive functional sub-layers to obtain L sub-student models and extracting the sub-layer energy vector corresponding to each sub-student model; collecting the device resource vector corresponding to each edge device node in the edge device node set, analyzing the sub-layer energy vector and the device resource vector based on a matching degree function, and redundantly deploying multiple copies of each sub-student model to multiple edge device nodes with a matching degree greater than a preset threshold to generate a multi-copy redundant deployment distillation model; when receiving an inference task, extracting the task requirement information of the inference task, establishing an optimization objective function based on the task requirement information, and collaboratively analyzing the multi-copy redundant deployment distillation model to obtain L preferred functional sub-layers; and having the L preferred functional sub-layers constitute a preferred teacher model to collaboratively execute the inference task.

[0008] Optionally, the teacher model is divided into L consecutive functional sub-layers according to a preset method, which includes at least one of fixed-depth division, functional module division, feature resolution division, and computational complexity division.

[0009] Optionally, a convolutional layer, a channel alignment layer, and a normalization layer with a unified interface are deployed in each of the L consecutive functional sub-layers to obtain L consecutive functional standard sub-layers; lightweight constraint processing is applied to the L consecutive functional standard sub-layers to obtain L initial sub-student models; sub-layer distillation training samples are constructed, and distillation loss is calculated on the L initial sub-student models based on the sub-layer distillation training samples until L sub-student models are trained to convergence.

[0010] Optionally, a multi-objective distillation loss function is constructed, which includes a weighted fitting function of feature distillation loss, soft label distillation loss, task supervision loss, and attention alignment loss; the distillation loss is calculated for each initial sub-student model according to the multi-objective distillation loss function, and the parameters of the L initial sub-student models are updated by backpropagation based on the distillation loss calculation results.

[0011] Optionally, the sub-layer energy vector and the device resource vector are analyzed based on the matching degree function to obtain computing power matching items, latency matching items, bandwidth matching items, energy consumption matching items, and stability matching items; when the value of each matching item is greater than a preset threshold, the first group of candidate edge device nodes for each sub-student model is obtained; the first group of candidate edge device nodes is subjected to a comprehensive matching value weighted calculation to obtain multiple edge device nodes with a matching degree greater than a preset threshold.

[0012] Optionally, obtain L replica expansion factors corresponding to the L consecutive functional sub-layers, wherein each replica expansion factor is calculated by identifying the historical call frequency of each consecutive functional sub-layer; and expand the deployment of the multi-replica redundant deployment distillation model according to the L replica expansion factors.

[0013] Optionally, the task requirement information is parsed to obtain a task requirement vector, including an allowable latency vector, an accuracy requirement vector, an energy consumption requirement vector, and a bandwidth usage requirement vector; the multi-replica redundant deployment distillation model is transformed into a collaborative inference weighted graph model, where the graph nodes of the collaborative inference weighted graph model correspond to edge device nodes, and the edges correspond to connections with inter-layer continuity; the optimization objective function is used to perform dynamic programming algorithm optimization on the collaborative inference weighted graph model to obtain a sequence of preferred functional sub-layers; L preferred functional sub-layers are selected from the sequence of preferred functional sub-layers.

[0014] Optionally, a dynamic programming state is defined, which includes the cumulative cost from the first sub-layer to the Lth sub-layer, including cumulative delay, cumulative energy consumption, cumulative bandwidth, and cumulative distillation accuracy. Under the objective function of minimizing the cumulative cost, the Lth sub-layer is selected by the dynamic programming algorithm, and the transition calculation of the dynamic programming state is performed. Based on the transition calculation result, a backtracking cumulative cost analysis of the Lth sub-layer is performed to obtain the preferred functional sub-layer sequence.

[0015] Optionally, edge device nodes with a cumulative cost threshold in the transfer calculation results are pruned to obtain a pruned collaborative reasoning weighted graph model; the L-1 sub-layer is selected based on the pruned collaborative reasoning weighted graph model, and the transfer calculation of the dynamic programming state is performed, and so on until the first sub-layer is obtained to obtain the preferred functional sub-layer sequence.

[0016] Secondly, this application also provides an edge device collaborative inference system based on model distillation, used to execute the edge device collaborative inference method based on model distillation as described in the first aspect, wherein the edge device collaborative inference system based on model distillation includes: a model partitioning module, used to obtain a pre-trained teacher model and partition the teacher model into L consecutive functional sub-layers; a local distillation training module, used to perform local distillation training on the L consecutive functional sub-layers to obtain L sub-student models and extract the sub-layer energy vector corresponding to each sub-student model; a matching degree analysis module, used to collect the device resource vector corresponding to each edge device node in the edge device node set, analyze the sub-layer energy vector and the device resource vector based on the matching degree function, and redundantly deploy each sub-student model to multiple edge device nodes with a matching degree greater than a preset threshold to generate a multi-replica redundant deployment distillation model; a collaborative analysis module, used to extract the task requirement information of the inference task when a reasoning task is received, establish an optimization objective function based on the task requirement information to perform collaborative analysis on the multi-replica redundant deployment distillation model, and obtain L preferred functional sub-layers; and a task execution module, used to have the preferred teacher model composed of the L preferred functional sub-layers collaboratively execute the reasoning task.

[0017] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0018] By acquiring a pre-trained teacher model, the teacher model is divided into L consecutive functional sub-layers; local distillation training is performed on the L consecutive functional sub-layers to obtain L sub-student models, and the sub-layer energy vector corresponding to each sub-student model is extracted; the device resource vector corresponding to each edge device node in the edge device node set is collected, and the sub-layer energy vector and the device resource vector are analyzed based on the matching degree function, and each sub-student model is redundantly deployed to multiple edge device nodes with a matching degree greater than a preset threshold to generate a multi-replica redundant deployment distillation model; when an inference task is received, the task requirement information of the inference task is extracted, and an optimization objective function is established based on the task requirement information to perform collaborative analysis on the multi-replica redundant deployment distillation model to obtain L preferred functional sub-layers; the L preferred functional sub-layers constitute a preferred teacher model to collaboratively execute the inference task. In other words, by decoupling the pre-trained teacher model through hierarchical distillation, and integrating resource-aware multi-copy dynamic deployment with task-driven online collaborative path optimization, resources and model capabilities are adaptively and dynamically allocated in complex and ever-changing edge environments. This provides optimal solutions for diverse tasks under multiple constraints in real time, and each inference task can obtain a customized optimal solution based on current resources and needs, thereby improving the model's adaptability and generalization ability.

[0019] The above description is merely an overview of the technical solution of this application. To better understand the technical means of this application and to facilitate its implementation according to the description, and to make the above and other objects, features, and advantages of this application more apparent, specific embodiments of this application are described below. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent through the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the collaborative reasoning method for edge devices based on model distillation proposed in this application.

[0022] Figure 2 This is a schematic diagram of the edge device collaborative reasoning system based on model distillation in this application.

[0023] Figure labeling: Model partitioning module 11, Local distillation training module 12, Matching degree analysis module 13, Collaborative analysis module 14, Task execution module 15. Detailed Implementation

[0024] This application provides a collaborative inference method and system for edge devices based on model distillation. It addresses the technical problem in existing technologies where the inherent resource constraints, device heterogeneity, and dynamic states of edge computing environments lead to conflicts between multiple objectives, such as service processing efficiency and model accuracy, further impacting the efficiency of inference tasks. By decoupling the pre-trained teacher model through hierarchical distillation, and integrating resource-aware multi-copy dynamic deployment with task-driven online collaborative path optimization, the method adaptively and dynamically allocates resources and model capabilities in complex and ever-changing edge environments. This provides optimal solutions under multiple constraints for diverse tasks in real time, allowing each inference task to obtain a customized optimal solution based on current resources and needs, thus improving the model's adaptability and generalization ability.

[0025] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. It should be understood that this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should also be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all of them.

[0026] Example 1, please refer to the appendix. Figure 1 This application provides a collaborative reasoning method for edge devices based on model distillation, wherein the collaborative reasoning method for edge devices based on model distillation is applied to a collaborative reasoning system for edge devices based on model distillation, and the collaborative reasoning method for edge devices based on model distillation specifically includes the following steps:

[0027] Obtain a pre-trained teacher model and divide the teacher model into L consecutive functional sub-layers.

[0028] Furthermore, this application also includes the following steps: dividing the teacher model into L consecutive functional sub-layers according to a preset method, wherein the preset method includes at least one of fixed depth division, functional module division, feature resolution division, and computational complexity division.

[0029] Specifically, a pre-trained teacher model is a deep learning model that has been trained on a large dataset and has achieved high accuracy. It is usually quite complex, containing a large number of parameters and hierarchical structures, which may place a significant burden on edge devices in terms of computation, but it can provide high inference accuracy in terms of performance.

[0030] Based on the actual needs of edge collaborative inference, such as device computing power distribution, communication bandwidth constraints, and task characteristics, one or more preset partitioning methods are selected to divide the entire model structure into L functionally continuous and clearly defined sub-layers. Preset methods include at least one of fixed-depth partitioning, functional module partitioning, feature resolution partitioning, and computational complexity partitioning.

[0031] Fixed-depth partitioning involves dividing the model into uniform or specified depths based on the number of layers, such as dividing a 100-layer model into sub-layers of 10 layers each. Functional module partitioning is based on the functional modules designed within the model itself, such as dividing each residual stage in ResNet into a sub-layer. Feature resolution partitioning is based on the spatial size variation of the intermediate feature maps in the network, grouping consecutive layers with the same resolution into a single sub-layer. Computational complexity partitioning is based on computational load (e.g., the number of floating-point operations) to ensure a roughly balanced computational load across each sub-layer. For example, using functional module partitioning, ResNet-50 can be divided into initial convolutional pooling layers, four residual stages, and fully connected layers at the end, based on its inherent hierarchical structure. During partitioning, it is crucial to ensure that the input and output tensor dimensions of each sub-layer match to maintain the continuity of the computational flow. Simultaneously, the structural definition, parameters, and position of each sub-layer within the complete model should be recorded to create a detailed list of sub-layers.

[0032] For example, taking image classification as an example, a ResNet-50 model pre-trained on the ImageNet dataset is selected as the teacher model. It contains approximately 25.5 million parameters and achieves a Top-1 classification accuracy of 76.1% on the ImageNet validation set. On edge computing devices, the average latency for complete model inference using a single 224x224 resolution image is approximately 45 milliseconds, with a power consumption of approximately 12 watts. ResNet-50 is a deep residual network whose core idea is to solve the gradient vanishing problem in deep network training through shortcut connections. The input layer receives a fixed-size input image of 224 pixels × 224 pixels × 3 channels (RGB). The initial convolutional and pooling layers include a 7×7 convolutional layer (stride 2, padding 3) with 64 output channels, followed by a batch normalization layer and a ReLU activation function; a 3×3 max-pooling layer (stride 2) reduces the output feature map size to 56×56; four consecutive residual stages (stages 1 to 4) constitute the main body of the model. Each stage consists of stacked residual blocks of varying numbers, in the order of 3, 4, 6, and 3 blocks. Each residual block is a basic functional unit with a clearly defined internal structure. Taking the bottleneck residual block as an example, its standard path contains three convolutional layers: the first layer is a 1×1 convolution for dimensionality reduction; the second layer is a 3×3 convolution for feature extraction; and the third layer is a 1×1 convolution for dimensionality increase. The output of this path is added element-wise to the block's input, such as through shortcut connections, and if necessary, using 1×1 convolutions for dimensionality matching, followed by ReLU activation of the output. Downsampling is performed between stages using a convolution operation with a stride of 2 within the first residual block, halving the feature map space size (e.g., from 56×56 to 28×28 to 14×14 to 7×7), while doubling the number of channels (e.g., from 64 to 256 to 512 to 1024 to 2048). The final classification layer includes a global average pooling layer, a fully connected layer, and a softmax function. A global average pooling layer is used to transform a 7×7×2048 feature map into a 1×1×2048 feature vector; a fully connected layer maps the 2048-dimensional feature vector to a 1000-dimensional vector, corresponding to the 1000 categories of the ImageNet dataset; and a softmax function is used to transform the output into probability distributions for each category. A publicly available dataset containing approximately 1.28 million training images and 50,000 validation images, covering 1000 object categories, is used. Images are trained using random cropping to 224×224, random horizontal flipping, and color jittering for data augmentation. The model is initialized using a cross-entropy loss function to measure the difference between the model's predicted probability distribution and the one-hot encoding of the ground truth labels.The optimizer uses stochastic gradient descent or the Adam optimizer; the batch size is set to 256, requiring multi-GPU parallel training; the initial learning rate is typically set to 1e-4, using Adam; the learning rate scheduling employs cosine annealing or a step-by-step descent strategy, such as multiplying the learning rate by 0.1 at 30%, 60%, and 80% of the training cycle; the total training cycle is 90 to 120 epochs, with one complete traversal of the entire training set constituting one epoch; the weight decay is set to 1e. -4 This is used to prevent overfitting. After the above training, a typical performance of a qualified ResNet-50 teacher model on the validation set is approximately 76.1% Top-1 accuracy and approximately 92.9% Top-5 accuracy. Its model file (containing approximately 25.56 million parameters) can be used as input for all subsequent steps.

[0033] Using a functional module partitioning approach, ResNet-50 is divided into 5 consecutive functional sub-layers: Sub-layer 1 contains an initial 7x7 convolution, batch normalization, ReLU activation, and a 3x3 max pooling layer, with an output feature map size of 56x56, 64 channels, and a computational cost of approximately 410 million floating-point operations (FLOPs); Sub-layer 2 corresponds to stage 1 of the original model, containing 3 residual blocks, with an output feature map size of 56x56, 256 channels, and a computational cost of approximately 390 million FLOPs; Sub-layer 3 corresponds to stage 2... The first sublayer contains 4 residual blocks, with an output feature map size of 28x28, 512 channels, and a computational cost of approximately 430 million FLOPs. Sublayer 4 corresponds to stage 3, containing 6 residual blocks, with an output feature map size of 14x14, 1024 channels, and a computational cost of approximately 400 million FLOPs. Sublayer 5 corresponds to stage 4, containing 3 residual blocks, a global average pooling layer, and a 1000-dimensional fully connected classification layer, with an output feature map size of 7x7, 2048 channels, and a computational cost of approximately 370 million FLOPs. After partitioning, each sublayer can be independently derived as a separate neural network module, with parameter scales of approximately 900,000 for sublayer 1, approximately 2.1 million for sublayer 2, approximately 3.2 million for sublayer 3, approximately 7.1 million for sublayer 4, and approximately 12.2 million for sublayer 5, maintaining the integrity of the model's functionality and providing clear module boundaries for subsequent steps.

[0034] Continuous functional sublayers divide a complete deep learning model into multiple logically continuous and functionally independent sub-parts according to its computational order or structural modules. Each sublayer receives the output of the previous sublayer as input and produces its own output for the next sublayer to use; they are chained together to complete the entire computational process of the original model. By dividing the teacher model into multiple continuous functional sublayers, the sublayers can be flexibly deployed according to the resource availability of edge devices. Each edge device only needs to execute its assigned sublayer, rather than processing the entire complex model, thus reducing the computational burden on a single device.

[0035] Local distillation training is performed on the L consecutive functional sub-layers to obtain L sub-student models, and the sub-layer energy vector corresponding to each sub-student model is extracted.

[0036] Furthermore, this application also includes the following steps: deploying a convolutional layer, a channel alignment layer, and a normalization layer with a unified interface in each of the L consecutive functional sub-layers to obtain L consecutive functional standard sub-layers; performing lightweight constraint processing on the L consecutive functional standard sub-layers to obtain L initial sub-student models; constructing sub-layer distillation training samples, and calculating the distillation loss of the L initial sub-student models based on the sub-layer distillation training samples, until L sub-student models that have been trained to convergence are obtained.

[0037] Furthermore, this application also includes the following steps: constructing a multi-objective distillation loss function, wherein the multi-objective distillation loss function includes a weighted fitting function of feature distillation loss, soft label distillation loss, task supervision loss, and attention alignment loss; calculating the distillation loss for each initial sub-student model according to the multi-objective distillation loss function; and updating the parameters of the L initial sub-student models through backpropagation based on the distillation loss calculation results.

[0038] Specifically, a structure is built for each consecutive functional sub-layer derived from the teacher model. Each functional sub-layer is connected to a configurable, unified-interface convolutional layer to standardize the number of channels in the input and output tensors. After the feature output points within each sub-layer, channel alignment and normalization layers are inserted to transform each functional sub-layer into a standardized computational unit, ensuring that each sub-layer has the same input / output format and computational rules. The unified-interface convolutional layer adds a standard, configurable 1×1 convolutional layer at both the input and output ends of each functional sub-layer. Its main purpose is to allow for flexible adjustment of the number of channels and feature fusion, ensuring that regardless of the original teacher sub-layer's input / output channel count, its corresponding student sub-layer can receive and transmit data with a uniform data specification (e.g., a fixed number of channels) by configuring the parameters of this convolutional layer. The channel alignment layer is a network layer specifically designed to match the number of channels in the feature maps. When the number of feature map channels at corresponding positions in the teacher sub-layer and the student sub-layer is inconsistent, this layer uses a learnable linear projection, typically a 1×1 convolution, to map the number of feature map channels in the student sub-layer to the same dimension as the teacher sub-layer's feature map. This facilitates subsequent channel-by-channel feature value comparison and knowledge distillation. The normalization layer, usually referring to a batch normalization layer or layer normalization layer, is deployed after the channel alignment layer to stabilize the internal data distribution of the student sub-layer during training, accelerate training convergence, and standardize the numerical range of intermediate features, making meaningful comparisons and loss calculations with the teacher model's features easier. After deployment, L consecutive functional standard sub-layers with standard external interfaces and internal observation points are obtained. For example, for a certain functional sub-layer, its corresponding standard sub-layer structure is as follows: a unified input interface convolutional layer—a 1×1 convolutional layer with the input channel number fixed to the output channel number of the previous standard sub-layer, such as 256; the output channel number is set according to design requirements, such as 128, ensuring all sub-layers receive input in a unified dimension. The lightweight core network is the main component replacing the complex structure of the original teacher sub-layer. For example, if it's a residual stage of ResNet, the lightweight core network can consist of several depthwise separable convolutional or inverse residual blocks, with the specific number of layers and channels determined by lightweight constraints. The channel alignment layer is a 1×1 convolutional layer with the same number of input channels as the lightweight core network's output channels. The number of output channels is strictly equal to the number of channels of the feature output at the corresponding position of this functional sub-layer. The purpose is to align the student's intermediate features with the teacher's in dimension, facilitating direct comparison. The normalization layer is usually a batch normalization layer, placed after the channel alignment layer, used for training stabilization. The output unified interface convolutional layer is a 1×1 convolutional layer with the same number of input channels as the number of channels of the feature output at the corresponding position of this functional sub-layer. The number of output channels is fixed, such as 256, serving as the standard output of this sub-layer for use by subsequent sub-layers.

[0039] Lightweight constraints are applied to L consecutive functional standard sub-layers to reduce the computational complexity and model parameters of each sub-layer, adapting to the resource constraints of edge devices. For example, techniques such as pruning, quantization, and low-rank decomposition can reduce the computational cost of each sub-layer. Lightweight constraints are design constraints imposed on the network structure during the construction of the initial sub-student model, aimed at reducing the number of parameters and computational cost. Specific methods include, but are not limited to, replacing standard convolutions with depthwise separable convolutions, reducing the number of convolutional filters (channels), reducing the number of residual blocks, or using more compact network modules, such as inverted residual blocks in MobileNetV2. The core objective is to significantly reduce model complexity without excessively compromising functionality.

[0040] The sub-layer distillation training samples are constructed, including not only the regular input images and ground truth labels, but more importantly, the intermediate feature maps generated by each corresponding functional sub-layer when the teacher model processes these samples, serving as supervision signals. During training, for each initial student model, sample images are input simultaneously, and its own generated features and outputs are obtained, along with the features generated by the corresponding sub-layer of the teacher model and the model's final soft labels. For each input image in the dataset, it is input into the already trained teacher model for forward propagation. In this process, not only the final output of the teacher model is saved, but also the intermediate feature maps of each sub-layer of interest, i.e., the L sub-layers. The specific steps are as follows: the input image undergoes the same preprocessing as during teacher model training (e.g., scaling, cropping, normalization, etc.). The preprocessed image is input into the teacher model, and forward propagation is performed sequentially through each layer. During the forward propagation of the teacher model, intermediate feature maps are intercepted and saved at the predefined L cut points, i.e., at the output of each functional sub-layer. These are typically four-dimensional tensors, with batch size, number of channels, height, and width specified. The final logits output of the teacher model is softened by a softmax layer (with a temperature parameter T, typically T>1) to obtain soft labels. For the l-th sub-layer, a triplet is saved: (input image, teacher l-th sub-layer feature map, teacher soft label). Simultaneously, the ground truth label of the input image is also saved for task supervision loss. Based on this, a distilled training sample set is obtained for each sub-layer. The size of each sample set is equal to the size of the base dataset, but each sample contains supervision information specific to that sub-layer.

[0041] A multi-objective distillation loss function is constructed, including a weighted fitting function for feature distillation loss, soft label distillation loss, task supervision loss, and attention alignment loss. For feature distillation loss, after the student model completes the calculation of the corresponding sublayer, the intermediate feature map generated is compared point-by-point with the teacher's reference answer (feature map), and the differences, such as mean squared error, are calculated. The final output (soft label) of the teacher model contains rich inter-class correlation information. The probability distribution of the student model's final output is required to be similar to the teacher's soft label distribution, allowing students to learn the teacher's relative grasp and fuzzy boundaries in judgment, rather than simply memorizing the final category. To prevent students from completely imitating the teacher and forgetting the most basic objective, the actual classification labels are still used to calculate conventional classification errors, such as cross-entropy loss. Excellent teacher models pay different attention to different channels and spatial locations when processing features. An attention map is extracted by calculating the mean of each channel of the feature map. Then, the student model's attention map is required to match the teacher's attention map, which is equivalent to requiring students not only to know the answer but also to learn where the teacher's focus is when solving the problem. The calculated feature distillation loss, soft label distillation loss, task supervision loss, and attention alignment loss are weighted and summed according to a preset importance ratio. For example, feature imitation is the most important and is given a weight of 1.0; style imitation is the next most important and is given a weight of 0.5; and so on, to obtain a comprehensive score. The goal of training is to adjust the parameters of the student model so that this comprehensive score gets lower and lower, that is, the student behaves more and more like the teacher.

[0042] A multi-objective distillation loss function is used to calculate the distillation loss for each initial sub-student model. Based on the calculated total loss, i.e., the distillation loss result, all parameters of that sub-student model are updated using the backpropagation algorithm, including its original lightweight core, the newly added unified interface layer, and the alignment layer. This process is performed independently and in parallel on L sub-student models, iterating continuously until the loss function of each sub-student model no longer decreases significantly, reaching a convergence state. This yields the final L high-performance lightweight quantum student models. The training objective is to adjust the parameters of the student models to make the overall score increasingly lower, i.e., the student behaves more and more like the teacher.

[0043] By standardizing the convolutional layers, channel alignment layers, and normalization layers with a unified interface for each sub-layer, the functional consistency and efficient computation of each sub-layer are ensured. Simultaneously, lightweight constraint processing reduces model complexity, making each sub-layer more adaptable to the resource constraints of edge devices, thereby improving the processing efficiency of inference tasks.

[0044] Guided by a multi-objective distillation loss function, the student model not only approaches the teacher model in accuracy but also achieves optimizations in multiple aspects, such as task supervision and attention mechanisms. Through backpropagation training, the student model effectively approximates the performance of the teacher model while reducing computational resource consumption, thus improving the speed and efficiency of inference tasks. Efficient inference task processing is achieved on edge devices, ensuring high inference accuracy while significantly reducing computational burden and resource consumption.

[0045] Collect the device resource vector corresponding to each edge device node in the edge device node set, analyze the sub-layer energy vector and the device resource vector based on the matching degree function, and redundantly deploy each sub-student model to multiple edge device nodes with a matching degree greater than a preset threshold to generate a multi-replica redundant deployment distillation model.

[0046] Furthermore, this application also includes the following steps: analyzing the sub-layer energy vector and the device resource vector based on the matching degree function to obtain computing power matching items, latency matching items, bandwidth matching items, energy consumption matching items, and stability matching items; when the value of each matching item is greater than a preset threshold, obtaining the first group of candidate edge device nodes for each sub-student model; performing a comprehensive matching value weighted calculation on the first group of candidate edge device nodes to obtain multiple edge device nodes with a matching degree greater than a preset threshold.

[0047] Specifically, the system collects the device resource vector for each edge device node in the edge device node set. This vector includes the computing power limit, load rate, available bandwidth, energy efficiency factor, and stability. The computing power limit is the maximum sustainable computing capacity the device can provide, such as a CPU computing power of 100 GFLOPS (gigaflops per second). The load rate is the percentage of computing resources currently used by the device (0%-100%), such as a current CPU utilization of 65%. Available bandwidth is the currently available uplink / downlink bandwidth (in Mbps or MB / s) of the device's network interface, such as a current available uplink bandwidth of 80 MB / s. The energy efficiency factor is the energy efficiency ratio of the device performing computations, usually expressed as performance / watt or the energy consumed to complete a unit of computation (e.g., 1 GFLOP), such as the device's energy efficiency being 10 GFLOPS / W. Stability is a comprehensive indicator of device reliability, possibly based on its historical online rate, mean time between failures (MTBF), or current power levels.

[0048] The sublayer energy vector includes computational complexity, inference latency, intermediate feature size, bandwidth requirements, and energy consumption per unit. Computational complexity is typically measured in floating-point operations or millions of instructions; for example, a sub-student model might require 500 million floating-point operations to complete one inference operation. Inference latency is the expected time for the model to run on a reference hardware (e.g., a 1GHz single-core CPU), such as 120 milliseconds, reflecting its sensitivity to computational speed. Intermediate feature size is the data size of the model's input / output feature maps (in megabytes, MB), such as 2MB for input features and 4MB for output features, affecting the amount of data transmitted between devices. Bandwidth requirements are the data transfer rate requirements derived from the feature size and the expected processing frame rate (e.g., 10 frames per second). Energy consumption per unit is the average energy consumed by the model running on the reference hardware, such as 0.5J for a single inference operation, reflecting the model's power density.

[0049] For a specific sub-student model, five matching terms are calculated for each edge device node in the set: computing power matching term, latency matching term, bandwidth matching term, energy consumption matching term, and stability matching term. The computing power matching item assesses whether the remaining computing power of the device is sufficient to support the model requirements. Remaining computing power = computing power limit * (1 - load rate). The matching value can be remaining computing power / model calculation requirements. The ratio must be greater than 1 (threshold) to indicate sufficient computing power. The latency matching item assesses whether the theoretical latency of running the model on the device meets the requirements. Estimated latency = model calculation requirements / remaining computing power. The matching value can be model latency requirements / estimated latency. The ratio must be greater than 1 (threshold) to indicate that it can be completed on time. The bandwidth matching item assesses whether the available bandwidth of the device can handle the data transmission caused by model inference, such as the output of the predecessor sublayer. The matching value can be available bandwidth / model bandwidth requirements. The ratio must be greater than 1 (threshold) to indicate sufficient bandwidth. The energy consumption matching item assesses whether the energy consumption of running the model on the device is acceptable or relatively efficient. The matching value can be the device energy consumption coefficient / model unit energy consumption index (or other reciprocal form). The higher the ratio, the better the energy efficiency. It must be greater than an energy efficiency threshold. The stability matching item directly uses the device's stability index and must be greater than a reliability threshold (e.g., 0.95).

[0050] A device is included in the first group of candidate edge device nodes in the sub-student model only when the values ​​for all five matching items are greater than the preset threshold, ensuring that each deployment point meets the most basic operating conditions. The preset threshold is a pre-defined value used to determine whether a matching item meets the requirements. When the value of a matching item in the device resource vector and sub-layer energy vector is greater than this threshold, it indicates that the device can meet the resource requirement. Each matching item has a separate threshold, typically set based on the priority and performance requirements of the specific task.

[0051] The candidate nodes selected in the first step are all qualified. To choose a better deployment location, a comprehensive matching value needs to be calculated. For each candidate node, the values ​​of its five matching items, which are dimensionless scalars, are weighted and summed according to the design objectives. For example, the comprehensive matching value = W1 * computing power matching value + W2 * latency matching value + W3 * bandwidth matching value + W4 * energy consumption matching value + W5 * stability matching value, where W1, W2, W3, W4, and W5 are the weights of different indicators, and W1 + W2 + W3 + W4 + W5 = 1. Different weights are assigned to each matching item according to the task requirements to better meet the specific requirements of the task when selecting the final device. For example, for tasks with high real-time requirements, the weight of the latency matching item may be larger, while for resource-constrained environments, the weight of the energy consumption matching item may be higher.

[0052] The matching degree of the first group of candidate edge device nodes is obtained by weighted calculation of the comprehensive matching value. All candidate edge device nodes are sorted according to the comprehensive matching value, and nodes with a comprehensive matching value greater than a preset threshold are selected to obtain multiple edge device nodes. For example, the energy vector of model A is: computational requirement 0.9 GFLOPs, latency requirement 30ms, output feature size 4MB, bandwidth requirement 40MB / s (estimated at 10fps), and unit energy consumption 0.3J. Device 1 has a maximum computing power of 50 GFLOPS (CPU), a load rate of 40%, an available uplink bandwidth of 100 MB / s, an energy efficiency coefficient of 15 GFLOPS / W, and a stability score of 0.99 (online rate). Device 2 has a maximum computing power of 20 GFLOPS (CPU), a load rate of 70%, an available bandwidth of 20 MB / s (4G network), an energy efficiency coefficient of 8 GFLOPS / W, and a stability score of 0.85 (50% battery). Device 3 has a maximum computing power of 200 GFLOPS (GPU), a load rate of 30%, an available bandwidth of 1000 MB / s (wired), an energy efficiency coefficient of 80 GFLOPS / W, and a stability score of 0.995. For Device 1, the computing power matching value of 33.3 is greater than 1.0, the latency matching value is 1.0, the bandwidth matching value of 2.5 is greater than 1.0, the energy efficiency matching value of 50 is greater than 1.0, and the stability matching value of 0.99 is greater than 0.95, so Device 1 is included in the candidate list. Device 2 is eliminated because the latency does not meet the requirements; Device 3 is included in the candidate list. Assuming the current system emphasizes low latency and reliability, and sets weights W1=0.2, W2=0.3, W3=0.1, W4=0.1, and W5=0.3, the overall matching value of device 1 is 12.5, the overall matching value of device 2 is 62.1, and the preset overall matching threshold is 10. Since both are greater than the threshold, device 1 and device 3 will be selected as the deployment targets of model A.

[0053] Based on the matching degree function, multiple device nodes can simultaneously deploy replicas of the same sub-student model, running them on different devices. This ensures that tasks can be processed in parallel across multiple devices, improving computational efficiency and fault tolerance. After redundant deployment, the generated multi-replica redundant deployment distillation model can utilize the resources of multiple devices simultaneously and optimize task execution through collaborative work. For example, when one device node is overloaded, the task can be transferred to another device node with a lower load. By redundantly deploying multiple replicas of the sub-student model across multiple device nodes, the system's fault tolerance and load balancing capabilities can be improved. When a device fails or becomes overloaded, other devices can take over the task, ensuring its continuous execution. Through multi-replica redundant deployment, tasks can be processed in parallel, reducing inference latency and improving the response speed of inference tasks, especially in large-scale edge computing environments.

[0054] Furthermore, this application also includes the following steps: obtaining L replica expansion factors corresponding to the L consecutive functional sub-layers, wherein each replica expansion factor is calculated by identifying the historical call frequency of each consecutive functional sub-layer; and expanding the deployment of the multi-replica redundancy deployment distillation model according to the L replica expansion factors.

[0055] Specifically, the historical call frequency of each consecutive functional sublayer is identified, recording the number of times each sublayer was executed in previous inference tasks. Based on the execution of inference tasks over a past period, the call frequency of each sublayer is statistically analyzed. A replica expansion factor is calculated for each sublayer based on the historical call frequency. The calculation of the replica expansion factor typically follows a rule: the higher the call frequency, the larger the replica expansion factor. The replica expansion factor is intended to reflect whether the sublayer may require more resource support in future tasks. Specifically, a call log is maintained within a sliding time window. For each inference task request, its execution path (i.e., the preferred functional sublayer sequence) determined after dynamic programming is recorded. Within the time window, the number of successfully executed inference tasks that used any replica of that sublayer is counted, and the frequency is calculated. To facilitate comparisons and threshold setting across different sublayers, the frequencies of all sublayers are typically normalized. The replica expansion factor is obtained based on the normalized historical call frequency through a mapping function. For example, several thresholds may be set. If the normalized history call frequency is <0.3, the replica expansion factor = 1 (basic replica); if the normalized history call frequency is 0.3 ≤ normalized history call frequency <0.6, the replica expansion factor = 2; if the normalized history call frequency is ≥0.6, the replica expansion factor = 3.

[0056] A strategy execution cycle is defined, and at the beginning of each cycle, an expansion deployment evaluation process is triggered. Based on the call logs from the previous cycle, the latest expansion factor is calculated. The number of actually deployed, healthy (devices online and resource matching meets standards) replicas in each sub-layer of the current system is read. For each sub-layer, the current number of replicas and the replica expansion factor are compared. If the replica expansion factor is greater than the current number of replicas, expansion is planned for that sub-layer, requiring the addition of replicas equal to the replica expansion factor minus the current number of replicas. If the replica expansion factor is less than the current number of replicas, scaling down is planned for that sub-layer, removing replicas equal to the current number of replicas minus the replica expansion factor, typically prioritizing the removal of replicas with lower overall matching values. If they are equal, the status quo is maintained.

[0057] For sublayers requiring scaling up, the resource matching process is re-executed, similar to the initial deployment. New, highly compatible deployment targets are searched from all currently available edge device nodes for the sublayer model. However, this time there's a significant optimization: devices that haven't yet deployed any replicas of the sublayer are prioritized to maximize replica distribution, thereby improving disaster recovery capabilities and parallel processing potential. The sublayer's student model files are distributed to the selected new devices, and model loading and initialization are initiated. Once the new replica is ready, it's registered in the system's collaborative inference service registry, declaring its availability. Subsequent dynamic programming optimization algorithms can then immediately recognize this newly added candidate node. For sublayers requiring scaling down, replicas are selected for decommissioning according to a predetermined strategy (e.g., least recently used, lowest overall matching value). These replicas are marked as empty and will no longer be selected by new dynamic programming paths; they wait for all currently processed tasks to complete; the replica is deregistered from the registry, and the model is unloaded from the device, releasing resources.

[0058] By calculating the replica expansion factor based on historical call frequency, the system predicts which sub-layers will be frequently invoked and allocates more replicas to these sub-layers, thereby achieving load balancing and improving computational efficiency. Redundant multi-replica deployment ensures that even if some devices fail, other replicas can continue to execute tasks, improving the system's fault tolerance.

[0059] When a reasoning task is received, the task requirement information of the reasoning task is extracted, and an optimization objective function is established based on the task requirement information to perform collaborative analysis on the multi-replica redundant deployment distillation model to obtain L preferred functional sub-layers.

[0060] Furthermore, this application also includes the following steps: parsing the task requirement information to obtain a task requirement vector, including an allowed latency vector, an accuracy requirement vector, an energy consumption requirement vector, and a bandwidth usage requirement vector; converting the multi-replica redundant deployment distillation model into a collaborative inference weighted graph model, wherein the graph nodes of the collaborative inference weighted graph model correspond to edge device nodes, and the edges correspond to connections with inter-layer continuity; the optimization objective function is used to perform dynamic programming algorithm optimization on the collaborative inference weighted graph model to obtain a sequence of preferred functional sub-layers; and selecting L preferred functional sub-layers from the sequence of preferred functional sub-layers.

[0061] Specifically, when a specific inference task arrives, each task carries explicit or implicit performance requirements. These requirements are quantified into a computable task demand vector, including an allowable latency vector, an accuracy requirement vector, an energy consumption requirement vector, and a bandwidth usage requirement vector. The allowable latency vector is the maximum end-to-end inference latency the task can tolerate; for example, a real-time video analytics task might require 150 milliseconds, while an offline image processing task might allow 2 seconds. The accuracy requirement vector is the minimum requirement for the accuracy of the inference results, quantified as a confidence threshold, such as 0.85, or an acceptable loss ratio relative to the accuracy of the teacher model, such as an accuracy drop of no more than 5%. The similarity of the sub-student model's evaluation accuracy on the validation set relative to the teacher features is mapped to this type of metric. The energy consumption requirement vector is the energy budget for task execution; for example, for a task initiated by a mobile device with limited battery power, the energy consumption per inference might be set to no more than 1 joule. The bandwidth usage requirement vector is the maximum network bandwidth that can be used to transmit intermediate data across devices during task execution; for example, in a congested network environment, it might be limited to no more than 50 MB / s to avoid impacting other critical services.

[0062] The multi-replica redundant deployment distillation model is transformed into a collaborative inference weighted graph model, where each node in the graph uniquely corresponds to a specific sub-student model replica on an edge device node. For example, node N{i,j} represents the replica of the i-th sub-layer (such as stage 2) deployed on device j. The weight attributes of the node itself include computation latency, computation energy consumption, model accuracy, and resource consumption. The edges of the graph connect nodes with continuous relationships between layers. Specifically, if the output of node N{i,j} (the replica of the i-th layer on device j) is the input of node N{i+1,k} (the replica of the (i+1)-th layer on device k), then there exists a directed edge from N{i,j} to N{i+1,k}. The weight attributes of the edge include transmission latency, transmission energy consumption, and bandwidth consumption. Finally, an L-layer hierarchical directed graph is obtained, where each layer contains multiple candidate nodes (replicas), and the layers are connected by edges, forming a network topology with multiple weight attributes from the input (first layer) to the output (Lth layer). The connections between each device and sublayer are also converted into edges of a graph, indicating whether the device can support the execution of the sublayer.

[0063] The task requirement vector is transformed into a multi-constraint optimization objective function, and a dynamic programming algorithm is used to find the optimal path in the graph. The merits of a complete inference path (consisting of L nodes connected sequentially) are quantitatively evaluated. This is typically a function of the task requirement vector, and requires optimization of one or more core metrics while satisfying hard constraints.

[0064] For each node in the collaborative reasoning weighted graph model, a state tuple is defined, recording the minimum cumulative cost required to reach that node and the optimal predecessor node. The process proceeds layer by layer, starting from the first layer. For each node in the first layer, its cumulative cost is the cost of performing computations on that node. For each node B in the i-th layer (i>1), consider all possible predecessor nodes A (from the (i-1)-th layer), calculate the transfer cost (transmission delay, transmission energy consumption, etc.) from A to B; calculate the total cost to reach B via A: the cumulative cost of A + transfer cost + computation cost on B; from all possible A's, select the one that minimizes the total cost as the optimal predecessor of B, and update the cumulative cost of B. During the recursive process, continuously check whether the cumulative cost exceeds the constraints specified by the task requirement vector. Once it is found that the cumulative delay of a node exceeds the allowable delay, or the cumulative energy consumption exceeds the energy budget, that node is immediately pruned, and subsequent paths originating from it are no longer considered. After reaching the last layer (layer L), find the node that satisfies all constraints and has the minimum total cost among all nodes, and then backtrack along the optimal predecessor pointer to obtain the complete sequence of preferred functional sublayers.

[0065] In the optimal functional sub-layer sequence, L optimal sub-layers are selected, ensuring that task requirements (such as latency, accuracy, and energy consumption) are best met. By optimizing the objective function and using dynamic programming, suitable devices and paths are selected to satisfy various task requirements, ensuring that the inference task is completed with reasonable resource consumption. By constructing a weighted graph model and applying dynamic programming, the optimal balance between performance and resource consumption is found, improving inference efficiency. By rationally allocating resources based on device availability and task requirements, the utilization efficiency of edge devices is maximized, avoiding resource waste.

[0066] Furthermore, this application also includes the following steps: defining a dynamic programming state, wherein the dynamic programming state includes the cumulative cost corresponding to the first sub-layer up to the Lth sub-layer, including cumulative delay, cumulative energy consumption, cumulative bandwidth and cumulative distillation accuracy; under the objective function of minimizing the cumulative cost, selecting the Lth sub-layer through a dynamic programming algorithm and performing the transition calculation of the dynamic programming state; performing backtracking cumulative cost analysis of the Lth sub-layer based on the transition calculation results to obtain the preferred functional sub-layer sequence.

[0067] Furthermore, this application also includes the following steps: pruning the edge device nodes in the transfer calculation results that are greater than the cumulative cost threshold to obtain a pruned collaborative reasoning weighted graph model; selecting the L-1 sub-layer based on the pruned collaborative reasoning weighted graph model, performing the transfer calculation of the dynamic programming state, and so on until the first sub-layer is obtained, to obtain the preferred functional sub-layer sequence.

[0068] Specifically, in the collaborative reasoning weighted graph model, the dynamic programming state needs to fully describe the cumulative cost of reaching a node in the graph. For any node N{i,j} (representing the model replica deployed on device j at layer i), the state is defined as the sum of all costs and performance along a feasible path from layer 1 to that node. The specific state vector includes four core dimensions: cumulative latency, cumulative energy consumption, cumulative bandwidth, and cumulative distillation accuracy. Cumulative latency is the total time spent from the start of the task (layer 1) to the current node N{i,j}, including the computational latency of all nodes and the transmission latency of all edges. Cumulative energy consumption is the total energy consumed from the start of the task to the current node, including computation at all nodes along the path and data transmission across all edges. Cumulative bandwidth is a constraint state, typically recorded as the maximum single-transmission bandwidth occupied across all transmission edges on the path, i.e., the bottleneck bandwidth, or a list of bandwidth usage for each transmission step for inspection, ensuring that the entire path does not exceed network capacity at any point. Cumulative distillation accuracy is the average functional fidelity or performance metric of all model replicas along the path from the start of the task to the current node. It is usually calculated using a geometric mean or a weighted average. For example, if the model replica accuracies along the path are [0.95, 0.92, 0.90], then the cumulative accuracy might be their product 0.95*0.92*0.90≈0.7866, or the weighted average (0.95+0.92+0.90) / 3≈0.923, representing the overall knowledge quality of the path.

[0069] Each state S(i,j) not only records the cost of reaching this state, but also implicitly defines a complete path from the first layer to the current node, obtained by backtracking the predecessor node. The goal of dynamic programming is to find, among all states that reach the L-th layer node, the state with the minimum overall cost while satisfying the task constraints, and to backtrack its complete path.

[0070] Traditional dynamic programming typically progresses from the starting point (layer 1) to the ending point (layer L) (forward DP). However, this scheme employs a goal-oriented backward dynamic programming approach, starting from the ending layer and calculating and pruning layer by layer. Its advantage lies in its ability to incorporate task objectives into the selection process earlier. For each node N{L,j} in layer L, it is considered the endpoint of the path. The initial value of its state S(L,j) is the cost of performing layer L computation alone at that node, excluding transmission costs (because it is the endpoint), i.e., including cumulative latency, cumulative energy consumption, cumulative bandwidth, and cumulative distillation accuracy. Based on the task requirement vector, all states in layer L are immediately filtered. For each state S(L,j), it is determined whether its cumulative latency, cumulative energy consumption, and cumulative accuracy meet the task's allowable latency, energy consumption, and accuracy requirements, respectively. Since it is the endpoint, bandwidth does not need to be checked, but bandwidth generated by intermediate transmissions will be checked during backtracking. If any node N{L,j} fails to meet the task requirement threshold, it is temporarily removed from the current computational graph model, marked as invalid, and no longer considered as a valid endpoint candidate. This step obtains the pruned collaborative reasoning weighted graph model, specifically the L-th layer subset.

[0071] Starting from layer L-1, calculate backwards layer by layer towards layer 1. For each node N{k,m} in layer k of the pruned graph, calculate the cost of all possible transitions from it to valid nodes in the next layer (layer k+1). Traverse all unpruned nodes N{k+1,n} in layer k+1. For each pair N{k,m} and N{k+1,n}, calculate the transition cost from m to n, i.e., the weight of the edge E{m to n}, including transmission delay, transmission energy consumption, and bandwidth usage. Add the transition cost to the state S(k+1,n) of N{k+1,n} to obtain the cost of a possible complete path with N{k,m} as the penultimate stop and N{k+1,n} as the endpoint. For node N{k,m}, it may have multiple subsequent nodes n to choose from. Select the optimal subsequent path based on the optimization objective function. For example, if the objective is to minimize cumulative latency, then for each m, the n that minimizes (transition latency + S(k+1,n) cumulative latency) is selected, and the corresponding total cost is recorded as the temporary state of S(k,m), while the selected successor node n is recorded as next(m). After obtaining the temporary state of N{k,m}, it is immediately checked using the task requirement vector. It is determined whether the cumulative latency, cumulative energy consumption, cumulative bandwidth, and cumulative accuracy of S(k,m) meet the task requirements. If any one of them exceeds the constraints, node N{k,m} is pruned and removed from the set of valid nodes in the current layer, meaning that starting from this node, it is impossible to reach any valid endpoint while satisfying all constraints. After completing the calculation and pruning of all nodes in the k-th layer, a new, sparser collaborative reasoning weighted graph model is obtained, namely the k-th layer subset. When the reverse computation and pruning reach the first layer, among all the unpruned nodes in the first layer, the node N{1,opt} with the minimum cost is selected according to the optimization objective function (such as minimizing the total delay). The state S(1,opt) represents the complete path cost from the starting point, satisfying all constraints and with the optimal objective function. Starting from this node, the path is backtracked, and the resulting node sequence is the final sequence of optimized functional sublayers, where each node corresponds to a specific copy of the sub-student model deployed on a specific device.

[0072] Through a backtracking process using dynamic programming, a preferred sequence of functional sublayers from the 1st to the Lth sublayer is obtained. L sublayers are selected from this preferred sequence to achieve the best inference task execution performance. These L preferred functional sublayers are then executed in parallel across multiple edge devices to complete the inference task. The dynamic programming algorithm precisely selects the optimal deployment path for each sublayer, ensuring that resource consumption such as latency, energy consumption, and bandwidth are minimized. Dynamic programming selects the optimal sublayer sequence, improving the execution efficiency of the inference task and ensuring that the task meets performance requirements while minimizing resource consumption.

[0073] The L preferred functional sub-layers constitute an preferred teacher model that collaboratively executes the inference task.

[0074] Specifically, L optimized functional sub-layers are organized to form an optimized teacher model, which includes tasks such as input data processing, feature extraction, and inference decision-making. The key to the optimized teacher model lies in making the inference task more efficient and accurate through the rational selection and integration of sub-layers. As a whole model, the optimized teacher model performs task inference through the collaborative action of multiple sub-layers. Within the distillation learning framework, the teacher model typically possesses strong inference capabilities, thus requiring appropriate selection and deployment based on task requirements. The collaborative work of each sub-layer is not simply sequential execution. To ensure efficient execution of the inference task, graph optimization algorithms are used to adjust the execution order and strategies of each sub-layer, ensuring optimal utilization of computational resources for each sub-layer. When the task needs to be executed, the input data is passed to the optimized teacher model. The model's L optimized functional sub-layers will collaboratively execute the inference task sequentially or in parallel until the final inference result is output. During execution, the task requirement vector continues to influence the execution of each sub-layer, ensuring that the inference task always meets the actual requirements.

[0075] For example, the task scheduler coordinates with lightweight executors on various edge devices to initiate a distributed pipeline. Following the order of the subtask chain, the scheduler issues a task start command to the device containing the first subtask (e.g., device A) and distributes a global task context. Device A's executor loads the specified model copy and is ready. Device A's executor receives initial input data, runs the layer 1 model copy, and produces an intermediate feature result F1. Device A transmits F1 (according to the execution plan) over the network to the device containing the second subtask, such as device B. Simultaneously, if the task is pipelined, device A can begin processing the layer 1 of the next task. Device B's executor receives F1, loads the layer 2 model copy, performs calculations using F1 as input, and produces an intermediate feature result F2. Device B transmits F2 to the device containing the third subtask, such as device C. This relay continues until the last device completes the layer L calculation. Throughout the process, the scheduler monitors the execution status, time consumption, and intermediate results of each subtask. If a subtask times out or fails, the scheduler can dynamically trigger fault recovery based on multi-replica deployment information. For example, it can reroute the failed subtask to another healthy replica in the same sublayer for execution, ensuring the overall task success. The device executing the last sublayer produces the final inference result, such as classification labels and detection boxes. This device returns the result to the task scheduler. The scheduler integrates the result with the task metadata and finally returns it to the task initiator, marking the successful completion of this collaborative inference task.

[0076] By selecting the L most suitable functional sublayers, the system ensures that each sublayer achieves optimal performance under resource and task requirements, thereby making the inference task more efficient. The optimal teacher model flexibly adjusts the execution order or strategy of each sublayer according to task requirements, ensuring that the inference task achieves optimal performance under various constraints.

[0077] In summary, the edge device collaborative reasoning method based on model distillation provided in this application has the following technical effects:

[0078] By acquiring a pre-trained teacher model, the teacher model is divided into L consecutive functional sub-layers; local distillation training is performed on the L consecutive functional sub-layers to obtain L sub-student models, and the sub-layer energy vector corresponding to each sub-student model is extracted; the device resource vector corresponding to each edge device node in the edge device node set is collected, and the sub-layer energy vector and the device resource vector are analyzed based on the matching degree function, and each sub-student model is redundantly deployed to multiple edge device nodes with a matching degree greater than a preset threshold to generate a multi-replica redundant deployment distillation model; when an inference task is received, the task requirement information of the inference task is extracted, and an optimization objective function is established based on the task requirement information to perform collaborative analysis on the multi-replica redundant deployment distillation model to obtain L preferred functional sub-layers; the L preferred functional sub-layers constitute a preferred teacher model to collaboratively execute the inference task. In other words, by decoupling the pre-trained teacher model through hierarchical distillation, and integrating resource-aware multi-copy dynamic deployment with task-driven online collaborative path optimization, resources and model capabilities are adaptively and dynamically allocated in complex and ever-changing edge environments. This provides optimal solutions for diverse tasks under multiple constraints in real time, and each inference task can obtain a customized optimal solution based on current resources and needs, thereby improving the model's adaptability and generalization ability.

[0079] Example 2: Based on the same inventive concept as the model distillation-based edge device collaborative inference method in Example 1, this application also provides a model distillation-based edge device collaborative inference system. Please refer to the appendix. Figure 2 The model-distillation-based edge device collaborative inference system includes:

[0080] The model partitioning module 11 is used to obtain a pre-trained teacher model and divide the teacher model into L consecutive functional sub-layers; the local distillation training module 12 is used to perform local distillation training on the L consecutive functional sub-layers to obtain L sub-student models and extract the sub-layer energy vector corresponding to each sub-student model; the matching degree analysis module 13 is used to collect the device resource vector corresponding to each edge device node in the edge device node set, analyze the sub-layer energy vector and the device resource vector based on the matching degree function, and redundantly deploy each sub-student model to multiple edge device nodes with a matching degree greater than a preset threshold to generate a multi-replica redundant deployment distillation model; the collaborative analysis module 14 is used to extract the task requirement information of the inference task when an inference task is received, establish an optimization objective function based on the task requirement information, perform collaborative analysis on the multi-replica redundant deployment distillation model, and obtain L preferred functional sub-layers; the task execution module 15 is used to have the preferred teacher model composed of the L preferred functional sub-layers collaboratively execute the inference task.

[0081] Furthermore, the model partitioning module 11 in the model-based edge device collaborative reasoning system is also used to: partition the teacher model into L consecutive functional sub-layers according to a preset method, wherein the preset method includes at least one of fixed-depth partitioning, functional module partitioning, feature resolution partitioning, and computational complexity partitioning.

[0082] Furthermore, the local distillation training module 12 in the model-based edge device collaborative inference system is also used to: deploy convolutional layers, channel alignment layers, and normalization layers with a unified interface in each of the L consecutive functional sub-layers to obtain L consecutive functional standard sub-layers; perform lightweight constraint processing on the L consecutive functional standard sub-layers to obtain L initial sub-student models; construct sub-layer distillation training samples, and calculate the distillation loss of the L initial sub-student models based on the sub-layer distillation training samples until L sub-student models are trained to convergence.

[0083] Furthermore, the local distillation training module 12 in the model-based edge device collaborative inference system is also used to: construct a multi-objective distillation loss function, which includes a weighted fitting function of feature distillation loss, soft label distillation loss, task supervision loss, and attention alignment loss; calculate the distillation loss for each initial sub-student model according to the multi-objective distillation loss function; and update the parameters of the L initial sub-student models through backpropagation based on the distillation loss calculation results.

[0084] Furthermore, the matching degree analysis module 13 in the model-based distillation-based edge device collaborative inference system is also used to: analyze the sub-layer energy vector and the device resource vector based on the matching degree function to obtain computing power matching items, latency matching items, bandwidth matching items, energy consumption matching items, and stability matching items; when the value of each matching item is greater than a preset threshold, obtain the first group of candidate edge device nodes for each sub-student model; perform a comprehensive matching value weighted calculation on the first group of candidate edge device nodes to obtain multiple edge device nodes with a matching degree greater than a preset threshold.

[0085] Furthermore, the matching degree analysis module 13 in the model-based distillation edge device collaborative inference system is also used to: obtain L replica expansion factors corresponding to the L consecutive functional sub-layers, wherein each replica expansion factor is calculated by identifying the historical call frequency of each consecutive functional sub-layer; and expand the deployment of the multi-replica redundant deployment distillation model according to the L replica expansion factors.

[0086] Furthermore, the collaborative analysis module 14 in the model-based distillation-based edge device collaborative inference system is also used for: parsing the task requirement information to obtain a task requirement vector, including an allowable latency vector, an accuracy requirement vector, an energy consumption requirement vector, and a bandwidth occupancy requirement vector; converting the multi-replica redundant deployment distillation model into a collaborative inference weighted graph model, wherein the graph nodes of the collaborative inference weighted graph model correspond to edge device nodes, and the edges correspond to connections with inter-layer continuity; the optimization objective function is used to perform dynamic programming algorithm optimization on the collaborative inference weighted graph model to obtain a sequence of preferred functional sub-layers; and selecting L preferred functional sub-layers from the sequence of preferred functional sub-layers.

[0087] Furthermore, the collaborative analysis module 14 in the model-based distillation-based edge device collaborative inference system is also used to: define a dynamic programming state, the dynamic programming state including the cumulative cost corresponding to the first sub-layer up to the Lth sub-layer, including cumulative latency, cumulative energy consumption, cumulative bandwidth and cumulative distillation accuracy; under the objective function of minimizing the cumulative cost, select the Lth sub-layer through a dynamic programming algorithm and perform the transition calculation of the dynamic programming state; perform backtracking cumulative cost analysis of the Lth sub-layer based on the transition calculation result to obtain the preferred functional sub-layer sequence.

[0088] Furthermore, the collaborative analysis module 14 in the model-based edge device collaborative reasoning system is also used to: prune edge device nodes in the transfer calculation results that are greater than the cumulative cost threshold to obtain a pruned collaborative reasoning weighted graph model; select the L-1 sub-layer based on the pruned collaborative reasoning weighted graph model, perform the transfer calculation of the dynamic programming state, and so on until the first sub-layer is obtained to obtain the preferred functional sub-layer sequence.

[0089] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The model distillation-based edge device collaborative reasoning method and specific examples in the aforementioned embodiment one are also applicable to the model distillation-based edge device collaborative reasoning system of this embodiment. Through the foregoing detailed description of the model distillation-based edge device collaborative reasoning method, those skilled in the art can clearly understand the model distillation-based edge device collaborative reasoning system of this embodiment. Therefore, for the sake of brevity, it will not be described in detail here.

[0090] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0091] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application also intends to include such modifications and variations.

Claims

1. An edge device collaborative reasoning method based on model distillation, characterized in that, The method includes: Obtain a pre-trained teacher model and divide the teacher model into L consecutive functional sub-layers; Local distillation training is performed on the L consecutive functional sub-layers to obtain L sub-student models, and the sub-layer energy vector corresponding to each sub-student model is extracted; Collect the device resource vector corresponding to each edge device node in the edge device node set, analyze the sub-layer energy vector and the device resource vector based on the matching degree function, and redundantly deploy each sub-student model to multiple edge device nodes with a matching degree greater than a preset threshold to generate a multi-replica redundant deployment distillation model; The method for analyzing the sub-layer energy vector and the device resource vector based on the matching degree function includes: The sub-layer energy vector and the device resource vector are analyzed based on the matching degree function to obtain computing power matching items, latency matching items, bandwidth matching items, energy consumption matching items, and stability matching items; When the value of each matching item is greater than the preset threshold, the first group of candidate edge device nodes for each sub-student model is obtained. The first group of candidate edge device nodes is subjected to a comprehensive matching value weighted calculation to obtain multiple edge device nodes with a matching degree greater than a preset threshold; When a reasoning task is received, the task requirement information of the reasoning task is extracted, and an optimization objective function is established based on the task requirement information to perform collaborative analysis on the multi-replica redundant deployment distillation model to obtain L preferred functional sub-layers. The task requirement information is parsed to obtain a task requirement vector, including an allowable latency vector, a precision requirement vector, an energy consumption requirement vector, and a bandwidth usage requirement vector; The multi-replica redundant deployment distillation model is transformed into a collaborative reasoning weighted graph model, where the graph nodes of the collaborative reasoning weighted graph model correspond to edge device nodes, and the edges correspond to connections with inter-layer continuity. The optimization objective function is used to perform dynamic programming algorithm optimization on the collaborative reasoning weighted graph model to obtain the preferred functional sub-layer sequence; Select L preferred functional sublayers from the preferred functional sublayer sequence; The L preferred functional sub-layers constitute an preferred teacher model that collaboratively executes the inference task.

2. The edge device collaborative inference method based on model distillation as described in claim 1, characterized in that, Local distillation training is performed on the L consecutive functional sub-layers to obtain L sub-student models. The method includes: Deploy a convolutional layer, a channel alignment layer, and a normalization layer with a unified interface in each of the L consecutive functional sub-layers to obtain L consecutive functional standard sub-layers; Lightweight constraints are applied to the L consecutive functional standard sub-layers to obtain L initial sub-student models; Construct sublayer distillation training samples, and calculate the distillation loss of the L initial sub-student models based on the sublayer distillation training samples until L sub-student models are trained to convergence.

3. The edge device collaborative inference method based on model distillation as described in claim 2, characterized in that, The method for calculating the distillation loss of the L initial sub-student models based on the sub-layer distillation training samples includes: A multi-objective distillation loss function is constructed, which includes a weighted fitting function of feature distillation loss, soft label distillation loss, task supervision loss, and attention alignment loss; Distillation loss is calculated for each initial sub-student model according to the multi-objective distillation loss function, and parameters are updated by backpropagation of the L initial sub-student models based on the distillation loss calculation results.

4. The edge device collaborative inference method based on model distillation as described in claim 1, characterized in that, After generating a multi-replica redundant deployment distillation model, the method also includes: Obtain L replica expansion factors corresponding to the L consecutive functional sub-layers, wherein each replica expansion factor is calculated by identifying the historical call frequency of each consecutive functional sub-layer; The multi-replica redundant deployment distillation model is expanded and deployed based on the L replica expansion factors.

5. The edge device collaborative inference method based on model distillation as described in claim 1, characterized in that, The optimization objective function is used to perform dynamic programming optimization on the collaborative reasoning weighted graph model to obtain the preferred functional sub-layer sequence. The method includes: Define a dynamic programming state, which includes the cumulative cost from the first sub-layer to the Lth sub-layer, including cumulative delay, cumulative energy consumption, cumulative bandwidth and cumulative distillation accuracy; Under the objective function of minimizing the cumulative cost, the Lth sub-layer is selected by dynamic programming algorithm, and the transition calculation of the dynamic programming state is performed. Based on the transfer calculation results, a backtracking cumulative cost analysis of the L sublayer is performed to obtain the preferred functional sublayer sequence.

6. The edge device collaborative inference method based on model distillation as described in claim 5, characterized in that, Based on the transfer calculation results, a backtracking cumulative cost analysis of the L sublayer is performed, including: The edge device nodes with values ​​greater than the cumulative cost threshold in the transfer calculation results are pruned to obtain the pruned collaborative reasoning weighted graph model. Based on the pruned collaborative reasoning weighted graph model, the (L-1)th sub-layer is selected, and the transition calculation of the dynamic programming state is performed. This process is repeated until the first sub-layer is obtained, thus acquiring the preferred functional sub-layer sequence.

7. The edge device collaborative inference method based on model distillation as described in claim 1, characterized in that, The teacher model is divided into L consecutive functional sub-layers according to a preset method, which includes at least one of fixed depth division, functional module division, feature resolution division, and computational complexity division.

8. An edge device collaborative inference system based on model distillation, characterized in that, The step of implementing the model distillation-based edge device collaborative inference method according to any one of claims 1 to 7, wherein the model distillation-based edge device collaborative inference system comprises: The model partitioning module is used to obtain the pre-trained teacher model and divide the teacher model into L consecutive functional sub-layers. The local distillation training module is used to perform local distillation training on the L consecutive functional sub-layers to obtain L sub-student models and extract the sub-layer energy vector corresponding to each sub-student model. The matching degree analysis module is used to collect the device resource vector corresponding to each edge device node in the edge device node set, analyze the sub-layer energy vector and the device resource vector based on the matching degree function, and redundantly deploy each sub-student model to multiple edge device nodes with a matching degree greater than a preset threshold to generate a multi-replica redundant deployment distillation model. The sub-layer energy vector and the device resource vector are analyzed based on the matching degree function to obtain computing power matching items, latency matching items, bandwidth matching items, energy consumption matching items, and stability matching items. When the value of each matching item is greater than a preset threshold, the first group of candidate edge device nodes for each sub-student model is obtained. The first group of candidate edge device nodes is then subjected to a weighted calculation of the comprehensive matching value to obtain multiple edge device nodes with a matching degree greater than a preset threshold. The collaborative analysis module is used to extract the task requirement information of the inference task when a reasoning task is received, establish an optimization objective function based on the task requirement information, perform collaborative analysis on the multi-replica redundant deployment distillation model, and obtain L preferred functional sub-layers. The task execution module is used to collaboratively execute the inference task by the preferred teacher model composed of the L preferred functional sub-layers.

Images