A large model adaptive method in heterogeneous cloud-edge scenarios

Abstract

This invention discloses a large model adaptive method in heterogeneous cloud-edge-device scenarios. The method is based on cloud servers, edge servers, and device groups, wherein: the cloud server assigns a personalized backbone architecture to each device group according to the device attributes of each group; the edge server initiates a first NAS stage based on its local shared dataset to generate a coarse-grained header architecture and distributes its architecture data to the device groups connected to it; the device groups refine the header architecture using local data through a second NAS stage, generating an architecture matrix, and upload the architecture matrix back to the edge server; the edge server aggregates all matrices of the group, customizes and distributes the final personalized model architecture to each device; this invention enables the large model to be gradually refined from coarse-grained to match different devices, and utilizes the collaboration between multiple devices to improve the local model architecture.

Description

Technical Field

[0001] This invention pertains to large model adaptive processing technology, and particularly relates to a large model adaptive method applied in heterogeneous cloud-edge scenarios. Background Art

[0002] In recent years, pre-trained large-scale Transformer-based models, such as Vision Transformer (ViT), BERT, and GPT-4, have demonstrated outstanding performance across a variety of tasks, significantly advancing the capabilities of personal intelligent virtual assistants. These models, traditionally deployed in cloud environments, face significant challenges related to data privacy and response latency, particularly in real-time applications where latency can severely impact user experience and operational efficiency. Therefore, deploying these models in cloud-edge-device systems has become a key research area, promising enhanced data privacy and improved system responsiveness through localized processing and reduced data transmission distances.

[0003] However, directly deploying existing manually designed models to devices often results in poor performance because these models are typically incompatible with the limited computing resources, diverse data environments, and stringent energy efficiency requirements of edge environments. To address these mismatches, there is an urgent need for automated design of models specifically optimized for the unique operational needs of devices. This approach promises to improve the efficiency of model deployment in cloud-edge-device systems.

[0004] 1. Lack of efficient collaboration for fine-grained model customization. Existing cloud-edge-device architectures often struggle to provide fine-grained model customization capabilities, hindering the efficient adaptation of models to the unique needs of individual devices. Without innovative collaborative methods that enable progressive and fine-grained model customization and efficient device-specific adaptation in the cloud, achieving optimal performance and resource utilization on heterogeneous devices will become extremely challenging.

[0005] 2. Performance Imbalance Due to Device Heterogeneity. The heterogeneity of devices in terms of capability and energy consumption characteristics can lead to performance imbalances when deploying models on different devices. A model optimized for one type of device may perform poorly or be inefficient on another device with different characteristics. Designing models based on a single performance or attribute fails to account for the diverse constraints imposed by device heterogeneity, potentially hindering the overall effectiveness of the model.

[0006] 3. Poor performance due to heterogeneity of device data. The statistical characteristics of data generated by different devices may vary significantly, requiring different optimal model architectures. If the model fails to consider the unique characteristics of the data distribution for each device, it may not be able to effectively capture the underlying patterns, resulting in poor performance. Summary of the Invention

[0007] To address the challenges of existing technologies, this invention provides a task offloading optimization method for edge computing systems with multimodal large models. This invention can achieve higher model accuracy while significantly reducing model size, and has superior evaluation results compared to existing large model lightweighting methods.

[0008] To address the problems in the existing technology, the present invention adopts the following technical solution (added after the technical solution is determined):

[0009] A large-model adaptive method for heterogeneous cloud-edge scenarios, the method being based on cloud servers, edge servers, and device clusters, including:

[0010] The cloud server assigns a personalized backbone architecture to each device group based on the device attributes of each group;

[0011] The edge server initiates the first NAS phase based on its local shared dataset to generate a coarse-grained header architecture and distributes this architecture data to the group of devices linked to it; where:

[0012] The coarse-grained header architecture establishes a large model architecture that matches device attributes using the following formula:

[0013] P1:

[0014] in: This represents a generalized public dataset in a cloud server. Indicates by s s The obtained intermediate model;

[0015] The device group generates a fine-grained header architecture using local data through the second NAS stage, and uploads the matrix of the fine-grained header architecture back to the edge server; where:

[0016] The fine-grained header architecture establishes a model architecture that matches device-specific data using the following formula:

[0017] P2:

[0018] in: This refers to each group of devices managed by the edge server;

[0019] The edge server aggregates all matrices of the device group, customizes and distributes personalized model architectures to each device.

[0020] Furthermore, the personalized backbone architecture generation process includes:

[0021] A customized backbone architecture is established based on the performance, energy consumption, and model size of the large model using the following formula:

[0022]

[0023] in: Is the device n using model θ n and local data D n Task-specific loss function; Calculate θ n The number of parameters; H, ξ h and ξ f These represent the number of parameters in all headers, the dimension of the hidden layer, and the dimension of the feedforward layer, respectively; where:

[0024] For a group of devices n spanning k training cycles, the total energy consumption is modeled using the following formula:

[0025]

[0026] Where: P n (·,·)T n (·,·) represent power consumption and average latency per training cycle, respectively;

[0027]

[0028] Where: ΔG n The power consumption increases with each additional Transformer layer, p n It is the number of blocks. This is the GPU power consumption estimated for batch size β, L n and ΔL n These represent the initial delay and the delay added at each layer, respectively.

[0029] Furthermore, the first NAS stage, to generate a coarse-grained header architecture process, includes:

[0030] The search space is generated by predefining different operation blocks;

[0031] The number of coarse-grained Header architectures generated by the operation block is selected according to the following formula;

[0032]

[0033] Where b represents the b-th block, and B represents the total number of blocks. Represents a set of operations;

[0034] Based on the LSTM controller parameters of the edge server Shared parameter ω of the sub-model sGenerate coarse-grained header architectures based on the number of coarse-grained header architectures.

[0035]

[0036] in: From The samples obtained through independent sampling; M represents the number of samples obtained through Monte Carlo sampling.

[0037] Furthermore, the device group generates a fine-grained header architecture process using local data through a second NAS phase, including:

[0038] The parameters of a coarse-grained header architecture are measured based on local data from a group of devices accessible during training.

[0039] The importance set of coarse-grained header architecture is obtained by evaluating the importance of parameters of the coarse-grained header architecture using the following formula;

[0040]

[0041] in: First-order Taylor expansion Nearby approximation Q n，r ;

[0042] Based on the importance set of the coarse-grained header architecture, edge server computing is allocated to the group according to the following formula. Similarity of devices W s ;

[0043]

[0044] in: It is the similarity between device i and device j, while It is the Wasserstein distance, i.e.

[0045]

[0046] in: From A small portion of data randomly sampled from the data. Similarly; It is a pre-trained model in The distribution of features extracted from the above; ∏(·,·) represents the joint distribution set between the two distributions; d(x,y) is the distance between x and y calculated using the L1 norm function;

[0047] The similarity W of the devices is calculated using the following formula. sOptimize to obtain a personalized model architecture;

[0048]

[0049] in: By combining knowledge from other devices, the edge server will Distribute to device group n.

[0050] Beneficial effects

[0051] Compared with traditional technical solutions, the beneficial effects of this invention are:

[0052] • Efficient Cloud-Edge-Device Collaborative Fine-Grained Model Customization Method: This invention provides a cloud-edge-device collaborative architecture that enables progressive fine-grained model customization for each device. This architecture allows for efficient device-specific adaptation, facilitating optimal performance and resource utilization on heterogeneous devices. By leveraging the capabilities of cloud and edge nodes, this invention achieves effective collaboration and customization of individual device models.

[0053] • Attribute-Aware Personalized Model Matching: This invention uses a gradient-based method to personalize the generation of the backbone network, providing a range of options to match device performance and storage requirements. It employs a grid method to find the Pareto front within model size constraints, enabling rapid acquisition of the model structure required by the device after front construction. This enhances the matching degree between the model and device attributes, thereby improving the model's cost-effectiveness.

[0054] • Data-aware personalized two-stage neural architecture search (NAS): This invention performs automatic header generation on different models distributed in the cloud, edge, and terminal, and fine-tunes them on the local device to obtain the structure matrix.

[0055] To address the issue of limited data, this invention also provides a personalized aggregation technique based on differences in data distribution. This technique leverages knowledge from other devices to fine-tune the local structure matrix, thereby obtaining a model structure that matches the local data.

[0056] This invention evaluated FormerTailor on different datasets. The results show that, within model size constraints, FormerTailor can achieve more cost-effective models. Compared to other methods, it achieves at least a 3% performance improvement, and when considering the trade-offs between performance, energy consumption, and size, relevant metrics are improved by nearly 20%, such as... Figure 5 As shown. Attached Figure Description

[0057] Figure 1 Cloud-edge-device system architecture diagram;

[0058] Figure 2Overall flowchart of the method of this invention;

[0059] Figure 3 Search methods and search space graph;

[0060] Figure 4 A diagram illustrating the acquisition of importance sets and model pruning;

[0061] Figure 5 Analysis chart comparing the results of different large models. Detailed Implementation

[0062] The following is in conjunction with the appendix Figure 1 - Appendix Figure 5 The present invention is described as follows:

[0063] I. Overall System Architecture

[0064] like Figure 1 As shown, this invention proposes a hierarchical cloud-edge-device system for generating personalized Transformer-based models. The system is defined as a tuple (C, S, N), where C represents a cloud server, and S = {s1, ..., s...}. S Let {} represent the set of edge servers, and N = {1,...,N} represent the set of devices. The system topology is characterized by a centralized connection from the cloud server to all edge servers, and device partitioning. Where N s It is assigned to the edge server s s A collection of devices. Each edge server... s The system receives models from cloud servers and customizes them for assigned device groups. Device partitioning is based on the similarity of computing and storage capabilities. Considering the resource constraints of mobile battery-powered devices (e.g., drones, robots), this invention focuses on optimizing their performance, storage, and energy consumption. In contrast, cloud servers and edge servers possess sufficient computing and storage resources.

[0065] The devices are located in different geographical locations and handle different tasks; therefore, the data between the devices should be heterogeneous. This invention simulates the heterogeneity of devices and their data: for each device, this invention defines a tuple (G... n C n ,θ n ), where G n Indicates GPU capacity, C n θ represents the maximum storable model volume. n It is a personalized model. and These represent the Header and Backbone components, respectively. The model architecture satisfies... Where |·| represents the model size. This reflects the typical composition of a Transformer-based model, where the backbone is much larger than the task-specific header.

[0066] Without loss of generality, the present invention will refer to the model Defined as the baseline. For device n, the transformation function δ(·) is used relative to... Parameterize its Backbone model, that is, in Indicates the width scaling factor. This indicates the number of Transformer layers. This representation provides a unified framework for analyzing and comparing different model architectures, facilitating consistent evaluation across heterogeneous devices.

[0067] FormerTailor comprises two phases: the first phase is the personalization of the backbone architecture, which involves generating a Pareto front to match the backbone and device attributes; the second phase is the personalization of the header architecture, developing a two-phase NAS that collaborates between edge servers and devices to create a header architecture adapted to local data distribution, such as... Figure 2 As shown.

[0068] Specifically, the collaborative method of the entire framework is as follows: Figure 1 As shown, the cloud server allocates the optimized backbone to each device group N based on the device attributes of each group. s Then, the edge server initiates the first NAS phase based on its local shared dataset to generate coarse-grained header architectures and distributes these architectures to its linked devices. The devices refine these header architectures using local data in a second NAS phase, generating architecture matrices, which they then upload back to the edge server. The edge server aggregates all matrices for the group, customizing and distributing the final personalized model architecture to each device. This framework allows models to be progressively refined from coarse-grained to match different devices and leverages collaboration among multiple devices to refine the local model architecture.

[0069] II. Optimize the energy consumption model

[0070] To optimize custom model generation, this invention quantifies the energy consumption of Transformer-based models on heterogeneous devices. This invention uses the energy consumption of the backbone network to approximate the total energy consumption. For device n spanning k training epochs, the total energy consumption can be modeled as follows:

[0071]

[0072] Where P n (·,·)Tn (·, ·) represent power consumption and average latency per training cycle, respectively. This invention defines...

[0073]

[0074] Where ΔG n The power consumption increases with each additional Transformer layer, p n It is the number of blocks. This is the GPU power consumption estimated for batch size β, L n and ΔL n These represent the initial delay and the delay increment at each layer, respectively. This energy consumption model enables the present invention to accurately optimize the model architecture across heterogeneous devices, balancing performance and energy efficiency.

[0075] III. Problem Modeling

[0076] This invention formalizes the problem of generating customized models for heterogeneous devices into a multi-objective optimization problem. The objective is to minimize a synthesis function that balances model performance, energy consumption, and model size while respecting device-specific storage constraints. Formally, this invention defines the optimization problem as follows:

[0077]

[0078] in: Is the device n using model θ n and local data D n Task-specific loss function. Calculate θ n The number of parameters. H, ξ h and ξ f These represent the number of parameters in all headers, the dimension of the hidden layer, and the dimension of the feedforward layer, respectively.

[0079] To address the complexity of the optimization problem in this invention, it is decomposed into two sub-problems:

[0080] (1) Match the model architecture with device attributes, and (2) match the model architecture with device-specific data. Given This invention primarily focuses on optimizing the backbone network in the first sub-problem. In contrast, the second sub-problem involves adjusting the header to adapt to specific data.

[0081] This decomposition strategy offers several advantages. The optimized backbone network provides effective general representations across diverse datasets without requiring extensive tuning. Reusing the backbone network enhances the model's ability to handle various tasks, improving its generalization capabilities. Header customization captures data-specific high-level semantics and category details, improving performance without significantly increasing computational overhead. By separating the optimization of the backbone network and the header, a balance is achieved between model generalization and task-specific performance while minimizing computational costs. This approach improves the efficiency and flexibility of custom model design, enabling it to effectively adapt to heterogeneous devices and diverse data distributions.

[0082] Phase 1 Problem: Matching Model Architecture with Device Attributes

[0083] The problem of matching model architecture with device attributes is formalized as a multi-objective optimization problem. This formula considers two device attributes, C... n and G n These factors respectively affect model size constraints and energy consumption. The goal is to balance model size, energy consumption, and accuracy within device constraints. Formally, this invention defines the optimization problem P1 as follows:

[0084] P1:

[0085] in This represents a generalized public dataset in a cloud server. Indicates by s s The obtained intermediate model.

[0086] Phase Two Problem: Matching the Model Architecture with Device-Specific Data

[0087] After matching the model architecture with device attributes, each edge group has a backbone architecture that is tailored to the attributes of the devices it manages. This invention customizes the header architecture based on the local data of each device while retaining the group-specific backbone. This method produces a customized model that adapts to both device attributes and local data. The second stage of this invention is formulated as an optimization problem P2:

[0088] P2:

[0089] in This allows for customization of the header while maintaining a shared, optimized backbone architecture within each device group managed by the edge server.

[0090] Figure 2 As shown, the first NAS phase involves generating and customizing a personalized backbone structure.

[0091] Personalized backbone structure generation

[0092] To adapt to devices with different properties, this invention proposes a systematic approach that generates multiple smaller backbone models from a complete backbone network by selectively removing components. This process targets both width (embedding dimension, attention head, multilayer perceptron (MLP) dimension) and depth.

[0093] Redundant elements in the (Transformer layer). Given the interdependence of the two dimensions in this invention's strategy, a two-step approach is employed. First, this invention derives a variable-width architecture. It serves as the final backbone architecture θ B The foundation.

[0094] This invention uses a small dataset To quantify the importance of the Header and neurons in the MLP within the Multi-Head Self-Attention (MSA) module. The output O of Header h... h Importance Indicator I h The definition is as follows:

[0095]

[0096] in: Indicates in the dataset The training loss function of the model containing the Header h, and This represents the loss function after removing the header h. h＝0 This is the model output without a header h. This invention uses a first-order Taylor expansion to approximate it. As shown below:

[0097] where R h＝0 This is a remainder term and can be ignored because it is much smaller than the preceding terms. When Header h is removed, O h＝0 =0.

[0098] Therefore, combining the two formulas above, the present invention yields the following derivation:

[0099]

[0100] The importance of neurons can be obtained using a similar method. Based on these importance indicators, this invention ranks the Header and neurons of each Transformer. Depending on the requirements, this invention discards the lower-ranked components to obtain the desired outcome. This has a dynamically adjustable width. The backbone model.

[0101] To generate a final backbone architecture with dynamic width and dynamic depth, this invention employs a knowledge distillation method, using... As a teacher model, the distillation objective function is as follows:

[0102]

[0103] Where l(·, ·) represents the mean square error function. (Term) and represents the logarithm, embedding, and hidden state of the teacher model, respectively. Similarly, y, E, and H correspond to the corresponding parts of the student model. Coefficients Used to balance the contributions of different components during the distillation process.

[0104] Personalized backbone structure customization

[0105] This invention decomposes each objective function in P1 into multiple subproblems and uses a grid method to find the Pareto front. This invention uses an approximation method to solve P1, specifically using the maximum energy consumption within each device group as a representative indicator:

[0106]

[0107] in and

[0108] This is the loss function of the model under the given dataset. To simplify the representation, this invention allows... express The l-th element.

[0109] This invention introduces a performance window γ p This invention quantifies the acceptable trade-offs between performance and other objectives. The objective space is divided into... There are several intervals, located between the ideal optimal point and the worst-case point, where l = 1 corresponds to the performance index. This partitioning is applied uniformly across all objectives. For each backbone architecture, the present invention calculates the grid coordinates for the l-th objective function as follows:

[0110]

[0111] Where σ > 0 is a small constant used to prevent division by zero. The solution set for the l-th objective within the k-th interval is defined as follows:

[0112]

[0113] For the objective function of each edge server This invention defines For the k-th interval Grid coordinates of all solutions The optimal value. Then the present invention constructs... As in China satisfies The solution set. All The union of these sets forms the Pareto Front Grid (PFG), which approximates the Pareto optimal set in the discrete target space.

[0114] Considering the aforementioned strict storage constraints, this invention proposes an efficient method for Pareto front construction and model selection. This invention defines... Make An upper limit on the model size was effectively set. Then, the invention constructed a truncated PFG by discarding all models exceeding this limit. Within this constrained PFG, the invention first identified the model with the highest performance. Using the grid positions of this high-performance model as the search space, the invention selected the final model. Minimize it to the ideal optimal point The Euclidean distance, i.e.:

[0115]

[0116] in and This represents the set of models that, while meeting storage constraints, are located at the highest-performing model grid position.

[0117] like Figures 3-4 As shown, the search space is generated based on modules.

[0118] Inspired by progressive NAS methods, this invention designs a header architecture for each edge server consisting of predefined operation blocks. The basic modular architecture of the header is represented as a directed acyclic graph (DAG) containing B blocks, i.e. This structure is repeated N times, followed by a pooling layer and a multilayer perceptron (MLP). Each block B in the DAG b Defined as a quintuple in This represents the input tensor. Block B is defined in this invention. b input set The union of all previous blocks, the backbone output, and the output of the penultimate Transformer layer in the backbone. This represents the operation applied to the input. Operation set. It includes a variety of functions commonly used in model headers, including z×z convolutional layers, pooling layers, and downsampling operations. The method for combining operation outputs is specified.

[0119] This invention combines functions By limiting the search space to element-wise addition, the expressive power is effectively reduced while maintaining its capacity. To accommodate size mismatches, 1×1 convolutional layers are inserted when necessary. To leverage the global feature representations captured by the backbone, the [CLS] tag is integrated into the final pooling output of the Header. This integration facilitates synergy between global information from the backbone and local features extracted by the Header. The figure illustrates the Header architecture consisting of U basic modules, where B=4, demonstrating the potential complexity and flexibility of the proposed search space.

[0120] To quantify the search space and demonstrate its ability to adapt to the different header requirements of various backbone architectures, this invention conducts the following analysis. This represents the possible architecture space of the b-th block. The cardinality is given by the following formula: in

[0121] Since U does not affect the search space, the total number of possible architectures for a Header with block B is represented as follows:

[0122]

[0123] Block selection and coarse header generation

[0124] Edge servers use a controller to determine block selection and connectivity strategies for their device groups, ultimately defining a coarse-grained model. To accommodate variable-sized inputs, this invention implements a Long Short-Term Memory (LSTM)-based controller capable of handling sequences of length 4 bytes. (Sequence element B) b Represents one of the four components of a block: and The input to LSTM is a data structure of size 10 ... or The one-hot encoded vector is followed by an embedding lookup. The final hidden state of the LSTM is transformed through a fully connected layer and a sigmoid function to estimate the accuracy of the verification.

[0125] To reduce search complexity, this invention employs a parameter-sharing strategy for sub-models. The search process involves two sets of learnable parameters: the LSTM controller parameters of the edge server. Shared parameter ω of the sub-model s .

[0126] The search alternates between the following two steps:

[0127] 1) Optimize ω using a shared datasets ;

[0128] 2) Update based on the performance of the sampling architecture This alternating optimization continues throughout the search process. In the implementation of this invention, the shared parameter ω... s The weights correspond to the operations within each block, because the method of this invention uses the same layer stack. This parameter sharing mechanism significantly reduces the computational overhead of architecture search by allowing multiple sub-models to reuse the same set of parameters.

[0129] Optimize shared parameter ω in edge servers s During this period, the strategy of the fixed controller of the present invention in This represents the sampled sub-model. This invention uses stochastic gradient descent to minimize the expected loss. Monte Carlo sampling estimates the gradient, i.e.:

[0130]

[0131] in From Samples obtained through independent sampling.

[0132] To optimize the controller parameters of the edge server, this invention fixes ω. s And maximize expected reward This invention uses the REINFORCE algorithm to calculate the policy gradient and introduces a moving average baseline to reduce variance.

[0133] Phase 2-2: The second phase of NAS employs personalized aggregation to generate fine-grained headers suitable for the data.

[0134] Header architecture generated by edge servers It is coarse-grained. This architecture may not be suitable for local data distribution on a single device. To address this limitation, this invention proposes a two-step process:

[0135] 1) Device-specific Header Adaptation: Each device generates a header optimized for its local data distribution; 2) Collaborative Knowledge Integration: Devices leverage collective information to refine their header architecture, improving model performance.

[0136] Architecture quantification based on importance, such as Figure 4 As shown:

[0137] Other studies have emphasized that parameter importance varies with task and data, and eliminating unnecessary parameters can improve model performance. In this step, considering device limitations, this invention chooses a minimum overhead strategy, measuring importance based on gradients accessible during training. As shown in Figure (a), this invention uses the above strategy to quantify the parameter importance of the coarse-grained header of local data on device n, and according to the importance set... Discard redundant parts to adapt the header to local data.

[0138] Device n receives from the edge server The parameters of Header therefore, It can be represented as Assumption The parameters in the invention are independent, and the invention can determine individually which parameters should be removed by considering their importance. From the complete... Initially, this invention gradually reduces... The parameters in the table are quantified by removing the introduced error. Based on the assumption of independent and identically distributed (IID), the introduced error can be quantified with and without parameters. The error is measured by the squared difference of the prediction error.

[0139]

[0140] To avoid calculating the r-th parameter Q n，r The importance of repeated evaluation Next, this invention uses a first-order Taylor expansion in Nearby approximation Q n，r :

[0141]

[0142] Therefore, the present invention obtains the following set of importance:

[0143]

[0144] After a certain number of iterations, this invention uses average importance as a standard to discard a preset number of neurons with low joint importance of their parameters.

[0145] Performance improvement of models using personalized aggregation

[0146] To overcome the limitations of limited data on each device, a personalized aggregation method is designed, allowing each device's header architecture to be refined by incorporating knowledge from other devices. As shown in Figure (b), this invention measures the similarity of data distribution on each device to avoid the negative impact of data bias. Ultimately, each device obtains a customized header that not only adapts to local data but also becomes more generalizable by acquiring knowledge from other devices.

[0147] This invention uses Wasserstein distance to measure the differences in data distribution between different devices, which is advantageous in capturing the geometry of complex distributions. Edge server computation is distributed to groups. The similarity of the devices is analyzed, and a similarity matrix W is generated. s :

[0148]

[0149] in It is the similarity between device i and device j, while yes

[0150] Wasserstein distance, i.e.

[0151]

[0152] From A small portion of data randomly sampled from the data. Same thing. It is a pre-trained model in The distribution of features extracted from the above. Π(·,·) represents the joint distribution set between the two distributions. d(x,y) is the distance between x and y calculated using the L1 norm function.

[0153] To further normalize the similarity matrix W s This invention first converts it into a symmetric matrix, and then normalizes it row by row:

[0154]

[0155] The final importance set of device n is a weighted convex combination of the other sets using a similarity matrix:

[0156]

[0157] By combining knowledge from other devices, the edge server will Distribute the header to device n to adjust the header architecture. In this way, each device can have a customized header that not only matches its local data but also incorporates knowledge of data from other devices.

[0158] Although the present invention has been described above, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many modifications under the guidance of the present invention without departing from the spirit of the present invention, and these modifications are all within the protection scope of the present invention.

Claims

1. A large-model adaptive method in heterogeneous cloud-edge scenarios, the method being based on cloud servers, edge servers, and device clusters, characterized in that... include: The cloud server assigns a personalized backbone architecture to each device group based on the device attributes of each group; wherein: the personalized backbone architecture generation process is as follows: A customized backbone architecture is established based on the performance, energy consumption, and model size of the large model using the following formula: ； in: Is device n using model and local data Task-specific loss function; calculate The number of parameters; H, and These represent the number of parameters in all headers, the dimension of the hidden layer, and the dimension of the feedforward layer, respectively; where: For device n spanning k training cycles, the total energy consumption is modeled using the following formula: ； in: These represent power consumption and average latency per training cycle, respectively. ； in: The power consumption increases with each additional Transformer layer. It is the number of blocks. This is the GPU power consumption estimated for batch size β. and These represent the initial delay and the delay increment for each layer, respectively. The edge server initiates the first NAS phase based on its local shared dataset to generate a coarse-grained header architecture and distributes this architecture data to the group of devices linked to it; where: The coarse-grained header architecture establishes a large model architecture that matches device attributes using the following formula: ； in: , ; This represents a generalized public dataset in a cloud server. Indicated by The obtained intermediate model; The device group generates a fine-grained header architecture using local data through the second NAS stage, and uploads the matrix of the fine-grained header architecture back to the edge server; where: The fine-grained header architecture establishes a model architecture that matches device-specific data using the following formula: ； in: This indicates that within each device group managed by the edge server, a shared and optimized backbone architecture is maintained, with a customized header; a tuple (C, S, N) is defined, where C represents the cloud server, and S = { ,..., } represents a set of edge servers. Represents a collection of devices; a partition of devices. ,in It is assigned to the edge server. A set of devices; for each device, define a tuple ( , , ),in Indicates GPU capacity. This represents the maximum storable model volume. It is a personalized model. and These represent the Header and Backbone components, respectively; Reference Model Defined as the baseline; for device n, a transformation function is used. Compared to Parameterize its Backbone model, that is, ,in Indicates the width scaling factor. Indicates the number of Transformer layers; The edge server aggregates all matrices of the device group, customizes and distributes personalized model architectures to each device.

2. The large model adaptive method in heterogeneous cloud-edge scenarios according to claim 1, characterized in that, The first NAS stage, a process for generating a coarse-grained header architecture, includes: The search space is generated by predefining different operation blocks; The number of coarse-grained Header architectures generated by the operation block is selected according to the following formula; ； Where b represents the b-th block, and B represents the total number of blocks. Represents a set of operations; Based on the LSTM controller parameters of the edge server Shared parameters of sub-models Generate coarse-grained header architectures based on the number of coarse-grained header architectures. ； in: From The samples obtained through independent sampling; M represents the number of samples obtained through Monte Carlo sampling.

3. The large model adaptive method in heterogeneous cloud-edge scenarios according to claim 1, characterized in that... The device group generates a fine-grained header architecture using local data through a second NAS phase, including: The parameters of a coarse-grained header architecture are measured based on local data from a group of devices accessible during training. ; The importance set of coarse-grained header architecture is obtained by evaluating the importance of parameters of the coarse-grained header architecture using the following formula; ； in: First-order Taylor expansion Nearby approximation ; Based on the importance set of the coarse-grained header architecture, edge server computing is allocated to the group according to the following formula. Similarity of Chinese devices ; ； in: It is the similarity between device i and device j, while It is the Wasserstein distance, i.e. ； in: From A small portion of data randomly sampled from the data. Similarly; It is a pre-trained model in The distribution of features extracted from the above; Represents the set of joint distributions between two distributions; d(x, y) is the distance between x and y calculated using the L1 norm function; The similarity of devices is determined by the following formula. Optimize to obtain a personalized model architecture; ； in: By combining knowledge from other devices, the edge server will Distribute to device n.

Images