Control method and device of edge AI model, edge equipment and storage medium

Abstract

The invention relates to the field of model deployment, and discloses an edge AI model control method and device, edge equipment and a storage medium, and the method comprises the steps: obtaining network state parameters and hardware performance parameters of the edge equipment in real time, judging whether the network state parameters and the hardware performance parameters are matched with a currently deployed AI model or not, and triggering a model switching condition when the network state parameters and the hardware performance parameters are not matched with the currently deployed AI model. And selecting a target model matched with the current environment parameters from a model library comprising the reference model and a plurality of lightweight AI models with different compression ratios, and executing seamless switching. The problem that an existing edge deployment scheme cannot adapt to diversified device performance and dynamic network environments due to the fact that a single fixed model is adopted is solved, dynamic balance between the model performance and resource occupation is achieved, and the self-adaptive capacity and user experience of the edge device in a cross-border scene are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model deployment and resource adaptive allocation in edge computing, and particularly relates to an edge AI model control method and device, an edge device and a storage medium. BACKGROUND

[0002] The existing AI model edge deployment scheme usually deploys a single lightweight model on a terminal device. This scheme alleviates the delay and bandwidth pressure of cloud computing to a certain extent, but has obvious limitations in actual application. Since the types and running environments of edge devices held by different users are different, a unified lightweight model cannot take into account the actual use of users, and has poor flexibility. SUMMARY

[0003] Therefore, it is necessary to propose an edge AI model control method to solve the technical problem that the lightweight AI model deployed in the prior art cannot meet the actual use of users.

[0004] In a first aspect, an edge AI model control method is provided, and the method comprises: obtaining network state parameters and hardware performance parameters of an edge device; determining whether the network state parameters and the hardware performance parameters match a current AI model deployed on the edge device, and if not, determining that a model switching condition is met; selecting a target AI model that matches the network state parameters and the hardware performance parameters from an AI model library; wherein the AI model library comprises one uncompressed benchmark AI model and at least two lightweight AI models, and each lightweight AI model has a different compression rate; switching the current AI model to the target AI model.

[0005] In a second aspect, an edge AI model control device is provided, and the device comprises: an obtaining module configured to obtain network state parameters and hardware performance parameters of an edge device; a determining module configured to determine whether the network state parameters and the hardware performance parameters match a current AI model deployed on the edge device, and if not, determine that a model switching condition is met; a matching module configured to select a target AI model that matches the network state parameters and the hardware performance parameters from an AI model library; wherein the AI model library comprises one uncompressed benchmark AI model and at least two lightweight AI models, and each lightweight AI model has a different compression rate; a switching module configured to switch the current AI model to the target AI model.

[0006] In a third aspect, an edge device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the control method of the edge AI model when executing the computer program.

[0007] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, wherein the computer program implements the steps of the control method of the edge AI model when executed by a processor.

[0008] The beneficial effects of the present application are as follows: by real-time sensing of network conditions and hardware states, the most suitable model version for the current environment can be dynamically selected. In the environment of high-quality network and high-performance hardware, the model with higher accuracy is automatically switched to, and in the environment of limited resources, the lightweight version is switched to, so that the model always maintains the best balance between performance and efficiency. By establishing a model library containing model versions with different compression rates, the defect that a single fixed model cannot adapt to diversified hardware configurations is overcome. The computing resources of high-end devices are fully utilized, and low-end devices can also obtain a smooth experience, so that the same system can adapt to edge devices with different performance levels. For cross-border application scenarios with dynamic changes in network conditions, the adaptive switching mechanism of the model can effectively cope with network fluctuations. Through the cooperation of multiple versions of lightweight AI models and complete benchmark models, fine-tuned adjustments can be made according to real-time resource conditions. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0010] In the drawings: Figure 1 An application environment diagram of the control method of the edge AI model in an embodiment; Figure 2 A flowchart of the control method of the edge AI model in an embodiment; Figure 3 A structural block diagram of the control device of the edge AI model in an embodiment; Figure 4 A structural block diagram of the edge device in an embodiment. DETAILED DESCRIPTION

[0011] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall into the scope of the present application.

[0012] The edge AI model control method provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 The AI model is deployed on an edge device (i.e., the client 110), and the server 120 is configured with a matching rule table, which includes a plurality of matching rules defining the parameter value ranges corresponding to AI models of different version numbers. The AI model library of the present application can be configured on the client or the server, and includes an uncompressed benchmark AI model and at least two lightweight AI models of different compression rates. The edge device obtains its network state parameters and hardware performance parameters, and determines whether the network state parameters and the hardware performance parameters match the current AI model deployed on the edge device. If not, it is determined that the model switching condition is met. If yes, a target AI model matching the network state parameters and the hardware performance parameters is selected from the AI model library, which includes an uncompressed benchmark AI model and at least two lightweight AI models, each having a different compression rate. The current AI model is switched to the target AI model. The present application can dynamically select the model version that best matches the current environment by real-time sensing of the network condition and the hardware state. In a high-quality network and high-performance hardware environment, it is automatically switched to a model with higher accuracy, and in a resource-limited environment, it is switched to a lightweight version, so that the model always maintains the best balance between performance and efficiency.

[0013] The client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers. The present application will be described in detail through specific embodiments.

[0014] Please refer to Figure 2 As shown in Figure 2 is a flowchart of the edge AI model control method provided by the embodiments of the present application, which includes the following steps: S1: Obtain the network state parameters and the hardware performance parameters of the edge device.

[0015] The network state parameter represents an index for evaluating the quality state of the data transmission link between the edge device and the communication network, and describes the smoothness, speed and stability of data transmission between the edge device and the network. The network state parameter includes: bandwidth, data transmission rate, communication delay and connection stability. Bandwidth: refers to the maximum theoretical capacity of the network channel for transmitting data per unit time. Data transmission rate represents the amount of effective data successfully transmitted per unit time in the actual communication process. Communication delay represents the time interval required for a data packet to be sent from the edge device to the target server and returned. Connection stability represents the ability of the network link to maintain continuous and reliable communication, which is generally measured by the data packet loss rate or the degree of transmission jitter.

[0016] The hardware performance parameter is a physical quantity for evaluating the availability and load degree of the computing, storage and energy resources possessed by the edge device at the current time, and describes the potential ability and current pressure of the edge device to perform computing tasks, which is a key basis for judging whether the model stable operation of a specific computing load can be supported. The hardware performance parameter includes: computing unit utilization, memory resource state and storage resource state. The computing unit utilization refers to the percentage of the computing capacity of the central processor or the dedicated neural network processor of the device occupied at the current time, reflecting the busy degree of the computing unit. The memory resource state refers to the capacity size of the device running memory currently available for immediate use by the application program, which determines the upper limit of the model volume that can be loaded and the data exchange space at runtime. The storage resource state refers to the remaining space capacity of the device persistent storage medium available for reading and writing, which is related to the storage and reading efficiency of the model library.

[0017] The edge device can periodically or triggered by a specific event, obtain its own network state parameter and hardware performance parameter.

[0018] The edge device can obtain the network state parameter based on the network probe, and the specific process includes: Bandwidth acquisition method: a series of data packet transmission requests are initiated to a target server with a known address and stable response (such as a speed test server or a content distribution network node maintained by a service provider) to implement measurement. Specifically, the edge device calculates the total amount of data successfully received within a preset time window, and divides the total amount of data by the duration of the time window, thereby obtaining the actual available bandwidth of the current link. As another implementation, the device can also call the network connection object interface provided by the operating system to directly read the real-time link rate estimated by the system underlying network stack.

[0019] The method for obtaining the communication delay comprises the following steps: the edge device sends a lightweight network control packet to the same target server, and accurately records the time interval from the moment of sending the packet to the moment of returning the confirmation response. The communication delay value representing the network response speed is finally obtained by continuously performing the above operation multiple times and taking the average value to eliminate the accidental fluctuations of single measurement.

[0020] The method for obtaining the connection stability comprises the following steps: the edge device counts the total number of sent data packets and the number of data packets that are not confirmed by the other party during the continuous data transmission process, and calculates the data packet loss rate by dividing the latter by the former. At the same time, the device calculates the standard deviation between the differences of multiple continuous delay measurement values, and the standard deviation is a quantitative indicator for measuring the degree of network jitter.

[0021] The method for obtaining the hardware performance parameters of the edge device comprises the following steps: The method for obtaining the utilization rate of the calculation unit comprises the following steps: the edge device queries the process scheduler of the operating system to obtain the ratio of the clock cycle number in the idle state to the total clock cycle number of the central processing unit and the dedicated neural network processor in each time unit. The current calculation unit utilization rate can be obtained by subtracting the idle ratio from 1. The data reflects the comprehensive load level of the calculation core.

[0022] The method for obtaining the memory resource state comprises the following steps: the edge device calls the application programming interface of the system memory manager to obtain the total capacity of the system physical memory at the current time, the capacity occupied by the application program, and the capacity in the cache state. The total capacity minus the occupied capacity is the capacity of the current available running memory.

[0023] The method for obtaining the storage resource state comprises the following steps: the edge device sends a query request to the file system to obtain the total space size of the storage partition where the model library is located and the remaining space size that is not allocated. The remaining space size is the storage resource capacity available for model read-write operation.

[0024] S2: determining whether the network state parameters and the hardware performance parameters match the current AI model deployed on the edge device, and if not, determining that the model switching condition is met.

[0025] The edge device internally or in the cloud stores a pre-set matching rule table, which shows the parameter value range (network state parameter value range and hardware performance parameter value range) required for stable operation of AI models of different version numbers. The edge device obtains the version number of the current AI model, queries the corresponding parameter value range in the matching rule table according to the version number, and judges whether the real-time obtained network state parameter and hardware performance parameter are within the parameter value range. If yes, it means that the current network state parameter and hardware performance parameter are matched with the current AI model, and switching is not required. If the obtained network state parameter and hardware performance parameter are not within the parameter value range, it is determined that the model switching condition is met. If the model switching condition is not met, the current AI model continues to run, and the network state parameter and hardware performance parameter of the edge device continue to be monitored.

[0026] It should be noted that, in order to avoid misjudgment due to instantaneous fluctuations of parameters, the edge device can set a state continuous monitoring mechanism, which requires that the mismatch state be maintained for a stable preset size of stable duration before finally confirming that the model switching condition is met. The stable duration can be determined according to actual needs. The setting method of the stable duration includes: The duration is set by analyzing the historical fluctuation data of network and hardware performance parameters in the target deployment scenario. The fluctuation frequency and amplitude of each parameter in the typical operating environment are collected, and the typical time period for the parameter to recover from abnormal fluctuation to normal range is calculated. The stable duration is set to be slightly larger than the typical recovery period, so that most transient jitter can be filtered out. For example, in the mobile device scenario, through the analysis of 4G network fluctuation data, it is found that more than 95% of the instantaneous bandwidth drop can recover within 3 seconds, so the stable duration can be preset to 5 seconds.

[0027] The overhead brought by model switching itself is considered as a factor. Systems with high switching cost should set a longer stable duration to avoid unnecessary frequent switching performance loss. The switching cost includes the time, computing resources and possible temporary service interruption consumed by loading a new model. An evaluation model including switching cost and misjudgment probability is established, and an optimal stable duration that minimizes the overall performance loss is calculated through the model. For example, loading a new model takes about two seconds and occupies a large amount of computing resources, so the stable duration of this operation is extended to 8 to 10 seconds to fully confirm the necessity of switching.

[0028] For example, the smartphone is currently running a visual recognition model that requires a high resource configuration. According to the version number of the model, the matching rule corresponding to the matching rule table is queried, and the stable running condition defined by the matching rule is that the network bandwidth is not less than 5 Mbps, and the available memory is greater than 1.2 GB. At this time, the edge device monitors that the actual network bandwidth is only 2 Mbps, and the available memory is only 800 MB. By comparing the measured values with the benchmark values required by the matching rule, it is found that both key parameters do not meet the basic running requirements, and it is determined that the model switching condition is met.

[0029] S3: Selecting a target AI model matching the network status parameter and the hardware performance parameter in the AI model library; wherein the AI model library includes one uncompressed baseline AI model and at least two lightweight AI models, each lightweight AI model having a different compression rate.

[0030] Wherein the edge device locally stores an AI model library, and the AI model library includes one uncompressed baseline AI model and at least two lightweight AI models with different compression rates. When it is determined in S2 that the model switching condition is met, the corresponding matching rule is queried in the preconfigured matching rule table according to the obtained network status parameter and hardware status parameter, i.e. the obtained network status parameter and hardware status parameter belong to the parameter value range defined by the matching rule, the version number corresponding to the matching rule is obtained, and the AI model corresponding to the version number is selected as the target AI model.

[0031] For example: the AI model library locally stored by the smartphone contains three different versions of speech recognition models, and the matching rules of each speech recognition model are as follows: Version number V1: uncompressed baseline model, network bandwidth ≥ 5 Mbps and memory ≥ 2 GB.

[0032] Version number V2: lightweight AI model with a compression rate of 50%, network bandwidth ≥ 2 Mbps and memory ≥ 1 GB.

[0033] Version number V3: lightweight AI model with a compression rate of 70%, network bandwidth ≥ 1 Mbps and memory ≥ 0.5 GB.

[0034] When it is monitored that the current network bandwidth is 1.5 Mbps and the available memory is 0.8 GB, the two parameters are compared with the matching rule table: the network bandwidth 1.5 Mbps falls in the interval [1, 2] and the available memory 0.8 GB falls in the interval [0.5, 1], according to the definition of the matching rule table, the matching model version number corresponding to the parameter combination is V3. The edge device determines that the lightweight AI model corresponding to the version number V3 is the target AI model for this switching.

[0035] S4: switch the current AI model to the target AI model.

[0036] Wherein, after the switching process is started, the edge device first loads the parameter file of the target AI model into the running memory in the background asynchronously, while keeping the current AI model to continue processing the existing operation task. After the target AI model completes the loading and initialization, the newly received data processing request is directed to the target AI model. Subsequently, on the premise of confirming that the target AI model runs stably, the memory resources occupied by the previous AI model are released. The whole switching process ensures that the device continuously provides data processing services during the model replacement period, and there is no service interruption or response stagnation.

[0037] It should be noted that during the model switching preparation stage, the resource occupancy rate of the edge device is continuously monitored, for example: memory occupancy rate and processor occupancy rate. When any indicator exceeds the preset safety threshold (for example, the safety threshold is 85% of the total resources), it is determined that there is a resource conflict, and the priority processing mechanism needs to be started.

[0038] The current running task is divided into three priorities: critical task: model inference and core business, keep running. Important task: data preprocessing and auxiliary function, moderate degradation. Ordinary task: background update and non-urgent task, priority pause.

[0039] When it is detected that the resource occupancy rate exceeds the threshold, all ordinary tasks are first paused; if the resources are still insufficient, the important tasks are taken downgrading measures, such as reducing the data processing frequency; the released resources are preferentially used for the loading process of the target AI model. After loading is completed, the paused tasks are gradually restored in priority order. In this way, it can be ensured that the model switching operation can still be successfully completed when the resources are tight, while the influence on user experience is minimized.

[0040] For example, a smart phone is using a benchmark model for real-time speech translation, at this time, according to the preset rules, it is decided to switch to a deep lightweight AI model. After the switching instruction is triggered, the phone starts to load the file of the deep lightweight AI model into the memory in the background, while the user's current speech translation service is still normally provided by the benchmark model, and the user does not perceive any lag. When the new model is ready for loading, the next sentence of the user's speech input is automatically routed to the lightweight AI model for processing. After ensuring that the new model continuously successfully processes multiple translation requests, the memory space occupied by the benchmark AI model is automatically cleaned up, thereby completing the whole switching operation.

[0041] In one possible embodiment of the present application, it further comprises: A1: extracting the structural data of the uncompressed benchmark AI model, the structural data comprising: the total number of layers, the type of each layer, the number of attention heads, the dimension of each attention head, and the number of neurons of the fully connected layer.

[0042] Specifically, the edge device can access a model configuration file stored locally or in the cloud, parse the model configuration file to obtain structure data of the benchmark AI model, and the structure data includes a plurality of parameters. The meaning of each parameter is described as follows.

[0043] The total number of layers refers to the total number of computing layers that constitute the benchmark model. This parameter is a core indicator for measuring the depth of the model structure. In a model architecture based on Transformer, each layer usually includes a self-attention calculation sublayer and a feedforward neural network sublayer. The total number of layers directly determines the expression ability and computational complexity of the model. The more the total number is, the stronger the theoretical expression ability of the model is, and the required computing resources and memory overhead also increase accordingly.

[0044] The type of each layer is used to identify the specific computing function performed by different layers in the model. In a complex model architecture, different layers can be designed to assume different roles. Typical layer types include, but are not limited to: self-attention layers that perform intra-sequence correlation calculations, feedforward neural network layers that perform nonlinear feature transformations, and classification layers for outputting final results, etc.

[0045] The number of attention heads specifically refers to the number of parallel computing units set in the self-attention layer of the model. Each attention head independently learns and focuses on different parts of the input information from different representation subspaces. Multiple attention heads work together to enable the model to simultaneously capture multiple types of dependencies in the input data, such as short-range dependencies and long-range dependencies, syntactic relationships and semantic relationships, etc.

[0046] The dimension of each attention head refers to the length of the feature vector allocated to a single attention head. This dimension determines the representation ability of the key, query, and value vectors within each head. Under the premise of a fixed total feature dimension, the number of heads and the dimension of each head are mutually restrictive. The smaller the dimension of each head, the more precise its ability to capture specific patterns, but the total amount of information it can accommodate is also limited.

[0047] The number of neurons in the fully connected layer refers to the number of nodes in the layer that are connected to all outputs of the previous layer. In a fully connected layer, each neuron integrates all input features to perform weighted summation and nonlinear transformation.

[0048] A2: inputting the structure data into a pre-trained large language model to obtain a module list including a plurality of modules.

[0049] Among them, the edge device organizes the structural data extracted in step A1 into a structural description text according to the pre-defined formatting requirements, and inputs it into the large language model deployed in the cloud. Based on the semantic understanding of the model components, the large language model parses and reasons the input structural data, identifies the functional units that can exist independently in the benchmark AI model, and systematically organizes these units into a structured module list. The module list completely covers all the trimmable components of the benchmark AI model, and establishes an independent identity for each component.

[0050] The module in the present application refers to the smallest structural unit in the model that can independently perform a specific computing function. Each module represents a complete computing subpart in the model, has a clear functional boundary and structural independence. The module is the basic constituent element formed after the decomposition of the model structure, and is the smallest granularity unit for importance evaluation and pruning operation.

[0051] In the model based on the Transformer architecture, the module mainly takes the following specific forms: Attention head module: Each attention head in the self-attention layer is regarded as an independent module. This module contains complete query, key, value projection matrix and corresponding attention calculation logic, and can independently complete the attention feature extraction of a specific subspace.

[0052] Feedforward sub-layer module: The feedforward neural network layer is decomposed into finer-grained functional units. Each independent nonlinear transformation branch or specific dimension calculation path can be regarded as a module, and these modules collectively constitute the complete feedforward calculation process.

[0053] Special function module: other computing units with clear independent functions in the model, such as position encoding layer, layer normalization component or specific type of embedding layer, can be identified as independent modules.

[0054] In the present embodiment, the training process of the large language model includes: 1. Training data preparation. The training data comes from the public deep learning model library and code repository, including but not limited to model definition files of TensorFlow, PyTorch, professional data sets containing model structure annotation data (such as ModelNet) and other frameworks. The data collection range covers source code, configuration files and corresponding technical documents of various neural network architectures. Through the parsing and cleaning of these data, a parallel corpus containing model structure description text and corresponding component labels is constructed. The finally formed training data set contains multiple model structure description instances, covering various model types from convolutional neural network to Transformer architecture.

[0055] 2. Model architecture design. A Transformer-based encoder-decoder architecture is adopted as the base framework. The encoder part is responsible for understanding the input model structure description text, consisting of 12 attention layers, each equipped with 8 attention heads, with a hidden dimension of 768. The decoder part is responsible for generating the module list, which is symmetrical in structure to the encoder, interacting with the encoder output through cross-attention mechanisms. The total parameter quantity of the model is about 240,000,000.

[0056] 3. Training task definition. A special structure understanding pre-training task is designed, including: The masked language modeling task randomly masks the key structure description words in the input text, training the model to accurately predict based on the context; the component relationship prediction task requires the model to judge the structural relationship between two components, such as hierarchical inclusion or data flow connection; the module boundary detection task trains the model to accurately identify the start and end boundaries of independent modules in the model.

[0057] 4. Execute the training process. Use the AdamW optimizer for training, with an initial learning rate of 0.00005, using linear warm-up and cosine annealing strategies. The batch size is set to 32 samples, and the training period is 50 rounds. During the training process, the model performance is evaluated on the validation set after each training cycle, and the model parameters with the best performance in structure parsing accuracy and module recognition F1 value are retained.

[0058] 5. Model verification and optimization. A special verification dataset is constructed, containing 1000 unseen model structure descriptions. For example: evaluation indicators include: module recognition accuracy reaches 95.3%; module type classification accuracy reaches 93.7%; module interdependence relationship recognition accuracy reaches 91.2%. Based on the verification results, the model is optimized in a targeted manner, focusing on improving the parsing ability of complex nested structures.

[0059] A3: Calculate the importance score of each module in the module list.

[0060] Specifically, the representative calibration dataset is used to perform inference calculation on the original model, and the running data of each module is collected during the calibration process, based on which the importance score of each module is calculated.

[0061] The calibration dataset refers to a representative subset of data sampled from the original training data distribution of the model, specifically used to evaluate the running characteristics of the internal components of the model. This dataset is not used to adjust model parameters, but as an observation sample to analyze the internal behavior of the model.

[0062] The method for constructing the calibration dataset comprises: randomly sampling 5000 samples from the original training data to form the calibration dataset, and maintaining the proportion of data of each category consistent with the original distribution during sampling. For a speech recognition model, the calibration dataset contains speech samples of different genders, ages, and accents; for an image recognition model, the calibration dataset contains image samples of different lighting conditions, shooting angles, and background environments.

[0063] Further, A3: the calculation of the importance score of each module in the module list comprises: A31: obtaining a calibration dataset.

[0064] Specifically, the calibration dataset is sampled from the original training data of the model, and the sample constitutes various data types and scene characteristics that the model may encounter in actual use. The calibration dataset is specifically used to analyze the behavior characteristics of the internal components of the model, and does not involve updating or adjusting the model parameters.

[0065] A32: calculating the average attention distribution entropy of the module on all sample data in the calibration dataset, and calculating the average gradient amplitude of the module in all training steps.

[0066] Specifically, this step performs forward inference and back propagation calculation on each sample in the calibration dataset, and systematically collects the running data of each module. For the attention distribution entropy, the attention weight distribution generated by each attention head when processing all samples is recorded, and the average value of the information entropy is calculated. For the average gradient amplitude, the gradient vector of each module parameter is recorded during back propagation, and the average value of the L2 norm is calculated.

[0067] For example: in the image recognition model of a smart phone, 384 attention head modules are analyzed. For one of the attention heads, the attention distribution entropy generated when processing 5000 test pictures is calculated, and the average value is 2.1. At the same time, the gradient of the parameters of the attention head is recorded during back propagation, and the average gradient amplitude is calculated as 0.03.

[0068] A33: normalizing the average attention distribution entropy to obtain an attention contribution score.

[0069] Specifically, the average attention distribution entropy values of all modules are normalized to map them to the interval of 0 to 1. Since the attention distribution entropy value is negatively correlated with the importance of the module, it is converted into an attention contribution score through mathematical conversion, so that the score value is positively correlated with the importance of the module.

[0070] A34: normalizing the average gradient amplitude to obtain a gradient contribution score.

[0071] Specifically, the average gradient amplitude of all modules is normalized and linearly mapped to the interval of zero to one. The gradient contribution score directly reflects the sensitivity of each module parameter during the training process, and a higher score indicates that the module has a more significant impact on the model performance.

[0072] A35: The attention contribution score and the gradient contribution score are weighted and averaged to obtain a final importance score.

[0073] Specifically, step A35 generates the final importance score through weighted fusion. The attention contribution score and the gradient contribution score are linearly combined according to a preset weight coefficient. The setting of the weight coefficient reflects the relative importance of the two indicators in importance evaluation, and the sum is kept as 1. Through this weighted fusion method, the final importance score can comprehensively reflect the overall performance of the module during the running period and the training period.

[0074] In this application, the weight distribution of the attention contribution score and the gradient contribution score needs to balance the contribution of these two dimensions to ensure that the final importance score can accurately predict the impact of module pruning on the model performance. This application can use the grid search method to determine the optimal weight combination. Within the weight setting range, all possible weight combinations are generated with a step size of 0.1. For each weight combination, a module pruning experiment is performed on the validation model to evaluate the performance retention rate of the pruned model. Finally, the weight combination that can achieve the highest model performance retention rate is selected as the formal weight coefficient.

[0075] First, a validation set containing multiple benchmark models is prepared, covering different types of neural network architectures. For each benchmark model, different weight combinations are used for module importance evaluation, and module pruning is performed according to the evaluation results. Then the performance of the pruned model on the standard test set is tested, and the average performance retention rate corresponding to each weight combination is recorded. Finally, all experimental results are analyzed comprehensively to determine the general weight coefficient setting.

[0076] After setting the initial weight coefficients, verification tests need to be performed on different types of models. If it is found that the evaluation effect is not ideal on some model types, targeted adjustments can be made according to the model characteristics. For example, for models that focus on attention mechanisms, the weight of the attention contribution score can be appropriately increased; for models that require a large amount of training, the weight of the gradient contribution score can be appropriately increased.

[0077] For example: Set the weight coefficient of the attention contribution score to 0.4 and the weight coefficient of the gradient contribution score to 0.6. Substitute the attention contribution score 0.72 and the gradient contribution score 0.85 obtained in the previous example into the calculation formula to obtain the final importance score of the attention head, which is 0.789.

[0078] The embodiment can generate accurate importance evaluation scores for each module. The scoring method considers the attention concentration ability of the module in actual operation and the parameter sensitivity in the training process, providing a scientific basis for subsequent model pruning and ensuring the best balance between model performance and compression efficiency in the lightweight process.

[0079] A4: descendingly sorting each module according to the importance score.

[0080] Specifically, the edge device first associates all modules in the module list with their corresponding importance scores. The unique identifier of each module and the importance score calculated therefrom form a data pair to form an original data set to be sorted.

[0081] The module is sorted in descending order using the quicksort algorithm. This algorithm is selected based on its high efficiency in the average case, with a time complexity of O(N log N), which can meet the requirements of the edge device for computational efficiency. In specific implementation, the system uses the importance score of the module as the sorting key value and arranges them in descending order.

[0082] The sorting process is completed through a recursive divide-and-conquer strategy. First, a reference value is selected from the data set, and then the data set is divided into three subsets: modules with scores higher than the reference value form the left subset, modules with scores equal to the reference value form the middle subset, and modules with scores lower than the reference value form the right subset. The same operation is recursively performed on the three subsets until all subsets contain only one element. Finally, the results are combined in the order of left subset, middle subset, and right subset to form a complete descending sequence.

[0083] After sorting, a new ordered module list is generated. The list is arranged from high to low in terms of module importance from beginning to end, providing a direct basis for subsequent threshold determination and module pruning. At the same time, the position index of each module in the ordered list is recorded to establish a mapping relationship between the module identifier and the sorting position.

[0084] A5: setting a pruning score threshold according to the target compression rate.

[0085] Specifically, the target parameter quantity is calculated according to the total parameter quantity of the model and the target compression rate, and then the module parameters are accumulated in order of high to low module importance. When the cumulative parameter quantity reaches the importance score of the module corresponding to the target parameter quantity, the pruning score threshold is obtained.

[0086] A6: generating a low-contribution module list according to the modules at the end of the sorted list and with importance scores lower than the pruning score threshold, and generating a lightweight AI model after module pruning based on the low-contribution module list, and adding the lightweight AI model to the AI model library.

[0087] Specifically, traverse the sorted module list in descending order, starting from the end of the list and scanning forward, identify all modules with importance scores below the pruning score threshold. Record the unique identifiers, hierarchical positions, module types, and total parameter amounts of these modules in the low-contribution module list. At the same time, the system calculates the total parameter amount of all modules in the list and verifies whether it meets the target compression rate requirement.

[0088] Perform a structured pruning operation according to the low-contribution module list. For each module to be pruned, remove the corresponding structure definition from the model configuration file and delete the related weight parameters. During the pruning process, the system maintains the integrity of the overall model structure and ensures the correct connection relationship between the remaining modules.

[0089] After completing the module removal, reconstruct the computational graph structure of the model. For attention layers with removed attention heads, the system updates their attention head number configuration; for partially pruned feedforward networks, the system adjusts their neuron number configuration. The reconstruction process ensures that the model can be normally loaded and run.

[0090] Perform knowledge distillation fine-tuning on the pruned model, using the original complete model as the teacher model to guide the training of the pruned student model. After fine-tuning, test the model performance on the validation set to ensure that the accuracy loss is controlled within an acceptable range.

[0091] The verified pruned model is converted into a format suitable for edge device deployment, generating the final lightweight AI model. The system also generates a model compression report, recording the number of pruned modules, parameter reduction ratio, and performance index changes.

[0092] After completing the generation and verification of the lightweight AI model, execute the process of integrating it into the AI model library. First, assign a unique version identifier to the new lightweight AI model, which can reflect its compression rate and target area characteristics. Package the model file and its configuration file into a standardized format and write them to the specified storage path of the AI model library. During storage, update the metadata index of the AI model library synchronously, recording the key attributes of the lightweight model, including the model structure hash value, parameter amount, expected resource demand, and applicable environmental parameter range. At the same time, establish the association between the model and the matching rule table to ensure that the edge device can correctly identify and call the new lightweight AI model.

[0093] In an embodiment of the present application, A5: setting the pruning score threshold according to the target compression rate, comprises: A51: Calculate the target parameter total amount to be reserved according to P_target = P_total * (1 - R), R represents the target compression rate, P_target represents the target parameter total amount, and P_total represents the parameter total amount of the baseline AI model; A52: Start from the first module in the sorted order, accumulate the parameter amount of each module one by one; A53: When the cumulative parameter total amount reaches the target parameter total amount, record the importance score corresponding to the last accumulated module, and set the importance score as the pruning score threshold.

[0094] Specifically, in A51, according to the preset target compression rate, the number of target compression rates can be multiple, and the target parameter total amount to be reserved is calculated by a mathematical formula. The specific calculation formula is P_target = P_total * (1 - R) to calculate the target parameter total amount to be reserved. R represents the target compression rate, P_target represents the target parameter total amount, and P_total represents the parameter total amount of the baseline AI model.

[0095] In A52, the list is sorted in descending order of module importance, and the most important module is processed in order. For each module in the list, read its parameter amount and add it to the running counter. This process continues, and the system monitors the cumulative parameter total amount in real time and compares it with the calculated target parameter total amount.

[0096] In A53, when the cumulative parameter total amount reaches or first exceeds the target parameter total amount, the importance score of the current module is immediately recorded. This score is the pruning score threshold, which will be used as the dividing line between the reserved modules and the pruned modules, and then this threshold is stored in the configuration file for subsequent pruning operations.

[0097] The embodiment can accurately convert the macro compression rate target into an executable pruning standard. This method ensures the scientificity and accuracy of the model pruning process, while achieving the predetermined compression target and maximizing the retention of the core functional modules of the model.

[0098] In one possible embodiment of the present application, it further includes: B1: Obtain language feature data of a target cross-border region; the target cross-border region represents the region where the AI model is deployed.

[0099] Specifically, language feature data refers to a structured data set that can quantitatively describe the key characteristics of a specific language system, and is extracted from the quantitative feature data of the authoritative language resources in the target cross-border region. These data systematically represent the essential properties of the target language from a linguistic perspective, and specifically include the following dimensions of feature information: Morphological features: Descriptive data of word form changes, including affix systems, inflection paradigms, and derivation. For example, the triconsonantal root pattern in Arabic, the agglutinative nature of Turkish, etc.

[0100] Syntactic structure features: Quantitative descriptions of syntactic categories, including word order rules, case marking systems, tense-aspect-mood categories, and syntactic relation markers. For example, the SOV word order in Japanese, the noun case system in German, etc.

[0101] Phonological features: For speech recognition tasks, including phoneme inventory, syllable structure, prosodic patterns, and phonological variation rules.

[0102] Semantic expression features: Covering semantic features such as lexical semantic fields, synonymy and antonymy relationships, idiomatic expressions, and culture-specific concepts.

[0103] Statistical distribution features: Empirical language data such as word frequency distribution, n-gram probability, and sentence length distribution based on large-scale corpus statistics.

[0104] The process of obtaining language feature data includes: Collect raw corpus from authoritative language resources in the target cross-border region, including local mainstream media texts, government documents, literary works, social media content, etc. Ensure the representativeness of the corpus in terms of style, domain, and time span, and build a balanced corpus with a size of over 10 million words.

[0105] A team of linguists proficient in the target language annotates the collected corpus. The annotation content includes but is not limited to: morphological analysis, syntactic structure tree, semantic role labeling, and culture-specific expression marking. The annotation process follows uniform linguistic annotation specifications to ensure consistency and accuracy of the data.

[0106] Automatically extract quantitative features from the annotated corpus using computational linguistics methods. Use morphological analyzers to obtain inflection rules, use syntactic analyzers to extract syntactic structure patterns, and use statistical models to calculate the distribution features of various language units. At the same time, key linguistic rules verified by linguists are retained.

[0107] Verify the extracted language features on an independent test set to evaluate their coverage and representativeness in actual use scenarios of the language. Through interviews and tests with local language users, ensure that the feature data truly reflects the actual use of the language.

[0108] Package the verified language feature data in a standardized data format, including feature type identification, feature weight coefficient, and applicable scenario description metadata, to form a feature data package that can be directly called by the model pruning system.

[0109] B2: Determine the reserved modules in the module list according to the language feature data.

[0110] Specifically, for each module in the module list, its functional feature vector is extracted. By analyzing the activation pattern, parameter distribution characteristics, and cooperation relationship with other modules of the module on the multi-language test set, a multi-dimensional functional feature vector is generated.

[0111] The functional feature vector of each module is calculated with the target language feature data. Cosine similarity algorithm is used to evaluate the matching degree of module function and language feature, and a matching score is generated, with the score range between 0 and 1.

[0112] Set the matching threshold to 0.75, when the module matches any key language feature to or above the threshold, mark the module as a reserved candidate module. This threshold is determined by a large number of cross-language experiments, which can effectively balance the protection strength and compression efficiency.

[0113] Further ablation experiments are conducted on the preliminary screened reserved candidate modules. By temporarily removing the module and observing the performance change of the model on the target language test set, the necessity of the module for processing the language feature is confirmed. For example: modules with performance decline of more than 5% are confirmed as key reserved modules.

[0114] Check the structural dependency relationship between the reserved module and other modules. For support modules with strong dependency relationship with the reserved module, even if their direct matching degree is slightly lower than the threshold, they are also included in the protection range to ensure that the reserved module can function properly.

[0115] When multiple language features require protection of the same module, a priority strategy is adopted. Based on the importance and frequency of use of language features, each feature is assigned a weight, and the protection requirement with the highest comprehensive weight is selected.

[0116] Add protective labels to the finally determined reserved modules, which will be exempted in the subsequent pruning process.

[0117] For example: in the Arabic adaptation scenario, it is found that: The 7th layer 3rd attention head module has a matching degree of 0.82 when processing morphological features; The 11th layer 5th attention head module has a matching degree of 0.79 when processing word order features; The 15th layer feedforward network has a matching degree of 0.85 when processing semantic features; After ablation verification, removing the 7th layer 3rd attention head module causes the Arabic named entity recognition F1 value to drop by 7.2%, confirming its key role. The system finally marks these three modules as reserved modules and records the protection reason as "Arabic morphological processing key module" in the metadata.

[0118] Another example, in the Chinese language adaptation scenario, the language feature data of Chinese as an isolating language is extracted from the language feature library, including: monosyllabic morpheme characteristics, tone disambiguation function (four tone system), Chinese character structure complexity (radical combination rules), and basic word order structure.

[0119] The following key modules are identified: the 5th layer 2nd attention head module: specifically processing tone features, matching degree 0.83, the 8th layer feedforward network module: responsible for Chinese character structure analysis, matching degree 0.79, the 12th layer 4th attention head module: processing word order structure, matching degree 0.81.

[0120] It is found that the 5th layer 2nd attention head module and the 8th layer feedforward network module appear in the low contribution module list at the same time, and their importance scores are 0.42 and 0.38 respectively. Remove these two modules from the low contribution module list, and keep the Chinese processing ability. After removal, the compression rate is adjusted from 65.1% to 64.3%, and the F1 value of the Chinese text classification task is improved from 76% of the general compression scheme to 94%.

[0121] Another example, in the English language adaptation scenario, the language feature data of English as an inflectional language is extracted, including: a rich tense variation system (16 tenses), subject-verb agreement rules, rules for using articles, and complex clause structures. At the same time, load the English inflection dictionary and the phrasal verb database.

[0122] The key modules are identified as follows: the 6th layer 3rd attention head module: processing tense variation, matching degree 0.86, the 9th layer 1st attention head module: responsible for subject-verb agreement detection, matching degree 0.82, the 14th layer feedforward network module: parsing clause structure, matching degree 0.84.

[0123] It is detected that the 6th layer 3rd attention head module and the 14th layer feedforward network module are in the low contribution module list, and their importance scores are 0.45 and 0.41 respectively. Remove these two modules from the low contribution module list. After adjustment, the accuracy of the model in the English grammar error detection task is improved from 78% to 96%, while the compression rate remains at a qualified level of 63.8%.

[0124] The present application significantly improves the adaptability of the model in a specific language scenario while ensuring the compression rate of the model through the reserved module determination mechanism.

[0125] B3: Determine whether the reserved module is in the low contribution module list.

[0126] Specifically, all marked reserved modules are traversed, and the unique identifier of each reserved module is compared with the entries in the low-contribution module list one by one. When the identifiers are found to be completely identical, it is determined that the reserved module is located in the low-contribution module list.

[0127] B4: If yes, the reserved module is removed from the low-contribution module list.

[0128] Specifically, according to the conflict module information recorded in the B3 step, the storage positions of these modules in the low-contribution module list are obtained. Each module is accurately positioned by its unique hierarchical position identifier and module type identifier. The corresponding module record is deleted from the low-contribution module list, and the metadata information of the list is updated, including the total number of modules, the total amount of parameters, and other statistical information of the current list. It is checked whether the removed module has a dependency. If the module is a prerequisite dependency of other reserved modules, the connection configuration of the related modules needs to be adjusted synchronously to ensure the integrity of the model structure. After completing the removal operation, the total amount of parameters of the low-contribution module list is recalculated to verify whether it still meets the compression rate requirement. If the compression rate does not meet the requirement due to the removal operation, the system will additionally select a small number of non-critical modules to supplement to the pruning list in the order of importance score from low to high.

[0129] When it is detected that none of the reserved modules is in the low-contribution module list, the low-contribution module list remains unchanged, and the subsequent model pruning process is directly continued.

[0130] In a possible embodiment of the present application, S2: determining whether the network status parameter and the hardware performance parameter match the current AI model deployed on the edge device, comprises: S21: searching for a corresponding matching rule in a matching rule table configured in the cloud or locally according to the version number of the current AI model; S22: determining a parameter value range defined by the matching rule; S23: determining whether the network status parameter and the hardware performance parameter are located in the parameter value range; S24: if yes, determining that the matching is matched; S25: if no, determining that the matching is not matched.

[0131] Specifically, in S21, according to the version number of the current AI model, a query operation is performed in a preset matching rule table. The matching rule table is stored in a key-value pair structure, where the version number is the primary key and the corresponding matching rule is the data value. When querying, the current model version number is exactly matched with the primary key in the rule table to obtain the operating environment parameter range required by the version model. For example: the current running speech recognition model of the smart phone is V2.1 version, the system retrieves the "V2.1" corresponding configuration record in the local rule table, and successfully obtains the matching rule of the version model: the network bandwidth is required to be ≥2Mbps, the memory is required to be ≥1GB, and the processor utilization is required to be ≤80%.

[0132] In S22, the range definition of each parameter is extracted from the retrieved matching rule. The rule parser decomposes the rule string according to the predetermined format, identifies the parameter type, comparison operator and threshold value, and converts them into a standardized parameter range data structure that can be processed by the system.

[0133] In S23, the real-time collected network state parameters and hardware performance parameters are compared one by one with the parsed parameter ranges. The judgment process adopts the full condition satisfaction principle, that is, all parameters must be within their corresponding specified range to produce a deterministic judgment result.

[0134] Referring to Figure 3 In an embodiment, an edge AI model control device is provided, and the device comprises: An acquisition module 301 is configured to acquire network state parameters and hardware performance parameters of an edge device. A judgment module 302 is configured to judge whether the network state parameters and the hardware performance parameters match a current AI model deployed on the edge device. If not, it is determined that the model switching condition is met. A matching module 303 is configured to select a target AI model that matches the network state parameters and the hardware performance parameters from an AI model library. The AI model library includes one uncompressed reference AI model and at least two lightweight AI models, and each lightweight AI model has a different compression rate. A switching module 304 is configured to switch the current AI model to the target AI model.

[0135] In one possible embodiment of the present application, it further comprises: A pruning module is configured to extract structural data of the uncompressed reference AI model, wherein the structural data includes: the total number of layers, the type of each layer, the number of attention heads, the dimension of each attention head, and the number of neurons in the fully connected layer. The structural data is input into a pre-trained large language model to obtain a module list comprising a plurality of modules. calculate an importance score of each module in the module list; sort each module in descending order according to the importance score; set a pruning score threshold according to a target compression rate; generate a low-contribution module list according to the modules at the end of the sorted list and with an importance score lower than the pruning score threshold, and generate a lightweight AI model after a module pruning process based on the low-contribution module list, and add the lightweight AI model to the AI model library.

[0136] In one possible embodiment of the present application, the calculation of the importance score of each module in the module list comprises: obtain a calibration dataset; calculate the average attention distribution entropy of the module on all sample data in the calibration dataset, and calculate the average gradient amplitude of the module in all training steps; normalize the average attention distribution entropy to obtain an attention contribution score; normalize the average gradient amplitude to obtain a gradient contribution score; weight average the attention contribution score and the gradient contribution score to obtain a final importance score.

[0137] In one possible embodiment of the present application, the setting of the pruning score threshold according to the target compression rate comprises: calculate the target total amount of parameters to be retained according to P_target = P_total * (1 - R), R represents the target compression rate, P_target represents the target total amount of parameters, and P_total represents the total amount of parameters of the baseline AI model; start from the first module in the sorted list, and accumulate the parameter amount of each module one by one; when the cumulative parameter amount reaches the target parameter amount, record the importance score corresponding to the last accumulated module, and set the importance score as the pruning score threshold.

[0138] In one possible embodiment of the present application, it further comprises: a screening module configured to obtain language feature data of a target cross-border region; the target cross-border region represents a region where the AI model is deployed; determine a retained module in the module list according to the language feature data; determine whether the retained module is in the low-contribution module list; if yes, remove the retained module from the low-contribution module list.

[0139] In a possible embodiment of the present application, the determining whether the network status parameter and the hardware performance parameter match the current AI model deployed on the edge device comprises: retrieving a corresponding matching rule from a matching rule table configured in the cloud or locally according to a version number of the current AI model; determining a parameter value range defined by the matching rule; determining whether the network status parameter and the hardware performance parameter are within the parameter value range; if yes, determining that the matching is matched; if no, determining that the matching is not matched.

[0140] In a possible embodiment of the present application, the network status parameter of the edge device is obtained based on a network probe, and the network status parameter comprises bandwidth, data transmission rate, and communication delay; and the hardware performance parameter comprises computing unit utilization, memory resource status, and storage resource status.

[0141] In an embodiment, an edge device, which can be a client, has an internal structure diagram as shown in Figure 4 The edge device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. The processor of the edge device is configured to provide computing and control capabilities. The memory of the edge device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the edge device is configured to communicate with an external server through a network connection. The computer program is executed by the processor to implement the functions or steps of the edge AI model control method on the client side.

[0142] In an embodiment, an edge device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor implements the following steps when executing the computer program: obtaining a network status parameter and a hardware performance parameter of an edge device; determining whether the network status parameter and the hardware performance parameter match a current AI model deployed on the edge device, and if not, determining that a model switching condition is met; if yes, selecting a target AI model that matches the network status parameter and the hardware performance parameter from an AI model library; wherein the AI model library includes one uncompressed reference AI model and at least two lightweight AI models, and each lightweight AI model has a different compression rate; switching the current AI model to the target AI model.

[0143] The present application solves the problem that the single fixed model adopted by the existing edge deployment scheme cannot adapt to diversified device performance and dynamic network environment, realizes dynamic balance of model performance and resource occupation, and significantly improves the adaptive ability and user experience of the edge device in a cross-border scenario.

[0144] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the following steps: obtain network state parameters and hardware performance parameters of an edge device; determine whether the network state parameters and the hardware performance parameters match a current AI model deployed on the edge device, and if not, determine that a model switching condition is met; if yes, select a target AI model that matches the network state parameters and the hardware performance parameters from an AI model library; wherein the AI model library includes one uncompressed baseline AI model and at least two lightweight AI models, and each lightweight AI model has a different compression rate; switch the current AI model to the target AI model.

[0145] The present application solves the problem that the single fixed model adopted by the existing edge deployment scheme cannot adapt to diversified device performance and dynamic network environment, realizes dynamic balance of model performance and resource occupation, and significantly improves the adaptive ability and user experience of the edge device in a cross-border scenario.

[0146] It should be noted that the functions or steps that the computer readable storage medium or the edge device can achieve correspond to the related descriptions of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions.

[0149] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not limit it. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features. Such modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A control method for an edge AI model, characterized in that, The method includes: Obtain network status parameters and hardware performance parameters of edge devices; Determine whether the network status parameters and hardware performance parameters match the current AI model deployed on the edge device; if not, determine that the model switching conditions are met. Select a target AI model from the AI ​​model library that matches the network state parameters and the hardware performance parameters; wherein the AI ​​model library includes an uncompressed baseline AI model and at least two lightweight AI models, each of which has a different compression ratio. Switch the current AI model to the target AI model.

2. The control method for the edge AI model according to claim 1, characterized in that, Before obtaining the network status parameters and hardware performance parameters of the edge device, the method further includes: Extract the structural data of the uncompressed benchmark AI model, which includes: the total number of layers, the type of each layer, the number of attention heads, the dimension of each attention head, and the number of neurons in the fully connected layer; The structured data is input into a pre-trained large language model to obtain a module list that includes multiple modules; Calculate the importance score for each module in the module list; Sort the modules in descending order based on the importance scores; Set the cropping score threshold based on the target compression ratio; A list of low-contribution modules is generated based on the modules that are at the end of the sorted list and whose importance scores are lower than the pruning score threshold. A lightweight AI model is generated after pruning the modules based on the list of low-contribution modules. The lightweight AI model is then added to the AI ​​model library.

3. The control method for the edge AI model according to claim 2, characterized in that, The calculation of the importance score for each module in the module list includes: Obtain the calibration dataset; The module calculates the average attention distribution entropy over all sample data in the calibration dataset, and calculates the average gradient magnitude over all training steps. The attention contribution score is obtained by normalizing the average attention distribution entropy. The average gradient magnitude is normalized to obtain the gradient contribution score; The final importance score is obtained by weighting the attention contribution score and the gradient contribution score.

4. The control method for the edge AI model according to claim 2, characterized in that, The step of setting the cropping score threshold based on the target compression ratio includes: The total number of target parameters to be retained is calculated according to P_target = P_total * (1 - R), where R represents the target compression ratio, P_target represents the total number of target parameters, and P_total represents the total number of parameters of the baseline AI model. Starting with the first module after sorting, increment the number of parameters for each module. When the total number of accumulated parameters reaches the target total number of parameters, the importance score corresponding to the last accumulated module is recorded, and this importance score is set as the pruning score threshold.

5. The control method for the edge AI model according to any one of claims 2-4, characterized in that, Also includes: Acquire language feature data of the target cross-border region; the target cross-border region represents the area where the AI ​​model is deployed. Based on the language feature data, determine the modules to be retained in the module list; Determine whether the reserved module is in the list of low contribution modules; If so, remove the reserved module from the list of low-contribution modules.

6. The control method for the edge AI model according to claim 5, characterized in that, The step of determining whether the network state parameters and the hardware performance parameters match the current AI model deployed on the edge device includes: Retrieve the corresponding matching rule from the matching rule table configured in the cloud or locally based on the current AI model version number; Determine the range of parameter values ​​defined in the matching rule; Determine whether the network status parameters and the hardware performance parameters are within the range of the parameter values; If yes, it is considered a match; If not, it is determined to be a mismatch.

7. The control method for the edge AI model according to claim 6, characterized in that, The network status parameters of the edge device are obtained based on network probes. The network status parameters include bandwidth, data transmission rate, and communication latency. The hardware performance parameters include computing unit utilization, memory resource status, and storage resource status.

8. A control device for an edge AI model, characterized in that, The device includes: The acquisition module is used to acquire network status parameters and hardware performance parameters of edge devices; The judgment module is used to determine whether the network status parameters and the hardware performance parameters match the current AI model deployed on the edge device. If not, it determines that the model switching conditions are met. A matching module is used to select a target AI model from the AI ​​model library that matches the network state parameters and the hardware performance parameters; wherein the AI ​​model library includes an uncompressed baseline AI model and at least two lightweight AI models, each of which has a different compression ratio. The switching module is used to switch the current AI model to the target AI model.

9. An edge device, characterized in that, The edge device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the control method for the edge AI model as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the control method for the edge AI model as described in any one of claims 1 to 7.

Images