Self-adaptive model evaluation and optimization method based on domestic operating system

Abstract

The invention relates to an adaptive model evaluation and optimization method based on a domestic operating system, and belongs to the technical field of deployment and optimization of AI models. According to the method, resource management characteristics of a domestic operating system are deeply integrated, an adaptive model evaluation and optimization framework is provided, and the core contradiction between resource limitation and model efficiency imbalance in a domestic environment is solved.

Description

Technical Field

[0001] This invention belongs to the field of AI model deployment and optimization technology, specifically involving an adaptive model evaluation and optimization method based on a domestic operating system. Background Technology

[0002] Under a domestically developed operating system environment, the deployment and optimization of AI models face numerous technical challenges. Existing technologies mainly suffer from the following problems: 1. Resource constraints and hardware compatibility issues: During the deployment of AI models, the traditional model routing method is unable to perceive the hardware status in real time due to the dynamic changes in hardware resources (such as video memory and GPU utilization) of domestic operating systems, resulting in unreasonable resource allocation and easy to cause video memory overflow or system crash.

[0003] 2. Disconnect between task semantics and hardware state: Existing hybrid expert model (MOE) methods mainly make routing decisions based on task semantic features, lacking the ability to perceive the runtime hardware state. They cannot dynamically adjust the model selection strategy in memory-constrained environments, resulting in an imbalance between model performance and hardware resources.

[0004] 3. Inefficient resource allocation in multi-tasking scenarios: In multi-tasking scenarios, traditional methods often ignore the dynamic changes in hardware resources, which can easily lead to high-complexity tasks consuming excessive resources, affecting the execution efficiency of other concurrent tasks, and thus reducing the overall system performance.

[0005] 4. Insufficient support for domestically produced hardware platforms Existing technologies lack deep integration with the characteristics of domestically produced hardware under domestic operating system environments, making it difficult to fully utilize hardware performance and affecting the reliability and stability of AI model deployment. Summary of the Invention

[0006] (a) Technical problems to be solved The technical problems this invention aims to solve are: 1. Solving the resource conflict problem caused by the traditional MoE architecture neglecting hardware status: especially in memory-constrained environments, preventing memory overflow and system crashes caused by blindly selecting large models. 2. Improving the reliability of AI deployment on domestically produced hardware platforms: ensuring service stability by dynamically adapting to hardware devices with different performance levels. 3. Optimizing resource allocation efficiency in multi-tasking scenarios: avoiding excessive resource consumption by high-complexity tasks that could affect the execution of other concurrent tasks.

[0007] (II) Technical Solution To address the aforementioned technical problems, this invention provides an adaptive model evaluation and optimization method based on a domestically developed operating system, comprising the following steps: Step 1: Calculate semantic complexity: Quantify the difficulty of understanding the input text from two aspects: content diversity and logical incoherence. This is achieved by analyzing text embedding vectors to quantify these two dimensions of understanding, and finally merging them into a comprehensive semantic complexity score. Step 1 is detailed below: S1. Input text and divide it into N 512-word blocks: Chunking rules: Divide the long text into multiple text blocks according to a fixed window. If the block length is less than 512 words, fill it with special characters to the length of the fixed window. S2. Embed the MiniLM model and generate block vectors: MiniLM is a lightweight pre-trained model similar to BERT, and its process for generating block vectors is as follows: 2) Constructing the input representation: Word segmentation: Using a pre-trained word segmenter to convert blocks of text into sequences of subwords; Add special markers: Start marker [CLS]: used to represent the aggregate features of the entire sequence; Separator marker [SEP]: used to separate different sentences; The final input sequence is: [CLS] + sub-word sequence + [SEP]; 2) Embedding layer mapping Input encoding: Convert the segmented sub-words into their corresponding IDs; Vectorization: Each ID is mapped to a 768-dimensional vector, i.e., a word embedding vector, through an embedding matrix; 3) Location coding Absolute position embedding: Generates a 768-dimensional vector for each position, i.e., the position embedding vector; Superimposed input: The word embedding vector is added to the position embedding vector to obtain the final block vector e_i; S3. Semantic diversity: The silhouette coefficient method is used for clustering to assess semantic diversity; specifically, the silhouette coefficient method is used to evaluate the compactness of each point with its own cluster and its separation from other clusters. Semantic diversity scoring formula:

[0008]

[0009] in Indicates the number of clusters; Indicates the first One cluster; Total number of text blocks; Indicates the first The percentage of each cluster; right Normalize the values ​​to fix their range to [0,1]; S4. Semantic incoherence Incoherence is measured by calculating the average similarity between block vectors of all text; the calculation formula is:

[0010] Cosine similarity is defined as:

[0011] Indicates semantic incoherence; The embedding vector is a 768-dimensional vector generated by the MiniLM model that represents the semantics of text chunks. Let cosine similarity be the value, and its range be 1. ; It is the L2 norm, that is, the vector Euclidean length; right Normalize the values ​​to fix their range to [0,1]; S5. Calculate semantic complexity:

[0012] α and β are coefficients; Step 2: Calculate the task type identifier Basic keyword matching: The keyword library consists of predefined task tags and their keyword sets; the matching frequency calculation formula is as follows:

[0013] Implicit Intent Recognition: Generating Task Probability Distributions Using the PanGu-πPro Model: Input: raw text x; Output: Task label distribution p(t|x); Task feature fusion: The vector is concatenated with p(t|x) to form the task feature vector, i.e., the task type identifier:

[0014] Step 3: Calculate the task adaptation score

[0015] Task fit score measures the relationship between the input task and the model. Compatibility level, calculated as follows:

[0016] Elements in the input feature vector include semantic complexity, task type identifier, and text length; Step 4: Calculate the hardware compatibility score

[0017] Hardware compatibility score measurement model The feasibility of deployment under the current hardware environment is shown in the following formula:

[0018] in, : Model The video memory requirements, The target GPU card's currently available video memory; GPU utilization; Suppression coefficient, used to control hardware sensitivity; Step 5: Determine the weighting coefficients and

[0019] Weighting coefficient and A weighting factor for task adaptation and hardware adaptation; Weighting coefficient and The task characteristics are determined based on real-time task features and hardware status. Real-time task features include text length, semantic complexity, and task type identifier. Hardware status includes available video memory and GPU utilization. Step Six: Calculation of Overall Score Obtain the text length L and semantic complexity Target type identifier, available video memory , , , The weights are automatically optimized through the learning module.

[0020] Calculate the overall score Then select the model with the highest score; use the selected model for model inference.

[0021] (III) Beneficial Effects 1. Hardware-state-aware dynamic routing mechanism: By monitoring operating system-level hardware metrics (available video memory C_d, GPU utilization U_d) in real time and combining them with the model's video memory requirement M_i, a hardware adaptability scoring function is constructed, which, together with the task semantic score, determines the expert model's routing.

[0022] 2. Semantic complexity measurement methods: A two-dimensional evaluation model that integrates semantic diversity (clustering distribution based on information entropy) and semantic incoherence (paragraph consistency based on cosine similarity) addresses the one-sidedness of traditional text length evaluation.

[0023] 3. Gated network dual-drive architecture: A weighted fusion mechanism of task adaptability gating (MLP network generates w_task) and hardware adaptability gating (sigmoid function generates w_hard) is used to achieve a balance between task requirements and resource constraints. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0025] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0026] In the context of domestically developed operating systems, AI model deployment faces challenges such as resource constraints, poor hardware compatibility, and difficulties in real-time performance optimization. Existing MOE methods primarily rely on task semantic features for routing decisions, lacking awareness of runtime hardware states. This invention provides an adaptive model evaluation and optimization method based on a domestically developed operating system. By deeply integrating the resource management features of the domestic operating system, an adaptive model evaluation and optimization framework is proposed, resolving the core contradiction between resource constraints and model performance imbalance in the domestic environment.

[0027] (I) Overall Architecture The architecture is divided into two main aspects: adapting to domestic operating systems and domestic hardware. 1. Input feature analyzer: includes features such as semantic complexity, task type identifier, and text length; 2. Multi-expert gating network: Confidence weights are generated for each model in the model library based on input features, and real-time hardware status feedback factors are introduced through the resource interface of a domestic operating system.

[0028] (II) Technical Details 1. Input Feature Analyzer 1.1 Semantic complexity 1.1.1 Core Objectives Semantic complexity quantifies the difficulty of understanding input text from two aspects: content diversity and logical incoherence. These two dimensions of understanding are quantified by analyzing text embedding vectors (i.e., using numerical vectors to represent text semantics) and are ultimately integrated into a comprehensive complexity score.

[0029] If the complexity exceeds a preset threshold, a large model is used for processing; otherwise, a small model is used.

[0030] Process description: S1. Input text and divide it into N 512-word blocks: Chunking rules: Long texts are divided into multiple text blocks by fixed windows (e.g., every 512 words or sub-words). If a block is less than 512 words long, it is padded with special symbols (e.g., [PAD]) to the length of the fixed window. For example, a 2000-word text is divided into 4 blocks (512 words each).

[0031] Objective: To adapt to the maximum input length limit of the embedded model.

[0032] S2. Embed the MiniLM model and generate block vectors: MiniLM is a lightweight pre-trained model similar to BERT, and its process for generating block vectors is as follows: Construct the input representation: Tokenization: Using a pre-trained tokenizer (such as WordPiece) to convert text blocks into sequences of subwords. Example: Input “Theoretical basis of quantum computing” is tokenized as ["quantum", "sub", "calculation", "of", "theory", "base", "foundation"].

[0033] Add special markers: Start marker [CLS]: used to represent the aggregate features of the entire sequence; Separator marker [SEP]: used to separate different sentences (if the input consists of multiple sentences); The final input sequence is: [CLS] + sub-word sequence + [SEP].

[0034] Embedding layer mapping Input encoding: Convert the segmented words into their corresponding IDs (based on a pre-trained vocabulary), for example: [CLS] → ID 101, quantity → ID 2345, sub → ID 5678, ... Vectorization: Each ID is mapped to a 768-dimensional vector, i.e., a word embedding vector, through an embedding matrix; Location coding Absolute position embedding: Generates a 768-dimensional vector for each position (e.g., position 0, position 1, etc.), i.e., the position embedding vector; Superimposed input: The word embedding vector is added to the position embedding vector to obtain the final block vector e_i.

[0035] S3. Semantic diversity: To measure semantic diversity, it is necessary to know how many "topics" these texts contain. Silhouette coefficient clustering is used to assess semantic diversity.

[0036] Silhouette coefficient method: This method evaluates the density of each point within its own cluster and its separation from other clusters. A higher coefficient indicates better clustering. The K value that maximizes the average silhouette coefficient is directly selected.

[0037] Semantic diversity scoring formula:

[0038]

[0039] Parameter description:

[0040] A higher score is achieved when text blocks are evenly distributed across multiple topics; a lower score is achieved when they are concentrated in only one or two topics. To facilitate comparison between different documents, normalization is performed, fixing the value range to [0,1].

[0041] S4. Semantic incoherence Incoherence is measured by calculating the average similarity between block vectors of all text. The more different the vectors are, the more inconsistent the meaning and the higher the incoherence.

[0042] Calculation formula:

[0043] Cosine similarity is defined as:

[0044] Parameter description:

[0045] Physical meaning: This formula measures the overall incoherence of the text by calculating the average semantic similarity between all pairs of text blocks.

[0046] • Minimum (0): Occurs when the mean cosine similarity of all block vectors is close to 1. This indicates that all text blocks are semantically highly consistent, the text is very coherent, and the themes are focused.

[0047] • Maximum value (2): Occurs when the average cosine similarity of all block vectors approaches -1 (theoretical extreme case). This indicates that the text content consists of a large number of contradictory semantics and is extremely disjointed.

[0048] • Median (≈1): Occurs when the mean cosine similarity of all block vectors approaches 0. This indicates that the text content consists of a large number of unrelated semantic topics and lacks coherence.

[0049] To facilitate comparison between different documents, normalization is performed. The value range is fixed to [0,1].

[0050] S5. Calculate semantic complexity:

[0051] Recommended configuration: Option 1 (Balanced): α = 0.6, β = 0.4 - Applicable scenarios: General text analysis, balancing diversity and incoherence. Option 2 (Diversity Priority): α = 0.8, β = 0.2 Option 3 (Prioritizing Inconsistency): α = 0.4, β = 0.6 1.2 Task Type Identifier 1.2.1 Basic Keyword Matching • Keyword library: Predefined task tags (such as "summary", "translation", "classification") and their keyword sets.

[0052] • Matching frequency calculation:

[0053] Example: Enter: "Please translate the following chapter into English:..."

[0054] Matching the keyword "translation" -> f_keyword=1 / 10=0.1 1.2.2 Implicit Intent Recognition • Lightweight GPT model: Uses domestically developed models (such as PanGu-πPro) to generate task probability distributions.

[0055] Input: Raw text x.

[0056] Output: Task label distribution p(t|x), for example:

[0057] Model structure: 20 Transformer layers, with output layers normalized using Softmax.

[0058] Task feature fusion: Concatenate f_keyword and p(t|x) to form a task feature vector, i.e., a task type identifier.

[0059] 1.3. Text Length Definition: The number of characters or words in the input text.

[0060] Application: Long texts may require a model with stronger computational power.

[0061] 2. Multi-expert gating network (Gating Net) Core objective: Traditional MoE (Mixture of Experts) gated networks select models based solely on input features (such as text semantics), while this design adds hardware state awareness at the level of a domestic operating system (such as video memory and GPU utilization) and dynamic weight correction to achieve multi-objective joint optimization.

[0062] 1. Task adaptability (selecting the model that best matches the input task) 2. Resource utilization (avoid memory overflow or computing power overload) 2.1 Hybrid Gating Function Formula expression:

[0063] in, Representing the selection model weights, It is a model The overall score.

[0064] Indicates the number of optional models 2.1.1 Task Adaptability Score

[0065] - Definition: Measuring the relationship between the input task and the model The degree of matching of abilities.

[0066] - Calculation method:

[0067] - Input feature vector: includes semantic complexity, task type identifier, and text length.

[0068] - MLP architecture: Lightweight multilayer perceptron (such as a 2-layer fully connected network) that outputs scalar scores.

[0069] 2.1.2 Hardware compatibility score

[0070] Definition: Measurement Model Feasibility of deployment in the current hardware environment.

[0071] formula:

[0072] Parameter explanation: : Model Video memory requirements (unit: GB) The target GPU card's currently available video memory (in GB). GPU utilization (0 indicates idle, 1 indicates full load) Suppression coefficient (controls hardware sensitivity, default value 0.5) Physical meaning: when near (Insufficient video memory) or When the load is too high (computing power overload), The lower the value, the more likely the model will be to be selected.

[0073] The larger the size, the stricter the hardware constraints.

[0074] 2.1.3 Weighting coefficients and

[0075] Definition: The weighting balancing factor between task adaptation and hardware adaptation.

[0076] Initial value: , (Priority task matching) Dynamic adjustment: Automatically optimized through the learning module.

[0077] Core idea: to and Instead of being a fixed value or a value adjusted by rules, it is now the output generated by a small neural network (attention mechanism).

[0078] Input: The input to this small network is real-time task characteristics (text length, semantic complexity, task type identifier) ​​and hardware status (available video memory, GPU utilization). Output: Directly output the optimal value. and Finally, the softmax function is added to make... and The sum of is 1.

[0079] Training methods: Offline training: The gating network is trained on the collected dataset, using the final task accuracy and inference efficiency (such as latency) as a joint loss function.

[0080] Dynamic weight calculation Input feature extraction: extracting text length L and semantic complexity And target type identifier.

[0081] Hardware status monitoring: Real-time acquisition of available video memory GPU utilization

[0082] Task fit score calculation: Input feature vector is processed by MLP, output... .

[0083] Hardware compatibility score calculation: Real-time acquisition of the target GPU's... and ,calculate .

[0084] Automatically optimize weights through the learning module: Quick calculations through learning modules and Match the current task characteristics and hardware status.

[0085] make sure :

[0086] Routing decision Calculate the overall score Choose the model with the highest score.

[0087] Use the selected model for model inference.

[0088] Flowchart as follows Figure 1 As shown.

[0089] Compared with the prior art, the present invention has the following beneficial effects: 1. Improve resource utilization and system stability: By monitoring hardware status in real time (such as video memory and GPU utilization) and combining it with task semantic features, the model selection strategy is dynamically adjusted to effectively avoid video memory overflow and system crashes, thereby improving the deployment reliability of AI models under the domestic operating system environment.

[0090] 2. Optimize resource allocation efficiency in multi-task scenarios. The dynamic routing mechanism proposed in this invention can intelligently allocate computing resources according to task complexity and hardware resource status, avoiding excessive resource consumption by high-complexity tasks and improving the overall system performance.

[0091] 3. Enhance the collaborative optimization capability between task semantics and hardware status. By integrating a dual-drive architecture of task adaptability gating and hardware adaptability gating, a balance between task requirements and resource constraints is achieved, significantly improving the accuracy of model selection and resource utilization.

[0092] 4. Enhance the AI ​​deployment capabilities of domestically produced hardware platforms. This invention deeply integrates the resource management features of domestic operating systems, fully explores the performance potential of domestically produced hardware, and provides technical support for the efficient deployment of AI models in a domestically produced environment.

[0093] 5. Reduce the risk of video memory overflow. By using a hardware compatibility scoring function, the dynamic relationship between video memory demand and available resources can be perceived in real time, effectively avoiding system crashes caused by insufficient video memory and improving the operational stability of AI models.

[0094] 6. Enhance the flexibility and adaptability of task processing. The semantic complexity quantification method proposed in this invention can more comprehensively evaluate the processing difficulty of input text. Combined with a dynamic routing mechanism, it significantly improves the flexibility and adaptability of model selection and is applicable to a variety of complex scenarios.

[0095] Through the above innovations, this invention provides a brand-new solution for the efficient deployment and optimization of AI models under a domestic operating system environment, which has important theoretical value and practical application significance.

[0096] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An adaptive model evaluation and optimization method based on a domestically developed operating system, characterized in that, Includes the following steps: Step 1: Calculate semantic complexity: Quantify the difficulty of understanding the input text from two aspects: content diversity and logical incoherence. This is achieved by analyzing text embedding vectors to quantify these two dimensions of understanding, and finally merging them into a comprehensive semantic complexity score. Step 1 is detailed below: S1. Input text and divide it into N 512-word blocks: Chunking rules: Divide the long text into multiple text blocks according to a fixed window. If the block length is less than 512 words, fill it with special characters to the length of the fixed window. S2. Embed the MiniLM model and generate block vectors: MiniLM is a lightweight pre-trained model similar to BERT, and its process for generating block vectors is as follows: 2) Constructing the input representation: Word segmentation: Using a pre-trained word segmenter to convert blocks of text into sequences of subwords; Add special markers: Start marker [CLS]: used to represent the aggregate features of the entire sequence; Separator marker [SEP]: used to separate different sentences; The final input sequence is: [CLS] + sub-word sequence + [SEP]; 2) Embedding layer mapping Input encoding: Convert the segmented sub-words into their corresponding IDs; Vectorization: Each ID is mapped to a 768-dimensional vector, i.e., a word embedding vector, through an embedding matrix; 3) Location coding Absolute position embedding: Generates a 768-dimensional vector for each position, i.e., the position embedding vector; Superimposed input: The word embedding vector is added to the position embedding vector to obtain the final block vector e_i; S3. Semantic diversity: The silhouette coefficient method is used for clustering to assess semantic diversity; specifically, the silhouette coefficient method is used to evaluate the compactness of each point with its own cluster and its separation from other clusters. Semantic diversity scoring formula: in Indicates the number of clusters; Indicates the first One cluster; Total number of text blocks; Indicates the first The percentage of each cluster; right Normalize the values ​​to fix their range to [0,1]; S4. Semantic incoherence Incoherence is measured by calculating the average similarity between block vectors of all text; the calculation formula is: Cosine similarity is defined as: Indicates semantic incoherence; The embedding vector is a 768-dimensional vector generated by the MiniLM model that represents the semantics of text chunks. Let cosine similarity be the value, and its range be 1. ; It is the L2 norm, that is, the vector Euclidean length; right Normalize the values ​​to fix their range to [0,1]; S5. Calculate semantic complexity: α and β are coefficients; Step 2: Calculate the task type identifier Basic keyword matching: The keyword library consists of predefined task tags and their keyword sets; the matching frequency calculation formula is as follows: Implicit Intent Recognition: Generating Task Probability Distributions Using the PanGu-πPro Model: Input: raw text x; Output: Task label distribution p(t|x); Task feature fusion: The vector is concatenated with p(t|x) to form the task feature vector, i.e., the task type identifier: Step 3: Calculate the task adaptation score Task fit score measures the relationship between the input task and the model. Compatibility level, calculated as follows: Elements in the input feature vector include semantic complexity, task type identifier, and text length; Step 4: Calculate the hardware compatibility score Hardware compatibility score measurement model The feasibility of deployment under the current hardware environment is shown in the following formula: in, : Model The video memory requirements, The target GPU card's currently available video memory; GPU utilization; Suppression coefficient, used to control hardware sensitivity; Step 5: Determine the weighting coefficients and Weighting coefficient and A weighting factor for task adaptation and hardware adaptation; Weighting coefficient and The task characteristics are determined based on real-time task features and hardware status. Real-time task features include text length, semantic complexity, and task type identifier. Hardware status includes available video memory and GPU utilization. Step Six: Calculation of Overall Score Obtain the text length L and semantic complexity Target type identifier, available video memory , , , The weights are automatically optimized through the learning module. Calculate the overall score Then select the model with the highest score; use the selected model for model inference.

2. The method as described in claim 1, characterized in that, The minimum value of 0 occurs when the average cosine similarity of all block vectors is close to 1.

3. The method as described in claim 1, characterized in that, The maximum value is 2: This occurs when the average cosine similarity of all block vectors is close to -1.

4. The method as described in claim 1, characterized in that, The median value occurs when the average cosine similarity of all block vectors is close to 0.

5. The method as described in claim 1, characterized in that, when near or When the threshold is exceeded, the selection of this model is suppressed.

6. The method as described in claim 1, characterized in that, The model is an AI model derived from a model library. The model library uses collected datasets and employs task accuracy and inference efficiency as joint loss functions to train a gating network.

7. The method as described in claim 6, characterized in that, The formula for the joint loss function is expressed as follows: in, Representing the selection model weights, It is a model The overall score, Indicates the number of optional models.

8. A system for implementing the method as described in any one of claims 1 to 7.

9. An application of the method as described in any one of claims 1 to 7 in the field of AI model deployment and optimization technology.

10. An application of the system as described in claim 8 in the field of AI model deployment and optimization technology.

Images