Big model inference-based computing power prediction method and system

Abstract

The application relates to a computing power prediction method and system based on large model reasoning. By performing semantic coding, route tendency prediction and semantic mutation detection on a token sequence, a semantic feature vector sequence is generated; a graphics processor memory access, instruction pipeline state and historical expert load are collected to generate a hardware state feature vector; based on the semantic feature vector sequence and the hardware state feature vector, a collaborative feature vector is obtained through time sequence alignment and cross attention fusion, the vector is input into a preset prediction neural network for prediction processing, and a graphics processor utilization prediction value sequence of a preset number of future time steps is output, and a computing power early warning signal is generated based on the graphics processor utilization prediction value sequence. The method effectively improves the accuracy and advance nature of graphics processor computing power fluctuation prediction in the mixed expert architecture large model reasoning scene through the dual-path collaborative prediction mechanism of text semantic mutation perception and hardware pipeline state.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a computing power prediction method and system based on large model inference. Background Technology

[0002] With the large-scale deployment of large language models in cloud-based inference services, accurate scheduling and load prediction of inference computing resources have become crucial for ensuring service level agreements and controlling computing power operating costs. The Mixture of Experts (MoE) architecture, through a routing gating unit, selects a subset of expert subnetworks to participate in computation for each input token. This maintains the expressive power of large parameter models while reducing the actual computational load per inference attempt using sparse activation, and has become one of the mainstream architectures for large language models.

[0003] The hybrid expert architecture selects a subset of expert subnetworks for each input token to participate in computation through a routing gating unit. This significantly reduces the actual computational cost per inference while maintaining the expressive power of large-parameter models. However, the inference computational power consumption of this architecture no longer has a stable linear relationship with the length of the input token, but rather depends directly on the expert activation combination triggered by the semantics of the input text. When the semantic domain of the input text changes, such as from code generation to mathematical reasoning, or from narrative description to logical argumentation, the expert selection mode of the router will change drastically, leading to instantaneous fluctuations in the utilization of the graphics processing unit (GPU). If such fluctuations cannot be predicted in advance, problems such as local overload of computing nodes and imbalance of resource utilization can easily occur, directly affecting the latency stability and service quality of the inference service.

[0004] However, existing methods for predicting computing power for large-scale model inference are mostly based on building time-series prediction models using historical load data. They estimate future computing power demand by extrapolating historical GPU utilization data over time. These methods rely on the assumption of smooth continuity of computing power changes over time, and cannot capture sudden, unexpected changes in computing power caused by semantic mutations. As a result, the prediction results are significantly lagging, leading to poor accuracy in computing power prediction. Summary of the Invention

[0005] Therefore, it is necessary to provide a method and system for predicting large-scale model inference computing power to address the aforementioned technical problems. This aims to improve the prediction accuracy and timeliness of inference computing power fluctuations under a hybrid expert architecture, and enhance the real-time performance and reliability of computing power resource scheduling.

[0006] Firstly, this application provides a computing power prediction method based on large model inference, including: The token sequence of the input text is semantically encoded to obtain the semantic embedding vector of each token in the token sequence; the routing tendency of each token in the hybrid expert model is predicted based on the semantic embedding vector to obtain the routing tendency feature; the semantic mutation between adjacent tokens is detected based on the routing tendency feature to generate a semantic mutation marker sequence. Semantic embedding vectors, routing tendency features, and semantic mutation marker sequences are temporally encoded to generate semantic feature vector sequences; Within a preset acquisition period, the memory access status of the graphics processor, the instruction pipeline execution status, and the load characteristics of the experts activated by the hybrid expert model in the previous time window are acquired to generate a hardware state feature vector. The start and end timestamps of the preset acquisition period are recorded. Among them, when the current inference window is the first inference, the load characteristics of the experts activated by the hybrid expert model in the previous time window are the preset baseline load characteristics. Obtain the physical completion timestamp of each token in the semantic feature vector sequence, and perform temporal alignment on the semantic feature vector sequence based on the physical completion timestamp, start timestamp, and end timestamp; The aligned semantic feature vector sequence is fused with the hardware state feature vector through cross-attention to generate a collaborative feature vector. The collaborative feature vector is predicted by a pre-set predictive neural network to obtain a sequence of predicted values ​​for the graphics processor utilization at a preset number of time steps, and a computing power warning signal is generated based on the sequence of predicted values ​​for the graphics processor utilization.

[0007] In some feasible embodiments, the token sequence of the input text is subjected to contextual semantic encoding, including: Convert the continuous token sequence within the current inference window into an initial embedding vector using an embedding lookup table; Concatenate the initial embedding vectors into an embedding matrix; The embedding matrix is ​​input to a Transformer encoder with 2 to 4 layers for contextual semantic encoding to obtain the semantic embedding vector for each token; Based on semantic embedding vectors, predict the routing tendency of each token among the experts in the hybrid expert model, including: Input the semantic embedding vector of each token into the routing tendency prediction head, and map the semantic embedding vector to a routing tendency logit vector with a dimension equal to the total number of experts in the hybrid expert model through a gated linear transformation, thus obtaining the routing tendency logit vector of each token. Semantic mutations between adjacent tokens are detected based on routing preference features, including: The routing preference logit vector of each token is normalized using Softmax to obtain the routing preference probability distribution; Calculate the Jensen-Shannon divergence between the routing tendency probability distributions of adjacent tokens; compare the Jensen-Shannon divergence with a preset mutation threshold to obtain the comparison result; Generate a binary semantic mutation marker sequence based on the comparison results; Temporal encoding of semantic embedding vectors, routing tendency features, and semantic mutation marker sequences includes: The semantic embedding vector, routing tendency logit vector, and the marker of the corresponding token position in the semantic mutation marker sequence of each token are concatenated to obtain the enhanced feature vector; Arrange the enhanced feature vectors in time sequence to form an enhanced feature matrix; The enhanced feature matrix is ​​input into a bidirectional gated recurrent unit for temporal encoding to obtain the encoding result; The encoded result is mapped through the output projection layer to generate a sequence of semantic feature vectors.

[0008] In some feasible embodiments, the expert's load characteristics include average computation time, average memory access frequency, and activation frequency; within a preset acquisition period, the graphics processor's memory access status, instruction pipeline execution status, and the load characteristics of experts activated by the hybrid expert model in the previous time window are acquired, including: Within a preset acquisition period, the number of row buffer hits and row buffer misses are read from the memory controller of the graphics processor; the memory bandwidth pressure index is calculated based on the number of row buffer hits and row buffer misses; the transaction queue depth is read, and the memory bandwidth pressure index and the transaction queue depth are combined into a register-level feature vector. Within a preset acquisition period, the tensor core instruction queue depth and tensor core idle cycle count are read from the instruction pipeline of the graphics processor; the tensor core utilization rate is calculated based on the tensor core idle cycle count and the tensor core active cycle count within the preset acquisition period; the streaming multiprocessor utilization rate is read, and the tensor core instruction queue depth, tensor core idle cycle count, tensor core utilization rate, and streaming multiprocessor utilization rate are combined into an instruction pipeline level feature vector; Obtain the expert index set activated by the hybrid expert model within the previous time window; for each expert in the expert index set, read the average computation time, average memory access, and activation frequency of each expert; concatenate the average computation time, average memory access, and activation frequency of each expert into an expert-level load feature vector; The register-level feature vector, instruction pipeline-level feature vector, and expert-level load feature vector are normalized and then concatenated to form the original hardware feature vector. The original hardware feature vector is then input into a two-layer multilayer perceptron for feature compression to generate a hardware state feature vector.

[0009] In some feasible embodiments, the semantic feature vector sequence is time-aligned based on the physical completion timestamp, start timestamp, and end timestamp, including: Calculate the total duration of the preset collection period based on the start and end timestamps; For each token, the relative position coefficient of the token within the preset collection period is calculated based on the difference between the token's physical completion timestamp and start timestamp, and the total duration. Calculate the token's time alignment weight based on the relative position coefficient; Multiply the time alignment weight by the semantic feature vector corresponding to the token in the semantic feature vector sequence to obtain the aligned semantic feature vector of the token, and then iterate through each token to generate the aligned semantic feature vector sequence.

[0010] In some feasible embodiments, the aligned semantic feature vector sequence is fused with the hardware state feature vector through cross-attention, including: Arrange the aligned semantic feature vector sequence into a semantic feature matrix; multiply the semantic feature matrix by the first query projection matrix, the first key projection matrix, and the first value projection matrix respectively to obtain the semantic query matrix, the semantic key matrix, and the semantic value matrix. The hardware state feature vector is expanded into a hardware feature matrix with the same length as the semantic feature vector sequence; the hardware feature matrix is ​​multiplied by the second query projection matrix, the second key projection matrix, and the second value projection matrix to obtain the hardware query matrix, the hardware key matrix, and the hardware value matrix. Calculate the first attention weight based on the semantic query matrix and the hardware key matrix; calculate the cross-attention output of the semantic query hardware based on the first attention weight and the hardware value matrix; calculate the second attention weight based on the hardware query matrix and the semantic key matrix; calculate the cross-attention output of the hardware query semantics based on the second attention weight and the semantic value matrix. The cross-attention output of the semantic query hardware and the cross-attention output of the hardware query semantics are respectively subjected to sequence-dimensional average pooling to obtain the first aggregate vector corresponding to the semantic query hardware and the second aggregate vector corresponding to the hardware query semantics; the first aggregate vector and the second aggregate vector are concatenated to obtain the concatenated aggregate vector. Based on the preset gate weight vector, preset gate bias, and spliced ​​aggregation vector, calculate the gate scalar; based on the gate scalar, perform weighted fusion processing on the first aggregation vector and the second aggregation vector to obtain the gated fusion vector; multiply the spliced ​​aggregation vector by the preset residual projection matrix to obtain the residual connection term; add the gated fusion vector and the residual connection term to generate the collaborative feature vector.

[0011] In some feasible embodiments, the predictive neural network is a fully connected network, which includes a first predictive layer, a second predictive layer, and an output layer; the computing power early warning signal includes an overload early warning signal, a resource idle early warning signal, a sudden increase in computing power early warning signal, a sudden decrease in computing power early warning signal, a fluctuation early warning signal, and a main frequency fluctuation early warning signal; The collaborative feature vector is predicted using a pre-defined predictive neural network, including: The collaborative feature vector is input into the prediction neural network, and the network is transformed through the first prediction layer, the second prediction layer and the output layer to output a sequence of predicted values ​​of the graphics processor utilization for a preset number of time steps in the future; wherein, the number of neurons in the first prediction layer is 256, the number of neurons in the second prediction layer is 128, and the number of neurons in the output layer is equal to the preset number in the future. A computing power early warning signal is generated based on the predicted value sequence of graphics processor utilization, including: Peak utilization, valley utilization, fluctuation amplitude, rising slope, falling slope, and main fluctuation frequency are extracted from the predicted value sequence of graphics processor utilization. The peak utilization rate is compared with a preset high load threshold. When the peak utilization rate is greater than the preset high load threshold, an overload warning signal is generated. The valley utilization rate is compared with the preset low load threshold. When the valley utilization rate is less than the preset low load threshold, a resource idle warning signal is generated. The rising edge slope is compared with a preset surge threshold. When the rising edge slope is greater than the preset surge threshold, a computing power surge warning signal is generated. The falling edge slope is compared with a preset sudden drop threshold. When the falling edge slope is less than the preset sudden drop threshold, a sudden drop warning signal for computing power is generated. The fluctuation amplitude is compared with a preset fluctuation stability threshold. When the fluctuation amplitude is greater than the preset fluctuation stability threshold, a fluctuation warning signal is generated. The main fluctuation frequency is compared with a preset high-frequency threshold. When the main fluctuation frequency is greater than the preset high-frequency threshold, a main frequency fluctuation warning signal is generated.

[0012] In some feasible embodiments, the time alignment weight is calculated using the following formula:

[0013]

[0014] in, For the first The physical completion timestamp of each token, measured in microseconds; The timestamp is the start time of the preset collection period, measured in microseconds. This is the end timestamp of the preset data collection period, measured in microseconds. For the first The relative position coefficient of each token within a preset collection period, with a value range of [value range missing]. ; For the first The time alignment weight of each token, with a value range of [value range missing]. .

[0015] Secondly, this application also provides a computing power prediction system based on large model inference, including: The semantic processing module is used to perform contextual semantic encoding on the token sequence of the input text to obtain the semantic embedding vector of each token in the token sequence; predict the routing tendency of each token in the hybrid expert model based on the semantic embedding vector to obtain the routing tendency feature; detect semantic mutations between adjacent tokens based on the routing tendency feature and generate a semantic mutation marker sequence. The encoding module is used to perform temporal encoding on semantic embedding vectors, routing tendency features, and semantic mutation marker sequences to generate a sequence of semantic feature vectors. The hardware awareness module is used to acquire the memory access status, instruction pipeline execution status, and load characteristics of experts activated in the previous time window of the graphics processor within a preset acquisition period, generate a hardware status feature vector, and record the start and end timestamps of the preset acquisition period; wherein, when the current inference window is the first inference, the load characteristics of experts activated in the previous time window of the hybrid expert model are the preset baseline load characteristics. The temporal alignment module is used to obtain the physical completion timestamp of each token in the semantic feature vector sequence, and to perform temporal alignment on the semantic feature vector sequence based on the physical completion timestamp, start timestamp, and end timestamp. The temporal fusion module is used to perform cross-attention fusion of the aligned semantic feature vector sequence and the hardware state feature vector to generate a collaborative feature vector. The computing power prediction module is used to perform predictive processing on the collaborative feature vector through a preset predictive neural network to obtain a preset number of time step graphics processor utilization prediction value sequences, and generate computing power early warning signals based on the graphics processor utilization prediction value sequences.

[0016] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method provided in the first aspect of this application.

[0017] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect of this application.

[0018] The aforementioned computing power prediction method and system based on large-model inference firstly, by performing contextual semantic encoding on the input text and predicting expert routing tendencies, can detect semantically driven computing power fluctuations in advance, overcoming the lag of traditional methods that rely solely on historical load extrapolation. Secondly, by detecting semantic mutations in adjacent tokens and generating mutation marker sequences, it can capture sudden changes in computing power without warning caused by semantic domain switching, compensating for the shortcomings of traditional time series models in predicting mutations. Furthermore, by acquiring memory access, instruction pipeline, and expert load characteristics of the graphics processing unit (GPU) to generate hardware state feature vectors, and by performing cross-attention fusion of semantic features and hardware states, a bidirectional causal relationship between semantically driven expert activation and underlying hardware execution efficiency is effectively established. This overcomes the prediction distortion problem caused by simply splicing features in existing technologies and improves the ability to accurately characterize computing power consumption. Finally, by using time-series alignment and predictive neural networks to output GPU utilization prediction sequences for future time steps, it achieves accurate prediction of computing power fluctuations while ensuring microsecond-level prediction timeliness, providing effective support for real-time overload protection and refined scheduling decisions. Attached Figure Description

[0019] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 The overall flowchart of the computing power prediction method based on large model inference in some embodiments is shown; Figure 2 The following are schematic diagrams illustrating the semantic processing of token sequences in some embodiments; Figure 3 A flowchart illustrating timing coding in some embodiments is shown; Figure 4 A flowchart illustrating the process of generating hardware state feature vectors in some embodiments is shown; Figure 5 The flowcharts illustrating temporal alignment of semantic feature vector sequences in some embodiments are shown. Figure 6 A flowchart illustrating the process of generating collaborative feature vectors in some embodiments is shown; Figure 7 A flowchart illustrating the computing power prediction process is shown in some embodiments; Figure 8 The diagram shows a schematic of the computing power prediction system based on large model inference in some embodiments. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0023] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0024] This application provides a computing power prediction method based on large model inference. This embodiment illustrates the method using a terminal as an example; however, it is understood that the method can also be applied to a server, or a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method is applied to a large language model inference scenario employing a hybrid expert architecture, used to predict the computing power fluctuation trend of the graphics processing unit in advance, providing support for resource scheduling and risk warning of the inference system.

[0025] Figure 1 The overall flowchart of a computing power prediction method based on large model inference in some embodiments is shown. For example... Figure 1 As shown, the method includes the following steps: S1. Perform contextual semantic encoding on the token sequence of the input text to obtain the semantic embedding vector of each token in the token sequence; predict the routing tendency of each token in the hybrid expert model based on the semantic embedding vector to obtain the routing tendency feature; detect semantic mutations between adjacent tokens based on the routing tendency feature to generate a semantic mutation marker sequence.

[0026] Specifically, semantic feature streams representing expert activation tendencies can be extracted from the token sequence of the input text. This serves as the semantic basis for subsequent computing power prediction, addressing the limitations of traditional time-series prediction methods in perceiving the semantic roots of computing power changes and their strong lag in responding to sudden changes in computing power. For example, a fixed-length inference window can be defined, and contextual semantic encoding can be performed on the token sequence within the current window. This involves first converting discrete tokens into continuous vector representations through embedding mapping, and then capturing contextual semantic dependencies through a shallow semantic encoding network to obtain a semantic embedding vector for each token that incorporates contextual information. Contextual modeling can compensate for the deficiencies in the semantic information of a single token, enabling features to accurately reflect the semantic connotation of the token in the current context and providing a reliable foundation for routing tendency prediction.

[0027] Subsequently, routing tendency prediction can be performed based on semantic embedding vectors. The semantic embeddings are mapped to the activation tendency values ​​of all experts corresponding to each token. These values ​​are consistent with the output distribution of the main router of the hybrid expert model, reflecting the probability of the token being distributed to each expert. The hybrid expert model is a conventional large language model built using a hybrid expert sparse activation architecture. Its core includes a routing gating unit and multiple parallel expert subnetworks. After each input token is allocated by the routing gating unit, only a portion of the expert subnetworks are activated to participate in the forward computation. The basic structure and inference principles of this type of model are common knowledge in the field of deep learning. Through semantic embedding mapping, expert activation tendencies can be obtained in advance without running the main model router, achieving proactive computational power prediction and capturing the driving factors of computational power changes at their source.

[0028] The location of semantic mutation can be identified based on the difference in activation tendency of adjacent tokens.

[0029] S2. Perform temporal encoding on the semantic embedding vector, routing tendency features, and semantic mutation marker sequence to generate a semantic feature vector sequence.

[0030] By temporally integrating semantic information, routing tendency information, and mutation information, a unified sequence of semantic feature vectors can be output. This step allows for the early capture of changes in expert activation trends from the input text, providing an early signal of computing power changes without waiting for the main model to actually execute, thus offering a semantic feature foundation for subsequent accurate predictions.

[0031] S3. Within a preset acquisition period, acquire the memory access status of the graphics processor, the instruction pipeline execution status, and the load characteristics of the experts activated by the hybrid expert model in the previous time window, generate a hardware status feature vector, and record the start and end timestamps of the preset acquisition period; wherein, when the current inference window is the first inference, the load characteristics of the experts activated by the hybrid expert model in the previous time window are the preset baseline load characteristics.

[0032] Specifically, data can be collected synchronously in pre-set collection cycles, including three types of data: the memory access status of the graphics processor, the instruction pipeline execution status, and the load characteristics of activated experts within the previous time window. For cold start scenarios with no historical load data for the first inference attempt, a preset baseline load characteristic can be used as the initial value to ensure normal startup. Normalization and fusion compression of the collected multi-source features generate a unified-dimensional hardware state feature vector. The start and end timestamps of the current collection cycle are also recorded for subsequent time-series alignment. The hardware state features obtained through this step comprehensively reflect the current operating efficiency and load pressure of the hardware pipeline, providing a realistic hardware-side constraint basis for computing power prediction.

[0033] S4. Obtain the physical completion timestamp of each token in the semantic feature vector sequence. Based on each physical completion timestamp, start timestamp, and end timestamp, perform temporal alignment on the semantic feature vector sequence to obtain the aligned semantic feature vector sequence.

[0034] Specifically, time alignment processing can be performed first, which involves calculating the temporal correlation between semantic features and hardware status based on the physical completion timestamp of each token and the time boundary of the hardware acquisition cycle, weighting and calibrating the semantic features, eliminating the temporal granularity differences between the two types of features, and ensuring the correspondence of the fused data in the time dimension.

[0035] S5. Perform cross-attention fusion between the aligned semantic feature vector sequence and the hardware state feature vector to generate a collaborative feature vector.

[0036] A cross-attention mechanism can be used to achieve bidirectional interaction and fusion of two types of features, uncovering the intrinsic relationship between semantic activation and hardware efficiency, and ultimately generating a collaborative feature vector. This collaborative feature vector simultaneously contains the expected information of expert activation and the execution efficiency information under the corresponding hardware conditions, which can more accurately map the actual computing power consumption and provide high-quality input features for subsequent predictions.

[0037] S6. The collaborative feature vector is predicted and processed by a preset predictive neural network to obtain a sequence of predicted values ​​of the graphics processor utilization for a preset number of time steps in the future, and a computing power warning signal is generated based on the sequence of predicted values ​​of the graphics processor utilization.

[0038] Specifically, by inputting the collaborative feature vector into the predictive neural network, a sequence of predicted GPU utilization values ​​for a predetermined number of time steps can be output through hierarchical transformation. Since complex deep networks lose their predictive value due to excessive inference latency, a lightweight predictive neural network can be used for this step. Subsequently, because a single utilization warning cannot cover multiple risk scenarios such as overload and drastic fluctuations, and multi-dimensional warnings can provide richer decision-making basis for the scheduling system, multiple fluctuation statistical features can be extracted from the predicted GPU utilization value sequence and compared with corresponding pre-set thresholds to generate corresponding computing power warning signals. This step can simultaneously identify multiple computing power risks such as high load overload and large fluctuations, providing clear trigger signals for scheduling operations such as load reduction, capacity expansion, and batch processing adjustments in the inference system.

[0039] The above method first performs semantic encoding, routing tendency prediction, semantic mutation detection, and temporal encoding on the input token sequence. This captures expert activation tendencies and semantic switching signals from the text side in advance, solving the problems of traditional temporal prediction's inability to predict semantic-driven computing power mutations and its strong prediction lag. Secondly, it collects multi-dimensional hardware state data of the graphics processor and historical expert load features, fusing them to generate a hardware state vector and supplementing the baseline fallback mechanism for the first inference. This addresses the issues of single-dimensional hardware state representation and missing initial running scenario data, comprehensively reflecting the real-time execution efficiency of the hardware pipeline. Furthermore, it aligns semantic features and hardware features temporally based on physical timestamps and achieves deep fusion of dual-path features through cross-attention, solving the problems of simple feature concatenation and lack of bidirectional correlation modeling. The generated collaborative features can simultaneously carry expert activation expectations and hardware execution efficiency information. Finally, it outputs future multi-time-step computing power prediction results and generates multi-dimensional computing power early warning signals through predictive processing, solving the problems of insufficient prediction granularity and single-dimensional early warning, providing a decision-making basis for real-time resource scheduling and overload protection of the inference system.

[0040] In some embodiments, Figure 2 The following are schematic diagrams illustrating the semantic processing of token sequences in some embodiments, such as... Figure 2 As shown, step S1 may include the following steps: S11. Convert the continuous token sequence in the current inference window into an initial embedding vector through an embedding lookup table, concatenate the initial embedding vectors into an embedding matrix, and input the embedding matrix into a Transformer encoder with 2 to 4 layers for contextual semantic encoding to obtain the semantic embedding vector of each token.

[0041] Specifically, for a continuous token sequence within the current inference window, a pre-trained embedding lookup table can be used to map discrete tokens to a continuous vector space. The inference window length can be set to 32 tokens. The embedding lookup table uses a vocabulary and embedding layer weights identical to the target hybrid expert large language model to ensure complete alignment of the semantic mapping space with the main model. Each token corresponds to a fixed-dimensional vector in a row of the lookup table as the initial embedding vector. The initial embedding vectors corresponding to all tokens within the current inference window are concatenated sequentially along the row dimension to generate an embedding matrix with the number of rows equal to the number of tokens in the inference window and the number of columns equal to the embedding dimension. Sine and cosine positional encodings are added to the embedding matrix to inject positional information into the sequence. The embedding matrix with added positional encodings is then input into a shallow Transformer encoder (i.e., 2 to 4 layers) to perform contextual semantic encoding. This encoder can have 3 layers, each using a pre-layer normalization structure, and incorporating a multi-head self-attention module and a feedforward neural network module. The number of heads in the multi-head self-attention module can be set to 4, and the hidden layer dimension of the feedforward neural network is set to twice the embedding dimension. The activation function uses Gaussian error linear units. Each layer's self-attention module and feedforward neural network module are connected to residual connections to ensure the stability of gradient propagation.

[0042] Through the above multi-layer self-attention calculation, the vector representation of each token can fuse the semantic information of all other tokens in the window, fully model the context dependency relationship, and finally output the semantic embedding vector of each token fused with global context information. Furthermore, by adopting a shallow Transformer encoder, the computational overhead and inference latency of the encoding stage can be controlled while ensuring the basic semantic modeling capability.

[0043] S12. Input the semantic embedding vector of each token into the routing tendency prediction head, and map the semantic embedding vector into a routing tendency logit vector with a dimension equal to the total number of experts in the hybrid expert model through a gated linear transformation, thus obtaining the routing tendency logit vector of each token.

[0044] Specifically, the routing preference prediction head can adopt a gated linear unit structure, containing two linear transformation layers and an intermediate gated activation branch. The first linear layer maps the semantic embedding vector to the intermediate dimension, and the gated branch controls the flow of features using a sigmoid activation function. The second linear layer maps the intermediate features to the target dimension. The routing preference prediction head is pre-trained offline. During training, the actual output distribution of the hybrid expert model's main router can be used as the supervision label, and KL divergence can be used as the loss function to optimize the network parameters, ensuring that the output routing preference logit vector is highly consistent with the output distribution of the main router, accurately reflecting the probability of the corresponding token being distributed to each expert. Furthermore, this prediction head is an independent network; during the inference phase, parameters can be directly loaded for forward computation without relying on the forward propagation of the main model router.

[0045] S13. Perform Softmax normalization on the routing tendency logit vector of each token to obtain the routing tendency probability distribution. Calculate the Jensen-Shannon divergence between the routing tendency probability distributions of adjacent tokens. Compare the Jensen-Shannon divergence with a preset mutation threshold to obtain the comparison result. Generate a binary semantic mutation tag sequence based on the comparison result.

[0046] Specifically, to ensure the stability of numerical calculations, the maximum value among all logit values ​​of a single token can be subtracted before calculation, followed by exponentiation and normalization. For example, the formula for Softmax normalization is as follows:

[0047] in, Indicates the first The token corresponds to the first The routing probability of an expert Indicates the first The token corresponds to the first An expert's routing preference is based on logit values. Indicates the first The maximum value among all experts corresponding to each token in the logit value. This represents the total number of experts in the hybrid expert model. This is the index variable for iterating through all experts. The route preference probability distribution is a dimensionless probability vector, with all elements ranging from [0, 1] and the sum of all elements being 1.

[0048] Furthermore, based on the routing preference probability distribution of adjacent tokens, the Jensen-Shannon divergence between adjacent distributions can be calculated to quantify the degree of difference between adjacent tokens in the expert activation distribution. During the calculation, a discrete form of the Kullback-Leibler divergence can be used, and a very small positive value is added to the logarithmic term to avoid zero logarithms, ensuring the numerical stability of the calculation process. For example, the formula for calculating the Jensen-Shannon divergence is as follows:

[0049] in, Indicates the first The probability distribution of routing preferences corresponding to each token. Indicates the first The probability distribution of routing preferences corresponding to each token. Let the mean distribution of two distributions satisfy the following condition: , This represents the Kullback-Leibler divergence.

[0050] The Jensen-Shannon divergence is a dimensionless index, with values ​​ranging from 1 to 10. A larger value indicates a more significant difference between the two route distributions, corresponding to a higher degree of semantic transition between adjacent tokens. The calculated divergence value of each pair of adjacent tokens is compared with a preset mutation threshold. If the divergence value is greater than the preset mutation threshold, a mutation state is marked at the corresponding position of the next token. The first token in the sequence has no preceding comparison object and is marked as non-mutated by default. Finally, a binary semantic mutation marker sequence with the same length as the token sequence is generated, where 0 marks the non-mutated state and 1 marks the mutation state. The preset mutation threshold can be set based on the divergence statistics of historical inference tasks, taking 90% of the historical divergence values ​​as the threshold to adapt to the semantic fluctuation characteristics of different models and different business scenarios.

[0051] In some embodiments, Figure 3 The following are schematic diagrams illustrating the timing coding process in some embodiments, such as... Figure 3 As shown, step S2 specifically includes the following steps: S21. Concatenate the semantic embedding vector, the routing tendency logit vector, and the marker of the corresponding token position in the semantic mutation marker sequence for each token to obtain the enhanced feature vector.

[0052] S22. Arrange the enhanced feature vectors in time sequence to form an enhanced feature matrix.

[0053] Specifically, after constructing the semantic embedding vector, routing preference logit vector, and semantic mutation marker, a concatenation operation can be performed along the feature dimensions for the three types of features corresponding to each token to obtain the enhanced feature vector of a single token. The semantic embedding vector can be set to 512 dimensions, the routing preference logit vector to 8 dimensions (consistent with the total number of experts), and the semantic mutation marker to a 1-dimensional scalar. The concatenated enhanced feature vector of a single token then has a dimension of 521. Arranging the enhanced feature vectors of all tokens within the inference window in temporal order forms an enhanced feature matrix with rows equal to the number of tokens in the window and columns equal to the enhanced feature dimensions, which is used for subsequent temporal coding processing.

[0054] S23. Input the enhanced feature matrix into the bidirectional gated recurrent unit for timing encoding to obtain the encoding result.

[0055] S24. The encoding result is mapped through the output projection layer to generate a sequence of semantic feature vectors.

[0056] Specifically, by inputting the enhanced feature matrix into a bidirectional gated recurrent unit (BRU) for temporal encoding, the forward and backward temporal dependencies in the feature sequence can be mined. The BRU contains two parallel standard gated recurrent units, forward and backward, extracting temporal features along the forward and backward directions of the token sequence, respectively. Each gated recurrent unit has built-in update and reset gates, controlling the retention and forgetting of historical information through a gating mechanism. The hidden layer dimension of this network can be set to 256 dimensions. After concatenating the forward and backward output features along the feature dimensions, the output feature dimension at each time step is 512 dimensions. The BRU effectively captures the overall changing trend and local mutation features of expert activation tendencies within the inference window, preserving long-distance semantic change information. The encoded result output from the BRU is input into the output projection layer, where a linear transformation without activation maps the features to a unified target dimension, ultimately generating a sequence of semantic feature vectors. The target dimension of the output projection layer can be set to 256 dimensions according to the input requirements of the subsequent fusion module to ensure the adaptability of the feature dimensions to the subsequent hardware-side features.

[0057] In some embodiments, the expert’s workload characteristics include average computation time, average memory accesses, and activation frequency.

[0058] In some embodiments, Figure 4 The following are schematic diagrams illustrating the process of generating hardware state feature vectors in some embodiments, such as... Figure 4 As shown, step S3 specifically includes the following steps: S31. Within a preset acquisition period, read the number of row buffer hits and row buffer misses from the memory controller of the graphics processor, calculate the video memory bandwidth pressure index based on the number of row buffer hits and row buffer misses, read the transaction queue depth, and combine the video memory bandwidth pressure index and transaction queue depth into a register-level feature vector.

[0059] Specifically, the preset sampling period can be set to 1 millisecond, matching the minimum sampling granularity of the graphics processor's hardware performance counters. The sampling action is triggered by the performance analysis interface provided by the graphics processor driver layer. Data from all hardware registers is read uniformly at the end of the sampling period to ensure consistent time boundaries for all statistical items within a single period. Within the preset sampling period, the number of row buffer hits and row buffer misses can be read from the graphics processor's memory controller registers. The row buffer is a high-speed cache unit within the memory controller. A row buffer hit indicates that the accessed data already exists in the row buffer, eliminating the need for actual memory read / write operations. A row buffer miss indicates that a memory row activation operation is required, resulting in additional access latency and bandwidth overhead. Based on these two types of counts, a memory bandwidth pressure index can be calculated to quantify the current load pressure on the memory interface. The calculation formula is as follows:

[0060] in, This represents the memory bandwidth pressure index, which is a dimensionless indicator with a value range of [0, 1]. The larger the value, the higher the memory bandwidth pressure. This indicates the number of row buffer misses within the preset collection period, expressed in times. This indicates the number of row buffer hits within the preset collection period, expressed in times.

[0061] Simultaneously, the transaction queue depth of the memory controller can be read. The transaction queue depth is the number of currently pending video memory access requests in the queue, expressed in units. A higher value indicates a higher degree of memory access congestion. Combining the video memory bandwidth pressure index with the transaction queue depth along the feature dimension yields a register-level feature vector.

[0062] S32. Within a preset acquisition period, read the tensor core instruction queue depth and tensor core idle cycle count from the instruction pipeline of the graphics processor. Calculate the tensor core utilization rate based on the tensor core idle cycle count and the tensor core active cycle count within the preset acquisition period. Read the streaming multiprocessor occupancy rate and combine the tensor core instruction queue depth, tensor core idle cycle count, tensor core utilization rate, and streaming multiprocessor occupancy rate into an instruction pipeline-level feature vector.

[0063] Specifically, within the same preset acquisition cycle, the tensor core instruction queue depth and the number of idle cycles of the tensor core can be read from the instruction pipeline register of the graphics processing unit (GPU). The tensor core is a dedicated computational unit in the GPU responsible for matrix multiplication and addition operations. It is the execution carrier of core computational tasks in the large language model inference process. The tensor core instruction queue depth is the number of instructions waiting to be sent to the tensor core for execution, measured in instructions, reflecting the degree of instruction backlog. Based on the number of idle cycles of the tensor core and the total number of clock cycles within the acquisition cycle, the tensor core utilization rate can be calculated to characterize the actual workload level of the tensor core. The calculation formula is as follows:

[0064] in, Tensor core utilization is a dimensionless index with a value range of [value range missing]. ; This indicates the number of idle cycles of the tensor core within the preset acquisition period, expressed in units of tensors. This indicates the total number of clock cycles within the preset acquisition period, expressed in units of cycles.

[0065] Furthermore, the streaming multiprocessor utilization rate can be read. This rate is the ratio of the number of currently active thread bundles to the maximum number of thread bundles supported by the streaming multiprocessor. It is a dimensionless indicator that reflects the saturation level of the GPU's general-purpose computing resources. Combining the tensor core instruction queue depth, tensor core idle cycles, tensor core utilization rate, and streaming multiprocessor utilization rate along the feature dimension yields an instruction pipeline-level feature vector.

[0066] S33. Obtain the expert index set activated by the hybrid expert model in the previous time window. For each expert in the expert index set, read the average computation time, average memory access, and activation frequency of each expert. Concatenate the average computation time, average memory access, and activation frequency of each expert into an expert-level load feature vector.

[0067] Specifically, the system obtains the set of expert indices activated in the previous time window for the hybrid expert model. This set is statistically derived from the actual distribution results of the main model router in the previous inference window and includes the IDs of all experts activated by at least one token. For each expert in the set, the system reads the average computation time, average memory access, and activation frequency from the performance statistics module of the inference engine. The average computation time is the average time taken for a single token to trigger the forward propagation of this expert, in microseconds; the average memory access is the average number of bytes accessed by this expert to access memory weights and intermediate data during the execution of a single token, in bytes; and the activation frequency is the proportion of the number of tokens activated by this expert in the previous inference window to the total number of tokens in the window, a dimensionless indicator. Subsequently, to ensure constant feature dimensions and adapt to the fixed input requirements of subsequent neural networks, the feature order is arranged according to the total number of experts in the hybrid expert model. The three feature bits corresponding to inactive experts are set to zero. Then, the three types of features corresponding to all experts are concatenated sequentially in order to obtain the expert-level load feature vector. In addition, for the cold start scenario of the first inference, the load characteristics of all experts are filled with preset baseline load values. The baseline values ​​can be obtained through pre-statistics of the model's no-load inference to ensure the effectiveness of the feature input in the initial stage.

[0068] S34. After normalizing the register-level feature vector, instruction pipeline-level feature vector, and expert-level load feature vector, they are concatenated into the original hardware feature vector. The original hardware feature vector is then input into a two-layer multilayer perceptron for feature compression to generate a hardware state feature vector.

[0069] Specifically, maximum and minimum value normalization can be performed on the register-level feature vector, instruction pipeline-level feature vector, and expert-level load feature vector, respectively. The maximum and minimum values ​​used for normalization can be obtained from the corresponding feature data of offline statistical historical inference tasks. After processing, the value range of all features can be mapped to the [0, 1] interval, eliminating the impact of differences in feature dimensions on subsequent fusion calculations. The three types of normalized feature vectors are concatenated along the feature dimensions to obtain the original hardware feature vector. The original hardware feature vector is input into a two-layer multilayer perceptron for feature compression and nonlinear transformation. The first layer of the multilayer perceptron is a linear transformation layer with Gaussian error linear unit activation, and the hidden layer dimension can be set to 128 dimensions. The second layer is a non-activated linear projection layer, and the output dimension can be set to 256 dimensions, ultimately outputting a fixed-dimensional hardware state feature vector. Through feature compression by the multilayer perceptron, multi-source heterogeneous hardware statistical features can be mapped to a unified high-dimensional feature space, filtering redundant statistical information and providing standardized input for subsequent cross-modal feature fusion.

[0070] In some embodiments, Figure 5The following are schematic diagrams illustrating the process of temporal alignment of semantic feature vector sequences in some embodiments, such as... Figure 5 As shown, step S4 specifically includes the following steps: S41. Obtain the physical completion timestamp corresponding to each token in the semantic feature vector sequence, and calculate the total duration of the preset collection period based on the start timestamp and end timestamp.

[0071] S42. For each token, calculate the relative position coefficient of the token within the preset collection period based on the difference between the token's physical completion timestamp and the start timestamp, combined with the total duration.

[0072] S43. Calculate the time alignment weight of the token based on the relative position coefficient.

[0073] S44. Multiply the time alignment weight by the semantic feature vector corresponding to the token in the semantic feature vector sequence to obtain the aligned semantic feature vector of the token, and traverse each token to generate the aligned semantic feature vector sequence.

[0074] Specifically, each vector in the semantic feature vector sequence corresponds to an independent token. The physical completion time of each token is determined by the inference progress, and the time interval is not fixed. However, the hardware state features are statistically generated based on a preset acquisition cycle of fixed duration. There is no natural correspondence between the sampling times of these two types of features. Therefore, in order to eliminate the time granularity difference between the semantic feature sequence and the hardware state features and avoid distortion of fused features due to time misalignment, we can first obtain the physical completion timestamp corresponding to each token in the semantic feature vector sequence. This timestamp is acquired through the graphics processor hardware event mechanism at the moment when the corresponding token completes the full-link forward computation. The timing reference and the system clock of the hardware performance counter are kept in the same source calibration, with the unit being microseconds, to avoid offset errors caused by different clock sources. At the same time, we can read the start timestamp and end timestamp corresponding to the current preset acquisition cycle, and calculate the total duration of the preset acquisition cycle by subtracting the end timestamp from the start timestamp. This total duration is used as the time reference for relative position calculation.

[0075] Furthermore, for each token, a time alignment weight can be calculated based on its physical completion timestamp and its relative position within the collection period. The calculation process consists of two steps: relative position coefficient calculation and weight mapping. First, the relative position coefficient of the token within the preset collection period can be calculated using the following formula:

[0076] in, For the first The physical completion timestamp of each token, measured in microseconds; The timestamp is the start time of the preset collection period, measured in microseconds. This is the end timestamp of the preset data collection period, measured in microseconds. For the first The relative position coefficients of each token within a preset collection period are dimensionless.

[0077] When the token's completion time falls exactly at the start of the collection cycle, the relative position coefficient is 0; when it falls at the end, the value is 1; when it is inside the cycle, the value is between 0 and 1; and when the token is completely outside the cycle, the corresponding value is less than 0 or greater than 1.

[0078] Subsequently, the time alignment weight can be obtained by mapping the relative position coefficient through a triangular window function, and the calculation formula is as follows:

[0079] in, For the first The time alignment weight of each token is dimensionless and ranges from 1. ; This is a function to maximize the value, used to truncate the weight of tokens that fall outside the collection period. ; This is the absolute value operator.

[0080] The weight calculation employs a triangular window weighting mechanism. When a token's completion time is at the midpoint of the acquisition cycle, its weight reaches its maximum value of 1, gradually decreasing linearly towards both ends of the cycle. The weight is 0 at the cycle boundaries, and tokens completely outside the cycle retain a weight of 0, not participating in feature fusion for the current cycle. This triangular window weighting mechanism reflects the correlation between token features at different times and the current hardware cycle. Tokens closer to the cycle center contribute more to the current hardware state, conforming to the time-dependent law of computing power transmission. Furthermore, the calculation process only involves basic arithmetic operations, resulting in extremely low computational overhead and no increase in prediction link latency.

[0081] Specifically, after obtaining the time alignment weight for each token, the weight is multiplied element-wise by the corresponding token's semantic feature vector to obtain the aligned semantic feature vector for that token. For tokens with a weight of 0, all elements of the aligned feature vector are set to zero and they do not participate in subsequent feature fusion calculations. This process is repeated for all tokens in the semantic feature vector sequence, performing weighting sequentially to obtain the aligned semantic feature vector sequence. After this time alignment process, the discrete token-level semantic features are mapped to a unified hardware acquisition cycle time axis. The contribution weight of the features corresponds to their time position, effectively solving the problem of temporal misalignment between semantic features and hardware features caused by fluctuations in token generation rate and inference latency jitter. This provides standardized feature input with a consistent time dimension for subsequent cross-attention fusion.

[0082] In some embodiments, the large model inference computing power prediction method S3 provided in this application performs cross-attention fusion between the aligned semantic feature vector sequence and the hardware state feature vector to generate a collaborative feature vector, which may include: In some embodiments, Figure 6 The following are schematic diagrams illustrating the process of generating collaborative feature vectors in some embodiments, such as Figure 6 As shown, step S5 specifically includes the following steps: S51. Arrange the aligned semantic feature vector sequence into a semantic feature matrix. Multiply the semantic feature matrix by the first query projection matrix, the first key projection matrix, and the first value projection matrix to obtain the semantic query matrix, the semantic key matrix, and the semantic value matrix.

[0083] Specifically, the aligned semantic feature vector sequence can be arranged in token time order to form a semantic feature matrix. The number of rows in the matrix is ​​the same as the length of the semantic feature vector sequence, and the number of columns is the same as the dimension of a single semantic feature vector, which can be set to 256 dimensions. The semantic feature matrix is ​​then multiplied by the first query projection matrix, the first key projection matrix, and the first value projection matrix to obtain the semantic query matrix, the semantic key matrix, and the semantic value matrix. These three types of projection matrices are trainable parameter matrices, and their dimensions can be set to 256 rows and 128 columns to map the original semantic features to a unified attention feature space, enabling the semantic features to have a basis for similarity matching with hardware features. The dimensions of the three projected semantic matrices are all the sequence length multiplied by 128 dimensions.

[0084] S52. Expand the hardware state feature vector into a hardware feature matrix with the same length as the semantic feature vector sequence. Multiply the hardware feature matrix by the second query projection matrix, the second key projection matrix, and the second value projection matrix to obtain the hardware query matrix, the hardware key matrix, and the hardware value matrix.

[0085] Specifically, a single hardware state feature vector is copied and expanded along the sequence dimension to generate a hardware feature matrix with the same number of rows as the semantic feature vector sequence length. The feature vector in each row of the matrix is ​​completely identical to the original hardware state feature vector, giving the hardware features a structural basis for positional interaction with the semantic sequence. The dimension of the hardware state feature vector can be set to 256. Matrix multiplication is then performed on the hardware feature matrix with the second query projection matrix, the second key projection matrix, and the second value projection matrix to obtain the hardware query matrix, the hardware key matrix, and the hardware value matrix. The dimension of the second set of projection matrices can all be set to 256 rows and 128 columns, consistent with the output dimension of the semantic side projection matrix, ensuring that attention calculation is completed in the same space for the features on both sides. The dimensions of the three types of hardware matrices after projection are all sequence length multiplied by 128 dimensions.

[0086] S53. Calculate the first attention weight based on the semantic query matrix and the hardware key matrix. Calculate the cross-attention output of the semantic query hardware based on the first attention weight and the hardware value matrix. Calculate the second attention weight based on the hardware query matrix and the semantic key matrix. Calculate the cross-attention output of the hardware query semantics based on the second attention weight and the semantic value matrix.

[0087] Specifically, a scaled dot product attention mechanism can be used. The first attention weight is calculated based on the semantic query matrix and the hardware key matrix. During the calculation, the values ​​are scaled using the square root of the dimension to avoid softmax gradient saturation caused by an excessively large dot product. A weighted summation operation is performed on the first attention weight and the hardware value matrix to obtain the cross-attention output for the semantic query hardware. Each position in this output sequence incorporates hardware state information under the corresponding semantic features, realizing the injection of hardware load constraints into semantic features. Similarly, a second attention weight can be calculated based on the hardware query matrix and the semantic key matrix. A weighted summation operation is then performed on the second attention weight and the semantic value matrix to obtain the cross-attention output for the hardware query semantics. Each position in this output sequence incorporates the semantic change trend corresponding to the hardware state, realizing the injection of semantic load expectations into hardware features. For example, the calculation formula for scaled dot product attention is as follows:

[0088] in, Represents the query matrix. Represents the key matrix, The value matrix is ​​a dimensionless eigenma matrix; The dimension of the key matrix is ​​dimensionless. Softmax is a scaling factor used to numerically scale the dot product result, maintaining the stability of the output value; ) is the normalized exponential function, used to map the weights to a probability distribution that sums to 1; Attention The output feature matrix after attention weighting is dimensionless. Through the above bidirectional cross-attention process, bidirectional information interaction between the two types of features can be achieved, and compared with unidirectional attention, it can more fully explore the intrinsic relationship between semantic activation and hardware efficiency, covering the bidirectional influence relationship between the two types of features.

[0089] S54. Perform average pooling of the sequence dimension on the cross-attention output of the semantic query hardware and the cross-attention output of the hardware query semantics to obtain the first aggregate vector corresponding to the semantic query hardware and the second aggregate vector corresponding to the hardware query semantics; concatenate the first aggregate vector and the second aggregate vector to obtain the concatenated aggregate vector.

[0090] Specifically, a global average pooling operation is performed along the sequence dimension on the cross-attention output of the semantic query hardware. This involves averaging the corresponding elements across all rows in the output matrix to obtain a first aggregated vector with a dimension of 128. This vector aggregates the semantically guided hardware fusion features within the entire inference window. The same sequence-dimensional average pooling operation is then performed on the cross-attention output of the hardware query semantics to obtain a second aggregated vector, also with a dimension of 128. This second vector aggregates the hardware-guided semantic fusion features within the entire inference window. Finally, the first and second aggregated vectors are concatenated along the feature dimension to obtain a concatenated aggregated vector with a dimension of 256. This concatenated vector contains fusion feature information from both directions.

[0091] S55. Based on the preset gate weight vector, preset gate bias, and spliced ​​aggregation vector, calculate the gate scalar; based on the gate scalar, perform weighted fusion processing on the first aggregation vector and the second aggregation vector to obtain the gate fusion vector; multiply the spliced ​​aggregation vector by the preset residual projection matrix to obtain the residual connection term; add the gate fusion vector and the residual connection term to generate the collaborative feature vector.

[0092] Specifically, a gated scalar can be calculated based on a preset gated weight vector, a preset gated bias, and a concatenated aggregation vector. The gated weight vector has the same dimension as the concatenated aggregation vector, which is 256 dimensions. The gated bias is a single-valued scalar. The calculation process involves performing a dot product operation between the concatenated aggregation vector and the gated weight vector, adding the gated bias, and then inputting the result into a Sigmoid activation function. The output is a gated scalar with a value ranging from 0 to 1, used to dynamically adjust the fusion weights of the two types of aggregation vectors. The formula for calculating the gated scalar is as follows:

[0093] in, This represents a gated scalar, dimensionless, with a value range of 1 / 2. ; This represents the preset gate weight vector, which is dimensionless. This represents a concatenated and aggregated vector, which is dimensionless. Indicates the preset gate bias, which is dimensionless; This represents the Sigmoid activation function, which maps input values ​​to the range of 0 to 1.

[0094] A weighted fusion of the first and second aggregated vectors is performed based on a gating scalar. The gating scalar serves as the weight of the first aggregated vector, and the difference between 1 and the gating scalar serves as the weight of the second aggregated vector. The weighted sum yields a gating fusion vector with 128 dimensions. This gating mechanism can adaptively adjust the fusion ratio of the two types of information according to the current input features. When the semantic mutation degree is high, it automatically increases the weight of semantic features; when the hardware load fluctuates greatly, it automatically increases the weight of hardware features, thereby improving the scene adaptability of the fused features.

[0095] Specifically, performing matrix multiplication on the concatenated aggregate vector and the preset residual projection matrix yields residual connect terms. The residual projection matrix is ​​set to 256 rows and 128 columns to map the concatenated aggregate vector to the same feature dimension as the gated fusion vector, ensuring dimensionality matching. Element-wise addition is then performed on the gated fusion vector and the residual connect terms to generate the final collaborative feature vector, which has 128 dimensions. The residual connect structure preserves the fundamental information in the original concatenated features, avoiding information loss during the gated fusion process. It also alleviates the gradient vanishing problem during network training, improving the training stability and inference robustness of the fusion module. The resulting collaborative feature vector integrates expert activation expectation information from the semantic side with actual hardware operating state information, and possesses adaptive feature weight adjustment capabilities, providing reliable input features for subsequent computational power prediction.

[0096] In some embodiments, the predictive neural network adopts a three-layer fully connected network structure. This structure is designed for fixed-dimensional collaborative feature vectors and features short computational links and low inference latency. The time consumption of a single round of inference can be controlled at the microsecond level, avoiding the computational overhead of the prediction process itself from offsetting the time margin of the early warning, thus ensuring the real-time and proactive nature of computing power prediction. The predictive neural network may include a first prediction layer, a second prediction layer, and an output layer; the computing power early warning signals include overload early warning signals, resource idle early warning signals, computing power surge early warning signals, computing power drop early warning signals, fluctuation early warning signals, and main frequency fluctuation early warning signals.

[0097] Figure 7 The following are schematic diagrams illustrating the computing power prediction process in some embodiments, such as Figure 7 As shown, step S6 specifically includes the following steps: S61. Input the collaborative feature vector into the prediction neural network, perform hierarchical transformation through the first prediction layer, the second prediction layer and the output layer, and output a sequence of predicted values ​​of the graphics processor utilization for a preset number of time steps in the future; wherein, the number of neurons in the first prediction layer is 256, the number of neurons in the second prediction layer is 128, and the number of neurons in the output layer is equal to the preset number in the future.

[0098] S62. Extract the peak utilization, valley utilization, fluctuation amplitude, rising edge slope, falling edge slope, and main fluctuation frequency from the graphics processor utilization prediction value sequence.

[0099] S63. Compare the peak utilization rate with the preset high load threshold. When the peak utilization rate is greater than the preset high load threshold, generate an overload warning signal. Compare the valley utilization rate with the preset low load threshold. When the valley utilization rate is less than the preset low load threshold, generate a resource idle warning signal.

[0100] S64. Compare the rising edge slope with a preset surge threshold. When the rising edge slope is greater than the preset surge threshold, generate a surge warning signal for computing power. Compare the falling edge slope with a preset drop threshold. When the falling edge slope is less than the preset drop threshold, generate a drop warning signal for computing power.

[0101] S65. Compare the fluctuation amplitude with the preset fluctuation stability threshold. When the fluctuation amplitude is greater than the preset fluctuation stability threshold, generate a fluctuation warning signal. Compare the main fluctuation frequency with the preset high-frequency threshold. When the main fluctuation frequency is greater than the preset high-frequency threshold, generate a main frequency fluctuation warning signal.

[0102] Specifically, after the collaborative feature vector is input into the prediction neural network, the features undergo hierarchical transformation through the first prediction layer, the second prediction layer, and the output layer. The first prediction layer is a fully connected layer with 256 neurons, followed by a linear rectified activation function. The input dimension is consistent with the dimension of the collaborative feature vector, which can be set to 128 dimensions. Nonlinear transformation is used to upgrade the feature dimension, uncover higher-order coupling relationships between feature components, and strengthen the expression of features strongly correlated with computing power changes. The second prediction layer is also a fully connected layer with 128 neurons, followed by a linear rectified activation function. This layer performs dimensionality reduction and aggregation on the upgraded features, filters redundant noise components, and retains core computing power-related features. The output layer is a linear fully connected layer without an activation function. The number of neurons is equal to the number of future preset time steps, which can be set to 10. It outputs the predicted value of the graphics processor utilization for the next 10 acquisition cycles. Each element in the output predicted value sequence is a dimensionless utilization scalar with a value range of [0, 1], corresponding to 0% to 100% of the actual hardware utilization. Furthermore, the predictive neural network can be optimized using offline supervised training. The training dataset consists of collaborative features collected from historical inference tasks and the corresponding real hardware utilization sequence for each time period. The loss function uses mean squared error loss, which enables iterative updates of network parameters through gradient descent algorithm, ensuring the numerical accuracy of the prediction results.

[0103] After obtaining the predicted sequence of graphics processor (GPU) utilization values, statistical features can be extracted from three dimensions: amplitude, rate of change, and frequency domain, providing a quantitative basis for multi-dimensional early warning. Peak utilization is the maximum value across all time steps in the prediction sequence, representing the highest level of hardware load within the prediction window and used to identify overload risk. Valley utilization is the minimum value across all time steps in the prediction sequence, representing the lowest level of hardware load within the prediction window and used to identify resource idle risk. Fluctuation amplitude is the difference between peak and valley utilization, reflecting the overall fluctuation range of load within the prediction window. The rising edge slope is calculated by traversing all consecutive rising intervals in the prediction sequence. For each consecutive rising interval, the ratio of the difference between the first and last utilization rates to the interval length is calculated, and the maximum value among all ratios is taken as the rising edge slope. This quantifies the severity of a rapid increase in computing power. The calculation formula is as follows:

[0104] in, This represents the rising slope, with the dimension being per time step; This represents the total number of time steps contained in the predicted value sequence, and is dimensionless. Indicates the first The predicted graphics processor utilization at each time step, dimensionless; Indicates the first The predicted graphics processor utilization at each time step, dimensionless; , The time step index is used to satisfy the condition of a monotonically increasing sequence.

[0105] Similarly, when calculating the falling edge slope, all consecutive falling intervals can be traversed, and the ratio of the difference between the first and last utilization rates to the interval length can be calculated. The minimum value among all ratios is taken as the falling edge slope, quantifying the severity of the rapid decline in computing power, with the dimension per time step. The main fluctuation frequency can be obtained by performing a discrete Fourier transform on the predicted value sequence, converting the time-domain utilization sequence into frequency-domain components, and extracting the frequency value corresponding to the frequency component with the largest amplitude as the main fluctuation frequency, used to characterize the speed of the periodic fluctuation of the load, with the dimension of the number of cycles per time step.

[0106] After completing the statistical feature extraction, multi-dimensional threshold comparisons can be performed sequentially to generate corresponding computing power warning signals. For example, peak utilization can be compared with a preset high-load threshold, which can be set according to hardware specifications and business stability requirements. When the peak utilization exceeds the preset high-load threshold, a hardware high-load overload state is predicted within the prediction window, generating an overload warning signal to trigger scheduling operations such as load reduction and task distribution in the inference system. Similarly, valley utilization can be compared with a preset low-load threshold, which can be set according to resource utilization performance indicators. When the valley utilization is less than the preset low-load threshold, a hardware resource idle state is predicted within the prediction window, generating a resource idle warning signal to trigger scheduling operations such as task merging and batch processing scale expansion in the inference system, thereby improving hardware resource utilization efficiency.

[0107] Specifically, the rising edge slope can be compared with a preset surge threshold, which corresponds to the maximum allowable increase in computing power per unit time. When the rising edge slope is greater than the preset surge threshold, a sudden increase in computing power within a short period of time is predicted within the prediction window, generating a computing power surge warning signal. This is used to trigger protective operations such as load buffering and resource reservation in advance, avoiding system lag and inference timeouts caused by sudden increases in computing power. Similarly, the falling edge slope can be compared with a preset drop threshold, which corresponds to the maximum allowable decrease in computing power per unit time. When the falling edge slope is less than the preset drop threshold, a sudden decrease in computing power within a short period of time is predicted within the prediction window, generating a computing power drop warning signal. This is used to trigger operations such as resource reclamation and task scheduling optimization in advance, avoiding resource waste caused by sudden drops in computing power.

[0108] Specifically, the fluctuation amplitude can be compared with a preset fluctuation stability threshold, which corresponds to the maximum allowable fluctuation range for stable load operation. When the fluctuation amplitude exceeds the preset fluctuation stability threshold, it is determined that the overall load fluctuation amplitude within the prediction window is too large, indicating insufficient operational stability. A fluctuation warning signal is generated to trigger operations such as load smoothing adjustment and dynamic adjustment of batch processing scale, thereby improving the stability of the inference process. Furthermore, the main fluctuation frequency can be compared with a preset high-frequency threshold, which corresponds to the maximum allowable fluctuation frequency for stable load operation. When the main fluctuation frequency exceeds the preset high-frequency threshold, it is determined that the load within the prediction window exhibits high-frequency periodic oscillations, which can easily cause hardware scheduling jitter and abnormal power consumption. A main frequency fluctuation warning signal is generated to trigger optimization operations such as input traffic smoothing and routing distribution equalization, suppressing the negative impact of high-frequency fluctuations on hardware lifespan and inference stability. All of the above warning signals are output in the form of Boolean flags, along with corresponding feature quantization values, to facilitate hierarchical response and fine-grained scheduling by the upper-layer scheduling system.

[0109] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0110] Based on the same inventive concept, this application also provides a computing power prediction system based on large model inference for implementing the computing power prediction method based on large model inference described above. The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the computing power prediction system based on large model inference provided below can be found in the limitations of the computing power prediction method based on large model inference described above, and will not be repeated here.

[0111] In one exemplary embodiment, such as Figure 8 As shown, a computing power prediction system 800 based on large model inference is provided, including: The semantic processing module 801 is used to perform contextual semantic encoding on the token sequence of the input text to obtain the semantic embedding vector of each token in the token sequence; predict the routing tendency of each token in the hybrid expert model based on the semantic embedding vector to obtain the routing tendency feature; detect semantic mutations between adjacent tokens based on the routing tendency feature to generate a semantic mutation marker sequence. Encoding module 802 is used to perform temporal encoding on the semantic embedding vector, the routing tendency feature, and the semantic mutation marker sequence to generate a semantic feature vector sequence; The hardware perception module 803 is used to acquire the memory access status, instruction pipeline execution status, and load characteristics of the experts activated by the hybrid expert model in the previous time window within a preset acquisition period, generate a hardware state feature vector, and record the start and end timestamps of the preset acquisition period; wherein, when the current inference window is the first inference, the load characteristics of the experts activated by the hybrid expert model in the previous time window are preset baseline load characteristics. The timing alignment module 804 is used to obtain the physical completion timestamp of each token in the semantic feature vector sequence, and to perform timing alignment on the semantic feature vector sequence based on the physical completion timestamp, the start timestamp, and the end timestamp. The temporal fusion module 805 is used to perform cross-attention fusion of the aligned semantic feature vector sequence and the hardware state feature vector to generate a collaborative feature vector. The computing power prediction module 806 is used to perform prediction processing on the collaborative feature vector through a preset prediction neural network to obtain a preset number of time step graphics processor utilization prediction value sequences, and generate a computing power early warning signal based on the graphics processor utilization prediction value sequences.

[0112] In some embodiments, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a computational power prediction method based on large model inference as described above.

[0113] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0114] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0115] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A computing power prediction method based on large model inference, characterized in that, The method includes: The token sequence of the input text is semantically encoded to obtain the semantic embedding vector of each token in the token sequence; the routing tendency of each token in the hybrid expert model is predicted based on the semantic embedding vector to obtain the routing tendency feature; the semantic mutation between adjacent tokens is detected based on the routing tendency feature to generate a semantic mutation marker sequence. The semantic embedding vector, the routing tendency feature, and the semantic mutation marker sequence are temporally encoded to generate a semantic feature vector sequence; Within a preset acquisition period, the memory access status, instruction pipeline execution status, and load characteristics of the experts activated by the hybrid expert model in the previous time window are acquired to generate a hardware state feature vector, and the start and end timestamps of the preset acquisition period are recorded; wherein, when the current inference window is the first inference, the load characteristics of the experts activated by the hybrid expert model in the previous time window are the preset baseline load characteristics. Obtain the physical completion timestamp of each token in the semantic feature vector sequence, and perform temporal alignment on the semantic feature vector sequence based on the physical completion timestamp, the start timestamp, and the end timestamp; The aligned semantic feature vector sequence is fused with the hardware state feature vector through cross-attention to generate a collaborative feature vector. The collaborative feature vector is predicted by a preset predictive neural network to obtain a preset number of time steps of the graphics processor utilization prediction value sequence, and a computing power warning signal is generated based on the graphics processor utilization prediction value sequence.

2. The method according to claim 1, characterized in that, The contextual semantic encoding of the token sequence of the input text includes: The consecutive token sequences within the current inference window are converted into an initial embedding vector using an embedding lookup table; The initial embedding vectors are concatenated into an embedding matrix; The embedding matrix is ​​input into a Transformer encoder with 2 to 4 layers for contextual semantic encoding to obtain the semantic embedding vector of each token; The step of predicting the routing tendency of each token in the hybrid expert model based on the semantic embedding vector includes: The semantic embedding vector of each token is input into the routing tendency prediction head, and the semantic embedding vector is mapped to a routing tendency logit vector with a dimension equal to the total number of experts in the hybrid expert model through a gated linear transformation, thus obtaining the routing tendency logit vector of each token. The step of detecting semantic mutations between adjacent tokens based on the routing tendency features includes: The routing tendency logit vector of each token is normalized using Softmax to obtain the routing tendency probability distribution; Calculate the Jensen-Shannon divergence between the routing tendency probability distributions of adjacent tokens; compare the Jensen-Shannon divergence with a preset mutation threshold to obtain the comparison result; Generate the semantic mutation marker sequence in binary form based on the comparison results; The temporal encoding of the semantic embedding vector, the routing tendency feature, and the semantic mutation marker sequence includes: The semantic embedding vector, the routing tendency logit vector, and the marker corresponding to the position of the token in the semantic mutation marker sequence of each token are concatenated to obtain the enhanced feature vector; The enhanced feature vectors are arranged in temporal order to form an enhanced feature matrix; The enhanced feature matrix is ​​input into a bidirectional gated recurrent unit for temporal encoding to obtain the encoding result; The encoding result is mapped through the output projection layer to generate the semantic feature vector sequence.

3. The method according to claim 1, characterized in that, The load characteristics of the experts include average computation time, average memory access frequency, and activation frequency; the acquisition of the graphics processor's memory access status, instruction pipeline execution status, and the load characteristics of the experts activated by the hybrid expert model in the previous time window within the preset acquisition period includes: Within the preset acquisition period, the number of row buffer hits and row buffer misses are read from the memory controller of the graphics processor; the memory bandwidth pressure index is calculated based on the number of row buffer hits and row buffer misses; the transaction queue depth is read, and the memory bandwidth pressure index and the transaction queue depth are combined into a register-level feature vector. Within the preset acquisition period, the tensor core instruction queue depth and the number of idle cycles of the tensor core are read from the instruction pipeline of the graphics processor; the tensor core utilization rate is calculated based on the number of idle cycles of the tensor core and the number of active cycles of the tensor core within the preset acquisition period; the streaming multiprocessor utilization rate is read, and the tensor core instruction queue depth, the number of idle cycles of the tensor core, the tensor core utilization rate, and the streaming multiprocessor utilization rate are combined into an instruction pipeline level feature vector; Obtain the expert index set activated by the hybrid expert model within the previous time window; for each expert in the expert index set, read the average computation time, average memory access volume, and activation frequency of each expert; concatenate the average computation time, average memory access volume, and activation frequency of each expert into an expert-level load feature vector; The register-level feature vector, the instruction pipeline-level feature vector, and the expert-level load feature vector are normalized and then concatenated to form the original hardware feature vector. The original hardware feature vector is then input into a two-layer multilayer perceptron for feature compression to generate the hardware state feature vector.

4. The method according to claim 1, characterized in that, The step of performing temporal alignment on the semantic feature vector sequence based on the physical completion timestamp, the start timestamp, and the end timestamp includes: Calculate the total duration of the preset collection period based on the start timestamp and the end timestamp; For each token, the relative position coefficient of the token within the preset collection period is calculated based on the difference between the physical completion timestamp and the start timestamp of the token, and the total duration. Calculate the time alignment weight of the token based on the relative position coefficient; The time alignment weight is multiplied by the semantic feature vector corresponding to the token in the semantic feature vector sequence to obtain the aligned semantic feature vector of the token, and the aligned semantic feature vector sequence is generated by traversing each token.

5. The method according to claim 4, characterized in that, The step of performing cross-attention fusion between the aligned semantic feature vector sequence and the hardware state feature vector includes: The aligned sequence of semantic feature vectors is arranged into a semantic feature matrix; the semantic feature matrix is ​​multiplied by the first query projection matrix, the first key projection matrix, and the first value projection matrix to obtain the semantic query matrix, the semantic key matrix, and the semantic value matrix. The hardware state feature vector is expanded into a hardware feature matrix with the same length as the semantic feature vector sequence; the hardware feature matrix is ​​multiplied by the second query projection matrix, the second key projection matrix, and the second value projection matrix to obtain the hardware query matrix, the hardware key matrix, and the hardware value matrix. Calculate a first attention weight based on the semantic query matrix and the hardware key matrix; calculate the cross-attention output of the semantic query hardware based on the first attention weight and the hardware value matrix; calculate a second attention weight based on the hardware query matrix and the semantic key matrix; calculate the cross-attention output of the hardware query semantics based on the second attention weight and the semantic value matrix. The cross-attention output of the semantic query hardware and the cross-attention output of the hardware query semantics are respectively subjected to sequence-dimensional average pooling to obtain a first aggregate vector corresponding to the semantic query hardware and a second aggregate vector corresponding to the hardware query semantics; the first aggregate vector and the second aggregate vector are concatenated to obtain a concatenated aggregate vector. Based on the preset gate weight vector, the preset gate bias, and the spliced ​​aggregation vector, a gate scalar is calculated; based on the gate scalar, the first aggregation vector and the second aggregation vector are weighted and fused to obtain a gated fusion vector; the spliced ​​aggregation vector is multiplied by a preset residual projection matrix to obtain a residual connection term; the gated fusion vector is added to the residual connection term to generate the collaborative feature vector.

6. The method according to claim 1, characterized in that, The predictive neural network is a fully connected network, comprising a first predictive layer, a second predictive layer, and an output layer; the computing power early warning signals include overload early warning signals, resource idle early warning signals, computing power surge early warning signals, computing power drop early warning signals, fluctuation early warning signals, and main frequency fluctuation early warning signals; The step of predicting the collaborative feature vector using a preset prediction neural network includes: The collaborative feature vector is input into the prediction neural network, and a hierarchical transformation is performed through the first prediction layer, the second prediction layer and the output layer to output a sequence of predicted values ​​of the graphics processor utilization for a preset number of time steps in the future; wherein, the number of neurons in the first prediction layer is 256, the number of neurons in the second prediction layer is 128, and the number of neurons in the output layer is equal to the preset number in the future. The generation of computing power early warning signals based on the graphics processor utilization prediction value sequence includes: The peak utilization, valley utilization, fluctuation amplitude, rising edge slope, falling edge slope, and main fluctuation frequency are extracted from the graphics processor utilization prediction value sequence. The peak utilization rate is compared with a preset high load threshold. When the peak utilization rate is greater than the preset high load threshold, the overload warning signal is generated. The valley utilization rate is compared with a preset low load threshold. When the valley utilization rate is less than the preset low load threshold, the resource idle warning signal is generated. The rising edge slope is compared with a preset surge threshold. When the rising edge slope is greater than the preset surge threshold, the computing power surge warning signal is generated. The falling edge slope is compared with a preset sudden drop threshold. When the falling edge slope is less than the preset sudden drop threshold, the computing power sudden drop warning signal is generated. The fluctuation amplitude is compared with a preset fluctuation stability threshold. When the fluctuation amplitude is greater than the preset fluctuation stability threshold, the fluctuation warning signal is generated. The main fluctuation frequency is compared with a preset high-frequency threshold. When the main fluctuation frequency is greater than the preset high-frequency threshold, the main frequency fluctuation warning signal is generated.

7. The method according to claim 4, characterized in that, The formula for calculating the time alignment weight is: ； ； in, For the first The physical completion timestamp of each token is measured in microseconds; The timestamp of the preset acquisition period is in microseconds. The timestamp of the end of the preset acquisition period is expressed in microseconds. For the first The relative position coefficient of each token within the preset collection period, with a value range of [value range missing]. ; For the first The time alignment weight of each token has a value range of [value range missing]. .

8. A computing power prediction system based on large model inference, characterized in that, The system includes: The semantic processing module is used to perform contextual semantic encoding on the token sequence of the input text to obtain the semantic embedding vector of each token in the token sequence; predict the routing tendency of each token in the hybrid expert model based on the semantic embedding vector to obtain the routing tendency feature; detect semantic mutations between adjacent tokens based on the routing tendency feature to generate a semantic mutation marker sequence. The encoding module is used to perform temporal encoding on the semantic embedding vector, the routing tendency feature, and the semantic mutation marker sequence to generate a semantic feature vector sequence; The hardware awareness module is used to acquire the memory access status, instruction pipeline execution status, and load characteristics of the experts activated by the hybrid expert model in the previous time window within a preset acquisition period, generate a hardware status feature vector, and record the start and end timestamps of the preset acquisition period; wherein, when the current inference window is the first inference, the load characteristics of the experts activated by the hybrid expert model in the previous time window are preset baseline load characteristics. The timing alignment module is used to obtain the physical completion timestamp of each token in the semantic feature vector sequence, and to perform timing alignment on the semantic feature vector sequence based on the physical completion timestamp, the start timestamp, and the end timestamp. The temporal fusion module is used to perform cross-attention fusion of the aligned semantic feature vector sequence and the hardware state feature vector to generate a collaborative feature vector; The computing power prediction module is used to perform prediction processing on the collaborative feature vector through a preset prediction neural network to obtain a preset number of time step graphics processor utilization prediction value sequences, and generate computing power early warning signals based on the graphics processor utilization prediction value sequences.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images