Heterogeneous feature alignment and real-time QoS (Quality of Service) prediction method and system for edge calculation

Abstract

The invention discloses a heterogeneous feature alignment and real-time QoS prediction method and system oriented to edge computing. The method comprises the following steps: reconstructing heterogeneous attribute metadata of a service to be predicted and a user into a standardized descriptive prompt text; extracting high-dimensional semantic features in an off-line manner by using the parameter-frozen teacher model, and mapping the high-dimensional semantic features to a low-dimensional public space through a teacher projection layer; constructing a lightweight student model, innovatively configuring an adaptive feature projection layer, and performing end-to-end supervision training by using a multi-target joint loss function including feature alignment loss, distillation loss and task loss to force the student model to accurately fit semantic distribution of a teacher model in a low-dimensional space; and when a QoS prediction request is received, adopting a bypass strategy, completely skipping teacher model and feature dictionary query, generating aligned semantic features in real time by utilizing a student model, and outputting a QoS prediction value. According to the method, high prediction precision, low reasoning delay and low storage overhead can be considered at the same time.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of service computing and artificial intelligence, and particularly relates to a heterogeneous feature alignment and real-time QoS (quality of service) prediction method and system for edge computing. BACKGROUND

[0002] Web service quality prediction is the core link of service recommendation system and service level agreement management. Traditional QoS prediction methods are mainly based on collaborative filtering or matrix decomposition, and such methods rely heavily on historical interaction data. In the face of the "cold start" scenario of new registered users or newly published services, the prediction accuracy will be greatly reduced due to the lack of interaction records.

[0003] In order to solve the cold start and data sparsity problem, existing technologies begin to introduce pre-trained large language models (such as BERT, RoBERTa, Llama or Phi series, etc.) to assist prediction. Such methods use the powerful semantic understanding ability of LLM to extract high-dimensional semantic features from the basic attribute description text of users and services (such as country, AS number, service function description, etc.), and then perform regression prediction after splicing with ID features. Studies have shown that the introduction of LLM features can significantly improve prediction accuracy.

[0004] However, the above-mentioned prediction method based on large model (hereinafter referred to as "existing heavy method") faces insurmountable engineering bottlenecks in actual industrial deployment, especially in edge computing scenarios, which includes the following three aspects: First, the inference delay is extremely high and cannot meet the real-time requirements.

[0005] The existing heavy method relies on a large model with tens of billions of parameters for online inference. When processing cold start requests, a single inference usually takes hundreds of milliseconds or even seconds. Modern recommendation systems usually require results to be returned within milliseconds, and the response speed of the existing method is far below the industrial standard.

[0006] Second, the storage overhead is huge and relies on a static feature dictionary.

[0007] In order to avoid the high delay of online inference, existing technologies usually adopt a "offline inference + online lookup table" compromise solution, that is, pre-compute the LLM features of all users / services and store them in a memory dictionary. This brings two new problems: first, it requires GB-level or even TB-level memory / display memory resources, which edge devices cannot bear; second, the feature dictionary is usually static and cannot handle dynamic changes in context information (such as user attribute changes).

[0008] Third, the heterogeneous model feature dimension is mismatched.

[0009] If a lightweight small model such as a multi-layer perceptron or a shallow Transformer is directly used to replace a large model, due to the huge dimensional gap between the feature space dimension (usually tens of dimensions) of the small model and the high-dimensional semantic space (usually thousands of dimensions, such as 3072 dimensions) of the large model, simple model compression will cause serious semantic information loss, resulting in prediction accuracy collapse.

[0010] Therefore, there is an urgent need for a real-time QoS prediction method that can inherit the high-precision semantic understanding ability of a large model, while breaking away from the dependence on online inference and large feature dictionary of the large model, and is suitable for edge-side low-resource environment. SUMMARY

[0011] In view of the above problems existing in the prior art, the present application provides a heterogeneous feature alignment and real-time QoS prediction method and system for edge computing, which can simultaneously consider high prediction accuracy, low inference delay and low storage overhead.

[0012] A heterogeneous feature alignment and real-time QoS prediction method for edge computing, comprising the following steps: (1) Multi-source attribute reconstruction and prompt text construction: obtaining sparse attribute metadata of Web services and Web users, reconstructing the attribute metadata into standardized descriptive prompt text using a preset prompt template, and obtaining a training sample set; (2) Teacher model offline feature dimension reduction and extraction: using a pre-trained large language model with frozen parameters as the backbone of the teacher model to perform offline inference on the descriptive prompt text in the training sample set; mapping the high-dimensional hidden vector output by the large language model to a low-dimensional public feature space through a teacher projection layer inside the teacher model, and extracting and caching the teacher semantic features and teacher soft labels; (3) Student model distillation training based on heterogeneous feature alignment: constructing a lightweight student model and configuring an adaptive student projection layer in the student model; using a multi-objective joint loss function including feature alignment loss, distillation loss and task loss to perform end-to-end supervised training on the student model, driving the student projection layer to map the original feature vector generated by the student model to a low-dimensional public feature space consistent with the dimension of the teacher semantic features, and obtaining student semantic features; (4) Dictionary-free online real-time inference: when receiving a QoS prediction request, using a bypass strategy to skip the teacher model and directly using the trained student model to perform real-time inference on the descriptive prompt text of the object to be predicted; the student model generates student semantic features in real time through its adaptive feature projection layer, and outputs a QoS prediction value in combination with the ID embedding features.

[0013] The method of the present application realizes cross-dimensional semantic migration of the feature space by constructing a generative framework of "large model guidance and small model inference".

[0014] In step (1), the sparse attribute metadata of the Web service and Web user includes, but is not limited to, the user's AS number and country code, as well as the service's WSDL address and provider information. Using a preset prompt template, the aforementioned discrete, unstructured sparse attribute data is reconstructed into natural language descriptive prompt text containing rich semantic information.

[0015] In step (2), firstly, a high-dimensional semantic latent vector (e.g., 3072-dimensional) is extracted from the backbone of the teacher model. Secondly, the high-dimensional semantic latent vector is dimensionality-reduced and mapped to a low-dimensional feature space (e.g., 16-dimensional or 32-dimensional, consistent with the ID embedding vector dimension) through a teacher projection layer to obtain the "teacher semantic features". Finally, the regression prediction values ​​of the teacher model are extracted as "teacher soft labels". In this step, the "teacher semantic features" and "teacher soft labels" are cached for subsequent use.

[0016] In step (2), the extracted teacher semantic features and teacher soft labels are stored in the local key-value cache; during the training process in step (3), the teacher information of the corresponding sample is read directly from the local key-value cache without performing real-time inference of the teacher model.

[0017] In step (3), a lightweight student model is constructed, specifically including a cascaded lightweight encoder, an ID embedding layer, an adaptive student projection layer, and a regression prediction layer: The lightweight encoder is used to receive descriptive prompt text and extract low-dimensional raw feature vectors.

[0018] The ID embedding layer is used to map user IDs and service IDs into dense vectors, i.e., ID embedding features.

[0019] The student projection layer contains trainable linear mapping weights to solve the feature dimension mismatch problem between the lightweight encoder and the teacher model, mapping the original feature vectors to a low-dimensional common feature space consistent with the teacher's semantic feature dimension to obtain the student's semantic features. The regression prediction layer is used to concatenate student semantic features and ID embedding features, and outputs the final service quality prediction value through a multilayer perceptron and a linear layer.

[0020] In step (3), the calculation formula performed by the student projection layer is as follows: in, This represents the low-dimensional original feature vector output by the lightweight encoder. This represents the cross-dimensional mapping weight matrix that the student projection layer can learn. This represents the bias vector. This represents the student semantic feature vector that is located in the low-dimensional common feature space after mapping.

[0021] In step (3), the feature alignment loss is used to calculate the Euclidean distance between the student's semantic features and the teacher's semantic features in the low-dimensional common feature space. By minimizing this distance, the student model is forced to learn the deep semantic distribution of the teacher model, thereby solving the dimensionality mismatch problem between heterogeneous models. Distillation loss is used to calculate the difference between the student model's predicted values ​​and the teacher's soft labels, enabling the student model to inherit the generalization reasoning logic of the teacher model. The task loss is used to calculate the difference between the student model's predicted values ​​and the actual QoS labels, ensuring the absolute accuracy of the prediction results.

[0022] In step (3), the calculation formula for the multi-objective joint loss function is as follows: ; in, It is the total loss function used for backpropagation. These are hyperparameters used to balance the weights of different loss terms. The feature alignment loss is calculated using mean squared error. Distillation losses are calculated using the mean square error. The mean squared error is used to calculate the task loss.

[0023] In step (4), the QoS prediction request includes a cold start request for newly registered users or newly published services, and a non-cold start request for existing objects.

[0024] For any type of request, the system directly generates and predicts features end-to-end through the descriptive prompt text constructed in real time by the student model, without loading the teacher model or maintaining or querying a static feature dictionary, thereby achieving low-latency response and low storage consumption at the edge.

[0025] Based on the same inventive principle, the present invention also provides a heterogeneous feature alignment and real-time QoS prediction system for edge computing, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned heterogeneous feature alignment and real-time QoS prediction method.

[0026] Based on the same inventive principle, this invention also provides a heterogeneous feature alignment and real-time QoS prediction system for edge computing, comprising: The data standardization processing module is used to convert heterogeneous user and service attribute metadata into descriptive prompt text in a unified format.

[0027] The offline knowledge transfer module is used to manage teacher models, extract dimensionality-reduced teacher semantic features and soft labels in batches, and cache them.

[0028] The heterogeneous distillation training module is used to build a lightweight student model and optimize the feature alignment loss so that the output of the student projection layer is aligned with the teacher's semantic features in a low-dimensional space.

[0029] The edge-side real-time inference module, deployed on edge servers or terminal devices, is used to provide millisecond-level responses to real-time QoS prediction requests without relying on external feature libraries.

[0030] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention breaks through the performance bottleneck of "accuracy-speed," achieving millisecond-level real-time response. Existing technologies rely on large models for online inference, resulting in latency up to seconds. This invention, through heterogeneous feature alignment technology, enables lightweight student models with only tens or hundreds of thousands of parameters to "mimic" the behavior of large models with billions of parameters in terms of mathematical distribution. Experiments show that this invention improves the single-inference speed by two orders of magnitude (approximately 100 times) while maintaining prediction accuracy comparable to large models (accuracy loss <2%), fully meeting the stringent requirements of industrial-grade real-time recommendation systems.

[0031] 2. A pioneering "zero-dictionary" inference architecture significantly reduces storage costs. Existing technologies, in order to avoid latency, are forced to maintain GB-level or even TB-level static "Prompt-Feature" dictionaries, which not only consume expensive memory / GPU resources but also cannot handle dynamic updates. This invention endows the student model with the ability to generate features in real time, completely eliminating the dependence on static feature dictionaries and reducing model storage overhead by more than 99%. This allows the system to be deployed at low cost on resource-constrained edge devices (such as IoT gateways and edge servers).

[0032] 3. This invention solves the "dimensionality mismatch" problem between heterogeneous models, effectively supporting cold starts. For newly registered users or services (cold start scenarios), traditional small models fail due to a lack of semantic understanding, while directly compressing large models leads to semantic collapse. The adaptive feature projection layer designed in this invention acts as a "semantic translator," successfully transferring high-dimensional semantic knowledge losslessly to a low-dimensional space. This enables the student model to provide high-precision QoS predictions even when facing cold start users it has never encountered before, through real-time analysis of their text descriptions, significantly improving the system's robustness.

[0033] 4. Possesses full-scenario adaptability and extremely high engineering application value. This invention unifies the processing logic for cold start and warm start, eliminating the need to maintain two sets of systems for different scenarios. Simultaneously, its lightweight nature allows for extremely rapid model updates, enabling it to adapt to the high-frequency dynamic changes in the Web service environment, thus possessing extremely high industrial practical value. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of a heterogeneous feature alignment and real-time QoS prediction method for edge computing according to the present invention.

[0036] Figure 2 This is a schematic diagram of a knowledge distillation training architecture based on heterogeneous feature alignment provided in an embodiment of the present invention.

[0037] Figure 3 This is a schematic diagram of the online real-time reasoning process without dictionary definition provided in an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0040] like Figure 1 As shown, a heterogeneous feature alignment and real-time QoS prediction method for edge computing is presented. This method achieves cross-dimensional semantic transfer of the feature space by constructing a generative framework of "large model guidance and small model inference". The specific process is as follows: Step S1: Descriptive Text Reconstruction of Multi-Source Attributes. This step aims to transform sparse, heterogeneous user and service metadata into a natural language sequence that can be understood by a large language model.

[0041] S11. Obtain raw attribute metadata. The system first obtains the raw attribute data of the object to be predicted from the Web service platform or public dataset (such as WSDream). For Web services, the obtained attributes include, but are not limited to: WSDL address, service provider, service hosting country, Autonomous System number (ASN), IP address range, etc. For Web users, the obtained attributes include, but are not limited to: user's country, user's ASN, geographic coordinates, network access type, etc.

[0042] S12. Construct structured descriptive prompt text. Using a preset prompt template, the discrete attributes mentioned above are concatenated into a coherent natural language description. Example of a user prompt text template: "This is a web user located in [Country], accessing the network via autonomous system [AS_Number], with IP range [IP_Segment]...". Example of a service prompt text template: "This is a web service provided by [Provider], hosted at [WSDL_URL], located in [Country], serving on autonomous system [AS_Number]...". Through this step, heterogeneous ID and attribute data, which originally could not be directly calculated for similarity, are transformed into text sequences containing rich semantic information. User text) and (Service text).

[0043] Step S2: Offline feature dimensionality reduction and extraction based on the teacher model. This step utilizes a pre-trained large language model with frozen parameters to extract high-level semantic knowledge. To ensure effective feature fusion, the teacher model employs a composite structure of "dual-tower input + projection dimensionality reduction + MLP fusion".

[0044] S21. Construct the teacher model architecture. The teacher model consists of the following five core components: Pre-trained large model backbone: Employs a pre-trained model with billions of parameters (such as Phi-3, Llama-3, or RoBERTa-Large). Its function is to process the input prompt text. Encoded as a high-dimensional latent vector Assume the output dimension of the main body of the large model is... (e.g., 3072 dimensions).

[0045] Teacher projection layer: A parameter-frozen linear mapping layer used to project high-dimensional vectors. Compressed to a low-dimensional common feature space (dimension 1) (e.g., 32-dimensional), to obtain teacher semantic features. .

[0046] Teacher ID embedding layer: used to map user IDs or service IDs to dense vectors Its dimensions are also .

[0047] Multilayer perceptron: used for semantic features and ID embedding features Perform nonlinear fusion.

[0048] Prediction Header: A linear regression layer that outputs the final QoS prediction value.

[0049] S22. Offline batch extraction and caching of "teacher semantic features". Input all training samples constructed in step S1 into the teacher model. For any sample (User or service), the calculation process is as follows: Semantic feature extraction: ; ; in, The “teacher semantic features” will be cached to guide feature alignment for student models.

[0050] Soft tag generation: ; ; in, The "teacher soft tag" will be cached for distillation training.

[0051] Caching strategy: The system only needs to cache the tuples. Stored in a local key-value database, subsequent training can directly read from this cache, eliminating the need to repeatedly run the large model.

[0052] Step S3: Distillation Training of the Student Model Based on Heterogeneous Feature Alignment. This step constructs a lightweight student model that structurally corresponds to the teacher model but has a very small number of parameters. Through a multi-objective joint loss function, the student model is forced to "align" with the teacher's semantic features in a low-dimensional common space. For example... Figure 2 As shown, the internal structure of the teacher and student models, including the projection layer, the dual-input mechanism of ID features, and the calculation path of the multi-objective joint loss function are illustrated in detail.

[0053] S31. Construct a lightweight student model architecture. The student model is designed as a micro-architecture suitable for edge deployment, containing the following components: Lightweight encoders: These employ 2-4 layer Transformer blocks or simple deep neural networks, with only one-thousandth the number of parameters of the teacher model. Their input is prompt text. The output is the student's original features. .Notice: Dimensions Typically lower in dimension (e.g., 64 or 128).

[0054] Adaptive Feature Projection Layer: A learnable linear transformation layer used to transform... Mapped to the same characteristics as teachers Public space. Its calculation formula is: ; in, Project weights to students, For "student semantic features".

[0055] Student ID Embedding Layer: Maps IDs to dimensions... vector .

[0056] Regression prediction layer: Contains a student MLP and a prediction head, with a structure similar to the teacher model but on a smaller scale. The specific calculations are as follows: ; S32. Constructing a multi-objective joint loss function. To achieve knowledge transfer from heterogeneous models, this invention defines a joint loss function consisting of three parts. : Feature alignment loss: used to minimize student semantic features semantic features of teachers Euclidean distance in low-dimensional public space.

[0057] ; in, For batch size, This represents the square of the L2 norm.

[0058] Distillation loss: used to improve the predicted output of the student model. Soft labels for fitting teacher models .

[0059] ; Task loss: Used to allow the student model to fit real QoS observations. .

[0060] ; S33. Perform end-to-end optimization. The final overall optimization objective is: in, To balance hyperparameters (e.g., take...) During training, the backpropagation algorithm is used to simultaneously update the lightweight encoder parameters and the student projection matrix. And the parameters of the regression prediction layer.

[0061] Step S4: Dictionary-free online real-time inference. After training is complete, the system enters the online service phase. At this point, the system architecture undergoes a qualitative change, completely removing the teacher model and no longer relying on any feature dictionary. For example... Figure 3 As shown, the logical path for end-to-end feature generation and QoS prediction using the student model is demonstrated after bypassing the teacher model.

[0062] S41. Receive real-time request. The system receives a request for a user... Services QoS prediction request.

[0063] S42. End-to-end real-time generation. The system no longer looks up tables, but instead executes the following real-time computation flow: Real-time word generation: Generates prompt text in real time based on attribute metadata in the current request. .

[0064] Real-time encoding and alignment: Input text into a pre-trained student model, pass it through a lightweight encoder and projection layer, and generate aligned semantic features in milliseconds. .

[0065] Hybrid prediction: Combining student ID embedded features, the final QoS prediction value is output through the student MLP and prediction header. .

[0066] Performance Evaluation and Validation of Beneficial Effects. To verify the technical effectiveness of this invention, comparative experiments were conducted on the WSDream dataset. The experimental environment consisted of a single NVIDIA T4 GPU, and the comparison object was the traditional method that directly uses the Phi-3-mini large model. The experimental data are shown in Table 1 below: Table 1 Experimental results show that this invention, through its innovative heterogeneous feature alignment architecture, successfully breaks the trade-off between accuracy and efficiency in QoS prediction, achieving a hundredfold increase in inference speed and extreme compression of storage space at a minimal cost to accuracy, and has extremely high industrial practical value.

[0067] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A heterogeneous feature alignment and real-time QoS prediction method for edge computing, characterized in that, Includes the following steps: (1) Obtain sparse attribute metadata of Web services and Web users, and use preset prompt templates to reconstruct the attribute metadata into standardized descriptive prompt text to obtain a training sample set; (2) Using the pre-trained large language model with parameter freezing as the backbone of the teacher model, offline reasoning is performed on the descriptive prompt text in the training sample set; through the teacher projection layer inside the teacher model, the high-dimensional latent vector output by the large language model is mapped to the low-dimensional common feature space, and the teacher semantic features and teacher soft labels are extracted and cached. (3) Construct a lightweight student model and configure an adaptive student projection layer in the student model; use a multi-objective joint loss function including feature alignment loss, distillation loss and task loss to conduct end-to-end supervised training of the student model, and drive the student projection layer to map the original feature vector generated by the student model to a low-dimensional common feature space consistent with the teacher's semantic feature dimension to obtain the student semantic features. (4) When a QoS prediction request is received, a bypass strategy is adopted to skip the teacher model and directly use the trained student model to perform real-time reasoning on the descriptive prompt text of the object to be predicted; the student model generates student semantic features in real time through its adaptive feature projection layer and outputs QoS prediction values ​​in combination with ID embedding features.

2. The heterogeneous feature alignment and real-time QoS prediction method for edge computing according to claim 1, characterized in that, In step (2), the extracted teacher semantic features and teacher soft labels are stored in the local key-value cache; during the training process in step (3), the teacher information of the corresponding sample is read directly from the local key-value cache without performing real-time inference of the teacher model.

3. The heterogeneous feature alignment and real-time QoS prediction method for edge computing according to claim 1, characterized in that, In step (3), a lightweight student model is constructed, specifically including a cascaded lightweight encoder, an ID embedding layer, an adaptive student projection layer, and a regression prediction layer: The lightweight encoder is used to receive descriptive prompt text and extract low-dimensional raw feature vectors; The ID embedding layer is used to map user IDs and service IDs into dense vectors, i.e., ID embedding features; The student projection layer contains trainable linear mapping weights to solve the feature dimension mismatch problem between the lightweight encoder and the teacher model, mapping the original feature vectors to a low-dimensional common feature space consistent with the teacher's semantic feature dimensions to obtain student semantic features. The regression prediction layer is used to concatenate student semantic features and ID embedding features, and outputs the final service quality prediction value through a multilayer perceptron and a linear layer.

4. The heterogeneous feature alignment and real-time QoS prediction method for edge computing according to claim 1, characterized in that, In step (3), the calculation formula performed by the student projection layer is as follows: ； in, This represents the low-dimensional original feature vector output by the lightweight encoder. This represents the cross-dimensional mapping weight matrix that the student projection layer can learn. This represents the bias vector. This represents the student semantic feature vector that is located in the low-dimensional common feature space after mapping.

5. The heterogeneous feature alignment and real-time QoS prediction method for edge computing according to claim 1, characterized in that, In step (3), the feature alignment loss is used to calculate the Euclidean distance between the student's semantic features and the teacher's semantic features in the low-dimensional common feature space. By minimizing this distance, the student model is forced to learn the deep semantic distribution of the teacher model. The distillation loss is used to calculate the difference between the student model's predicted value and the teacher's soft label, so that the student model inherits the generalization reasoning logic of the teacher model. The task loss is used to calculate the difference between the student model's predicted value and the real QoS label, so as to ensure the absolute accuracy of the prediction result.

6. The heterogeneous feature alignment and real-time QoS prediction method for edge computing according to claim 1, characterized in that, In step (3), the calculation formula for the multi-objective joint loss function is as follows: ； in, It is the total loss function used for backpropagation. These are hyperparameters used to balance the weights of different loss terms. The feature alignment loss is calculated using mean squared error. Distillation losses are calculated using the mean square error. The mean squared error is used to calculate the task loss.

7. The heterogeneous feature alignment and real-time QoS prediction method for edge computing according to claim 1, characterized in that, In step (4), the QoS prediction request includes a cold start request for newly registered users or newly published services, and a non-cold start request for existing objects; For any type of request, the system directly generates and predicts features end-to-end through the descriptive prompt text constructed in real time by the student model, without loading the teacher model or maintaining or querying a static feature dictionary, thereby achieving low-latency response and low storage consumption at the edge.

8. A heterogeneous feature alignment and real-time QoS prediction system for edge computing, characterized in that, The method includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the heterogeneous feature alignment and real-time QoS prediction method according to any one of claims 1-7.

Images