Distributed sensing multi-stage large model fine tuning method and device

Abstract

The embodiment of the invention provides a distribution perception multi-stage large model fine tuning method and device. The large model to be finely tuned comprises a pre-trained backbone network and a prediction layer. In the fine tuning method, samples in a data set are used for performing first training on a large model to obtain a first large model, and the first training comprises updating parameters of a prediction layer in the large model and not updating parameters of a backbone network in the large model. Then, samples in the data set are used for carrying out second training on the large model to obtain a second large model, and the second training comprises updating parameters of a backbone network in the large model and parameters in a prediction layer; in this way, two trained large models are obtained, then parameter valuing can be carried out in a range formed by corresponding parameters in the two trained large models, and a fine-tuned large model is obtained. The data set contains privacy data, and privacy protection needs to be carried out in the data processing process.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer technology, and in particular, to a distributed-aware multi-stage large model fine-tuning method and device. Background Art

[0002] In recent years, the rise of large models such as GPT, LLaMA, and GLM has marked a new era in the field of natural language processing. These large models have set new performance records in many natural language processing (NLP) tasks and demonstrated great potential, thanks to the massive world knowledge encoded in their weights and excellent inherent reasoning capabilities. In view of this, many studies have begun to use the powerful capabilities of large models for task modeling. They use pre-trained large models as the basic framework and fine-tune the large models on labeled datasets designed for specific tasks so that the large models can quickly adapt to specific task processing. There is also a need to protect privacy data when large models process specific tasks. Although pre-trained large models show good generalization potential, fine-tuning for specific tasks will cause the large models to over-specialize the training data, resulting in a significant decrease in generalization ability when facing changes in the distribution of real-world data.

[0003] Therefore, we hope to have an improved solution that can improve the generalization ability of the fine-tuned large model when facing different data distributions in the real world. Summary of the invention

[0004] One or more embodiments of this specification describe a distribution-aware multi-stage large model fine-tuning method and device to improve the generalization ability of the fine-tuned large model when facing different data distributions in the real world. The specific technical solution is as follows.

[0005] In a first aspect, an embodiment provides a multi-stage large model fine-tuning method, wherein the large model to be fine-tuned includes a pre-trained backbone network and a prediction layer, and the method includes:

[0006] Get the dataset;

[0007] Performing a first training on the large model using samples in the data set to obtain a first large model; the first training includes: updating parameters of the prediction layer in the large model, and not updating parameters of the backbone network in the large model;

[0008] Performing a second training on the large model using samples in the data set to obtain a second large model; the second training comprises: updating parameters of the backbone network and parameters in the prediction layer in the large model;

[0009] Parameter values ​​are taken within a range formed by parameters in the first large model and corresponding parameters in the second large model to obtain a fine-tuned large model.

[0010] In one embodiment, the step of taking parameter values ​​within a range formed by the parameters in the first large model and the corresponding parameters in the second large model includes:

[0011] Based on the interpolation coefficients, interpolation is performed between the parameters in the first large model and the corresponding parameters in the second large model.

[0012] In one implementation, the step of interpolating includes:

[0013] Determining first weights of parameters in the first large model based on the interpolation coefficients, and determining second weights of corresponding parameters in the second large model;

[0014] Interpolation is performed between parameters in the first large model and corresponding parameters in the second large model using the first weight and the second weight.

[0015] In one embodiment, the interpolation coefficient is a parameter to be trained; the step of interpolating between the parameter in the first large model and the corresponding parameter in the second large model based on the interpolation coefficient includes:

[0016] Based on the values ​​of the interpolation coefficients, interpolation is performed between the parameters in the first large model and the corresponding parameters in the second large model to obtain a third large model;

[0017] Inputting samples in the data set into the third largest model to obtain prediction results, and optimizing and updating the values ​​of the interpolation coefficients based on the prediction results;

[0018] When determining the optimal value of the interpolation coefficient, the third largest model corresponding to the optimal value is used as the fine-tuned large model.

[0019] In one embodiment, the step of acquiring a data set includes:

[0020] Get the original data set and some unlabeled data;

[0021] Based on a first data distribution of the original data set, identifying target data outside the first data distribution from a plurality of unlabeled data;

[0022] The target data is added to the original data set to obtain the data set.

[0023] In one embodiment, the step of identifying target data outside the first data distribution from a plurality of unlabeled data includes:

[0024] Determine a global distance between any unlabeled data and the first data distribution;

[0025] Determine the nearest neighbor distance between any unlabeled data and the samples in the original data set;

[0026] When the global distance of the unlabeled data is greater than a first threshold, and the nearest neighbor distance of the unlabeled data is greater than a second threshold, the unlabeled data is determined as the target data.

[0027] In one implementation, the step of adding the target data to the original data set includes: determining the labeling data of the target data, and adding the target data and the labeling data to the original data set correspondingly.

[0028] In one embodiment, the step of performing a second training on the large model using samples in the data set includes:

[0029] The first large model is used as an object of second training, and the samples in the data set are used to perform second training on the first large model to obtain a second large model.

[0030] In one embodiment, the prediction layer is a binary classification classifier; the label of the sample includes a binary classification label in the binary classification task and a label distribution value reflecting the classification level; the classification in the binary classification task includes a first classification and a second classification, and the label distribution value includes a probability value on the first classification and a probability value on the second classification; the step of first training the large model using the samples in the data set includes:

[0031] Input the samples in the data set into the large model, and determine the predicted probability distribution of the samples in the binary classification task through the classifier included in the large model;

[0032] Determining a distribution loss based on a distribution difference between the predicted probability distribution and the label distribution value;

[0033] The large model is first trained based on the distribution loss.

[0034] In one embodiment, the label distribution value of the sample is determined in the following manner: the label distribution value corresponding to the classification level of the sample is determined based on a prior distribution, and the prior distribution includes label distribution values ​​corresponding to multiple classification levels.

[0035] In one embodiment, the step of first training the large model based on the distribution loss includes:

[0036] Based on the difference between the binary classification label and the label prediction value corresponding to the sample in the binary classification task, the classification loss is determined; and the large model is first trained based on the distribution loss and the classification loss.

[0037] In a second aspect, the embodiment provides a multi-stage large model fine-tuning device, wherein the large model to be fine-tuned includes a pre-trained backbone network and a prediction layer, and the device includes:

[0038] A data set acquisition module, configured to acquire a data set;

[0039] A first training module is configured to perform a first training on the large model using samples in the data set to obtain a first large model; the first training includes: updating parameters of the prediction layer in the large model, and not updating parameters of the backbone network in the large model;

[0040] A second training module is configured to perform a second training on the large model using samples in the data set to obtain a second large model; the second training includes: updating parameters of the backbone network and parameters in the prediction layer in the large model;

[0041] The parameter value taking module is configured to take parameter values ​​within a range formed by the parameters in the first large model and the corresponding parameters in the second large model to obtain a fine-tuned large model.

[0042] In a third aspect, an embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute any one of the methods described in the first aspect.

[0043] In a fourth aspect, an embodiment provides a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in any one of the first aspects is implemented.

[0044] The method and apparatus provided in the embodiments of this specification train a large model through multiple stages, and the backbone network in the first large model obtained retains the parameters of the original large model. The parameters of the backbone network in the second large model obtained are fine-tuned based on the data set, and the parameter values ​​are taken within the range of the corresponding parameters of the two large models. This can reduce the over-specialization of the large model on the training data during the fine-tuning process, thereby improving the generalization ability of the large model after fine-tuning when facing different data distributions. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification;

[0047] Figure 2 A schematic diagram of an implementation architecture of an embodiment disclosed in this specification;

[0048] Figure 3 A schematic flow chart of a multi-stage large model fine-tuning method provided in an embodiment;

[0049] Figure 4 A schematic diagram of two labels of a sample x and the prior distribution q(x);

[0050] Figure 5 A comparative diagram of the statistical results of the prediction results of the large model;

[0051] Figure 6 A schematic block diagram of a multi-stage large model fine-tuning device provided in an embodiment. DETAILED DESCRIPTION

[0052] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0053] Large Language Model (LLM), also called a large model, is a language model that usually contains tens of billions or more parameters, has strong semantic understanding capabilities, and can understand and generate natural language text. In the pre-training stage, self-supervised learning or semi-supervised learning is used to train on a large amount of text data, which can come from various sources such as the Internet, books, and news. Large models use data from various application fields during pre-training, which contains a lot of knowledge, so they have stronger language processing capabilities and wider applicability.

[0054] In order to enable the large language model to handle various downstream tasks, such as text classification, question answering, document summarization, and text generation, in the fine-tuning stage, the large language model can also be fine-tuned in combination with specific downstream tasks or features of specific fields, as well as training sets corresponding to downstream tasks, thereby improving the performance of the model in specific scenarios. This application relates to improvements to the fine-tuning stage of the large language model.

[0055] In order to fine-tune the large language model in conjunction with downstream tasks, the corresponding training set for fine-tuning is required. For example, when the downstream task is to identify the intent of user questions, a user question may be: how to remit money to the other party's bank card safely and quickly. The intents to be selected include: transfer remittance, cardless remittance, unbinding, etc. The labeled intent of the above user question may be transfer remittance. The user question and the labeled intent can be used as training samples in the downstream task of intent identification. When the downstream task is search relevance prediction, the training samples include query data and queried items. The large model is used to predict the relevance between the query data and the queried items to obtain a relevance score. Search relevance modeling plays a vital role in search engines. Its task is to select the most relevant products (i.e., project items) for a given query from the product candidate set, thereby improving the user's search experience.

[0056] When the fine-tuned large model is put into a specific task, the data faced by the large model, that is, the data to be processed as input to the large model, may be data in the in-distribution data (ID) set or data in the out-of-distribution data (OOD) set. The in-distribution data set refers to a data set with a similar distribution to the data used by the large model during the training process, and the data set maintains a distribution similar to that of the data set in the training phase. The out-of-distribution data set refers to a data set with a dissimilar distribution to the data used by the model during the training process, and the data distribution is different from the distribution in the model training phase.

[0057] In practical applications, the generalization of the model on out-of-distribution datasets is an important consideration. The generalization of a model refers to the ability of the model to predict new, unseen data. A model with good generalization ability can learn universal rules from limited training data and make accurate predictions on new data. In the real world, the data encountered by the model is often diverse, and may encounter a variety of data distributions. In practical applications, it is necessary to pay attention to the generalization ability and robustness of the model.

[0058] Although fine-tuning techniques can enable large models to quickly adapt to specific downstream tasks and improve model performance, naive fine-tuning methods still have limitations. When faced with changes in data distribution in real-world applications, large models after naive fine-tuning often experience significant performance degradation. Although pre-trained large models show impressive generalization capabilities in cross-task applications, fine-tuning may damage the inherent generalization capabilities of large models due to reasons such as shortcut learning and feature distortion, resulting in a huge performance gap between the model on in-distribution data and out-of-distribution data.

[0059] Based on this, in order to improve the generalization ability of the fine-tuned large model when facing different data distributions in the real world, the embodiments of this specification have made improvements in data enhancement, multi-stage large model fine-tuning and loss function design, etc. The following are detailed descriptions.

[0060] Figure 1 The present invention is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. It includes multiple computing devices, such as device 1 and device 2. Device 1 is used to perform data enhancement, that is, to identify out-of-distribution data from an unlabeled data set and add it to the original data set, and perform data enhancement on the original data set. Device 2 is used to perform multi-stage training on a large model to obtain two large models, and merge the parameters of the two large models to obtain a fine-tuned large model. Figure 1 This is just an application example. In actual applications, device 1 and the operations performed by device 1 can be performed by one computing device, or the operations performed by device 2 can be performed by multiple computing devices, which is feasible.

[0061] Figure 2 This is a schematic diagram of the implementation architecture of an embodiment disclosed in this specification. The left side shows the implementation process framework of using an unlabeled dataset to supplement the original dataset with out-of-distribution data. Specifically, it includes using an embedding model to determine the embedding vectors of samples in the original dataset and the embedding vectors of samples in the unlabeled dataset, using the embedding vectors of these two samples to perform OOD detection, identifying out-of-distribution data from the unlabeled dataset, and adding it to the original dataset to obtain an enhanced training set.

[0062] The data enhancement method provided in the embodiment of this specification is described in detail below through the following steps 1 to 3, and the method is executed by a computing device.

[0063] Step 1: Get the original dataset D id , obtain some unlabeled data. Some includes one or more.

[0064] Original dataset D id It is a dataset used to fine-tune large models in specific tasks. id The samples in the dataset are data that the large model can recognize and process, such as text data or image data. The sample feature values ​​in the dataset usually follow a certain probability distribution. id There is a first data distribution, which may be, but is not limited to, Gaussian distribution, beta distribution, uniform distribution, Poisson distribution, etc.

[0065] Unlabeled data comes from unlabeled datasets. Data in unlabeled datasets can be obtained through search. Unlabeled data is data that can be recognized and processed by large models, such as text data or image data. Unlabeled datasets include sample data in the distribution (ID) and out of distribution (OOD).

[0066] Step 2: Based on the first data distribution, identify target data outside the first data distribution from a plurality of unlabeled data.

[0067] Step 3: Add the target data to the original dataset D id , and get the enhanced data set D aug .

[0068] Among them, the original data set D id The data in follows the first data distribution and belongs to the in-distribution data. The target data outside the first data distribution belongs to the out-of-distribution OOD data.

[0069] Labeling the entire unlabeled dataset requires a lot of manual work. Therefore, in order to improve processing efficiency, the embodiment also provides a new distribution-aware strategy, which uses the OOD detection algorithm to identify OOD target data from the unlabeled dataset. These identified target data will be labeled and used to enrich the original dataset D id .

[0070] Step 2 can process a number of unlabeled data according to the following steps 21 to 23 to identify target data from the number of unlabeled data.

[0071] Step 21: determine the global distance between any unlabeled data and the first data distribution.

[0072] Step 22: Determine whether any unlabeled data is related to the original data set D id The nearest neighbor distance between samples in is also called the local distance.

[0073] Step 23: when the global distance of the unlabeled data is greater than the first threshold d1 and the nearest neighbor distance of the unlabeled data is greater than the second threshold d2, the unlabeled data is determined as the target data.

[0074] When determining the global distance, the Mahalanobis distance can be used to determine the distance between the unlabeled data and the first data distribution based on the embedding vector of the unlabeled data. Take the first data distribution as the following Gaussian distribution as an example:

[0075]

[0076] in, and Σ are the mean and variance of the Gaussian distribution, which can be obtained from the original data set D id The Mahalanobis distance is used to measure the distance between each unlabeled sample and the Gaussian distribution:

[0077]

[0078] Among them, d Mahal is the unlabeled data x u Compared with the original dataset D id The Mahalanobis distance between u ) is the unlabeled data x u The embedding vector of the sample is mapped to the representation space by the function g. The function g can be implemented by an embedding model, including but not limited to Bert, BGE, T5 or GPT models. The Mahalanobis distance is only an example of determining the distance between a sample and a data distribution. In practical applications, other algorithms such as the Euclidean distance can also be used.

[0079] In step 22, the unlabeled data x is determined u Compared with the original dataset D id When the nearest neighbor distance between samples in , we can calculate the unlabeled data x separately u Compared with the original dataset D id The similarity between each sample in the , multiple similarity values ​​are obtained, and the largest similarity is used as the nearest neighbor distance. When calculating the similarity, algorithms such as cosine similarity or Manhattan distance can be used. In practical applications, the nearest neighbor distance can be calculated by the following formula:

[0080]

[0081] Among them, cos represents the unlabeled data x u Compared with the original dataset D id Sample x in i The cosine similarity between i ) represents the sample x i The embedding vector d knn is the nearest neighbor distance.

[0082] After obtaining the global distance and the nearest neighbor distance, in step 23, the target data u can be determined based on the following formula ood :

[0083] u ood ={x u :d Mahal (x u ,D id )>d1∧d knn (x u )>d2} (4)

[0084] Among them, ∧ is the logical AND symbol, that is, the data where both metrics exceed a certain threshold are regarded as OOD samples u ood , the OOD sample u ood Add OOD sample set U ood .

[0085] In step 3, the target data u can be determined ood The labeled data of the target data u ood Add the corresponding labeled data to the original dataset D id , so that the target data u ood Used for large model fine-tuning to alleviate the original dataset D id Therefore, the distribution-aware enhanced dataset D aug It can be expressed as

[0086] D aug =D id ∪D ood (5)

[0087] For the target data u ood Labeling can be done manually or with a large model such as GPT. The amount of out-of-distribution data that is filtered out is much smaller than that of the unlabeled dataset, so there is no need to label the entire unlabeled dataset, and only the filtered out-of-distribution data can be labeled.

[0088] In this embodiment, the global distance and local distance of the sample can reflect the relationship between the unlabeled data and the first data distribution from different angles. The unlabeled data whose global distance is greater than the first threshold d1 and whose nearest neighbor distance is greater than the second threshold d2 is taken as the target data, which can make OOD detection more accurate.

[0089] If the global distance of an unlabeled data is greater than the first threshold d1, it can also be added as the target data to the original data set D id If the nearest neighbor distance of an unlabeled data is greater than the second threshold d2, it can also be added as the target data to the original data set D id middle.

[0090] The above content describes in detail the data enhancement method provided in the embodiment. Using out-of-distribution data in unlabeled data to perform data enhancement on the data set used to fine-tune the large model can effectively improve the generalization ability of the large model when processing out-of-distribution data. In particular, this data enhancement can be continued intermittently, so as to continuously add out-of-distribution data to the data set and continuously enhance the generalization ability of the large model.

[0091] Combine the following Figure 3The multi-stage fine-tuning method provided in the embodiments of this specification is described.

[0092] Figure 3 A flow chart of a multi-stage large model fine-tuning method provided for an embodiment. The large model N to be fine-tuned includes a pre-trained backbone network and a prediction layer. The backbone network is used to extract sample features, obtain hidden states, and input the hidden states into the prediction layer. The hidden state is an intermediate feature of the input data or sample. The prediction layer is usually the last layer of the large model, which is related to a specific task and is used to convert the hidden state of the large model into a prediction result. The prediction layer can be a classification layer (i.e., a classification head or classifier), a regression layer (i.e., a regression head), or a layer of other types of tasks. When the prediction layer is a classification layer, its output result is a classification result, that is, a classification result of data or samples in a specific task, and the classification result can specifically be a classification probability or a classification score. When the prediction layer is a regression layer, its output result is a regression value, that is, a classification result of data or samples in a specific task. The backbone network is a pre-trained large model with good generalization ability. The prediction layer is a layer added to the end of the large model based on a specific task. Initially, the parameters in the prediction layer may be untrained parameters. This fine-tuning method is executed by a computing device and specifically includes the following steps.

[0093] Step S310, obtaining data set D.

[0094] The dataset D here can be the perceptually enhanced dataset D aug , or the original data set can be used. id , the fine-tuning method of this embodiment can also improve the generalization ability of the large model. aug When the number of datasets is increased, the generalization ability of large models can be improved from the two aspects of data set and fine-tuning.

[0095] Step S320, use the samples in the data set D to perform a first training on the large model N to obtain a first large model N1. The large model N here is the large model to be fine-tuned, including a backbone network and a prediction layer.

[0096] The first training includes: updating the parameters of the prediction layer in the large model N, and not updating the parameters of the backbone network in the large model N, that is, freezing the parameters of the backbone network. In other words, the fine-tuning performed in step S320 only trains the prediction layer using the data set D, and does not update the parameters in the backbone network.

[0097] Specifically, the first training includes: inputting the samples in the data set D into the large model N, processing them through the backbone network and the prediction layer in the large model N, outputting the prediction results, determining the prediction loss based on the difference between the prediction results and the labeled data of the samples, and updating the parameters of the prediction layer based on the prediction loss. The data set D may also include labeled data of the samples. Training the large model N using all the samples in the data set D can be used as a convergence condition for the first training. After the first training of the large model N using all the samples in the data set D, the first large model N1 is obtained. In the specific implementation, the parameter efficient fine-tuning method LoRA, BitFit or IA3 can be used to train the prediction layer.

[0098] Step S330, use the samples in the data set D to perform a second training on the large model to obtain a second large model N2. The large model here can be the large model N to be fine-tuned that has not undergone the first training, that is, the same as the large model before training in step S320; or it can be the first large model N1 after the first training, that is, the first large model N1 is used as the object of the second training, and the first large model N1 is trained for the second time using the samples in the data set D to obtain the second large model N2. That is, the prediction layer obtained by the first training can also be used to initialize the prediction layer of this stage.

[0099] The second training includes: updating the parameters of the backbone network and the parameters in the prediction layer in the large model, that is, fine-tuning all the parameters in the large model.

[0100] Specifically, the second training includes: inputting samples in the data set D into a large model (the large model can be N or N1. For the sake of simplicity in description, N is used as an example below), and after being processed by the backbone network and prediction layer in the large model N, the prediction result is output, and the prediction loss is determined based on the difference between the prediction result and the labeled data of the sample, and the parameters of the prediction layer are updated based on the prediction loss. Training the large model N using all samples in the data set D can be used as a convergence condition for the first training. After the second training of the large model N using all samples in the data set D, the second large model N2 is obtained. In specific implementation, the prediction layer can be trained using the parameter efficient fine-tuning methods LoRA, BitFit or IA3.

[0101] The large model training performed in steps S320 and S330 can be called different training stages. These two training stages or these two steps can be performed simultaneously, or step S320 can be performed first and then step S330. Both are feasible. The two stages mentioned here are not limited to a certain time sequence. After the two steps are executed, the parameters of the first large model N1 and the parameters of the second large model can be saved respectively for use in subsequent parameter merging.

[0102] Step S340, parameter values ​​are taken within the range of the parameters in the first large model N1 and the corresponding parameters in the second large model N2 to obtain a fine-tuned large model. The fine-tuned large model is the large model required in this embodiment and can also be used to perform specific tasks.

[0103] It can be understood that the parameters of the backbone network in the first large model N1 are the same as those in the large model N (because the backbone network parameters are not updated in the first training), and are not affected by the data set D corresponding to the specific task, so they maintain good generalization ability for out-of-distribution data. The parameters of the backbone network in the second large model N2 are different from those in the large model N. They have been affected by the data set D and have good prediction ability for in-distribution data. In order to achieve a performance balance between in-distribution data and out-of-distribution data, the embodiment of this specification adds step S340 after steps S320 and S330, that is, the parameters of the two large models obtained in multiple stages are merged, so that the obtained large model achieves a certain performance balance between in-distribution data and out-of-distribution data.

[0104] Since the two large models have exactly the same model structure and the same number of parameters, and the parameters of the two large models correspond one to one. That is, the parameters of one large model are the same as those of the other large model, but the parameter values ​​of the two large models may be different. When the parameter value of a certain parameter of the two large models is the same, the parameter value can be directly used as the parameter value of the fine-tuned large model. When the parameter value of a certain parameter of the two large models is different, the parameter value can be selected within the range of the two parameter values ​​of the two large models. After all the parameters are selected in this way, the fine-tuned large model is obtained.

[0105] The following is a detailed description of the method for parameter value selection. The method for parameter value selection within the range of two parameter values ​​may include interpolation, random value selection, or value selection based on experience, etc. The following is a detailed description of the interpolation method.

[0106] In one implementation, interpolation may be performed between parameters in the first large model N1 and corresponding parameters in the second large model N2 based on the interpolation coefficient α. The interpolation here may generally include linear interpolation and nonlinear interpolation.

[0107] Linear interpolation is used to estimate the coordinates of an unknown point between two known points based on their coordinates. The basic idea of ​​linear interpolation is to use a straight line to approximate a complex function curve between two known points. The basic idea of ​​nonlinear interpolation is to use a curve to approximate a complex function curve between two known points.

[0108] When interpolation is performed, a first weight of a parameter in the first large model N1 can be determined based on the interpolation coefficient α, and a second weight of a corresponding parameter in the second large model N2 can be determined, and interpolation is performed between the parameter in the first large model N1 and the corresponding parameter in the second large model N2 using the first weight and the second weight. The interpolation here can be linear interpolation.

[0109] For example, the interpolation coefficient α is a value between 0 and 1. The interpolation coefficient α can be used as the first weight, and 1-α can be used as the second weight to perform linear interpolation on the two parameter values. The formula is expressed as

[0110] φ=αφ ft +(1-α)φ lp (6)

[0111] Among them, φ is the parameter value obtained after linear interpolation, which is also the parameter value in the fine-tuned large model. ft is the parameter in the first largest model N1, φ lp is the corresponding parameter in the second largest model N2. The interpolation coefficient α may be a default value or a value set based on experience.

[0112] In order to use more accurate interpolation coefficients to interpolate parameters, the interpolation coefficient α may be set as a parameter to be trained, and the interpolation coefficient α may be optimized and updated in the third training.

[0113] That is to say, Figure 3 Based on the embodiment, the interpolation coefficient α can also be optimized and updated through a third stage of training in addition to the training stages corresponding to steps S320 and S330. Figure 2 The schematic diagram of the optimization α shown in the rightmost dashed box in the figure is used for explanation. In the above step S430, the step of interpolating between the parameters in the first large model N1 and the corresponding parameters in the second large model N2 based on the interpolation coefficient α can be specifically performed according to the following steps 4 to 6.

[0114] Step 4: Based on the value of the interpolation coefficient α, interpolate between the parameters in the first large model N1 and the corresponding parameters in the second large model N2 to obtain the third large model N3.

[0115] Step 5: Input the samples in the data set D into the third largest model N3 to obtain the prediction result, and optimize and update the value of the interpolation coefficient α based on the prediction result.

[0116] Step 6: When the optimal value of the interpolation coefficient α is determined, the third largest model N3 corresponding to the optimal value is used as the fine-tuned large model.

[0117] The above steps 4 and 5 can be performed by iterative training. Initially, the value of the interpolation coefficient α can be a random value or a value set based on experience. When step 4 is implemented, it can include determining the first weight of the parameter in the first large model N1 based on the interpolation coefficient α, and determining the second weight of the corresponding parameter in the second large model N2, and using the first weight and the second weight, interpolating between the parameters in the first large model N1 and the corresponding parameters in the second large model N2 to obtain the third large model N3.

[0118] In the above step 5, the samples in the data set D are input into the third largest model N3, and the prediction results are obtained after being processed by the backbone network and the prediction layer in the third largest model N3. The prediction loss is determined based on the difference between the prediction result and the labeled data of the sample, and the value of the interpolation coefficient α is optimized and updated based on the prediction loss. The value of the interpolation coefficient α is optimized and updated using multiple samples in the data set D until the above difference is less than the preset minimum value.

[0119] When the above difference is smaller than the preset value, it is considered that the value of the interpolation coefficient α is a more optimal value, and the third largest model N3 corresponding to the optimal value α can be used as the fine-tuned large model.

[0120] This embodiment optimizes the value of the interpolation coefficient α through iterative training, and selects the third largest model N3 corresponding to the more optimized interpolation coefficient α as the fine-tuned large model, which can enable the large model to achieve a good balance between in-distribution data and out-of-distribution data, thereby improving the generalization ability of the large model.

[0121] Combine the following Figure 4 The design scheme of the loss function provided in the embodiment is described in detail.

[0122] Take the prediction layer as a binary classifier as an example. The label of any sample x includes the binary label in the binary classification task and the label distribution value reflecting the classification level. The classification in the binary classification task includes the first classification and the second classification. The label distribution value includes the probability value on the first classification and the probability value on the second classification.

[0123] Figure 4 A schematic diagram of two labels and prior distribution q(x) for a sample x. The sample contains the query statement q i And multiple searched items i The corresponding relationship between them. i The binary classification labels between item1, item2, item3 and item4 are relevant, irrelevant, irrelevant and relevant respectively. iThey constitute different samples with item1, item2, item3 and item4 respectively. The classification levels of the binary classification labels of these samples are strong correlation, weak correlation, irrelevant and strong correlation respectively. The label distribution value corresponding to strong correlation is [0.1, 0.9], the label distribution value corresponding to weak correlation is [0.2, 0.8], and the label distribution value corresponding to irrelevant is [0.9, 0.1]. Among them, the numbers on the left in the brackets represent the probability of irrelevance, and the numbers on the right represent the probability of relevance. Each classification level is represented by the label distribution value, which will be explained in detail later.

[0124] This example corresponds to the task of predicting search relevance. The goal of the search relevance task is to predict the query statement q i With any item i In this task, the binary classification labels usually include two categories: relevant and irrelevant, and the corresponding label values ​​are y=1 and y=0 respectively. The binary classification label value of sample x may be 0 or 1. The classification levels of correlation can include strong correlation (SR), weak correlation (WR) and irrelevant (IR).

[0125] That is to say, the embodiment sets two labels for each sample, one is the original binary label, and the other is a label distribution reflecting the classification level of the binary label. The label level or label distribution is used to more finely represent the classification of the sample.

[0126] It should be noted that in the task of search relevance prediction (or evaluation), in order to distinguish the relevance between different queries and products (or projects) and improve user experience, multiple fine-grained relevance levels are usually required to represent the relevance, such as strong relevance, weak relevance or irrelevant, rather than just a simple "yes (relevant)" or "no (irrelevant)" answer. In other words, in practical applications, the goal of the big model in the specific task of search relevance is that the score of the relevance classification score output by the big model for the sample should be able to reflect the different classification levels of relevance, that is, different scores represent different relevance levels, and accurately define the fine-grained relationship in the search. Similarly, in other similar specific tasks that require more refined classification, there is also such a demand, that is, the score of the classification score output by the big model for the sample should be able to reflect the different classification levels of classification.

[0127] Next, we will continue to use the specific task of search relevance prediction as an example to explain why classification scores need to reflect different classification levels. Search engines are designed to help users quickly and accurately locate goods, services, or other products that meet their needs. These goods, services, or other products can be collectively referred to as searched items. Relevance modeling plays a central role. Relevance modeling is achieved by adjusting the large model to predict the relevance between queries and products through data sets and fine-tuning. The goal of relevance modeling is to ensure that the returned results are highly consistent with user intent through deep understanding and matching of queries and products, thereby improving the user's search experience. Although large models perform well in understanding general world knowledge, they cannot fully grasp the nuances of knowledge and concepts in the specific task of relevance modeling. Therefore, conventional fine-tuning strategies that lack targeted adjustments will cause large models to provide biased or overconfident classification scores when distinguishing fine-grained relevance levels. At the same time, when faced with changes in data distribution in real-world applications, large models after conventional fine-tuning often suffer significant performance degradation.

[0128] In order to solve this problem, this embodiment sets a label distribution that reflects the classification level for the sample during the fine-tuning stage of the large model, and uses the difference between the label distribution value of the sample and the classification score predicted by the large model to adjust the parameters of the large model, thereby guiding the large model to output a classification score that can reflect the refined classification level, which undoubtedly improves the accuracy of the large model prediction results. When the classification score can reflect the refined classification level, the classification score can play a variety of important roles. One is that the classification score output by the large model can be converted into a classification level, such as a relevance level; the other is that the classification score can be used as a key sorting criterion for the subsequent sorting layer.

[0129] The following describes how to convert the classification level of a sample into a label distribution value. The label distribution value can represent the label distribution. The classification level of a sample can be set manually or by large model prediction, and the label distribution value can be determined based on the prior distribution q(x). The prior distribution q(x) containing three classification levels of strong correlation (SR), weak correlation (WR) and irrelevant (IR) can be expressed by the following formula:

[0130]

[0131] Here, ρ is a distribution smoothing factor, and its default value can be 0.1, 0.15 or other values.

[0132] From the above example of prior distribution q(x), we can know that the prior distribution contains label distribution values ​​corresponding to multiple classification levels. The label distribution value can be determined by the distribution smoothing factor ρ and the preset formula. The factor ρ can be determined by grid parameter search (Grid Search) or manually set by expert experience.

[0133] When the large model outputs a classification score, the classification score can also be converted into a classification level through the above-mentioned prior distribution q(x).

[0134] The following describes in detail a specific embodiment of adjusting the parameters of the large model using the difference between the label distribution value of the sample and the classification score predicted by the large model.

[0135] In step S320, the first training of the large model N is performed using samples in the data set D, which can be specifically implemented by following steps 7 to 9.

[0136] Step 7: Input the samples in the dataset D into the large model N, and determine the predicted probability distribution of the samples in the binary classification task through the classifier contained in the large model N. The predicted probability distribution is also the classification score. For example, the predicted probability distribution is [0.2, 0.8], where 0.2 is the probability score that the sample is irrelevant, and 0.8 is the probability score that the sample is relevant.

[0137] Step 8: Determine the distribution loss L based on the distribution difference between the predicted probability distribution and the label distribution value of the sample. kl . In determining the distribution loss L kl When , it can be obtained based on KL divergence.

[0138] Step 9, based on the distribution loss L kl The large model N is first trained.

[0139] In this step, the distribution loss can be used alone to update the parameters of the classifier in the large model N, or the multivariate loss can be used to update the parameters. For example, the classification loss L can be determined based on the difference between the binary classification label and the corresponding label prediction value of the sample in the binary classification task. ce , based on the distribution loss L kl and classification loss L ce First train the large model. Classification loss L ce It can be determined by the cross entropy function for binary classification tasks.

[0140] For example, the binary label of sample x is y=1, that is, sample x is a relevant sample. The probability of relevance in the label prediction value determined by the large model is 0.8, so the classification loss can be determined based on the difference between 1 and 0.8.

[0141] The embodiment introduces an auxiliary loss L based on KL divergence kl To adjust the distribution of predicted values, the total loss can be expressed as

[0142] Among them, L kl =KL(p(x),q(x)) is the KL divergence, which is used to quantify the difference between the model's predicted distribution p(x) and the prior distribution q(x). The predicted distribution p(x) is the predicted probability distribution determined by the large model. ce is the cross entropy loss for the binary classification task of relevance scoring.

[0143] Similarly, when the samples in the data set D are used to perform the second training on the large model in step S330, the multivariate loss combining the distribution loss and the classification loss can also be used for training. Specifically, it includes: inputting the samples in the data set D into the large model, determining the predicted probability distribution of the samples in the binary classification task through the classifier contained in the large model, determining the distribution loss based on the distribution difference between the predicted probability distribution and the label distribution value of the sample, determining the classification loss based on the difference between the binary classification label and the label prediction value corresponding to the sample in the binary classification task, and performing the second training on the large model based on the distribution loss and the classification loss, that is, updating the parameters in the backbone network and the classifier.

[0144] Similarly, in step 5 included in step S340, that is, inputting the samples in the data set D into the third largest model N3 to obtain the prediction results, and optimizing and updating the value of the interpolation coefficient α based on the prediction results, the multivariate loss combined with the classification loss can also be used for training. Specifically, it includes: inputting the samples in the data set D into the third largest model N3, determining the predicted probability distribution of the samples in the binary classification task through the classifier included in the third largest model N3, determining the distribution loss based on the distribution difference between the predicted probability distribution and the label distribution value of the sample, determining the classification loss based on the difference between the binary classification label and the label prediction value corresponding to the sample in the binary classification task, and optimizing and updating the interpolation coefficient α based on the distribution loss and the classification loss.

[0145] Figure 5It is a comparative schematic diagram of the statistical results of the prediction results of the large model. Among them, Figure (a) is a statistical schematic diagram of the prediction results of the large model under the conventional fine-tuning strategy, and Figure (b) is a statistical schematic diagram of the prediction results of the large model under the fine-tuning strategy improved in the embodiment. The horizontal axis is the predicted classification score, i.e., the probability value, and the vertical axis is the number of samples. As can be seen from Figure (a), the classification scores of the weakly correlated samples of the large model are mainly concentrated at the two extremes of the [0,1] interval, resulting in a significant overlap with the classification scores of the samples considered to be strongly correlated. Classification scores are also called correlation scores, classification probabilities, or prediction scores, etc. As can be seen from Figure (b), after the large model is fine-tuned using the method of this embodiment, the classification scores of the weakly correlated samples obviously move toward the endpoint of 0, and the classification scores of the strongly correlated samples obviously move toward the endpoint of 1. The classification scores can be used to more accurately divide the classification levels of the samples.

[0146] The above content describes the embodiment in detail from aspects such as data enhancement, multi-stage large model fine-tuning and loss function design. Figure 2 The architectural diagrams shown illustrate various embodiments.

[0147] Figure 2 The left part shows the process framework of enhancing the data set based on distribution awareness. Figure 2 The right part of the diagram shows that the large model is trained in multiple stages using samples from the enhanced dataset, such as samples from the search relevance task. This includes a training stage for updating the prediction layer, a training stage for updating all parameters, and a training stage for optimizing the interpolation coefficient α. In each training stage, the loss function used in the fine-tuning process of the large model uses the distribution loss L kl With the classification loss L ce The combined approach achieves calibration of overconfidence.

[0148] In summary, the embodiments of this specification propose a distribution-aware robust learning framework, which improves the modeling based on large models in specific tasks by designing distribution-aware sample enhancement and effective loss functions, and adopts a multi-stage fine-tuning strategy to improve the generalization ability of the model under different data distributions. Distribution-aware sample enhancement can be implemented in conjunction with the multi-stage fine-tuning strategy, or it can be implemented separately, which can improve the generalization ability of the large model to a certain extent. The combined implementation of distribution-aware sample enhancement, distribution loss and multi-stage fine-tuning strategy can further improve the generalization ability of the large model and reduce the prediction difference between the large model on out-of-distribution data and in-distribution data.

[0149] In this specification, the word "first" in terms such as the first large model, the first training, the first weight, etc., and the corresponding "second" (if any) in the text, etc., are merely for the convenience of distinction and description and do not have any limiting meaning.

[0150] In this specification, the computing device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities.

[0151] The foregoing describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0152] Figure 6 A schematic block diagram of a multi-stage large model fine-tuning device provided in an embodiment. The large model to be fine-tuned includes a pre-trained backbone network and a prediction layer. Figure 2 The device 600 includes:

[0153] A data set acquisition module 610 is configured to acquire a data set;

[0154] A first training module 620 is configured to perform a first training on the large model using samples in the data set to obtain a first large model; the first training includes: updating parameters of the prediction layer in the large model, and not updating parameters of the backbone network in the large model;

[0155] The second training module 630 is configured to perform a second training on the large model using the samples in the data set to obtain a second large model; the second training includes: updating the parameters of the backbone network and the parameters in the prediction layer in the large model;

[0156] The parameter value taking module 640 is configured to take parameter values ​​within a range formed by the parameters in the first large model and the corresponding parameters in the second large model to obtain a fine-tuned large model.

[0157] In one embodiment, the parameter value taking module 640 is specifically configured to interpolate between the parameters in the first large model and the corresponding parameters in the second large model based on the interpolation coefficients.

[0158] In one embodiment, the parameter value module 640 includes: a weight submodule and an execution submodule (not shown in the figure). The weight submodule is configured to determine a first weight of a parameter in the first large model based on an interpolation coefficient, and to determine a second weight of a corresponding parameter in the second large model. The execution submodule is configured to interpolate between the parameter in the first large model and the corresponding parameter in the second large model using the first weight and the second weight.

[0159] In one embodiment, the interpolation coefficient is a parameter to be trained. The parameter value module 640 includes: an interpolation submodule 641, an optimization submodule 642 and a fine-tuning submodule 643. The interpolation submodule 641 is configured to interpolate between the parameters in the first large model and the corresponding parameters in the second large model based on the value of the interpolation coefficient to obtain a third large model. The optimization submodule 642 is configured to input the samples in the data set into the third large model to obtain a prediction result, and optimize and update the value of the interpolation coefficient based on the prediction result. The fine-tuning submodule 643 is configured to use the third large model corresponding to the optimized value as the fine-tuned large model when determining the optimized value of the interpolation coefficient.

[0160] In one implementation, the data set acquisition module 610 includes: an acquisition submodule 611, an identification submodule 612, and an addition submodule 613;

[0161] The acquisition submodule 611 is configured to acquire an original data set; acquire a number of unlabeled data;

[0162] An identification submodule 612 is configured to identify target data outside the first data distribution from a plurality of unlabeled data based on the first data distribution of the original data set;

[0163] The adding submodule 613 is configured to add the target data to the original data set to obtain the data set.

[0164] In one embodiment, the identification submodule 612 includes: a first determination unit, a second determination unit, and a third determination unit (not shown in the figure). The first determination unit is configured to determine the global distance between any unlabeled data and the first data distribution. The second determination unit is configured to determine the nearest neighbor distance between any unlabeled data and the sample in the original data set. The third determination unit is configured to determine the unlabeled data as the target data when the global distance of the unlabeled data is greater than a first threshold and the nearest neighbor distance of the unlabeled data is greater than a second threshold.

[0165] In one implementation, the adding submodule 613 is specifically configured to: determine the labeling data of the target data, and add the target data and the labeling data to the original data set correspondingly.

[0166] In one embodiment, the second training module 630 is specifically configured as follows: taking the first large model as the object of the second training, performing the second training on the first large model using the samples in the data set, and obtaining the second large model.

[0167] In one embodiment, the prediction layer is a binary classification classifier; the label of the sample includes the binary classification label in the binary classification task and the label distribution value reflecting the classification level; the classification in the binary classification task includes the first classification and the second classification, and the label distribution value includes the probability value on the first classification and the probability value on the second classification; the first training module 620 includes: a prediction submodule 621, a determination submodule 622 and a training submodule 623. The prediction submodule 621 is configured to input the sample in the data set into the large model, and determine the predicted probability distribution of the sample in the binary classification task through the classifier included in the large model. The determination submodule 622 is configured to determine the distribution loss based on the distribution difference between the predicted probability distribution and the label distribution value. The training submodule 623 is configured to perform a first training on the large model based on the distribution loss.

[0168] In one embodiment, the device 600 also includes a distribution determination submodule (not shown in the figure), which is configured to determine the label distribution value of the sample in the following manner: determine the label distribution value corresponding to the classification level of the sample based on a prior distribution, and the prior distribution includes label distribution values ​​corresponding to multiple classification levels.

[0169] In one embodiment, the training submodule 623 includes: a loss unit and a training unit (not shown in the figure). The loss unit is configured to determine the classification loss based on the difference between the binary classification label and the label prediction value corresponding to the sample in the binary classification task. The training unit is configured to perform a first training on the large model based on the distribution loss and the classification loss.

[0170] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0171] The present specification also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute Figures 1 to 5 Any of the methods described above.

[0172] The embodiment of the present specification also provides a computing device, including a memory and a processor, wherein the memory stores an executable code, and when the processor executes the executable code, Figures 1 to 5 Any of the methods described above.

[0173] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0174] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0175] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A multi-stage large model fine-tuning method, wherein the large model to be fine-tuned includes a pre-trained backbone network and a prediction layer, and the method comprises: Get the dataset; Performing a first training on the large model using samples in the data set to obtain a first large model; The first training includes: updating parameters of the prediction layer in the large model, and not updating parameters of the backbone network in the large model; Performing a second training on the large model using samples in the data set to obtain a second large model; the second training comprises: updating parameters of the backbone network and parameters in the prediction layer in the large model; Parameter values ​​are taken within a range formed by parameters in the first large model and corresponding parameters in the second large model to obtain a fine-tuned large model.

2. According to the method of claim 1, the step of taking parameter values ​​within the range formed by the parameters in the first large model and the corresponding parameters in the second large model comprises: Based on the interpolation coefficients, interpolation is performed between the parameters in the first large model and the corresponding parameters in the second large model.

3. The method according to claim 2, wherein the step of interpolating comprises: Determining first weights of parameters in the first large model based on the interpolation coefficients, and determining second weights of corresponding parameters in the second large model; Interpolation is performed between parameters in the first large model and corresponding parameters in the second large model using the first weight and the second weight.

4. According to the method of claim 2, the interpolation coefficient is a parameter to be trained; the step of interpolating between the parameters in the first large model and the corresponding parameters in the second large model based on the interpolation coefficient comprises: Based on the values ​​of the interpolation coefficients, interpolation is performed between the parameters in the first large model and the corresponding parameters in the second large model to obtain a third large model; Inputting samples in the data set into the third largest model to obtain prediction results, and optimizing and updating the values ​​of the interpolation coefficients based on the prediction results; When determining the optimal value of the interpolation coefficient, the third largest model corresponding to the optimal value is used as the fine-tuned large model.

5. The method according to claim 1, wherein the step of acquiring the data set comprises: Get the original data set; Get some unlabeled data; Based on a first data distribution of the original data set, identifying target data outside the first data distribution from a plurality of unlabeled data; The target data is added to the original data set to obtain the data set.

6. The method according to claim 5, wherein the step of identifying target data outside the first data distribution from a plurality of unlabeled data comprises: Determine a global distance between any unlabeled data and the first data distribution; Determine the nearest neighbor distance between any unlabeled data and the samples in the original data set; When the global distance of the unlabeled data is greater than a first threshold, and the nearest neighbor distance of the unlabeled data is greater than a second threshold, the unlabeled data is determined as the target data.

7. The method according to claim 5, wherein the step of adding the target data to the original data set comprises: Determine the labeled data of the target data, and add the target data and the labeled data to the original data set accordingly.

8. The method according to claim 1, wherein the step of performing a second training on the large model using samples in the data set comprises: The first large model is used as an object of second training, and the samples in the data set are used to perform second training on the first large model to obtain a second large model.

9. According to the method of claim 1, the prediction layer is a binary classification classifier; the label of the sample includes a binary classification label in the binary classification task and a label distribution value reflecting the classification level; the classification in the binary classification task includes a first classification and a second classification, and the label distribution value includes a probability value on the first classification and a probability value on the second classification; The step of performing a first training on the large model using samples in the data set comprises: Input the samples in the data set into the large model, and determine the predicted probability distribution of the samples in the binary classification task through the classifier included in the large model; Determining a distribution loss based on a distribution difference between the predicted probability distribution and the label distribution value; The large model is first trained based on the distribution loss.

10. According to the method of claim 9, the label distribution value of the sample is determined in the following manner: A label distribution value corresponding to the classification level of the sample is determined based on a prior distribution, where the prior distribution includes label distribution values ​​corresponding to multiple classification levels.

11. The method according to claim 9, wherein the step of first training the large model based on the distribution loss comprises: Determining a classification loss based on a difference between the binary classification label and a label prediction value corresponding to the sample in the binary classification task; The large model is first trained based on the distribution loss and the classification loss.

12. A multi-stage large model fine-tuning device, wherein the large model to be fine-tuned includes a pre-trained backbone network and a prediction layer, and the device comprises: A data set acquisition module, configured to acquire a data set; A first training module is configured to perform a first training on the large model using samples in the data set to obtain a first large model; The first training includes: updating parameters of the prediction layer in the large model, and not updating parameters of the backbone network in the large model; A second training module is configured to perform a second training on the large model using samples in the data set to obtain a second large model; the second training includes: updating parameters of the backbone network and parameters in the prediction layer in the large model; The parameter value taking module is configured to take parameter values ​​within a range formed by the parameters in the first large model and the corresponding parameters in the second large model to obtain a fine-tuned large model.

13. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 11.

14. A computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 11 is implemented.

Images