A model determination method, apparatus, and computer-readable storage medium

Abstract

This application provides a model determination method, apparatus, and computer-readable storage medium. The method includes: acquiring multiple training samples corresponding to M training tasks, where M ≥ 2; determining the dataset features corresponding to the multiple training samples corresponding to the i-th training task through the feature extraction sub-model included in the initial multi-scene model, where M ≥ i ≥ 1; determining the prediction results corresponding to the multiple training samples corresponding to the i-th training task through the prediction sub-model included in the initial multi-scene model based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples; generating the sub-model loss function corresponding to the i-th training task; and performing inner and outer loops of training the initial multi-scene model through the sub-model loss functions corresponding to the M training tasks to obtain the multi-scene model. This method can improve the accuracy of multi-scene models applied to multiple application scenarios.

Description

Technical Field

[0001] This application belongs to the field of data processing, and specifically relates to a model determination method, apparatus and computer-readable storage medium. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this application as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] Humans possess the ability to quickly adapt to new scenarios based on prior knowledge. For example, an infant can recognize a cat or dog after seeing it only a few times, and learn to run with limited walking practice. In contrast, models typically require a large number of samples to learn for specific application scenarios and lack corresponding generalization ability.

[0004] To address this, Model-Agnostic Meta-Learning (MAML) has been proposed to compensate for the shortcomings of models requiring a large number of samples for training and having poor generalization ability. Specifically, MAML does not improve the model's ability in a specific application scenario through a large number of samples, but rather improves the model's ability in a range of application scenarios through a small number of samples from different datasets. For example, compared with a speech recognition model that can only be applied to speech recognition or an image recognition model that can only be applied to image recognition, MAML can obtain a multi-scenario recognition model that can be applied to a range of application scenarios such as speech recognition and image recognition by training with a small number of samples from different datasets.

[0005] Although MAML can be used to obtain multi-scenario models applicable to a range of application scenarios, the performance of these models in specific application scenarios is often poor because they require universal initialization across multiple scenarios. For example, the accuracy of a multi-scenario recognition model obtained through MAML may not be high when applied to the specific application scenario of speech recognition. In other words, multi-scenario models obtained through MAML have certain limitations when applied to multiple application scenarios.

[0006] Therefore, improving the accuracy of multi-scenario models that can be applied to multiple application scenarios is an urgent problem to be solved. Summary of the Invention

[0007] To address the problems existing in the prior art, a model determination method, apparatus, and computer-readable storage medium are proposed, which can solve the aforementioned problems.

[0008] This application provides the following solutions.

[0009] Firstly, a model determination method is provided, including:

[0010] Obtain multiple training samples corresponding to M training tasks, where M is an integer and M≥2; for the i-th training task among the M training tasks, the multiple training samples corresponding to the i-th training task correspond to multiple training datasets, and the multiple training datasets correspond one-to-one with multiple application scenarios, where i is an integer and M≥i≥1;

[0011] Multiple training samples corresponding to the i-th training task are input into the initial multi-scene model. The feature extraction sub-model in the initial multi-scene model is used to determine the dataset features corresponding to the multiple training samples of the i-th training task. The dataset features are used to represent the correlation between the corresponding training sample and the multiple training datasets.

[0012] Based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples respectively, the prediction results corresponding to the multiple training samples corresponding to the i-th training task are determined through the prediction sub-model in the initial multi-scene model.

[0013] Based on the differences between the prediction results and sample labels of the multiple training samples corresponding to the i-th training task, a sub-model loss function corresponding to the i-th training task is generated.

[0014] The initial multi-scene model is trained through inner and outer loops using the sub-model loss functions and outer loop learning rates corresponding to the M training tasks, thereby obtaining the multi-scene model.

[0015] Optionally, when the prediction sub-model includes K convolutional layers, where K is an integer and K≥2, the step of determining the prediction results corresponding to the multiple training samples corresponding to the i-th training task based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples respectively, through the prediction sub-model in the initial multi-scene model, includes:

[0016] For the first convolutional layer among the K convolutional layers, the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples are input into the first convolutional layer. Through the first convolutional layer, the first layer prediction sub-results corresponding to the multiple training samples corresponding to the i-th training task are determined.

[0017] For the k-th convolutional layer among the K convolutional layers, the (k-1)-th layer prediction sub-results corresponding to the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple samples are input into the k-th convolutional layer. Through the k-th convolutional layer, the k-th layer prediction sub-results corresponding to the multiple training samples corresponding to the i-th training task are determined; where k is an integer, K≥k≥2;

[0018] The Kth layer prediction sub-results corresponding to the multiple training samples corresponding to the i-th training task are respectively used as the prediction results corresponding to the multiple training samples corresponding to the i-th training task.

[0019] Optionally, the feature extraction sub-model and the prediction sub-model have the same model architecture.

[0020] Optionally, the step of determining the prediction results corresponding to the multiple training samples corresponding to the i-th training task through the prediction sub-model in the initial multi-scene model, based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples respectively, includes:

[0021] The dataset features corresponding to the multiple training samples are input into the bidirectional long short-term memory network in the initial multi-scenario model. The bidirectional long short-term memory network is used to determine the memory dataset features corresponding to the multiple training samples. The memory dataset features contain more information than the corresponding dataset features.

[0022] Based on the features of the multiple training samples corresponding to the i-th training task and the memory datasets corresponding to the multiple training samples, the prediction results corresponding to the multiple training samples corresponding to the i-th training task are determined by the prediction sub-model in the initial multi-scene model.

[0023] Optionally, the step of training the initial multi-scene model using the sub-model loss function and outer loop learning rate corresponding to the M training tasks to obtain the multi-scene model includes:

[0024] The inner loop of training the initial multi-scene model is performed using the sub-model loss function corresponding to the i-th training task to obtain the model update parameters corresponding to the i-th training task;

[0025] Based on the model update parameters and outer loop learning rate corresponding to the M training tasks, the outer loop of the initial multi-scene model is trained to obtain the multi-scene model.

[0026] Optionally, the inner loop of training the initial multi-scene model using the sub-model loss function corresponding to the i-th training task to obtain the model update parameters corresponding to the i-th training task includes:

[0027] The inner loop of training the initial multi-scene model using the sub-model loss function corresponding to the i-th training task yields multiple model update sub-parameters corresponding to the i-th training task; the multiple model update sub-parameters corresponding to the i-th training task correspond one-to-one with the multi-step update corresponding to the i-th training task.

[0028] The outer loop training the initial multi-scene model based on the model update parameters and outer loop learning rate corresponding to the M training tasks, to obtain the multi-scene model, includes:

[0029] Based on the multiple model update sub-parameters and outer loop learning rate corresponding to the M training tasks, the outer loop of the initial multi-scene model is trained to obtain the multi-scene model.

[0030] Optionally, in the inner loop of training the initial multi-scene model using the sub-model loss function corresponding to the i-th training task, the method further includes:

[0031] Configure corresponding sample weights for the multi-step updates corresponding to the i-th training task.

[0032] Optionally, configuring corresponding sample weights for the multiple training samples corresponding to the i-th training task includes:

[0033] At the beginning of the inner loop of the training of the initial multi-scene model, the corresponding initial sample weights are configured for the multi-step updates corresponding to the i-th training task; the values ​​of the multiple initial sample weights corresponding to the i-th training task are the same.

[0034] During the inner loop of training the initial multi-scene model, the weights of multiple initial samples corresponding to the i-th training task are adjusted; specifically, for the start update step in the multi-step update corresponding to the i-th training task, the weights of the initial samples corresponding to the start update step of the i-th training task are decreased, and for the subsequent update steps in the multi-step update corresponding to the i-th training task, the weights of the initial samples corresponding to the subsequent update steps of the i-th training task are increased.

[0035] Optionally, the method further includes:

[0036] During the outer loop of training the initial multi-scene model, the outer loop learning rate is annealed; wherein the outer loop learning rate decreases as the outer loop of training the initial multi-scene model progresses.

[0037] Secondly, a model determining apparatus is provided, comprising:

[0038] An acquisition unit is used to acquire multiple training samples corresponding to M training tasks, where M is an integer and M≥2; for the i-th training task among the M training tasks, the multiple training samples corresponding to the i-th training task have corresponding sample labels, the multiple training samples corresponding to the i-th training task correspond to multiple training datasets, the multiple training datasets correspond one-to-one with multiple application scenarios, and the multiple application scenarios have the same scenario type, where i is an integer and M≥i≥1;

[0039] The first determining unit is used to input multiple training samples corresponding to the i-th training task into an initial multi-scene model, and determine the dataset features corresponding to the multiple training samples corresponding to the i-th training task through the feature extraction sub-model in the initial multi-scene model; the dataset features are used to represent the correlation between the corresponding training sample and the multiple training datasets.

[0040] The second determining unit is used to determine the prediction results corresponding to the multiple training samples corresponding to the i-th training task based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples respectively, through the prediction sub-model in the initial multi-scene model.

[0041] The generation unit is used to generate a sub-model loss function corresponding to the i-th training task based on the difference between the prediction results and the corresponding sample labels of the multiple training samples corresponding to the i-th training task.

[0042] The first training unit is used to train the initial multi-scene model through the sub-model loss function corresponding to the i-th training task in an inner loop, so as to obtain the model update parameters corresponding to the i-th training task.

[0043] The second training unit is used to train the initial multi-scenario model in the outer loop according to the model update parameters and outer loop learning rate corresponding to the M training tasks respectively, so as to obtain the multi-scenario model, which is applied to the multiple application scenarios.

[0044] Thirdly, a model determining apparatus is provided, comprising:

[0045] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform: the method described above.

[0046] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a program that, when executed by a multi-core processor, causes the multi-core processor to perform the method described above.

[0047] One advantage of the above embodiments is that, in order to obtain a multi-scenario model applicable to multiple application scenarios with the same scenario type, multiple training samples corresponding to M training tasks are obtained, where M is an integer and M≥2. Taking the i-th training task among the M training tasks as an example, where i is an integer and M≥i≥1, the multiple training samples corresponding to the i-th training task do not come from a single training dataset, but from multiple training datasets that correspond one-to-one with multiple application scenarios. The multiple training samples corresponding to the i-th training task are input into the initial multi-scenario model for training and optimization. The initial multi-scenario model includes a feature extraction sub-model and a prediction sub-model. The feature extraction sub-model can first determine the dataset features corresponding to the multiple training samples corresponding to the i-th training task. The dataset features are used to represent the correlation between the corresponding training sample and multiple training datasets, that is, the dataset features can represent the correlation between the training sample and the multiple training datasets. The most relevant training dataset for the corresponding training samples is selected to determine the application scenario corresponding to each training sample. Then, based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to each of these training samples, a prediction sub-model determines the prediction results for each of the multiple training samples corresponding to the i-th training task. Since the prediction sub-model considers not only the training samples but also the dataset features representing the application scenario of the training samples during the prediction process, it can increase the accuracy of the prediction results to a certain extent. Based on the differences between the prediction results and the corresponding sample labels for the multiple training samples corresponding to the i-th training task, a sub-model loss function for the i-th training task can be generated. The initial multi-scenario model is then trained using the sub-model loss functions for each of the M training tasks and the outer loop learning rate through inner and outer loops to obtain the multi-scenario model. During the training process of the initial multi-scenario model, including the feature extraction sub-model and the prediction sub-model, the initial multi-scenario model learns how to identify the application scenario corresponding to the training samples, enabling the obtained multi-scenario model to be better applied to specific application scenarios in multiple application scenarios, thus improving the accuracy of the multi-scenario model applied to multiple application scenarios.

[0048] Other advantages of this application will be explained in more detail with reference to the following description and figures.

[0049] It should be understood that the above description is merely an overview of the technical solution of this application, so as to enable a clearer understanding of the technical means of this application and thus allow for its implementation in accordance with the contents of the specification. To make the above and other objects, features, and advantages of this application more apparent and understandable, specific embodiments of this application are illustrated below. Attached Figure Description

[0050] By reading the detailed description of the exemplary embodiments below, those skilled in the art will understand the advantages and benefits described herein, as well as other advantages and benefits. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0051] Figure 1 A comparison diagram of MAML and pre-trained MAML provided for embodiments of this application.

[0052] Figure 2 An accuracy verification graph for MAML training is provided for an embodiment of this application.

[0053] Figure 3 This is a schematic diagram of a hardware operating environment provided in an embodiment of this application.

[0054] Figure 4 This is a flowchart illustrating a model determination method provided in an embodiment of this application.

[0055] Figure 5 This is a model structure diagram of an initial multi-scene model provided in an embodiment of this application.

[0056] Figure 6 This is a schematic diagram of multiple datasets provided in an embodiment of this application.

[0057] Figure 7 A comparison chart of the accuracy variance of MMAML and MMAML+ provided for embodiments of this application.

[0058] Figure 8 This diagram illustrates an application scenario of a model determination method provided in this application.

[0059] Figure 9 This is a schematic diagram of a model determination device provided in an embodiment of this application.

[0060] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0061] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0062] In the description of embodiments of this application, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of the disclosed features, figures, steps, behaviors, components, portions or combinations thereof in this specification, and do not exclude the possibility of the presence of one or more other features, figures, steps, behaviors, components, portions or combinations thereof.

[0063] Unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In this article, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A alone, A and B at the same time, and B alone.

[0064] The terms "first," "second," etc., are used only for ease of description to distinguish identical or similar technical features and should not be construed as indicating or implying the relative importance or number of these technical features. Therefore, a feature defined by "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, unless otherwise stated, the term "multiple" means two or more.

[0065] It should also be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0066] In related technologies, the basic training method usually uses a loss function to supervise the learning of the model, and the corresponding formula is shown below:

[0067] .

[0068] in, This represents the parameters after training. This represents the parameters during the training process. Let X represent the training data, and let X represent the input feature vector. y Indicates the corresponding label, Represents the loss function. This represents the corresponding model parameters.

[0069] The model can be trained using the above formula. Specifically, the training data is usually in the form of data points. The data is input into the initial model in batches, with each data point containing an input feature vector x and a corresponding label y. The initial model learns using this training data. The training process optimizes the model parameters based on the input-output relationship observed in the training data, thereby minimizing the loss function in the formula. This method can be applied to linear regression, support vector machines (SVM), and decision trees, among others.

[0070] Although the models in related technologies can be applied to specific application scenarios, compared with humans who can quickly learn new skills based on prior knowledge, the models usually require a large number of samples to be trained before they can adapt to specific application scenarios, and lack corresponding generalization ability. Specific defects include:

[0071] 1. Data Greed: Requires a large amount of labeled data; models typically require a significant amount of labeled data for training. This means that manually labeling the dataset is necessary, which can be time-consuming, expensive, and impractical. Furthermore, when applied to new application scenarios, it is often necessary to collect and label new data again, thus limiting the model's applicability and scalability.

[0072] 2. Lack of generalization ability: The model usually performs poorly when applied to new application scenarios. This is because the model often learns specific abilities by adjusting parameters on a large amount of training data. This leads to the model lacking the ability to share knowledge and transfer learning between different application scenarios. In other words, the model is difficult to apply to new application scenarios.

[0073] 3. Slow training and high time cost: Different application scenarios usually require a large amount of training data for independent training, and the training time is usually long.

[0074] 4. Lack of flexibility and adaptability: The model is usually static and cannot be dynamically adjusted according to changes in the application scenario. That is, the model lacks self-learning and adaptive capabilities and requires manual parameter adjustment and optimization.

[0075] To address the shortcomings of data greed, lack of generalization ability, slow training, high time cost, and lack of flexibility and adaptability, pre-training has been proposed as a solution. Pre-training involves initial training on multiple large-scale datasets before fine-tuning on the target application scenario. This involves learning features common to different application scenarios from multiple large-scale datasets to adjust model parameters, and then transferring this learned knowledge to the target application scenario. Specifically, pre-training may include the following steps:

[0076] 1. Initial training: Initial training can be performed using the following formula:

[0077]

[0078] in, Let P represent P large-scale datasets, and p represent the p-th large-scale dataset among the P large-scale datasets. This indicates that training is performed using the p-th large-scale dataset.

[0079] The above formula can be used to train on P large-scale datasets. For example, it can be trained through self-supervised learning, unsupervised learning, or supervised learning. After the initial training phase, a model with good parameters that performs well on multiple large-scale datasets will be obtained.

[0080] 2. Fine-tuning: After initial training, the model parameters obtained from the initial training are applied to the target application scenario and fine-tuned on the dataset corresponding to the target application scenario. The purpose of fine-tuning is to further optimize the model parameters to adapt to the specific needs of the target application scenario. The dataset for fine-tuning is usually a relatively small dataset related to the target application scenario.

[0081] Inspired by pre-training, meta-learning has been proposed to address the shortcomings of model training, such as the need for large amounts of data and poor generalization ability. Meta-learning does not improve a model's performance in a specific application scenario through a large number of samples, but rather improves its performance across a range of application scenarios using a small number of samples from different datasets. Meta-learning is also known as learning how to learn. It has been extensively studied in areas such as few-shot regression, few-shot classification, and fast reinforcement learning. Researchers have proposed three main meta-learning paradigms: metric-learning, model-based learning, and optimizer-learning. Metric-learning aims to learn a kernel function that generates predicted probabilities; model-based learning aims to adjust the model's internal state according to changes in the application scenario; and MAML (Meta-Learning Objective-C) is the optimizer-learning paradigm. Inspired by fine-tuning and transfer learning on the ImageNet dataset, MAML aims to find a global initialization that can quickly and efficiently adapt to new application scenarios. The corresponding formula in MAML is shown below:

[0082]

[0083] in, This represents the parameters after the i-th training iteration. Let represent the parameters in the i-th training process. This represents a series of training tasks. This represents a dataset derived from a series of training tasks. This represents the loss function corresponding to the dataset. The data represents the training data from the dataset, and X represents the input feature vector. y Indicates the corresponding label, This represents the model parameters corresponding to the i-th training iteration.

[0084] MAML differs from the aforementioned basic training methods in both input and output. The input of basic training methods is a series of training data, while the input of MAML is a series of training tasks. The output of basic training methods is a specific function to solve application scenarios similar to the training data, while the output of MAML is a function with generalization ability to quickly adapt to multiple application scenarios. The biggest difference between basic training methods and MAML lies in the different objective functions during MAML training. If the basic training method aims to obtain experts who are proficient in a certain field, then MAML aims to obtain experts who can be applied in multiple fields and have rapid learning capabilities.

[0085] Furthermore, the main difference between MAML and pre-training lies in the loss function. Pre-training results in a model that performs well across multiple application scenarios, while MAML results in a model that performs well in multiple application scenarios, especially in new ones. In other words, compared to pre-training, MAML focuses more on whether the resulting model can quickly adapt to new application scenarios; that is, the model obtained through MAML possesses the ability to learn how to learn.

[0086] like Figure 1 As shown, pre-training updates the model parameters all at once after training multiple training tasks, so that the direction of its gradient change is the same as the first update direction of the training task. However, MAML updates the model parameters after the second gradient descent step, that is, for the training task, the parameters are updated only after two gradient descent steps.

[0087] Specifically, the training process for MAML includes the following steps:

[0088] 1. For each training task, perform the first step of gradient descent using the initial model parameters, and calculate the gradient of the loss function with respect to the model parameters.

[0089] 2. Based on the gradients calculated from all training tasks, perform a second step of gradient descent to obtain the final model parameters.

[0090] In contrast, pre-training updates the model parameters directly after performing only one gradient descent step in each training task.

[0091] MAML chooses to update model parameters after two gradient descent steps to fully utilize the gradient information from the training task and better adapt to new application scenarios. By performing more gradient descent steps, MAML can better capture the gradient directions specific to the training task, thereby achieving faster and more efficient adaptive learning.

[0092] In addition, related technologies have provided methods such as transfer learning, reinforcement learning, and continuous learning to address the shortcomings of basic training methods.

[0093] Although MAML can generate multi-scenario models applicable to a range of applications and demonstrates excellent results in few-shot supervised learning and reinforcement learning, it still has the following limitations:

[0094] 1. Limitations when applied to multiple application scenarios: Because the MAML process requires ensuring the general initialization of the multi-scenario model when applied to multiple application scenarios, the performance of the multi-scenario model when applied to specific application scenarios will be poor. That is, the multi-scenario model obtained through MAML will have certain limitations when applied to multiple application scenarios. For example, the accuracy of the multi-scenario recognition model obtained through MAML may not be high when applied to the specific application scenario of speech recognition.

[0095] 2. Training instability: such as Figure 2 As shown, due to its special inner and outer loop design, MAML requires multiple backpropagation derivative calculations, which may lead to gradient explosion and gradient vanishing problems. In other words, MAML may be very unstable during training.

[0096] 3. Fixed outer loop learning rate: In MAML, related techniques often use a fixed outer loop learning rate for training optimization, such as using the Adam optimizer with a fixed outer loop rate. Using a fixed outer loop learning rate reduces the generalization performance of MAML and also requires more training time to adjust the outer loop learning rate.

[0097] In view of this, embodiments of this application provide a model determination method, apparatus, and computer-readable storage medium.

[0098] like Figure 3 As shown, Figure 3This is a schematic diagram of the hardware operating environment provided in an embodiment of this application.

[0099] It should be noted that, Figure 3 This provides a schematic diagram of the hardware operating environment of the device determined by the model. In this application, the device determined based on the model can be a PC, a portable computer, or other terminal device.

[0100] like Figure 3 As shown, the model defines a device that may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0101] Those skilled in the art will understand that Figure 3 The model-determining device structure shown does not constitute a limitation on the model-determining device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0102] like Figure 3 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a model determination program. The operating system is a program that manages and controls the hardware and software resources of the model determination device, supporting the operation of the model determination program and other software or programs.

[0103] exist Figure 3 In the model determination device shown, the user interface 1003 is mainly used to receive requests and data sent by the first terminal, the second terminal, and the monitoring terminal; the network interface 1004 is mainly used to connect the backend server and communicate with it; and the processor 1001 can be used to call the model determination program stored in the memory 1005 and perform the following operations:

[0104] Obtain multiple training samples corresponding to M training tasks, where M is an integer and M≥2; for the i-th training task among the M training tasks, the multiple training samples corresponding to the i-th training task correspond to multiple training datasets, and the multiple training datasets correspond one-to-one with multiple application scenarios, where i is an integer and M≥i≥1.

[0105] Multiple training samples corresponding to the i-th training task are input into the initial multi-scene model. The feature extraction sub-model in the initial multi-scene model determines the dataset features corresponding to the multiple training samples corresponding to the i-th training task. The dataset features are used to represent the correlation between the corresponding training sample and multiple training datasets.

[0106] Based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples respectively, the prediction results corresponding to the multiple training samples corresponding to the i-th training task are determined through the prediction sub-model in the initial multi-scene model.

[0107] Based on the differences between the prediction results and sample labels of multiple training samples corresponding to the i-th training task, a sub-model loss function corresponding to the i-th training task is generated.

[0108] The initial multi-scene model is trained through inner and outer loops using the sub-model loss functions and outer loop learning rates corresponding to M training tasks, thus obtaining the multi-scene model.

[0109] Therefore, during the training process of the initial multi-scenario model, including the feature extraction sub-model and the prediction sub-model, the initial multi-scenario model can learn how to identify the application scenarios corresponding to the training samples. This enables the obtained multi-scenario model to be better applied to specific application scenarios in multiple application scenarios, thereby improving the accuracy of the multi-scenario model applied to multiple application scenarios.

[0110] Figure 4 This is a flowchart illustrating a model determination method provided in an embodiment of this application. In this process, from a device perspective, the executing entity can be one or more electronic devices, more specifically, a processing module of an electronic device; from a program perspective, the executing entity can correspondingly be a program mounted on these electronic devices. In this embodiment, the executing entity of the model determination method can be… Figure 3 The model determination device in the illustrated embodiment.

[0111] S401. Obtain multiple training samples corresponding to M training tasks, where M is an integer and M≥2.

[0112] In this embodiment, unlike the usual model training method which only requires setting one training task, M training tasks are required, where M is an integer and M≥2. This is because the training method in this embodiment is based on MAML, which mainly involves two loops: an inner loop and an outer loop. The inner loop adapts the model parameters to the update based on each specific training task among the M training tasks, that is, it adapts to the specific training task. The outer loop initializes and updates the model parameters based on the M training tasks.

[0113] In order to train and optimize the model through M training tasks, we can obtain multiple training samples corresponding to each of the M training tasks. Training samples refer to the samples used to train and optimize the model. For ease of explanation, we will take any one of the M training tasks, i.e., the i-th training task, as an example, where i is an integer and M≥i≥1.

[0114] In order to enable supervised training of the model in subsequent steps, the multiple training samples corresponding to the i-th training task each have corresponding sample labels. The sample labels can identify the true value corresponding to the training sample. For example, when the training sample corresponds to the application scenario of cat and dog recognition, the sample label can be used to identify whether the corresponding training sample is a cat or a dog.

[0115] In related technologies, when using the MAML method for training optimization, multiple training samples corresponding to each training task are constructed by selecting multiple training data of different categories from a training dataset. That is, each training task targets a training dataset.

[0116] In this embodiment, when constructing multiple training samples corresponding to the i-th training task, multiple training data of different categories can be selected from multiple training datasets. For example, a 5-way 1-shot mode can be selected, where 5-way means determining 5 categories of training data from 5 training datasets, and 1-shot means determining 1 training sample from each category of training data. That is, for one training task, the number of samples involved is 5, and these 5 samples come from 5 training datasets respectively. In other words, the multiple training samples corresponding to the i-th training task correspond to multiple training datasets, rather than just one training dataset.

[0117] A training dataset refers to a collection of training data that matches an application scenario. Each training dataset includes training data that corresponds to its specific application scenario. For example, when the application scenario is cat and dog recognition, the corresponding training dataset could include multiple cat images and multiple dog images. In order to train and optimize the model across multiple application scenarios, the multiple training datasets involved in this embodiment correspond one-to-one with multiple application scenarios.

[0118] Since multiple training datasets correspond one-to-one with multiple application scenarios, compared with MAML where each training task corresponds to one application scenario, each training task in this embodiment can correspond to multiple application scenarios.

[0119] It should be noted that in order for the model to be applicable to multiple application scenarios after training, multiple application scenarios must have the same scenario type. Scenario type refers to the type of application scenario, which can include recognition, regression, etc. For example, when the scenario type is recognition, multiple application scenarios can include cat and dog recognition, face recognition, etc.

[0120] Furthermore, in this embodiment, in order to verify the performance of the obtained model in subsequent steps, corresponding query samples can be obtained. Query samples refer to samples used to verify the performance of the model. In order to accurately verify the generalization ability of the model, the query samples and training samples are not the same.

[0121] S402. Input the multiple training samples corresponding to the i-th training task into the initial multi-scene model, and determine the dataset features corresponding to the multiple training samples corresponding to the i-th training task through the feature extraction sub-model in the initial multi-scene model.

[0122] A multi-scenario model is a model that can be applied to multiple application scenarios. Compared with a regular model that can only be applied to a single application scenario, a multi-scenario model can be applied to multiple different application scenarios.

[0123] An initial multi-scene model refers to an initial model that can be used to train on training samples to obtain a multi-scene model, i.e., a multi-scene model that has not completed model training.

[0124] In related technologies, in order to ensure the universal initialization of multi-scenario models when applied to multiple application scenarios, the resulting multi-scenario models may perform poorly when applied to specific application scenarios.

[0125] Therefore, in this embodiment, in order to improve the accuracy of the multi-scenario model, the initial multi-scenario model may include a feature extraction sub-model in addition to the prediction sub-model. The feature extraction sub-model is used to determine the application scenario corresponding to the training sample, and the prediction sub-model is used to determine the prediction result corresponding to the training sample.

[0126] Specifically, after inputting multiple training samples corresponding to the i-th training task into the initial multi-scenario model, the feature extraction sub-model within the initial multi-scenario model can first determine the dataset features corresponding to each training sample for the i-th training task. These dataset features represent the correlation between the corresponding training sample and multiple training datasets; that is, the dataset features represent the training dataset most relevant to the corresponding training sample. This allows us to determine that the corresponding training sample comes from the most relevant training dataset, and consequently, the application scenario corresponding to the most relevant training dataset can be identified as the application scenario corresponding to the training sample. For example, when training sample A comes from the training dataset corresponding to the application scenario of cat and dog recognition, the feature extraction sub-model in the initial multi-scenario model can first determine the dataset feature A corresponding to training sample A. Dataset feature A indicates that the corresponding training sample A comes from the training dataset corresponding to the application scenario of cat and dog recognition. In other words, during the training of the initial multi-scenario model, for training samples from multiple training datasets, the initial multi-scenario model, through the feature extraction sub-model, will determine the relationship between the training sample and these multiple training datasets, and the resulting dataset features are strongly correlated with the corresponding application scenario.

[0127] S403. Based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples respectively, determine the prediction results corresponding to the multiple training samples corresponding to the i-th training task respectively through the prediction sub-model in the initial multi-scene model.

[0128] The prediction result refers to the corresponding result obtained by predicting after the training sample is input into the model. For example, when the training sample A comes from the training dataset corresponding to the application scenario of cat and dog recognition, the prediction result of the training sample A may be that there is a probability of 0.8 that it is a cat.

[0129] After determining the dataset features corresponding to the multiple training samples for the i-th training task in S402, the prediction sub-model can determine the prediction results for each training sample corresponding to the i-th training task based on these features. For example, when training sample A comes from the training dataset corresponding to the application scenario of cat and dog recognition, compared to directly obtaining the prediction result of training sample A, the initial multi-scenario model can obtain the corresponding prediction result based on training sample A and its corresponding dataset features through the prediction sub-model. Since the prediction sub-model considers not only the training samples but also the dataset features representing the application scenario corresponding to the training samples in determining the prediction result, it can increase the accuracy of the prediction result to a certain extent. In other words, the dataset features obtained through the feature extraction sub-model can modify the prediction sub-model, enabling it to adapt to different application scenarios.

[0130] In one possible implementation, when the prediction sub-model includes K convolutional layers, where K is an integer and K≥2, in step S403, based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to each of the multiple training samples, the prediction results corresponding to the multiple training samples corresponding to the i-th training task are determined through the prediction sub-model in the initial multi-scene model, including:

[0131] For the first convolutional layer among the K convolutional layers, input the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples respectively into the first convolutional layer. Through the first convolutional layer, determine the first layer prediction sub-results corresponding to the multiple training samples corresponding to the i-th training task respectively.

[0132] For the k-th convolutional layer out of K convolutional layers, the (k-1)-th layer prediction sub-results corresponding to multiple training samples for the i-th training task and the dataset features corresponding to multiple samples are input into the k-th convolutional layer. Through the k-th convolutional layer, the k-th layer prediction sub-results corresponding to multiple training samples for the i-th training task are determined; where k is an integer, K≥k≥2.

[0133] The K-th layer prediction sub-results corresponding to the multiple training samples corresponding to the i-th training task are respectively used as the prediction results corresponding to the multiple training samples corresponding to the i-th training task.

[0134] In the actual use of this embodiment, the prediction sub-model may include K convolutional layers, where K is an integer and K≥2. In order to obtain accurate prediction results, the dataset features can be used as part of the input of each of the K convolutional layers. That is, the dataset features are not only used as simple input items of the prediction sub-model, but also used to adjust each convolutional layer in the prediction sub-model.

[0135] For example, such as Figure 5 As shown, when the prediction sub-model includes four convolutional layers, the channel sizes of each convolutional layer are 32, 64, 128, and 256, respectively. The kernel size of each convolutional layer is 3, the stride is 3, and valid padding is used. Each convolutional layer is followed by batch normalization and rectified linear unit (ReLU) activation layers. After obtaining the dataset features through the feature extraction sub-model, the dataset features can be in the form of feature vectors. The dataset features can be transformed into four different dimensions through four fully connected linear layers, that is, transformed into the form of embedding vectors. Then, the sigmoid function is taken for each dimension of the dataset features, and it is multiplied with each convolutional layer in the prediction sub-model to complete the adjustment of each convolutional layer in the prediction sub-model.

[0136] By using dataset features as input to each convolutional layer in the prediction sub-model, the accuracy of each convolutional layer in the prediction sub-model can be adjusted, thereby improving the accuracy of multi-scenario models.

[0137] In one possible implementation, the feature extraction sub-model and the prediction sub-model have the same model architecture.

[0138] In the practical application of this embodiment, in order to reduce the amount of computation, the feature extraction sub-model and the prediction sub-model can have the same model architecture. Similar to the prediction sub-model, the feature extraction sub-model can also adopt a multi-layer convolutional neural network architecture, that is, the feature extraction sub-model can also include four convolutional layers. The channel size of each convolutional layer can be 32, 64, 128 and 256 respectively. The kernel size of each convolutional layer can also be 3, the stride can also be 3, and effective padding is also used. Each convolutional layer is followed by batch normalization and modified linear unit activation layer. After the four convolutional layers of the feature extraction sub-model, a 256-dimensional dataset feature can be obtained by average pooling.

[0139] In one possible implementation, in step S403, based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to each of the multiple training samples, the prediction results corresponding to the multiple training samples corresponding to the i-th training task are determined through the prediction sub-model in the initial multi-scene model, including:

[0140] The dataset features corresponding to multiple training samples are input into the bidirectional long short-term memory network in the initial multi-scenario model. The bidirectional long short-term memory network determines the memory dataset features corresponding to multiple training samples. The memory dataset features contain more information than the corresponding dataset features.

[0141] Based on the features of multiple training samples corresponding to the i-th training task and the memory datasets corresponding to the multiple training samples, the prediction results corresponding to the multiple training samples corresponding to the i-th training task are determined through the prediction sub-model in the initial multi-scene model.

[0142] Based on the dataset features corresponding to multiple training samples for the i-th training task obtained through the initial feature extraction sub-model, such as... Figure 5 As shown, the dataset features corresponding to multiple training samples for the i-th training task can be input into the bidirectional long short-term memory network in the initial multi-scene model. The bidirectional long short-term memory (LSTM) network is an artificial recurrent neural network (RNN). The LSTM network can take feature vectors as input, where the hidden layer size can be 128. Through the bidirectional long short-term memory network, the memory dataset features corresponding to multiple training samples for the i-th training task can be determined. The memory dataset features contain more information than the corresponding dataset features. In other words, the LSTM network enhances the memory of dataset features, thus obtaining the corresponding memory dataset features.

[0143] Based on obtaining the memory dataset features corresponding to the multiple training samples for the i-th training task, the prediction results corresponding to the multiple training samples for the i-th training task can be determined through the prediction sub-model in the initial multi-scene model, according to the multiple training samples for the i-th training task and the memory dataset features corresponding to the multiple training samples.

[0144] By adding an LSTM network as a memory layer, the multi-scenario model's ability to handle complex application scenarios can be further improved, allowing it to better utilize learned knowledge when dealing with new application scenarios. Corresponding experimental results also demonstrate that in complex application scenarios, such as simultaneously processing five application scenarios, the added LSTM can further improve the accuracy of the multi-scenario model in specific application scenarios, further enhancing the performance of the initial multi-scenario model.

[0145] S404. Based on the differences between the prediction results and the corresponding sample labels of the multiple training samples corresponding to the i-th training task, generate the sub-model loss function corresponding to the i-th training task.

[0146] Since the prediction result refers to the corresponding result obtained by prediction after the training sample is input into the model, and the sample label refers to the label that can identify the true value corresponding to the training sample, for the i-th training task, based on the difference between the prediction results and the corresponding sample labels of the multiple training samples corresponding to the i-th training task, a sub-model loss function corresponding to the i-th training task can be generated. The sub-model loss function corresponding to the i-th training task can represent the direction of model training optimization for the i-th training task.

[0147] S405. Using the sub-model loss function and outer loop learning rate corresponding to M training tasks respectively, the inner and outer loops are used to train the initial multi-scene model to obtain the multi-scene model.

[0148] After obtaining the sub-model loss functions corresponding to the M training tasks through S404, since the sub-model loss functions corresponding to the M training tasks can represent the model training optimization direction corresponding to the M training tasks respectively, the initial multi-scene model can be trained according to the sub-model loss functions corresponding to the M training tasks and the outer loop learning rate. That is, the inner and outer loops of training the initial multi-scene model are used to obtain the multi-scene model, which can be applied to multiple application scenarios.

[0149] In one possible implementation, S405 uses the sub-model loss functions and outer loop learning rates corresponding to M training tasks to train the initial multi-scene model in both inner and outer loops, resulting in a multi-scene model including:

[0150] The inner loop of training the initial multi-scene model is obtained by using the sub-model loss function corresponding to the i-th training task;

[0151] Based on the model update parameters and outer loop learning rate corresponding to the M training tasks, the outer loop of the initial multi-scene model is trained to obtain the multi-scene model.

[0152] Since the sub-model loss function corresponding to the i-th training task can represent the training optimization direction of the model corresponding to the i-th training task, the initial multi-scene model can be trained according to the sub-model loss function corresponding to the i-th training task. That is, the inner loop of training the initial multi-scene model is completed to update the initial multi-scene model with multiple training samples corresponding to the i-th training task, thereby obtaining the model update parameters corresponding to the i-th training task.

[0153] The model update parameters for each of the M training tasks can be obtained from the sub-model loss functions. After obtaining the model parameters for each of the M training tasks, to obtain an initial scene model applicable to multiple application scenarios, an outer loop can be run to train the initial multi-scene model based on the model update parameters for each of the M training tasks and the outer loop learning rate. For example, the outer loop training can be completed using the following formula:

[0154] )

[0155] in, Indicates parameters, This represents the outer loop learning rate. This indicates gradient descent. Indicates M training tasks. This represents the i-th training task. This represents the sub-model loss function corresponding to the i-th training task. This represents the model update parameters corresponding to the i-th training task.

[0156] Using the above formula, the model update parameters corresponding to the i-th training task can be obtained through one-step gradient descent. After obtaining the model update parameters corresponding to M training tasks, two-step gradient descent can be performed on the model update parameters corresponding to M training tasks, and the model parameters can be converged through the outer loop learning rate to obtain the corresponding multi-scene model.

[0157] During the training process of the initial multi-scenario model, including the feature extraction sub-model and the prediction sub-model, the initial multi-scenario model can learn how to identify the application scenarios corresponding to the training samples. This enables the obtained multi-scenario model to be better applied to specific application scenarios in multiple application scenarios, thereby improving the accuracy of the multi-scenario model applied to multiple application scenarios.

[0158] In one possible implementation, the model update parameters for the i-th training task are obtained through the inner loop of training the initial multi-scene model using the sub-model loss function corresponding to the i-th training task. These parameters include:

[0159] The inner loop of training the initial multi-scene model through the sub-model loss function corresponding to the i-th training task obtains multiple model update sub-parameters corresponding to the i-th training task; the multiple model update sub-parameters corresponding to the i-th training task correspond one-to-one with the multi-step update corresponding to the i-th training task.

[0160] Based on the model update parameters and outer loop learning rate corresponding to the M training tasks, the outer loop of the initial multi-scene model is trained to obtain the multi-scene model, including:

[0161] Based on the multiple model update sub-parameters and outer loop learning rate corresponding to the M training tasks, the outer loop of the initial multi-scene model is trained to obtain the multi-scene model.

[0162] Due to its unique inner and outer loop design, MAML requires multiple backpropagation derivative calculations, which may lead to gradient explosion and gradient vanishing problems. In other words, MAML may be very unstable during training.

[0163] Therefore, in the inner loop of training the initial multi-scene model using the sub-model loss function corresponding to the i-th training task, compared to only obtaining the model update parameters corresponding to the i-th training task, where the model update parameters corresponding to the i-th training task correspond to the last update step in the multi-step update corresponding to the i-th training task, in this embodiment, multiple model update sub-parameters corresponding to the i-th training task can be obtained, and these multiple model update sub-parameters correspond one-to-one with the multi-step update corresponding to the i-th training task. That is, in this embodiment, the model update sub-parameters for each step of the update corresponding to the i-th training task can be obtained to calculate the sub-loss for each step, instead of only obtaining the loss of the last step corresponding to the i-th training task.

[0164] After obtaining multiple model update sub-parameters corresponding to M training tasks, the outer loop of the initial multi-scene model can be trained based on the multiple model update sub-parameters and the outer loop learning rate corresponding to the M training tasks. This allows the training process of the initial multi-scene model to be smoothed through multi-step loss during the outer loop of model training, thus avoiding training instability.

[0165] In one possible implementation, during the inner loop of training the initial multi-scene model using the sub-model loss function corresponding to the i-th training task, the method further includes:

[0166] Configure the corresponding sample weights for the multi-step updates corresponding to the i-th training task.

[0167] When training the initial multi-scene model using the sub-model loss function corresponding to the i-th training task in the inner loop, in order to control the impact of each training sample on the training optimization of the initial multi-scene model, taking the i-th training task as an example, the corresponding sample weights can be configured for the multi-step updates corresponding to the i-th training task, as shown in the following formula:

[0168] )

[0169] in, Indicates parameters, This represents the outer loop learning rate. This indicates gradient descent. Indicates M training tasks. Let N represent the i-th training task, N represent the N-step updates corresponding to the i-th training task, and j represent the j-th update in the N-step updates corresponding to the i-th training task. This represents the sample weight updated in step j of the N-step update for the i-th training task. This represents the sub-model loss function corresponding to the i-th training task. This represents the model update sub-parameter corresponding to the j-th step update for the i-th training task.

[0170] It should be noted that a higher sample weight indicates that the corresponding update step is more important during the training process, while a lower sample weight indicates that the corresponding update step is less important during the training process. For example, in order to reduce the amount of computation, the same sample weight can be configured for multiple training samples corresponding to the i-th training task.

[0171] In one possible implementation, corresponding sample weights are configured for each of the multi-step updates corresponding to the i-th training task, including:

[0172] At the beginning of the inner loop of the initial multi-scene model training, the corresponding initial sample weights are configured for the multi-step updates corresponding to the i-th training task; the values ​​of the multiple initial sample weights corresponding to the i-th training task are the same.

[0173] During the inner loop of training the initial multi-scene model, the weights of multiple initial samples corresponding to the i-th training task are adjusted. Specifically, for the start update step in the multi-step update corresponding to the i-th training task, the weights of the initial samples corresponding to the start update step of the i-th training task are decreased, and for the subsequent update steps in the multi-step update corresponding to the i-th training task, the weights of the initial samples corresponding to the subsequent update steps of the i-th training task are increased.

[0174] In this embodiment, for the i-th training task, the weights of multiple samples corresponding to the i-th training task can change with the training process. Specifically, at the beginning of the inner loop of the initial multi-scene model training, the corresponding initial sample weights can be configured for the multi-step updates corresponding to the i-th training task. At this time, it can be assumed that the multi-step updates corresponding to the i-th training task have equal contributions to the final update, that is, the values ​​of the multiple initial sample weights corresponding to the i-th training task are the same. During the inner loop of the initial multi-scene model training, as the number of iterations increases, the subsequent update steps of the inner loop become more important, while the initial update step becomes less important. Therefore, the weights of the multiple initial samples corresponding to the i-th training task can be adjusted. Specifically, for the initial update step in the multi-step update corresponding to the i-th training task (the starting step of the inner loop), the weights of the initial samples corresponding to the initial update step of the i-th training task can be reduced, thus reducing the influence of the starting step of the inner loop. For the subsequent update steps in the multi-step update corresponding to the i-th training task (the subsequent steps of the inner loop), the weights of the initial samples corresponding to the subsequent update steps of the i-th training task can be increased, thus increasing the influence of the subsequent steps of the inner loop.

[0175] Since the loss function is calculated more accurately as it is calculated later in the training and optimization process, the sample weights can be adjusted during training to allow the inner loop to converge to a relatively lower loss, thereby obtaining a set of model update parameters that perform well on the corresponding training task.

[0176] In one possible implementation, the outer loop learning rate is annealed during the outer loop training of the initial multi-scene model; wherein the outer loop learning rate decreases as the outer loop training of the initial multi-scene model progresses.

[0177] In MAML, related techniques often use a fixed outer loop learning rate for training optimization, such as employing the Adam optimizer with a fixed outer loop rate. Using a fixed outer loop learning rate reduces the generalization performance of MAML and also requires more training time to adjust the outer loop learning rate.

[0178] In this embodiment, instead of using a fixed outer loop learning rate, the outer loop learning rate is annealed during the outer loop training of the initial multi-scene model. For example, a learning rate scheduler such as Cosine Annealing can be used to anneal the outer loop learning rate, so that the outer loop learning rate decreases as the initial multi-model training outer loop progresses, thereby improving the generalization performance of the obtained multi-scene model and reducing training time.

[0179] Therefore, to obtain a multi-scenario model applicable to multiple application scenarios with the same scenario type, multiple training samples corresponding to M training tasks are obtained, where M is an integer and M≥2. Taking the i-th training task among the M training tasks as an example, where i is an integer and M≥i≥1, the multiple training samples corresponding to the i-th training task do not come from a single training dataset, but from multiple training datasets that correspond one-to-one with multiple application scenarios. The multiple training samples corresponding to the i-th training task are input into the initial multi-scenario model for training and optimization. The initial multi-scenario model includes a feature extraction sub-model and a prediction sub-model. The feature extraction sub-model can first determine the dataset features corresponding to the multiple training samples of the i-th training task. The dataset features are used to represent the correlation between the corresponding training sample and multiple training datasets, that is, the dataset features can represent the correlation between the corresponding training sample and the multiple training datasets. The most relevant training dataset is used to identify the application scenarios corresponding to the training samples. Then, based on the multiple training samples for the i-th training task and the dataset features corresponding to each of these training samples, a prediction sub-model determines the prediction results for each of the multiple training samples for the i-th training task. Since the prediction sub-model considers not only the training samples but also the dataset features representing the application scenarios of the training samples, it increases the accuracy of the prediction results to some extent. Based on the differences between the prediction results and the corresponding sample labels for the multiple training samples for the i-th training task, a sub-model loss function for the i-th training task is generated. The inner and outer loops of the initial multi-scenario model are then trained using the sub-model loss functions for each of the M training tasks and the outer loop learning rate to obtain the multi-scenario model. During the training of the initial multi-scenario model, including the feature extraction sub-model and the prediction sub-model, the initial multi-scenario model learns how to identify the application scenarios corresponding to the training samples, enabling the obtained multi-scenario model to be better applied to specific application scenarios in multiple application scenarios and improving the accuracy of the multi-scenario model applied to multiple application scenarios.

[0180] The multi-scene model obtained in this application will be validated using experimental datasets. These datasets may include multiple datasets, such as... Figure 6 As shown, Figure 6 (a) in the image represents the OMNIGLOT dataset, which mainly includes images corresponding to handwritten characters. Figure 6 (b) in the image represents the MINI-IMAGENET dataset, which mainly includes images of common objects. Figure 6 (c) in the dataset represents the FC100 dataset, which mainly includes images of common objects. Figure 6In the diagram, (d) represents the CUB dataset, which mainly includes images of birds. Figure 6 In the table, (e) represents the AIRCRAFT dataset, which mainly includes images of aircraft. Experiments can be conducted to verify the fusion of multiple datasets, such as mixing two or three datasets. The fusion methods are shown in Table 1.

[0181] Table 1. Dataset Fusion Methods

[0182]

[0183] Since the training method in this application is an improvement based on MAML, by adding a feature extraction sub-model to the initial multi-scene model, the multi-scene model has the ability to train across training datasets in a single training task, expanding the accuracy of MAML in multiple application scenarios and improving the ability of MAML to be applied in multiple models. This can be represented by the multi-task model-agnostic meta-learning algorithm (mt-MAML). By comparing the accuracy of MAML and mt-MAML on multiple datasets, the accuracy of mt-MAML in multiple application scenarios can be verified. The corresponding verification data are shown in Tables 2 and 3.

[0184] Table 2 shows the evaluation process accuracy of MAML and MMAML across multiple datasets.

[0185]

[0186] Table 3. Statistics and hypotheses for MAML and MMAML across multiple datasets.

[0187]

[0188] As shown in Table 2, the evaluation process accuracy of mt-MAML is higher than that of MAML across multiple datasets, and the difference in evaluation process accuracy between mt-MAML and MAML increases with the number of datasets. As shown in Table 3, by setting a null hypothesis that MAML does not improve accuracy and a converse hypothesis that the training method described in this application does indeed improve accuracy, the null hypothesis can be definitively rejected based on the obtained statistical values. This demonstrates that MAML and mt-MAML have a significant difference, meaning that the training method described in this application can significantly improve the accuracy of MAML.

[0189] MAML in related technologies suffers from training instability. This application addresses this issue by improving model stability during training through the acquisition of multi-step loss within the inner loop. This training method can be represented as mt-MAML+. Experiments show that mt-MAML+ significantly outperforms MMAML in training stability, especially in multi-dataset training scenarios. Figure 7 As shown, Figure 7 In the middle (a), the data refers to the combined data from the two datasets. Figure 7 (b) refers to the mixed data from the three datasets. Figure 7 In the middle (c), the data refers to the data corresponding to the combination of the four datasets. Figure 7 In the middle (d), the data corresponds to the mixture of five datasets. By comparing the accuracy variance during the training process, it can be analyzed that the change in accuracy variance of mt-MAML+ is smaller than that of mt-MAML, that is, mt-MAML+ training is more stable. In addition, the experimental results are analyzed, and the relevant data are shown in Table 4. By quantitatively comparing the accuracy variance, it can be seen that the accuracy variance of mt-MAML+ is smaller. The experimental data are interpreted using Wilcoxon comparison. The null hypothesis is set as mt-MAML+ has no improvement in stability, while the converse hypothesis is that the mt-MAML+ method does improve training stability. Based on the obtained hypothesis values, the null hypothesis can be rejected with certainty, thus proving that mt-MAML+ and mt-MAML have significant differences, that is, mt-MAML+ can significantly improve training stability.

[0190] Table 4. Relevant data on MMAML+ training stability.

[0191]

[0192] The training method presented in this application, as an improvement on MAML, retains the advantages of MAML, namely, its ability to learn from few samples, good generalization ability, and fast learning capability. In practical applications, many scenarios lack sufficiently large training sets. This application utilizes a meta-learning training method, enabling the model to learn quickly from a limited number of samples. Basic training methods, after learning a task, often only perform well in specific application scenarios and struggle to generalize to others. This application, by performing meta-training on multiple relevant application scenarios, allows the model to better generalize to new application scenarios, thus improving the model's generalization ability.

[0193] The training method presented in this application, as an improvement on MAML, also possesses another advantage of MAML: model independence, or model-agnostic nature. Theoretically, this training method is applicable to various types of models, such as neural networks and support vector machines. Furthermore, MAML has demonstrated excellent performance in many fields, including few-shot regression, few-shot classification, and fast reinforcement learning. In the few-shot regression domain, the training method can be applied to marketing campaign configuration optimization. For example, based on multi-dimensional rule elements of the campaign configuration, including basic rules, transaction methods, merchant rules, user rules, and display rules, it can predict the expected results of the marketing campaign, such as budget execution rate, coupon redemption / discount rate, and marketing leverage (a one-yuan discount driving transaction volume). This helps operations personnel and campaign organizers appropriately adjust campaign formats, audiences, and other elements. In the few-shot classification domain, the training method can be applied to traditional image classification and typical Natural Language Processing (NLP) tasks. The training method provided in this application can also be implemented in the merchant chain model optimization project. By training through multiple sub-tasks (named entity recognition, name similarity calculation), a better initialization parameter can be obtained.

[0194] The model determination method of this application will be described below based on application scenario examples.

[0195] Marketing campaigns rely heavily on campaign configuration. However, the sheer volume of configuration elements often makes it difficult to review and adjust existing campaigns, or to design new ones. If the effectiveness of a campaign could be directly predicted based on its configuration elements, configuration efficiency and marketing results would be significantly improved. Currently, however, there is a problem of a large variety of campaigns but limited training sample data. The model determination method proposed in this application can be well applied to this scenario.

[0196] For the specific implementation process and the parameter updates and data flow involved, please refer to [link / reference]. Figure 8 And Table 5. (For example...) Figure 8 As shown, the model determination method of this application can be applied to the optimization of marketing campaigns, where the key steps are the creation of the dataset and the training of the model.

[0197] Dataset Creation: Following the few-shot learning paradigm, the collected raw data can be divided into a series of training tasks according to the type of activity. For example, one training task contains information on 5 activities: Activity 1, Activity 2, Activity 3, Activity 4, and Activity 5. These five activities belong to coupon activities, instant discount activities, targeted marketing activities, UnionPay red envelope activities, and cashback points activities, respectively. The configuration elements of these five activities are quite different, but a standardized evaluation system can be used to evaluate them, such as the utilization rate of activity funds, marketing leverage ratio, monthly active user promotion, and monthly active transaction increase. Each task is a combination of different activity categories. After the data is divided according to the tasks, it needs to be converted into a data format before being input into the initial multi-scenario model for training, i.e., preprocessing. Since the activity configuration elements include both text and numbers such as time, they can be uniformly treated as text information, truncated and supplemented, then the text is converted into a one-dimensional vector, and then the one-dimensional vector is converted into a three-dimensional vector as the input of the initial multi-scenario model.

[0198] The model training phase is divided into two parts: an inner loop and an outer loop. The inner loop updates the model parameters only for a specific training task, requiring only 5 training data points per iteration, thus demonstrating the advantages of few-shot learning. The outer loop, however, integrates all training tasks to obtain cross-task model parameters. Since the primary goal of training is to enable the model to quickly adapt to new application scenarios, the model parameters obtained through this method can converge quickly and be applied to the prediction of entirely new application scenarios.

[0199] Table 5 Inner and Outer Loop Data Flow Tables

[0200]

[0201] The model determination method presented in this application offers significant flexibility in model selection. By focusing on optimizing the training process itself (inner and outer loops) rather than the model, model-insensitive optimization is achieved. Its small training data requirement also facilitates implementation in lightweight applications such as marketing campaign configuration optimization. Traditional few-shot learning is highly unstable throughout the training process due to the small amount of data used in the inner loop. By incorporating multi-step loss optimization, the training of the inner loop is smoothed out, improving overall training stability and enabling faster model convergence. Furthermore, real-world application scenarios involve a wide variety of activities with diverse configuration parameters. The feature extraction sub-model added in this proposal effectively identifies different activity categories, resulting in high accuracy for specific activities and good generalization ability, making it suitable for future additions to the activity mix.

[0202] In the description of this specification, references to terms such as "some possible implementations," "some implementations," "example," "specific example," or "some examples" indicate that a specific feature, structure, material, or characteristic described in connection with that implementation or example is included in at least one implementation or example of this application, and the aforementioned terms do not necessarily refer to the same implementation or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more implementations or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different implementations or examples described in this specification, as well as the features of different implementations or examples.

[0203] The method flowcharts for embodiments of this application describe certain operations as different steps performed in a certain order. Such flowcharts are illustrative and not restrictive. Some steps described herein may be grouped together and performed in a single operation, or some steps may be divided into multiple sub-steps, and some steps may be performed in an order different from that shown herein. The various steps shown in the flowcharts may be implemented in any way by any circuit structure and / or tangible mechanism (e.g., by software running on a computer device, hardware (e.g., logic functions implemented by a processor or chip), and / or any combination thereof).

[0204] Those skilled in the art will understand that in the methods described in the above specific embodiments, the order in which the steps are written does not imply a strict execution order, and the specific execution order of each step should be determined by its function and possible internal logic.

[0205] The following description, based on an embodiment of the apparatus, illustrates a model determination apparatus provided in this application. Figure 9 As shown, the model determination device 900 includes an acquisition unit 901, a first determination unit 902, a second determination unit 903, a generation unit 904, and a training unit 905.

[0206] The acquisition unit 901 is used to acquire multiple training samples corresponding to M training tasks respectively, where M is an integer and M≥2; for the i-th training task among the M training tasks, the multiple training samples corresponding to the i-th training task correspond to multiple training datasets, and the multiple training datasets correspond one-to-one with multiple application scenarios, where i is an integer and M≥i≥1.

[0207] The first determining unit 902 is used to input multiple training samples corresponding to the i-th training task into the initial multi-scene model, and determine the dataset features corresponding to the multiple training samples corresponding to the i-th training task through the feature extraction sub-model in the initial multi-scene model; the dataset features are used to represent the correlation between the corresponding training sample and multiple training datasets.

[0208] The second determining unit 903 is used to determine the prediction results corresponding to the multiple training samples corresponding to the i-th training task based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples respectively, through the prediction sub-model in the initial multi-scene model.

[0209] The generation unit 904 is used to generate the sub-model loss function corresponding to the i-th training task based on the difference between the prediction results and the corresponding sample labels of the multiple training samples corresponding to the i-th training task.

[0210] Training unit 905 is used to train the initial multi-scene model through inner and outer loops using the sub-model loss functions and outer loop learning rates corresponding to M training tasks, respectively, to obtain the multi-scene model.

[0211] In one possible implementation, the second determining unit 903 is used for:

[0212] When the prediction sub-model includes K convolutional layers, where K is an integer and K≥2, for the first convolutional layer among the K convolutional layers, multiple training samples corresponding to the i-th training task and the dataset features corresponding to the multiple training samples are input into the first convolutional layer. Through the first convolutional layer, the prediction sub-results of the first layer corresponding to the multiple training samples corresponding to the i-th training task are determined.

[0213] For the k-th convolutional layer out of K convolutional layers, the (k-1)-th layer prediction sub-results corresponding to multiple training samples for the i-th training task and the dataset features corresponding to multiple samples are input into the k-th convolutional layer. Through the k-th convolutional layer, the k-th layer prediction sub-results corresponding to multiple training samples for the i-th training task are determined; where k is an integer, K≥k≥2.

[0214] The K-th layer prediction sub-results corresponding to the multiple training samples corresponding to the i-th training task are respectively used as the prediction results corresponding to the multiple training samples corresponding to the i-th training task.

[0215] In one possible implementation, the feature extraction sub-model and the prediction sub-model have the same model architecture.

[0216] In one possible implementation, the second determining unit 903 is used for:

[0217] The dataset features corresponding to multiple training samples are input into the bidirectional long short-term memory network in the initial multi-scenario model. The bidirectional long short-term memory network determines the memory dataset features corresponding to multiple training samples. The memory dataset features contain more information than the corresponding dataset features.

[0218] Based on the features of multiple training samples corresponding to the i-th training task and the memory datasets corresponding to the multiple training samples, the prediction results corresponding to the multiple training samples corresponding to the i-th training task are determined through the prediction sub-model in the initial multi-scene model.

[0219] In one possible implementation, training unit 905 is used for:

[0220] The inner loop of training the initial multi-scene model is obtained by using the sub-model loss function corresponding to the i-th training task;

[0221] Based on the model update parameters and outer loop learning rate corresponding to the M training tasks, the outer loop of the initial multi-scene model is trained to obtain the multi-scene model.

[0222] In one possible implementation, training unit 905 is used for:

[0223] The inner loop of training the initial multi-scene model through the sub-model loss function corresponding to the i-th training task obtains multiple model update sub-parameters corresponding to the i-th training task; the multiple model update sub-parameters corresponding to the i-th training task correspond one-to-one with the multi-step update corresponding to the i-th training task.

[0224] Based on the multiple model update sub-parameters and outer loop learning rate corresponding to the M training tasks, the outer loop of the initial multi-scene model is trained to obtain the multi-scene model.

[0225] In one possible implementation, the model determining device 900 further includes a configuration unit for:

[0226] Configure the corresponding sample weights for the multi-step updates corresponding to the i-th training task.

[0227] In one possible implementation, the configuration unit is used for:

[0228] At the beginning of the inner loop of the initial multi-scene model training, the corresponding initial sample weights are configured for the multi-step updates corresponding to the i-th training task; the values ​​of the multiple initial sample weights corresponding to the i-th training task are the same.

[0229] During the inner loop of training the initial multi-scene model, the weights of multiple initial samples corresponding to the i-th training task are adjusted. Specifically, for the start update step in the multi-step update corresponding to the i-th training task, the weights of the initial samples corresponding to the start update step of the i-th training task are decreased, and for the subsequent update steps in the multi-step update corresponding to the i-th training task, the weights of the initial samples corresponding to the subsequent update steps of the i-th training task are increased.

[0230] In one possible implementation, the model determining device 900 further includes an annealing unit for:

[0231] During the outer loop training of the initial multi-scene model, the outer loop learning rate is annealed; the outer loop learning rate decreases as the initial multi-scene model is trained in the outer loop.

[0232] It should be noted that the apparatus in the embodiments of this application can implement the various processes of the aforementioned method embodiments and achieve the same effects and functions, which will not be repeated here.

[0233] One advantage of the above embodiments is that, in order to obtain a multi-scenario model applicable to multiple application scenarios with the same scenario type, multiple training samples corresponding to M training tasks are obtained, where M is an integer and M≥2. Taking the i-th training task among the M training tasks as an example, where i is an integer and M≥i≥1, the multiple training samples corresponding to the i-th training task do not come from a single training dataset, but from multiple training datasets that correspond one-to-one with multiple application scenarios. The multiple training samples corresponding to the i-th training task are input into the initial multi-scenario model for training and optimization. The initial multi-scenario model includes a feature extraction sub-model and a prediction sub-model. The feature extraction sub-model can first determine the dataset features corresponding to the multiple training samples corresponding to the i-th training task. The dataset features are used to represent the correlation between the corresponding training sample and multiple training datasets, that is, the dataset features can represent the correlation between the training sample and the multiple training datasets. The most relevant training dataset for the corresponding training samples is selected to determine the application scenario corresponding to each training sample. Then, based on the multiple training samples corresponding to the i-th training task and the dataset features corresponding to each of these training samples, a prediction sub-model determines the prediction results for each of the multiple training samples corresponding to the i-th training task. Since the prediction sub-model considers not only the training samples but also the dataset features representing the application scenario of the training samples during the prediction process, it can increase the accuracy of the prediction results to a certain extent. Based on the differences between the prediction results and the corresponding sample labels for the multiple training samples corresponding to the i-th training task, a sub-model loss function for the i-th training task can be generated. Using the sub-model loss functions for each of the M training tasks and the outer loop learning rate, the inner and outer loops of the initial multi-scenario model are trained to obtain the multi-scenario model. During the training process of the initial multi-scenario model, including the feature extraction sub-model and the prediction sub-model, the initial multi-scenario model learns how to distinguish the application scenario corresponding to the training samples, thus enabling the obtained multi-scenario model to be better applied to specific application scenarios in multiple application scenarios and improving the accuracy of the multi-scenario model applied to multiple application scenarios.

[0234] According to some embodiments of this application, a model determination apparatus according to an embodiment of this application is provided for performing... Figure 4 The model determination method shown includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in the above embodiments.

[0235] According to some embodiments of this application, a non-volatile computer storage medium for model determination is provided, having stored computer-executable instructions configured to execute, when run by a processor, the method described in the above embodiments.

[0236] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory, read-only memory, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. Furthermore, although the operations of the methods of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple sub-steps.

[0237] While the spirit and principles of this application have been described above with reference to several specific embodiments, it should be understood that this application is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined. This application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A model determination method characterized by, The method comprises the following steps: obtaining a plurality of training samples corresponding to M training tasks, M being an integer and M≥2; for the i-th training task in the M training tasks, the plurality of training samples corresponding to the i-th training task correspond to a plurality of training data sets, the plurality of training data sets correspond to a plurality of application scenarios one by one, wherein i is an integer and M≥i≥1, and the training data set comprises a plurality of images; inputting the plurality of training samples corresponding to the i-th training task into an initial multi-scene model, determining the data set features corresponding to the plurality of training samples corresponding to the i-th training task through a feature extraction sub-model in the initial multi-scene model, wherein the data set features are used to represent the correlation between the corresponding training sample and the plurality of training data sets; determining the prediction results corresponding to the plurality of training samples corresponding to the i-th training task through a prediction sub-model in the initial multi-scene model according to the plurality of training samples corresponding to the i-th training task and the data set features corresponding to the plurality of training samples; generating a sub-model loss function corresponding to the i-th training task based on the difference between the prediction results corresponding to the plurality of training samples corresponding to the i-th training task and the sample labels corresponding to the plurality of training samples; performing an inner loop and an outer loop of training the initial multi-scene model through the sub-model loss functions corresponding to the M training tasks and an outer loop learning rate to obtain a multi-scene model; wherein, at the beginning of the inner loop of training the initial multi-scene model, a corresponding initial sample weight is configured for the multi-step update corresponding to the i-th training task, the plurality of initial sample weights corresponding to the i-th training task have the same value, and the plurality of initial sample weights corresponding to the i-th training task are adjusted in the inner loop of training the initial multi-scene model; wherein, for the starting update step in the multi-step update corresponding to the i-th training task, the initial sample weight corresponding to the starting update step of the i-th training task is reduced, and for the subsequent update step in the multi-step update corresponding to the i-th training task, the initial sample weight corresponding to the subsequent update step of the i-th training task is increased.

2. The method of claim 1, wherein, When the prediction sub-model comprises K convolutional layers, wherein K is an integer and K≥2, the determination of the prediction results corresponding to the plurality of training samples corresponding to the i-th training task through the prediction sub-model in the initial multi-scene model according to the plurality of training samples corresponding to the i-th training task and the data set features corresponding to the plurality of training samples comprises: for the first convolutional layer in the K convolutional layers, inputting the plurality of training samples corresponding to the i-th training task and the data set features corresponding to the plurality of training samples into the first convolutional layer, and determining the first layer prediction sub-results corresponding to the plurality of training samples corresponding to the i-th training task through the first convolutional layer. For the kth convolutional layer in the K convolutional layers, the (k-1)th layer prediction sub-results corresponding to the plurality of training samples corresponding to the ith training task and the dataset features corresponding to the plurality of training samples are input into the kth convolutional layer, and the kth layer prediction sub-results corresponding to the plurality of training samples corresponding to the ith training task are determined by the kth convolutional layer; wherein k is an integer, K≥k≥2; The Kth layer prediction sub-results corresponding to the plurality of training samples corresponding to the ith training task are taken as the prediction results corresponding to the plurality of training samples corresponding to the ith training task.

3. The method of claim 1, wherein, The feature extraction sub-model and the prediction sub-model have the same model architecture.

4. The method of claim 1, wherein, The determination of the prediction results corresponding to the plurality of training samples corresponding to the ith training task by the prediction sub-model in the initial multi-scene model according to the plurality of training samples corresponding to the ith training task and the dataset features corresponding to the plurality of training samples comprises: The dataset features corresponding to the plurality of training samples are input into the bidirectional long short-term memory network in the initial multi-scene model, and the memory dataset features corresponding to the plurality of training samples are determined by the bidirectional long short-term memory network; the memory dataset features include more information than the corresponding dataset features; The determination of the prediction results corresponding to the plurality of training samples corresponding to the ith training task by the prediction sub-model in the initial multi-scene model according to the plurality of training samples corresponding to the ith training task and the memory dataset features corresponding to the plurality of training samples.

5. The method of claim 1, wherein, The inner loop and the outer loop for training the initial multi-scene model by the sub-model loss functions corresponding to the M training tasks and the outer loop learning rate to obtain a multi-scene model comprises: The inner loop for training the initial multi-scene model by the sub-model loss function corresponding to the ith training task to obtain the model update parameters corresponding to the ith training task; The outer loop for training the initial multi-scene model according to the model update parameters corresponding to the M training tasks and the outer loop learning rate to obtain a multi-scene model.

6. The method of claim 5, wherein, The inner loop for training the initial multi-scene model by the sub-model loss function corresponding to the ith training task to obtain the model update parameters corresponding to the ith training task comprises: The inner loop for training the initial multi-scene model by the sub-model loss function corresponding to the ith training task to obtain a plurality of model update sub-parameters corresponding to the ith training task; the plurality of model update sub-parameters corresponding to the ith training task are one-to-one corresponding to the plurality of update steps corresponding to the ith training task; The outer loop for training the initial multi-scene model according to the model update parameters corresponding to the M training tasks and the outer loop learning rate to obtain a multi-scene model comprises: According to the multiple model update sub-parameters and the outer loop learning rate corresponding to the M training tasks, the initial multi-scene model is trained in an outer loop to obtain a multi-scene model.

7. The method of claim 6, wherein, In the inner loop of training the initial multi-scene model by the sub-model loss function corresponding to the i-th training task, the method further comprises: The corresponding sample weights are respectively configured for the multiple-step updates corresponding to the i-th training task.

8. The method of claim 1, wherein, The method further comprises: In the outer loop process of training the initial multi-scene model, the outer loop learning rate is annealed; wherein the outer loop learning rate becomes smaller along with the outer loop process of training the initial multi-scene model.

9. A model determination apparatus characterized by comprising: configured to perform the method of any one of claims 1-8, comprising: An acquisition unit is configured to acquire a plurality of training samples corresponding to M training tasks, M being an integer, M≥2; for an i-th training task in the M training tasks, the plurality of training samples corresponding to the i-th training task correspond to a plurality of training data sets, the plurality of training data sets correspond to a plurality of application scenarios one by one, wherein i is an integer, M≥i≥1, and the training data set includes a plurality of images; A first determination unit is configured to input the plurality of training samples corresponding to the i-th training task into an initial multi-scene model, and determine dataset features corresponding to the plurality of training samples corresponding to the i-th training task by a feature extraction sub-model in the initial multi-scene model; the dataset features are used to represent the correlation between the corresponding training sample and the plurality of training data sets; A second determination unit is configured to determine prediction results corresponding to the plurality of training samples corresponding to the i-th training task by a prediction sub-model in the initial multi-scene model according to the plurality of training samples corresponding to the i-th training task and the dataset features corresponding to the plurality of training samples; A generation unit is configured to generate a sub-model loss function corresponding to the i-th training task based on the difference between the prediction results corresponding to the plurality of training samples corresponding to the i-th training task and the sample labels corresponding to the plurality of training samples; A training unit is configured to train an inner loop and an outer loop of the initial multi-scene model by the sub-model loss functions corresponding to the M training tasks and an outer loop learning rate to obtain a multi-scene model, wherein in a beginning stage of the inner loop of training the initial multi-scene model, the corresponding initial sample weights are respectively configured for the multiple-step updates corresponding to the i-th training task; the plurality of initial sample weights corresponding to the i-th training task have the same value; in the inner loop process of training the initial multi-scene model, the plurality of initial sample weights corresponding to the i-th training task are adjusted; wherein for a beginning update step in the multiple-step updates corresponding to the i-th training task, the initial sample weight corresponding to the beginning update step corresponding to the i-th training task is reduced, and for a subsequent update step in the multiple-step updates corresponding to the i-th training task, the initial sample weight corresponding to the subsequent update step corresponding to the i-th training task is increased.

10. A computer readable storage medium having stored thereon a program which, when executed by a multi-core processor, causes the multi-core processor to perform the method of any one of claims 1-8.

Images