A model training method, device and general model framework

Abstract

The present application discloses a model training method, comprising: obtaining training data, and training a target model using the training data and a general model framework. The general model framework comprises a feature processing module and a task abstraction module, the feature processing module being used to perform feature processing on multimodal data, and the task abstraction module being used to calculate loss functions corresponding to a variety of prediction tasks. Multimodal data includes data of the type to which the training data belongs, and a variety of prediction tasks include prediction tasks of the target model. When training the target model, the feature processing module can be used to perform feature processing on the training data, and the task abstraction model can be used to calculate the loss function. It can be seen that the use of this solution can effectively improve the efficiency of training the target module. As an example, in a scenario where there are many scenarios that require model training, the model training in each scenario can adopt the solution provided by the embodiment of the present application, thereby effectively improving the efficiency of training models corresponding to multiple scenarios.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model training method, device, and general model framework. Background Art

[0002] In some scenarios, models can be trained to predict outcomes. For example, in a content recommendation scenario, a model can be trained to determine the likelihood of a user interacting with content, thereby providing reasonable recommendations.

[0003] The current way to train models is to train corresponding models for each business scenario (such as recommendation scenarios). Accordingly, R&D personnel need to write corresponding software codes for each model, which leads to low efficiency of model training.

[0004] Therefore, a solution is urgently needed to solve the above problems. Summary of the Invention

[0005] In order to solve or at least partially solve the above technical problems, the embodiments of the present application provide a model training method, device and general model framework.

[0006] In a first aspect, an embodiment of the present application provides a model training method, the method comprising:

[0007] Get training data;

[0008] The target model is trained using the training data and the general model framework; wherein:

[0009] The general model framework includes a feature processing module and a task abstraction module. The feature processing module is used to perform feature processing on multimodal data, and the task abstraction module is used to calculate loss functions corresponding to multiple prediction tasks. The multimodal data includes data of the type to which the training data belongs, and the multiple prediction tasks include prediction tasks of the target model. When training the target model, the feature processing module is used to perform feature processing on the training data, and the task abstraction module is used to calculate the loss function. The multimodal data includes at least one of images, videos, audio and text.

[0010] Optionally, the feature processing module includes:

[0011] Feature extraction module and feature fusion module;

[0012] The feature extraction module is used to extract features of each type of data in the multimodal data respectively;

[0013] The feature fusion module is used to fuse features of at least two types of data among the features of the various types of data to obtain fused features.

[0014] Optionally, the feature fusion module includes:

[0015] A feature fusion submodule corresponding to each of the multiple feature fusion methods.

[0016] Optionally, the multiple fusion modes include at least one of the following:

[0017] Average pooling, concatenation fusion, and feature fusion based on attention mechanism.

[0018] Optionally, performing feature processing on the training data using the feature processing module includes:

[0019] The feature extraction module and the target feature fusion submodule in the feature fusion module are used to perform feature processing on the training data.

[0020] Optionally, the multiple prediction tasks include:

[0021] Classification tasks, matching tasks, knowledge distillation tasks, and behavior sequence modeling tasks.

[0022] Optionally, the task abstraction module includes task abstraction submodules corresponding to each prediction task, and the calculation of the loss function using the task abstraction module includes:

[0023] The loss function of the target model is calculated using a task abstraction submodule corresponding to the prediction task of the target model.

[0024] In a second aspect, an embodiment of the present application provides a model training device, the device comprising:

[0025] An acquisition unit, used to acquire training data;

[0026] A training unit is used to train a target model using the training data and a general model framework; wherein:

[0027] The general model framework includes a feature processing module and a task abstraction module. The feature processing module is used to perform feature processing on multimodal data, and the task abstraction module is used to calculate loss functions corresponding to multiple prediction tasks. The multimodal data includes data of the type to which the training data belongs, and the multiple prediction tasks include prediction tasks of the target model. When training the target model, the feature processing module is used to perform feature processing on the training data, and the task abstraction module is used to calculate the loss function. The multimodal data includes at least one of images, videos, audio and text.

[0028] Optionally, the feature processing module includes:

[0029] Feature extraction module and feature fusion module;

[0030] The feature extraction module is used to extract features of each type of data in the multimodal data respectively;

[0031] The feature fusion module is used to fuse features of at least two types of data among the features of the various types of data to obtain fused features.

[0032] Optionally, the feature fusion module includes:

[0033] A feature fusion submodule corresponding to each of the multiple feature fusion methods.

[0034] Optionally, the multiple fusion modes include at least one of the following:

[0035] Average pooling, concatenation fusion, and feature fusion based on attention mechanism.

[0036] Optionally, performing feature processing on the training data using the feature processing module includes:

[0037] The feature extraction module and the target feature fusion submodule in the feature fusion module are used to perform feature processing on the training data.

[0038] Optionally, the multiple prediction tasks include:

[0039] Classification tasks, matching tasks, knowledge distillation tasks, and behavior sequence modeling tasks.

[0040] Optionally, the task abstraction module includes task abstraction submodules corresponding to each prediction task, and the calculation of the loss function using the task abstraction module includes:

[0041] The loss function of the target model is calculated using a task abstraction submodule corresponding to the prediction task of the target model.

[0042] On the third aspect, an embodiment of the present application provides a general model framework, which includes a feature processing module and a task abstraction module. The feature processing module is used to perform feature processing on multimodal data, and the task abstraction module is used to calculate the loss functions corresponding to multiple prediction tasks. The multimodal data includes: at least one of: images, videos, audio and text.

[0043] Optionally, the feature processing module includes:

[0044] Feature extraction module and feature fusion module;

[0045] The feature extraction module is used to extract features of each type of data in the multimodal data respectively;

[0046] The feature fusion module is used to fuse features of at least two types of data among the features of the various types of data to obtain fused features.

[0047] Optionally, the feature fusion module includes:

[0048] A feature fusion submodule corresponding to each of the multiple feature fusion methods.

[0049] Optionally, the multiple fusion modes include at least one of the following:

[0050] Average pooling, concatenation fusion, and feature fusion based on attention mechanism.

[0051] Optionally, the multiple prediction tasks include:

[0052] Classification tasks, matching tasks, knowledge distillation tasks, and behavior sequence modeling tasks.

[0053] In a fourth aspect, an embodiment of the present application provides a device, comprising a processor and a memory;

[0054] The processor is configured to execute instructions stored in the memory, so that the device performs the method as described in any one of the first aspects above.

[0055] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions, wherein the instructions instruct a device to execute a method as described in any one of the above first aspects.

[0056] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0057] Compared with the prior art, the embodiments of the present application have the following advantages:

[0058] The embodiment of the present application provides a model training method, which can obtain training data. After obtaining the training data, the training data and the general model framework can be used to train the target model. The general model framework includes a feature processing module and a task abstraction module. The feature processing module is used to perform feature processing on multimodal data, and the task abstraction module is used to calculate the loss function corresponding to multiple prediction tasks. Therefore, the general model framework can be used to train models corresponding to multiple scenarios. In one example, the multimodal data includes data of the type to which the training data belongs, and the multiple prediction tasks include the prediction tasks of the target model. In this way, when training the target model, the feature processing module can be used to perform feature processing on the training data, and the task abstraction model can be used to calculate the loss function. It can be seen that in the embodiment of the present application, when training the target model, model training is performed based on the general model framework. For R&D personnel, they do not need to write software code corresponding to the target model. They can focus on training the training data of the target model and the prediction tasks of the target model. Using this solution, the efficiency of training the target module can be effectively improved. As an example, in a scenario where there are many scenarios that require model training, the model training in each scenario can adopt the solution provided by the embodiments of the present application, thereby effectively improving the efficiency of training models corresponding to multiple scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1 A flowchart of a model training method provided in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of the structure of a general model framework provided in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of the structure of a model training device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0064] After research, the inventors of this application discovered that in some business scenarios, models can be trained to predict results. For example, in the context of content recommendation, a model can be trained to determine the likelihood of user interaction with content, thereby reasonably recommending content to users. Currently, content comes in a variety of genres, including short videos, videos, articles, and so on. Each genre of content can also include multimodal data. For example, for videos, it includes data such as video titles, video frames, voice, background music, and so on.

[0065] The current model training method is to train a corresponding model for each business scenario (such as recommendation scenario). For example: for the scenario of video recommendation article, a corresponding model is trained; for the scenario of article recommendation video, a corresponding model is trained; for the scenario of article recommendation article, a corresponding model is trained; for the scenario of video recommendation video, a corresponding model is trained.

[0066] For each model to be trained, developers need to write the corresponding software code for each model. This takes a considerable amount of time, resulting in low model training efficiency. This is especially true when there are many models to train, as writing the corresponding software code for each model individually makes model training particularly inefficient.

[0067] In order to solve the above problems, the embodiments of the present application provide a model training method, device and general model framework.

[0068] Various non-limiting embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0069] Exemplary Methods

[0070] See also Figure 1 , which is a flow chart of a model training method provided in an embodiment of the present application. The model training method provided in an embodiment of the present application can be executed by a client or a server, and the embodiment of the present application does not make any specific limitation.

[0071] In this embodiment, Figure 1 The method shown may include, for example, the following steps: S101 - S102 .

[0072] S101: Obtain training data.

[0073] The embodiments of the present application do not specifically limit the training data, and the training data is related to the prediction task of the target model to be trained. As an example, if the prediction task is to predict the possibility of an article or video pushed to a user by a video playback application (Application, APP) being viewed by the user, the training data may, for example, include information about videos that the user has historically watched in the video playback APP. The video information mentioned here may be one or more of the video title, video frame, voice, and background music of the video. As another example, if the prediction task is to predict the possibility of a video pushed to a user by a article reading APP being viewed by the user, the training data may, for example, include information about articles that the user has historically watched in the article reading APP. The article information mentioned here may be one or more of the article title, article body, and so on.

[0074] S102: Utilize the training data and the general model framework to train the target model, wherein: the general model framework includes a feature processing module and a task abstraction module, the feature processing module is used to perform feature processing on multimodal data, and the task abstraction module is used to calculate loss functions corresponding to multiple prediction tasks; the multimodal data includes data of the type to which the training data belongs, and the multiple prediction tasks include prediction tasks of the target model; when training the target model, the feature processing module is used to perform feature processing on the training data, and the task abstraction model is used to calculate the loss function, and the multimodal data includes: at least one of: image, video, audio and text.

[0075] In the embodiments of the present application, when using the training data to train the target model, the R&D personnel do not need to separately write corresponding software code for the target model. In other words, the target model does not need to be trained using software code written by the R&D personnel for the target model. Instead, the target model is trained using a general model framework. In other words, after obtaining the training data, the target model can be trained using the training data and the general model framework.

[0076] Regarding the general model framework, it should be noted that it can be used to train models for a variety of business scenarios. In one example, the software code corresponding to the general model framework can be pre-written. In this way, when models corresponding to multiple business scenarios need to be trained, the general model framework can be used for training, eliminating the need for R&D personnel to write corresponding software code for each business scenario, thereby improving the efficiency of model training.

[0077] In the embodiment of the present application, the general model framework includes a feature processing module and a task abstraction module.

[0078] The feature processing module is used to perform feature processing on multimodal data. The multimodal data mentioned here includes but is not limited to at least one of images, videos, audio and text.

[0079] In one example, considering that in some business scenarios, the training data used to train the model may include multimodal data, in this case, it is often necessary to fuse the features of the multimodal data included in the training data. The general model framework is applicable to a variety of business scenarios. Therefore, in one example, the feature processing module may include a feature extraction module and a feature fusion module. Among them:

[0080] The feature extraction module is used to extract features of each type of data in the multimodal data. As an example, the feature extraction module may include multiple feature processing submodules, each of which is used to extract features of one type of data. For example, the feature processing module includes three submodules, namely a text feature extraction submodule, an image feature extraction submodule, and an audio feature extraction submodule. The text feature extraction submodule is used to extract features of text, the image feature extraction submodule is used to extract features of images, and the audio feature extraction submodule is used to extract features of audio.

[0081] The feature fusion module is used to fuse the features of at least two types of data among the features of the various types of data to obtain fused features. As an example, considering that for different business scenarios, the data types corresponding to the features that need to be fused are different. The general module framework can be applied to various business scenarios. Therefore, the fusion module can fuse the features of at least two types of data among the features of the various types of data to obtain fused features. For example, the feature fusion module can fuse the features of text and the features of images; the feature fusion module can fuse the features of text and the features of audio; the feature fusion module can fuse the features of images and the features of audio; the feature fusion module can also fuse the features of text, the features of audio, and the features of images.

[0082] In one example, considering the fusion of features of multimodal data, there can be multiple fusion methods. Different fusion methods may bring different model training effects. In one example, in order to make the general model framework applicable to the requirements of model training effects in different business scenarios, the feature fusion module may include feature fusion sub-modules corresponding to each of the multiple feature fusion methods. When training the target model, R&D personnel can select the corresponding feature fusion sub-module to participate in the training of the target model according to the specific business scenario.

[0083] Regarding the multiple fusion methods, the embodiments of the present application do not specifically limit them. In one example, the multiple fusion methods may include mean pooling, concatenate fusion, and feature fusion based on attention mechanism. Among them:

[0084] Average pooling: refers to averaging the features of multimodal data, and then inputting the averaged features into several layers of fully connected neural networks to obtain fused features.

[0085] Cascade fusion: This involves concatenating the features of multimodal data along the feature dimension and then passing them through several layers of fully connected neural networks to generate fused features. Compared to average pooling, cascade fusion can more precisely model the information interactions between multimodal data.

[0086] Feature fusion based on the attention mechanism, also known as cross-modality attention (CMA) fusion, refers to the use of transformers to model the interaction between multimodal data. CMA fusion is not described in detail here.

[0087] In one example, if the training data used to train the model only includes one type of data, for example, only text. In this case, in one example, it is not necessary to fuse the features output by the feature extraction module. In another example, the feature fusion module can use a single mapping method to process the features output by the feature extraction module. The so-called single mapping means that the feature fusion module does not perform special processing on the features output by the feature extraction module and directly outputs the features output by the feature extraction module.

[0088] In one example, when the training data includes multimodal data, the feature processing module may be used to perform feature processing on the training data. In a specific implementation, the feature extraction module and the target feature fusion submodule in the feature fusion module may be used to perform feature processing on the training data. For example, at least one feature extraction submodule corresponding to the training data may be used to extract features corresponding to various types of data in the training data, and then the target feature fusion submodule in the feature fusion module may be used to perform feature fusion on the features output by the at least one feature extraction submodule.

[0089] For example, if the training data includes text, images, and audio, the text feature extraction submodule can be used to extract features from the text in the training data to obtain text features. The image feature extraction submodule can be used to extract features from the images in the training data to obtain image features. The audio feature extraction submodule can be used to extract features from the audio in the training data to obtain audio features. Subsequently, the feature fusion submodule corresponding to the attention-based feature fusion can be used to fuse the text features, image features, and audio features to obtain fused features.

[0090] As mentioned above, the general model framework includes a feature processing module and a task abstraction module. Next, the task abstraction module will be introduced.

[0091] In an embodiment of the present application, the task abstraction module is used to calculate loss functions corresponding to various prediction tasks. The prediction tasks are related to the business scenarios corresponding to the models. The present embodiment does not specifically limit the various prediction tasks. In one example, the various prediction tasks may include: classification tasks, matching tasks, knowledge distillation tasks, and behavior sequence modeling tasks.

[0092] A classification task can be categorizing data, for example, identifying the data's category. In this case, when training the target model, the loss function can be directly calculated using the output of the feature fusion module and the labels of the training data. The labels of the training data indicate the actual classification of the training data.

[0093] The so-called matching task can be modeling the degree of matching between multiple training samples (for example, two training samples). In this case, a loss function can be calculated based on the fused features corresponding to the multiple training samples. For example, the loss function can be calculated based on training sample 1 and training sample 2. In another example, the loss function can be calculated using the MoCo model.

[0094] The so-called knowledge distillation task can be to use the output of the teacher network to supervise the learning of the student network. The teacher network mentioned here can be the fusion feature obtained by feature processing different data, and the corresponding student network can be the fusion feature obtained by feature processing the training data.

[0095] The so-called behavior sequence modeling task can be to predict a user's future behavior using their historical behavior. In this case, the user's historical behavior can correspond to multiple fused features. These multiple fused features can be further fused to produce a feature that can be used to predict the user's future behavior. For example, the (N-1) videos a user has clicked on can correspond to N fused features. These (N-1) fused features can be fused together to predict whether the next video will be clicked. In this case, during the model training phase, assuming that the user's historical behavior includes M actions, a loss function can be calculated based on the fused features of the first (M-1) actions and the fused features of the Mth action. In one example, the task abstraction module can include task abstraction submodules corresponding to each prediction task. For example, these can include task abstraction submodules corresponding to classification tasks, matching tasks, knowledge distillation tasks, and behavior sequence modeling tasks. In this case, when training the target model, the task abstraction submodule corresponding to the prediction task can be used to calculate the target model's loss function. Accordingly, the calculated loss function is used to update the target model's parameters. For example, if a target model is used to predict whether a user will click on a video recommended to them based on their historical clicks on videos, the loss function of the target model can be calculated using the task abstraction submodule corresponding to the behavior sequence modeling task.

[0096] Next, combine Figure 2 Introduce this general model framework.

[0097] See also Figure 2 , which is a structural diagram of a general model framework provided in an embodiment of the present application.

[0098] like Figure 2As shown, the general model framework includes a feature processing module and a task abstraction model, wherein the feature processing module includes: a feature extraction module and a feature fusion module, and the feature extraction module includes a text feature extraction submodule, an image feature extraction submodule, and an audio feature extraction submodule. The feature fusion module includes a fusion submodule corresponding to average pooling, a fusion submodule corresponding to serial fusion, a fusion submodule corresponding to cross-model attention fusion, and a fusion submodule corresponding to single mapping. The task abstraction module includes a task abstraction submodule corresponding to the classification task, a task abstraction submodule corresponding to the matching task, a task abstraction submodule corresponding to the knowledge distillation task, and a task abstraction submodule corresponding to behavior sequence modeling.

[0099] When training the target model, the training data is first processed by one or more submodules in the feature extraction module. The features output by one or more submodules in the feature extraction module are then processed by a fusion submodule (specifically, this can be specified by the developer). Furthermore, the output of this fusion submodule is processed by a task abstraction submodule (specifically, this can be specified by the developer) to obtain a corresponding loss function. Furthermore, based on this loss function, the parameters of the target model are updated.

[0100] It should be noted that Figure 2 It is only for the convenience of understanding the solution of the embodiment of the present application. The structure of the general model framework is not limited to Figure 2 As shown, for example, the feature extraction module may include Figure 2 In addition to the three submodules shown, other submodules may also be included. Similarly, the feature fusion module includes Figure 2 In addition to the four sub-modules shown, other sub-modules may also be included; the task abstraction module includes Figure 2 In addition to the four sub-modules shown, other sub-modules may also be included, etc., which will not be described in detail here.

[0101] Exemplary devices

[0102] Based on the method provided in the above embodiment, the embodiment of the present application further provides a device, which is described below in conjunction with the accompanying drawings.

[0103] See also Figure 3 , which is a schematic diagram of the structure of a model training device provided in an embodiment of the present application. The device 300 may specifically include: an acquisition unit 301 and a training unit 302.

[0104] An acquisition unit 301 is used to acquire training data;

[0105] The training unit 302 is configured to train a target model using the training data and the general model framework; wherein:

[0106] The general model framework includes a feature processing module and a task abstraction module. The feature processing module is used to perform feature processing on multimodal data, and the task abstraction module is used to calculate loss functions corresponding to multiple prediction tasks. The multimodal data includes data of the type to which the training data belongs, and the multiple prediction tasks include prediction tasks of the target model. When training the target model, the feature processing module is used to perform feature processing on the training data, and the task abstraction module is used to calculate the loss function. The multimodal data includes at least one of images, videos, audio and text.

[0107] Optionally, the feature processing module includes:

[0108] Feature extraction module and feature fusion module;

[0109] The feature extraction module is used to extract features of each type of data in the multimodal data respectively;

[0110] The feature fusion module is used to fuse features of at least two types of data among the features of the various types of data to obtain fused features.

[0111] Optionally, the feature fusion module includes:

[0112] A feature fusion submodule corresponding to each of the multiple feature fusion methods.

[0113] Optionally, the multiple fusion modes include at least one of the following:

[0114] Average pooling, concatenation fusion, and feature fusion based on attention mechanism.

[0115] Optionally, performing feature processing on the training data using the feature processing module includes:

[0116] The feature extraction module and the target feature fusion submodule in the feature fusion module are used to perform feature processing on the training data.

[0117] Optionally, the multiple prediction tasks include:

[0118] Classification tasks, matching tasks, knowledge distillation tasks, and behavior sequence modeling tasks.

[0119] Optionally, the task abstraction module includes task abstraction submodules corresponding to each prediction task, and the calculation of the loss function using the task abstraction module includes:

[0120] The loss function of the target model is calculated using a task abstraction submodule corresponding to the prediction task of the target model.

[0121] Since the device 300 is a device corresponding to the method provided in the above method embodiment, the specific implementation of each unit of the device 300 is based on the same concept as the above method embodiment. Therefore, for the specific implementation of each unit of the device 300, please refer to the description part of the above method embodiment, which will not be repeated here.

[0122] An embodiment of the present application provides a general model framework, which includes a feature processing module and a task abstraction module. The feature processing module is used to perform feature processing on multimodal data, and the task abstraction module is used to calculate loss functions corresponding to multiple prediction tasks. The multimodal data includes at least one of: images, videos, audio and text.

[0123] Optionally, the feature processing module includes:

[0124] Feature extraction module and feature fusion module;

[0125] The feature extraction module is used to extract features of each type of data in the multimodal data respectively;

[0126] The feature fusion module is used to fuse features of at least two types of data among the features of the various types of data to obtain fused features.

[0127] Optionally, the feature fusion module includes:

[0128] A feature fusion submodule corresponding to each of the multiple feature fusion methods.

[0129] Optionally, the multiple fusion modes include at least one of the following:

[0130] Average pooling, concatenation fusion, and feature fusion based on attention mechanism.

[0131] Optionally, the multiple prediction tasks include:

[0132] Classification tasks, matching tasks, knowledge distillation tasks, and behavior sequence modeling tasks.

[0133] Regarding the general model framework, please refer to the relevant description section above and will not be repeated here.

[0134] An embodiment of the present application further provides a device, the device comprising a processor and a memory;

[0135] The processor is used to execute instructions stored in the memory so that the device executes the model training method provided by the above method embodiment.

[0136] An embodiment of the present application provides a computer-readable storage medium, including instructions, which instruct a device to execute the model training method provided by the above method embodiment.

[0137] An embodiment of the present application also provides a computer program product, which, when run on a computer, enables the computer to execute the model training method provided by the above method embodiment.

[0138] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0139] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0140] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A model training method, characterized in that: The method comprises: Acquire training data, where the training data includes: information about videos that the user has historically watched in a video playback application, or information about articles that the user has historically watched in an article reading application; The target model is trained using the training data and the general model framework; wherein: The general model framework is used to train models corresponding to various business scenarios. The general model framework includes a feature processing module and a task abstraction module. The feature processing module is used to perform feature processing on multimodal data, and the task abstraction module is used to calculate loss functions corresponding to various prediction tasks. The multimodal data includes data of the type to which the training data belongs, and the various prediction tasks include prediction tasks of the target model. When training the target model, the feature processing module is used to perform feature processing on the training data, and the task abstraction module is used to calculate the loss function. The multimodal data includes at least one of images, videos, audio and text. The various prediction tasks include predicting the possibility of an article or video pushed to a user in the video playback application being viewed by the user, and predicting the possibility of a video pushed to a user in the article reading application being viewed by the user.

2. The method according to claim 1, characterized in that The feature processing module includes: Feature extraction module and feature fusion module; The feature extraction module is used to extract features of each type of data in the multimodal data respectively; The feature fusion module is used to fuse features of at least two types of data among the features of the various types of data to obtain fused features.

3. The method according to claim 2, characterized in that The feature fusion module includes: A feature fusion submodule corresponding to each of the multiple feature fusion methods.

4. The method according to claim 3, characterized in that The multiple feature fusion methods include at least one of the following: Average pooling, concatenation fusion, and feature fusion based on attention mechanism.

5. The method according to claim 3, characterized in that Performing feature processing on the training data using the feature processing module includes: The feature extraction module and the target feature fusion submodule in the feature fusion module are used to perform feature processing on the training data.

6. The method according to claim 1, characterized in that The various prediction tasks include: Classification tasks, matching tasks, knowledge distillation tasks, and behavior sequence modeling tasks.

7. The method according to claim 6, characterized in that The task abstraction module includes task abstraction submodules corresponding to each prediction task. The calculation of the loss function using the task abstraction module includes: The loss function of the target model is calculated using a task abstraction submodule corresponding to the prediction task of the target model.

8. A model training device, characterized in that: The device comprises: An acquisition unit, configured to acquire training data, wherein the training data includes information about videos that a user has historically watched in a video playback application, or information about articles that a user has historically watched in an article reading application; A training unit is used to train a target model using the training data and a general model framework; wherein: The general model framework is used to train models corresponding to various business scenarios. The general model framework includes a feature processing module and a task abstraction module. The feature processing module is used to perform feature processing on multimodal data, and the task abstraction module is used to calculate loss functions corresponding to various prediction tasks. The multimodal data includes data of the type to which the training data belongs, and the various prediction tasks include prediction tasks of the target model. When training the target model, the feature processing module is used to perform feature processing on the training data, and the task abstraction module is used to calculate the loss function. The multimodal data includes at least one of images, videos, audio and text. The various prediction tasks include predicting the possibility of an article or video pushed to a user in the video playback application being viewed by the user, and predicting the possibility of a video pushed to a user in the article reading application being viewed by the user.

9. A general model framework, characterized in that The general model framework is used to train models corresponding to various business scenarios. The general model framework includes a feature processing module and a task abstraction module. The feature processing module is used to perform feature processing on multimodal data. The task abstraction module is used to calculate loss functions corresponding to various prediction tasks. The multimodal data includes: at least one of: images, videos, audio and text. The various prediction tasks include: predicting the possibility of articles or videos pushed to users by video playback applications being viewed by users, and predicting the possibility of videos pushed to users by article reading applications being viewed by users.

10. A device, characterized in that The device includes a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the device performs the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that The method comprises instructions for instructing a device to execute the method according to any one of claims 1 to 7.

Images