Training method and device of model

Abstract

The application provides a model training method and device. The model training method comprises: obtaining fusion feature information based on input data, wherein the fusion feature information fuses multi-modal information contained in the input data; pre-training a model for a plurality of basic tasks based on the fusion feature information, and training the model for at least one target task to update parameters of the model. The technical solution of the application can improve the processing capability of the model for multi-modal information, and thus can improve the performance of the model on the target task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a model training method and device. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, the processing of multi-modal information has attracted more and more attention and application. The processing of multi-modal information mainly involves computer vision (CV) and natural language processing (NLP) technology. Generally, the model obtained by training based on the extracted multi-modal information can implement inference tasks for complex scenes. However, the existing training method is difficult to obtain a high-performance model, that is, the performance of the model has a bottleneck, especially for inference tasks of more complex scenes. SUMMARY

[0003] Therefore, the embodiments of the present application provide a model training method and device, which can improve the processing capability of the model for multi-modal information, and thus can improve the performance of the model on target tasks.

[0004] In a first aspect, the embodiments of the present application provide a model training method, comprising: obtaining fusion feature information based on input data, wherein the fusion feature information fuses multi-modal information contained in the input data; pre-training a model on a plurality of basic tasks based on the fusion feature information, and training the model on at least one target task to update parameters of the model.

[0005] In some embodiments of the present application, the pre-training of the model on the plurality of basic tasks based on the fusion feature information comprises: obtaining a classification result corresponding to each basic task in the plurality of basic tasks based on the fusion feature information; and pre-training the model based on the plurality of classification results corresponding to the plurality of basic tasks.

[0006] In some embodiments of the present application, the plurality of basic tasks includes at least two of a disorderly rearrangement task, an object detection task, a text recognition task, and a speech recognition task.

[0007] In some embodiments of the present application, the input data includes at least one image, wherein the obtaining of the fusion feature information based on the input data comprises: partitioning each image in the at least one image to obtain a plurality of regions corresponding to the image; obtaining region feature information corresponding to each region in the plurality of regions, the region feature information fusing multi-modal information contained in the region, and the fusion feature information including a plurality of region feature information corresponding to the plurality of regions.

[0008] In some embodiments of the present application, the obtaining the fusion feature information based on the input data further comprises: performing position encoding on each of the multiple regions to obtain first position encoding information matched with the region feature information of the region, and the fusion feature information further comprises multiple first position encoding information matched with the multiple region feature information.

[0009] In some embodiments of the present application, the multiple basic tasks comprise a disorderly rearrangement task, and the pre-training the model on the multiple basic tasks based on the fusion feature information comprises: performing disorderly matching on the multiple region feature information and the multiple first position encoding information in the fusion feature information to obtain disorderly feature information, the disorderly feature information comprising the multiple region feature information and multiple second position encoding information matched with the multiple region feature information; and pre-training the model on the disorderly rearrangement task based on the disorderly feature information.

[0010] In some embodiments of the present application, the multiple basic tasks further comprise an object detection task and / or a text recognition task, and the pre-training the model on the multiple basic tasks based on the fusion feature information further comprises: pre-training the model on the object detection task and / or the text recognition task based on the fusion feature information.

[0011] In some embodiments of the present application, the training the model on the at least one target task to update the parameters of the model comprises: during the pre-training process, training the model on the at least one target task based on a reinforcement learning method to update the parameters of the model.

[0012] In some embodiments of the present application, the training the model on the at least one target task based on the reinforcement learning method to update the parameters of the model comprises: using the model to perform inference based on sample data corresponding to each of the at least one target task to obtain a target inference result; evaluating the target inference result based on a preset evaluation index to obtain a reward signal; and updating the parameters of the model based on the reward signal.

[0013] In some embodiments of the present application, the training the model on the at least one target task to update the parameters of the model comprises: training the model on the at least one target task based on a supervised learning method to update the parameters of the model.

[0014] In some embodiments of the present application, the at least one target task comprises at least one of the following: visual question answering, layout analysis, product recommendation, scene description, and image-text retrieval.

[0015] In a second aspect, embodiments of the present application provide a model training apparatus, comprising: an acquisition module configured to acquire fusion feature information based on input data, wherein the fusion feature information fuses multi-modal information contained in the input data; and a training module configured to pre-train a model on a plurality of base tasks based on the fusion feature information, and train the model on at least one target task to update parameters of the model.

[0016] In a third aspect, embodiments of the present application provide an electronic device, comprising: a processor; and a memory configured to store processor-executable instructions, wherein the processor is configured to execute the model training method of the first aspect.

[0017] In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program configured to execute the model training method of the first aspect.

[0018] Embodiments of the present application provide a model training method and apparatus, which acquires fusion feature information that fuses multi-modal information contained in input data based on the input data, and trains a model on base tasks and target tasks based on the fusion feature information, so that the model can extract unified modal feature information, improve the processing capability of the model on multi-modal information, and further improve the performance of the model on the target task. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Fig. 1 shows a system architecture schematic diagram of a model training system according to an example embodiment of the present application.

[0020] Figure 2 Fig. 2 shows a flow schematic diagram of a model training method according to an example embodiment of the present application.

[0021] Figure 3 Fig. 3 shows a flow schematic diagram of a model training method according to another example embodiment of the present application.

[0022] Figure 4 Fig. 4 shows a structure schematic diagram of a model according to an example embodiment of the present application.

[0023] Figure 5 Fig. 5 shows a structure schematic diagram of a model training apparatus according to an example embodiment of the present application.

[0024] Figure 6 Fig. 6 shows a block diagram of an electronic device configured to execute the model training method according to an example embodiment of the present application. DETAILED DESCRIPTION

[0025] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0026] SUMMARY

[0027] The processing of multi-modal information mainly involves computer vision and natural language processing technology. Computer vision is a science of teaching machines to "understand" the content of a scene, and the types of tasks involved can include image recognition, object detection, image segmentation, object tracking, video monitoring, etc. The core purpose of natural language processing is to teach machines to "understand" the natural language used by humans in daily life. Common natural language processing tasks include machine translation, automatic text summarization, text classification, text question answering, reading comprehension, text semantic analysis, speech recognition, language modeling, optical character recognition (OCR), etc. Computer vision tasks and natural language processing tasks involving single modal information can be regarded as tasks in a simple scenario.

[0028] With the development of intelligence, people have more and more demands for tasks in complex scenarios, and the performance requirements for inference models in complex scenarios are getting higher and higher. For example, given a scene (picture or video), the inference model needs to understand the complex multi-modal information (such as objects and texts) in the scene, and generate a text description that conforms to the semantics of the scene, at this time the inference task can be called scene description (IC), or answer the questions related to the scene, at this time the inference task can be called visual question answering (VQA). Scene description technology can be used for automatic captioning of movies, helping visually impaired people quickly understand the surrounding environment, etc. Visual question answering technology can help humans explore unknown environments in an interactive way, such as being used for visual navigation and chat robots, etc. Therefore, it is of great practical significance to research and solve inference tasks in complex scenarios. Accurate understanding of multi-modal information in complex scenarios by inference models is the key to improving model performance.

[0029] In an embodiment, a method for training an inference model can include: a first stage, extracting text feature information from input data using a text encoder, and extracting object feature information from input data using an object encoder; a second stage, training the model based on the text feature information and the object feature information for a basic task, and the trained model can have the function of identifying occluded texts or objects; a third stage, training the trained model for a target task.

[0030] This training method can obtain an inference model capable of handling multimodal information. It first trains the model on a basic task, then trains the model on the target task, addressing the issue of limited target task samples. However, because the second-stage training process handles information from two different modalities acquired using models with different structures (e.g., text encoders and object encoders), it's difficult to effectively integrate this information. This limits the model's inference performance on the basic task, and consequently, its inference performance on the target task.

[0031] Exemplary System

[0032] Figure 1 The diagram shown is a schematic representation of the system architecture of a model training system 100 provided in an exemplary embodiment of this application, illustrating an application scenario for training a model based on input data containing multimodal information. The training system 100 includes a first computing device 110.

[0033] The first computing device 110 may be a terminal or a server, etc.

[0034] A model can be run on the first computing device 110. Specifically, the first computing device 110 can train the model, such as by training the model based on input data. Specific training methods can be found in the description of the exemplary methods section below. Furthermore, the first computing device 110 can use the trained model to perform corresponding tasks.

[0035] In one embodiment, the input data may be stored on the first computing device 110; in other embodiments, the input data may be stored on other computing devices, such as the second computing device 120. The first computing device 110 may obtain the input data from the second computing device 120 and then train the model based on the input data. The second computing device 120 is similar to the first computing device 110 and may be a terminal or server, etc.

[0036] It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited thereto. Rather, the embodiments of this application can be applied to any applicable scenario.

[0037] Exemplary methods

[0038] Figure 2 The diagram shown is a flowchart illustrating a training method for a model provided in an exemplary embodiment of this application. Figure 2 The method can be executed by a computing device. For example... Figure 2 As shown, the training method for this model includes the following.

[0039] 210: obtain fusion feature information based on the input data, wherein the fusion feature information fuses the multi-modal information contained in the input data.

[0040] Specifically, the input data can include at least one image, and the at least one image contains multi-modal information such as text information and object information. The at least one image can include one or more images, for example, one or more images in a video or one or more images collected by photographing or the like. The input data can be used as sample data corresponding to a basic task.

[0041] Alternatively, the input data can include a video clip, that is, at least one image and voice information corresponding to the at least one image. At this time, the multi-modal information can include voice information and object information in the at least one image, or voice information, object information in the at least one image and text information.

[0042] The fusion feature information can be directly extracted from the input data, and the fusion feature information fuses the multi-modal information contained in the input data, which is equivalent to extracting feature information of a unified modality from the input data. For example, the input data is input into a model, and the model extracts the fusion feature information based on the input data. In this way, the model can be trained by avoiding extracting feature information of different modalities from the input data, thereby solving the problems that the model is difficult to well fuse feature information of different modalities and the performance of the model is difficult to improve.

[0043] For example, feature information of different modalities is extracted by different models (such as encoders). The models can be pre-trained and used to extract feature information of a single modality. Since the models for obtaining feature information of different modalities differ in structure, training method and training samples, the feature information of different modalities extracted by different models is difficult to well fuse. Training a model for performing a target task by using these different modalities of feature information which are difficult to well fuse will limit the performance of the model. The embodiments of the present application can avoid separately extracting feature information of different modalities by directly extracting fusion feature information from the input data, unify the feature extraction process, obtain feature information (fusion feature information) of a unified modality (one type), which can improve the fusion ability of the model for multi-modal information and further improve the performance of the model.

[0044] 220: pre-train the model for multiple basic tasks based on the fusion feature information, and train the model for at least one target task to update parameters of the model.

[0045] The base task in the embodiments of the present application can also be referred to as an upstream task, and the target task can also be referred to as a downstream task. Specifically, in terms of the reasoning process, the base task can be a simple task relative to the target task, and the training of the base task can enable the model to have a basic ability to solve the target task. The base task can include at least two of the following: object detection, text recognition, speech recognition, and the like. The target task can be a complex reasoning task, such as scene description and / or visual question answering, and the like.

[0046] The training process of the base task has a certain guiding effect on the model to solve the target task, and can provide a large amount of prior knowledge. For example, training the model to do the target task can be compared to training a pupil to do the college entrance examination question. Here, the pupil is equivalent to the randomly initialized model. If the pupil is directly asked to solve the college entrance examination question (target task), it is very difficult. It is necessary to first teach the pupil junior high school knowledge (base task). However, it is impossible to really answer the college entrance examination question by only doing the junior high school question. Therefore, the model needs to be trained in combination with the base task and the target task.

[0047] In an implementation manner, the model can be first subjected to a pre-training process, and a trained intermediate model can be obtained after the pre-training process ends, and then the intermediate model is further trained for the target task to obtain a final model. For example, the model is first pre-trained for multiple base tasks based on the fused feature information, and then the model is further trained based on the sample data corresponding to the target task to update the parameters of the model.

[0048] In another implementation manner, the model can be simultaneously subjected to the training processes of the base task and the target task. For example, the model is pre-trained for multiple base tasks based on the fused feature information, and the model is trained based on the sample data corresponding to the target task to update the parameters of the model in real time. Here, the pre-training process of the base task can be started before the training of the target task, but the training process of the target task does not need to wait until the pre-training process of the model is completely finished. For example, the training process of the target task can be started after the pre-training process of the model meets a certain preset condition. The preset condition can be that the pre-training process has proceeded to a certain degree, such as 20%, 30%, 40%, 50%, and the like. In an embodiment, the sample data corresponding to the target task can be input into the model immediately after the input data corresponding to the base task is input into the model. That is, the training of the base task and the target task has a sequence, but the two training processes are not completely separated, and can be regarded as one training process, and the training of the target task does not need to wait until the model is completely trained for the base task.

[0049] The embodiment of the present application provides a model training method, which obtains fusion feature information fused with multi-modal information contained in input data based on the input data, and trains a model based on the fusion feature information to perform a basic task and a target task, so that the model can extract unified modal feature information, improves the processing capability of the model on the multi-modal information, and further improves the performance of the model on the target task.

[0050] According to an embodiment of the present application, the pre-training of the model based on the fusion feature information includes: obtaining a classification result corresponding to each of the plurality of basic tasks based on the fusion feature information; and pre-training the model based on the plurality of classification results corresponding to the plurality of basic tasks.

[0051] Specifically, in the pre-training process, the model can be trained on the plurality of basic tasks. For example, based on the same fusion feature information, the model can be trained on the plurality of basic tasks, and the training processes of the plurality of basic tasks can be performed in parallel.

[0052] In the training process of the model on the basic task, each basic task can correspond to a classifier, and the parameters of the model can be adjusted based on the classification result of the classifier. For example, the parameters of the model are adjusted based on the difference between the classification result of the classifier and the annotation result corresponding to the input data, and the difference between the classification result and the annotation result can be represented by a loss function.

[0053] The embodiment of the present application can improve the generalization capability of the model by pre-training the model on the plurality of basic tasks for the same input data, and lay a good foundation for the model to have good inference performance on the target task.

[0054] According to an embodiment of the present application, the plurality of basic tasks includes at least two of a disorderly rearrangement task, an object detection task, a text recognition task, and a speech recognition task.

[0055] Specifically, the plurality of basic tasks can be set according to the number and type of the target task. For example, when the target task includes scene description and / or commodity recommendation, the basic task can include text recognition and object detection. When the target task includes layout analysis, the basic task can include text recognition, object detection, and disorderly rearrangement. When the target task includes visual question and answer, the basic task can include text recognition, object detection, and speech recognition.

[0056] When the types of the target task are more, a plurality of basic tasks can be set, so that the finally obtained model can have the capability of processing a plurality of target tasks.

[0057] The embodiments of the present application can improve the generalization ability of the model by setting multiple basic tasks, facilitate the training of the model for multiple target tasks, and improve the adaptability of the model. In addition, pre-training the model for multiple basic tasks can strengthen the model and improve the inference ability of the model for target tasks.

[0058] In an embodiment, the at least one target task includes at least one of the following: visual question answering, layout analysis, commodity recommendation, scene description, and image-text retrieval.

[0059] Specifically, the target task can be one or more, and the more the number of target tasks, the better the adaptability of the model, which can adapt to the inference process in multiple complex scenarios. According to the complexity of each target task, multiple basic tasks can be set to pre-train the model to lay the foundation for the training of the target task. For example, the multiple basic tasks can include a disorder rearrangement task, an object detection task, and a text recognition task. Based on these basic tasks, the model can adapt to the inference process of the following multiple target tasks: visual question answering, layout analysis, commodity recommendation, scene description, and image-text retrieval.

[0060] According to an embodiment of the present application, the input data includes at least one frame of image, wherein the fusion feature information is obtained based on the input data, including: partitioning each frame of image in the at least one frame of image to obtain a plurality of regions corresponding to the image; obtaining region feature information corresponding to each region in the plurality of regions, the region feature information fuses the multi-modal information contained in the region, and the fusion feature information includes a plurality of region feature information corresponding to the plurality of regions.

[0061] Specifically, a complete fusion feature information can be extracted for each frame of image, that is, the fusion feature information fuses the multi-modal information contained in the entire image, and then the model is pre-trained based on the fusion feature information, which can reduce the amount of calculation and improve the training efficiency of the model.

[0062] Optionally, each frame of image can be partitioned to obtain a plurality of regions corresponding to the image, and region feature information corresponding to each region is obtained respectively, that is, the region feature information fuses the multi-modal information contained in the region. The shape and size of each region can be the same or different, for example, the image can be evenly divided into multiple regions. By partitioning the image to obtain region feature information corresponding to different regions, the model can better learn local information, thereby improving the inference effect of the model.

[0063] Further, the model can adopt a Transformer architecture, which has good performance in natural language processing tasks and computer vision tasks. The self-attention mechanism in the Transformer architecture can realize sufficient interaction between region feature information corresponding to different regions, thereby facilitating the model to better construct global information. The sufficient fusion of local information (region feature information) and global information can obtain more rich feature information, thereby facilitating the improvement of the performance of the model and enabling the model to better complete the target task.

[0064] In some embodiments, if the multiple base tasks include speech recognition, the fused feature information can fuse the multi-modal information (such as text information, object information, and speech information) corresponding to the entire image, or the region feature information can fuse the multi-modal information (such as text information, object information, and speech information) corresponding to the region. That is, the fused feature information / region feature information fuses the information of multiple modalities contained in the input data.

[0065] The embodiments of the present application can obtain multiple regions corresponding to the image by partitioning the image, obtain the corresponding region feature information for each region, and then pre-train the model based on the region feature information corresponding to each partition, so that the model can better learn local information and construct global information based on the local information. The training process combining local information and global information can obtain a model with better performance.

[0066] Further, based on the input data, obtaining the fused feature information further includes: performing position encoding on each region in the multiple regions to obtain first position encoding information matched with the region feature information of the region, and the fused feature information further includes multiple first position encoding information matched with the multiple region feature information.

[0067] In an implementation, the input data can include an image frame, and the multiple first position encoding information obtained by performing position encoding on each region of the image can represent the positions of the multiple regions in the image, that is, can be used to distinguish the region feature information corresponding to different regions. For example, the multiple regions in the image can be position encoded in a clockwise or counterclockwise direction.

[0068] In another implementation, the input data can include multiple image frames, and the multiple image frames can have a certain order. The multiple first position encoding information obtained by performing position encoding on each region of each image in the multiple image frames can represent the position of the image to which the multiple regions belong in the multiple image frames and the specific position in the image.

[0069] In this embodiment, the position encoding method can be decimal, binary, or other methods. The specific partitioning method and position encoding method can be set according to actual needs, and this embodiment does not limit them.

[0070] This application embodiment obtains first location encoding information that matches the regional feature information of each region by performing location encoding on each of the multiple regions. This can facilitate the labeling of each region's feature information, thereby better distinguishing each region's feature information and facilitating the model training process.

[0071] According to one embodiment of this application, multiple basic tasks include a disordered rearrangement task. The pre-training of the model for multiple basic tasks based on fused feature information includes: performing disordered matching of multiple regional feature information and multiple first position encoding information in the fused feature information to obtain disordered feature information, wherein the disordered feature information includes multiple regional feature information and multiple second position encoding information matched with the multiple regional feature information; and performing pre-training of the model for the disordered rearrangement task based on the disordered feature information.

[0072] Specifically, the matching relationship between some or all of the regional feature information and the first positional encoding information can be changed. For example, multiple regions can be arranged in a certain order. By randomly shuffling the order of the multiple regions, multiple regions in a random order can be obtained. Based on the positions of the multiple regions in the random order, second positional encoding information matching the feature information of each region can be obtained. Alternatively, the first positional encoding information corresponding to each region's feature information can be directly modified to obtain the second positional encoding information. The value ranges of multiple second positional encoding information and multiple first positional encoding information can be the same; only the matching relationships between the two and the multiple region feature information are different.

[0073] The input data may include at least one image frame. Random matching of multiple region feature information and multiple first positional encoding information can adjust the positional relationships between different regions within the same image frame. This random reordering task can improve the model's performance in target tasks such as layout analysis. Alternatively, the random matching can adjust the positional relationships between multiple images without adjusting the positional relationships between different regions within the same image frame. This random reordering task can also improve the model's performance in target tasks such as video action recognition and layout analysis. Or, the random matching can adjust the positional relationships between all regions in multiple images; that is, it can adjust both the position of different regions within the multiple images and the specific distribution of different regions within their respective images. This random reordering task can also improve the model's performance in target tasks such as video action recognition and layout analysis. For example, the original position of a certain region is (X... i Y j ), where X iThis indicates the location of the image to which this region belongs within multiple frames, Y. j This indicates the specific location of the region within its corresponding image. After random matching, the location of this region can be (X... i Y j+k ), (X i+k Y j ) or (X i+k Y j+l Here, k and l can be integers, specifically positive, negative or 0.

[0074] This application's embodiments train the model using a randomized rearrangement task as the basic task. This forces the model to restore randomly shuffled regions, similar to a "puzzle restoration" operation. This allows the model to pay more attention to overlapping edges between regions, thereby better completing information exchange between regions and constructing richer global information. Therefore, training with a randomized rearrangement task can improve the model's inference performance for various target tasks.

[0075] Furthermore, the multiple basic tasks also include object detection and / or text recognition tasks. The pre-training of the model for multiple basic tasks based on fused feature information also includes: pre-training the model for object detection and / or text recognition tasks based on fused feature information.

[0076] Specifically, object detection tasks can detect object categories between regions, forcing models to possess object detection capabilities. Text recognition tasks can detect text within a region, such as letters, numbers, English characters, etc.

[0077] Different preprocessing techniques can be applied to the input data for different basic tasks. For example, for the scrambling task, multiple region features and multiple first position encodings in the fused feature information can be matched in a random order to obtain scrambling feature information, and the model can be pre-trained for the scrambling task based on this scrambling feature information. Here, the process of obtaining fused feature information from the input data and the scrambling matching process can be considered as preprocessing. For object detection and / or text recognition tasks, the model can be directly pre-trained for these tasks based on the fused feature information. Here, the process of obtaining fused feature information from the input data can also be considered as preprocessing.

[0078] When the target task is scene description, by combining the scrambling and object detection in the basic task, the model can predict the scene category based on global information (e.g., by comprehensively analyzing object information such as athletes, spectators, and basketballs, the model can predict the scene as "stadium"); by combining the scrambling and text recognition in the basic task, the model can correctly concatenate text lines to obtain meaningful text information; by combining the text recognition and object detection in the basic task, the model can predict occluded text and occluded objects. In addition, the feature information corresponding to the text recognition and object detection tasks can participate in the entire training process, avoiding the model parameters from getting stuck in local optima.

[0079] In this application, the input data obtained for different basic tasks can be the same, but different preprocessing can be performed on the same input data for different basic tasks to adapt to the training process of different basic tasks, thereby effectively improving the performance of the model.

[0080] According to one embodiment of this application, training a model on at least one target task to update the model's parameters includes: training the model on at least one target task based on a supervised learning method to update the model's parameters.

[0081] Specifically, supervised learning methods can be used to train the model for the target task. For example, the model can be pre-trained first, and after the pre-training process, an intermediate trained model can be obtained. Then, the intermediate model can be trained on the target task to obtain the final model.

[0082] Supervised learning trains a model using labeled sample data and updates its parameters based on the differences between the labeled data and the inference results obtained during training. For example, supervised learning can compare the differences between the labeled data and the inference results word by word, using this difference as a penalty to force the model to update its parameters in the direction of human annotation.

[0083] The supervised learning process for the target task can be carried out after the pre-training phase to avoid the problem of the model being difficult to control the optimization direction due to simultaneous training of multiple tasks (basic task and target task).

[0084] In this embodiment of the application, before training the model for the target task, a pre-training process for the basic task is first performed on the model to obtain a trained intermediate model, and then the intermediate model is trained for the target task, thereby ensuring that the model can obtain reliable performance on the target task.

[0085] Furthermore, since there is a certain gap between the basic task and the target task, pre-training the model on the basic task before training it on the target task will lead to the final model's performance being affected by the pre-training results. In other words, the intermediate model obtained through pre-training determines the upper limit of the final model's performance on the target task. For example, training on the basic task aims to achieve good generalization performance across many target tasks, while training on the target task aims to achieve optimal performance on a specific task. Therefore, the training on the basic task and the training on the target task may have inconsistent optimization directions, resulting in performance bottlenecks. To further improve model performance, pre-training can be performed on basic tasks that closely match the target task and on basic tasks with higher difficulty levels.

[0086] According to one embodiment of this application, training a model on at least one target task to update the model's parameters includes: during pre-training, training the model on at least one target task based on a reinforcement learning method to update the model's parameters.

[0087] Specifically, during the pre-training of the model for the basic task, the model can be simultaneously trained for the target task. For example, reinforcement learning methods can be used to further train the model for the target task during the pre-training process. The pre-training process for the basic task can directly contribute to the construction or adjustment of model parameters, while the training process for the target task can indirectly guide the model to update parameters through reward signals.

[0088] The pre-training process for the basic task can begin before the training for the target task, but it doesn't need to wait until the model pre-training process is completely finished before executing the target task training process. For example, the target task training process can begin after the model pre-training process meets certain preset conditions. Here, the preset conditions can be that the pre-training process has progressed to a certain extent, such as 20%, 30%, 40%, 50%, etc. In one embodiment, the input data corresponding to the basic task can be input into the model immediately after the input data corresponding to the basic task is input into the model. That is, the training of the basic task and the target task have a sequential order, but the two training processes are not separated and can be regarded as a single training process, without waiting for the model to be fully trained on the basic task before training the target task.

[0089] Reinforcement learning improves model performance through interaction with the environment. It doesn't require the inference results obtained during training to be completely identical to the labeled results; instead, it uses pre-defined evaluation metrics to assess the inference results. If the inference result meets certain expectations, a reward signal is given, and the model's parameters are updated based on this reward signal. For example, reinforcement learning can calculate the overall difference between the inference result and the labeled result, using this difference as a reward to encourage the model to update its parameters in the direction of maximizing the reward. Ultimately, reinforcement learning aims to maximize the reward, not to achieve an exact match between the inference result and the manually labeled result.

[0090] Since there is a certain gap between the basic task and the target task in terms of the information (such as sample data) they process, pre-training the model on the basic task before training it on the target task means that the model needs to learn the labeled results corresponding to multiple different sample data. Although intuitively, the model learns multiple different scene information and should obtain better inference performance, in reality, it is difficult for the model to balance the information conflicts between different tasks, thus making it difficult to obtain a highly reliable model.

[0091] This embodiment employs reinforcement learning to train the model for the target task. This provides guidance during the pre-training process for the basic task, directing the model to update parameters in the direction of the target task and reducing the impact of the gap between the basic and target tasks on model performance. Furthermore, by using reinforcement learning to train the model for the target task, the two previously separate training processes (pre-training for the basic task and training for the target task) can be simplified into a single training process. Unlike the basic task, the target task does not directly participate in model parameter adjustment; instead, it guides the model to update parameters through reward signals. This reinforcement learning method can guide the model to achieve better performance.

[0092] This application's embodiments employ reinforcement learning to train the model for the target task, allowing the training processes for the basic task and the target task to proceed simultaneously. This shortens the model's training cycle and improves training efficiency. Furthermore, by using reinforcement learning to train the model for the target task, the design difficulty of the basic task can be reduced to some extent, improving model performance and ultimately resulting in a robust model that meets usage requirements.

[0093] In other embodiments, the model can also be trained based on other learning methods, such as unsupervised learning methods, and the specific learning method can be determined by combining the type of the basic task and the target task.

[0094] According to one embodiment of this application, training a model on at least one target task based on a reinforcement learning method to update the model's parameters includes: using the model to infer a target inference result based on sample data corresponding to each target task in at least one target task; evaluating the target inference result based on a preset evaluation index to obtain a reward signal; and updating the model's parameters based on the reward signal.

[0095] Specifically, during pre-training, sample data corresponding to the target task can be input into the model. The model performs inference based on this sample data to obtain the target inference result. The target inference result is then evaluated based on a preset evaluation metric to obtain a reward signal.

[0096] Preset evaluation metrics can be designed based on the type of target task. When the target inference result meets the preset evaluation metrics, the model can receive a corresponding reward signal.

[0097] For example, when the target task is scene description, the preset evaluation metric could be a bilingual evaluation understudy (BLEU) metric such as language fluency. Evaluating the reasoning results based on the BLEU metric yields a reward signal, and different reward signals are determined based on the magnitude of the BLEU value. When the target task is visual question answering or product recommendation, the preset evaluation metric could be accuracy, and different reward signals are determined based on the accuracy level.

[0098] Taking accuracy as the preset evaluation metric and product recommendation and scene description as target tasks as examples, the specific process of obtaining reward signals is described. Sample data from the product recommendation and scene description tasks are input into the model. The labeled result for the product recommendation task is product A, and the labeled result for the scene description task is "There is a red water glass on the table, and a monitor next to it." For the product recommendation task, the model's target inference result based on the sample data is product B. This target inference result is inconsistent with the labeled result, i.e., the accuracy is 0, thus obtaining a low reward score (reward signal). For the scene description task, the model's target inference result based on the sample data is "The monitor is on the table, and there is a red water glass next to it." This target inference result is consistent with the labeled result in overall meaning, i.e., the accuracy is 100%, thus obtaining a high reward score (reward signal). In this example, for the scene description task, if a supervised learning method is used, the target inference result will be inconsistent with the labeled result because supervised learning compares the differences between the target inference result and the labeled result word by word, while reinforcement learning compares the differences between the target inference result and the labeled result in overall meaning. Therefore, reinforcement learning can effectively improve the reliability of the model in reasoning for the target task.

[0099] In one implementation, the loss function value can be determined using the following formula, and then the model parameters can be updated based on the loss function value. In the formula below, reward is the reward score, and P(output) is the model's output, such as the probability value corresponding to the target inference result.

[0100]

[0101] This application embodiment trains the model on the target task using reinforcement learning during pre-training, evaluates the target inference results based on a preset evaluation metric to obtain a reward signal, and then updates the model's parameters based on the reward signal. This effectively improves the model's performance on the target task.

[0102] Figure 3 The diagram shown is a flowchart illustrating a training method for a model provided in another exemplary embodiment of this application. Figure 3 The example is Figure 2 To avoid repetition, the similarities in the examples provided will not be repeated. Figure 3 The method can be achieved through Figure 4 Model 400 is used for execution. For example... Figure 3 As shown, the training method for this model includes the following.

[0103] 310: Divide each frame of at least one image into multiple regions corresponding to each frame, and obtain the region feature information corresponding to each region in the multiple regions.

[0104] like Figure 4 As shown, input data can be fed into model 400. Model 400 may include a feature extraction module 410 and multiple classifiers corresponding to multiple basic tasks, such as a randomized reordering classifier 420, an object detection classifier 430, and a text recognition classifier 440. The feature extraction module 410 may employ a Transformer architecture or other network structures. Since the feature extraction module 410 can obtain unified modality feature information from the multimodal information contained in the input data, it can also be called a unified vision-text transformer. The input data may include at least one image frame.

[0105] The feature extraction module 410 can partition the image and obtain the regional feature information corresponding to each region. The regional feature information is fused with the multimodal information contained in the corresponding region. Optionally, in other embodiments, the image can be partitioned by a computing device to obtain multiple small region images, and the multiple small region images can be input into the model 400 so that the model 400 can obtain the regional feature information corresponding to each small region image.

[0106] 320: Perform location encoding on each of the multiple regions to obtain first location encoding information that matches the region's regional feature information.

[0107] For example, the input data may include N images (N greater than or equal to 1), each image having a size of H*W, where H and W represent the length and width of the image, respectively. Dividing the N images into partitions yields P regions. The feature information for each region is then obtained; this feature information can be represented by a feature vector f. i This indicates that it can contain text information and object information in the corresponding region, where 1≤i≤P. All region feature information can be represented by matrix F. Further, the feature extraction module 410 can perform position encoding on each region to obtain first position encoding information that matches the region feature information of each region.

[0108] 330: Randomly match multiple regional feature information with multiple first position coding information to obtain random feature information, which includes multiple regional feature information and multiple second position coding information that match the multiple regional feature information.

[0109] The process of obtaining disordered feature information can be found in the description of the above embodiments, and will not be repeated here to avoid repetition. The acquisition of disordered feature information can be performed by the feature extraction module 410.

[0110] 340: Pre-training the model for the task of disordered rearrangement based on disordered feature information, and pre-training the model for the tasks of object detection and text recognition based on multiple region feature information and multiple first position encoding information.

[0111] Multiple basic tasks may include scrambling, object detection, and text recognition. The feature extraction module 410 can further process the extracted feature information. The feature information output by the feature extraction module 410 can be sent to the scrambling classifier 420, the object detection classifier 430, and the text recognition classifier 440, respectively. Based on the feature information, the scrambling classifier 420, the object detection classifier 430, and the text recognition classifier 440 can obtain their respective classification results.

[0112] Adjust the model parameters based on the difference between the classification result and the corresponding annotation result.

[0113] Specifically, the feature extraction module 410 (Transformer architecture) obtains feature information as a matrix F based on the input data. In one embodiment, the model parameters can be adjusted based on the following formula, where, It can be a weight calculation function (such as the Attention value), W Q W Kand W V Here, d represents the model parameters, and f can be a feature vector. i Dimensions.

[0114]

[0115] By using different learning parameters, the feature information contained in matrix F can be mapped to different subspaces for interaction, thereby obtaining richer global information.

[0116] 350: During pre-training, the model is trained on at least one target task based on reinforcement learning methods to update the model's parameters.

[0117] At least one target task includes at least one of the following: visual question answering, layout analysis, product recommendation, scene description, and image and text retrieval.

[0118] During the pre-training process of the basic task, the model is simultaneously trained for the target task. For example... Figure 4 As shown, sample data corresponding to the target task can be input into model 400. Feature extraction module 410 can extract fused feature information based on the sample data. The fused feature information integrates the multimodal information contained in the sample data. The fused feature information extracted by feature extraction module 410 based on sample data can be similar to the feature information extracted by feature extraction module 410 based on the input data of the basic task. For example, the fused feature information may include multiple region feature information, first position encoding information, and second position encoding information.

[0119] Model 400 may further include a target task inference module 450, with different target tasks corresponding to different target task inference modules. Feature extraction module 410 can further process the extracted fused feature information, and the feature information output by feature extraction module 410 can be transmitted to target task inference module 450. Target task inference module 450 can obtain the corresponding target inference result based on the feature information.

[0120] Furthermore, a reward signal can be obtained by evaluating the target inference result based on a preset evaluation index, and the model parameters can be updated based on the reward signal. The process of obtaining the reward signal can be found in the description of the above embodiments.

[0121] Exemplary device

[0122] Figure 5 The diagram shown is a structural schematic of a model training device 500 provided in an exemplary embodiment of this application. Figure 5 As shown, the training device 500 includes an acquisition module 510 and a training module 520.

[0123] The acquisition module 510 is used to acquire fused feature information based on the input data, wherein the fused feature information integrates multimodal information contained in the input data. The training module 520 is used to pre-train the model on multiple basic tasks based on the fused feature information, and to train the model on at least one target task to update the model's parameters.

[0124] This application provides a model training device that obtains fusion feature information that integrates multimodal information contained in the input data based on the input data, and trains the model on basic tasks and target tasks based on the fusion feature information. This enables the model to extract feature information of a unified modality, improves the model's ability to process multimodal information, and thus improves the model's performance on the target task.

[0125] According to one embodiment of this application, the training module 520 is used to: obtain the classification result corresponding to each of the multiple basic tasks based on the fused feature information; and pre-train the model based on the multiple classification results corresponding to the multiple basic tasks.

[0126] According to one embodiment of this application, the multiple basic tasks include at least two of the following: a shuffle task, an object detection task, a text recognition task, and a speech recognition task.

[0127] According to one embodiment of this application, the input data includes at least one frame of image, wherein the acquisition module 510 is used to: partition each frame of image in the at least one frame of image to obtain multiple regions corresponding to the image; acquire region feature information corresponding to each region in the multiple regions, wherein the region feature information is fused with multimodal information contained in the region, and the fused feature information includes multiple region feature information corresponding to multiple regions.

[0128] According to one embodiment of this application, the acquisition module 510 is further configured to: perform position encoding on each region in the multiple regions to obtain first position encoding information that matches the region feature information of the region, and the fused feature information further includes multiple first position encoding information that matches the feature information of the multiple regions.

[0129] According to an embodiment of this application, multiple basic tasks include a disordered rearrangement task, wherein the training module 520 is used to: perform disordered matching of multiple regional feature information and multiple first position coding information in the fused feature information to obtain disordered feature information, the disordered feature information including multiple regional feature information and multiple second position coding information matching the multiple regional feature information; and pre-train the model on the disordered rearrangement task based on the disordered feature information.

[0130] According to one embodiment of this application, the multiple basic tasks also include object detection tasks and / or text recognition tasks, wherein the training module 520 is further used to: pre-train the model on object detection tasks and / or text recognition tasks based on fused feature information.

[0131] According to one embodiment of this application, the training module 520 is used to: train the model on at least one target task based on a reinforcement learning method during the pre-training process to update the model's parameters.

[0132] According to one embodiment of this application, the training module 520 is used to: use the model to infer based on sample data corresponding to each target task in at least one target task to obtain a target inference result; evaluate the target inference result based on a preset evaluation index to obtain a reward signal; and update the parameters of the model based on the reward signal.

[0133] According to one embodiment of this application, the training module 520 is used to: train the model on at least one target task based on a supervised learning method to update the model's parameters.

[0134] According to one embodiment of this application, at least one target task includes at least one of the following: visual question answering, layout analysis, product recommendation, scene description, and image and text retrieval.

[0135] It should be understood that the operation and function of the acquisition module 510 and the training module 520 in the above embodiments can be referred to the above. Figure 2 or Figure 3 The description of the training method of the model provided in the embodiments will not be repeated here to avoid repetition.

[0136] Figure 6 The diagram shown is a block diagram of an electronic device 600 for performing a model training method according to an exemplary embodiment of this application.

[0137] Reference Figure 6 The electronic device 600 includes a processing component 610, which further includes one or more processors, and memory resources represented by memory 620 for storing instructions executable by the processing component 610, such as application programs. The application programs stored in memory 620 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 610 is configured to execute instructions to perform the training method of the aforementioned model.

[0138] Electronic device 600 may also include a power supply component configured to perform power management of electronic device 600, a wired or wireless network interface configured to connect electronic device 600 to a network, and an input / output (I / O) interface. Electronic device 600 can be operated based on an operating system stored in memory 620, such as Windows Server.TM Mac OSX TM Unix TM Linux TM FreeBSD TM Or similar.

[0139] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the electronic device 600, enables the electronic device 600 to execute a model training method, comprising: acquiring fused feature information based on input data, wherein the fused feature information fuses multimodal information contained in the input data; pre-training the model on multiple basic tasks based on the fused feature information, and training the model on at least one target task to update the model's parameters.

[0140] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0141] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0142] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0146] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program verification codes, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0147] It should be noted that in the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0148] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications or equivalent substitutions made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a model, characterized in that, The method comprises: obtaining fusion feature information based on input data, wherein the fusion feature information fuses multi-modal information contained in the input data, and the fusion feature information is directly extracted from the input data; pre-training a model on a plurality of basic tasks based on the fusion feature information, and when the model meets a preset condition, training the model on at least one target task to update parameters of the model, wherein the target task is a complex reasoning task, the complexity of each task in the plurality of basic tasks is lower than that of the target task, the pre-training of the plurality of basic tasks is performed based on the same fusion feature information, and the preset condition includes that the pre-training process of the model reaches a preset degree. 2.The method of Claim 1, wherein, The pre-training of the model on the plurality of basic tasks based on the fusion feature information comprises: obtaining classification results corresponding to each basic task in the plurality of basic tasks based on the fusion feature information; pre-training the model based on the plurality of classification results corresponding to the plurality of basic tasks. 3.The method of Claim 1, wherein, The plurality of basic tasks comprises at least two of a disorder rearrangement task, an object detection task, a text recognition task, and a speech recognition task. 4.The method of Claim 1, wherein The input data comprises at least one image, and the obtaining of the fusion feature information based on the input data comprises: partitioning each image in the at least one image to obtain a plurality of regions corresponding to the image; obtaining region feature information corresponding to each region in the plurality of regions, wherein the region feature information fuses multi-modal information contained in the region, and the fusion feature information comprises a plurality of region feature information corresponding to the plurality of regions.

5. The method of training a model of claim 4, wherein, The obtaining of the fusion feature information based on the input data further comprises: position encoding each region in the plurality of regions to obtain first position encoding information matched with the region feature information of the region, and the fusion feature information further comprises a plurality of first position encoding information matched with the plurality of region feature information.

6. The method of training a model of claim 5, wherein, The plurality of basic tasks comprises a disorder rearrangement task, and the pre-training of the model on the plurality of basic tasks based on the fusion feature information comprises: disorder matching the plurality of region feature information in the fusion feature information with the plurality of first position encoding information to obtain disorder feature information, wherein the disorder feature information comprises the plurality of region feature information and a plurality of second position encoding information matched with the plurality of region feature information; pre-training the model on the disorder rearrangement task based on the disorder feature information.

7. The method of training a model of claim 6, wherein, The plurality of basic tasks further comprises an object detection task and / or a text recognition task, and the pre-training of the model on the plurality of basic tasks based on the fusion feature information further comprises: pre-training the model on the object detection task and / or the text recognition task based on the fusion feature information. 8.The method of Claim 1, wherein The training of the model on the at least one target task to update the parameters of the model comprises: in the pre-training process, training the model on the at least one target task based on a reinforcement learning method to update the parameters of the model. 9.The method of Claim 8, wherein, The reinforcement learning-based method trains the model on the at least one target task to update parameters of the model, including: inference based on the model to obtain a target inference result based on sample data corresponding to each of the at least one target task; evaluating the target inference result based on a preset evaluation index to obtain a reward signal; updating the parameters of the model based on the reward signal. 10.The method of Claim 1, wherein The training of the model on the at least one target task to update the parameters of the model includes: training the model on the at least one target task based on a supervised learning method to update the parameters of the model. 11.The method of training a model according to any one of claims 1 to 10, characterized in that, The at least one target task includes at least one of the following: visual question answering, layout analysis, product recommendation, scene description, and image-text retrieval.

12. A device for training a model, the device comprising: It includes: an acquisition module configured to acquire fusion feature information based on input data, wherein the fusion feature information fuses multi-modal information contained in the input data, and the fusion feature information is directly extracted from the input data; a training module configured to pre-train a model on a plurality of basic tasks based on the fusion feature information, and when the model meets a preset condition, train the model on at least one target task to update parameters of the model, wherein the target task is a complex inference task, the complexity of each of the plurality of basic tasks is lower than that of the target task, the pre-training of the plurality of basic tasks is performed in parallel based on the same fusion feature information, and the preset condition includes that the pre-training process of the model reaches a preset degree.

13. An electronic device, comprising: It includes: a processor; a memory for storing instructions executable by the processor, wherein the processor is configured to execute the training method of the model according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the training method of the model according to any one of claims 1 to 11.

Images