Method, device and equipment for model training and readable medium

Abstract

The embodiment of the invention provides a method, device and equipment for model training and a readable medium. The method comprises the steps that a universal sample set is used for training a feature extractor, the universal sample set comprises a plurality of universal sample image-universal text label pairs, and universal text labels indicate semantic information corresponding to universal sample images and at least one item in categories corresponding to the universal sample images; adding a target classifier for the target scene to the trained feature extractor to obtain a target model; and at least training a target classifier by using a target sample set related to the target scene to determine a trained target model, the target sample set comprising a plurality of target data sample-target label pairs, the target data samples being extracted from the data samples of the target scene, and the target label pairs being extracted from the target data samples of the target scene. And the target label indicates the category to which the corresponding target data sample is divided in the target scene. In this way, the robustness and generalization of the target model can be improved.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computer technology, and more specifically, to methods, apparatus, devices, and computer-readable storage media for model training. Background Technology

[0002] With the development of machine learning technology, it is now possible to utilize machine learning models, especially large models, to perform tasks in various application environments. When performing classification tasks in specific scenarios, a trained large model can determine and output the classification result of a received object. The training performance of the large model affects its classification ability in practical applications. Therefore, to improve the classification ability of large models when performing classification tasks in specific scenarios, it is desirable to train the large model specifically for that particular scenario to enhance its training performance. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for model training is provided. The method includes: training a feature extractor using a general sample set, the general sample set including multiple general sample image-general text label pairs, the general text labels indicating at least one of semantic information of the corresponding general sample image and the category corresponding to the corresponding general sample image; adding a target classifier for a target scene to the trained feature extractor to obtain a target model; and determining the trained target model by at least training the target classifier using a target sample set related to the target scene, the target sample set including multiple target data sample-target label pairs, the target data samples being extracted from data samples of the target scene, and the target labels indicating the category to which the corresponding target data sample is classified within the target scene.

[0004] In a second aspect of this disclosure, an apparatus for model training is provided. The apparatus includes: an extractor training module configured to train a feature extractor using a general sample set, the general sample set including multiple general sample image-general text label pairs, the general text labels indicating at least one of semantic information of the corresponding general sample image and a category corresponding to the corresponding general sample image; a model acquisition module configured to add a target classifier for a target scene to the trained feature extractor to obtain a target model; and a classifier training module configured to determine the trained target model by training the target classifier using at least a target sample set related to the target scene, the target sample set including multiple target data sample-target label pairs, the target data samples being extracted from data samples of the target scene, and the target labels indicating the category to which the corresponding target data sample is classified within the target scene.

[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method of the first aspect of this disclosure when executed by the at least one processing unit.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program that can be executed by a processor to perform the method according to a first aspect of this disclosure.

[0007] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of various implementations of this disclosure will become more apparent in the following detailed description, taken in conjunction with the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0010] Figure 2 A flowchart of a process for model training according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram of a process for model training according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A schematic structural block diagram of an apparatus for model training according to some embodiments of the present disclosure is shown; and

[0013] Figure 5 A block diagram of a computing device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0016] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," "machine learning network," "neural network," or simply "network," and these terms are used interchangeably.

[0017] A neural network is a machine learning network based on deep learning. A neural network processes input and provides a corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications often include many hidden layers, thus increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output serves as the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above.

[0018] Machine learning typically comprises three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating its parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0019] It should be noted that the acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0021] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0022] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0023] In recent years, machine learning models, especially large-scale models, have developed rapidly and have been widely used in various intelligent processing scenarios. Compared with traditional models, large-scale models can handle more complex and high-dimensional features, have stronger fitting capabilities, and can model and learn deep-seated patterns. However, large-scale models have a large number of parameters, high computational resource requirements, and need to rely on technologies such as cloud computing for support. In terms of model performance, large-scale models have strong learning capabilities, can identify more complex and novel patterns, and achieve higher accuracy. Furthermore, large-scale models can quickly process massive amounts of data in parallel, resulting in faster retrieval and computation speeds. Overall, large-scale models have more powerful feature learning and modeling capabilities, can deeply mine the value of data, and demonstrate enormous application potential in various intelligent processing scenarios.

[0024] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Figure 1 In environment 100, it is desirable to train and use a large model (i.e., model 130), which is configured for various application environments. For example, if model 120 is a classification model, it is possible to determine the category corresponding to a user input based on that user input, and so on.

[0025] like Figure 1 As shown, environment 100 includes model training system 150. Figure 1The upper part illustrates the model training phase, and the lower part illustrates the model application phase. Before training, the parameter values ​​of model 130 can have initial values ​​or pre-trained parameter values ​​obtained through a pre-training process. Model 130 can be trained via forward and backward propagation, during which the parameter values ​​of model 130 can be updated and adjusted. After training is complete, model 130' is obtained. At this point, the parameter values ​​of model 130' have been updated, and based on the updated parameter values, model 130 can be used to implement recommendation tasks in the model application phase.

[0026] During the model training phase, model 130 can be trained using a training sample set 110 comprising multiple training samples 112 and a model training system 150. Each training sample 112 can involve a binary format. For example, for a classification task, training sample 112 can include sample 120 and label 122 from the classification task. Model training system 150 can train model 130 using the training sample set 110 comprising sample 120 and label 122. Specifically, the training process can be performed iteratively using a large number of training samples. After training is complete, model 130 can include knowledge about the task to be processed. During the model application phase, model application system 160 can use model 130' (which now has the trained parameter values) to perform the corresponding task. For example, it can receive model input 142 from a classification task and output the corresponding model output 144.

[0027] exist Figure 1 In this system, the model training system 150 and the model application system 160 are configured on an electronic device. It is understood that the model training system 150 and the model application system 160 can be configured on the same electronic device or on different electronic devices. For example, the model training system 150 is configured on a first electronic device, which can provide the trained model to a second electronic device, whereby the model application system 160 can then use the trained model to perform corresponding tasks.

[0028] Electronic devices can include any computing device with computing capabilities, such as terminal devices or server devices. Terminal devices can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof.

[0029] Server-side equipment can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server-side equipment may include, for example, computing systems / servers, such as mainframes, edge computing nodes, computing devices in cloud environments, and so on.

[0030] It should be understood that Figure 1 The components and arrangements shown in environment 100 are merely examples, and a computing system suitable for implementing the exemplary implementations described in this disclosure may include one or more different components, other components, and / or different arrangements. Implementations of this disclosure are not limited in this respect.

[0031] As discussed above, when performing classification tasks in specific scenarios, a trained large model (or simply model) can determine and output the classification result of the received object. The training effect of the large model affects its classification ability in practical applications. When training a large model, the diversity of the training data, label quality, iteration cycle, and number of parameters all influence its performance. Traditionally, because large models have very strict requirements for training data, they often require training sample sets containing a large amount of labeled data specific to the scenario to ensure accurate execution of the corresponding task. However, the large amount of labeled data for a scenario requires extremely high manual annotation, significantly increasing the labor cost of model training. Moreover, in some specific scenarios, the scenario changes rapidly, requiring frequent model updates and iterations to maintain the model's task processing capability in that scenario. Traditional training methods result in long training iteration cycles. Furthermore, traditionally, models are often trained based on single-modality training data, which can easily lead to label misdirection, overfitting, and other issues, affecting the model's robustness.

[0032] In view of this, embodiments of the present disclosure provide an improved scheme for model training. The method includes: training a feature extractor using a general sample set, the general sample set including multiple general sample image-general text label pairs, where the general text label indicates at least one of semantic information of the corresponding general sample image and the category to which the corresponding general sample image belongs. A target classifier for a target scene is added to the trained feature extractor to obtain a target model. The trained target model is determined by training the target classifier using at least a target sample set related to the target scene, the target sample set including multiple target data sample-target label pairs, where the target data samples are extracted from data samples in the target scene, and the target labels indicate the category to which the corresponding target data sample is classified within the target scene.

[0033] In this way, a robust and improved feature extractor can be trained using a general sample set containing a large amount of labeled data, including multimodal data. Furthermore, an adaptation layer and classifier can be trained using a target sample set from the target scene, including a small amount of labeled data. This allows the final target model, including the feature extractor, adaptation layer, and classifier, to be applied to the target scene. This reduces the human resource cost of labeled data while enabling convenient and rapid training of the target model. Moreover, the model training scheme disclosed herein improves the robustness and generalization of the target model, contributing to enhanced task processing capabilities in the target scene.

[0034] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0035] Figure 2 A flowchart of a process 200 for model training according to some embodiments of the present disclosure is shown. Process 200 can be implemented at an electronic device configured with a model training system 150. For ease of discussion, reference will be made to... Figure 1 The environment 100 is used to describe the process 200.

[0036] In box 210, the electronic device uses a general sample set to train a feature extractor. This general sample set includes multiple general sample image-general text label pairs, where the general text label indicates at least one of the semantic information of the corresponding general sample image and the category corresponding to that general sample image. This general sample set (exemplarily referred to as the base class Session D0) can be, for example, a sample set from a general scene. The feature extractor can, for example, include a neural network based on a window attention mechanism. This window attention-based neural network can be a self-attention-based visual Transformer network, such as the Swin Transformer. Its key feature is that it retains the main advantages of the Transformer while introducing a window attention mechanism to replace global self-attention.

[0037] A feature extractor is configured to extract effective feature representations from raw input data for subsequent model use. A robust feature extractor needs to extract stable and consistent features across a variety of varying input data, which is crucial for the model's understanding of the real world. Because feature extraction is a critical step connecting input and prediction, only a sufficiently robust feature extractor can provide truly effective and reliable features for subsequent tasks. A robust feature extractor allows the model to handle a variety of complex scenarios, not just those seen in the training data, which is essential for the model to handle unknown inputs and tasks. Improving the robustness of the feature extractor is a vital step in achieving model stability and generalization. Therefore, in embodiments of this disclosure, to improve the robustness of the feature extractor, the electronic device will use a general sample set including a large amount of multimodal data to train the feature extractor.

[0038] Regarding the specific method of training the feature extractor, in some embodiments, the electronic device can determine multiple image feature representations in visual space for multiple general sample images in a multiple general sample image-general text label pair. For example, taking a shift window converter as the feature extractor, the electronic device can divide the multiple general sample images into a certain number (e.g., M) of non-overlapping image patches, and the set of these image patches can be represented as... Where p represents an image patch, i represents the i-th patch, and p iLet represent the i-th image patch. The electronic device inputs these image patches into a feature extractor. The feature extractor can extract multiple image embedding patches corresponding to these multiple non-overlapping image patches based on the following formula.

[0039]

[0040] These multiple images are embedded in image blocks. It can also be viewed as multiple image feature representations corresponding to multiple general sample images.

[0041] In some embodiments, the electronic device may further process these multiple image feature representations to transform them into a visual space. Multiple image feature representations in the visual space (also known as global visual feature representations), where v is vision. Electronic devices can perform any appropriate transformation, such as linear transformation, fully connected transformation, feedforward neural network (FFN) transformation, multilayer perceptron (MLP) transformation, etc., on the multiple image feature representations extracted by the feature extractor to transform these multiple image feature representations into multiple image feature representations in the visual space.

[0042] Electronic devices can, for example, transform these multiple image feature representations into a visual space based on the following formula. Multiple image feature representations in:

[0043]

[0044] in This instruction performs an average pooling operation on multiple image feature representations.

[0045] An electronic device can determine a first loss value corresponding to multiple general sample images based on at least multiple general sample image-general text label pairs, each corresponding to a category and multiple image feature representations. Each general text label in these multiple general sample image-general text label pairs can be, for example, "image of category XX". Such general text labels can be directly obtained by the electronic device. In some embodiments, the electronic device can only obtain the category label (e.g., category XX) of each general sample image. The electronic device can obtain the general text label by embedding the category label into a sentence template such as "image of {}". The electronic device can determine the category corresponding to the corresponding general sample image-general text label pair based on the multiple general text labels in the multiple general sample image-general text label pairs. The first loss value here can be, for example, cross-entropy loss. The electronic device can determine the first loss value L based, for example, on the following formula. VCE :

[0046]

[0047] Where T0 can be the number of multiple general sample image-general text label pairs in the general sample set, y i This represents the category corresponding to the i-th general sample image-general text label pair in the general sample set. and [b1, ..., b] N ] represents the weights and biases in the fully connected classification layer. An electronic device can train the feature extractor by updating its parameters to reduce the first loss value, and at least by reducing the first loss value.

[0048] In some embodiments, the electronic device can also determine multiple generic text labels in multiple generic sample image-generic text label pairs. In semantic space Multiple text feature representations in the semantic space, where 's' represents semantics. Electronic devices can further map these multiple text feature representations from the semantic space to the visual space (i.e., transformation). To obtain multiple text feature representations in visual space This process can also be viewed as performing a dimensionality transformation on multiple text feature representations in the semantic space to obtain multiple text feature representations in the visual space. The electronic device can determine a second loss value corresponding to multiple common text labels based on at least multiple common sample images and common text labels, each corresponding to its category and multiple text feature representations in the visual space. This second loss value can, for example, be the cross-entropy loss. The electronic device can, for example, determine the second loss value L based on the following formula: SCE :

[0049]

[0050] Similarly, T0 here can be the number of multiple universal sample image-universal text label pairs in the universal sample set, y i This represents the category corresponding to the i-th general sample image-general text label pair in the general sample set. and [b1, ..., b] N ] represents the weights and biases in the fully connected classification layer. The electronic device can train the feature extractor by updating its parameters to reduce the second loss value, and at least by reducing the second loss value.

[0051] In some embodiments, in order to combine image feature representation and text feature representation, the electronic device may determine the total loss value L of the first loss value and the second loss value based on the following formula. VSCE :

[0052] L VSCE =L VCE +λLSCE (5)

[0053] Where λ represents L VCE and L SCE The balance coefficient (i.e., the balance coefficient between the first loss value and the second loss value). Electronic devices, for example, can directly reduce this total loss value by updating the parameters of the feature extractor, and train the feature extractor by reducing this total loss value.

[0054] refer to Figure 3 , Figure 3 A schematic diagram of a process 300 for model training according to some embodiments of the present disclosure is shown. Figure 3 As shown, process 300 may include three training phases: a first training phase 302, a second training phase 304, and a third training phase 306. In the first training phase 302, the electronic device may use a general sample set 312 to train the feature extractor 330. The general sample set 312 may include general text labels 314 and general sample images 316. The general text labels 314 and general sample images 316 may exist in the form of general sample image-general text label pairs. For example, the general sample image 316 may be an image containing oranges, and the corresponding general text label 314 may be the text "picture of oranges," where the text "oranges" can indicate the classification of the image. The general text labels 314 are provided to the text encoder 320. The text encoder 320 may output the text feature representation 332 of the general text labels 314 in the semantic space. The general sample images 316 are provided to the feature extractor 330. The feature extractor 330 may extract the image feature representation 334 of the general sample images 316. The text feature representation 332 and the image feature representation 334 in the semantic space are respectively provided to the corresponding feature processing modules (i.e., feature processing module 340-1 and feature processing module 340-2) to obtain the text feature representation 332 in the visual space and the image feature representation 334 in the visual space.

[0055] The electronic device can determine a first loss value 352 and a second loss value 354 corresponding to the text feature representation 332 in visual space and the image feature representation 334 in visual space, respectively. The electronic device can, for example, train a feature extractor 330 by reducing the first loss value 352 and the second loss value 354, respectively. The electronic device can further determine, for example, the sum of the first loss value 352 and the second loss value 354 (i.e., a third loss value 356). The electronic device can directly train the feature extractor 330 by reducing the third loss value 356. The electronic device can, for example, update the parameters of the feature extractor 330 using gradient descent to reduce the first loss value 352, the second loss value 354, and the third loss value 356.

[0056] In this way, a feature extractor (e.g., feature extractor 330) can be trained using a general sample set containing a large amount of labeled data (i.e., general text labels) and multimodal data. This fully leverages the complementarity of information, improves the robustness of the feature extractor, reduces the risk of underestimation, and helps improve the stability and generalization of the target model (e.g., model 130).

[0057] Return to reference Figure 2 In box 220, the electronic device adds a target classifier for the target scene to the trained feature extractor to obtain the target model. Here, the target classifier is a classifier adapted to the target scene. The classifier can be, for example, any suitable neural network. In some embodiments, the electronic device can directly acquire a target classifier adapted to the target scene. The target scene may, for example, include a risk control scenario of an application (e.g., a risk control scenario activated by the application). In some embodiments, the electronic device can also acquire multiple classifiers and, based on the performance (i.e., classification effect) of the multiple classifiers in the target scene, determine the classifier that best fits the target scene and designate this classifier as the target classifier.

[0058] In box 230, the electronic device determines the trained target model by training a target classifier using at least a target sample set relevant to the target scene. The target sample set includes multiple target data sample-target label pairs, where the target data samples are extracted from the data samples of the target scene, and the target labels indicate the category to which the corresponding target data sample is classified within the target scene. The target data samples here may include, for example, any suitable data such as device attribute data from the target scene, statistical data from the target scene, or in-site behavior data from the target scene.

[0059] Device attribute data can be, for example, device attribute data obtained from other electronic devices in the target scenario, such as the memory size, device name, etc., of these other electronic devices. Statistical data can be calculated based on device attribute data. For example, when a large amount of device data from other electronic devices is obtained, this large amount of other electronic devices can include multiple electronic devices provided by multiple providers (each provider can also provide multiple electronic devices). Statistical data can, for example, indicate the statistical results of the multiple electronic devices provided by each provider. In-site behavior data in the target scenario can, for example, indicate data associated with various appropriate user behaviors in the target scenario (e.g., browsing behavior, viewing behavior, etc.).

[0060] In some embodiments, the target model further includes an adaptation layer connected between the feature extractor and the target classifier. This adaptation layer can be, for example, any suitable neural network such as a feedforward application network or a multilayer perceptron. The adaptation layer can transform the feature representation extracted by the feature extractor into a target feature representation that matches the target scene. Compared to models traditionally trained directly on data from the target scene, the target model with an adaptation layer (i.e., a target model including a feature extractor, an adaptation layer, and a target classifier) ​​retains the general learning capability of the feature extractor and adapts to the target scene, which can greatly improve the target model's robustness, spatial structure representation ability, and classification performance.

[0061] The adaptation layer can be trained independently of the target classifier, or it can be trained jointly with the target classifier. The following describes the case of training the adaptation layer separately (i.e., training the adaptation layer and the target classifier separately).

[0062] In some embodiments, the electronic device may utilize a trained feature extractor to extract multiple first feature representations corresponding to multiple target data samples in a multiple target data sample-target label pair. As mentioned above, the multiple target data samples may include, for example, any suitable data such as device attribute data in the target scene, statistical data in the target scene, and in-site behavior data in the target scene. The multiple first feature representations may include, for example, device attribute feature representations corresponding to device attribute data, statistical feature representations corresponding to statistical data, in-site behavior feature representations corresponding to in-site behavior data, and so on.

[0063] The electronic device can then utilize an adaptation layer to transform multiple first feature representations to obtain multiple target feature representations that match the target scene. For example, the electronic device can obtain multiple target feature representations that match the target scene by transforming the dimensions of the multiple first feature representations. The electronic device can then use a trained first classifier to process the multiple target feature representations to determine multiple first prediction classification results corresponding to the multiple target feature representations. It is important to note that the first classifier here is a different trained classifier than the target classifier.

[0064] An electronic device can input multiple target feature representations into a trained first classifier to obtain multiple first predicted classification results corresponding to these target feature representations. Each predicted classification result can indicate the category of the corresponding target data sample in the target scenario. For example, in a risk control scenario activated by an application, there may be two categories: "risk exists" and "risk does not exist". The first classifier can determine the predicted classification result of the corresponding target data sample as either "risk exists" or "risk does not exist" based on each acquired target feature representation.

[0065] Since the accuracy of the target feature representation affects the accuracy of multiple first-prediction classification results, the electronic device can train the adaptation layer based on the errors between multiple first-prediction classification results and multiple target labels in multiple target data sample-target label pairs (i.e., the errors between the categories of the target data samples indicated by the multiple first-prediction classification results and multiple target labels). It is also important to note that the parameters of the feature extractor are not changed during the training of the adaptation layer. That is, the electronic device can reduce the errors between multiple first-prediction classification results and multiple target labels in multiple target data sample-target label pairs by adjusting the parameters of the adaptation layer. The electronic device can determine that the training of the adaptation layer is complete when this error falls below a threshold. The trained adaptation layer can accurately convert the feature representation extracted by the feature extractor into the target feature representation in the target scene.

[0066] The electronic device can then train a target classifier using a trained feature extraction model and a trained adaptation layer. The electronic device can use the trained adaptation layer to transform multiple first feature representations extracted by the trained feature extractor to obtain multiple target feature representations that match the target scene. The electronic device can then use a target classifier to process the multiple target feature representations to determine multiple target prediction classification results corresponding to the multiple target feature representations. Taking a risk control scenario activated by an application as an example, target labels can indicate whether a corresponding target label is risky. The electronic device can train the target classifier based on the error between multiple target prediction classification results and multiple target labels. It should be noted that during the training of the target classifier, the parameters of the feature extractor (e.g., the weights of the feature extractor layers) can be changed or kept constant. The electronic device can determine the target model as a combination of the trained feature extractor, the trained adaptation layer, and the trained classifier. Alternatively, the electronic device can train the target model by training the feature extractor, the adaptation layer, and the classifier separately.

[0067] Continue to refer to Figure 3 ,like Figure 3As shown, in response to the feature extractor 330 completing training in the first training phase 302, the electronic device can add an adaptation layer 360 after the trained feature extractor 330 in the second training phase 304. The electronic device can provide a target sample set 318 to the trained feature extractor 330. The target sample set 318 is a sample set including multiple target data samples within the target scene and their corresponding target labels. The number of multiple target data samples in the target sample set 318 can be less than the number of general sample images 316 in the general sample set 312. The trained feature extractor 330 can extract multiple first feature representations corresponding to the multiple target data samples in the target sample set 318. These multiple first feature representations are provided to the adaptation layer 360. The adaptation layer 360 can perform transformations on these multiple first feature representations to obtain multiple target feature representations that match the target scene. The electronic device can use a trained first classifier to process the multiple target feature representations to determine multiple first predicted classification results corresponding to the multiple target feature representations. The electronic device can determine a fourth loss value 370 based on multiple first predicted classification results and multiple target labels corresponding to these multiple target data samples in the target sample set 318. Here, the fourth loss value 370 can, for example, be a cross-entropy loss. The electronic device can, for example, train the adaptation layer 360 by reducing the fourth loss value 370. In response to the fourth loss value 370 being less than a threshold, the electronic device can determine that the adaptation layer 360 has completed training.

[0068] In response to the completion of training of the adaptation layer 360 in the second training phase 304, the electronic device can add a pre-determined target classifier (i.e., classifier 382) matching the target scene after the trained adaptation layer 360 in the third training phase 304. The electronic device can obtain multiple target feature representations using the trained feature extractor 330 and the trained adaptation layer 360. These multiple target feature representations will be provided to the classifier 382. The classifier 382 can process the multiple target feature representations to determine multiple target prediction classification results corresponding to the multiple target feature representations. The electronic device can determine a fifth loss value 390 based on the multiple target prediction classification results and multiple target labels. Here, the fifth loss value 390 can also be, for example, a cross-entropy loss. The electronic device can update the parameters of the classifier 382, ​​for example, through backpropagation to reduce the fifth loss value 390. The electronic device can perform multiple iterations, and in response to the fifth loss value 390 being less than a threshold, the electronic device can determine that the classifier 382 has completed training. The electronic device can identify the combination of a trained feature extractor 330, a trained adaptation layer 360, and a trained classifier 382 as the target model 380.

[0069] The following describes the scenario where the adaptation layer and the target classifier are jointly trained. In some embodiments, the electronic device uses a trained feature extractor to extract multiple second feature representations corresponding to multiple target data samples in multiple target data sample-target label pairs. The electronic device then uses the adaptation layer to perform a transformation on the multiple second feature representations to obtain multiple target feature representations that match the target scene. The electronic device uses the target classifier to process the multiple target feature representations to determine multiple second predicted classification results corresponding to the multiple target feature representations. The electronic device can determine the error between the second predicted classification results and multiple target labels in the multiple target data sample-target label pairs (e.g., a sixth loss value, which can also be, for example, cross-entropy loss). The electronic device can use this error to train the adaptation layer and the target classifier. For example, the electronic device can update the parameters of the adaptation layer and the target classifier by reducing the sixth loss value. The electronic device can also determine that the adaptation layer and the target classifier are trained in response to the sixth loss value being less than a threshold.

[0070] In summary, in the embodiments of this disclosure, a robust and improved feature extractor can be obtained by training a general sample set containing a large amount of labeled data, including multimodal data. Furthermore, the adaptation layer and classifier can be trained using a target sample set from the target scene, which includes a small amount of labeled data. This allows the final target model, including the feature extractor, adaptation layer, and classifier, to be applied to the target scene (i.e., the target model can be deployed in the target scene). This fully utilizes the multimodal data in the general sample set to improve the model performance of the target model. It reduces the manpower cost of labeling data while conveniently and quickly training the target model. It improves the robustness and generalization of the target model, helping to enhance its task processing capabilities in the target scene. Moreover, only the target sample set needs to be used to train the adaptation layer and classifier, which accelerates the model's adaptation to the target scene and shortens the model's iteration cycle.

[0071] Accordingly, after the target model is trained, during the application phase of the target model, the electronic device equipped with the model application system 160 can use the trained feature extractor of the target model for the target scenario to extract the feature representation corresponding to the target data in the target scenario. The feature extractor is trained for a general scenario. Here, the target scenario can be, for example, the risk control scenario of the application (e.g., the risk control scenario activated by the application).

[0072] In some embodiments, the electronic device may also utilize a target classifier of the target model to process the feature representation to determine the classification result corresponding to the feature representation. The target classifier is trained for the target scene, and the classification result indicates the category to which the target data is classified in the target scene.

[0073] Specifically, in some embodiments, the electronic device can directly utilize the target classifier to process the feature representation extracted by the feature extractor. Alternatively or additionally, in some embodiments, to improve the accuracy of the target model in processing target data, the target model may further include an adaptation layer for the target scene. The adaptation layer may be located between the feature extractor and the target classifier, and it can be used to transform the feature representation extracted by the feature extractor into a feature representation in the target scene. Thus, the target classifier can directly process the feature representation in the target scene to determine the classification result. This helps improve the accuracy of the classification result.

[0074] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes.

[0075] Figure 4 A schematic structural block diagram of an apparatus 400 for model training according to certain embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in a model training system 150. The various modules / components in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0076] like Figure 4 As shown, the apparatus 400 includes an extractor training module 410 configured to train a feature extractor using a general sample set, the general sample set including multiple general sample image-general text label pairs, where the general text label indicates at least one of the semantic information of the corresponding general sample image and the category corresponding to the corresponding general sample image. The apparatus 400 also includes a model acquisition module 420 configured to add a target classifier for the target scene to the trained feature extractor to obtain a target model. The apparatus 400 further includes a classifier training module 430 configured to determine the trained target model by training a target classifier using at least a target sample set related to the target scene, the target sample set including multiple target data sample-target label pairs, where the target data samples are extracted from data samples of the target scene, and the target labels indicate the category to which the corresponding target data sample is classified within the target scene.

[0077] In some embodiments, the extractor training module 410 is further configured to: determine multiple image feature representations of multiple general sample images in visual space in multiple general sample image-general text label pairs; determine a first loss value corresponding to the multiple general sample images based at least on the categories corresponding to the multiple general sample image-general text label pairs and the multiple image feature representations; and train the feature extractor at least by reducing the first loss value.

[0078] In some embodiments, the extractor training module 410 is further configured to: determine multiple text feature representations in semantic space for multiple general text labels in multiple general sample image-general text label pairs; map the multiple text feature representations in semantic space to visual space to obtain multiple text feature representations in visual space; determine a second loss value corresponding to the multiple general text labels based at least on the categories corresponding to the multiple general sample image-general text label pairs and the multiple text feature representations in visual space; and train the feature extractor at least by reducing the second loss value.

[0079] In some embodiments, the target model further includes an adaptation layer connected between the feature extractor and the target classifier, and the adaptation layer is trained independently of the target classifier, or the adaptation layer is trained jointly with the target classifier.

[0080] In some embodiments, the apparatus 400 further includes an adaptation layer separate training module, configured to separately train the adaptation layer by: extracting multiple first feature representations corresponding to multiple target data samples in multiple target data sample-target label pairs using a trained feature extractor; performing a transformation on the multiple first feature representations using the adaptation layer to obtain multiple target feature representations that match the target scene; processing the multiple target feature representations using a trained first classifier to determine multiple first predicted classification results corresponding to the multiple target feature representations; and training the adaptation layer based on the error between the multiple first predicted classification results and multiple target labels in the multiple target data sample-target label pairs, wherein the parameters of the feature extractor are not changed during the training of the adaptation layer.

[0081] In some embodiments, the classifier training module 430 is further configured to: process multiple target feature representations using a target classifier to determine multiple target prediction classification results corresponding to the multiple target feature representations; and train a target classifier based on the error between the multiple target prediction classification results and the multiple target labels.

[0082] In some embodiments, the apparatus 400 further includes a joint training module configured to jointly train an adaptation layer and a target classifier by: extracting multiple second feature representations corresponding to multiple target data samples in multiple target data sample-target label pairs using a trained feature extractor; performing a transformation on the multiple second feature representations using the adaptation layer to obtain multiple target feature representations that match the target scene; processing the multiple target feature representations using a target classifier to determine multiple second predicted classification results corresponding to the multiple target feature representations; and training the adaptation layer and the target classifier based on the error between the second predicted classification results and multiple target labels in the multiple target data sample-target label pairs.

[0083] In some embodiments, the feature extractor includes at least a neural network based on a window attention mechanism.

[0084] The units and / or modules included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units and / or modules in device 400 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0085] Figure 5 A block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 5 The illustrated electronic device 500 can be used to implement an electronic device equipped with a model training system 150 and / or a model application system 160.

[0086] like Figure 5 As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors or processing units 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0087] Electronic device 500 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data and accessible within electronic device 500.

[0088] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0089] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0090] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0091] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0092] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0094] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0096] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for model training, comprising: A feature extractor is trained using a general sample set, which includes multiple general sample image-general text label pairs, wherein the general text label indicates at least one of the semantic information of the corresponding general sample image and the category corresponding to the corresponding general sample image; Add a target classifier for the target scene to the trained feature extractor to obtain the target model; as well as The trained target model is determined by training the target classifier using at least a target sample set associated with the target scene, the target sample set comprising multiple target data sample-target label pairs, the target data samples being extracted from data samples of the target scene, and the target labels indicating the category to which the corresponding target data sample is classified within the target scene.

2. The method of claim 1, wherein training the feature extractor comprises: Determine multiple image feature representations in visual space for the multiple general sample images in the multiple general sample image-general text label pairs; Based at least on the categories corresponding to the plurality of general sample images and general text labels and the feature representations of the plurality of images, a first loss value corresponding to the plurality of general sample images is determined; as well as The feature extractor is trained by at least reducing the first loss value.

3. The method according to claim 1, wherein training the feature extractor further comprises: Determine the semantic space representation of multiple text features of the multiple general text labels in the multiple general sample image-general text label pairs; Multiple text feature representations in the semantic space are mapped to the visual space to obtain multiple text feature representations in the visual space; Based at least on the multiple general sample images-general text labels corresponding to their respective categories and multiple text feature representations in the visual space, a second loss value corresponding to the multiple general text labels is determined; as well as The feature extractor is trained by at least reducing the second loss value.

4. The method of claim 1, wherein the target model further comprises an adaptation layer connected between the feature extractor and the target classifier, and The adaptive layer is trained independently of the target classifier, or the adaptive layer is trained jointly with the target classifier.

5. The method of claim 4, wherein the adaptation layer is trained separately by: The trained feature extractor is used to extract multiple first feature representations corresponding to multiple target data samples in the multiple target data sample-target label pairs; The adaptation layer is used to transform the plurality of first feature representations to obtain a plurality of target feature representations that match the target scene; The trained first classifier is used to process the multiple target feature representations to determine multiple first prediction classification results corresponding to the multiple target feature representations; as well as The adaptation layer is trained based on the errors between the multiple first predicted classification results and the multiple target labels in the multiple target data sample-target label pairs, wherein the parameters of the feature extractor are not changed during the training of the adaptation layer.

6. The method of claim 5, wherein training the target classifier comprises: The target classifier is used to process the multiple target feature representations to determine the multiple target prediction classification results corresponding to the multiple target feature representations; as well as The target classifier is trained based on the error between the multiple target prediction classification results and the multiple target labels.

7. The method of claim 4, wherein the adaptation layer and the target classifier are jointly trained by: The trained feature extractor is used to extract multiple second feature representations corresponding to multiple target data samples in the multiple target data sample-target label pairs; The adaptation layer is used to transform the multiple second feature representations to obtain multiple target feature representations that match the target scene; The target classifier is used to process the multiple target feature representations to determine multiple second predicted classification results corresponding to the multiple target feature representations; as well as The adaptive layer and the target classifier are trained based on the error between the second predicted classification result and the multiple target labels in the multiple target data sample-target label pairs.

8. The method of claim 1, wherein the feature extractor comprises at least a neural network based on a window attention mechanism.

9. An apparatus for model training, comprising: The extractor training module is configured to train a feature extractor using a general sample set, which includes multiple general sample image-general text label pairs, wherein the general text label indicates at least one of the semantic information of the corresponding general sample image and the category corresponding to the corresponding general sample image; The model acquisition module is configured to add a target classifier for the target scene to the trained feature extractor to obtain the target model. as well as The classifier training module is configured to determine a trained target model by training the target classifier using at least a target sample set associated with the target scene, the target sample set including multiple target data sample-target label pairs, the target data samples being extracted from data samples of the target scene, and the target labels indicating the category to which the corresponding target data sample is classified within the target scene.

10. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 8.

Images