Media content classification model training method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a media content classification model training method and device, equipment and a storage medium. Extracting a first media content text and a second media content text from the media content text set; respectively inputting the first media content text and the second media content text into a first media content classification model to obtain a first media content feature and a first media content label category corresponding to the first media content text, and a second media content feature and a second media content label category corresponding to the second media content text; determining the similarity between the first media content feature and the second media content feature and the similarity between the first media content label category and the second media content label category to obtain a first similarity and a second similarity; and training a second media content classification model based on any one of the first media content text and the second media content text, the first similarity and the second similarity to obtain a target media content classification model.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of data processing technology, and in particular to a training method, apparatus, device and storage medium for a media content classification model. Background Art

[0002] With the rapid development of the Internet and the popularity of mobile devices, short videos are closely related to people's lives, resulting in a huge amount of short video data. By classifying a large number of short videos with hierarchical labels, it can bring huge benefits to downstream businesses, such as: as label features for video recommendations. Therefore, it is particularly important to quickly and accurately classify short videos with hierarchical labels. Summary of the invention

[0003] The embodiments of the present disclosure provide a method, apparatus, device and storage medium for training a media content classification model, which can improve the generalization and accuracy of the media content classification model.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for training a media content classification model, comprising:

[0005] Extracting a first media content text and a second media content text from a media content text set; wherein each media content text in the media content text set includes at least one first type of text with a signal-to-noise ratio lower than a first preset threshold and at least one second type of text with a signal-to-noise ratio higher than a second preset threshold;

[0006] Inputting the first media content text and the second media content text into a first media content classification model respectively, obtaining first media content features and first media content label categories corresponding to the first media content text, and second media content features and second media content label categories corresponding to the second media content text; wherein the first media content classification model is a pre-trained model;

[0007] Determine the similarity between the first media content feature and the second media content feature and the similarity between the first media content tag category and the second media content tag category to obtain a first similarity and a second similarity;

[0008] A second media content classification model is trained based on any one of the first media content text and the second media content text, the first similarity and the second similarity to obtain a target media content classification model; wherein the second media content classification model has the same structure as the first media content classification model.

[0009] In a second aspect, the present disclosure also provides a training device for a media content classification model, including:

[0010] A media content text extraction module, used to extract a first media content text and a second media content text from a media content text set; wherein each media content text in the media content text set includes at least one first type of text with a signal-to-noise ratio lower than a first preset threshold and at least one second type of text with a signal-to-noise ratio higher than a second preset threshold;

[0011] a media content feature and media content label category acquisition module, which inputs the first media content feature and the second media content text into a first media content classification model respectively, and obtains the first media content feature and the first media content label category corresponding to the first media content text, and the second media content feature and the second media content label category corresponding to the second media content text; wherein the first media content classification model is a pre-trained model;

[0012] A similarity determination module, configured to determine the similarity between the first media content feature and the second media content text and the similarity between the first media content tag category and the second media content tag category, to obtain a first similarity and a second similarity;

[0013] A target media content classification model acquisition module is used to train a second media content classification model based on any one of the first media content text and the second media content text, the first similarity and the second similarity to obtain a target media content classification model; wherein the second media content classification model has the same structure as the first media content classification model.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electrical device, the electrical device comprising:

[0015] one or more processors;

[0016] a storage device for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the training method of the media content classification model described in the embodiment of the present disclosure.

[0018] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the training method for the media content classification model described in the embodiments of the present disclosure.

[0019] The present disclosure embodiment discloses a training method, apparatus, device and storage medium for a media content classification model, the method comprising: extracting a first media content text and a second media content text from a media content text set; wherein each media content text in the media content text set comprises at least one first type of text with a signal-to-noise ratio lower than a first preset threshold and at least one second type of text with a signal-to-noise ratio higher than a second preset threshold; inputting the first media content text and the second media content text into a first media content classification model respectively, obtaining a first media content feature and a first media content label category corresponding to the first media content text, and a second media content feature and a second media content label category corresponding to the second media content text; wherein the first media content classification model is a pre-trained model; determining the similarity between the first media content feature and the second media content text and the similarity between the first media content label category and the second media content label category, obtaining a first similarity and a second similarity; training a second media content classification model based on any one of the first media content text and the second media content text, the first similarity and the second similarity, obtaining a target media content classification model; wherein the second media content classification model has the same structure as the first media content classification model. The training method of the media content classification model provided in the embodiment of the present disclosure trains a second media content classification model based on media content text, media content features and media content labels output by the first media content classification model, and can correct the media content label category output by the second media content classification model, thereby improving the accuracy and generalization of the media content classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 It is a flowchart of a method for training a media content classification model provided by an embodiment of the present disclosure;

[0022] Figure 2 is a structural diagram of a media content classification model provided by an embodiment of the present disclosure;

[0023] Figure 3 is a structural schematic diagram of a feature adjustment module provided in an embodiment of the present disclosure;

[0024] Figure 4 It is a structural schematic diagram of a training device for a media content classification model provided by an embodiment of the present disclosure;

[0025] Figure 5It is a structural schematic diagram of an electrical device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0027] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0028] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0029] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0030] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0031] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0032] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0033] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electrical device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0034] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electrical device.

[0035] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0036] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0037] In the field of natural language understanding, the classification of hierarchical labels of text based on deep learning methods is a popular topic with important academic and application value. In short videos, the texts involved include video title text, optical character recognition (OCR) text, and automatic speech recognition (ASR) text. These video texts often correspond to multiple levels of semantic labels. For example: the hierarchical label categories can be: "Leisure and Entertainment-Store Exploration Guide-Deduction Board Games" or "Fashion-Outfit-Accessories" and other three-level labels. This type of application scenario generally requires the construction of a neural network model including a text encoder and a hierarchical label task classifier, which is trained through manually labeled data, so that the hierarchical label category corresponding to the video text can be predicted.

[0038] In the classification learning of hierarchical labels, there are generally two aspects of data noise: one is that the multi-source input in the training data contains text with low signal-to-noise ratio, such as OCR text and ASR text; the other is the label data input in the training data. Since the labels are manually labeled, it is inevitable that there will be problems such as inconsistent data labeling and low accuracy. In the prior art, the text with low signal-to-noise ratio in the multi-source input in the training data is discarded and eliminated, which causes information waste of training data and low model generalization. For the problem of low label accuracy in the training data, label smoothing and soft labeling are generally used. This method essentially adds noise to the data and cannot truly reflect the distribution of labels. The improvement of the model generalization ability is limited and the number of training data sets cannot be increased.

[0039] Figure 1 A flow chart of a method for training a media content classification model provided in an embodiment of the present disclosure is provided. The embodiment of the present disclosure is applicable to the situation where a media content classification model is trained. The method can be executed by a training device for a media content classification model, and the device can be implemented in the form of software and / or hardware, and optionally, by an electrical device, which can be a mobile terminal, a PC or a server, etc.

[0040] like Figure 1 As shown, the method includes:

[0041] S110: Extracting a first media content text and a second media content text from a media content text set.

[0042] Among them, each media content text in the media content text set includes at least one first type of text with a signal-to-noise ratio lower than a first preset threshold and at least one second type of text with a signal-to-noise ratio higher than a second preset threshold. Among them, the first preset threshold and the second preset threshold can be the same or different. If they are different, the first preset threshold is less than the second preset threshold. In this embodiment, the signal-to-noise ratio of the text can be understood as the ratio between the information in the text that is useful for classification and the interference information (also referred to as noise information). The media content can be one or more of video (such as short video), audio, image and plain text.

[0043] The media content text set can be understood as a pre-collected media content text training sample set, which includes media content texts and real media content label categories corresponding to the media content texts. Exemplarily, the media content text set can be expressed as: D = [(x1, y1), (x2, y2), ..., (x d ,y d )], where x i represents the media content text of the i-th training sample, y iis the real media content label category corresponding to the ith media content text. In this embodiment, the media content label category may be a hierarchical label category.

[0044] Among them, the second type of text is title text. The first type of text is OCR text and / or ASR text. That is, the signal-to-noise ratio of the title text is higher than the second preset threshold. The signal-to-noise ratio of the OCR text and the ASR text is lower than the first preset threshold. The title text can be understood as the text corresponding to the title of the media content, the OCR text can be understood as the text obtained by performing text recognition on optical characters in the media content, and the ASR can be understood as the text obtained by performing text recognition on audio in the media content.

[0045] In this embodiment, two media content texts may be randomly selected from the media content text set as the first media content text and the second media content text, denoted by x i and x j .

[0046] S120, inputting the first media content text and the second media content text into a first media content classification model respectively, obtaining first media content features and first media content label categories corresponding to the first media content text, and second media content features and second media content label categories corresponding to the second media content text.

[0047] Among them, the first media content classification model is a pre-trained model. Figure 2 is a structural diagram of a first media content classification model in an embodiment of the present disclosure, such as Figure 2 As shown, the first media content classification model includes: a text feature encoding module, a feature adjustment module, a feature fusion module and a label classification module.

[0048] Specifically, the text feature encoding module is used to extract semantic features of the first category text and the second category text in the media content text, and output the first category semantic features and the second category semantic features. The feature adjustment module is used to adjust the first category semantic features according to the second category semantic features to obtain the adjusted first category semantic features. The feature fusion module is used to fuse the second category semantic features and the adjusted first category semantic features to obtain media content features. The label classification module is used to classify media content features and output media content label categories.

[0049] The media content text is the first media content text or the second media content text, the media content feature is the first media content feature or the second media content feature, and the media content tag category is the first media content tag category or the second media content tag category.

[0050] In this embodiment, the text feature encoding module includes: a word segmentation unit, an embedding unit and a feature extraction unit (Bert). The process of the text feature encoding module for extracting semantic features from the first category text and the second category text in the media content text can be: firstly, the first category text and the second category text are respectively segmented by the word segmentation unit to obtain a word sequence; then the word sequence is processed by the embedding unit, and then the processing result is input into the feature extraction unit to obtain the semantic features of each word, thereby obtaining the first category semantic features and the second category semantic features.

[0051] Among them, both the first type of semantic features and the second type of semantic features can be represented by vectors.

[0052] For example, the second type of text is the title text, and the first type of text is the OCR text and the ASR text. The text feature encoding module is used to extract semantic features from the first type of text and the second type of text in the media content text. The process can be: first, the title text, the OCR text, and the ASR text are segmented by the word segmentation unit to obtain a word sequence; then the word sequence is processed by the embedding unit, and the processing result is input into the feature extraction unit to obtain the semantic features of each word, thereby obtaining the title semantic features, the OCR semantic features, and the ASR semantic features.

[0053] Among them, the title semantic feature is represented by the title semantic feature vector, expressed as C ti =[c ti1 ,c ti2 ,…c tim ], that is, the title semantic feature of the i-th media content text. The OCR semantic feature is represented by the OCR semantic feature vector, expressed as: C oi =[c oi1 ,c oi2 ,…c oin ], that is, the OCR semantic features of the i-th media content text. The ASR semantic features are represented by the ASR semantic feature vector, expressed as: C ai =[c ai1 ,c ai2 ,…c aik ], that is, the ASR semantic features of the i-th media content text.

[0054] Optionally, the feature adjustment module includes a first average pooling layer and a feature adjustment unit. The input of the first average pooling layer is the second type of semantic features, and the input of the feature adjustment unit is the first type of semantic features.

[0055] Among them, the feature adjustment unit includes: a fully connected layer, an activation layer, a fusion layer and a second average pooling layer; the fully connected layer, the activation layer, the fusion layer and the second average pooling layer are connected in sequence; the output of the first average pooling layer is connected to the input of the fully connected layer; the input of the fully connected layer is jump-connected to the input of the fusion layer.

[0056] Specifically, the data processing process of the feature adjustment module is as follows: the first average pooling layer performs an average pooling operation on the second-class semantic features to obtain the second-class global semantic features, and then the second-class global semantic features and the first-class semantic features are sequentially passed through the fully connected layer and the activation layer, and then the output of the activation layer is fused with the original input (the first-class semantic features) through the fusion layer, and finally the fused features are processed through the second average pooling layer to obtain the adjusted second-class semantic features. Among them, the function of the fusion layer is to multiply the corresponding elements of the vectors of the two input features.

[0057] For example, Figure 3 is a structural diagram of a feature adjustment module in an embodiment of the present disclosure, such as Figure 3 As shown, it includes a first average pooling layer and two parallel feature adjustment units. The input of the first average pooling layer is the title semantic feature, the input of one feature adjustment unit is the OCR semantic feature, and the input of the other feature adjustment unit is the ASR semantic feature. Specifically, Figure 3 As shown, the first average pooling layer is used to calculate the title semantic feature C ti After average pooling operation, the global semantic features of the title are obtained Then the global semantic features of the title Respectively with OCR semantic features C oi and ASR semantic features C ai After passing through the fully connected layer and the activation layer in sequence, the output of the activation layer is combined with the original input (OCR semantic feature C oi Or ASR semantic feature C ai ) is fused through the fusion layer, and finally the fused features are processed through the second average pooling layer to obtain the adjusted OCR semantic features And the adjusted ASR semantic features The function of the fusion layer is to multiply the corresponding elements of the two input feature vectors. In this embodiment, the signal-to-noise ratio of the title semantic feature is high, and the signal-to-noise ratio of the OCR semantic feature and the ASR semantic feature is low. By adjusting the OCR semantic feature and the ASR semantic feature with a low signal-to-noise ratio through the title semantic feature with a high signal-to-noise ratio, the input text information can be adjusted, the confidence of the input text can be improved, and the accuracy and generalization of the classification model can be further improved.

[0058] In this embodiment, the function of the feature fusion module is to complete feature fusion by summing the second type of global semantic features output by the feature adjustment module and the adjusted first type of semantic features, thereby outputting media content features. Exemplarily, the function of the feature fusion module is to complete feature fusion by summing the title global semantic features output by the feature adjustment module, the adjusted OCR semantic features, and the adjusted ASR semantic features, thereby outputting media content features.

[0059] In this embodiment, the tag classification module can use multiple fully connected layers to fit the output of multiple tag categories. Assuming that there are N tag categories in total, after the media content features are input into the tag classification module, the output media content tag category can be represented as Represents the probability that the i-th media content belongs to the n-th label category.

[0060] In this embodiment, the training method of the first media content classification model is: extracting a third media content text from a media content text set; inputting the third media content text into the first media content classification model, and outputting a predicted media content label category corresponding to the third media content text; determining a third loss function based on the predicted media content label category and the real media content label category corresponding to the third media content text; and training the first media content classification model based on the third loss function.

[0061] The third loss function may be a BCEWithLogitsLoss loss function. Specifically, extract the third media content text x from the media content text set. k , the third media content text x k The text feature encoding module, feature adjustment module, feature fusion module and label classification module in the first media content classification model are input in sequence, and the predicted media content label category corresponding to the third media content text is output. Assuming that the first media content classification model is represented by f1(), then Third media content text x k The corresponding real media content label category is y k , then the third loss function can be expressed as: After obtaining the third loss function, the first media content classification model is trained based on the third loss function, so as to train the second media content classification model based on the trained first media content classification model.

[0062] S130: Determine the similarity between the first media content feature and the second media content feature and the similarity between the first media content tag category and the second media content tag category to obtain a first similarity and a second similarity.

[0063] In this embodiment, both the first media content feature and the second media content feature can be represented by a feature vector. Therefore, the method for determining the similarity between the first media content feature and the second media content feature can be: determine the cosine similarity between the feature vector of the first media content feature and the feature vector of the second media content feature, and use the cosine similarity as the first similarity. The first media content label category and the second media content label category can also be represented by a vector. Therefore, the method for determining the similarity between the first media content label category and the second media content label category can be: determine the cosine similarity between the vector corresponding to the first media content label category and the vector corresponding to the second media content label category, and use the cosine similarity as the second similarity. Assume that the first media content feature is represented by c i , the second media content feature is represented by c j , then the first similarity can be expressed as cos(c i ,c j ); The first media content label category is represented as The second media content label category is represented as Then the second similarity can be expressed as

[0064] S140: Training a second media content classification model based on any one of the first media content text and the second media content text, the first similarity, and the second similarity to obtain a target media content classification model.

[0065] Among them, the second media content classification model has the same structure as the first media content classification model, that is, the second media content classification model also includes: a text feature encoding module, a feature adjustment module, a feature fusion module and a label classification module. The processing process of the second media content classification model on the input media content text is similar to the processing process of the first media content classification model on the input media content text, which will not be repeated here.

[0066] Specifically, based on any one of the first media content text and the second media content text, the first similarity and the second similarity, the second media content classification model is trained to obtain the target media content classification model as follows: any one of the first media content text and the second media content text is input into the second media content classification model, and a predicted media content label category is output; a first loss function is determined based on the predicted media content label category and the actual media content label category of the input media content text; a second loss function is determined based on the first similarity and the second similarity; and the second media content classification model is trained based on the first loss function and the second loss function to obtain the target media content classification model.

[0067] The first loss function may be a BCEWithLogitsLoss loss function. Assume that the first media content text xi Input the second media content classification model and output the first media content text x i Corresponding predicted media content label category Assuming that the second media content classification model is represented by f2(), then First media content text x i The corresponding real media content label category is y i , then the first loss function can be expressed as: The second loss function can be determined according to the first similarity and the second similarity by first determining the fusion coefficient according to the first similarity, then linearly superimposing the first similarity and the second similarity, and multiplying the linear superposition result by the fusion coefficient to obtain the second loss function. The second loss function can be expressed as: Among them, λ is the fusion coefficient.

[0068] Among them, the method of determining the fusion coefficient according to the first similarity can be: if the first similarity is greater than or equal to the set threshold, the fusion coefficient takes the first set value (such as: 1); if the first similarity is less than the set threshold, the fusion coefficient takes the second set value (such as: 0). In this embodiment, samples of the same media content label category are similar or related in nature, and there are obvious differences between samples of different media content label categories. Two texts with similar content should be predicted as the same media content category label. Therefore, the second similarity of two samples of the same media content label category should be as close as possible to the second similarity. Based on this, training the second media content classification model can achieve the effect of correcting the annotated labels, further improving the accuracy of the training of the second media content classification model, thereby improving the accuracy and generalization of the trained second media content classification model.

[0069] Specifically, the second media content classification model is trained according to the first loss function and the second loss function to obtain the target media content classification model. The method can be: fusing the first loss function and the second loss function to obtain the target loss function; training the second media content classification model based on the target loss function to obtain the target media content classification model.

[0070] The fusion of the first loss function and the second loss function can be understood as: superimposing the first loss function and the second loss function. The formula is: L = L1 + L2. Finally, the second media content classification model is trained based on the target loss function to obtain the target media content classification model.

[0071] In this embodiment, label correction is achieved through the trained first media content classification model, and the adjustment of OCR semantic features and ASR semantic features with low noise ratio is achieved through the feature adjustment module in the model, so as to process the noise data in the sample and improve the accuracy and generalization of the neural network.

[0072] The technical solution of this embodiment is to extract a first media content text and a second media content text from a media content text set; wherein each media content text in the media content text set includes at least one first type of text with a signal-to-noise ratio lower than a first preset threshold and at least one second type of text with a signal-to-noise ratio higher than a second preset threshold; the first media content text and the second media content text are respectively input into a first media content classification model to obtain a first media content feature and a first media content label category corresponding to the first media content text, and a second media content feature and a second media content label category corresponding to the second media content text; wherein the first media content classification model is a pre-trained model; determine the similarity between the first media content feature and the second media content text and the similarity between the first media content label category and the second media content label category to obtain a first similarity and a second similarity; train a second media content classification model based on any one of the first media content text and the second media content text, the first similarity and the second similarity to obtain a target media content classification model; wherein the second media content classification model has the same structure as the first media content classification model. The training method of the media content classification model provided in the embodiment of the present disclosure trains a second media content classification model based on media content text, media content features and media content labels output by the first media content classification model, and can correct the media content label category output by the second media content classification model, thereby improving the accuracy and generalization of the media content classification model.

[0073] Figure 4 A structural diagram of a media content classification model training device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the device comprises:

[0074] The media content text extraction module 410 is used to extract the first media content text and the second media content text from the media content text set; wherein each media content text in the media content text set includes at least one first type of text with a signal-to-noise ratio lower than a first preset threshold and at least one second type of text with a signal-to-noise ratio higher than a second preset threshold;

[0075] The media content feature and media content label category acquisition module 420 inputs the first media content text and the second media content text into a first media content classification model, respectively, to obtain first media content features and first media content label categories corresponding to the first media content text, and second media content features and second media content label categories corresponding to the second media content text; wherein the first media content classification model is a pre-trained model;

[0076] A similarity determination module 430 is used to determine the similarity between the first media content feature and the second media content feature and the similarity between the first media content tag category and the second media content tag category to obtain a first similarity and a second similarity;

[0077] The target media content classification model acquisition module 440 is used to train a second media content classification model based on any one of the first media content text and the second media content text, the first similarity and the second similarity to obtain a target media content classification model; wherein the second media content classification model has the same structure as the first media content classification model.

[0078] Optionally, the first media content classification model and the second media content classification model both include: a text feature encoding module, a feature adjustment module, a feature fusion module and a label classification module.

[0079] Optionally, the media content feature and media content tag category acquisition module 420 is further used to:

[0080] The text feature encoding module is used to extract semantic features of the first type of text and the second type of text in the media content text, and output the first type of semantic features and the second type of semantic features; wherein the media content text is the first media content text or the second media content text;

[0081] The feature adjustment module is used to adjust the first type of semantic features according to the second type of semantic features to obtain adjusted first type of semantic features;

[0082] The feature fusion module is used to fuse the second type of semantic features and the adjusted first type of semantic features to obtain media content features; wherein the media content features are the first media content features or the second media content features;

[0083] The label classification module is used to classify the media content features and output a media content label category; wherein the media content label category is the first media content label category or the second media content label category.

[0084] Optionally, the feature adjustment module includes a first average pooling layer and a feature adjustment unit; the input of the first average pooling layer is the second type of semantic features, and the input of the feature adjustment unit is the first type of semantic features;

[0085] Among them, the feature adjustment unit includes: a fully connected layer, an activation layer, a fusion layer and a second average pooling layer; the fully connected layer, the activation layer, the fusion layer and the second average pooling layer are connected in sequence; the output of the first average pooling layer is connected to the input of the fully connected layer; the input of the fully connected layer is jump-connected to the input of the fusion layer.

[0086] Optionally, the target media content classification model acquisition module 440 is further used to:

[0087] Inputting any one of the first media content text and the second media content text into the second media content classification model, and outputting a predicted media content label category;

[0088] Determine a first loss function according to the predicted media content label category and the actual media content label category of the input media content text;

[0089] Determine a second loss function according to the first similarity and the second similarity;

[0090] A second media content classification model is trained according to the first loss function and the second loss function to obtain a target media content classification model.

[0091] Optionally, the target media content classification model acquisition module 440 is further used to:

[0092] The first loss function and the second loss function are combined to obtain a target loss function;

[0093] A second media content classification model is trained based on the target loss function to obtain a target media content classification model.

[0094] Optionally, it further includes: a first media content classification model training module, used to:

[0095] extracting a third media content text from the media content text set;

[0096] Inputting the third media content text into the first media content classification model, and outputting a predicted media content label category corresponding to the third media content text;

[0097] Determine a third loss function according to the predicted media content label category and the real media content label category corresponding to the third media content text;

[0098] The first media content classification model is trained based on the third loss function.

[0099] The training device for the media content classification model provided in the embodiments of the present disclosure can execute the training method for the media content classification model provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0100] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0101] Figure 5 This is a schematic diagram of the structure of an electrical device provided by an embodiment of the present disclosure. Figure 5 , which shows an electrical device (eg, Figure 5 The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electrical device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0102] like Figure 5 As shown, the electric device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electric device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0103] Typically, the following devices may be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 509. The communication devices 509 may allow the electrical device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electrical device 500 is shown with various devices, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead.

[0104] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0105] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0106] The electrical device provided in the embodiment of the present disclosure and the training method of the media content classification model provided in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0107] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the training method of the media content classification model provided in the above embodiments is implemented.

[0108] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0109] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0110] The computer-readable medium may be included in the electrical device, or may exist independently without being installed in the electrical device.

[0111] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electrical device, the electrical device:

[0112] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electrical device, the electrical device: extracts a first media content text and a second media content text from a media content text set; wherein each media content text in the media content text set includes at least one first type of text with a signal-to-noise ratio lower than a first preset threshold and at least one second type of text with a signal-to-noise ratio higher than a second preset threshold; inputs the first media content text and the second media content text into a first media content classification model respectively to obtain a first media content feature and a first media content label category corresponding to the first media content text, and a second media content feature and a second media content label category corresponding to the second media content text; wherein the first media content classification model is a pre-trained model; determines the similarity between the first media content feature and the second media content feature and the similarity between the first media content label category and the second media content label category to obtain a first similarity and a second similarity; trains a second media content classification model based on any one of the first media content text and the second media content text, the first similarity and the second similarity to obtain a target media content classification model; wherein the second media content classification model has the same structure as the first media content classification model.

[0113] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0116] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0117] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0119] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable combination.

[0120] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for training a media content classification model, characterized in that: include: Extracting a first media content text and a second media content text from a media content text set; wherein each media content text in the media content text set includes at least one first type of text with a signal-to-noise ratio lower than a first preset threshold and at least one second type of text with a signal-to-noise ratio higher than a second preset threshold; Inputting the first media content text and the second media content text into a first media content classification model respectively, obtaining first media content features and first media content label categories corresponding to the first media content text, and second media content features and second media content label categories corresponding to the second media content text; wherein the first media content classification model is a pre-trained model; Determine the similarity between the first media content feature and the second media content feature and the similarity between the first media content tag category and the second media content tag category to obtain a first similarity and a second similarity; A second media content classification model is trained based on any one of the first media content text and the second media content text, the first similarity and the second similarity to obtain a target media content classification model; wherein the second media content classification model has the same structure as the first media content classification model.

2. The method according to claim 1, characterized in that The first media content classification model and the second media content classification model both include: a text feature encoding module, a feature adjustment module, a feature fusion module and a label classification module.

3. The method according to claim 2, characterized in that Inputting the first media content text and the second media content text into a first media content classification model respectively, obtaining first media content features and first media content label categories corresponding to the first media content text, and second media content features and second media content label categories corresponding to the second media content text, including: The text feature encoding module is used to extract semantic features from the first type of text and the second type of text in the media content text, and output the first type of semantic features and the second type of semantic features; wherein the media content text is the first media content text or the second media content text; The feature adjustment module is used to adjust the first type of semantic features according to the second type of semantic features to obtain adjusted first type of semantic features; The feature fusion module is used to fuse the second type of semantic features and the adjusted first type of semantic features to obtain media content features; wherein the media content features are the first media content features or the second media content features; The label classification module is used to classify the media content features and output a media content label category; wherein the media content label category is the first media content label category or the second media content label category.

4. The method according to claim 2 or 3, characterized in that: The feature adjustment module includes a first average pooling layer and a feature adjustment unit; the input of the first average pooling layer is the second type of semantic features, and the input of the feature adjustment unit is the first type of semantic features; Among them, the feature adjustment unit includes: a fully connected layer, an activation layer, a fusion layer and a second average pooling layer; the fully connected layer, the activation layer, the fusion layer and the second average pooling layer are connected in sequence; the output of the first average pooling layer is connected to the input of the fully connected layer; the input of the fully connected layer is jump-connected to the input of the fusion layer.

5. The method according to claim 1, characterized in that Training a second media content classification model based on any one of the first media content text and the second media content text, the first similarity, and the second similarity to obtain a target media content classification model includes: Inputting any one of the first media content text and the second media content text into the second media content classification model, and outputting a predicted media content label category; Determine a first loss function according to the predicted media content label category and the actual media content label category of the input media content text; Determine a second loss function according to the first similarity and the second similarity; A second media content classification model is trained according to the first loss function and the second loss function to obtain a target media content classification model.

6. The method according to claim 5, characterized in that Training a second media content classification model according to the first loss function and the second loss function to obtain a target media content classification model includes: The first loss function and the second loss function are combined to obtain a target loss function; A second media content classification model is trained based on the target loss function to obtain a target media content classification model.

7. The method according to claim 1, characterized in that The training method of the first media content classification model is: extracting a third media content text from the media content text set; Inputting the third media content text into the first media content classification model, and outputting a predicted media content label category corresponding to the third media content text; Determine a third loss function according to the predicted media content label category and the real media content label category corresponding to the third media content text; The first media content classification model is trained based on the third loss function.

8. The method according to claim 1, characterized in that: The second type of text is title text; the first type of text is OCR text and / or ASR text.

9. A training device for a media content classification model, characterized in that: include: A media content text extraction module, used to extract a first media content text and a second media content text from a media content text set; wherein each media content text in the media content text set includes at least one first type of text with a signal-to-noise ratio lower than a first preset threshold and at least one second type of text with a signal-to-noise ratio higher than a second preset threshold; a media content feature and media content label category acquisition module, which inputs the first media content text and the second media content text into a first media content classification model respectively, and obtains first media content features and first media content label categories corresponding to the first media content text, and second media content features and second media content label categories corresponding to the second media content text; wherein the first media content classification model is a pre-trained model; A similarity determination module, configured to determine a similarity between the first media content feature and the second media content feature and a similarity between the first media content tag category and the second media content tag category, to obtain a first similarity and a second similarity; A target media content classification model acquisition module is used to train a second media content classification model based on any one of the first media content text and the second media content text, the first similarity and the second similarity to obtain a target media content classification model; wherein the second media content classification model has the same structure as the first media content classification model.

10. An electrical device, characterized in that: The electrical equipment comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the training method for the media content classification model as described in any one of claims 1-8.

11. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the training method of the media content classification model as described in any one of claims 1 to 8 when executed by a computer processor.

Images