Content processing method, device, computer equipment and storage medium

Abstract

The present application relates to a content processing method, device, computer equipment and storage medium. The method includes: determining the current classification task in the classification task sequence; obtaining the current training category, wherein the training categories corresponding to at least two classification tasks in the classification task sequence are different; obtaining the current training sample, extracting features of the current training sample based on the feature extraction model to be trained, and obtaining the current sample extraction features; classifying based on the current sample extraction features, and obtaining the current classification result; training the feature extraction model to be trained based on the current classification result, and obtaining the trained feature extraction model, wherein the feature extraction model to be trained is obtained by training the forward classification task corresponding to the current classification task; using the feature extraction model trained according to each classification task in the classification task sequence as the content feature extraction model. The present method can improve the accuracy of feature extraction.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a content processing method, apparatus, computer device and storage medium. Background Art

[0002] With the development of computer technology, machine learning technology has emerged. Through machine learning, machine learning models for various tasks can be trained. For example, a feature extraction model for feature extraction tasks can be trained. The feature extraction model can extract features corresponding to the content, and the content can be identified based on the features to obtain content identification results. For example, a sentence can be feature extracted to obtain a feature vector representing the sentence, and the sentence can be translated based on the feature vector.

[0003] In traditional technology, feature extraction models can be trained through training samples. However, it is often the case that the features extracted by the trained feature extraction models have relatively low accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a content processing method, apparatus, computer device and storage medium to address the above technical issues.

[0005] A content processing method, the method comprising: determining a current classification task in a classification task sequence; wherein the classification task sequence comprises a plurality of classification tasks; obtaining a current training category corresponding to the current classification task, wherein the training categories corresponding to at least two classification tasks in the classification task sequence are different; obtaining a current training sample corresponding to the current training category, performing feature extraction on the current training sample based on a feature extraction model to be trained, and obtaining current sample extraction features; performing classification based on the current sample extraction features, and obtaining a current classification result; training the feature extraction model to be trained based on the current classification result, and obtaining a trained feature extraction model corresponding to the current classification task, wherein the feature extraction model to be trained is obtained by training a forward classification task corresponding to the current classification task in the classification task sequence; using the feature extraction model trained according to each classification task in the classification task sequence as a content feature extraction model, and performing feature extraction on content based on the content feature extraction model.

[0006] A content processing device, the device comprising: a task determination module, used to determine a current classification task in a classification task sequence; wherein the classification task sequence includes multiple classification tasks; a training category acquisition module, used to acquire a current training category corresponding to the current classification task, wherein at least two classification tasks in the classification task sequence correspond to different training categories; a feature extraction module, used to acquire a current training sample corresponding to the current training category, and perform feature extraction on the current training sample based on a feature extraction model to be trained to obtain current sample extraction features; a classification module, used to perform classification based on the current sample extraction features to obtain a current classification result; a training module, used to train the feature extraction model to be trained based on the current classification result to obtain a trained feature extraction model corresponding to the current classification task, wherein the feature extraction model to be trained is obtained by training a forward classification task corresponding to the current classification task in the classification task sequence; and a model acquisition module, used to use the feature extraction model trained according to each classification task in the classification task sequence as a content feature extraction model, so as to perform feature extraction on content based on the content feature extraction model.

[0007] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program: determining a current classification task in a classification task sequence; wherein the classification task sequence comprises a plurality of classification tasks; obtaining a current training category corresponding to the current classification task, wherein the training categories corresponding to at least two classification tasks in the classification task sequence are different; obtaining a current training sample corresponding to the current training category, performing feature extraction on the current training sample based on a feature extraction model to be trained, and obtaining current sample extraction features; performing classification based on the current sample extraction features, and obtaining a current classification result; training the feature extraction model to be trained based on the current classification result, and obtaining a trained feature extraction model corresponding to the current classification task, wherein the feature extraction model to be trained is obtained by training a forward classification task corresponding to the current classification task in the classification task sequence; using the feature extraction model trained according to each classification task in the classification task sequence as a content feature extraction model, and performing feature extraction on content based on the content feature extraction model.

[0008] A computer-readable storage medium stores a computer program, which implements the following steps when executed by a processor: determining a current classification task in a classification task sequence; wherein the classification task sequence includes multiple classification tasks; obtaining a current training category corresponding to the current classification task, wherein the training categories corresponding to at least two classification tasks in the classification task sequence are different; obtaining a current training sample corresponding to the current training category, performing feature extraction on the current training sample based on a feature extraction model to be trained, and obtaining current sample extraction features; performing classification based on the current sample extraction features, and obtaining a current classification result; training the feature extraction model to be trained based on the current classification result, and obtaining a trained feature extraction model corresponding to the current classification task, wherein the feature extraction model to be trained is obtained by training a forward classification task corresponding to the current classification task in the classification task sequence; using the feature extraction model trained according to each classification task in the classification task sequence as a content feature extraction model, and performing feature extraction on content based on the content feature extraction model.

[0009] In some embodiments, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, the following steps are implemented: determining a current classification task in a classification task sequence; wherein the classification task sequence includes multiple classification tasks; obtaining a current training category corresponding to the current classification task, wherein the training categories corresponding to at least two classification tasks in the classification task sequence are different; obtaining a current training sample corresponding to the current training category, and extracting features of the current training sample based on a feature extraction model to be trained to obtain current sample extraction features; classifying based on the current sample extraction features to obtain a current classification result; training the feature extraction model to be trained based on the current classification result to obtain a trained feature extraction model corresponding to the current classification task, wherein the feature extraction model to be trained is obtained by training the forward classification task corresponding to the current classification task in the classification task sequence; using the feature extraction model trained according to each classification task in the classification task sequence as a content feature extraction model, so as to extract features of the content based on the content feature extraction model.

[0010] The above-mentioned content processing method, device, computer equipment, storage medium and computer program product determine the current classification task in the classification task sequence, obtain the current training category corresponding to the current classification task, obtain the current training sample corresponding to the current training category, extract features from the current training sample based on the feature extraction model to be trained, obtain the current sample extraction feature, classify based on the current sample extraction feature to obtain the current classification result, train the feature extraction model to be trained based on the current classification result to obtain the trained feature extraction model corresponding to the current classification task. Since the trained feature extraction model is obtained by training based on the classification result, and since the feature extraction model obtained by training according to each classification task in the classification task sequence is used as the content feature extraction model, the classification task sequence includes multiple classification tasks, wherein at least two classification tasks correspond to different training categories. Therefore, the feature extraction model learns a variable classification task during the training process. Due to the change of the classification task during the training process, the content feature extraction model can learn the ability to distinguish the features of different categories of content while maintaining the feature extraction ability, thereby improving the accuracy of feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A diagram of an application environment of a content processing method in some embodiments;

[0012] Figure 2 is a flowchart of a content processing method in some embodiments;

[0013] Figure 3 is a flowchart of content processing steps in some other embodiments;

[0014] Figure 4 Schematic diagram of jointly training a feature extraction model and a classification model in some other embodiments;

[0015] Figure 5 A schematic diagram of training a content classification model under a target task in some embodiments;

[0016] Figure 6 is a schematic diagram of a retrieval process in some embodiments;

[0017] Figure 7 A schematic diagram of a process for retrieving an image in some embodiments

[0018] Figure 8 is a structural block diagram of a content processing device in some embodiments;

[0019] Fig. 9 is an internal structure diagram of a computer device in some embodiments;

[0020] Fig.10 1 is a diagram of the internal structure of a computer device in some other embodiments. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0022] The content processing method provided in the embodiment of the present application involves artificial intelligence computer perspective, machine learning and other technologies, among which:

[0023] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0024] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0025] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to using cameras and computers to replace human eyes to identify, follow and measure targets, and further process graphics so that computer processing becomes an image that is more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, automatic driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0026] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0027] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0028] The computer vision, machine learning and other technologies involved in this application are specifically described through the following embodiments:

[0029] The content processing method provided by this application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104. The network can be a wired network or a wireless network, and the wireless network can be any one of a local area network, a metropolitan area network and a wide area network. The terminal 102 can be one of a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart TV or a wearable device, but is not limited to this. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0030] Specifically, the server determines the current classification task in the classification task sequence, obtains the current training category corresponding to the current classification task, obtains the current training sample corresponding to the current training category, extracts features from the current training sample based on the feature extraction model to be trained, obtains the current sample extracted features, classifies based on the current sample extracted features, obtains the current classification result, trains the feature extraction model to be trained based on the current classification result, obtains the trained feature extraction model corresponding to the current classification task, uses the feature extraction model trained according to each classification task in the classification task sequence as the content feature extraction model, and extracts features from the content based on the content feature extraction model, wherein at least two classification tasks in the classification task sequence correspond to different training categories, and the feature extraction model to be trained is obtained by training based on the forward classification task corresponding to the current classification task in the classification task sequence.

[0031] In some embodiments, after obtaining the trained content feature extraction model, the server 104 can deploy the content feature extraction model. Thereafter, the server 104 can receive the content retrieval request sent by the terminal 102, extract features of the content carried in the content retrieval request through the deployed content feature extraction model, classify based on the extracted features, determine the feature distance threshold corresponding to the classification result from the classification threshold table based on the classification result, and then retrieve from the content database based on the feature distance threshold, recall the content less than the distance threshold as the retrieval result and return it to the terminal. Among them, the content retrieval request of the terminal 102 can be generated by the user inputting the retrieval content in the terminal interface and triggering the retrieval control. In the content database, each content and its corresponding feature vector are stored correspondingly.

[0032] It can be understood that the above application scenario is only an example and does not constitute a limitation on the user data processing method provided by the embodiment of the present application. The method provided by the embodiment of the present application can also be applied in other application scenarios. For example, the terminal can pre-generate a classification task sequence, send a model training request to the server and carry the classification task sequence, and the server determines the current classification task in the classification task sequence from the classification task sequence to perform model training based on the current classification task to obtain the trained feature extraction model corresponding to the current classification task. After all classification tasks in the classification task sequence are executed, the feature extraction model trained according to each classification task in the classification task sequence is returned to the terminal. The terminal uses the feature extraction model trained according to each classification task in the classification task sequence as a content feature extraction model to extract features from the content based on the content feature extraction model, thereby classifying the content according to the extracted features to implement a business process based on the classification results. For example, the terminal can classify the content that the user has browsed historically to recommend the same type of content to the user.

[0033] In some embodiments, Figure 2 As shown, a content processing method is provided. It can be understood that the content processing method can be executed by a terminal or a server alone, or by a terminal and a server in collaboration. In this embodiment, the method is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0034] Step 202, determining a current classification task in a classification task sequence; wherein the classification task sequence includes a plurality of classification tasks.

[0035] Among them, the classification task refers to the task of identifying the category of content, and the classification result obtained based on the classification task can be used to train the feature extraction model to be trained. The content refers to data containing semantic information, and the content can be any one of text, audio or image. The classification task can be generated by determining the category to be identified. The classification task sequence refers to the sequence composed of all classification tasks in one round (epoch) of training. The classification task sequence includes multiple classification tasks. Multiple classification tasks can be arranged to obtain a classification task sequence. Multiple refers to at least two. In some embodiments, in order to ensure the accuracy of model training, multiple rounds of training can be performed during the entire training process, and each round of training corresponds to a classification task sequence. The current classification task refers to the classification task that needs to be performed at the current moment in the classification task sequence.

[0036] In some embodiments, the classification task sequence can be pre-generated, that is, multiple classification tasks are pre-determined, and these classification tasks are arranged and stored as a classification task sequence. Specifically, the server can obtain a training category set including multiple training categories, and pre-divide the training categories in the training category set into multiple training category subsets, each training category subset corresponds to a classification task, and these classification tasks are numbered and a corresponding relationship is established with the training category subset, wherein the training category refers to the category to be identified in the process of training based on the classification task. The server can arrange the classification tasks in the order of numbers from small to large to obtain a classification task sequence, for example: task 1-task 2-task 3-...-task N. In some embodiments, the server can execute the classification tasks in the classification task sequence in sequence, and after the training based on the current classification task is completed, the classification task corresponding to the next number in the classification task sequence is determined as the current classification task. In some other embodiments, the server can randomly extract a classification task from the classification task sequence without replacement as the current classification task, that is, after the server extracts a classification task, the extracted classification task is removed from the classification task sequence.

[0037] In some other embodiments, the classification task sequence can be generated while training, that is, after each training based on a classification task is completed, the next classification task is determined. Specifically, after the training based on the current classification task is completed, the server can extract one or more training categories from the training category set to generate a new classification task, and determine the generated classification task as the current classification task. After the current round of training is completed, all classification tasks are arranged in chronological order to obtain a classification task sequence. Among them, when the server extracts the training category to generate the classification task, it can be extracted with replacement or without replacement.

[0038] Step 204 , obtaining a current training category corresponding to the current classification task, wherein the training categories corresponding to at least two classification tasks in the classification task sequence are different.

[0039] Among them, the current training category corresponding to the current classification task refers to the category to be identified in the process of training based on the current classification task. The current training category corresponding to the current classification task can be one training category or multiple training categories, and multiple refers to at least two. In some embodiments, the different training categories corresponding to at least two classification tasks in the classification task sequence can be that the training categories corresponding to at least two classification tasks are completely different. For example, assuming that there are two classification tasks in the classification task sequence, namely task A and task B, then for each training category in the multiple training categories corresponding to task A, there is no training category that is the same as the training category in the multiple training categories corresponding to task B. In other embodiments, the different training categories corresponding to at least two classification tasks in the classification task sequence can also be that the training categories corresponding to at least two classification tasks in the classification task sequence are partially different. For example, assuming that there are two classification tasks in the classification task sequence, namely task A and task B, then there is at least one training category in the multiple training categories corresponding to task A that is different from any of the multiple training categories corresponding to task B.

[0040] In some embodiments, the classification task sequence is pre-generated, and there is a correspondence between the classification tasks in the classification task sequence and the training category subsets. After determining the current classification task, the server can obtain the training category subset corresponding to the current classification task according to the correspondence, and use the training category in the obtained training category subset as the current training category.

[0041] In some other embodiments, the classification task sequence is generated during training, and the server extracts training categories from the training category set to generate the current classification task. When the server determines the current classification task, it obtains the current training category corresponding to the current classification task.

[0042] Step 206, obtaining a current training sample corresponding to the current training category, performing feature extraction on the current training sample based on the feature extraction model to be trained, and obtaining a current sample extraction feature.

[0043] Among them, the current training sample corresponding to the current training category refers to the training sample whose category label is the current training category. The training sample is a content sample, and the content can be any one of text, audio or image. The feature extraction model is a machine learning model used to extract features and output feature vectors. The feature extraction model includes at least an embedding model. The embedding model is used to output feature vectors. The feature vector output by the embedding model can be called embedding. The embedding model can be a model composed of one or more fully connected layers. The feature extraction model to be trained refers to the feature extraction model to be trained to perform the current classification task.

[0044] In some embodiments, the feature extraction model may only include an embedding model, and the input end of the embedding model may be connected to the output end of the trained basic neural network model to receive the output of the basic neural network model as input. The basic neural network model may be a model for extracting feature information contained in the content, and the basic neural network model may be a neural network based on artificial intelligence, such as a convolutional neural network (CNN), or a network such as ResNet101 (deep residual network 101) or ResNet18 (deep residual network 18). The convolutional neural network is a type of feedforward neural network (Feedforward Neural Networks) that includes convolution calculations and has a deep structure. Specifically, after the server obtains the current training sample corresponding to the current training category, the current training sample is input into the trained basic neural network model, feature information is extracted through the trained basic neural network model, and the extracted feature information is input into the embedding model to be trained to obtain a feature vector, which is the current sample extraction feature corresponding to the current training sample. For example, taking the current training sample as an image, the image can be input into the trained CNN model to extract the deep features of the image, and the extracted deep features can be input into the embedding model to obtain the feature vector of the image.

[0045] In some other embodiments, the feature extraction model may include a basic neural network model and an embedding model, in which case the basic neural network model and the embedding model are trained together as feature extraction models. Specifically, after obtaining the current training sample corresponding to the current training category, the server inputs the current training sample into the basic neural network model to be trained, extracts feature information through the basic neural network model to be trained, and inputs the extracted feature information into the embedding model to be trained to obtain a feature vector, which is the current sample extraction feature corresponding to the current training sample.

[0046] Step 208: extract features based on the current sample and perform classification to obtain the current classification result.

[0047] Classification refers to classifying the current training sample based on the features extracted from the current sample, so as to determine the specific category to which the current training sample belongs from the current training category corresponding to the current classification task. The current classification result is used to characterize the category information of the specific category to which the current training sample belongs.

[0048] Specifically, the server can use the classification model to classify the features extracted from the current sample, that is, input the features extracted from the current sample into the classification model, and obtain the output of the classification model as the current classification result. Among them, the classification model refers to a machine learning model that can perform category recognition. In some embodiments, the classification model may include one or more fully connected layers. When the classification model includes multiple fully connected layers, the first few fully connected layers can extract semantic features from the features extracted from the current sample, so it can be called a semantic layer Semantic_layer, and the last fully connected layer is used for classification, which is called a classification layer. The number of semantic layers can be determined as needed, and the semantic layer can be deepened for situations where more feature intersections are required. By setting the semantic layer, high-order features can be abstracted to more fully mine classification information. In some embodiments, the current classification result can be a category identifier for characterizing the category to which the current training sample belongs. For example, if the classification model classifies N categories, the first category can be represented by an N-dimensional vector (1, 0, 0, 0, ..., 0), and the second category can be represented by a vector (0, 1, 0, 0, ..., 0), and so on. In other embodiments, the current classification result may be a probability for characterizing the category to which the current training sample belongs. For example, the classification model may be used to classify N categories. Then the classification model may output an N-dimensional probability vector, wherein the probability of each dimension is used to characterize the possibility that the current training sample belongs to the category.

[0049] In some other embodiments, the server performs classification based on the features extracted from the current sample, which may be to calculate the feature distance between the features extracted from the current sample and the category features corresponding to each current training category. When the feature distance between the features extracted from the current sample and the category features is less than the distance threshold, the current training category corresponding to the category features is determined as the classification category to which the current training sample belongs. At this time, the current classification result may be the category identifier of the classification category. The category features corresponding to the current training category may be determined in the following manner: collect multiple content samples corresponding to the training category, extract the feature vectors of these content samples through the embedding model, and then calculate an average vector as the category feature corresponding to the training category.

[0050] Step 210, training the feature extraction model to be trained based on the current classification result to obtain a trained feature extraction model corresponding to the current classification task, wherein the feature extraction model to be trained is obtained by training the forward classification task corresponding to the current classification task in the classification task sequence.

[0051] Among them, the feature extraction model to be trained is obtained by training based on the forward classification task corresponding to the current classification task in the classification task sequence. The forward classification task corresponding to the current classification task refers to all classification tasks performed before the current classification task in this round of training. For example, when the classification tasks in the classification task sequence are performed in sequence, the forward classification task corresponding to the current classification task refers to all classification tasks sorted before the current classification task. Since the feature extraction model to be trained is obtained by training based on the forward classification task corresponding to the current classification task in the classification task sequence, the features extracted by the feature extraction model to be trained can learn the training categories corresponding to the forward classification tasks, and can well support the classification tasks of the training categories corresponding to the forward classification tasks.

[0052] Specifically, the current training sample is a content sample marked with a category label. After obtaining the current classification result corresponding to the current training sample, the server can determine the target model loss value based on the difference between the current classification result and the category label, adjust the model parameters of the feature extraction model to be trained based on the target model loss value and continue training until the training stop condition is met, and obtain the trained feature extraction model corresponding to the current classification task. The training stop condition can be any one of the loss value reaching the minimum value, the training time reaching the preset time, or the number of training times reaching the preset number of times. Model parameters refer to variable parameters inside the model, and for neural network models, they can also be called neural network weights (weight). It can be understood that for the first classification task executed, the feature extraction model to be trained can be an initialized feature extraction model, or a pre-trained feature extraction model. The trained feature extraction model corresponding to the current classification task has learned the current training category corresponding to the previous classification task, and can support the classification task corresponding to the current training category.

[0053] In some embodiments, the current classification result is obtained based on the classification model, and the server determines the target model loss value based on the difference between the current classification result and the category label, which can be: determine the difference between the current classification result and the category label based on the predicted probability corresponding to the category label in the current classification result, and then determine the target model loss value. In some other embodiments, the current classification result is obtained based on the feature distance between the current sample extraction feature and the category features corresponding to each current training category, and the server determines the target model loss value based on the difference between the current classification result and the category label, which can be: obtain the feature distance between the category feature of the category corresponding to the current classification result and the category feature corresponding to the category label, use the feature distance as the difference between the current classification result and the category label, and then determine the target model loss value.

[0054] Step 212: Using the feature extraction model obtained by training according to each classification task in the classification task sequence as a content feature extraction model, so as to extract features from the content based on the content feature extraction model.

[0055] Specifically, for each classification task in the classification task sequence, the server executes the above steps 204-206 to obtain the trained feature extraction model corresponding to the classification task. After each classification task is executed and the training based on the classification task is completed, the trained feature extraction model corresponding to the classification task is saved, and when executing the next classification task, the currently saved feature extraction model is used as the feature extraction model to be trained. This cycle is repeated until all classification tasks are completed, and the feature extraction model finally obtained is used as the content feature extraction model to extract features from the content based on the content feature extraction model. Since in the process of cyclic execution, the feature extraction model to be trained corresponding to each classification task is obtained by training based on the forward classification task corresponding to the current classification task in the classification task sequence, the classification categories learned by the feature extraction model to be trained are always accumulated, and the content feature extraction model obtained after all classification tasks are completed has learned the training categories corresponding to each classification task in the classification task sequence. Among them, when executing each classification task in the classification task sequence, it can be executed in sequence according to the order in the classification task sequence, or it can be executed by random sampling.

[0056] In the above-mentioned content processing method, by determining the current classification task in the classification task sequence, obtaining the current training category corresponding to the current classification task, obtaining the current training sample corresponding to the current training category, performing feature extraction on the current training sample based on the feature extraction model to be trained, obtaining the current sample extraction feature, classifying based on the current sample extraction feature, obtaining the current classification result, training the feature extraction model to be trained based on the current classification result, and obtaining the trained feature extraction model corresponding to the current classification task. Since the trained feature extraction model is obtained based on the classification result, and since the feature extraction model obtained by training according to each classification task in the classification task sequence is used as the content feature extraction model, the classification task sequence includes multiple classification tasks, wherein at least two classification tasks correspond to different training categories. Therefore, the feature extraction model learns a changing classification task during the training process. Due to the change of the classification task during the training process, the content feature extraction model can learn the ability to distinguish the features of different categories of content while maintaining the feature extraction capability, thereby improving the accuracy of feature extraction.

[0057] In some embodiments, obtaining the current training category corresponding to the current classification task includes: obtaining the current category set corresponding to the current classification task; wherein the current category set is obtained by removing the forward training category from the training category set, and the forward training category is the training category corresponding to the forward classification task of the current classification task in the classification task sequence; extracting categories from the current category set, and using the extracted categories as the current training categories corresponding to the current classification task.

[0058] The training category set refers to the set of all categories to be recognized in this round of training. Removal means to remove, that is, to discard. For example, assuming that the original training categories of the training category set are A, B, C, and D, and the forward training category is A, then after removing the forward training category, the categories included in the current category set are B, C, and D.

[0059] Specifically, in this embodiment, the classification task sequence is generated while training, that is, after each training based on a classification task is completed, multiple training categories are extracted to generate new classification tasks. After a round of training is completed, all classification tasks form a classification task sequence. For each classification task in the classification task sequence, after the server extracts the training category used to generate the classification task, it will remove the extracted classification task from the candidate category set to achieve extraction without replacement. As the number of classification tasks gradually increases, the number of training categories in the candidate category set gradually decreases until all training categories in the candidate category set are extracted, and this round of training ends.

[0060] In some embodiments, when the number of training categories in the candidate category set is less than the number of training categories required for the classification task at the last extraction, it can be supplemented by randomly extracting from the categories that have been extracted. For example, assuming that the number of training categories for each classification task is fixed to 10 categories, if the number of training categories in the candidate category set is 6 at the last extraction, 4 training categories are randomly extracted from the categories that have been extracted to make up for 10 categories.

[0061] In some other embodiments, when the number of training categories in the candidate category set at the last extraction is less than the number of training categories required for the classification task, the insufficient categories can be set as invalid categories. For example, assuming that the number of training categories for each classification task is fixed to 10 categories, if the number of training categories in the candidate category set at the last extraction is 6, the 7th to 10th categories can be set as invalid categories. Invalid categories refer to categories that do not require training samples to be prepared.

[0062] In the above embodiment, by obtaining the current category set corresponding to the current classification task, categories are extracted from the current category set, and the extracted categories are used as the current training categories corresponding to the current classification task. After the extraction, the extracted categories are removed from the current category set, and the remaining categories are used as the current category set for the next classification task, thereby achieving category extraction without replacement, ensuring that all training categories in the candidate category set can be learned, so that the obtained feature extraction model can support the classification of as many categories as possible.

[0063] In some embodiments, the above method also includes: classifying the unextracted categories in the current category set into the same reference category, the reference category is the shielded category corresponding to the current classification task; obtaining a reference training sample corresponding to the reference category, extracting features from the reference training sample based on the feature extraction model to be trained, and obtaining reference sample extracted features; classifying based on the reference sample extracted features to obtain a reference classification result; training the feature extraction model to be trained based on the current classification result to obtain a trained feature extraction model corresponding to the current classification task, including: training the feature extraction model to be trained based on the current classification result and the reference classification result to obtain a trained feature extraction model corresponding to the current classification task.

[0064] The shielded classification category refers to the category that cannot be identified by the current classification task. For example, if the current classification task is to identify which specific category of category A, category B, and category C the content belongs to, then category D is a category that cannot be identified by the current classification task. The reference training sample can be a training sample of any category under the reference category. For example, assuming that the reference category includes category D, category E, and category F, the reference training sample can be any one of the training samples labeled as category D, the training samples labeled as category E, or the training samples labeled as category F.

[0065] Specifically, after extracting the training categories used to generate the current classification task, the server divides the remaining categories in the current category set into the same reference category, where the reference category is the shielded classification category corresponding to the current classification task, and obtains the training samples corresponding to any category under the reference category as reference training samples. The reference training samples are subjected to feature extraction based on the feature extraction model to be trained to obtain reference sample extracted features, and then classified based on the reference sample extracted features to obtain reference classification results. The server can then train the feature extraction model to be trained based on the current classification results and the reference classification results to obtain the trained feature extraction model corresponding to the current classification task.

[0066] In some embodiments, the current classification result includes the current category probability that the current training sample is the current training category, and the reference classification result includes the shielding category probability corresponding to the reference training sample being the shielding classification category; the feature extraction model to be trained is trained based on the current classification result and the reference classification result, and the trained feature extraction model corresponding to the current classification task is obtained, including: obtaining a first model loss value based on the current category possibility, and the first model loss value is positively correlated with the current category possibility; obtaining a second model loss value based on the shielding category possibility, and the second model loss value is positively correlated with the shielding category possibility; obtaining a target model loss value based on the first model loss value and the second model loss value; and adjusting the parameters of the feature extraction model to be trained based on the target model loss value to obtain the trained feature extraction model corresponding to the current classification task.

[0067] Among them, the possibility is used to characterize the size of the possibility and can be expressed as a probability. The current category possibility is used to characterize the possibility that the current training sample is the current training category. The shielding category possibility is used to characterize the possibility that the reference training sample is the shielding classification category. A positive correlation means that when other conditions remain unchanged, the two variables change in the same direction. When one variable changes from large to small, the other variable also changes from large to small. It can be understood that the positive correlation here means that the direction of change is consistent, but it does not require that when one variable changes a little, the other variable must also change. For example, when variable a is 10 to 20, variable b can be set to 100, and when variable a is 20 to 30, variable b is 120. In this way, the direction of change of a and b is that when a increases, b also increases. But when a is in the range of 10 to 20, b can be unchanged.

[0068] Specifically, the current training sample is a training sample under the current training category, then the current training sample is a positive sample of the previous training category, and its sample label is 1. The server obtains the first model loss value based on the difference between the current category possibility and 1. The shielded training sample is a training sample under the shielded training category, then the shielded training sample is a positive sample of the shielded training category, and its sample label is also 1. The server can obtain the second model loss value based on the difference between the shielded category possibility and the sample label corresponding to the reference training sample, add the first model loss value to the second model loss value to obtain the target model loss value, and then adjust the parameters of the feature extraction model to be trained based on the target model loss value to obtain the trained feature extraction model corresponding to the current classification task.

[0069] In the above embodiment, by classifying the categories that have not been extracted in the current category set into the same reference category, the server can train the feature extraction model to be trained based on the reference training samples corresponding to the reference category during the training process. Since the sample features of the masked classification category are taken into consideration, the reliability of the trained feature extraction model can be guaranteed.

[0070] In some embodiments, obtaining the current training category corresponding to the current classification task includes: obtaining multiple candidate category sets; removing the forward candidate category set from the multiple candidate category sets to obtain the remaining candidate category set; the forward candidate category set is the candidate category set corresponding to the forward classification task of the current classification task in the classification task sequence; extracting the candidate category set from the remaining candidate category set, and using the candidate category in the extracted candidate category set as the current training category corresponding to the current classification task.

[0071] The candidate category set includes multiple categories, where multiple refers to at least two. A candidate category set can be used to generate a classification task, and the generated classification task can identify all categories in the candidate category set.

[0072] Specifically, in this embodiment, the classification task sequence is generated while training, that is, after each training based on a classification task is completed, a candidate category set is extracted to generate a classification task. After a round of training is completed, all classification tasks form a classification task sequence. For each classification task in the classification task sequence, after the server extracts the candidate category set used to generate the classification task, it will remove the extracted candidate category set from multiple candidate category sets to achieve extraction without replacement. As the number of classification tasks gradually increases, the number of candidate category sets gradually decreases until all candidate category sets are extracted and this round of training ends.

[0073] In the above embodiment, when generating a training task, a candidate category set may be extracted to generate the training task, which may improve the extraction efficiency and thus the training efficiency.

[0074] In some embodiments, obtaining multiple candidate category sets includes: obtaining a training category set for classification training; obtaining the number of tasks in a classification task sequence, and dividing the training category set based on the number of tasks to obtain multiple candidate category sets.

[0075] Among them, the number of tasks refers to the number of classification tasks. The number of classification tasks is determined in advance, and the server can divide the training category set based on the number of tasks to obtain multiple candidate category sets. In some embodiments, the server can divide the training category set into equal numbers based on the number of classification tasks to obtain multiple candidate category sets. For example, the number of classification tasks is 10, and the training category set includes 1,000 training categories, then the training category set can be divided into 10 candidate category sets, and each candidate category set includes 10 training categories. In some other embodiments, the server can randomly divide the training category set based on the number of classification tasks to obtain multiple candidate category sets, and the training categories and numbers in each candidate category set are random, and the number of training categories in each candidate category set can be the same or different.

[0076] In the above embodiment, by obtaining the number of tasks in the classification task sequence, the training category set is divided based on the number of tasks to obtain multiple candidate category sets, ensuring that all classification tasks in the training category set can be learned during the training process, thereby improving the generalization performance of the feature extraction model.

[0077] In some embodiments, a feature extraction model to be trained is trained based on the current classification result to obtain a trained feature extraction model corresponding to the current classification task, including: obtaining a comparison sample extraction feature corresponding to a comparison training sample of the current training sample, and obtaining a feature extraction loss value based on the difference between the current sample extraction feature and the comparison sample extraction feature; obtaining a classification loss value based on the current classification result; obtaining a target model loss value based on the feature extraction loss value and the classification loss value; and adjusting parameters of the feature extraction model to be trained based on the target model loss value to obtain a trained feature extraction model corresponding to the current classification task.

[0078] The comparison training sample refers to a content sample used for comparison with the current training sample to determine the feature extraction loss value. The comparison training sample may include at least one of a positive comparison training sample or a negative comparison training sample, wherein the positive comparison training sample refers to a content sample similar to the current training sample, and the negative comparison training sample refers to a content sample dissimilar to the current training sample.

[0079] Specifically, the server can perform feature extraction on the comparison training sample based on the feature extraction model to be trained, obtain the sample extraction feature corresponding to the comparison training sample as the comparison sample extraction feature, obtain the feature extraction loss value based on the difference between the current sample extraction feature and the comparison sample extraction feature, obtain the classification loss value based on the difference between the current classification result and the category label of the current training sample, and calculate the target model loss value based on the feature extraction loss value and the classification loss value and refer to the following formula (1), where L totalis the target model loss value, L em is the feature extraction loss value, L class is the classification loss value, w1 and w2 are the loss value weights respectively.

[0080] L total =w1L em +w2L class Formula (1)

[0081] In some embodiments, the server obtains the feature extraction loss value based on the difference between the current sample extraction feature and the comparison sample extraction feature by: calculating the cosine similarity between the comparison sample extraction feature and the current sample extraction feature, using the obtained cosine similarity to characterize the difference between the current sample extraction feature and the comparison sample extraction feature, so as to calculate the difference between the cosine similarity and the training label to obtain the feature extraction loss value. Wherein, when the comparison sample extraction feature is a positive comparison training sample, the training label is 1, and when the comparison sample extraction feature is a negative comparison training sample, the training label is 0.

[0082] In some embodiments, the server obtains the feature extraction loss value based on the difference between the current sample extracted features and the comparison sample extracted features by: classifying the current sample extracted features and the comparison sample extracted features by similarity, dividing them into two categories, similar and dissimilar, obtaining the probability of similarity and the probability of dissimilarity, taking the larger probability value as the classification result, and using the probability value of the classification result to characterize the difference between the current sample extracted features and the comparison sample extracted features, and then calculating the difference between the probability of the classification result and the training label to obtain the feature extraction loss value. Wherein, when the comparison sample extracted features are positive comparison training samples, the training label is 1, and when the comparison sample extracted features are negative comparison training samples, the training label is 0.

[0083] In some embodiments, the server obtains the feature extraction loss value based on the difference between the current sample extraction feature and the comparison sample extraction feature by: calculating the feature distance between the current sample extraction feature and the comparison sample extraction feature, using the feature distance to characterize the difference between the current sample extraction feature and the comparison sample extraction feature, and using the feature distance as the feature extraction loss value, wherein the feature distance can be, for example, the Euclidean distance or the L2 distance.

[0084] In the above embodiment, when the server trains the feature extraction model to be trained, it adjusts the parameters by combining the classification loss value obtained by the classification result and the feature extraction loss value obtained based on the difference between the current sample extracted features and the comparison sample extracted features, so that the features extracted by the trained feature extraction model are more accurate.

[0085] In some embodiments, the comparison sample extraction features include positive sample extraction features corresponding to positive comparison training samples and negative sample extraction features corresponding to negative comparison training samples; obtaining the feature extraction loss value based on the difference between the current sample extraction features and the comparison sample extraction features includes: obtaining the positive feature difference value, the positive feature difference value is the feature difference value between the current sample extraction features and the positive sample extraction features; obtaining the negative feature difference value, the negative feature difference value is the feature difference value between the current sample extraction features and the negative sample extraction features; determining the feature extraction loss value based on the positive feature difference value and the negative feature difference value.

[0086] Specifically, the server extracts features from the positive comparison training samples based on the feature extraction model to be trained to obtain positive sample extraction features corresponding to the positive comparison training samples, extracts features from the negative comparison training samples based on the feature extraction model to be trained to obtain negative sample extraction features corresponding to the negative comparison training samples, obtains the feature difference value between the current sample extraction feature and the positive sample extraction feature to obtain the positive feature difference value, obtains the feature difference value between the current sample extraction feature and the negative sample extraction feature to obtain the negative feature difference value, and finally determines the feature extraction loss value based on the positive feature difference value and the negative feature difference value.

[0087] In some embodiments, the server may determine the feature extraction loss value by referring to the following formula (2), where x a Extract features for the current sample, x p Extract features for positive samples, x n is a negative feature difference value, ||x a -x p || represents x a and x p The L2 distance between them, that is, the positive feature difference value, ||x a -x n || represents x a and x n The L2 distance between them, that is, the negative feature difference value, the purpose of formula (2) is to make the distance between the current training sample and the negative comparison training sample larger than the distance between the current training sample and the positive comparison training sample by α, where α is the margin (interval term), and the value of α can be set as needed, for example, α can be 4.

[0088] L tri =max(||x a -x p ||-||x a -x n ||+α,0) Formula (2)

[0089] It can be seen from formula (2) that the feature extraction loss value is 0 only when the distance between the current training sample and the negative comparison training sample is greater than the distance between the current training sample and the positive comparison training sample by α, otherwise, the feature extraction loss value is greater than 0. Therefore, in the process of reducing the loss value, the distance between the current training sample and the negative comparison training sample develops in the direction of being greater than the distance between the current training sample and the positive comparison training sample by α, so that the features extracted by the feature extraction model can better focus on maintaining the classification features.

[0090] In some embodiments, negative comparison training samples can be obtained by the following steps: feature extraction of other training samples in the sample set to which the current training sample belongs, obtaining comparison sample extraction features, calculating the feature distance between the current sample extraction features corresponding to the current training sample and each comparison sample extraction feature, and sorting the other training samples in the sample set from small to large according to the feature distance, so as to select training samples with relatively small feature distances as negative comparison training samples. The sample set to which the current training sample belongs can be pre-deduplicated, and by selecting training samples with relatively small feature distances as negative comparison training samples, difficult samples (Hard Sample) can be selected to alleviate the problem of sample imbalance, so that the model obtained by training is more accurate. Among them, difficult samples refer to these negative comparison training samples that belong to different categories from the current training sample but have small feature distances, which are difficult for classification. Among them, deduplication operation refers to retaining one of the similar one or more training samples and removing the others.

[0091] In the above embodiment, since the feature extraction loss value is determined based on the positive feature difference value and the negative feature difference value, the feature extraction model takes into account the impact of the small inter-class feature distance on classification when performing similarity metric learning, thereby improving the accuracy of the feature extraction model when used for classification.

[0092] In some embodiments, Figure 3 As shown, a content processing method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0093] Step 302, determining a current classification task in a classification task sequence; wherein the classification task sequence includes a plurality of classification tasks.

[0094] Step 304: obtaining a current training category corresponding to the current classification task, wherein the training categories corresponding to at least two classification tasks in the classification task sequence are different.

[0095] Step 306, obtaining a current training sample corresponding to the current training category, performing feature extraction on the current training sample based on the feature extraction model to be trained, and obtaining a current sample extraction feature.

[0096] Step 308, obtaining a current classification model corresponding to the current classification task, classifying the sample extraction features based on the current classification model, and obtaining a classification result.

[0097] The current classification model corresponding to the current classification task may be an initialized classification model or a pre-trained classification model. Specifically, the server inputs the sample extraction features into the current classification model to obtain a classification result.

[0098] Step 310 , based on the current classification result, jointly adjust the model parameters of the feature extraction model to be trained and the model parameters of the current classification model to obtain an adjusted feature extraction model and a content classification model with adjusted parameters.

[0099] Specifically, the server determines the target model loss value based on the classification result, and then back-propagates the loss value, jointly adjusts the model parameters of the feature extraction model to be trained and the model parameters of the current classification model, and obtains the adjusted feature extraction model and the adjusted current classification model. In some embodiments, the model parameters of the feature extraction model to be trained and the model parameters of the current classification model can be adjusted using a stochastic gradient descent algorithm, an Adagrad (Adaptive Gradient) algorithm, an Adadelta (an improvement of the AdaGrad algorithm), an RMSprop (an improvement of the AdaGrad algorithm), an Adam (Adaptive Moment Estimation) algorithm, etc.

[0100] Step 312 , determining whether the current classification task satisfies the task stopping condition, if so, proceeding to step 314 , if not, proceeding to step 306 .

[0101] The task stop condition refers to the training stop condition for training based on the classification task, and the training stop condition can be any one of the following: the loss value reaches the minimum value, the training duration reaches the preset duration, or the number of training times reaches the preset number of times. Whether the current classification task satisfies the task stop condition refers to whether the training based on the current classification task satisfies the training stop condition.

[0102] Step 314: Use the adjusted feature extraction model as the trained feature extraction model corresponding to the current classification task.

[0103] Step 316: Use the feature extraction model trained according to each classification task in the classification task sequence as a content feature extraction model to extract features from the content based on the content feature extraction model.

[0104] In some embodiments, Figure 4 As shown in Figure 1, it is a schematic diagram of jointly training feature extraction model and classification model. Figure 4 , the server obtains the triplet sample corresponding to the current classification task, the triplet sample includes the current training sample and the positive contrast training sample and negative contrast training sample corresponding to the current training sample, inputs each sample in the triplet sample into the convolutional neural network to extract basic feature information, then inputs the extracted basic feature information into the embedding model, extracts features from each sample in the triplet sample based on the embedding model to obtain the feature vector of each sample, then calculates the feature extraction loss value by referring to the above formula (2), inputs the feature vector of any sample in the triplet sample into the classification model, further extracts semantic features through the semantic layer in the classification model, and then classifies through the classification layer (Fc_class), then obtains the classification loss value based on the difference between the classification result and the category label, obtains the target model loss value by weighted summation of the feature extraction loss value and the classification loss value, and jointly trains the convolutional neural network, the embedding model and the classification model based on the target model loss value to obtain the trained embedding model corresponding to the current classification task. In the process of training based on the classification task, the classification model learns the changing classification task, that is, it learns a different classification task each time. In some embodiments, for a sample labeled with a category label, the server may calculate the classification loss value according to the following formula (3), where t[i] is the true value label vector and o[i] is the predicted probability value of each training category by the classification model:

[0105]

[0106] In the above formula (3), when the true value of a label bit is 1, the left side of the plus sign takes effect, and when the true value is 0, the right side takes effect, so the supervision information of the sample under all labels can be learned.

[0107] In the above embodiment, by simultaneously learning the joint feature extraction and classification tasks, merging the feature extraction and classification tasks, learning the feature extraction model and the classification model in the same model structure, the weight control can be updated by gradient to avoid the classification affecting the feature expression of the feature extraction model.

[0108] In some embodiments, the above method also includes: obtaining a target classification category corresponding to the target classification task; obtaining a target classification model corresponding to the target classification category; splicing the content feature extraction model with the target classification model to obtain a content classification model; classifying the target training samples corresponding to the target classification category based on the content classification model to obtain a target classification result; while keeping the model parameters of the content feature extraction model unchanged, adjusting the parameters of the target classification model in the content classification model according to the target classification result to obtain an adjusted content classification model.

[0109] The target classification task refers to the classification task that needs to be performed in the actual business scenario. The target classification category corresponding to the target classification task refers to the category to be identified in the process of performing the target classification task. The target classification category corresponding to the target classification task can be the same as the category involved in the classification task sequence during the training process, or it can be different from the category involved in the classification task sequence during the training process. The target classification model corresponding to the target classification category can be an initialized classification model or a pre-trained classification model.

[0110] Specifically, the server concatenates the content feature extraction model with the target classification model to obtain the content classification model. The input of the content feature extraction model is used as the input of the target classification model, and the target classification model outputs the classification result. In order to enable the target classification task to accurately perform category recognition, the content classification model needs to be trained first. Since the content feature extraction model in the content classification model is a model trained according to the classification task sequence, it learns the variable classification tasks during the training process and can well support the target classification tasks of any category combination. Therefore, in the process of training the content classification model, the model parameters of the content feature extraction model can be fixed, and only the model parameters of the target classification model can be adjusted. Specifically, the server inputs the target training sample corresponding to the target classification category into the content classification model, extracts the feature vector through the content feature extraction model of the content classification model, inputs the obtained feature vector into the target classification model of the content classification model, obtains the target classification result, obtains the target model loss value based on the difference between the target classification result and the category label of the target training sample, adjusts the parameters of the target classification model according to the target model loss value while keeping the model parameters of the content feature extraction model unchanged, repeats the step of inputting the target training sample corresponding to the target classification category into the content classification model, until the training stop condition is met, and obtains the adjusted content classification model. The adjusted content classification model can be applied to actual business scenarios to classify content.

[0111] In some embodiments, Figure 5The figure shows a schematic diagram of training a content classification model under a target task. The server obtains training samples corresponding to the target classification task, which include content samples of each target classification category corresponding to the target classification task, inputs the training samples into CNN, extracts basic feature information, and then inputs the extracted basic feature information into the embedding model. The feature vector is extracted from the training samples based on the embedding model, and the feature vector is input into the classification model. The semantic features are further extracted through the semantic layer in the classification model, and then the classification is performed through the classification layer (Fc_class). Then, the classification loss value is obtained based on the difference between the classification result and the category label, and the classification loss value is used as the target model loss value. When the model parameters of the CNN and the embedding model are fixed, the semantic layer and the classification layer of the classification model are trained based on the target model loss value to obtain the trained content classification model.

[0112] In the above embodiment, the content classification model can be trained by only learning the classification task without updating the model parameters of the feature extraction model, so that the feature extraction model can quickly learn new semantic categories to meet the changes in semantic level requirements.

[0113] In some embodiments, the above method also includes: obtaining search content, performing feature extraction on the search content based on a content feature extraction model to obtain search features corresponding to the search content; performing classification based on the search features to obtain search classification results, and obtaining a difference threshold corresponding to the classification category of the search classification results; calculating the feature difference between the content extraction features and the search features of each target content in the target content set; selecting target content whose feature difference is less than the difference threshold from the target content set, and obtaining a search result corresponding to the search content based on the selected target content.

[0114] Among them, the search content refers to the content that needs to be searched for similar content. The search feature corresponding to the search content refers to the feature vector corresponding to the search content. The classification category of the classification result refers to the classification category represented by the classification result. The difference threshold corresponding to the classification category refers to the feature distance threshold based on which the content is screened when searching for similar content. The target content set refers to the set composed of the content stored in the search database. The search database is a pre-established database that stores a large number of target contents and content extraction features of each target content. Each target content and the content extraction features of the target content are stored correspondingly. The content extraction feature of the target content refers to the feature vector extracted from the target content.

[0115] Specifically, when performing a search, after the server obtains the search content, it extracts features of the search content based on the content feature extraction model, obtains a feature vector corresponding to the search content as the search feature corresponding to the search content, performs classification based on the search feature, obtains a classification result corresponding to the search feature as the search classification result, obtains a difference threshold corresponding to the classification category of the search classification result, calculates the feature distance between the content extraction feature and the search feature of each target content in the target content set, obtains the feature vector distance between the content extraction feature and the search feature of the target content as the feature difference, and because the feature distance is negatively correlated with the similarity, that is, the smaller the feature distance, the greater the similarity, therefore, the server selects target content whose feature difference is less than the difference threshold from the target content set, sorts each target content from small to large according to the feature difference, and selects the target content with the highest sorting to obtain the search result corresponding to the search content.

[0116] In some embodiments, the difference threshold corresponding to the classification category of the retrieved classification result obtained by the server may be obtained from a pre-established threshold table, in which the feature distance threshold corresponding to each category is pre-stored.

[0117] In the above embodiment, the feature vector of the search content is extracted by the trained content feature extraction model, so the required search results can be accurately retrieved. Since the difference threshold used in the search process is obtained according to the classification category of the classification result, the business needs of different business scenarios can be met.

[0118] In some other embodiments, it can be understood that the feature distance threshold can be changed dynamically in actual retrieval applications. Taking images as an example, if the retrieved image is a selfie type image of a person, the feature distance threshold should be adjusted higher due to changes in the background / expression decoration of the person. If the retrieved image is a basketball game, game type, etc., the feature distance threshold should be lowered to avoid mismatching due to reasons such as the large similarity of the person's clothing / environment. On the other hand, due to different business needs, in the retrieval application, it is also necessary to face uncertain customized needs to adjust the feature distance threshold. Taking images as an example, if applied to image retrieval of a certain theme, the feature distance threshold of the theme (such as the category of costume dramas) needs to be increased so that there are more images of the theme that users are interested in in the retrieval results. Based on this, it is necessary to dynamically adjust the feature distance threshold according to different application needs.

[0119] In some other embodiments, the step of determining the difference threshold corresponding to the classification category of the retrieved classification result includes: obtaining multiple similar training sample pairs; obtaining sample feature difference values ​​between training samples in the similar training sample pairs, and performing statistics on the sample feature difference values ​​to obtain difference statistics; determining a threshold adjustment direction corresponding to the classification category of the retrieved classification result, determining a corresponding adjustment value based on the threshold adjustment direction, and adjusting the difference value statistics based on the adjustment value to obtain a difference threshold corresponding to the classification category of the retrieved classification result.

[0120] The sample feature difference value between the training samples in the training sample pair may be a feature distance between the training samples in the training sample pair. The statistics may be one of averaging and median.

[0121] Specifically, the server can obtain multiple similar training sample pairs from a similar training sample pair data set, obtain sample feature difference values ​​between training samples in the similar training sample pairs, perform statistics on the sample feature difference values, obtain difference statistics, and determine a threshold adjustment direction corresponding to the classification category of the retrieved classification result. The threshold adjustment direction can be one of increasing the feature distance threshold or decreasing the feature distance threshold. When the threshold adjustment direction is to increase the feature distance threshold, the corresponding adjustment value is determined based on the threshold adjustment direction, and the difference value statistics are increased based on the adjustment value to obtain the difference threshold corresponding to the classification category of the retrieved classification result; and when the threshold adjustment direction is to decrease the feature distance threshold, the corresponding adjustment value is determined based on the threshold adjustment direction, and the difference value statistics are decreased based on the adjustment value to obtain the difference threshold corresponding to the classification category of the retrieved classification result.

[0122] In some embodiments, the server can obtain all similar training sample pairs from the similar training sample pair data set, and perform statistics on the sample feature difference values ​​of all similar sample pairs to obtain difference statistics. The obtained difference statistics are the global feature distance threshold, which can be used to characterize the similarity between similar training sample pairs in the entire data set. For the feature distance threshold that needs to be increased, the adjustment value can be determined to be 1 / 2 of the global feature distance threshold, and the global feature distance + 1 / 2 global feature distance (thr / 2+thr) can be used as the adjusted difference threshold. For the feature distance threshold that needs to be increased, the global feature distance is directly used as the adjusted difference threshold, and the adjustment value is 0 at this time.

[0123] In the above embodiment, the difference statistical value is obtained by statistics, the threshold adjustment direction corresponding to the classification category of the retrieval classification result is determined, the corresponding adjustment value is determined based on the threshold adjustment direction, and the difference value statistical value is adjusted based on the adjustment value to obtain the difference threshold corresponding to the classification category of the retrieval classification result. This realizes the automatic determination of the difference threshold according to different business needs, thereby improving the retrieval efficiency while meeting the business needs.

[0124] In some embodiments, Figure 6 The figure shows a schematic diagram of the retrieval process. The server constructs a content database based on the feature vectors of the historical inventory and the feature vectors extracted from the new content in advance. Each content in the content database is stored in correspondence with the corresponding feature vector. When searching, the feature vector is extracted from the searched content and classified. The similarity threshold is obtained based on the classification result. Then, the content database is searched according to the threshold. During the search process, the similarity between the feature vector of the searched content and the feature vector in the content database is calculated, and the content with a similarity greater than the similarity threshold is selected as the output. Among them, the feature vector of the historical inventory can be obtained from a third-party server, or saved by the server in a historical time period. The similarity between the feature vector of the searched content and the feature vector in the content database can be calculated by cosine similarity.

[0125] The present application also provides an application scenario, which is applicable to the above-mentioned content processing method. In this application scenario, the content is an image, and the method can be applied to image retrieval. In the business scenario of image retrieval, the downstream classification requirements are variable and uncertain. How to support these uncertain classification requirements in embedding metric learning is a problem in business expansion. In the embodiment of the present application, by variably learning random classification tasks in the learning phase, these classification supports are maintained while maintaining the embedding metric learning ability. When downstream applications face new classifications, the model only needs to train new classification layer tasks in a limited manner to be competent, and the original embedding does not need to be updated when applying new categories. Only the classification parameters need to be updated. It is very user-friendly for massive data services. Specifically, the application of the image processing method in this application scenario is as follows:

[0126] First, obtain similar sample pairs as training samples and label the similar sample pairs with category labels. Each similar sample only needs to label one image. Since it is a similar sample pair, the other image has the same category. The labeled image can be regarded as the base image (anchor), and the other similar image can be regarded as the positive sample corresponding to the labeled image. Due to the existence of downstream uncertain classification tasks, the model needs to have learned as many categories as possible during training (at least covering the classification task services of similar sample pairs).

[0127] 2. Model training.

[0128] The model to be trained includes three parts: basic feature extraction model, embedding model, and classification model. The basic feature extraction model and the embedding model together constitute the feature extraction model. The basic feature extraction model training uses resnet101, and the parameters are shown in Table 1. It includes 5 parts of Conv1, Conv2_x-Conv5_x. Conv1 is a 7×7×64 convolution with a stride of 2. Conv2_x includes a 3×3 maximum pooling layer and 3 ResNet modules. Conv3_x-Conv5_x include 3 ResNet modules, 4 ResNet modules, 23 ResNet modules, and 3 ResNet modules, respectively.

[0129] Table 1

[0130]

[0131] The parameters of the embedding model are shown in Table 2, including a fully connected layer. The parameters of the classification model are shown in Table 3, including a semantic layer (Semantic_layer) and a classification layer (Fc_class). The semantic layer is used to obtain first-order information for the embedding features, and the Fc_class classification layer is used to extract second-order information as classification. For applications that require more feature crossover, the semantic layer can be deepened.

[0132] Table 2

[0133] The name of the layer Output size layer Embedding 1x128 full connection

[0134] Table 3

[0135] The name of the layer Output size layer Semantic_layer 1x128 full connection Fc_class 1x11 full connection

[0136] Since the number of categories and classification categories of uncertain classification problems are uncertain, a fixed number of categories M = 10 can be used to reduce uncertainty (the number of categories that require special processing is often less than 10). In addition, in addition to the M categories that need to be learned, there are many samples in the training category set that do not belong to these M categories, so it is necessary to set a reference category to classify these categories that do not belong to these M categories into the reference category, so Fc_class outputs M+1 categories. The advantage of a fixed number of categories is that the model can learn a stable classification paradigm while avoiding the impact of too much uncertainty on model convergence. It should be noted that the basic feature extraction model, embedding model and classification model can also adopt other model structures, such as the basic features using the resnet18CNN module, such as the embedding model using multi-layer fully connected layer connections, etc. The specific training process is as follows:

[0137] 1. Parameter initialization: Conv1-Conv5 can use the parameters of ResNet101 pre-trained on the ImageNet dataset. The newly added layers (semantic layer, classification layer, and fully connected layer of the embedding model) can be initialized with a Gaussian distribution with a variance of 0.01 and a mean of 0.

[0138] 2. Set learning parameters: Setting all layer parameters requires learning.

[0139] 3. Learning rate lr: The basic feature extraction model, embedding model, and semantic layer all use lr1=0.005 learning rate, and the classification layer Fc_class uses lr2=0.05 (10 times that of other structures). After every 10 rounds of iteration, the learning rate lr becomes 0.1 times the original. Because classification learning is a strong supervision of the image, it converges faster than the similarity metric loss of learning global features (that is, the loss of the embedding model) and is therefore more prone to overfitting. By setting the learning rate of the classification layer to 10 times that of other structures, it can prevent the classification information from being transferred to other layers too quickly.

[0140] 4. Learning process: For the training categories in the training category set, perform epoch iterations; each iteration processes all categories once, and a total of 10 rounds of learning are performed. Among them, for a certain round of iteration, 10 classification categories (and all similar sample pairs under the category) are extracted from the training category set without replacement each time until all categories are traversed once, which is considered an epoch (round) of learning. If the last extraction is less than 10 categories, 10 categories are randomly extracted from the categories that have been learned to make up for it. The specific steps in each epoch iteration are as follows:

[0141] 4.1. Construct a certain (i) classification task and preprocessing under this task. For the i-th classification task, since it is a new classification task, the parameters of the Fc_class layer need to be reinitialized: specifically, a Gaussian distribution with a variance of 0.01 and a mean of 0 can be used for initialization. From the training category set, 10 categories are randomly selected, and the remaining categories that have not been selected are classified as the 11th category (reference category). The number of sample pairs of the category with the largest number of training samples among the 10 categories is recorded (for example, the second category has the largest number of images, with K image pairs). Iterative learning is performed without replacement from these 11 categories. Each time, (bs / 11) similar sample pairs are extracted from all these 11 categories. For 10 target categories, at most K / (bs / 11) = 11K / bs iter is required to complete the data traversal of all 10 categories, and a total of five times, so 11K / bs*5 iter. After completing all iter of this classification task, extract the next (i+1) classification task, until all classifications are trained once and the epoch learning is finished, and the trained image feature extraction model is obtained. Among them, bs refers to the number of samples (batch) extracted each time. In each iteration, the model training process is as follows:

[0142] 4.1.1. Set all parameters of the model to a state where learning is required.

[0143] 4.1.2. Obtain the triplets corresponding to all similar sample pairs in the batch, input any image in the triplets into the feature extraction model to obtain a feature vector, classify it through the classification model, obtain the classification result, and calculate the classification loss value based on the classification result and sample label and refer to the formula (3) above, where the true value label vector t[i] is a 1*11 vector of 0 and 1, and o[i] is the predicted probability value of the classification model for 11 categories (10 randomly selected categories plus one reference category).

[0144] For each image in the triplet, a feature vector is obtained through the feature extraction model and the feature extraction loss value is calculated by referring to the triplet loss function in the above formula (2). The two loss values ​​are weightedly summed to obtain the target model loss value.

[0145] For each similar sample pair x in the batch, a triplet can be generated in the following way: randomly select an image from each of the remaining similar sample pairs in the batch to calculate the embedding (the output of the current model embedding module), calculate the distance between the embedding and x, sort them from small to large by distance, take the first 10 samples as negative samples, and form a triplet with the similar sample pair x respectively, so each similar sample pair generates 10 triplets, and the entire batch obtains 10*bs triplets. It can be understood that the triplet samples can also be generated before training and stored in correspondence with the corresponding similar sample pairs, and the corresponding triplet can be directly obtained according to the similar sample pairs.

[0146] 4.1.3. Using the stochastic gradient descent algorithm, the target model loss value obtained in the previous step is subjected to gradient backward calculation to obtain the updated value of all model parameters, and the updated value is used to update the parameters of all models.

[0147] 5. Classification learning on target classification tasks. For new business needs, new classification task learning is required, in which the image feature extraction model (including the basic feature extraction model and the embedding model) is no longer updated, and only the semantic layer and classification layer parameters of the classification model are updated. When the target categories are less than 10, training can continue. Assuming that the target has P categories, the P+1 to 10 categories are invalid categories, and the 11th category is still other categories. Among them, since the image feature extraction model no longer needs to be updated, similar sample pairs are not prepared, and only a single image with a category label is required. Assume that the maximum number of category samples for the target 10 categories is K images. The training steps are similar to the above, in which the parameters in Tables 1 and 2 are fixed and not updated; each batch extracts bs / 11 samples from 10+1 categories for training, and only the classification loss value is used to calculate the gradient and update the parameters in Table 3. The training stops after 11K / bs*5 iters.

[0148] 3. Image Retrieval

[0149] 1. For the case where the threshold is fixed during the retrieval process. For the retrieval image, directly input it into the trained image feature extraction model to obtain the feature vector embedding, calculate the feature distance between the feature vector embedding and the embedding of each image in the image database, select images with feature distance less than the preset threshold for recall, and obtain similar images corresponding to the retrieval image.

[0150] 2. Different categories correspond to different thresholds during the retrieval process. Figure 7, is a schematic diagram of the process of retrieving an image. For the retrieved image, after extracting the feature vector through the basic feature extraction model CNN and the embedding model, it is classified through the trained classification model to obtain the classification result. According to the classification result, the feature distance threshold of the category corresponding to the classification result is queried from the pre-established classification threshold table, and then the image less than the feature distance threshold is selected from the image database for recall according to the feature distance threshold. Among them, the pre-established classification threshold table stores the feature distance threshold corresponding to each category. The image database stores each image and the feature vector corresponding to the image. The feature vector corresponding to the image can be extracted by the basic feature extraction model CNN and the embedding model.

[0151] The embodiments of the present application have the following beneficial effects in the above application scenarios:

[0152] 1. Support embedding learning for downstream tasks with uncertain classification: By randomly constructing classification tasks in embedding learning, embedding can satisfy the similarity triplet measurement while supporting subsequent classification learning. Under the above learning, embedding can converge to an optimal model parameter position for similarity measurement that supports semantic expression. Therefore, both have better application performance after joint learning.

[0153] 2. A deeper classification feature extraction layer can resist overfitting and semantic information extraction: Adding a semantic layer can extract deeper semantic information from the embedding, and on the other hand, it can also prevent classification from affecting embedding learning too early.

[0154] 3. Supports downstream applications that only learn classification without updating the image embedding (feature vector), which improves the expansion of embedding in downstream semantic tasks to meet uncertain and customized business needs without updating inventory features, avoiding the cost consumption caused by upgrading the embedding database.

[0155] The present application also provides an application scenario, which can be applied to the above-mentioned content processing method. In this application scenario, an image feature extraction model is trained through the same steps as the above application scenario, and the image feature extraction model is spliced ​​with the initialized classification model to obtain an image classification model. While keeping the model parameters of the image feature extraction model unchanged, the classification model is trained under the target classification task through the feature extraction model. After the training is completed, the trained image classification model is obtained and applied to the advertising insertion business. For example, for basketball game videos, advertisements are randomly inserted, and for non-basketball games, advertisements are not inserted. For each frame of the video that the user is about to play, it is judged whether the frame classification is basketball. If yes, the video time point corresponding to the frame is returned.

[0156] It should be understood that although Figure 2-Figure 7 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-Figure 7 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0157] In some embodiments, Figure 8 As shown, a content processing device 800 is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes:

[0158] The task determination module 802 is used to determine the current classification task in the classification task sequence; wherein the classification task sequence includes multiple classification tasks;

[0159] A training category acquisition module 804 is used to acquire a current training category corresponding to the current classification task, wherein the training categories corresponding to at least two classification tasks in the classification task sequence are different;

[0160] A feature extraction module 806 is used to obtain a current training sample corresponding to the current training category, and perform feature extraction on the current training sample based on a feature extraction model to be trained to obtain a current sample extraction feature;

[0161] A classification module 808 is used to extract features based on the current sample to perform classification and obtain a current classification result;

[0162] A training module 810 is used to train a feature extraction model to be trained based on the current classification result to obtain a trained feature extraction model corresponding to the current classification task, wherein the feature extraction model to be trained is obtained by training a forward classification task corresponding to the current classification task in the classification task sequence;

[0163] The model acquisition module 812 is used to use the feature extraction model trained according to each classification task in the classification task sequence as a content feature extraction model, so as to extract features from the content based on the content feature extraction model.

[0164] The above-mentioned content processing device determines the current classification task in the classification task sequence, obtains the current training category corresponding to the current classification task, obtains the current training sample corresponding to the current training category, extracts features from the current training sample based on the feature extraction model to be trained, obtains the current sample extraction feature, classifies based on the current sample extraction feature, obtains the current classification result, trains the feature extraction model to be trained based on the current classification result, and obtains the trained feature extraction model corresponding to the current classification task. Since the trained feature extraction model is obtained by training based on the classification result, and since the feature extraction model obtained by training according to each classification task in the classification task sequence is used as the content feature extraction model, the classification task sequence includes multiple classification tasks, wherein at least two classification tasks correspond to different training categories. Therefore, the feature extraction model learns variable classification tasks during the training process. Due to the change of the classification tasks during the training process, the content feature extraction model can learn the ability to distinguish the features of different categories of content while maintaining the feature extraction capability, thereby improving the accuracy of feature extraction.

[0165] In some embodiments, the training category acquisition module is also used to obtain a current category set corresponding to the current classification task; wherein the current category set is obtained by removing forward training categories from the training category set, and the forward training categories are the training categories corresponding to the forward classification tasks of the current classification task in the classification task sequence; categories are extracted from the current category set, and the extracted categories are used as the current training categories corresponding to the current classification task.

[0166] In the above embodiment, by obtaining the current category set corresponding to the current classification task, categories are extracted from the current category set, and the extracted categories are used as the current training categories corresponding to the current classification task. After the extraction, the extracted categories are removed from the current category set, and the remaining categories are used as the current category set for the next classification task, thereby achieving category extraction without replacement, ensuring that all training categories in the candidate category set can be learned, so that the obtained feature extraction model can support the classification of as many categories as possible.

[0167] In some embodiments, the training category acquisition module is also used to classify the unextracted categories in the current category set into the same reference category, and the reference category is the shielded category corresponding to the current classification task; the feature extraction module is also used to obtain the reference training samples corresponding to the reference category, and perform feature extraction on the reference training samples based on the feature extraction model to be trained to obtain reference sample extracted features; the classification module is also used to perform classification based on the reference sample extracted features to obtain reference classification results; the training module is also used to train the feature extraction model to be trained based on the current classification results and the reference classification results to obtain the trained feature extraction model corresponding to the current classification task.

[0168] In some embodiments, the current classification result includes the current category possibility that the current training sample is the current training category, and the reference classification result includes the shielding category possibility corresponding to the reference training sample being the shielding classification category; the training module is also used to obtain a first model loss value based on the current category possibility, and the first model loss value is positively correlated with the current category possibility; obtain a second model loss value based on the shielding category possibility, and the second model loss value is positively correlated with the shielding category possibility; obtain a target model loss value based on the first model loss value and the second model loss value; and adjust the parameters of the feature extraction model to be trained based on the target model loss value to obtain the trained feature extraction model corresponding to the current classification task.

[0169] In the above embodiment, by classifying the categories that have not been extracted in the current category set into the same reference category, the server can train the feature extraction model to be trained based on the reference training samples corresponding to the reference category during the training process. Since the sample features of the masked classification category are taken into consideration, the reliability of the trained feature extraction model can be guaranteed.

[0170] In some embodiments, the training category acquisition module is also used to obtain multiple candidate category sets; remove the forward candidate category set from the multiple candidate category sets to obtain the remaining candidate category set; the forward candidate category set is the candidate category set corresponding to the forward classification task of the current classification task in the classification task sequence; extract the candidate category set from the remaining candidate category set, and use the candidate category in the extracted candidate category set as the current training category corresponding to the current classification task.

[0171] In the above embodiment, the candidate category set is obtained by dividing the training category set according to the number of tasks in the classification task sequence. When generating training tasks, the candidate category set can be extracted to generate training tasks, which can improve the extraction efficiency and thus improve the training efficiency.

[0172] In some embodiments, the training category acquisition module is further used to acquire a training category set for classification training; acquire the number of tasks in a classification task sequence, and divide the training category set based on the number of tasks to obtain multiple candidate category sets.

[0173] By obtaining the number of tasks in the classification task sequence, the training category set is divided based on the number of tasks to obtain multiple candidate category sets, ensuring that all classification tasks in the training category set can be learned during the training process, thereby improving the generalization performance of the feature extraction model.

[0174] In some embodiments, the training module is also used to obtain a comparison sample extraction feature corresponding to the comparison training sample of the current training sample, and obtain a feature extraction loss value based on the difference between the current sample extraction feature and the comparison sample extraction feature; obtain a classification loss value based on the current classification result; obtain a target model loss value based on the feature extraction loss value and the classification loss value; adjust the parameters of the feature extraction model to be trained based on the target model loss value to obtain a trained feature extraction model corresponding to the current classification task.

[0175] In the above embodiment, when training the feature extraction model to be trained, the classification loss value obtained by the classification result and the feature extraction loss value obtained based on the difference between the current sample extracted features and the comparison sample extracted features are combined to perform parameter adjustment, so that the features extracted by the trained feature extraction model are more accurate.

[0176] In some embodiments, the comparison sample extraction features include positive sample extraction features corresponding to positive comparison training samples and negative sample extraction features corresponding to negative comparison training samples; the training module is also used to obtain positive feature difference values, which are feature difference values ​​between the current sample extraction features and the positive sample extraction features; obtain negative feature difference values, which are feature difference values ​​between the current sample extraction features and the negative sample extraction features; and determine a feature extraction loss value based on the positive feature difference value and the negative feature difference value.

[0177] In the above embodiment, since the feature extraction loss value is determined based on the positive feature difference value and the negative feature difference value, the feature extraction model takes into account the impact of the small inter-class feature distance on classification when performing similarity metric learning, thereby improving the accuracy of the feature extraction model when used for classification.

[0178] In some embodiments, the classification module is also used to obtain the current classification model corresponding to the current classification task; classify the sample extraction features based on the current classification model to obtain a classification result; the training module is also used to jointly adjust the model parameters of the feature extraction model to be trained and the model parameters of the current classification model based on the current classification result to obtain an adjusted feature extraction model and an adjusted current classification model; return to the step of obtaining the current training sample corresponding to the current training category until the current classification task meets the task stop condition, and obtain the trained feature extraction model corresponding to the current classification task.

[0179] In the above embodiment, by learning the joint feature extraction and classification tasks simultaneously, merging the feature extraction and classification tasks, learning the feature extraction model and the classification model in the same model structure, the weight control can be updated by gradient to avoid the classification affecting the feature vector expression of feature extraction.

[0180] In some embodiments, the above-mentioned device also includes a content classification model acquisition module, which is used to obtain a target classification category corresponding to a target classification task; obtain a target classification model corresponding to the target classification category; splice the content feature extraction model with the target classification model to obtain a content classification model; classify the target training samples corresponding to the target classification category based on the content classification model to obtain a target classification result; while keeping the model parameters of the content feature extraction model unchanged, adjust the parameters of the target classification model in the content classification model according to the target classification result to obtain an adjusted content classification model.

[0181] In the above embodiment, the content classification model can be trained by only learning the classification task without updating the model parameters of the feature extraction model, so that the feature extraction model can quickly learn new semantic categories to meet the changes in semantic level requirements.

[0182] In some embodiments, the retrieval module of the above-mentioned device is used to obtain retrieval content, perform feature extraction on the retrieval content based on the content feature extraction model to obtain retrieval features corresponding to the retrieval content; perform classification based on the retrieval features to obtain retrieval classification results, and obtain a difference threshold corresponding to the classification category of the retrieval classification results; calculate the feature difference between the content extraction features of each target content in the target content set and the retrieval features; select target content whose feature difference is less than the difference threshold from the target content set, and obtain the retrieval result corresponding to the retrieval content based on the selected target content.

[0183] In the above embodiment, the feature vector of the search content is extracted by the trained content feature extraction model, so the required search results can be accurately retrieved. Since the difference threshold used in the search process is obtained according to the classification category of the classification result, the business needs of different business scenarios can be met.

[0184] In some embodiments, the retrieval module is also used to obtain multiple similar training sample pairs; obtain sample feature difference values ​​between the training samples in the similar training sample pairs, perform statistics on the sample feature difference values, and obtain difference statistics; determine a threshold adjustment direction corresponding to the classification category of the retrieved classification result, determine a corresponding adjustment value based on the threshold adjustment direction, adjust the difference value statistics based on the adjustment value, and obtain a difference threshold corresponding to the classification category of the retrieved classification result.

[0185] In the above embodiment, the difference statistical value is obtained by statistics, the threshold adjustment direction corresponding to the classification category of the retrieval classification result is determined, the corresponding adjustment value is determined based on the threshold adjustment direction, and the difference value statistical value is adjusted based on the adjustment value to obtain the difference threshold corresponding to the classification category of the retrieval classification result. This realizes the automatic determination of the difference threshold according to different business needs, thereby improving the retrieval efficiency while meeting the business needs.

[0186] For the specific definition of the content processing device, please refer to the definition of the content processing method above, which will not be repeated here. Each module in the above-mentioned content processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0187] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store content and feature vectors corresponding to the content. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a content processing method is implemented.

[0188] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a content processing method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0189] Those skilled in the art will understand that Fig. 9 as well as Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0190] In some embodiments, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0191] In some embodiments, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0192] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0193] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0194] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0195] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A content processing method, characterized in that: The content is any one of text, audio or image, and the method includes: Determine a current classification task in a classification task sequence; wherein the classification task sequence includes a plurality of classification tasks; Acquire a current category set corresponding to the current classification task, wherein at least two classification tasks in the classification task sequence correspond to different training categories; Extracting a category from the current category set, and using the extracted category as a current training category corresponding to the current classification task; Classifying the unextracted categories in the current category set into the same reference category, wherein the reference category is the shielded category corresponding to the current classification task, and the current category set is obtained by removing the forward training category from the training category set, and the forward training category is the training category corresponding to the forward classification task of the current classification task in the classification task sequence; Acquire reference training samples corresponding to the reference category, perform feature extraction on the reference training samples based on the feature extraction model to be trained, and obtain reference sample extraction features; Extract features based on the reference sample to perform classification and obtain a reference classification result; Acquire a current training sample corresponding to the current training category, perform feature extraction on the current training sample based on the feature extraction model to be trained, and obtain a current sample extraction feature; Extract features based on the current sample to perform classification and obtain a current classification result; The feature extraction model to be trained is trained based on the current classification result and the reference classification result to obtain a trained feature extraction model corresponding to the current classification task, wherein the feature extraction model to be trained is obtained by training a forward classification task corresponding to the current classification task in the classification task sequence; The feature extraction model obtained by training according to each classification task in the classification task sequence is used as a content feature extraction model, so as to extract features of the content based on the content feature extraction model.

2. The method according to claim 1, characterized in that The current classification result includes the current category possibility that the current training sample is the current training category, and the reference classification result includes the shielding category possibility that the reference training sample is the shielding category corresponding to the shielding category; The training of the feature extraction model to be trained based on the current classification result and the reference classification result to obtain the trained feature extraction model corresponding to the current classification task comprises: Obtaining a first model loss value based on the current category possibility, wherein the first model loss value is positively correlated with the current category possibility; Obtaining a second model loss value based on the shielding category possibility, wherein the second model loss value is positively correlated with the shielding category possibility; Obtaining a target model loss value based on the first model loss value and the second model loss value; Based on the target model loss value, the parameters of the feature extraction model to be trained are adjusted to obtain a trained feature extraction model corresponding to the current classification task.

3. The method according to claim 1, characterized in that The obtaining of the current training category corresponding to the current classification task comprises: Obtain multiple candidate category sets; Removing a forward candidate category set from the multiple candidate category sets to obtain a remaining candidate category set; the forward candidate category set is a candidate category set corresponding to a forward classification task of the current classification task in the classification task sequence; A candidate category set is extracted from the remaining candidate category sets, and the candidate categories in the extracted candidate category set are used as current training categories corresponding to the current classification task.

4. The method according to claim 3, characterized in that: The obtaining of multiple candidate category sets comprises: Get the training category set for classification training; The number of tasks in the classification task sequence is obtained, and the training category set is divided based on the number of tasks to obtain a plurality of candidate category sets.

5. The method according to claim 1, characterized in that The extracting features based on the current sample to perform classification and obtain the current classification result includes: Obtaining a current classification model corresponding to the current classification task; Classify the sample extraction features based on the current classification model to obtain a classification result; The training of the feature extraction model to be trained based on the current classification result and the reference classification result to obtain the trained feature extraction model corresponding to the current classification task comprises: Based on the current classification result and the reference classification result, jointly adjusting the model parameters of the feature extraction model to be trained and the model parameters of the current classification model to obtain an adjusted feature extraction model and an adjusted current classification model; Return to the step of obtaining the current training sample corresponding to the current training category, until the current classification task meets the task stop condition, and obtain the trained feature extraction model corresponding to the current classification task.

6. The method according to claim 1, characterized in that The method further comprises: Obtain the target classification category corresponding to the target classification task; Obtaining a target classification model corresponding to the target classification category; The content feature extraction model is combined with the target classification model to obtain a content classification model; Classifying the target training samples corresponding to the target classification category based on the content classification model to obtain a target classification result; Under the condition that the model parameters of the content feature extraction model remain unchanged, the parameters of the target classification model in the content classification model are adjusted according to the target classification result to obtain an adjusted content classification model.

7. The method according to claim 1, characterized in that The method further comprises: Acquire search content, perform feature extraction on the search content based on the content feature extraction model, and obtain search features corresponding to the search content; Classify based on the search feature to obtain a search classification result, and obtain a difference threshold value corresponding to the classification category of the search classification result; Calculating feature differences between content extraction features of each target content in the target content set and the retrieval features; Target content whose feature difference is less than the difference threshold is selected from the target content set, and a search result corresponding to the search content is obtained based on the selected target content.

8. The method according to claim 7, characterized in that The step of determining the difference threshold corresponding to the classification category of the retrieved classification result includes: Obtain multiple similar training sample pairs; Obtaining sample feature difference values ​​between the training samples in the similar training sample pairs, and performing statistics on the sample feature difference values ​​to obtain difference statistics values; Determine a threshold adjustment direction corresponding to the classification category of the retrieval classification result, determine a corresponding adjustment value based on the threshold adjustment direction, adjust the difference statistical value based on the adjustment value, and obtain a difference threshold corresponding to the classification category of the retrieval classification result.

9. A content processing device, characterized in that: The content is any one of text, audio or image, and the device includes: A task determination module, used to determine a current classification task in a classification task sequence; wherein the classification task sequence includes a plurality of classification tasks; A training category acquisition module is used to acquire a current category set corresponding to the current classification task, wherein the training categories corresponding to at least two classification tasks in the classification task sequence are different; extract categories from the current category set, and use the extracted categories as the current training categories corresponding to the current classification task; and is also used to classify the categories that have not been extracted in the current category set into the same reference category, wherein the reference category is a shielded classification category corresponding to the current classification task, and the current category set is obtained by removing the forward training category from the training category set, wherein the forward training category is a training category corresponding to the forward classification task of the current classification task in the classification task sequence; acquire reference training samples corresponding to the reference categories, perform feature extraction on the reference training samples based on the feature extraction model to be trained, and obtain reference sample extracted features; perform classification based on the reference sample extracted features, and obtain reference classification results; A feature extraction module is used to obtain a current training sample corresponding to the current training category, perform feature extraction on the current training sample based on the feature extraction model to be trained, and obtain a current sample extraction feature; A classification module, used to extract features based on the current sample to perform classification and obtain a current classification result; A training module, used for training the feature extraction model to be trained based on the current classification result and the reference classification result to obtain a trained feature extraction model corresponding to the current classification task, wherein the feature extraction model to be trained is obtained by training a forward classification task corresponding to the current classification task in the classification task sequence; The model acquisition module is used to use the feature extraction model trained according to each classification task in the classification task sequence as the content feature extraction model, so as to extract features from the content based on the content feature extraction model.

10. The device according to claim 9, characterized in that The current classification result includes the current category possibility that the current training sample is the current training category, and the reference classification result includes the shielding category possibility that the reference training sample is the shielding category corresponding to the shielding category; The training module is also used to obtain a first model loss value based on the current category possibility, and the first model loss value is positively correlated with the current category possibility; obtain a second model loss value based on the shielding category possibility, and the second model loss value is positively correlated with the shielding category possibility; obtain a target model loss value based on the first model loss value and the second model loss value; and adjust the parameters of the feature extraction model to be trained based on the target model loss value to obtain a trained feature extraction model corresponding to the current classification task.

11. The device according to claim 9, characterized in that The training category acquisition module is also used to obtain multiple candidate category sets; remove the forward candidate category set from the multiple candidate category sets to obtain the remaining candidate category set; the forward candidate category set is the candidate category set corresponding to the forward classification task of the current classification task in the classification task sequence; extract the candidate category set from the remaining candidate category set, and use the candidate category in the extracted candidate category set as the current training category corresponding to the current classification task.

12. The device according to claim 11, characterized in that The training category acquisition module is further used to acquire a training category set for classification training; acquire the number of tasks in a classification task sequence, and divide the training category set based on the number of tasks to obtain a plurality of candidate category sets.

13. The device according to claim 9, characterized in that The classification module is also used to obtain a current classification model corresponding to the current classification task; classify the sample extraction features based on the current classification model to obtain a classification result; The training module is also used to jointly adjust the model parameters of the feature extraction model to be trained and the model parameters of the current classification model based on the current classification result and the reference classification result to obtain an adjusted feature extraction model and an adjusted current classification model; return to the step of obtaining the current training sample corresponding to the current training category until the current classification task meets the task stop condition, and obtain the trained feature extraction model corresponding to the current classification task.

14. The device according to claim 9, characterized in that The device also includes: The content classification model acquisition module is used to obtain the target classification category corresponding to the target classification task; obtain the target classification model corresponding to the target classification category; splice the content feature extraction model with the target classification model to obtain the content classification model; classify the target training samples corresponding to the target classification category based on the content classification model to obtain the target classification result; while keeping the model parameters of the content feature extraction model unchanged, adjust the parameters of the target classification model in the content classification model according to the target classification result to obtain the adjusted content classification model.

15. The device according to claim 9, characterized in that The device also includes: The retrieval module is used to obtain retrieval content, extract features from the retrieval content based on the content feature extraction model to obtain retrieval features corresponding to the retrieval content; classify based on the retrieval features to obtain retrieval classification results, and obtain a difference threshold value corresponding to the classification category of the retrieval classification results; calculate the feature difference between the content extraction features of each target content in the target content set and the retrieval features; select target content whose feature difference is less than the difference threshold value from the target content set, and obtain the retrieval result corresponding to the retrieval content based on the selected target content.

16. The device according to claim 15, characterized in that The retrieval module is also used to obtain multiple similar training sample pairs; obtain sample feature difference values ​​between training samples in the similar training sample pairs, perform statistics on the sample feature difference values, and obtain difference statistics; determine a threshold adjustment direction corresponding to the classification category of the retrieval classification result, determine a corresponding adjustment value based on the threshold adjustment direction, adjust the difference statistics based on the adjustment value, and obtain a difference threshold corresponding to the classification category of the retrieval classification result.

17. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

18. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

19. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images