Video theme classification method and electronic equipment

Abstract

The invention provides a video theme classification method and electronic equipment, and relates to the technical field of data processing. The method comprises the following steps: acquiring a description text of a video, and converting the description text into a corresponding feature vector; inputting the feature vector corresponding to the description text into a pre-trained main classification model, and determining at least one activation tag cluster corresponding to the video according to the feature vector corresponding to the description text; obtaining an intra-cluster classifier corresponding to each activated label cluster; according to the intra-cluster classifier corresponding to each activated tag cluster, determining a theme tag result corresponding to each activated tag cluster; and performing fusion processing on the theme tag results corresponding to the activated tag clusters to obtain the theme tag of the video. According to the invention, a hierarchical dynamic classification architecture of the main classification model and the intra-cluster classifier is provided, and the video content is automatically analyzed to accurately generate the corresponding theme tag, so that the recommendation accuracy of the video content and the watching experience of the user are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a video subject classification method and electronic equipment. Background Art

[0002] Currently, video platforms primarily rely on human editors or reviewers to label long videos (such as movies and TV series) with subject matter tags. This is done by selecting the most suitable tag from a list of preset tags to describe the video content. While this manual tagging method achieves the classification of video subject matter, the process relies on the staff's expertise and experience, resulting in low efficiency and high subjectivity. Furthermore, with the increasing diversification of video content and the increasing frequency of updates, manual tagging methods are no longer able to meet the real-time needs of video platforms for high-quality tags. This can easily lead to issues such as incomplete tag coverage and inaccurate classification, which in turn impacts the user viewing experience and the effectiveness of the video platform's content recommendations. Summary of the Invention

[0003] The purpose of this application is to provide a video subject classification method and electronic device to address the deficiencies in the above-mentioned existing technologies, so as to solve the problem that the manual labeling method in the existing technology can no longer meet the real-time requirements of the video platform for high-quality labels, and is prone to cause problems such as incomplete label coverage and inaccurate classification, which in turn affects the user's viewing experience and the content recommendation effect of the video platform.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows: In a first aspect, an embodiment of the present application provides a method for classifying video themes, the method comprising: Obtaining a description text of the video and converting the description text into a corresponding feature vector; Inputting the feature vector corresponding to the description text into a pre-trained main classification model, the main classification model determining at least one activated label cluster corresponding to the video based on the feature vector corresponding to the description text, each activated label cluster including at least one subject label; Obtaining an intra-cluster classifier corresponding to each of the activation label clusters, wherein the intra-cluster classifier is obtained in advance through training; Determining, according to the intra-cluster classifier corresponding to each of the activated label clusters, a subject label result corresponding to each of the activated label clusters, wherein the subject label result includes: at least one activated label; The subject label results corresponding to each of the activated label clusters are fused to obtain the subject label of the video.

[0005] As a possible implementation manner, converting the description text into a corresponding feature vector includes: Performing word segmentation processing on the description text to obtain multiple description words; Determining the weight of each of the description words according to the frequency of occurrence of each of the description words in the description text; A feature vector corresponding to the description text is generated according to the weight of each description word.

[0006] As a possible implementation manner, determining at least one activation tag cluster corresponding to the video according to the feature vector corresponding to the description text includes: Determining the activation probability of each preset tag cluster according to the feature vector corresponding to the description text, each preset tag cluster including at least one subject tag; Each preset tag cluster is traversed, and for a current tag cluster traversed, the activation probability of the current tag cluster is compared with a first threshold. If the activation probability of the current tag cluster is greater than the first threshold, the current tag cluster is used as an activated tag cluster.

[0007] As a possible implementation, determining the subject label result corresponding to each of the activated label clusters according to the intra-cluster classifier of each of the activated label clusters includes: Determining the activation probability of each theme tag in each of the activation tag clusters according to the intra-cluster classifier corresponding to each of the activation tag clusters; For each of the activated tag clusters, traverse each of the theme tags within the activated tag cluster, and for the traversed current theme tag, compare the activation probability of the current theme tag with a second threshold. If the activation probability of the current theme tag is greater than the second threshold, use the current theme tag as an activated tag.

[0008] As a possible implementation, the training process of the main classification model includes: Obtaining a training data set, wherein the training data set includes a plurality of video samples, each video sample includes a video description text and a subject label set, and the subject label set includes a plurality of different types of subject labels; Converting the video description text of each video sample into a corresponding feature vector; Determining a label similarity matrix based on the subject label set of each video sample; Clustering the subject matter tags according to the tag similarity matrix and the semantic feature vectors of the subject matter tags to obtain a plurality of tag clusters, wherein the subject matter tags in each tag cluster have high correlation; Determine, based on the subject label corresponding to each video sample and the plurality of label clusters, a true cluster vector corresponding to each video sample, wherein the true cluster vector is a representation of the label cluster to which the subject label corresponding to the video sample belongs; The main classification model is obtained by training according to the feature vector corresponding to the video description text of each video sample and the real cluster vector corresponding to each video sample.

[0009] As a possible implementation, determining a label similarity matrix based on the subject label set of each video sample includes: Initialize a label matrix, wherein the first row and the first column of the label matrix represent subject labels respectively, and the initial value of each cell in the label matrix except the first row and the first column is zero; For each of the video sample's subject label sets, multiple subject labels in the subject label set are combined in pairs to obtain multiple label pairs, each label pair is traversed, and for the currently traversed label pair, a target cell and a mirror cell of the label pair in the label matrix are determined, and the values ​​of the target cell and the mirror cell are updated to obtain a label co-occurrence matrix, wherein the first row and the first column of the label co-occurrence matrix respectively represent the subject labels, and the values ​​of each cell other than the first row and the first column represent the co-occurrence frequency of the corresponding label pair; Normalizing the values ​​of each cell except the first row and the first column in the label co-occurrence matrix to obtain a label co-occurrence probability matrix; The tag similarity matrix is ​​determined according to the tag co-occurrence probability matrix.

[0010] As a possible implementation manner, determining the tag similarity matrix according to the tag co-occurrence probability matrix includes: Each row in the label co-occurrence probability matrix is ​​used as a label feature vector; Determining the modulus of each of the label feature vectors, and determining the dot product of each two label feature vectors; The label similarity matrix is ​​determined according to the module length of each label feature vector and the dot product of every two label feature vectors.

[0011] As a possible implementation, clustering the subject labels according to the label similarity matrix and the semantic feature vectors of the subject labels to obtain multiple label clusters, including: For each theme tag, determining the semantic density of the theme tag based on the semantic feature vector of the theme tag and the semantic feature vectors of other theme tags except the theme tag; Determining a plurality of target theme tags as initial anchor points from the plurality of theme tags according to the semantic density of each theme tag; A first fusion distance from each of the subject labels to each of the initial anchor points is determined, and each subject label is divided into different label clusters according to the first fusion distance to obtain a plurality of label clusters.

[0012] As a possible implementation method, after clustering the subject labels to obtain multiple label clusters, the method further includes: Step A: For each tag cluster, determine the semantic center of the tag cluster based on the semantic feature vectors of each theme tag in the tag cluster, determine the target theme tag closest to the semantic center in the tag cluster, and use the target theme tag as the new anchor point of the tag cluster; Step B: determining a second fusion distance from each theme label in the label cluster to the new anchor point, and redistributing each theme label according to the second fusion distance to obtain multiple new label clusters; Step C: Iterate the above steps A to B until the anchor point change rate of the label cluster is less than the third threshold, or the number of iterations reaches the preset maximum number of iterations, stop the iteration, and obtain multiple target label clusters.

[0013] In a second aspect, an embodiment of the present application provides a video subject classification device, the device comprising: A conversion module, configured to obtain a description text of a video and convert the description text into a corresponding feature vector; A first determination module is configured to input a feature vector corresponding to the description text into a pre-trained main classification model, and the main classification model determines at least one active label cluster corresponding to the video based on the feature vector corresponding to the description text, each active label cluster including at least one subject label; An acquisition module, configured to acquire an intra-cluster classifier corresponding to each of the activation label clusters, wherein the intra-cluster classifier is obtained in advance through training; A second determining module is configured to determine a subject label result corresponding to each of the activation label clusters based on an intra-cluster classifier corresponding to each of the activation label clusters, wherein the subject label result includes at least one activation label; The fusion processing module is used to perform fusion processing on the subject label results corresponding to each of the activated label clusters to obtain the subject label of the video.

[0014] As a possible implementation, the conversion module is specifically configured to: Performing word segmentation processing on the description text to obtain multiple description words; Determining the weight of each of the description words according to the frequency of occurrence of each of the description words in the description text; A feature vector corresponding to the description text is generated according to the weight of each description word.

[0015] As a possible implementation manner, the first determining module is specifically configured to: Determining the activation probability of each preset tag cluster according to the feature vector corresponding to the description text, each preset tag cluster including at least one subject tag; Each preset tag cluster is traversed, and for a current tag cluster traversed, the activation probability of the current tag cluster is compared with a first threshold. If the activation probability of the current tag cluster is greater than the first threshold, the current tag cluster is used as an activated tag cluster.

[0016] As a possible implementation manner, the second determining module is specifically configured to: Determining the activation probability of each theme tag in each of the activation tag clusters according to the intra-cluster classifier corresponding to each of the activation tag clusters; For each of the activated tag clusters, traverse each of the theme tags within the activated tag cluster, and for the traversed current theme tag, compare the activation probability of the current theme tag with a second threshold. If the activation probability of the current theme tag is greater than the second threshold, use the current theme tag as an activated tag.

[0017] As a possible implementation, the video subject classification device further includes a training module, and the training module is specifically configured to: Obtaining a training data set, wherein the training data set includes a plurality of video samples, each video sample includes a video description text and a subject label set, and the subject label set includes a plurality of different types of subject labels; Converting the video description text of each video sample into a corresponding feature vector; Determining a label similarity matrix based on the subject label set of each video sample; Clustering the subject matter tags according to the tag similarity matrix and the semantic feature vectors of the subject matter tags to obtain a plurality of tag clusters, wherein the subject matter tags in each tag cluster have high correlation; Determine, based on the subject label corresponding to each video sample and the plurality of label clusters, a true cluster vector corresponding to each video sample, wherein the true cluster vector is a representation of the label cluster to which the subject label corresponding to the video sample belongs; The main classification model is obtained by training according to the feature vector corresponding to the video description text of each video sample and the real cluster vector corresponding to each video sample.

[0018] As a possible implementation, the training module is specifically used to: Initialize a label matrix, wherein the first row and the first column of the label matrix represent subject labels respectively, and the initial value of each cell in the label matrix except the first row and the first column is zero; For each of the video sample's subject label sets, multiple subject labels in the subject label set are combined in pairs to obtain multiple label pairs, each label pair is traversed, and for the currently traversed label pair, a target cell and a mirror cell of the label pair in the label matrix are determined, and the values ​​of the target cell and the mirror cell are updated to obtain a label co-occurrence matrix, wherein the first row and the first column of the label co-occurrence matrix respectively represent the subject labels, and the values ​​of each cell other than the first row and the first column represent the co-occurrence frequency of the corresponding label pair; Normalizing the values ​​of each cell except the first row and the first column in the label co-occurrence matrix to obtain a label co-occurrence probability matrix; The tag similarity matrix is ​​determined according to the tag co-occurrence probability matrix.

[0019] As a possible implementation, the training module is specifically used to: Each row in the label co-occurrence probability matrix is ​​used as a label feature vector; Determining the modulus of each of the label feature vectors, and determining the dot product of each two label feature vectors; The label similarity matrix is ​​determined according to the module length of each label feature vector and the dot product of every two label feature vectors.

[0020] As a possible implementation, the training module is specifically used to: For each theme tag, determining the semantic density of the theme tag based on the semantic feature vector of the theme tag and the semantic feature vectors of other theme tags except the theme tag; Determining a plurality of target theme tags as initial anchor points from the plurality of theme tags according to the semantic density of each theme tag; A first fusion distance from each of the subject labels to each of the initial anchor points is determined, and each subject label is divided into different label clusters according to the first fusion distance to obtain a plurality of label clusters.

[0021] As a possible implementation, the training module is further configured to: Step A: For each tag cluster, determine the semantic center of the tag cluster based on the semantic feature vectors of each theme tag in the tag cluster, determine the target theme tag closest to the semantic center in the tag cluster, and use the target theme tag as the new anchor point of the tag cluster; Step B: determining a second fusion distance from each theme label in the label cluster to the new anchor point, and redistributing each theme label according to the second fusion distance to obtain multiple new label clusters; Step C: Iterate the above steps A to B until the anchor point change rate of the label cluster is less than the third threshold, or the number of iterations reaches the preset maximum number of iterations, stop the iteration, and obtain multiple target label clusters.

[0022] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the video subject classification method as described in any one of the first aspects above.

[0023] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the video subject classification method as described in any one of the above-mentioned first aspects are executed.

[0024] According to the video subject classification method and electronic device of the embodiment of the present application, the description text of the video is obtained, and the description text is converted into a corresponding feature vector, and the feature vector corresponding to the description text is input into the pre-trained main classification model. The main classification model determines at least one activation label cluster corresponding to the video based on the feature vector corresponding to the description text, and each activation label cluster includes at least one subject label. Then, the intra-cluster classifier corresponding to each activation label cluster is obtained, and the subject label result corresponding to each activation label cluster is determined based on the intra-cluster classifier corresponding to each activation label cluster. The subject label result corresponding to each activation label cluster is fused to obtain the subject label of the video. According to the embodiment of the present application, the video description text is automatically converted into a feature vector, and the main classification model is used to quickly predict the activation label cluster to which the video belongs, thereby narrowing the subsequent prediction range. Then, the intra-cluster classifier corresponding to each activation label cluster that has been pre-trained is used to perform fine-grained label prediction on each activation label cluster, and the prediction results of each intra-cluster classifier are fused to output a complete subject label. The entire prediction process does not require human intervention, which meets the video platform's needs for fast and high-quality labeling of massive content. This application provides a hierarchical dynamic classification architecture of a main classification model plus an intra-cluster classifier. By automatically analyzing the video content and automatically and accurately generating corresponding subject labels, it significantly improves the accuracy of video content recommendations and the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram showing a flow chart of a video subject classification method provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of a process for determining an activation tag cluster provided in an embodiment of the present application is shown; Figure 3 A flow chart of a method for determining a subject label result provided in an embodiment of the present application is shown; Figure 4 A schematic diagram of a process flow of a main classification model training method provided in an embodiment of the present application is shown; Figure 5 A flowchart of a method for determining a tag similarity matrix provided in an embodiment of the present application is shown; Figure 6 A schematic diagram of a flow chart of a tag cluster determination method provided in an embodiment of the present application is shown; Figure 7 A schematic diagram of a process flow of a subject label clustering method provided in an embodiment of the present application is shown; Figure 8 A schematic diagram showing a process flow of another subject label clustering method provided in an embodiment of the present application is shown; Figure 9 A flowchart of a tag prediction method provided in an embodiment of the present application is shown; Figure 10 A schematic diagram of the structure of a video subject classification device provided in an embodiment of the present application is shown; Figure 11 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0028] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0029] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0030] Figure 1 FIG2 shows a flow chart of a video subject classification method provided by an embodiment of the present application. Figure 1 As shown, the method specifically includes the following steps: S101: Obtain a description text of a video, and convert the description text into a corresponding feature vector.

[0031] Optionally, the video in this application is a long video such as a TV series or a movie, and the description text includes text information such as the video title, starring actor, director, plot summary, episode plot, etc. These text information are merged into a long text, and the text feature extraction method (TF-IDF, Term Frequency - Inverse Document Frequency) is used to convert the long text into a numerical feature vector.

[0032] S102: Inputting a feature vector corresponding to the description text into a pre-trained main classification model, and the main classification model determines at least one activation label cluster corresponding to the video based on the feature vector corresponding to the description text.

[0033] Optionally, the primary classification model is a pre-trained deep neural network (DNN). Its input is, for example, a 1024-dimensional TF-IDF feature vector, and its output is, for example, a K-dimensional vector representing the activation probability of a video belonging to each label cluster. Based on the activation probabilities of the video belonging to each label cluster, at least one activated label cluster corresponding to the video is determined, each of which includes at least one subject label. This method selects activated label clusters relevant to the video from multiple label clusters, narrowing the range of subsequent predictions for the final subject label, thereby improving the efficiency and accuracy of video subject label classification.

[0034] Optionally, multiple tag clusters are pre-determined by tag clustering, and the tags within each tag cluster have high correlation. This application obtains multiple original subject matter tags and performs cluster analysis on all the original subject matter tags, grouping the subject matter tags with strong semantic and content correlation into a tag cluster, so that the tags within each tag cluster have similar themes or semantic features, and the tags between different tag clusters are significantly different.

[0035] S103: Obtain the intra-cluster classifier corresponding to each activated label cluster.

[0036] Optionally, the intra-cluster classifier is pre-trained, with each label cluster having a pre-trained intra-cluster classifier. The input of the intra-cluster classifier is the intermediate layer feature vector of the main classification model, such as the 512-dimensional hidden layer feature vector of the main classification model, and the output is the predicted activation probability of all labels within the label cluster. Accordingly, after determining at least one activated label cluster corresponding to a video, the intra-cluster classifier corresponding to each activated label cluster can be obtained. Furthermore, using the intra-cluster classifier corresponding to each activated label cluster, the subject matter label results corresponding to each activated label cluster can be determined, including at least one activated label, i.e., multiple subject matter labels suitable for the video.

[0037] S104 : Determine the subject label result corresponding to each activated label cluster according to the intra-cluster classifier corresponding to each activated label cluster.

[0038] Optionally, the 12-dimensional hidden layer feature vector of the main classification model 512 is used as the input of the intra-cluster classifier and is input into the intra-cluster classifier corresponding to each activated label cluster. The intra-cluster classifier corresponding to each activated label cluster outputs an n-dimensional vector, where each dimension of the n-dimensional vector represents the predicted probability of the corresponding label, and the predicted probability is a specific value between 0 and 1. Here, n can be determined based on the number of subject matter labels in the label cluster. For example, if there are 10 subject matter labels in the label cluster, the n-dimensional vector output by the intra-cluster classifier is a 10-dimensional vector. Optionally, the subject labeling result includes at least one active label. Based on the n-dimensional vectors output by the intra-cluster classifiers corresponding to each active label cluster, a predicted probability that the video belongs to a particular subject label, as predicted by the intra-cluster classifiers corresponding to each active label cluster, is obtained. Based on the predicted probabilities, a more credible subject label is selected from the multiple subject labels as the active label, thereby obtaining subject labeling results corresponding to each active label cluster.

[0039] S105: Fusing the subject label results corresponding to each activated label cluster to obtain the subject label of the video.

[0040] Optionally, since the subject labels in each activation label cluster are highly correlated, and each activation label cluster is independent, the subject label results corresponding to each activation label cluster are also different. Therefore, the subject label results corresponding to each activation label cluster can be fused to obtain a more complete and comprehensive subject label result, which can more comprehensively reflect the multiple subject labels corresponding to the video. Specifically, the subject label results corresponding to each activation label cluster can be merged. For example, if the subject label result of one activation label cluster is "science fiction, space", and the subject label result of another activation label cluster is "action, fighting", then the subject label of the video obtained after the fusion process includes "science fiction, space, action, fighting", that is, the subject label results output by different activation label clusters are merged.

[0041] Based on this, according to the video subject classification method of the embodiment of the present application, the video description text is automatically converted into a feature vector, and the activation label cluster to which the video belongs is quickly predicted through the main classification model, thereby narrowing the subsequent prediction range. Then, the intra-cluster classifiers corresponding to each pre-trained activation label cluster are used to perform fine-grained label prediction on each activation label cluster, and the prediction results of each intra-cluster classifier are fused to output a complete subject label. The entire prediction process does not require manual intervention, which meets the video platform's demand for fast and high-quality labeling of massive content. The present application provides a hierarchical dynamic classification architecture of a main classification model plus an intra-cluster classifier, which automatically analyzes the video content and automatically and accurately generates corresponding subject labels, significantly improving the accuracy of video content recommendations and the user's viewing experience.

[0042] As a possible implementation method, the above-mentioned step S101 converts the description text into a corresponding feature vector, including: performing word segmentation processing on the description text to obtain multiple description words, determining the weight of each description word according to the frequency of occurrence of each description word in the description text, and generating a feature vector corresponding to the description text according to the weight of each description word.

[0043] Optionally, converting the description text into a corresponding feature vector is a text vectorization process. Specifically, the description text is a continuous text containing text information such as the video title, starring actor, director, plot summary, episode plot, etc. The word segmentation processing of the description text is to divide the continuous description text into individual description words and remove the stop words therein, such as meaningless words such as "is, of, and, etc.", and then count the word frequency of each valid description word, that is, the frequency of each description word in the entire description text, and perform weighted calculation on each description word according to the frequency of each description word in the description text to obtain the weight of each description word, and then convert the weight of each description word into a numerical vector, and use the numerical vector as the feature vector corresponding to the description text.

[0044] Based on this, the description text is segmented, and the word frequency of each description word obtained by the word segmentation is weighted to obtain the weight of each description word. The weight of each description word is converted to generate a numerical vector of fixed length, and the numerical vector is used as the feature vector corresponding to the description text, so that the feature vector can be used as the input of the main classification model to determine at least one activation label cluster corresponding to the video.

[0045] Figure 2 A flow chart of a method for determining an activation tag cluster provided by an embodiment of the present application is shown. As a possible implementation method, refer to Figure 2 As shown, the above step S102 determines at least one activation tag cluster corresponding to the video according to the feature vector corresponding to the description text, including: S201: Determine the activation probability of each preset tag cluster according to the feature vector corresponding to the description text.

[0046] Optionally, the preset tag cluster is a tag set that has been grouped by a clustering algorithm, each preset tag cluster includes at least one subject tag, and each subject tag has similar themes or semantic features, while the tags between different tag clusters are significantly different.

[0047] Optionally, a feature vector corresponding to the description text, such as a 1024-dimensional TF-IDF feature vector, is used as model input and input into the input layer of the main classification model. The hidden layer of the main classification model converts the 1024-dimensional TF-IDF feature vector into a 512-dimensional hidden layer feature vector. In this way, based on the binary classification principle, it is determined whether each preset label cluster is activated, and finally the output layer outputs a K-dimensional vector to represent the activation probability of each preset label cluster.

[0048] S202 , traverse each preset tag cluster, and for the traversed current tag cluster, compare the activation probability of the current tag cluster with a first threshold. If the activation probability of the current tag cluster is greater than the first threshold, treat the current tag cluster as an activated tag cluster.

[0049] Optionally, after obtaining the activation probability of each preset tag cluster, each preset tag cluster is traversed, and for the current tag cluster traversed, the activation probability of the current tag cluster is compared with a first threshold value, such as 0.5. If the activation probability of the current tag cluster is greater than the first threshold value 0.5, the current tag cluster is used as an activated tag cluster. Otherwise, if the activation probability of the current tag cluster is less than or equal to the first threshold value 0.5, the current tag cluster will not be activated, nor will it be used as an activated tag cluster, and can be directly skipped. In this way, a tag cluster with a high correlation with the video can be screened out from multiple preset tag clusters as the activated tag cluster corresponding to the video.

[0050] Based on this, the main classification model is used to predict which label clusters the video may belong to based on the feature vector corresponding to the video's description text, and then a pre-set first threshold is used to filter out at least one activated label cluster with a high correlation with the video from multiple label clusters, so that the intra-cluster classifier of the activated label cluster can be used to perform intra-cluster prediction and determine the multiple subject labels corresponding to the video.

[0051] Figure 3 The flowchart of a method for determining a subject label result provided by an embodiment of the present application is shown. As a possible implementation method, refer to Figure 3 As shown, the above step S104 determines the subject label results corresponding to each activated label cluster according to the intra-cluster classifier corresponding to each activated label cluster, including: S301 : Determine the activation probability of each subject label in each activated label cluster according to the intra-cluster classifier corresponding to each activated label cluster.

[0052] Optionally, each active label cluster corresponds to a pre-trained intra-cluster classifier. The input of the intra-cluster classifier is the intermediate layer feature vector of the main classification model, such as the 512-dimensional hidden layer feature vector of the main classification model, and the output is the activation probability of all subject labels within the active label cluster. Using the 512-dimensional hidden layer feature vector of the main classification model as the input of the intra-cluster classifier saves computing resources because it does not need to repeatedly extract feature vectors. Furthermore, the main classification model and the intra-cluster classifier use the same semantic space, ensuring feature space consistency.

[0053] Optionally, the intra-cluster classifiers corresponding to each activated label cluster are all binary classification structures. Based on the binary classification principle, an n-dimensional vector can be predicted and output. The dimension of the n-dimensional vector is determined by the number of subject labels in the activated label cluster, and each dimension represents the activation probability of a subject label.

[0054] S302: For each activated tag cluster, traverse each theme tag in the activated tag cluster, and for the current theme tag traversed, compare the activation probability of the current theme tag with a second threshold. If the activation probability of the current theme tag is greater than the second threshold, then use the current theme tag as an activated tag.

[0055] Optionally, after obtaining the activation probability of each subject tag within each activation tag cluster, for each activation tag cluster, each subject tag within the activation tag cluster is traversed, and for the current subject tag traversed, the activation probability of the current subject tag is compared with a second threshold value, such as 0.5. If the activation probability of the current subject tag is greater than the second threshold value of 0.5, the current subject tag is used as an activation tag. Otherwise, if the activation probability of the current subject tag is less than or equal to the second threshold value of 0.5, the current subject tag will not be activated and will not be used as an activation tag, and can be directly skipped. In this way, a subject tag with a high correlation with the video can be screened out from multiple subject tags as the activation tag corresponding to the video.

[0056] Based on this, the intra-cluster classifier corresponding to each activation label cluster takes the 512-dimensional hidden layer feature vector of the main classification model as input to predict which of the multiple subject labels contained in the activation label cluster the video may belong to, and then uses a pre-set second threshold to filter out activation labels with a higher correlation with the video from the multiple subject labels, thereby obtaining the subject label results corresponding to each activation label cluster.

[0057] Figure 4 The flowchart of a main classification model training method provided by the embodiment of the present application is shown. As a possible implementation method, refer to Figure 4 As shown in Figure 2, the training process of the main classification model specifically includes the following steps: S401: Obtain a training data set.

[0058] Optionally, the training data set includes multiple video samples, each video sample includes a video description text and a subject label set, and the subject label set includes multiple subject labels of different types.

[0059] Optionally, multiple original videos can be obtained from multiple video platforms, and multiple subject labels for each original video under multiple video platforms can be obtained. For each original video, the multiple subject labels of the original video under each video platform are compared. If the multiple subject labels of the original video under a preset number of video platforms are consistent, the original video is taken as a video sample, and a training data set is constructed based on the description text of the original video and the subject label set.

[0060] Exemplarily, the preset number can be set according to the total number of multiple video platforms. For example, there are 5 video platforms, and multiple subject labels of the same original video are obtained from these 5 video platforms. If a preset number (for example, 3) of the 5 video platforms have the same multiple subject labels for the same original video, then this original video can be used as a video sample, and the title, starring actor, director, plot summary, episode plot and other descriptive text of the video sample and the identifier of the video sample are obtained. Then, a training data set is constructed based on the subject label set consisting of the descriptive text of the video sample, the identifier of the video sample and the corresponding multiple subject labels. The sample format of the video samples in the training data set is as follows: { "video_id": "tt1234567", "features": { "title": "XXX", "actors": ["Zhang San", "Li Si"], "director": "Wang Wu", "synopsis": "narrate..." }, "labels": ["Ancient Costume", "Politics", "History", "Martial Arts"]} Among them, video_id represents the identifier of the video sample, features represents the description text of the video sample, title represents the title of the video sample, actors represents the starring actors of the video sample, director represents the director of the video sample, synopsis represents the plot summary of the video sample, and labels represents multiple subject labels in the subject label set of the video sample.

[0061] S402: Convert the video description text of each video sample into a corresponding feature vector.

[0062] Optionally, the video description text of each video sample also includes text information such as the title, starring actor, director, plot summary, episode plot, etc. of the video sample. For each video sample, the text information in the video description text can also be merged into a long text, and the long text can be converted into a numerical feature vector using the text feature extraction method TF-IDF, so as to obtain the feature vector corresponding to the video description text of each video sample.

[0063] S403: Determine a label similarity matrix based on the subject label set of each video sample.

[0064] Optionally, each video sample corresponds to a subject label set. By integrating the subject labels contained in the subject label sets of each video sample, N subject labels can be obtained. By comparing the N subject labels pairwise and calculating the semantic similarity between each two subject labels, an N*N label similarity matrix can be constructed, where N is the total number of subject labels.

[0065] For example, the first row and first column of the tag similarity matrix represent the subject tags, and the values ​​of each cell other than the first row and first column are used to represent the semantic similarity between the corresponding two subject tags. Specifically, the tag similarity matrix is ​​a two-dimensional table, as shown below:

[0066] S404 : Clustering each theme tag according to the tag similarity matrix and the semantic feature vector of each theme tag to obtain a plurality of tag clusters.

[0067] Optionally, the semantic similarity between each subject label in the label similarity matrix is ​​used as the clustering basis, and each subject label is clustered in combination with the semantic feature vector of each subject label to output multiple label clusters. The subject labels in each label cluster have high correlation, which means that each label cluster contains multiple subject labels with similar semantics.

[0068] S405 : Determine a true cluster vector corresponding to each video sample based on the subject label and multiple label clusters corresponding to each video sample.

[0069] Optionally, the true cluster vector represents the label cluster to which the subject label corresponding to the video sample belongs. Each video sample corresponds to a subject label set, which includes multiple subject labels. Multiple label clusters are obtained by pre-clustering the multiple subject labels. For each video sample, by determining whether the subject label corresponding to the video sample is within the corresponding label cluster, the true cluster vector corresponding to the video sample is constructed to indicate which label cluster the video sample belongs to.

[0070] For example, in the true cluster vector corresponding to the video sample, 0 may indicate that the video sample does not belong to a label cluster, and 1 may indicate that the video sample belongs to a label cluster.

[0071] S406 , training a main classification model based on the feature vector corresponding to the video description text of each video sample and the real cluster vector corresponding to each video sample.

[0072] Optionally, the feature vector corresponding to the video description text of each video sample is used as the model input, and the true cluster vector corresponding to each video sample is used as the training target for model iterative training. During each iteration, the predicted cluster vector corresponding to each video sample output by the model is compared with the true cluster vector corresponding to each video sample to determine the loss value between the two. The model parameters are adjusted to minimize the loss value or reach a preset value, and the iteration is stopped to obtain a trained main classification model.

[0073] Based on this, by analyzing the semantic correlation between each subject label, multiple subject labels are clustered to construct a cluster structure, and the cluster vector is used as the training target for model training to obtain a trained main classification model. The main classification model can predict the label cluster to which the video belongs based on the video description text, thereby providing input basis for subsequent intra-cluster classifiers.

[0074] Figure 5 FIG2 shows a flow chart of a method for determining a tag similarity matrix provided by an embodiment of the present application. Figure 5 As shown, the above step S403 determines the label similarity matrix based on the subject label set of each video sample, which specifically includes the following steps: S501: Initialize the label matrix.

[0075] For example, an empty label matrix is ​​created to record the frequency of appearance of any two subject labels in the video, wherein the first row and the first column of the label matrix represent the subject labels respectively, and the initial value of each cell in the label matrix except the first row and the first column is set to zero.

[0076] S502. For the subject label set of each video sample, multiple subject labels in the subject label set are combined in pairs to obtain multiple label pairs, each label pair is traversed, and for the currently traversed label pair, the target cell and the mirror cell of the label pair in the label matrix are determined, and the values ​​of the target cell and the mirror cell are updated to obtain a label co-occurrence matrix.

[0077] Optionally, the first row and the first column of the label co-occurrence frequency matrix represent the subject labels respectively, and the values ​​of each cell except the first row and the first column represent the co-occurrence frequency of the corresponding label pairs.

[0078] For example, each video sample in the training data set is traversed, and for the current video sample traversed, the subject label set of the current video sample is extracted, and all the subject labels in the subject label set are combined in pairs. For example, the subject label set of a video sample includes the three subject labels of "science fiction, adventure, and disaster", then multiple subject labels in this subject label set are combined in pairs, and the resulting multiple label pairs are (science fiction, adventure), (science fiction, disaster), and (adventure, disaster).

[0079] For example, taking the above-mentioned theme tag set including the three theme tags of "science fiction, adventure, and disaster" as an example, the initialized tag matrix is ​​as follows:

[0080] Exemplarily, after obtaining multiple label pairs, each label pair is traversed, and for the currently traversed label pair, the target cell and mirror cell of the label pair in the label matrix are determined. Taking the label pair (science fiction, adventure) as an example, the target cell (that is, the intersection position) of the label pair (science fiction, adventure) in the label matrix shown above is the cell corresponding to the second row and third column, and the mirror cell (that is, the mirror position) is the mirror image corresponding to the label pair (science fiction, adventure). The target cell of the label pair (adventure, science fiction) in the label matrix is ​​also the cell corresponding to the third row and second column in the label matrix shown above.

[0081] For example, the label matrix update rule is to determine the target cell and mirror cell corresponding to each label pair in the label matrix, and increase the values ​​of the target cell and mirror cell by 1. Similarly, the above process is performed on the subject label set of each video sample, and for each label pair, the values ​​of the corresponding target cell and mirror cell are increased by 1, thereby obtaining a label co-occurrence matrix.

[0082] S503 : Normalize the values ​​of each cell except the first row and the first column in the label co-occurrence frequency matrix to obtain a label co-occurrence probability matrix.

[0083] For example, the values ​​of each cell except the first row and the first column in the label co-occurrence frequency matrix represent the co-occurrence frequency of the corresponding label pairs. By normalizing the values ​​of each cell, the frequency is converted into a probability measure, that is, the value of each cell is converted to the range of 0-1, thereby obtaining the label co-occurrence probability matrix.

[0084] For example, the value at each cell may be normalized by dividing the value at each cell by the total number of video samples in the training dataset.

[0085] S504: Determine a tag similarity matrix based on the tag co-occurrence probability matrix.

[0086] Optionally, based on the label co-occurrence probability matrix, label associations are quantified using cosine similarity to obtain a label similarity matrix. Specifically, each row in the label co-occurrence probability matrix is ​​used as a label feature vector, the modulus of each label feature vector is determined, and the dot product of each two label feature vectors is determined. Based on the modulus of each label feature vector and the dot product of each two label feature vectors, a label similarity matrix is ​​determined.

[0087] For example, each row in the label co-occurrence probability matrix is ​​used as a label feature vector, and the modulus of each label feature vector is determined using the following formula (1): (1) in, represents the modulus of the label feature vector, Indicates the number of label feature vectors, ranging from 1 to , Represents the label feature vector.

[0088] For example, the dot product of each two label feature vectors is determined using the following formula (2): (2) in, and represents two label feature vectors, Represents the dot product of two label feature vectors.

[0089] For example, the tag similarity matrix is ​​determined using the following formula (3) based on the modulus of each tag feature vector and the dot product of every two tag feature vectors: (3) in, Represents the label similarity matrix The value at the corresponding position, represents the dot product of two label feature vectors, and Represent the modulus lengths of the two label feature vectors respectively.

[0090] Based on this, by constructing a label co-occurrence matrix and normalizing the cell values ​​in the label co-occurrence matrix, we obtain a label co-occurrence probability matrix. Then, based on the label co-occurrence probability matrix, we quantify the correlation between subject labels through cosine similarity to obtain a label similarity matrix.

[0091] Figure 6 FIG2 shows a flow chart of a tag cluster determination method provided by an embodiment of the present application. Figure 6 As shown, the above step S404 clusters each theme tag according to the tag similarity matrix and the semantic feature vector of each theme tag to obtain multiple tag clusters, which specifically includes the following steps: S601 : For each theme tag, determine the semantic density of the theme tag based on the semantic feature vector of the theme tag and the semantic feature vectors of other theme tags except the theme tag.

[0092] For example, for each theme tag, the language model BERT generates a semantic feature vector corresponding to each theme tag. Furthermore, for each theme tag, the average cosine similarity of the semantic feature vector of the theme tag and the semantic feature vectors of all other theme tags is calculated. The semantic density of the theme tag is then determined based on the average cosine similarity. This semantic density reflects the coreness of the theme tag in the semantic space.

[0093] For example, the semantic density of the subject tag can be calculated using the following formula (4): (4) in, Indicates cosine similarity calculation, Indicates the subject label and The semantic feature vector of Indicates subject matter tags The semantic density of Matrix dimension representing the label similarity matrix.

[0094] S602: Determine a plurality of target theme tags as initial anchor points from a plurality of theme tags according to the semantic density of each theme tag.

[0095] For example, after obtaining the semantic density of each subject tag, the semantic density of each subject tag is sorted in descending order, and the first m high-density subject tags are selected as initial anchor points. For example, there are 200 subject tags in total. After sorting in descending order of semantic density, the first 20 high-density subject tags can be selected as initial anchor points.

[0096] S603: Determine a first fusion distance from each theme label to each initial anchor point, and divide each theme label into different label clusters according to the first fusion distance to obtain multiple label clusters.

[0097] For example, for each theme label, the first fusion distance from the theme label to each initial anchor point is calculated. Specifically, the first fusion distance is calculated using the following formula (5): (5) in, Indicates subject matter tags To the initial anchor point The first fusion distance, Indicates subject matter tags With the initial anchor point The cosine similarity of Indicates the subject labels and The semantic feature vector of and are two constant parameters, and , .

[0098] Exemplarily, according to the first fusion distance from each theme label to each initial anchor point, each theme label is assigned to a cluster corresponding to the anchor point with the shortest fusion distance, thereby obtaining multiple label clusters.

[0099] Based on this, the initial anchor point is determined by calculating the semantic density of the semantic feature vector of each subject label, and then the fusion distance from each subject label to the initial anchor point is calculated, and the label is assigned according to the fusion distance, so that multiple subject labels are divided into corresponding anchor point clusters through anchor point propagation clustering, thereby obtaining multiple label clusters.

[0100] Figure 7 FIG2 shows a flow chart of a method for clustering theme labels provided in an embodiment of the present application. Figure 7 As shown, after the above steps cluster the subject labels to obtain multiple label clusters, the method further includes: S701 , for each tag cluster, determine the semantic center of the tag cluster based on the semantic feature vectors of each theme tag in the tag cluster, determine the target theme tag closest to the semantic center in the tag cluster, and use the target theme tag as the new anchor point of the tag cluster.

[0101] For example, the anchor point position can be updated for each tag cluster. Specifically, the semantic center of each tag cluster is calculated, that is, the average value of the semantic feature vectors of all the theme tags in the tag cluster content. The distance between each theme tag in each tag cluster and the semantic center is then determined. The theme tag closest to the semantic center in the tag cluster is used as the target theme tag, and the target theme tag is used as the new anchor point of the tag cluster.

[0102] S702: Determine a second fusion distance from each subject label in the label cluster to the new anchor point, and redistribute each subject label according to the second fusion distance to obtain multiple new label clusters.

[0103] For example, the above formula (5) is used to determine the second fusion distance from each theme label in the label cluster to the new anchor point. According to the second fusion distance from each theme label to the new anchor point, each theme label is assigned to the cluster corresponding to the anchor point with the shortest fusion distance, so as to redistribute each theme label and obtain multiple new label clusters.

[0104] S703 , iteratively execute the above steps S701 to S702 until the anchor point change rate of the tag cluster is less than the third threshold, or the number of iterations reaches the preset maximum number of iterations, stop the iteration, and obtain multiple target tag clusters.

[0105] Exemplarily, steps S701 and S702 are iteratively executed, i.e., the process of updating the anchor point positions and reallocating labels is iteratively executed until the anchor point change rate of the label cluster is less than a third threshold, such as 0.001, or the number of iterations reaches a preset maximum number of iterations, such as 100. The iterations are then stopped, resulting in multiple target label clusters, each of which has a semantically stable content. The anchor point change rate can be obtained by calculating the anchor point movement distance.

[0106] Optionally, refer to Figure 8 As shown, in this application, by calculating the semantic density of each theme label and sorting them according to the semantic density, determining the initial anchor point, and determining the fusion distance of each theme label to the initial anchor point, and assigning labels based on the fusion distance, multiple label clusters are obtained. Furthermore, the semantic center of each label cluster is updated until the semantics within the label cluster are stable, and multiple target label clusters are obtained. These multiple target label clusters are the final clustering results of the multiple theme labels.

[0107] Based on this, this application performs label classification through the fusion distance between the anchor point and the subject label to obtain multiple initial label clusters, and then redistributes the subject labels by calculating the semantic center within the label cluster to update the anchor point position until the multiple target label clusters finally obtained are semantically stable, thereby obtaining multiple accurately classified target label clusters.

[0108] As a possible implementation method, the present application also provides a training method for an intra-cluster classifier. The difference from training the main classification model is that the training samples are multiple activation label clusters. The other training processes are similar to those of the main classification model and will not be repeated here.

[0109] Optionally, refer to Figure 9 As shown, this application uses the feature vector corresponding to the description text of the video as input, and predicts K activation label clusters through the main classification prediction model, and then uses the intra-cluster classifier corresponding to each activation label cluster to predict the activation label of the activation label cluster content. By fusing the prediction results of the intra-cluster classifiers, the final multi-label prediction result of the video can be obtained. In this way, this application innovatively proposes a hierarchical dynamic classification architecture of a main classification model plus an intra-cluster classifier, which realizes efficient and accurate multi-label prediction and provides an automated content labeling service for the Internet TV platform. By automatically analyzing the video content and automatically and accurately generating the corresponding subject labels, it can not only improve the user experience, but also help the platform better manage and recommend video content.

[0110] Based on the same inventive concept, the embodiment of the present application also provides a video subject classification device corresponding to the video subject classification method. Since the principle of solving the problem by the video subject classification device in the embodiment of the present application is similar to the above-mentioned video subject classification method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0111] Reference Figure 10 FIG. 1 is a schematic diagram of the structure of a video subject classification device provided in an embodiment of the present application. The video subject classification device 1000 includes: a conversion module 1001, a first determination module 1002, an acquisition module 1003, a second determination module 1004, and a fusion processing module 1005, wherein: The conversion module 1001 is used to obtain the description text of the video and convert the description text into a corresponding feature vector; A first determination module 1002 is configured to input a feature vector corresponding to the description text into a pre-trained primary classification model, and the primary classification model determines at least one active label cluster corresponding to the video based on the feature vector corresponding to the description text, each active label cluster including at least one subject label; An acquisition module 1003 is used to acquire an intra-cluster classifier corresponding to each activation label cluster, where the intra-cluster classifier is obtained in advance through training; The second determining module 1004 is configured to determine the subject label results corresponding to each activation label cluster according to the intra-cluster classifier corresponding to each activation label cluster, where the subject label results include: at least one activation label; The fusion processing module 1005 is used to perform fusion processing on the subject label results corresponding to each activated label cluster to obtain the subject label of the video.

[0112] Based on this, according to the video subject classification device of the embodiment of the present application, the video description text is automatically converted into a feature vector, and the activation label cluster to which the video belongs is quickly predicted through the main classification model, thereby narrowing the subsequent prediction range. Then, the pre-trained intra-cluster classifiers corresponding to each activation label cluster are used to perform fine-grained label predictions on each activation label cluster, and the prediction results of each intra-cluster classifier are fused to output a complete subject label. The entire prediction process does not require manual intervention, which meets the video platform's demand for fast and high-quality labeling of massive content. The present application provides a hierarchical dynamic classification architecture of a main classification model plus an intra-cluster classifier, which significantly improves the accuracy of video content recommendations and the user's viewing experience by automatically analyzing the video content and automatically and accurately generating corresponding subject labels.

[0113] In a possible implementation, the conversion module 1001 is specifically configured to: Perform word segmentation on the description text to obtain multiple description words; Determine the weight of each descriptive word according to its frequency of occurrence in the description text; According to the weight of each description word, a feature vector corresponding to the description text is generated.

[0114] In a possible implementation, the first determining module 1002 is specifically configured to: Determine the activation probability of each preset tag cluster based on the feature vector corresponding to the description text, each preset tag cluster including at least one subject tag; Each preset tag cluster is traversed, and for the traversed current tag cluster, the activation probability of the current tag cluster is compared with the first threshold. If the activation probability of the current tag cluster is greater than the first threshold, the current tag cluster is regarded as an activated tag cluster.

[0115] In a possible implementation manner, the second determining module 1004 is specifically configured to: Determine the activation probability of each subject label in each activation label cluster based on the intra-cluster classifier corresponding to each activation label cluster; For each activated tag cluster, each theme tag in the activated tag cluster is traversed, and for the current theme tag traversed, the activation probability of the current theme tag is compared with the second threshold. If the activation probability of the current theme tag is greater than the second threshold, the current theme tag is used as an activated tag.

[0116] In a possible implementation, the video subject classification device further includes a training module, wherein the training module is specifically configured to: Obtain a training data set, the training data set including multiple video samples, each video sample including video description text and a subject label set, the subject label set including multiple different types of subject labels; Convert the video description text of each video sample into a corresponding feature vector; Determine the label similarity matrix based on the subject label set of each video sample; Based on the label similarity matrix and the semantic feature vectors of each theme label, each theme label is clustered to obtain multiple label clusters. The theme labels in each label cluster have high correlation; Determine the true cluster vector corresponding to each video sample based on the subject label and multiple label clusters corresponding to each video sample, where the true cluster vector represents the label cluster to which the subject label corresponding to the video sample belongs; The main classification model is trained based on the feature vector corresponding to the video description text of each video sample and the true cluster vector corresponding to each video sample.

[0117] In one possible implementation, the training module is specifically used to: Initialize the label matrix. The first row and first column of the label matrix represent the subject labels, and the initial value of each cell in the label matrix except the first row and first column is zero. For the subject label set of each video sample, multiple subject labels in the subject label set are combined in pairs to obtain multiple label pairs. Each label pair is traversed, and for the currently traversed label pair, the target cell and the mirror cell of the label pair in the label matrix are determined, and the values ​​of the target cell and the mirror cell are updated to obtain a label co-occurrence matrix. The first row and the first column of the label co-occurrence matrix represent the subject labels respectively, and the values ​​of each cell except the first row and the first column represent the co-occurrence frequency of the corresponding label pair; Normalize the values ​​of each cell except the first row and first column in the label co-occurrence matrix to obtain the label co-occurrence probability matrix; According to the label co-occurrence probability matrix, the label similarity matrix is ​​determined.

[0118] In one possible implementation, the training module is specifically used to: Each row in the label co-occurrence probability matrix is ​​used as a label feature vector; Determine the modulus of each label feature vector, and determine the dot product of each two label feature vectors; The label similarity matrix is ​​determined based on the modulus of each label feature vector and the dot product of every two label feature vectors.

[0119] In one possible implementation, the training module is specifically used to: For each theme tag, determine the semantic density of the theme tag based on the semantic feature vector of the theme tag and the semantic feature vectors of other theme tags except the theme tag; According to the semantic density of each theme tag, multiple target theme tags are determined from multiple theme tags as initial anchor points; A first fusion distance from each subject label to each initial anchor point is determined, and each subject label is divided into different label clusters according to the first fusion distance to obtain multiple label clusters.

[0120] In a possible implementation, the training module is further configured to: Step A: For each tag cluster, determine the semantic center of the tag cluster based on the semantic feature vectors of each theme tag in the tag cluster, and determine the target theme tag closest to the semantic center in the tag cluster, and use the target theme tag as the new anchor point of the tag cluster; Step B: Determine the second fusion distance from each theme label in the label cluster to the new anchor point, and redistribute each theme label according to the second fusion distance to obtain multiple new label clusters; Step C: Iterate the above steps A to B until the anchor point change rate of the label cluster is less than the third threshold, or the number of iterations reaches the preset maximum number of iterations, stop the iteration, and obtain multiple target label clusters.

[0121] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0122] The embodiment of the present application further provides an electronic device 1100, such as Figure 11 FIG. 1 is a schematic diagram of the structure of an electronic device 1100 provided in an embodiment of the present application, comprising a processor 1101, a memory 1102, and optionally, a bus 1103. The memory 1102 stores machine-readable instructions executable by the processor 1101. When the electronic device 1100 is running, the processor 1101 communicates with the memory 1102 via the bus 1103. When the machine-readable instructions are executed by the processor 1101, the steps of the video subject classification method described above are performed.

[0123] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the video subject classification method described above are executed.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0125] In addition, the functional units in the various embodiments of the present application can be integrated into a single processing unit, each unit can exist physically separately, or two or more units can be integrated into a single unit. If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0126] The above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the protection scope of the present application.

Claims

1. A video subject classification method, characterized in that: include: Obtaining a description text of the video and converting the description text into a corresponding feature vector; Inputting the feature vector corresponding to the description text into a pre-trained main classification model, the main classification model determining at least one activated label cluster corresponding to the video based on the feature vector corresponding to the description text, each activated label cluster including at least one subject label; Obtaining an intra-cluster classifier corresponding to each of the activation label clusters, wherein the intra-cluster classifier is obtained in advance through training; Determining, according to the intra-cluster classifier corresponding to each of the activated label clusters, a subject label result corresponding to each of the activated label clusters, wherein the subject label result includes: at least one activated label; The subject label results corresponding to each of the activated label clusters are fused to obtain the subject label of the video.

2. The method according to claim 1, characterized in that The converting the description text into a corresponding feature vector includes: Performing word segmentation processing on the description text to obtain multiple description words; Determining the weight of each of the description words according to the frequency of occurrence of each of the description words in the description text; A feature vector corresponding to the description text is generated according to the weight of each description word.

3. The method according to claim 1, characterized in that The determining, based on the feature vector corresponding to the description text, at least one activation tag cluster corresponding to the video includes: Determining the activation probability of each preset tag cluster based on the feature vector corresponding to the description text, each preset tag cluster including at least one subject tag; Each preset tag cluster is traversed, and for a current tag cluster traversed, the activation probability of the current tag cluster is compared with a first threshold. If the activation probability of the current tag cluster is greater than the first threshold, the current tag cluster is used as an activated tag cluster.

4. The method according to claim 1, wherein The determining, based on the intra-cluster classifier corresponding to each of the activated label clusters, the subject label result corresponding to each of the activated label clusters includes: Determining the activation probability of each subject label in each activation label cluster according to the intra-cluster classifier corresponding to each activation label cluster; For each of the activated tag clusters, traverse each of the theme tags within the activated tag cluster, and for the traversed current theme tag, compare the activation probability of the current theme tag with a second threshold. If the activation probability of the current theme tag is greater than the second threshold, use the current theme tag as an activated tag.

5. The method according to any one of claims 1 to 4, characterized in that The training process of the main classification model includes: Obtaining a training data set, wherein the training data set includes a plurality of video samples, each video sample includes a video description text and a subject label set, and the subject label set includes a plurality of different types of subject labels; Converting the video description text of each video sample into a corresponding feature vector; Determining a label similarity matrix based on the subject label set of each video sample; Clustering the subject matter tags according to the tag similarity matrix and the semantic feature vectors of the subject matter tags to obtain a plurality of tag clusters, wherein the subject matter tags in each tag cluster have high correlation; Determine, based on the subject label corresponding to each video sample and the plurality of label clusters, a true cluster vector corresponding to each video sample, wherein the true cluster vector is a representation of the label cluster to which the subject label corresponding to the video sample belongs; The main classification model is obtained by training according to the feature vector corresponding to the video description text of each video sample and the real cluster vector corresponding to each video sample.

6. The method according to claim 5, characterized in that Determining a label similarity matrix based on the subject label set of each video sample includes: Initialize a label matrix, wherein the first row and the first column of the label matrix represent subject labels respectively, and the initial value of each cell in the label matrix except the first row and the first column is zero; For each of the video sample's subject label sets, multiple subject labels in the subject label set are combined in pairs to obtain multiple label pairs, each label pair is traversed, and for the currently traversed label pair, a target cell and a mirror cell of the label pair in the label matrix are determined, and the values ​​of the target cell and the mirror cell are updated to obtain a label co-occurrence matrix, wherein the first row and the first column of the label co-occurrence matrix respectively represent the subject labels, and the values ​​of each cell other than the first row and the first column represent the co-occurrence frequency of the corresponding label pair; Normalizing the values ​​of each cell except the first row and the first column in the label co-occurrence matrix to obtain a label co-occurrence probability matrix; The tag similarity matrix is ​​determined according to the tag co-occurrence probability matrix.

7. The method according to claim 6, characterized in that Determining the tag similarity matrix according to the tag co-occurrence probability matrix includes: Each row in the label co-occurrence probability matrix is ​​used as a label feature vector; Determining the modulus of each of the label feature vectors, and determining the dot product of each two label feature vectors; The label similarity matrix is ​​determined according to the module length of each label feature vector and the dot product of every two label feature vectors.

8. The method according to claim 5, characterized in that The method of clustering the subject matter tags according to the tag similarity matrix and the semantic feature vectors of the subject matter tags to obtain multiple tag clusters includes: For each theme tag, determining the semantic density of the theme tag based on the semantic feature vector of the theme tag and the semantic feature vectors of other theme tags except the theme tag; Determining a plurality of target theme tags as initial anchor points from the plurality of theme tags according to the semantic density of each theme tag; A first fusion distance from each of the subject labels to each of the initial anchor points is determined, and each subject label is divided into different label clusters according to the first fusion distance to obtain a plurality of label clusters.

9. The method according to claim 5, characterized in that After clustering the subject labels to obtain a plurality of label clusters, the method further includes: Step A: For each tag cluster, determine the semantic center of the tag cluster based on the semantic feature vectors of each theme tag in the tag cluster, determine the target theme tag closest to the semantic center in the tag cluster, and use the target theme tag as the new anchor point of the tag cluster; Step B: determining a second fusion distance from each theme label in the label cluster to the new anchor point, and redistributing each theme label according to the second fusion distance to obtain multiple new label clusters; Step C: Iterate the above steps A to B until the anchor point change rate of the label cluster is less than the third threshold, or the number of iterations reaches the preset maximum number of iterations, stop the iteration, and obtain multiple target label clusters.

10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor executes the machine-readable instructions to perform the steps of the video subject classification method according to any one of claims 1 to 9.

Images