A multimedia content classification method and related device
A multimedia content and classification method technology, applied in the field of multimedia content classification methods and related devices, can solve the problems of inaccurate classification of fusion semantic features, poor interactivity, etc., and achieve the effect of improving interaction complexity, accurate classification, and good interactivity
Active Publication Date: 2021-10-15
TENCENT TECH (SHENZHEN) CO LTD
View PDF8 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] However, the above methods such as feature splicing and Hadamard product are relatively simple, resulting in poor interaction of different modal semantic features represented by the fusion semantic features obtained by this method, which leads to inaccurate classification of the fusion semantic features.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0039] Embodiments of the present application are described below in conjunction with the accompanying drawings.
[0040] Such as figure 1 In the overall schematic diagram of multimedia content classification shown, the video A to be classified is input into the classification model, and the output category of the video A to be classified is "game". In related technologies, specific implementation, such as figure 2 A schematic diagram of the specific implementation of multimedia content classification in a related technology is shown. The classification model includes a BERT (Bidirectional Encoder Representation from Transformers) model, a residual network (Residual Network, ResNet) model, a feature fusion sub-model and a classification sub-model; to be The video title information of the classified video A is "This character A is hopeless, the economy is suppressed, and he can't get up at all. Here you can play with your mobile phone!" Input the BERT model and output the tex...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The embodiment of the present application discloses a multimedia content classification method and a related device, which relate to natural language processing and machine learning in artificial intelligence, and obtain the first modal information and the second modal information of different modalities of the multimedia content to be classified ; input it into the feature extraction sub-model of the classification model, and output the first modal semantic feature and the second modal semantic feature; input it into the first feature fusion sub-model of the classification model, and feature fusion output the first fused semantic feature. Input it into the second feature fusion sub-model of the classification model, further perform convolution fusion on the first fusion semantic features through convolution parameters, and output the second fusion semantic features; introduce convolution parameters for convolution fusion to improve the interaction of feature fusion The complexity makes the semantic features of different modes more interactive; the classification sub-model of the classification model uses the second fusion semantic feature to determine the category of the multimedia content to be classified, so that the classification of the multimedia content to be classified is more accurate.
Description
technical field [0001] The present application relates to the field of data processing, in particular to a multimedia content classification method and a related device. Background technique [0002] With the rapid development of science and technology, the classification of multimedia content is very important in scenarios such as searching and recommending multimedia content. Wherein, the multimedia content generally includes at least two modal information among text information, image information, and voice information. [0003] At present, different modal information of multimedia content is usually used as input. After extracting different modal semantic features corresponding to different modal information, feature fusion is performed on different modal semantic features by means of feature splicing or Hadamard product, etc., to obtain Classify multimedia content by fusing semantic features. [0004] However, the above methods such as feature splicing and Hadamard pr...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/62G06N3/04
CPCG06N3/045G06F18/24G06F18/253G06F18/214
Inventor 黄剑辉
Owner TENCENT TECH (SHENZHEN) CO LTD