A method and system for automatic classification of video content recognition

Abstract

The present invention discloses a method and system for automatic classification of video content recognition, the method comprising S1, multimodal feature extraction: extracting visual features, audio features and text features of video data respectively based on deep learning model, audio processing technology and natural language processing technology; S2, multimodal information fusion: fusing the extracted visual features, audio features and text features by feature splicing or weighted summation or deep learning fusion to obtain a multimodal feature representation of the video; S3, video classification: classifying the multimodal feature representation of the video based on a deep learning classifier provided with a domain adversarial neural network to obtain the classification result of the video. The advantage is that by adopting a deep learning algorithm and combining the multimodal information fusion technology, effective feature representation can be automatically learned from the video, thereby improving the accuracy of video classification. In complex scenes, it can still maintain a high accuracy.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology at the intersection of computer vision and natural language processing, and in particular to a method and system for automatic classification of video content recognition. Background Art

[0002] At present, video content recognition and automatic classification technologies mostly use single-modal data processing strategies, such as visual feature extraction based on image sequences or analysis based on audio signals. However, these traditional methods have significant limitations in processing multimodal complex content. First, single-modal feature expression cannot fully capture the semantic diversity and contextual information in video content, especially in complex videos involving multiple scenes or multiple characters, where both recognition accuracy and classification robustness are limited. In addition, existing methods generally lack modeling of spatiotemporal dependencies and dynamic features, and cannot fully capture the temporal changes between consecutive frames, thus affecting the accurate analysis of dynamic plots.

[0003] For videos containing subtitles or descriptive text, traditional methods often fail to effectively integrate natural language processing (NLP) technology to make full use of text information, which leads to weak cross-modal feature correlation between visual, audio, and text modalities, making it difficult to improve classification results. Even though some methods introduce text features, most of the current multimodal fusion methods are based on simple splicing or weighted summation, and do not fully adopt the adaptive fusion mechanism of deep learning. Therefore, information loss between modalities is prone to occur during information integration. In addition, due to the lack of powerful multimodal feature fusion algorithms in existing technologies, the classification results lack contextual relevance and semantic integrity, making it difficult to meet the needs of high-precision automatic classification of complex video content. These technical defects leave much room for improvement in existing methods in multimodal fusion, temporal feature modeling, and cross-modal association processing. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for automatic classification of video content recognition, so as to solve the above-mentioned problems existing in the prior art.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for automatic classification of video content recognition comprises the following steps:

[0007] S1. Multimodal feature extraction: Extract visual features, audio features, and text features of video data based on deep learning models, audio processing technology, and natural language processing technology;

[0008] S2. Multimodal information fusion: The extracted visual features, audio features and text features are fused by feature concatenation, weighted summation or deep learning fusion to obtain a multimodal feature representation of the video;

[0009] S3. Video classification: Based on a deep learning classifier equipped with a domain adversarial neural network, the multimodal feature representation of the video is classified to obtain the classification result of the video.

[0010] Preferably, step S1 specifically includes the following contents:

[0011] S11. Use the pre-trained convolutional neural network model to extract visual features from the image sequence of the video data and obtain the feature vector of each frame; input the feature vectors of adjacent frames into the pre-trained LSTM model through time series modeling to extract the dynamic features of the video in the time dimension;

[0012] S12, using Mel frequency cepstrum coefficients to extract features from the audio signal of the video data to obtain audio feature representation of the video; and combining a multi-scale feature extraction algorithm to perform multi-level convolution operations on the MFCC data to obtain multi-level audio features;

[0013] S13. For video data with subtitles or text descriptions, natural language processing technology is used to extract features of the text descriptions to obtain text feature representations of the video.

[0014] Preferably, in step S11, a large amount of labeled video data is used to train the convolutional neural network model to obtain a pre-trained convolutional neural network model; during the training process, stochastic gradient descent and cross entropy loss function are used to perform parameter optimization;

[0015] In the pre-trained LSTM model, the output of the last fully connected layer of the model is selected as the visual feature representation, or the output of the intermediate layer of the model is fused to obtain a richer visual feature representation.

[0016] Preferably, step S13 specifically includes performing word segmentation, stop word removal, and stem extraction operations on the subtitles or text descriptions of the video, and using a bag-of-words model to represent the processed text as a vector, where each element of the vector represents the number of times a word appears in the text; or using a deep learning model to model the processed text to obtain a richer text feature representation.

[0017] Preferably, in step S2,

[0018] Feature splicing: simply splice the extracted visual, audio and text feature representations, assign different weights to each modality through the attention mechanism, and obtain the multimodal feature representation of the video;

[0019] Weighted summation: The extracted visual, audio, and text feature representations are weighted and summed to obtain a multimodal feature representation of the video; the weight values ​​are obtained through training or set based on experience;

[0020] Deep learning fusion: Use a deep learning model or a multimodal fusion network to fuse the extracted visual, audio, and text feature representations to obtain a multimodal feature representation of the video.

[0021] Preferably, step S3 specifically comprises:

[0022] A deep learning classifier is used to classify the fused multimodal features, and ensemble learning and cross-validation techniques are applied in the classification process to improve the accuracy and robustness of the classification. Adversarial training technology is introduced to make the classifier more robust to slight perturbations or adversarial samples of the input video, preventing it from being easily attacked. A domain adversarial neural network is added to the classifier, and the domain adversarial neural network is trained to help the classifier extract common features from videos in different fields, so that it can generalize across domains and adapt to different scenarios.

[0023] Preferably, before step S1, the step further includes:

[0024] S0. Video data preprocessing: decode the collected video data, convert it into image sequence and audio signal, and perform corresponding preprocessing operations on the image sequence and audio signal respectively.

[0025] Preferably, the step S3 further includes:

[0026] S4. Result output: Output the video classification results, including the category label and confidence of the video, and output the key frames and summary information of the video according to user needs, so that users can quickly understand the video content.

[0027] Preferably, step S4 specifically includes the following contents:

[0028] S41, outputting the category label of the video and its confidence information; if the confidence is lower than a preset threshold, prompting that reclassification is required;

[0029] S42, according to the classification results, grouping frames with similar features by using a K-means clustering algorithm, selecting the most representative frame in each group as a key frame; sorting the frames generated per second by confidence, retaining the top 10 key frames with the highest confidence, and forming a key frame sequence of the video content;

[0030] S43. Generate a video summary based on the text features of the video data using natural language generation technology, converting the text content into a brief description; the summary content includes an overview of the video category, key plots, and important scenes;

[0031] S44, storing the final output content in JSON format; the final output content includes the category label, confidence, key frame sequence, and summary text of the video.

[0032] The present invention also aims to provide a system for automatically classifying video content, which can implement the above-mentioned method. The system comprises:

[0033] Video acquisition module: collects video data from various data sources;

[0034] Video preprocessing module: decodes the collected video data, converts it into image sequence and audio signal, and performs corresponding preprocessing operations on the image sequence and audio signal respectively;

[0035] Multimodal feature extraction module: extracts visual features, audio features, and text features of video data based on deep learning models, audio processing technology, and natural language processing technology;

[0036] Multimodal information fusion module: It uses feature concatenation, weighted summation or deep learning fusion to fuse the extracted visual features, audio features and text features to obtain a multimodal feature representation of the video;

[0037] Video classification module: Based on a deep learning classifier equipped with a domain adversarial neural network, the multimodal feature representation of the video is classified to obtain the classification results of the video;

[0038] Result output module: Outputs video classification results, including the category label and confidence level of the video, and outputs the key frames and summary information of the video according to user needs, so that users can quickly understand the video content.

[0039] The beneficial effects of the present invention are: 1. Improving the accuracy of video classification: The present invention adopts a deep learning algorithm, combined with multimodal information fusion technology, and can automatically learn effective feature representations from videos, thereby improving the accuracy of video classification. In complex scenes, such as changes in illumination, changes in viewing angles, occlusion, etc., the present invention can still maintain a high accuracy. With reference to the multimodal learning framework of M2-CLIP and Act ionCLIP, the semantic information of multimodal information such as video and text is mined to further improve the classification accuracy. 2. Enhancing the robustness of video classification: By preprocessing and extracting features from video data, the present invention can effectively resist noise and interference and enhance the robustness of video classification. Even in the case of poor video quality, the present invention can accurately classify video content. 3. Improving computational efficiency: The present invention adopts efficient algorithm design and optimization technology, which can achieve rapid classification and processing on large-scale video data to meet real-time requirements. Compared with existing algorithms, the computational efficiency of the present invention has been significantly improved. For example, the use of pre-trained models and the "pre-training, prompting and fine-tuning" paradigm in Act ionCLIP can be used as a reference to reduce computational costs and improve computational efficiency. 4. Make full use of multimodal information: The present invention not only considers the visual features of the video, but also makes full use of information from other modalities such as audio and text to achieve multimodal information fusion, thereby improving the accuracy and comprehensiveness of video classification. M2-CLIP and Act ionCLIP also emphasize the importance of multimodal learning and provide us with a good reference. 5. Provide personalized video recommendation services: By analyzing the user's video viewing history and behavior data, the present invention can provide users with more personalized video recommendation services and improve user satisfaction and loyalty. 6. Wide scope of application: The present invention can be widely used in industries such as intelligent video content recognition, automatic classification and video data management, mainly to extract meaningful multimodal features (visual, audio, text) from the video content, and classify the video as a whole into a specific category. The present invention is also applicable to a variety of application scenarios such as video recommendation, content review, security monitoring, advertising placement, etc., to meet users' efficient and intelligent needs for video content management and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a method in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation methods described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] like Figure 1As shown, in this embodiment, an efficient video content recognition and automatic classification method is provided to solve the problems of low efficiency, insufficient accuracy, and difficulty in multimodal information fusion in video classification in the prior art. By performing multimodal feature extraction (including visual, audio, and text features) and information fusion on video data, the present invention can fully understand the video content and achieve accurate automatic classification. At the same time, the present invention adopts deep learning models and advanced preprocessing methods to improve the quality of video data and the effect of feature extraction, ensuring that the algorithm is efficient and robust when processing massive and diversified video data, and through a set of efficient multimodal feature extraction and fusion strategies, combined with deep learning models, it can automatically classify input videos. The method of the present invention specifically includes the following six parts:

[0043] 1. Video Data Collection

[0044] Collect video data from various data sources, including video websites, social media platforms, security monitoring systems, etc.

[0045] 2. Video Data Preprocessing

[0046] Decode the collected video data, convert it into image sequence and audio signal, and perform corresponding preprocessing operations on the image sequence and audio signal respectively. The specific contents include the following:

[0047] 2.1. First, the input video data is converted into image sequences and audio signals through a video decoder.

[0048] 2.2. Perform preprocessing operations such as denoising, enhancement, and cropping on the image sequence, and use denoising algorithms to improve image quality while reducing unnecessary computational burden.

[0049] 2.3. Optimize audio quality by using noise reduction and enhancement techniques such as adaptive filters.

[0050] In this embodiment, in order to solve the problem of data labeling, a semi-supervised learning method is combined to improve the scale and diversity of the data set by learning from a small amount of labeled data and combining it with unlabeled data for automatic labeling. A generative adversarial network (GAN) is used to generate synthetic data to expand the data set and enhance the generalization ability of the model, especially in areas where labeled data is scarce, for pre-training of subsequent models.

[0051] 3. Multimodal feature extraction

[0052] Based on deep learning models, audio processing technology and natural language processing technology, visual features, audio features and text features of video data are extracted respectively. Specifically, the following contents are included:

[0053] 3.1. Use the pre-trained convolutional neural network model (ResNet-50) to extract visual features from the image sequence of the video data and obtain the feature vector of each frame. Through time series modeling, the feature vectors of adjacent frames are input into the pre-trained LSTM model (Long Short-Term Memory Network) to extract the dynamic features of the video in the time dimension.

[0054] Specifically: a large amount of annotated video data is used to train the convolutional neural network model (ResNet-50) to learn effective visual feature representation; during the training process, random gradient descent and cross entropy loss function are used to optimize parameters, and finally a pre-trained convolutional neural network model is obtained; the pre-processed video image sequence is input into the trained convolutional neural network model to obtain the visual feature representation of the video. The output of the last fully connected layer of the model can be selected as the visual feature representation, or the output of the intermediate layer of the model can be fused to obtain a richer visual feature representation.

[0055] 3.2. Use audio processing technology, such as Mel-Frequency Cepstral Coefficient (MFCC), to extract features from the preprocessed audio signal and obtain the audio feature representation of the video. Combined with the multi-scale feature extraction algorithm, perform multi-level convolution operations on the MFCC data to obtain multi-level audio features.

[0056] 3.3. For video data with subtitles or text descriptions, natural language processing techniques, such as the bag-of-words model (BoW) and deep learning models, are used to extract features from the text descriptions and obtain text feature representations of the videos.

[0057] Specifically: perform preprocessing operations such as word segmentation, stop word removal, and stem extraction on the subtitles or text descriptions of the video. The bag-of-words model can be used to represent the preprocessed text as a vector, where each element of the vector represents the number of times a word appears in the text. A deep learning model can also be used. 如循环神经网络( RNN ) and long short-term memory network (LSTM) to model the preprocessed text and obtain richer text feature representation.

[0058] 4. Multimodal Information Fusion

[0059] The extracted visual features, audio features, and text features are fused by feature concatenation, weighted summation, or deep learning fusion to obtain a multimodal feature representation of the video.

[0060] (1) Feature splicing: The extracted visual, audio, and text feature representations are simply spliced, and different weights are assigned to each modality through the attention mechanism. The multimodal feature representation of the video is obtained. This method is simple and intuitive, but it may lead to too high a feature dimension and increase the amount of calculation.

[0061] (2) Weighted summation: The extracted visual, audio, and text feature representations are weighted and summed to obtain a multimodal feature representation of the video. The weights can be obtained through training or set based on experience. This method can effectively reduce the feature dimension, but the weights need to be set reasonably to ensure that the fused feature representation has good performance.

[0062] (3) Deep learning fusion: A deep learning model, the multimodal fusion network (MMFN), is used to fuse the extracted visual, audio, and text feature representations to obtain a multimodal feature representation of the video. The multimodal fusion network can automatically learn the relationship between features of different modalities, thereby achieving more effective information fusion. The multimodal adapter design of M2-CLIP is referenced to optimize the multimodal information fusion method.

[0063] 5. Video Classification

[0064] Based on a deep learning classifier equipped with a domain adversarial neural network, the multimodal feature representation of the video is classified to obtain the classification result of the video.

[0065] Specifically: A deep learning classifier is used to classify the fused multimodal features, and ensemble learning and cross-validation techniques are applied in the classification process to improve the accuracy and robustness of the classification.

[0066] The introduction of adversarial training technology makes the classifier more robust to slight perturbations of the input video or adversarial samples, preventing it from being easily attacked.

[0067] In order to improve the generalization ability of the model in different fields, a domain adversarial neural network (DANN) is added to the classifier. By training the domain adversarial neural network, the classifier is helped to extract common features from videos in different fields, so that it can generalize across domains and adapt to different scenarios (such as medical, security, entertainment, etc.).

[0068] 6. Result Output

[0069] Output video classification results, including video category labels, confidence, and output video key frames and summary information according to user needs, so that users can quickly understand the video content. Specifically, it includes the following:

[0070] 6.1. Output the category label of the video and its confidence information (e.g. the confidence of the label "Entertainment" is 94%); if the confidence is lower than the preset threshold, it will prompt that reclassification is needed to improve the reliability of the classification.

[0071] 6.2. Based on the classification results, the frames with similar features are grouped using the K-means clustering algorithm, and the most representative frames in each group are selected as key frames. The frames generated per second are sorted by confidence, and the top 10 key frames with the highest confidence are retained to form a key frame sequence of the video content.

[0072] 6.3. Use natural language generation technology to generate video summaries based on the text features of video data, converting text content into brief descriptions; the summary content includes an overview of the video category, key plots, and important scenes, making it easier for users to quickly understand the video content.

[0073] 6.4. The final output includes: video category label, confidence, key frame sequence, summary text, stored in JSON format for subsequent retrieval and display.

[0074] In this embodiment, the automatic classification process can be automatically updated and optimized: as more video data is input, the model can be automatically updated and optimized. Through online learning and adaptive training mechanisms, the automatic classification function can continuously improve its accuracy and robustness to adapt to different video content and classification tasks. The above method is implemented using an efficient computing architecture (such as GPU acceleration, distributed computing, etc.) to ensure efficient classification speed when processing large-scale video streams and support real-time video stream classification tasks.

[0075] In this embodiment, in order to implement the above method, a system for automatic classification of video content recognition is provided, which refers to the system architecture of M2-CLIP and Act ionCLIP, integrates advanced multimodal learning components, and improves system performance. The system includes:

[0076] 1. Video acquisition module: collects video data from various data sources and transmits it to the preprocessing module.

[0077] 2. Video preprocessing module: decode the collected video data, convert it into image sequence and audio signal, and perform corresponding preprocessing operations on the image sequence and audio signal respectively.

[0078] 3. Multimodal feature extraction module: Based on deep learning models, audio processing technology and natural language processing technology, the visual features, audio features and text features of video data are extracted respectively.

[0079] 4. Multimodal information fusion module: The extracted visual features, audio features and text features are fused by feature splicing, weighted summation or deep learning fusion to obtain a multimodal feature representation of the video.

[0080] 5. Video classification module: Based on the deep learning classifier equipped with a domain adversarial neural network, the multimodal feature representation of the video is classified to obtain the classification results of the video.

[0081] 6. Result output module: Outputs video classification results, including the category label and confidence level of the video, and outputs the key frames and summary information of the video according to user needs, so that users can quickly understand the video content.

[0082] By adopting the above technical solution disclosed in the present invention, the following beneficial effects are obtained:

[0083] The present invention provides a method and system for automatic classification of video content recognition, which improves the accuracy of video classification: the present invention adopts a deep learning algorithm, combined with multimodal information fusion technology, and can automatically learn effective feature representation from the video, thereby improving the accuracy of video classification. In complex scenes, such as changes in illumination, changes in viewing angle, occlusion, etc., the present invention can still maintain a high accuracy. With reference to the multimodal learning framework of M2-CL IP and ActiONCL IP, the semantic information of multimodal information such as video and text is mined to further improve the classification accuracy. Enhance the robustness of video classification: By preprocessing and extracting features of video data, the present invention can effectively resist noise and interference and enhance the robustness of video classification. Even in the case of poor video quality, the present invention can accurately classify video content. Improve computing efficiency: The present invention adopts efficient algorithm design and optimization technology, which can achieve rapid classification and processing on large-scale video data and meet real-time requirements. Compared with existing algorithms, the computing efficiency of the present invention has been significantly improved. For example, the pre-training model and the "pre-training, prompting and fine-tuning" paradigm used in ActiONCL IP can be used for reference to reduce computing costs and improve computing efficiency. Make full use of multimodal information: The present invention not only considers the visual features of the video, but also makes full use of information from other modes such as audio and text to achieve multimodal information fusion, thereby improving the accuracy and comprehensiveness of video classification. M2-CLIP and Acti onCL IP also emphasize the importance of multimodal learning, providing us with a good reference. Provide personalized video recommendation services: By analyzing the user's video viewing history and behavior data, the present invention can provide users with more personalized video recommendation services and improve user satisfaction and loyalty. Wide scope of application: The present invention can be widely used in industries such as intelligent video content recognition, automatic classification and video data management, mainly to extract meaningful multimodal features (visual, audio, text) from video content, and classify the video as a whole into a specific category. The present invention is also applicable to a variety of application scenarios such as video recommendation, content review, security monitoring, advertising placement, etc., to meet users' efficient and intelligent needs for video content management and classification.

[0084] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention.

Claims

1. A method for automatic classification of video content recognition, characterized in that: The following steps are included: S1. Multimodal feature extraction: Extract visual features, audio features, and text features of video data based on deep learning models, audio processing technology, and natural language processing technology; S2. Multimodal information fusion: The extracted visual features, audio features and text features are fused by feature concatenation, weighted summation or deep learning fusion to obtain a multimodal feature representation of the video; S3. Video classification: Based on a deep learning classifier equipped with a domain adversarial neural network, the multimodal feature representation of the video is classified to obtain the classification result of the video.

2. The method for automatic classification of video content recognition according to claim 1, characterized in that: Step S1 specifically includes the following contents: S11. Use the pre-trained convolutional neural network model to extract visual features from the image sequence of the video data and obtain the feature vector of each frame; input the feature vectors of adjacent frames into the pre-trained LSTM model through time series modeling to extract the dynamic features of the video in the time dimension; S12, using Mel frequency cepstrum coefficients to extract features from the audio signal of the video data to obtain audio feature representation of the video; and combining a multi-scale feature extraction algorithm to perform multi-level convolution operations on the MFCC data to obtain multi-level audio features; S13. For video data with subtitles or text descriptions, natural language processing technology is used to extract features of the text descriptions to obtain text feature representations of the video.

3. The method for automatic classification of video content recognition according to claim 2, characterized in that: In step S11, a large amount of annotated video data is used to train the convolutional neural network model to learn effective visual feature representation. During the training process, random gradient descent and cross entropy loss function are used to optimize parameters, and finally a pre-trained convolutional neural network model is obtained; Select the output of the last fully connected layer of the model as the visual feature representation, or fuse the output of the intermediate layers of the model to obtain a richer visual feature representation.

4. The method for automatic classification of video content recognition according to claim 2, characterized in that: Step S13 specifically includes: performing word segmentation, stop word removal, and stem extraction operations on the subtitles or text descriptions of the video, and using a bag-of-words model to represent the processed text as a vector, where each element of the vector represents the number of times a word appears in the text; or using a deep learning model to model the processed text to obtain a richer text feature representation.

5. The method for automatic classification of video content recognition according to claim 1, characterized in that: In step S2, Feature splicing: simply splice the extracted visual, audio and text feature representations, assign different weights to each modality through the attention mechanism, and obtain the multimodal feature representation of the video; Weighted summation: Perform weighted summation on the extracted visual, audio, and text feature representations to obtain a multimodal feature representation of the video; The weight values ​​are obtained through training or set based on experience; Deep learning fusion: Use a deep learning model or a multimodal fusion network to fuse the extracted visual, audio, and text feature representations to obtain a multimodal feature representation of the video.

6. The method for automatic classification of video content recognition according to claim 1, characterized in that: Step S3 specifically comprises: A deep learning classifier is used to classify the fused multimodal features, and ensemble learning and cross-validation techniques are applied in the classification process to improve the accuracy and robustness of the classification. Adversarial training technology is introduced to make the classifier more robust to slight perturbations or adversarial samples of the input video, preventing it from being easily attacked. A domain adversarial neural network is added to the classifier, and the domain adversarial neural network is trained to help the classifier extract common features from videos in different fields, so that it can generalize across domains and adapt to different scenarios.

7. The method for automatic classification of video content recognition according to claim 1, characterized in that: Before step S1, the method further includes: S0. Video data preprocessing: decode the collected video data, convert it into image sequence and audio signal, and perform corresponding preprocessing operations on the image sequence and audio signal respectively.

8. The method for automatic classification of video content recognition according to claim 1, characterized in that: After step S3, the step further includes: S4. Result output: Output the video classification results, including the category label and confidence of the video, and output the key frames and summary information of the video according to user needs, so that users can quickly understand the video content.

9. The method for automatic classification of video content recognition according to claim 8, characterized in that: Step S4 specifically includes the following contents: S41, outputting the category label of the video and its confidence information; if the confidence is lower than a preset threshold, prompting that reclassification is required; S42, according to the classification results, grouping frames with similar features by using a K-means clustering algorithm, selecting the most representative frame in each group as a key frame; sorting the frames generated per second by confidence, retaining the top 10 key frames with the highest confidence, and forming a key frame sequence of the video content; S43. Generate a video summary based on the text features of the video data using natural language generation technology, converting the text content into a brief description; the summary content includes an overview of the video category, key plots, and important scenes; S44, storing the final output content in JSON format; the final output content includes the category label, confidence, key frame sequence, and summary text of the video.

10. A system for automatic classification of video content recognition, characterized in that: The system can implement the method described in any one of claims 1 to 9, and the system includes: Video acquisition module: collects video data from various data sources; Video preprocessing module: decodes the collected video data, converts it into image sequence and audio signal, and performs corresponding preprocessing operations on the image sequence and audio signal respectively; Multimodal feature extraction module: extracts visual features, audio features, and text features of video data based on deep learning models, audio processing technology, and natural language processing technology; Multimodal information fusion module: It uses feature concatenation, weighted summation or deep learning fusion to fuse the extracted visual features, audio features and text features to obtain a multimodal feature representation of the video; Video classification module: Based on a deep learning classifier equipped with a domain adversarial neural network, the multimodal feature representation of the video is classified to obtain the classification results of the video; Result output module: Outputs video classification results, including the category label and confidence level of the video, and outputs the key frames and summary information of the video according to user needs, so that users can quickly understand the video content.

Images