System for realizing automatic scene classification of video picture based on artificial intelligence

The AI-based automatic scene classification system for video footage solves the problems of poor feature fusion and insufficient semantic parsing depth, achieving efficient and accurate video scene classification and semantic parsing, supporting edge computing and anomaly detection, and ensuring privacy and security.

CN120876943APending Publication Date: 2025-10-31HANGZHOU XIFU INFORMATION TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202510944108.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-09
Publication Date
2025-10-31

AI Technical Summary

Technical Problem

Existing AI-based automatic scene classification systems for video footage suffer from poor feature fusion and insufficient depth of scene semantic parsing.

Method used

An AI-based automatic scene classification system for video footage is adopted, including a data receiving module, a video and audio preprocessing module, a feature extraction unit, a dynamic weighting module, a feature fusion module, a scene classification module, a scene semantic parsing module, and a result output module. The system processes video data through dynamic image stabilization and adaptive illumination, extracts features using EfficientNet, ResNet, I3D, C3D, VGGish, and Librosa tools, and employs a multimodal large model for scene understanding and semantic parsing.

Benefits of technology

It achieves efficient fusion of video image features and deep semantic parsing, improves the accuracy and efficiency of scene classification, supports edge computing and anomaly detection, ensures privacy and security, and provides user-friendly output results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120876943A_ABST
    Figure CN120876943A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a system for realizing automatic scene classification of video pictures based on artificial intelligence. Comprising a data receiving module, a video preprocessing unit, an audio preprocessing module, an encryption transmission module, a feature extraction unit, a dynamic weighting module, a feature fusion module, a scene classification module, a scene semantic analysis module, a result output module and a feedback module. The video preprocessing unit is connected with the data receiving module, the audio preprocessing unit is connected with the video preprocessing unit, and the scene classification module understands scenes of received image picture feature vectors based on a multi-modal large model, automatically classifies a large number of pictures according to the understanding, determines the scene category to which a video belongs, and stores the video. In this way, the technical problems that in the prior art, a video image automatic scene classification system based on artificial intelligence is poor in feature fusion effect and insufficient in scene semantic analysis depth are solved, and quality improvement and efficiency improvement are achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a system for automatically classifying video scenes based on artificial intelligence. Background Technology

[0002] With the rapid development of information technology, video data has experienced explosive growth in various fields such as the internet, social media, security monitoring, and the entertainment industry. Faced with massive amounts of video data, efficiently and accurately classifying video scenes automatically has become an extremely challenging task. Traditional video scene classification methods mainly rely on manual annotation or classification algorithms based on simple rules. This approach is not only inefficient but also has limited accuracy and generalization ability, making it difficult to meet the needs of rapid processing and accurate classification of large-scale video data in practical applications.

[0003] With the continuous advancement of artificial intelligence technology, especially the widespread application of deep learning in computer vision and speech recognition, AI-based automatic scene classification technology for video footage has gradually become a research hotspot. However, existing AI-based automatic scene classification systems for video footage suffer from poor feature fusion and insufficient depth of scene semantic parsing. Summary of the Invention

[0004] The purpose of this invention is to provide a system for automatic scene classification of video footage based on artificial intelligence, aiming to solve the technical problems of poor feature fusion effect and insufficient depth of scene semantic parsing in existing artificial intelligence-based automatic scene classification systems for video footage.

[0005] To achieve the above objectives, the present invention employs a system for automatic scene classification of video images based on artificial intelligence, comprising a data receiving module, a video preprocessing unit, an audio preprocessing module, an encrypted transmission module, a feature extraction unit, a dynamic weighting module, a feature fusion module, a scene classification module, a scene semantic parsing module, a result output module, and a feedback module.

[0006] The video preprocessing unit is connected to the data receiving module, the audio preprocessing unit is connected to the video preprocessing unit, the encrypted transmission module is connected to both the audio preprocessing module and the feature extraction unit, the feature fusion module is located between the feature extraction unit and the scene classification module, the scene semantic parsing module is connected to the scene classification module, the result output module is connected to the scene classification module, the feedback module is connected to the result output module, and the dynamic weighting module is connected to the feature extraction module.

[0007] The data receiving module is used to receive data from different data sources, decrypt the data to obtain the original video, and then transmit it to the video preprocessing unit.

[0008] The video preprocessing unit is used to preprocess the received video data;

[0009] The audio preprocessing unit is used to extract audio streams from video data, perform preprocessing operations on the audio data, and transmit it to the feature extraction unit through the encrypted transmission module.

[0010] The feature extraction unit is used to extract various features from the preprocessed video and audio data, and the dynamic weighting module dynamically adjusts the weight of each feature in the feature extraction unit.

[0011] The feature fusion module is used to fuse video features and audio features to generate a comprehensive feature vector. The scene classification module combines the fused feature vector with the scene understanding of the received image feature vector based on the multimodal large model, and automatically performs a large number of image classifications based on the understanding to determine the scene category to which the video belongs.

[0012] The scene semantic parsing module is used to further perform semantic parsing on the scene classification results and extract key information from the scene;

[0013] The result output module is used to output the results of scene classification and semantic parsing in a user-friendly manner.

[0014] The video preprocessing unit includes a dynamic image stabilization module and an illumination adaptive module.

[0015] The dynamic image stabilization module uses the SIFT feature matching algorithm to achieve motion compensation through feature point trajectory fitting.

[0016] The adaptive lighting module combines the CLAHE algorithm with a brightness compensation model based on depth estimation to automatically adjust the lighting conditions of video frames.

[0017] The feature extraction unit includes a visual feature extraction module, a temporal feature extraction module, and an audio feature extraction module.

[0018] The visual feature extraction module uses EfficientNet or ResNet deep neural networks to extract spatial features from video frames.

[0019] The temporal feature extraction module uses I3D or C3D networks to extract temporal features between video frames;

[0020] The audio feature extraction module uses VGGish or Librosa tools to extract features from audio data.

[0021] The system for automatic scene classification of video footage based on artificial intelligence also includes a model optimization module and an edge computing module. The model optimization module is connected to both the scene classification module and the feedback module, and the edge computing module is connected to the data receiving module.

[0022] The model optimization module uses a deep learning model optimization algorithm to optimize and train the scene classification model in the scene classification module.

[0023] The system for automatic scene classification of video footage based on artificial intelligence also includes a human-computer interaction module, which is connected to the result output module.

[0024] The system also includes a model evaluation and verification module, which is connected to the model optimization module.

[0025] The model evaluation and verification module periodically evaluates and verifies the performance of the model optimized by the model optimization module, using metrics (such as accuracy, recall, F1 score, etc.) to quantify the model effect, providing clear reference for model optimization and ensuring that the model continuously outputs high-quality classification results.

[0026] The system for automatic scene classification of video footage based on artificial intelligence also includes an anomaly detection and processing module and a log recording module.

[0027] The anomaly detection and processing module is connected to the data receiving module; the log recording module is connected to the scene classification module.

[0028] The anomaly detection and processing module is used to identify abnormal events in video data (such as screen freezing, audio interruption, scene abrupt change, etc.). When an anomaly is detected, the abnormal frame is automatically marked, or remedial measures are taken (such as skipping the abnormal frame or going back to re-acquire data).

[0029] The system for automatic scene classification of video footage based on artificial intelligence also includes a privacy protection enhancement module, which is connected to the encrypted transmission module.

[0030] This invention discloses a system for automatic scene classification of video footage based on artificial intelligence. In practical use, the data receiving module receives data from different data sources, decrypts it to obtain the original video, and then transmits it to the video preprocessing unit for preprocessing. The audio preprocessing unit extracts the audio stream from the video data, preprocesses the audio data, and transmits it to the feature extraction unit via the encrypted transmission module. The feature extraction unit extracts various features from the preprocessed video and audio data and dynamically adjusts the weights of each feature in the feature extraction unit using the dynamic weighting module. The feature fusion module combines the video features and audio features... The features are fused to generate a comprehensive feature vector. The scene classification module combines the fused feature vector with the scene understanding of the received image feature vector based on a multimodal large model. Based on the understanding, it automatically performs a large number of image classifications to determine the scene category to which the video belongs. Subsequently, the scene semantic parsing module performs further semantic parsing on the scene classification results to extract key information in the scene. Finally, the result output module outputs the scene classification and semantic parsing results in a user-friendly manner. This approach solves the technical problems of poor feature fusion effect and insufficient scene semantic parsing depth in existing AI-based automatic scene classification systems for video images, thereby improving quality and efficiency. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the first embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the second embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram of the third embodiment of the present invention.

[0035] 101-Data Receiving Module, 102-Video Preprocessing Unit, 103-Audio Preprocessing Module, 104-Encrypted Transmission Module, 105-Feature Extraction Unit, 106-Dynamic Weighting Module, 107-Feature Fusion Module, 108-Scene Classification Module, 109-Scene Semantic Parsing Module, 110-Result Output Module, 111-Feedback Module, 112-Model Optimization Module, 113-Edge Computing Module, 114-Dynamic Anti-shake Module, 115-Illumination Adaptation Module, 116-Visual Feature Extraction Module, 117-Temporal Feature Extraction Module, 118-Audio Feature Extraction Module, 201-Human-Computer Interaction Module, 202-Model Evaluation and Verification Module, 203-Anomaly Detection and Handling Module, 204-Log Recording Module, 205-Privacy Protection Enhancement Module, 301-Login Module, 302-Authentication Module, 303-Permission Management Module. Detailed Implementation

[0036] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0037] The first embodiment of this application is as follows:

[0038] Please see Figure 1 , Figure 1 This is a schematic diagram of the first embodiment of the present invention.

[0039] This invention provides a system for automatic scene classification of video footage based on artificial intelligence, comprising a data receiving module 101, a video preprocessing unit 102, an audio preprocessing module 103, an encrypted transmission module 104, a feature extraction unit 105, a dynamic weighting module 106, a feature fusion module 107, a scene classification module 108, a scene semantic parsing module 109, a result output module 110, a feedback module 111, a model optimization module 112, and an edge computing module 113. The video preprocessing unit 102 includes a dynamic image stabilization module 114 and an illumination adaptation module 115; the feature extraction unit 105 includes a visual feature extraction module 116, a temporal feature extraction module 117, and an audio feature extraction module 118. The aforementioned solution solves the technical problems of poor feature fusion effect and insufficient scene semantic parsing depth in existing artificial intelligence-based automatic scene classification systems for video footage.

[0040] In this specific embodiment, the data receiving module 101 is used to receive data from different data sources, decrypt the data to obtain the original video, and then transmit it to the video preprocessing unit 102.

[0041] The video preprocessing unit 102 is used to perform preprocessing operations on the received video data, such as decoding, denoising, color space conversion (e.g., from RGB to YUV or HSV space to facilitate subsequent feature extraction), and frame rate adjustment (unifying the video frame rate for easier processing), in order to optimize the quality of the video data and improve the accuracy and efficiency of subsequent feature extraction.

[0042] The audio preprocessing unit is used to extract audio streams from video data and perform preprocessing operations such as noise reduction, audio enhancement, and audio format conversion on the audio data, so as to extract useful feature information from the audio, such as voice content and background music type, to assist in scene classification, and transmit it to the feature extraction unit 105 through the encrypted transmission module 104.

[0043] The feature extraction unit 105 is used to extract various features from the preprocessed video and audio data. The dynamic weighting module 106 dynamically adjusts the weight of each feature in the feature extraction unit 105. The dynamic weighting module 106 introduces a dual attention mechanism, including spatial attention (SA) and temporal attention (TA), to dynamically weight key features and improve the accuracy of feature extraction.

[0044] The feature fusion module 107 is used to fuse video features and audio features to generate a comprehensive feature vector. The scene classification module 108 combines the fused feature vector with the scene understanding of the received image feature vector based on the multimodal large model, and automatically performs a large number of image classifications based on the understanding to determine the scene category to which the video belongs.

[0045] The scene semantic parsing module 109 is used to further perform semantic parsing on the scene classification results and extract key information in the scene;

[0046] The result output module 110 is used to output the results of scene classification and semantic parsing in a user-friendly manner.

[0047] The video preprocessing unit 102 is connected to the data receiving module 101, the audio preprocessing unit is connected to the video preprocessing unit 102, the encrypted transmission module 104 is connected to both the audio preprocessing module 103 and the feature extraction unit 105, the feature fusion module 107 is located between the feature extraction unit 105 and the scene classification module 108, the scene semantic parsing module 109 is connected to the scene classification module 108, the result output module 110 is connected to the scene classification module 108, the feedback module 111 is connected to the result output module 110, and the dynamic weighting module 106 is connected to the feature extraction module. In practical use, the data receiving module 101 receives data from different data sources, decrypts it to obtain the original video, and then transmits it to the video preprocessing unit 102 for preprocessing. The audio preprocessing unit extracts the audio stream from the video data, preprocesses the audio data, and transmits it through... The encrypted transmission module 104 transmits data to the feature extraction unit 105; the feature extraction unit 105 extracts various features from the preprocessed video and audio data, and dynamically adjusts the weights of each feature in the feature extraction unit 105 through the dynamic weighting module 106; the feature fusion module 107 fuses the video features and audio features to generate a comprehensive feature vector; the scene classification module 108 combines the fused feature vector with the scene understanding of the received image feature vector based on a multimodal large model, and automatically performs a large number of image classifications based on the understanding to determine the scene category to which the video belongs; subsequently, the scene semantic parsing module 109 performs further semantic parsing on the scene classification results to extract key information in the scene; finally, the result output module 110 outputs the scene classification and semantic parsing results in a user-friendly manner, thereby solving the technical problems of poor feature fusion effect and insufficient scene semantic parsing depth in existing artificial intelligence-based automatic scene classification systems for video images.

[0048] Secondly, the dynamic image stabilization module 114 adopts the SIFT feature matching algorithm to achieve motion compensation through feature point trajectory fitting, thereby improving video stability.

[0049] The illumination adaptive module 115 combines the CLAHE algorithm with a brightness compensation model for depth estimation to automatically adjust the illumination conditions of video frames, making them more suitable for feature extraction.

[0050] Meanwhile, the visual feature extraction module 116 uses EfficientNet or ResNet deep neural networks to extract spatial features from video frames;

[0051] The temporal feature extraction module 117 uses an I3D or C3D network to extract temporal features between video frames;

[0052] The audio feature extraction module 118 uses VGGish or Librosa tools to extract features from the audio data.

[0053] In addition, the model optimization module 112 is connected to the scene classification module 108 and the feedback module 111, and the edge computing module 113 is connected to the data receiving module 101;

[0054] The model optimization module 112 uses a deep learning model optimization algorithm to optimize and train the scene classification model in the scene classification module 108.

[0055] When the classification result is inaccurate or the model performance deteriorates, the model optimization module 112 triggers an adaptive learning mechanism to update the model. Based on the classification result and feedback information, it updates the model parameters online to improve the system's dynamic adaptability. Specifically, it receives the classification result and feedback information output by the scene classification module 108, employs incremental learning, transfer learning, or meta-learning strategies to achieve online model updating and optimization, and outputs the updated model parameters to the scene classification module 108. The edge computing module 113 performs edge computing processing on the video data, preprocessing and extracting features from some data on edge devices close to the data source. This reduces data transmission volume and the computational burden on the cloud server, improving the system's response speed and real-time performance.

[0056] Using the AI-based system for automatic scene classification of video footage according to this embodiment, in practical use, the data receiving module 101 receives data from different data sources, decrypts it to obtain the original video, and then transmits it to the video preprocessing unit 102 for preprocessing. The audio preprocessing unit extracts the audio stream from the video data, preprocesses the audio data, and transmits it to the feature extraction unit 105 through the encrypted transmission module 104. The feature extraction unit 105 extracts various features from the preprocessed video and audio data, and dynamically adjusts the weights of each feature in the feature extraction unit 105 through the dynamic weighting module 106. The feature fusion module 107... The video and audio features are fused to generate a comprehensive feature vector. The scene classification module 108 combines the fused feature vector with a multimodal large model to understand the scene of the received image feature vector and automatically classifies a large number of images to determine the scene category to which the video belongs. Subsequently, the scene semantic parsing module 109 performs further semantic parsing on the scene classification results to extract key information in the scene. Finally, the result output module 110 outputs the scene classification and semantic parsing results in a user-friendly manner. This approach solves the technical problems of poor feature fusion and insufficient scene semantic parsing depth in existing AI-based automatic scene classification systems for video images, thereby improving quality and efficiency.

[0057] The second embodiment of this application is as follows:

[0058] Please see Figure 2 , Figure 2 This is a schematic diagram of the second embodiment of the present invention.

[0059] Based on the first embodiment, the present invention provides a system for automatic scene classification of video images based on artificial intelligence, which also includes a human-computer interaction module 201, a model evaluation and verification module 202, an anomaly detection and processing module 203, a log recording module 204, and a privacy protection enhancement module 205.

[0060] In this specific embodiment, the human-computer interaction module 201 is connected to the result output module 110. The human-computer interaction module 201 transforms the scene classification results generated by the system into visual charts, voice broadcasts, or interactive interfaces, reducing the user's understanding threshold. Simultaneously, the human-computer interaction module 201 provides a user feedback interface, allowing users to manually annotate or correct the classification results; the feedback data is used for model optimization. Furthermore, it supports user-defined queries, enabling secondary filtering of results.

[0061] The model evaluation and verification module 202 is connected to the model optimization module 112.

[0062] The model evaluation and verification module 202 periodically evaluates and verifies the performance of the model optimized by the model optimization module 112, using indicators (such as accuracy, recall, F1 score, etc.) to quantify the model effect, providing a clear reference for model optimization and ensuring that the model continuously outputs high-quality classification results.

[0063] Secondly, the anomaly detection and processing module 203 is connected to the data receiving module 101; the log recording module 204 is connected to the scene classification module 108.

[0064] The anomaly detection and processing module 203 is used to identify abnormal events in video data (such as screen freezing, audio interruption, scene change, etc.). When an anomaly is detected, it automatically marks the abnormal frame or takes remedial measures (such as skipping the abnormal frame or re-acquiring data).

[0065] The log recording module 204 uses Apache HBase (column-oriented storage) or Elasticsearch (document storage) to record the work logs of the scene classification module 108, and segments the logs by timestamp or video ID (e.g., generating one segment per hour) to distribute storage pressure. Through its highly scalable, high-performance, and highly reliable design, the log recording module 204 significantly improves the log management capabilities of the video scene classification system.

[0066] Furthermore, the privacy protection enhancement module 205 significantly improves the privacy security and compliance of the video scene classification system through functions such as data anonymization and differential privacy, effectively protecting user privacy from being leaked and reducing the risk of data leakage.

[0067] In this embodiment, a system for automatic scene classification of video footage based on artificial intelligence is used. In practical application, the human-computer interaction module 201 is connected to the result output module 110. The human-computer interaction module 201 transforms the scene classification results generated by the system into visual charts, voice broadcasts, or interactive interfaces, reducing the user's understanding threshold. Simultaneously, the human-computer interaction module 201 provides a user feedback interface, allowing users to manually annotate or correct the classification results; the feedback data is used for model optimization. Furthermore, it supports user-defined queries, enabling secondary filtering of results.

[0068] The anomaly detection and processing module 203 is used to identify abnormal events in video data (such as screen freezing, audio interruption, scene change, etc.). When an anomaly is detected, it automatically marks the abnormal frame or takes remedial measures (such as skipping the abnormal frame or re-acquiring data).

[0069] The log recording module 204 uses Apache HBase (column-oriented storage) or Elasticsearch (document storage) to record the work logs of the scene classification module 108, and segments the logs by timestamp or video ID (e.g., generating one segment per hour) to distribute storage pressure. Through its highly scalable, high-performance, and highly reliable design, the log recording module 204 significantly improves the log management capabilities of the video scene classification system.

[0070] The third embodiment of this application is as follows:

[0071] Please see Figure 3 , Figure 3 This is a schematic diagram of the third embodiment of the present invention.

[0072] Based on the second embodiment, the present invention provides a system for automatic scene classification of video images based on artificial intelligence, which also includes a login module 301, an identity verification module 302 and an access control module 303.

[0073] In this specific embodiment, the login module 301 is connected to the human-computer interaction module 201, and the identity verification module 302 is connected to the login module 301. The login module 301 is directly connected to the human-computer interaction module 201, allowing users to access system functions through the login module 301, simplifying the operation process and improving the user experience. The identity verification module 302 adopts a multi-factor authentication mechanism (such as password + dynamic verification code) to effectively prevent unauthorized users from logging in, ensuring that the system is only open to authorized users and reducing security risks.

[0074] The permission management module 303 is connected to the identity verification module 302. The permission management module 303 and the identity verification module 302 work together to dynamically allocate functional permissions based on user roles (such as administrators and ordinary users). For example, ordinary users can only view scene classification results and cannot access the original video data to ensure data isolation and compliance.

[0075] The system for automatic scene classification of video footage based on artificial intelligence, as described in this embodiment, allows users to access system functions through the login module 301, simplifying the operation process and improving user experience. The identity verification module 302 employs a multi-factor authentication mechanism (such as password + dynamic verification code) to effectively prevent unauthorized user login, ensuring that the system is only open to authorized users and reducing security risks. The permission management module 303 is connected to the identity verification module 302; the permission management module 303 and the identity verification module 302 work together to dynamically allocate functional permissions based on user roles (such as administrator, ordinary user). For example, ordinary users can only view scene classification results and cannot access the original video data, ensuring data isolation and compliance.

[0076] This invention employs the video preprocessing unit 102 and the audio preprocessing module 103 to perform noise reduction and enhancement operations on video and audio data respectively, avoiding cross-interference. Simultaneously, the dynamic weighting module 106 dynamically adjusts the weights of video and audio features according to the scene (e.g., motion features have higher weights in sports events, and audio content has higher weights in news reports).

[0077] The feature fusion module 107 uses deep learning algorithms (such as attention mechanisms and graph neural networks) to deeply fuse video and audio features, rather than simply splicing them together.

[0078] After scene classification, the independent scene semantic parsing module 109 is used to further analyze the key information in the scene (such as shopper behavior and product display in the "shopping mall" scene). Combined with video (visual elements) and audio (voice and background music) information, the scene semantics are comprehensively analyzed (such as atmosphere and singer style in the concert scene). Furthermore, the feedback module 111 collects user feedback on the semantic parsing results to continuously optimize the parsing model.

[0079] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A system for automatic scene classification of video footage based on artificial intelligence, characterized in that, It includes a data receiving module, a video preprocessing unit, an audio preprocessing module, an encrypted transmission module, a feature extraction unit, a dynamic weighting module, a feature fusion module, a scene classification module, a scene semantic parsing module, a result output module, and a feedback module; The video preprocessing unit is connected to the data receiving module, the audio preprocessing unit is connected to the video preprocessing unit, the encrypted transmission module is connected to both the audio preprocessing module and the feature extraction unit, the feature fusion module is located between the feature extraction unit and the scene classification module, the scene semantic parsing module is connected to the scene classification module, the result output module is connected to the scene classification module, the feedback module is connected to the result output module, and the dynamic weighting module is connected to the feature extraction module. The data receiving module is used to receive data from different data sources, decrypt the data to obtain the original video, and then transmit it to the video preprocessing unit. The video preprocessing unit is used to preprocess the received video data; The audio preprocessing unit is used to extract audio streams from video data, perform preprocessing operations on the audio data, and transmit it to the feature extraction unit through the encrypted transmission module. The feature extraction unit is used to extract various features from the preprocessed video and audio data, and the dynamic weighting module dynamically adjusts the weight of each feature in the feature extraction unit. The feature fusion module is used to fuse video features and audio features to generate a comprehensive feature vector. The scene classification module combines the fused feature vector with the scene understanding of the received image feature vector based on the multimodal large model, and automatically performs a large number of image classifications based on the understanding to determine the scene category to which the video belongs. The scene semantic parsing module is used to further perform semantic parsing on the scene classification results and extract key information from the scene; The result output module is used to output the results of scene classification and semantic parsing in a user-friendly manner.

2. The system for automatic scene classification of video footage based on artificial intelligence as described in claim 1, characterized in that, The video preprocessing unit includes a dynamic image stabilization module and an illumination adaptive module; The dynamic image stabilization module uses the SIFT feature matching algorithm to achieve motion compensation through feature point trajectory fitting. The adaptive lighting module combines the CLAHE algorithm with a brightness compensation model based on depth estimation to automatically adjust the lighting conditions of video frames.

3. The system for automatic scene classification of video footage based on artificial intelligence as described in claim 2, characterized in that, The feature extraction unit includes a visual feature extraction module, a temporal feature extraction module, and an audio feature extraction module; The visual feature extraction module uses EfficientNet or ResNet deep neural networks to extract spatial features from video frames. The temporal feature extraction module uses I3D or C3D networks to extract temporal features between video frames; The audio feature extraction module uses VGGish or Librosa tools to extract features from audio data.

4. The system for automatic scene classification of video footage based on artificial intelligence as described in claim 3, characterized in that, The system for automatic scene classification of video footage based on artificial intelligence also includes a model optimization module and an edge computing module. The model optimization module is connected to the scene classification module and the feedback module, and the edge computing module is connected to the data receiving module. The model optimization module uses a deep learning model optimization algorithm to optimize and train the scene classification model in the scene classification module.

5. The system for automatic scene classification of video footage based on artificial intelligence as described in claim 4, characterized in that, The system for automatic scene classification of video footage based on artificial intelligence also includes a human-computer interaction module, which is connected to the result output module.

6. The system for automatic scene classification of video footage based on artificial intelligence as described in claim 5, characterized in that, The system also includes a model evaluation and verification module, which is connected to the model optimization module. The model evaluation and verification module periodically evaluates and verifies the performance of the model optimized by the model optimization module.

7. The system for automatic scene classification of video footage based on artificial intelligence as described in claim 6, characterized in that, The system for automatic scene classification of video footage based on artificial intelligence also includes an anomaly detection and processing module and a log recording module; The anomaly detection and processing module is connected to the data receiving module; the log recording module is connected to the scene classification module.

8. The system for automatic scene classification of video footage based on artificial intelligence as described in claim 7, characterized in that, The system for automatic scene classification of video footage based on artificial intelligence also includes a privacy protection enhancement module, which is connected to the encrypted transmission module.

Citation Information

Cited By

  • Audio transcoding method and device, equipment and storage medium

    CN121239891A

  • An audio transcoding method, apparatus, device and storage medium

    CN121239891B