Silent short video intelligent abstract generation system based on deep learning
The deep learning-based intelligent summary generation system for silent short videos solves the problems of information redundancy and logical confusion in silent video scenarios. It achieves end-to-end semantic understanding and text summary generation, and the generated summaries are logically coherent, concise, and suitable for silent video scenarios, thus improving the accuracy of description and system performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-04
- Publication Date
- 2026-04-14
AI Technical Summary
Existing video summarization methods are not well adapted to silent video scenarios, lack end-to-end processing, resulting in information redundancy, logical confusion, or missing key information, and are severely limited by multimodal information dependencies.
A deep learning-based silent short video intelligent summarization system is adopted, which includes video preprocessing, keyframe extraction, visual semantic encoding, semantic integration and summarization generation modules. It achieves end-to-end semantic understanding and text summarization generation through bimodal encoding and Transformer structure.
It achieves high-level semantic understanding without audio or subtitles, generates logically coherent and concise text summaries, improves information integrity and descriptive accuracy, and has continuous optimization capabilities.
Smart Images
Figure CN121865070A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning technology, specifically to a silent short video intelligent summarization system based on deep learning. Background Technology
[0002] With the rapid development of short video platforms, video content has exploded, and users face a serious problem of content overload when trying to quickly obtain core information from massive amounts of video data.
[0003] Video summarization technology, as a key means to solve this problem, can compress and summarize content while preserving the main semantic information of the video, greatly improving the efficiency of information acquisition. It has been widely used in many fields such as monitoring and analysis, education and teaching, and news dissemination. Among them, deep learning is an important branch of machine learning. By constructing a multi-layer neural network model to simulate the information processing mechanism of the human brain, it can automatically learn complex feature representations and semantic associations from massive amounts of data. It has made breakthrough progress in fields such as computer vision and natural language processing, providing strong technical support for the deep understanding and intelligent processing of video content. This has led to the development of summarization generation technology, which extracts key information from the original data, integrates and reconstructs it to generate concise, coherent output results that accurately reflect the core semantics of the original content. Traditional methods for video summarization can be broadly categorized into two types: one is frame selection based on visual features, which analyzes inter-frame differences, optical flow features, or uses clustering algorithms to select representative frames. While computationally efficient, these methods only address low-level visual features and lack a deep understanding of video semantics, potentially leading to summaries that fail to reflect the core content. The other type is content understanding based on semantics, which utilizes deep neural networks for semantic encoding and feature extraction. However, these methods often rely on multimodal information such as audio tracks, subtitles, or speech recognition results as supplementary information, making them unsuitable for applications lacking audio information, such as surveillance footage, silent instructional videos, and silent news materials.
[0004] Therefore, existing video summarization methods generally have significant limitations: some methods presuppose the availability of multimodal inputs, resulting in insufficient adaptability to completely silent videos; most solutions lack a complete end-to-end processing flow from raw video frames to the final text summary, requiring manual intervention in multiple stages, leading to low processing efficiency; at the same time, they lack systematic optimization strategies for the semantic coherence, information coverage, and language conciseness of the generated summaries, easily resulting in semantic redundancy, logical confusion, or missing key information.
[0005] In summary, a deep learning-based intelligent summary generation system for silent short videos is needed to address the aforementioned problems. Summary of the Invention
[0006] The purpose of this invention is to provide a silent short video intelligent summarization system based on deep learning to solve the problems mentioned in the background art.
[0007] To achieve the above objectives, the present invention provides the following technical solution: This invention proposes a deep learning-based intelligent summarization system for silent short videos, comprising: The video preprocessing module receives raw silent short video data and performs video format decoding, frame sequence extraction, image standardization and grayscale conversion operations, outputting a standardized frame sequence with uniform size and color channels. The keyframe extraction module, based on a standardized frame sequence, performs redundant frame removal and representative keyframe selection operations through inter-frame difference analysis, content-time dual feature vector construction, K-means clustering, and intra-cluster frame selection strategies, outputting a temporally ordered set of keyframes. The visual semantic encoding module employs a dual-modal encoding mechanism to extract visual features and model semantics in keyframes. It uses a cross-attention mechanism to achieve visual-linguistic semantic alignment, generating frame-level natural language descriptions. The semantic integration module is used to perform semantic vector mapping, similarity clustering, consistency detection, and global semantic reorganization on frame-level descriptions, remove redundant and erroneous descriptions, and output a core semantic stream sequence. The summary generation module performs global semantic fusion on the core semantic stream based on the Transformer structure, and generates and outputs a coherent and concise text summary through dynamic temperature sampling and length constraint mechanisms.
[0008] Preferably, it further includes: The storage module is used to persistently store the original video data, standardized frame sequences, keyframe sets, frame-level semantic descriptions, core semantic streams, and final text summaries, supporting data backup, historical tracing, and rapid retrieval.
[0009] Preferably, it further includes: The iterative optimization module records key parameters, model performance indicators, and semantic matching results throughout the system process. It performs incremental learning, feature extraction algorithm optimization, visual language model parameter updates, and performance monitoring to achieve dynamic iterative upgrades of system accuracy.
[0010] Based on the system, this invention also proposes a summary generation method for a deep learning-based silent short video intelligent summary generation system, comprising the following steps: S1. Receives silent short video raw input, completes format decoding and frame sequence extraction through the video preprocessing module, performs image standardization and grayscale conversion on each frame with uniform size and color channels, and outputs a standardized frame sequence; S2. Based on the standardized frame sequence from step S1, the keyframe extraction module performs redundant frame removal, dual feature vector construction, K-means clustering, frame selection within clusters, and time sequence restoration operations to output a set of keyframes. ; S3. The keyframe set is processed through a visual semantic encoding module. Perform bimodal feature extraction, semantic alignment, and autoregressive decoding to generate frame-level natural language descriptions; S4. The semantic integration module performs semantic vector mapping, similarity filtering, consistency detection, and time-series rearrangement on the frame-level descriptions, outputting a core semantic stream sequence. ; S5. The core semantic stream sequence is processed through the summarization module. Global semantic fusion, length constraint control, and post-processing optimization are performed to output the final text summary. S6. Record the key parameters and performance indicators of steps S1-S5 through an iterative optimization module, continuously optimize the feature extraction algorithm and model parameters based on incremental learning, improve the accuracy and coherence of summary generation, and update the semantic similarity threshold quarterly based on newly added short video data. , Every six months, the bimodal encoder of the visual semantic coding module is retrained, and the weight calculation method of the cross-attention mechanism is optimized. Every year, the summary length constraint parameters and dynamic temperature sampling coefficient are adjusted based on user feedback to ensure the semantic integrity and language fluency of the generated summary.
[0011] Preferably, the specific implementation process of step S2 is as follows: S21. Calculate the absolute difference between frames at the pixel level, and remove redundant frames with a difference value lower than the threshold, as shown in equation (1): (1); in Image width and height, For the first Frame grayscale image, This refers to the inter-frame variation. S22. Construct content feature vectors for the remaining frames. With time feature vector The content-time dual feature vector is formed by splicing the two features together, as shown in equation (2): (2); In the formula, , The total number of frames, ; S23. Based on the dual feature vector set, K-means clustering is used to divide frames that are semantically similar and temporally close into the same semantic cluster. The number of clusters K in the K-means clustering is adaptively adjusted according to the video length and content complexity, with a value range of 5-20. Specifically, K=5-8 when the video length is ≤1 minute, K=9-15 when the video length is <1 minute and ≤5 minutes, and K=16-20 when the video length is >5 minutes. S24. Within each cluster, select the frame closest to the center, the frame farthest from the center, and the time interval sampling frame. After temporal reconstruction, output the keyframe set. .
[0012] Preferably, the content feature vector in step S22 The build process is as follows: A hybrid structure combining convolutional feature extraction, multi-head self-attention semantic modeling, and global average pooling + linear projection is used to encode features in frame images. Convolutional layers extract low-level visual features, the multi-head self-attention structure captures long-range semantic dependencies, and global average pooling compresses the spatial structure, ultimately outputting a semantic vector of dimension 256. .
[0013] Preferably, the specific implementation process of step S3 is as follows: S31. The visual branch uses a CNN or ViT architecture to extract spatial semantic features. The language branch encodes semantic lexical vectors through a pre-trained text embedding model. ; S32. The association weights between visual regions and semantic words are calculated through a multi-layer cross-attention mechanism to achieve semantic alignment based on bimodal coding. The bimodal coding adopts the BLIP vision-language pre-training framework. Through the collaborative work of the visual encoder, text encoder and image and text decoder, the mapping from visual semantics to natural language is realized. S33. Based on semantic fusion features, a frame-level description is generated through an autoregressive decoder, and a semantic consistency constraint loss function is introduced, as shown in equation (3): (3); In the formula, For cross-entropy loss, To optimize the generated results, we use semantic matching loss.
[0014] Preferably, the specific implementation process of step S4 is as follows: S41. Map frame-level descriptions to vectors in a unified semantic space. Calculate the cosine similarity between each pair of pairs, as shown in equation (4): (4); Used to remove filters with similarity exceeding a threshold. Redundant description; S42. Calculate the semantic vector of a single frame and the global center vector. Similarity, remove those with similarity below a threshold. Exception description; S43. After hierarchical clustering and temporal rearrangement, output the core semantic stream sequence. .
[0015] Preferably, the threshold in step S41 The value is 0.85, which is the threshold value in step S42. The value is dynamically adjusted based on the statistical characteristics of the dataset, and the range is 0.6-0.75.
[0016] Preferably, the specific implementation process of step S5 is as follows: S51. Construct a global context attention matrix using the Transformer structure, as shown in equation (5): (5); Used to capture the temporal and causal relationships of semantic events; S52. An initial summary is generated using a dynamic temperature sampling method and a length constraint mechanism. After repeated phrase trimming and grammatical correction, the final text summary is output. The average length of the final text summary is 35-50 characters, and the length threshold can be customized according to the application scenario, ranging from 20-100 characters.
[0017] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention achieves high-level semantic understanding of videos under the constraints of no audio and no subtitles through a pure visual semantic understanding path, based on keyframe extraction, bimodal coding, and semantic integration technology. It realizes end-to-end summary generation of silent short videos. Through content feature encoding structures using convolutional feature extraction and multi-head self-attention semantic modeling, combined with the BLIP visual-language pre-training framework and cross-attention mechanism, it achieves precise alignment between visual features and linguistic semantics, ensuring that keyframes fully cover the core semantics of the video while maintaining temporal coherence. This avoids the information redundancy or missing information problems caused by insufficient keyframe representativeness in traditional methods. Furthermore, it eliminates redundancy based on cosine similarity calculation. By combining description with global center vector detection to remove abnormal descriptions, and then using the Transformer structure for global semantic fusion, this invention effectively solves the problems of semantic redundancy and logical confusion in traditional summary generation. The generated summary is highly concise and logically coherent, with strong readability. Through incremental learning, model parameter updates, and dynamic threshold adjustment, the system performance is continuously improved. In summary, compared with traditional summary generation systems, this invention has the advantages of breaking through the limitations of multimodal information dependence, adapting to silent scenarios, improving semantic understanding accuracy, strengthening description accuracy, optimizing keyframe selection strategy, ensuring information integrity, achieving semantic conciseness and logical coherence, improving summary quality, and having continuous optimization capabilities and flexible scalability. Attached Figure Description
[0018] Figure 1 The diagram shows the topology of the silent short video intelligent summarization system based on deep learning according to the present invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] Example 1, please refer to Figure 1 This invention proposes a deep learning-based intelligent summary generation system for silent short videos, comprising: The video preprocessing module receives raw silent short video data and performs video format decoding, frame sequence extraction, image standardization and grayscale conversion operations, outputting a standardized frame sequence with uniform size and color channels. The keyframe extraction module, based on a standardized frame sequence, performs redundant frame removal and representative keyframe selection operations through inter-frame difference analysis, content-time dual feature vector construction, K-means clustering, and intra-cluster frame selection strategies, outputting a temporally ordered set of keyframes. The visual semantic encoding module employs a dual-modal encoding mechanism to extract visual features and model semantics in keyframes. It uses a cross-attention mechanism to achieve visual-linguistic semantic alignment, generating frame-level natural language descriptions. The semantic integration module is used to perform semantic vector mapping, similarity clustering, consistency detection, and global semantic reorganization on frame-level descriptions, remove redundant and erroneous descriptions, and output a core semantic stream sequence. The summary generation module performs global semantic fusion on the core semantic stream based on the Transformer structure, and generates and outputs a coherent and concise text summary through dynamic temperature sampling and length constraint mechanisms.
[0021] In this embodiment, the system further includes: The storage module is used to persistently store the original video data, standardized frame sequences, keyframe sets, frame-level semantic descriptions, core semantic streams, and final text summaries, supporting data backup, historical tracing, and rapid retrieval.
[0022] In this embodiment, the system further includes: The iterative optimization module records key parameters, model performance indicators, and semantic matching results throughout the system process. It performs incremental learning, feature extraction algorithm optimization, visual language model parameter updates, and performance monitoring to achieve dynamic iterative upgrades of system accuracy.
[0023] Example 2, in practical applications, the summary generation method of the silent short video intelligent summary generation system specifically includes the following steps: S1. Receives silent short video raw input, completes format decoding and frame sequence extraction through the video preprocessing module, performs image standardization and grayscale conversion on each frame with uniform size and color channels, and outputs a standardized frame sequence; S2. Based on the standardized frame sequence from step S1, the keyframe extraction module performs redundant frame removal, dual feature vector construction, K-means clustering, frame selection within clusters, and time sequence restoration operations to output a set of keyframes. ; In this embodiment, the specific implementation process of step S2 is as follows: S21. Calculate the absolute difference between frames at the pixel level, and remove redundant frames with a difference value lower than the threshold, as shown in equation (1): (1); in Image width and height, For the first Frame grayscale image, This refers to the inter-frame variation. S22. Construct content feature vectors for the remaining frames. With time feature vector The content-time dual feature vector is formed by splicing the two features together, as shown in equation (2): (2); In the formula, , The total number of frames, ; S23. Based on the dual feature vector set, K-means clustering is used to divide frames that are semantically similar and temporally close into the same semantic cluster. The number of clusters K in the K-means clustering is adaptively adjusted according to the video length and content complexity, with a value range of 5-20. Specifically, K=5-8 when the video length is ≤1 minute, K=9-15 when the video length is <1 minute and ≤5 minutes, and K=16-20 when the video length is >5 minutes. S24. Within each cluster, select the frame closest to the center, the frame farthest from the center, and the time interval sampling frame. After temporal reconstruction, output the keyframe set. ; In this embodiment, the content feature vector mentioned in step S22 The build process is as follows: A hybrid structure combining convolutional feature extraction, multi-head self-attention semantic modeling, and global average pooling + linear projection is used to encode features in frame images. Convolutional layers extract low-level visual features, the multi-head self-attention structure captures long-range semantic dependencies, and global average pooling compresses the spatial structure, ultimately outputting a semantic vector of dimension 256. ; S3. The keyframe set is processed through a visual semantic encoding module. Perform bimodal feature extraction, semantic alignment, and autoregressive decoding to generate frame-level natural language descriptions; In this embodiment, the specific implementation process of step S3 is as follows: S31. The visual branch uses a CNN or ViT architecture to extract spatial semantic features. The language branch encodes semantic lexical vectors through a pre-trained text embedding model. ; S32. The association weights between visual regions and semantic words are calculated through a multi-layer cross-attention mechanism to achieve semantic alignment based on bimodal coding. The bimodal coding adopts the BLIP vision-language pre-training framework. Through the collaborative work of the visual encoder, text encoder and image and text decoder, the mapping from visual semantics to natural language is realized. S33. Based on semantic fusion features, a frame-level description is generated through an autoregressive decoder, and a semantic consistency constraint loss function is introduced, as shown in equation (3): (3); In the formula, For cross-entropy loss, Optimize the generated results by adjusting the semantic matching loss; S4. The semantic integration module performs semantic vector mapping, similarity filtering, consistency detection, and time-series rearrangement on the frame-level descriptions, outputting a core semantic stream sequence. ; In this embodiment, the specific implementation process of step S4 is as follows: S41. Map frame-level descriptions to vectors in a unified semantic space. Calculate the cosine similarity between each pair of pairs, as shown in equation (4): (4); Used to remove filters with similarity exceeding a threshold. Redundant description; S42. Calculate the semantic vector of a single frame and the global center vector. Similarity, remove those with similarity below a threshold. Exception description; S43. After hierarchical clustering and temporal rearrangement, output the core semantic stream sequence. ; In this embodiment, the threshold mentioned in step S41 The value is 0.85, which is the threshold value in step S42. The value is dynamically adjusted based on the statistical characteristics of the dataset, ranging from 0.6 to 0.75. S5. The core semantic stream sequence is processed through the summarization module. Global semantic fusion, length constraint control, and post-processing optimization are performed to output the final text summary. In this embodiment, the specific implementation process of system step S5 is as follows: S51. Construct a global context attention matrix using the Transformer structure, as shown in equation (5): (5); Used to capture the temporal and causal relationships of semantic events; S52. An initial summary is generated using a dynamic temperature sampling method and a length constraint mechanism. After repeated phrase trimming and grammatical correction, the final text summary is output. The average length of the final text summary is 35-50 characters, and a custom length threshold of 20-100 characters is supported according to the application scenario. S6. Record the key parameters and performance indicators of steps S1-S5 through an iterative optimization module, continuously optimize the feature extraction algorithm and model parameters based on incremental learning, improve the accuracy and coherence of summary generation, and update the semantic similarity threshold quarterly based on newly added short video data. , ; The bimodal encoder of the visual semantic coding module is retrained every six months to optimize the weight calculation method of the cross-attention mechanism; Each year, the abstract length constraint parameters and dynamic temperature sampling coefficient are adjusted based on user feedback to ensure the semantic integrity and linguistic fluency of the generated abstracts.
[0024] Through the above steps, this invention has advantages over traditional summary generation systems, including overcoming the limitations of multimodal information dependence, adapting to silent scenarios, improving semantic understanding accuracy, enhancing descriptive accuracy, optimizing keyframe selection strategies, ensuring information integrity, achieving semantic refinement and logical coherence, improving summary quality, and possessing continuous optimization capabilities and flexible scalability.
[0025] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A silent short video intelligent summarization generation system based on deep learning, characterized in that... ,include: The video preprocessing module receives raw silent short video data and performs video format decoding, frame sequence extraction, image standardization and grayscale conversion operations, outputting a standardized frame sequence with uniform size and color channels. The keyframe extraction module, based on a standardized frame sequence, performs redundant frame removal and representative keyframe selection operations through inter-frame difference analysis, content-time dual feature vector construction, K-means clustering, and intra-cluster frame selection strategies, outputting a temporally ordered set of keyframes. The visual semantic encoding module employs a dual-modal encoding mechanism to extract visual features and model semantics in keyframes. It uses a cross-attention mechanism to achieve visual-linguistic semantic alignment, generating frame-level natural language descriptions. The semantic integration module is used to perform semantic vector mapping, similarity clustering, consistency detection, and global semantic reorganization on frame-level descriptions, remove redundant and erroneous descriptions, and output a core semantic stream sequence. The summary generation module performs global semantic fusion on the core semantic stream based on the Transformer structure, and generates and outputs a coherent and concise text summary through dynamic temperature sampling and length constraint mechanisms.
2. The deep learning-based silent short video intelligent summarization system according to claim 1, characterized in that... Also includes: The storage module is used to persistently store the original video data, standardized frame sequences, keyframe sets, frame-level semantic descriptions, core semantic streams, and final text summaries, supporting data backup, historical tracing, and rapid retrieval.
3. The deep learning-based silent short video intelligent summarization system according to claim 2, characterized in that... Also includes: The iterative optimization module records key parameters, model performance indicators, and semantic matching results throughout the system process. It performs incremental learning, feature extraction algorithm optimization, visual language model parameter updates, and performance monitoring to achieve dynamic iterative upgrades of system accuracy.
4. The summarization method of the deep learning-based silent short video intelligent summarization system according to any one of claims 1-3, characterized in that... This includes the following steps: S1. Receives silent short video raw input, completes format decoding and frame sequence extraction through the video preprocessing module, performs image standardization and grayscale conversion on each frame with uniform size and color channels, and outputs a standardized frame sequence; S2. Based on the standardized frame sequence from step S1, the keyframe extraction module performs redundant frame removal, dual feature vector construction, K-means clustering, frame selection within clusters, and time sequence restoration operations to output a set of keyframes. ; S3. The keyframe set is processed through a visual semantic encoding module. Perform bimodal feature extraction, semantic alignment, and autoregressive decoding to generate frame-level natural language descriptions; S4. The semantic integration module performs semantic vector mapping, similarity filtering, consistency detection, and time-series rearrangement on the frame-level descriptions, outputting a core semantic stream sequence. ; S5. The core semantic stream sequence is processed through the summarization module. Global semantic fusion, length constraint control, and post-processing optimization are performed to output the final text summary. S6. Record the key parameters and performance indicators of steps S1-S5 through an iterative optimization module, continuously optimize the feature extraction algorithm and model parameters based on incremental learning, improve the accuracy and coherence of summary generation, and update the semantic similarity threshold quarterly based on newly added short video data. , The bimodal encoder of the visual semantic coding module is retrained every six months, and the weight calculation method of the cross-attention mechanism is optimized. The summary length constraint parameters and dynamic temperature sampling coefficient are adjusted annually based on user feedback.
5. The summarization method of the deep learning-based silent short video intelligent summarization system according to claim 4, characterized in that... The specific implementation process of step S2 is as follows: S21. Calculate the absolute difference between frames at the pixel level, and remove redundant frames with a difference value lower than the threshold, as shown in equation (1): (1); in Image width and height, For the first Frame grayscale image, This refers to the inter-frame variation. S22. Construct content feature vectors for the remaining frames. With time feature vector The content-time dual feature vector is formed by splicing the two features together, as shown in equation (2): (2); In the formula, , The total number of frames, ; S23. Based on the dual feature vector set, K-means clustering is used to divide frames that are semantically similar and temporally close into the same semantic cluster. The number of clusters K in the K-means clustering is adaptively adjusted according to the video length and content complexity, with a value range of 5-20. Specifically, K=5-8 when the video length is ≤1 minute, K=9-15 when the video length is <1 minute and ≤5 minutes, and K=16-20 when the video length is >5 minutes. S24. Within each cluster, select the frame closest to the center, the frame farthest from the center, and the time interval sampling frame. After temporal reconstruction, output the keyframe set. .
6. The summarization method of the silent short video intelligent summarization system based on deep learning according to claim 5, characterized in that... The content feature vector in step S22 The build process is as follows: A hybrid structure combining convolutional feature extraction, multi-head self-attention semantic modeling, and global average pooling + linear projection is used to encode features in frame images. Convolutional layers extract low-level visual features, the multi-head self-attention structure captures long-range semantic dependencies, and global average pooling compresses the spatial structure, ultimately outputting a semantic vector of dimension 256. .
7. The summarization method of the silent short video intelligent summarization system based on deep learning according to claim 6, characterized in that... The specific implementation process of step S3 is as follows: S31. The visual branch uses a CNN or ViT architecture to extract spatial semantic features. The language branch encodes semantic lexical vectors through a pre-trained text embedding model. ; S32. The association weights between visual regions and semantic words are calculated through a multi-layer cross-attention mechanism to achieve semantic alignment based on bimodal coding. The bimodal coding adopts the BLIP vision-language pre-training framework. Through the collaborative work of the visual encoder, text encoder and image and text decoder, the mapping from visual semantics to natural language is realized. S33. Based on semantic fusion features, a frame-level description is generated through an autoregressive decoder, and a semantic consistency constraint loss function is introduced, as shown in equation (3): (3); In the formula, For cross-entropy loss, To optimize the generated results, we use semantic matching loss.
8. The summarization method of the silent short video intelligent summarization system based on deep learning according to claim 7, characterized in that... The specific implementation process of step S4 is as follows: S41. Map frame-level descriptions to vectors in a unified semantic space. Calculate the cosine similarity between each pair of pairs, as shown in equation (4): (4); Used to remove filters with similarity exceeding a threshold. Redundant description; S42. Calculate the semantic vector of a single frame and the global center vector. Similarity, remove those with similarity below a threshold. Exception description; S43. After hierarchical clustering and temporal rearrangement, output the core semantic stream sequence. .
9. The summarization method of the silent short video intelligent summarization system based on deep learning according to claim 8, characterized in that... The threshold in step S41 The value is 0.85, which is the threshold value in step S42. The value is dynamically adjusted based on the statistical characteristics of the dataset, and the range is 0.6-0.
75.
10. The summarization method of the deep learning-based silent short video intelligent summarization system according to claim 9, characterized in that... The specific implementation process of step S5 is as follows: S51. Construct a global context attention matrix using the Transformer structure, as shown in equation (5): (5); Used to capture the temporal and causal relationships of semantic events; S52. An initial summary is generated using a dynamic temperature sampling method and a length constraint mechanism. After repeated phrase trimming and grammatical correction, the final text summary is output. The average length of the final text summary is 35-50 characters, and the length threshold can be customized according to the application scenario, ranging from 20-100 characters.