Knowledge unit slicing and explainable indexing propagation method for short video or live streaming content

Abstract

The application discloses a short video or live content knowledge unit slicing and explainable indexing propagation method, and belongs to the technical field of digital data processing. The method comprises the following steps: acquiring video stream data and audio data, analyzing and processing the video stream data and the audio data to generate slicing driving parameters, segmenting, generating a knowledge unit segment set, and generating segment index information; performing multi-modal feature extraction and fusion processing on the knowledge unit segment set, matching the controlled vocabulary, generating alignment candidate labels; referencing the segment index information to generate an evidence index set, checking the multi-modal fusion representation vector, and mapping the platform rules to generate publishable indexing data and recover online interaction feedback; updating the multi-modal feature extraction and fusion processing and the controlled vocabulary matching process, and storing the generated log fingerprint record. The method combines multi-modal feature fusion and explainable reasoning, and constructs a feedback closed-loop data processing mode, which can realize slicing, traceable indexing and optimization of knowledge units.

Description

Technical Field

[0002] This invention relates to the field of digital data processing technology, and in particular to a method for disseminating knowledge unit slices and interpretable indexing of short video or live streaming content. Background Technology

[0004] With the development of internet technology, short videos and live streaming have become the main carriers of digital information dissemination. Effective content understanding and structuring of this massive amount of unstructured multimedia data is crucial for achieving accurate recommendations, efficient retrieval, and effective supervision. Knowledge unit slicing and indexing are the core steps in this process, aiming to divide continuous video streams into independent, meaningful content segments and assign each segment an accurate tag reflecting its theme. This is essentially a complex digital data processing task.

[0005] In existing technologies, processing such multimedia data typically employs some basic computer data processing methods. For example, in video segmentation, methods such as segmentation based on fixed time intervals or simple scene segmentation based on shot transition detection are commonly used. In content indexing, some schemes rely on single-modal information, such as performing speech recognition only on the audio stream of the video and then extracting keywords as tags from the converted text; or randomly selecting a few keyframes from the video and using image classification models to obtain some high-level visual classification labels.

[0006] However, segmentation methods based on fixed durations or simple shot transitions often disrupt the semantic coherence of a complete knowledge point, resulting in segmentation results that do not match the natural thematic boundaries of the content. Secondly, relying on a single information modality for content understanding limits the dimensions of information acquired, failing to comprehensively and accurately summarize the true theme of video clips, easily leading to one-sided or omitted tags. Furthermore, tags generated by existing technologies often lack interpretability; the system cannot explain why it assigns a particular tag, and its decision-making process is opaque. Simultaneously, these systems are typically static; once deployed, their models and rule bases are difficult to dynamically adjust and optimize based on online feedback. Summary of the Invention

[0008] To address the aforementioned issues, this invention provides a method for knowledge unit slicing and interpretable indexing propagation of short video or live streaming content. It employs a combination of multimodal feature fusion and interpretable reasoning, and constructs a feedback closed-loop data processing approach, enabling the slicing, traceable indexing, and optimization of knowledge units.

[0009] The above objectives can be achieved through the following approach:

[0010] A method for knowledge unit slicing and interpretable indexing of short video or live streaming content includes: acquiring video stream data and audio data; performing time synchronization and audio-visual separation processing on the video stream data and audio data to generate media basic data; analyzing the content rhythm pattern of the media basic data to generate slice driving parameters; segmenting the media basic data based on the slice driving parameters to generate a set of knowledge unit fragments, and generating fragment index information for the knowledge unit fragment set; performing multimodal feature extraction and fusion processing on the knowledge unit fragment set to generate a multimodal fusion representation vector; and matching the multimodal fusion representation vector with a preset controlled lexicon to generate alignment candidate tags. The fragment index information is used to generate an evidence index set for the aligned candidate tags; the multimodal fusion representation vector and the evidence index set are semantically consistent and mapped with platform rules to generate publishable index data; online interaction feedback of the publishable index data is collected to generate feedback feature vectors, and feedback feature vectors are used to generate backflow update parameters to update the multimodal feature extraction and fusion processing process and the controlled lexicon matching process; using the SHA-256 hash algorithm and a timestamp from NTP or a local secure crystal oscillator, the structured indexing results, the fragment index information, and the backflow update parameters are used as binding objects for evidence storage to generate log fingerprint records.

[0011] Optionally, the generation of basic media data includes: acquiring video stream data and audio data and aligning them with timestamps to generate synchronized media data; performing demultiplexing on the synchronized media data to generate audio-visual separation data; decoding the audio-visual separation data to generate a frame sequence and an audio sequence; performing visual feature change detection on the frame sequence to generate a keyframe set; performing speech energy analysis on the audio sequence to generate a speech energy feature sequence; and integrating the frame sequence, the audio sequence, the keyframe set, and the speech energy feature sequence to generate basic media data.

[0012] Optionally, the step of generating slice driving parameters involves: analyzing the switching frequency of the frame sequence and the speech rate changes of the audio sequence in the media basic data to generate content rhythm pattern features; calculating the information entropy and mutation point density of the content rhythm pattern features to generate a content complexity index; and dynamically adjusting the slice granularity threshold based on the content complexity index to generate slice driving parameters.

[0013] Optionally, generating a set of knowledge unit fragments and generating fragment index information for the set of knowledge unit fragments includes: segmenting the frame sequence of the media basic data using the slice driving parameters to generate shot slice fragments; performing boundary smoothing processing on the shot slice fragments to generate a smooth boundary sequence; performing speech recognition and topic mutation detection on the audio sequence of the media basic data to generate semantic slice fragments; aligning the time positions of the smooth boundary sequence and the semantic slice fragments to generate aligned fragment boundaries; merging fragments according to the aligned fragment boundaries to generate a set of knowledge unit fragments; assigning a unique identifier to the set of knowledge unit fragments and associating it with the time position, frame range, and subtitle range to generate fragment index information.

[0014] Optionally, generating the multimodal fusion representation vector includes: extracting image sequences corresponding to frame ranges in the knowledge unit fragment set and performing visual feature encoding to generate visual vectors; extracting text data corresponding to subtitle ranges in the knowledge unit fragment set and performing text feature encoding to generate speech recognition text data; recognizing text regions in the image sequence and performing text encoding to generate optical character recognition text data; concatenating the visual vectors, the speech recognition text data, and the optical character recognition text data to generate a multimodal raw vector; and performing gating and attention weighting calculations on the multimodal raw vector to generate a multimodal fusion representation vector.

[0015] Optionally, generating alignment candidate tags includes: calculating the similarity between the multimodal fusion representation vector and the vectors of each entry in a preset controlled lexicon to generate a lexicon matching result; querying the higher-level concepts of the lexicon matching result in a preset knowledge graph of relationships between descriptive concepts to generate an ontology association result; merging the lexicon matching result and the ontology association result to generate a candidate tag set; performing semantic similarity aggregation on the candidate tag set to generate a synonym aggregation set; filtering out non-compliant items in the synonym aggregation set using a preset list of prohibited words to generate an available aggregation set; and extracting representative items from the available aggregation set to generate alignment candidate tags.

[0016] Optionally, the step of using the segment index information to generate an evidence index set for the alignment candidate tags includes: extracting time identifiers of frame ranges from the segment index information to generate a frame identifier set; extracting interval identifiers of subtitle ranges from the segment index information to generate a subtitle interval identifier set; associating the frame identifier set and the subtitle interval identifier set with the corresponding alignment candidate tags to generate tag evidence binding pairs; aggregating the tag evidence binding pairs to construct a traceable index structure and generate an evidence index set.

[0017] Optionally, generating publishable indexed data includes: performing semantic consistency verification and interpretable reasoning on the multimodal fusion representation vector and the evidence index set to generate structured indexing results; calculating the coverage ratio of the evidence index set to each tag in the structured indexing results to generate an evidence coverage index; verifying the consistency between the evidence coverage index and the tag content to generate a consistency verification result; mapping platform topics and categories to the passing items in the consistency verification result to generate a platform mapping result; synthesizing indexed data based on the platform mapping result and performing rule detection on its length, prohibited words, and format to generate a rule detection result; and fixing violations based on the rule detection result to generate publishable indexed data.

[0018] Optionally, generating backflow update parameters using the feedback feature vector includes: collecting user click, dwell, and appeal records corresponding to the publishable index data to generate interaction feedback records; normalizing the interaction feedback records to generate a feedback indicator sequence; encoding the correlation between the feedback indicator sequence and the platform mapping results, and performing dimensionality reduction processing to generate a feedback feature vector; incrementally updating the gating and attention weighting parameters in the multimodal feature extraction and fusion processing using the feedback feature vector to generate fusion parameter update values; correcting the lexicon matching results using the feedback feature vector to generate lexicon matching update values; and merging the fusion parameter update values ​​and the lexicon matching update values ​​to generate backflow update parameters.

[0019] Based on the same inventive concept, this invention also provides a knowledge unit slicing and interpretable indexing dissemination system for short video or live streaming content. The system includes: a data acquisition module for acquiring video stream data and audio data, and performing time synchronization and audio-visual separation processing on the video stream data and audio data to generate basic media data; a slice analysis module for analyzing the content rhythm pattern of the basic media data and generating slice driving parameters; a data segmentation module for segmenting the basic media data based on the slice driving parameters to generate a set of knowledge unit fragments, and generating fragment index information for the set of knowledge unit fragments; and a feature representation module for performing multi-modal analysis on the set of knowledge unit fragments. The system includes a feature extraction and fusion processing module to generate a multimodal fusion representation vector; a tag generation module to match the multimodal fusion representation vector with a preset controlled lexicon to generate alignment candidate tags; and an evidence generation module to generate an evidence index set for the alignment candidate tags by referencing the segment index information. This includes extracting time identifiers for frame ranges from the segment index information to generate a frame identifier set; extracting interval identifiers for subtitle ranges from the segment index information to generate a subtitle interval identifier set; associating the frame identifier set and the subtitle interval identifier set with the corresponding alignment candidate tags to generate tag-evidence binding pairs; and aggregating the tag-evidence binding pairs to construct a traceable index structure. The system generates an evidence index set; a verification and reasoning module is used to perform semantic consistency verification and platform rule mapping on the multimodal fusion representation vector and the evidence index set to generate publishable indexed data; including: performing semantic consistency verification and interpretable reasoning on the multimodal fusion representation vector and the evidence index set to generate structured indexing results; calculating the coverage ratio of the evidence index set to each tag in the structured indexing results to generate an evidence coverage index; verifying the consistency between the evidence coverage index and the tag content to generate a consistency verification result; mapping the passed items in the consistency verification result to platform topics and categories to generate platform mapping results; and based on the platform mapping results... The system synthesizes indexing data and performs rule checks on its length, prohibited words, and format to generate rule check results. Based on these results, violations are corrected, and publishable indexing data is generated. A feedback module collects online interactive feedback on the publishable indexing data, generates feedback feature vectors, and uses these vectors to generate backflow update parameters for updating the parameters of multimodal feature extraction and fusion processing and the mapping relationship of the controlled lexicon. A notarization module uses the SHA-256 hash algorithm and a timestamp from an NTP or local secure crystal oscillator to notarize the structured indexing results, the fragment index information, and the backflow update parameters as binding objects, generating a log fingerprint record.

[0020] Based on the same inventive concept, the present invention also provides a computer storage medium storing one or more programs, which, when executed, can implement any of the methods described above.

[0021] Based on the same inventive concept, the present invention also provides a device including a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface, and the memory communicate with each other through the communication bus; the processor is used to execute a program stored in the aforementioned computer-readable storage medium.

[0022] Compared with the prior art, the present invention has the following advantages:

[0023] 1. This invention, from the acquisition of audio and video streams to the generation and feedback optimization of publishable indexing data, can process massive amounts of short video and live content on a large scale. It transforms the tedious video understanding and annotation work that relies on human experience into a standardized machine computing process, thereby improving the efficiency and scalability of content structuring processing.

[0024] 2. This invention constructs a traceable evidence index set for each generated tag and precisely binds the tag generation to specific frames and subtitle text in the original video. Combined with subsequent semantic consistency verification and interpretable reasoning, the indexing results have source evidence and logical support, enhancing the transparency, credibility, and auditability of the indexing system.

[0025] 3. This invention collects real user interaction behaviors with indexing data and transforms them into feedback feature vectors. The system can use this data from real application scenarios to incrementally update the parameters of the upstream multimodal feature fusion model and the lexicon matching strategy, thereby enabling the system to have the ability to continuously learn and iterate, ensuring that the accuracy of indexing can dynamically adapt to changes in the content ecosystem and user preferences. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating the knowledge unit slicing and interpretable indexing propagation method for short video or live stream content according to an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram illustrating the relationship between content complexity and dynamic slice granularity in an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the structure of the knowledge unit slice and interpretable indexing propagation system for short video or live broadcast content according to an embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Reference Figure 1 One embodiment of the present invention proposes a knowledge unit slicing and interpretable indexing propagation method for short video or live streaming content. It adopts a combination of multimodal feature fusion and interpretable reasoning, and constructs a data processing method with feedback closed loop, which can realize the slicing, traceable indexing and optimization of knowledge units.

[0035] The method described in this embodiment specifically includes:

[0036] Acquire video stream data and audio data, and perform time synchronization and audio-visual separation processing on the video stream data and audio data to generate basic media data;

[0037] Analyze the content rhythm pattern of the media basic data to generate slice driving parameters;

[0038] The media base data is segmented based on the slice driving parameters to generate a knowledge unit fragment set, and fragment index information is generated for the knowledge unit fragment set.

[0039] Multimodal feature extraction and fusion processing are performed on the knowledge unit fragment set to generate a multimodal fusion representation vector;

[0040] The multimodal fusion representation vector is matched with a preset controlled lexicon to generate aligned candidate tags;

[0041] The fragment index information is used to generate an evidence index set for the alignment candidate labels;

[0042] The multimodal fusion representation vector and the evidence index set are subjected to semantic consistency verification and platform rule mapping to generate publishable index data;

[0043] The online interactive feedback of the publishable index data is collected, a feedback feature vector is generated, and the feedback feature vector is used to generate backflow update parameters to update the multimodal feature extraction and fusion process and the controlled lexicon matching process.

[0044] Using the SHA-256 hash algorithm and a timestamp derived from an NTP or local secure crystal oscillator, the structured indexing result, the fragment index information, and the reflow update parameters are used as binding objects for evidence storage, generating a log fingerprint record.

[0045] Optionally, the generated media base data includes:

[0046] Acquire video stream data and audio data, align timestamps, and generate synchronized media data;

[0047] Perform a demultiplexing operation on the synchronous media data to generate audio-visual separated data;

[0048] Decode the audio-visual separation data to generate a frame sequence and an audio sequence;

[0049] Visual feature change detection is performed on the frame sequence to generate a keyframe set;

[0050] The audio sequence is subjected to speech energy analysis to generate a speech energy feature sequence;

[0051] The frame sequence, the audio sequence, the keyframe set, and the speech energy feature sequence are integrated to generate basic media data.

[0052] Specifically, this method generates basic media data by transforming the raw, mixed video and audio streams into a structured, standardized dataset containing multi-dimensional features, providing a foundation for subsequent knowledge unit slicing. The system acquires video and audio stream data, aligns the timestamps by reading the timestamp information within the data packets, ensuring the audio-visual error is within 40 milliseconds, and generates synchronized media data. Next, the system performs demultiplexing on this synchronized media data, separating independent compressed video and audio streams from container formats such as MP4 or FLV, generating audio-visual separation data. Subsequently, the system calls a decoder to decode the audio-visual separation data, converting the video stream into a time-ordered sequence of original image frames (i.e., a frame sequence), and converting the audio stream into a PCM pulse code modulation format audio sequence. To identify visual transition points in the content, the system performs visual feature change detection on the frame sequence. This detection locates scene transitions by quantifying the visual differences between consecutive frames. The system calculates the structural similarity between two adjacent frames using the Structural Similarity Index (SSIM), and its change is expressed by the following formula:

[0053] ,

[0054] Where D represents the calculated visual change, and S represents the mean of the structural similarity (SSIM) calculation results between the two frames, which can be expressed as:

[0055] ,

[0056] in, , SSIM represents a local window at the same location in two images, such as an 8×8 pixel block. SSIM is calculated based on local regions to avoid errors in global statistics. , This represents the average brightness of the corresponding window. , The average of the squared deviations of the pixel values ​​within the corresponding window from the mean value reflects the brightness and contrast. Covariance measures the degree of linear correlation within a window; a larger value indicates greater structural similarity. , This is a very small constant to avoid a denominator of zero or unstable values; among which, , L is the pixel dynamic range, such as L=255 for 8-bit. k1 and k2 are small constants, typically k1=0.01 and k2=0.03, to ensure the formula is sensitive to weak texture regions, k1∈[0.01,0.02], k2∈[0.03,0.04]. For frame t and frame t−1, after downsampling to 224×224, a block-level SSIM is performed using an 11×11 Gaussian window, and the mean S of the entire image is taken.

[0057] When the D value exceeds a preset visual change threshold, such as 0.7, the current frame is identified as a keyframe and stored in the keyframe set. Simultaneously, to locate effective information intervals in the audio, the system performs speech energy analysis on the audio sequence. This analysis distinguishes speech segments from silent or noisy segments by calculating the energy amplitude of the audio signal within a short time window. The system calculates the short-time energy E of each audio frame using the following formula:

[0058] ,

[0059] in, Frame length refers to the number of sampling points per frame. To balance temporal resolution and computational efficiency, the frame length is typically set to 20-30 milliseconds. For example, for a 44.1kHz sampling rate, L=1024 corresponds to approximately 23.2ms as one frame; for a 16kHz sampling rate, L=256 corresponds to 16ms or L=512 corresponds to 32ms. In this embodiment, for ease of explanation, subsequent calculations will use a 16kHz sampling rate and L=256 (approximately 16ms) as an example; E represents the energy of the current audio frame. This represents the amplitude value of the i-th sample point within the audio frame, which is obtained from the audio sequence. For have:

[0060] ,

[0061] in, For the t-th frame The original values ​​of each sampling point Let be the index of the starting sampling point of frame t. For the first frame A global index of points; For the window function in the first The value of each point is used to weight the signal within the frame, such as attenuation at the edge of the Hamming window.

[0062] The calculated energy value sequence constitutes the speech energy feature sequence. Finally, the system integrates the decoded frame sequence and audio sequence, as well as the keyframe set generated through analysis and the speech energy feature sequence. The integration method involves associating each data stream and feature set through a unified time axis, encapsulating them into a structured object, thereby generating media foundation data containing rich temporal and feature information.

[0063] Optionally, the slice generation driving parameters are:

[0064] Analyze the switching frequency of frame sequences and the speech rate changes of audio sequences in the media basic data to generate content rhythm pattern features;

[0065] Calculate the information entropy and mutation point density of the content rhythm pattern features to generate a content complexity index;

[0066] The slice granularity threshold is dynamically adjusted based on the content complexity index to generate slice driving parameters.

[0067] Specifically, such as Figure 2As shown, this method generates slice-driven parameters to quantify the information density and rhythm changes of video content, thus providing a dynamic and adaptive granular control basis for subsequent knowledge unit segmentation. The system analyzes the media's basic data, extracting the frame sequence switching frequency and audio sequence speech rate changes to generate content rhythm pattern features. Specifically, the system counts the number of times keyframes appear in the keyframe set within a preset time window, such as 5 seconds, as the frame sequence switching frequency; simultaneously, the system performs automatic speech recognition (ASR) on the audio sequence, obtains words and their timestamps, calculates the number of words per second as the instantaneous speech rate, and calculates the standard deviation of this speech rate within a sliding time window as the speech rate change index. These two indices together constitute the temporal content rhythm pattern features. Next, the system calculates the information entropy and mutation point density of this content rhythm pattern feature to generate a content complexity index. This index is a comprehensive quantitative value that reflects the information content and rhythm stability of the content per unit time. Its calculation formula is:

[0068] ,

[0069] Where C is the content complexity index, H is the information entropy calculated after discretization of the content rhythm pattern feature sequence within a specific segment, used to measure the unpredictability of rhythm changes. Dt is the mutation point density, obtained by calculating the first derivative of the content rhythm pattern feature sequence and counting the number of points whose values ​​exceed a predetermined threshold, divided by the segment duration, reflecting the degree of drastic rhythm changes. W1 and w2 are preset weighting coefficients used to balance the influence of entropy and mutation point density; their values ​​are set empirically, typically between 0.3 and 0.7. Finally, the system dynamically adjusts the slice granularity threshold based on the content complexity index to generate slice driving parameters. The slice granularity threshold is a parameter that determines the shortest or longest slice duration. Its adjustment logic is that the higher the content complexity, the finer the slice granularity should be, i.e., the shorter the duration should be, to ensure the purity of knowledge units. The adjustment formula is as follows:

[0070] ,

[0071] Here, T is the final generated slice-driving parameter, i.e., the dynamically adjusted slice granularity threshold. T0 is a base duration benchmark, such as 30 seconds. k is a coefficient for adjusting sensitivity, typically between 0.5 and 1.5, used to control the degree of influence of the content complexity index C on the final slice duration. This parameter T will directly guide the next segmentation operation.

[0072] Optionally, generating a set of knowledge unit fragments and generating fragment index information for the set of knowledge unit fragments includes:

[0073] The frame sequence of the media base data is segmented using the slice driving parameters to generate shot slice fragments;

[0074] The lens slices are subjected to boundary smoothing processing to generate a smooth boundary sequence;

[0075] Speech recognition and topic mutation detection are performed on the audio sequence of the media basic data to generate semantic segment fragments;

[0076] Align the smooth boundary sequence with the temporal position of the semantic slice fragment to generate aligned fragment boundaries;

[0077] Based on the alignment of fragment boundaries, fragments are merged to generate a set of knowledge unit fragments;

[0078] A unique identifier is assigned to the knowledge unit fragment set and associated with the time position, frame range, and subtitle range to generate fragment index information.

[0079] Specifically, this method generates a set of knowledge unit fragments and generates fragment index information for them. It segments the original media stream into independent, meaningful content units based on both visual and semantic cues, and creates a structured index for each unit that can be read and referenced by a machine. The system uses the slice-driving parameters generated in the previous step to guide the segmentation of frame sequences in the media base data. Specifically, the system uses the time points in the keyframe set as initial segmentation points and uses the slice-driving parameters as duration constraints, merging segments with durations much shorter than the parameter with adjacent segments to generate preliminary shot slices. To avoid excessively short or uneven transitions, the system performs boundary smoothing on the shot slices, for example, merging shot segments with durations less than 1 second into neighboring segments, or fine-tuning the cut point to the lowest audio energy point within 0.5 seconds before and after, thereby generating a smooth boundary sequence in the form of a timestamp list. In parallel, the system performs speech recognition on the audio sequence of the media base data to obtain timestamped text, and then applies a topic mutation detection algorithm. This detection identifies shifts in content theme by calculating the cosine distance between the vector representations of adjacent text blocks, such as text every 10 seconds. When this distance exceeds a preset threshold, a theme switch is considered to have occurred, and a semantic boundary is set at that time point, thereby generating semantic slice fragments. Next, the system aligns the smooth boundary sequence with the temporal positions of the semantic slice fragments to generate aligned fragment boundaries. This alignment operation considers visual and semantic cut points as candidate boundaries. If the time difference between a visual and a semantic cut point is less than 2 seconds, the boundary weight at that position is enhanced, forming a strong boundary; otherwise, a cut point existing alone is a weak boundary. The system prioritizes retaining all strong boundaries and, based on the target average duration determined by the slice-driving parameters, selectively retains some weak boundaries among the strong boundaries, ultimately forming a set of aligned fragment boundaries. Based on the timestamps in this set, the system performs final segmentation of the media base data, generating a set of knowledge unit fragments. Finally, the system generates segment index information for each segment in the knowledge unit segment set. Specifically, it assigns a UUID to each segment as a unique identifier, records its start and end timestamps in the original video, converts them into a frame number range (i.e., a frame range), and associates them with the text content recognized by speech within the corresponding time period (i.e., the subtitle range), forming a structured index record containing a unique identifier, time, frame, and subtitle information.

[0080] Optionally, the generation of the multimodal fusion representation vector includes:

[0081] Extract the image sequence corresponding to the frame range in the knowledge unit fragment set, and encode the visual features to generate a visual vector;

[0082] Extract the text data corresponding to the subtitle range in the knowledge unit fragment set, and perform text feature encoding to generate speech recognition text data;

[0083] The text regions in the image sequence are identified and encoded to generate optical character recognition text data.

[0084] The visual vector, the speech recognition text data, and the optical character recognition text data are concatenated to generate a multimodal raw vector;

[0085] The original multimodal vectors are subjected to gating and attention weighting calculations to generate a multimodal fusion representation vector.

[0086] Specifically, this method generates a multimodal fusion representation vector by extracting various information, such as visual, speech-to-text, and image-text information, from a set of knowledge unit fragments. This information is encoded and fused into a unified, high-dimensional digital vector to quantify the core semantics of the fragment and achieve machine-understandable representation. The system extracts the corresponding image sequence from the set of knowledge unit fragments based on the frame range in the fragment index information. The system samples this image sequence at a frequency of one frame per second and uses a pre-trained visual feature encoding model, such as ResNet50, to encode each frame into a feature vector. Then, it performs average pooling on all frame vectors within a fragment to generate a fixed-dimensional visual vector, for example, 2048 dimensions. Simultaneously, the system extracts the text data corresponding to the subtitle range in the fragment index information, i.e., speech recognition text data. This text data is processed using a pre-trained text feature encoding model, such as BERT, to extract its corresponding semantic vector as the feature encoding result. Similarly, the system performs optical character recognition (OCR) on the text regions in the image sequence. After concatenating the extracted text, it is encoded using the same text feature encoding model to generate an OCR text vector. Then, the system concatenates the visual vector, the speech recognition text feature encoding result, and the optical character recognition text vector to generate a temporary multimodal raw vector. Finally, the system performs gating and attention-weighted calculations on this multimodal raw vector. This calculation uses a learnable neural network module to dynamically evaluate the importance of visual, speech, and text information in the current segment and assigns different weights for weighted fusion. This process can be expressed by the following formula.

[0087] ,

[0088] Where M is the final generated multimodal fusion representation vector. Pv, Pa, and Po are the visual vector, speech recognition text vector, and optical character recognition text vector, respectively, mapped to a vector with the same target dimension, such as 768, after being processed by independent linear projection layers. gv, ga, and go are three gating weights corresponding to the visual, speech, and text modalities, respectively. Their values ​​are calculated by a small feedforward neural network using the concatenation of the three projected vectors as input, and normalized by the Softmax function to ensure that the weights sum to one. This weighted summation process ultimately generates a multimodal fusion representation vector that comprehensively reflects the semantics of the fragment content.

[0089] Optionally, generating alignment candidate labels includes:

[0090] Calculate the similarity between the multimodal fusion representation vector and the vectors of each entry in the preset controlled lexicon, and generate lexicon matching results;

[0091] The matching results from the lexicon are queried for their parent concepts in a pre-defined knowledge graph of relationships between descriptive concepts, and ontology association results are generated.

[0092] Merge the lexicon matching results with the ontology association results to generate a candidate tag set;

[0093] Semantic similarity aggregation is performed on the candidate tag set to generate a synonym aggregation set;

[0094] The non-compliant items in the synonym aggregation set are filtered out by a preset list of banned words to generate an available aggregation set;

[0095] Representative items are extracted from the available aggregate set to generate alignment candidate labels.

[0096] Specifically, this method generates aligned candidate tags by transforming machine-readable multimodal fusion representation vectors into a series of pre-screened and standardized text tags aligned with human cognition, providing a semantically clear candidate pool for subsequent indexing. The system performs lexicon matching by calculating the cosine similarity between the multimodal fusion representation vectors and each entry vector in a pre-defined controlled lexicon, generating lexicon matching results. This controlled lexicon is a pre-constructed professional dictionary where each term has been converted into a vector using a text feature encoding model. The system retains terms with similarity scores higher than a pre-defined threshold, such as 0.85, as initial matching items, forming lexicon matching results. Next, to expand the breadth of tags, the system queries the higher-level concepts in a pre-defined knowledge graph describing the relationships between concepts, i.e., broader parent concepts, such as "husky" whose higher-level concept could be "canine". The system traverses one to two levels upwards in the graph for each matching term, obtaining its parent nodes as supplementary information to generate ontology association results. Subsequently, the system merges the dictionary matching results with the ontology association results to obtain an undeduplicated candidate tag set. To address the synonym and near-synonym issues, the system performs semantic similarity aggregation on this candidate tag set. Specifically, the system uses pre-stored word vectors in the dictionary to calculate the similarity between each pair of tags within the set, aggregating tags with similarity exceeding a high threshold, such as 0.95, into a cluster to generate a synonym aggregation set. Then, the system filters the synonym aggregation sets using a pre-defined list of prohibited words. This list contains words that do not conform to platform specifications or are of low quality; if any aggregation cluster contains a prohibited word, the entire cluster is removed, thus generating a usable aggregation set. Finally, the system extracts a representative item from each cluster of the usable aggregation sets to generate the final aligned candidate tags. The extraction rule can be to select the word in the cluster with the highest similarity to the original video clip vector, or to select the most commonly used and standardized word, ensuring that the output candidate tags are concise and representative.

[0097] Optionally, the step of referencing the fragment index information to generate an evidence index set for the alignment candidate labels includes:

[0098] Extract the time identifier of the frame range from the fragment index information to generate a frame identifier set;

[0099] Extract the subtitle range interval identifiers from the segment index information to generate a subtitle range identifier set;

[0100] Associate the frame identifier set and the subtitle interval identifier set with the corresponding alignment candidate tags to generate tag evidence binding pairs;

[0101] Aggregate the aforementioned tag evidence binding pairs to construct a traceable index structure and generate an evidence index set.

[0102] Specifically, this method uses the fragment index information to generate an evidence index set for alignment candidate tags, establishing a source link accurate to the frame and word levels for each automatically generated candidate tag. This makes the indexing results not only usable but also interpretable and auditable. The system parses the fragment index information of the knowledge unit fragment, extracts the start and end timestamps corresponding to the frame range, and combines it with the keyframe set to filter out the time identifiers of all keyframes within that time period, generating a frame identifier set. Simultaneously, the system extracts the subtitle range from the fragment index information, i.e., the speech recognition text and optical character recognition text corresponding to the fragment, and obtains the timestamp information of each word or phrase, generating a subtitle interval identifier set containing the text content and the corresponding time interval. Next, the system performs an association operation, associating the frame identifier set and the subtitle interval identifier set with the corresponding alignment candidate tags, generating tag evidence binding pairs. This association is a type of attribution analysis, where the system traces back the generation source of each alignment candidate tag. If a tag, such as "machine learning," primarily originates from caption text, the system locates the time interval in which the word "machine learning" appears within the caption interval identifier set and uses this interval as its textual evidence. If a tag, such as "cat," is primarily contributed by visual features, the system locates the timestamp of the image frame containing the cat within the frame identifier set as its visual evidence. A tag can be bound to multiple visual or textual evidence simultaneously, forming a tag-evidence binding pair called a "tag-evidence list." Finally, the system aggregates all tag-evidence binding pairs to construct a traceable index structure, generating an evidence index set. This traceable index structure is a set of key-value pairs with aligned candidate tags as keys and their evidence lists as values. Each evidence item is explicitly labeled with its type (e.g., visual or text), source (e.g., ASR or OCR), and precise time position in the original video. This structured evidence index set provides direct data support for subsequent semantic consistency verification and allows any indexing result to be traced back to its specific performance in the source video, achieving transparency in the indexing process.

[0103] Optionally, generating publishable indexing data includes:

[0104] The multimodal fusion representation vector and the evidence index set are subjected to semantic consistency verification and interpretable reasoning to generate structured indexing results;

[0105] Calculate the coverage ratio of the evidence index set to each tag in the structured indexing results, and generate an evidence coverage index;

[0106] Verify the consistency between the evidence coverage index and the label content, and generate a consistency verification result;

[0107] The items that pass the consistency check result are mapped to platform topics and categories to generate platform mapping results;

[0108] Indexing data is synthesized based on the platform mapping results, and rule detection is performed on its length, prohibited words, and format to generate rule detection results;

[0109] Based on the detection results of the rules, correct the violations and generate publishable indexed data.

[0110] Specifically, this method generates publishable indexed data by performing a series of rigorous verifications, screenings, and normalization processes on the candidate tag set generated in the preceding steps. This ensures that the final output indexed data is not only semantically accurate and sufficiently evidence-based, but also fully conforms to the format and content specifications of the publishing platform, achieving the transformation from raw candidates to usable tags. The system performs semantic consistency verification and interpretable reasoning on the multimodal fusion representation vector and the evidence index set. This verification checks whether the semantics of a tag matches the evidence content it is bound to. For example, for textual evidence, the system uses a natural language inference model to determine whether the tag can be reasonably inferred from the text; for visual evidence, the system calls an object detection or scene classification model to verify the evidence frame. Only when the model's confidence score is higher than a preset threshold, such as 0.9, is the evidence considered valid. This step retains tags supported by valid evidence, generating structured indexing results. Next, the system calculates the evidence coverage index for each tag in the structured indexing results. This index quantifies the importance or persistence of a tag in the entire knowledge unit fragment, and its calculation formula is:

[0111] ,

[0112] Where R is the evidence coverage index, Te is the sum of the time spans of all valid evidence for that tag, obtained from the evidence index set, and Ts is the total duration of the current knowledge unit fragment, obtained from the fragment index information. Subsequently, the system checks the consistency of the tag content based on this index. If the R value of a tag is lower than a preset coverage threshold, such as 0.15, it is considered secondary information and removed. Tags that pass this screening constitute the consistency check result. Then, the system maps the passing items in the consistency check result to platform topics and categories, that is, maps the tags generated internally to the platform's publicly available topic pool or preset classification system, generating platform mapping results. Finally, the system synthesizes preliminary indexing data based on the platform mapping results and performs rule checks on its length, prohibited words, and format. For example, it checks whether the total number of tags exceeds 5, whether the total character length exceeds 100 characters, and whether there are any newly added platform prohibited words. This is the rule check result. Based on the detection results of the rules, the system automatically corrects violations, such as truncating tags that exceed the limit by sorting them by importance, or removing tags containing prohibited words, and finally generating fully compliant and publishable indexed data.

[0113] Optionally, generating backflow update parameters using the feedback feature vector includes:

[0114] Collect user click, dwell, and appeal records corresponding to the publishable index data, and generate interaction feedback records;

[0115] The interaction feedback records are normalized to generate a feedback index sequence;

[0116] The correlation between the feedback index sequence and the platform mapping result is encoded and then dimensionality reduction is performed to generate a feedback feature vector;

[0117] The feedback feature vector is used to incrementally update the gating and attention weighting parameters in the multimodal feature extraction and fusion process to generate updated fusion parameter values;

[0118] The feedback feature vector is used to correct the dictionary matching result, and a dictionary matching update value is generated;

[0119] The fusion parameter update value is merged with the dictionary matching update value to generate the backflow update parameter.

[0120] Specifically, this method utilizes the feedback feature vector to generate backflow update parameters, establishing a data-driven closed-loop optimization system. By analyzing real online user interaction behavior, it dynamically and incrementally corrects and improves the accuracy and timeliness of the upstream multimodal feature fusion and tag generation model. The system periodically collects user interaction feedback records for a period after the publishable indexing data goes live, such as 24 hours. This includes user click-through rates on tags, completion rates of corresponding knowledge unit segments, and user-initiated appeals or "irrelevant" tags. Next, the system normalizes these heterogeneous interaction feedback records, for example, converting click-through rates and completion rates to values ​​between 0 and 1, and assigning appeal records as -1, generating a feedback indicator sequence with uniform dimensions. Then, the system associates this feedback indicator sequence with the platform mapping results that generated these feedbacks, i.e., the published tags, encoding it into a high-dimensional feature matrix. This matrix is ​​then processed using a pre-trained dimensionality reduction model, such as a variational autoencoder, to generate a feedback feature vector, for example, 128 dimensions, that characterizes the quality of this indexing. After obtaining this vector, the system executes two parallel update tasks. First, the system incrementally updates the gating and attention weighting parameters in the multimodal feature extraction and fusion process using the feedback feature vector. This update involves treating the feedback feature vector as a reward or penalty signal, calculating a gradient using a backpropagation algorithm, and fine-tuning the gating and attention weighting parameters with a small learning rate, such as 0.00001, thereby generating updated fusion parameter values. Second, the system uses the feedback feature vector to correct the lexicon matching results. Specifically, if a tag continuously receives positive feedback, the system increases its base weight in the controlled lexicon matching process or decreases its triggered similarity threshold; conversely, if it continuously receives negative feedback, its weight is decreased or it is added to a temporary negative sample set, generating updated lexicon matching values. Subsequently, the system merges and encapsulates the updated fusion parameter values ​​with the updated lexicon matching values ​​to generate unified backflow update parameters. Finally, the system uses the SHA-256 hash algorithm and obtains a reliable timestamp generated by a Network Time Protocol (NTP) server or a local high-precision secure crystal oscillator. The initial structured indexing results, fragment index information, and the generated backflow update parameters are packaged and encrypted as a joint binding object, and a unique log fingerprint record is calculated to ensure the authenticity, timeliness, and traceability of the entire feedback optimization link.

[0121] Based on the same inventive concept, such as Figure 3 As shown, the present invention also provides a knowledge unit slicing and interpretable indexing dissemination system for short video or live streaming content, characterized in that the system includes:

[0122] The data acquisition module is used to acquire video stream data and audio data, and to perform time synchronization and audio-visual separation processing on the video stream data and audio data to generate basic media data.

[0123] The slice analysis module is used to analyze the content rhythm pattern of the media basic data and generate slice driving parameters;

[0124] The data segmentation module is used to segment the media basic data based on the slice driving parameters, generate a knowledge unit fragment set, and generate fragment index information for the knowledge unit fragment set;

[0125] The feature representation module is used to perform multimodal feature extraction and fusion processing on the knowledge unit fragment set to generate a multimodal fusion representation vector;

[0126] The tag generation module is used to match the multimodal fusion representation vector with a preset controlled lexicon to generate aligned candidate tags;

[0127] The evidence generation module is used to generate an evidence index set for the alignment candidate tags by referencing the segment index information; including extracting time identifiers of frame ranges from the segment index information to generate a frame identifier set; extracting interval identifiers of subtitle ranges from the segment index information to generate a subtitle interval identifier set; associating the frame identifier set and the subtitle interval identifier set with the corresponding alignment candidate tags to generate tag evidence binding pairs; aggregating the tag evidence binding pairs to construct a traceable index structure and generate the evidence index set;

[0128] The verification and reasoning module is used to perform semantic consistency verification and platform rule mapping on the multimodal fusion representation vector and the evidence index set to generate publishable indexed data. This includes: performing semantic consistency verification and interpretable reasoning on the multimodal fusion representation vector and the evidence index set to generate structured indexing results; calculating the coverage ratio of the evidence index set to each tag in the structured indexing results to generate an evidence coverage index; verifying the consistency between the evidence coverage index and the tag content to generate a consistency verification result; mapping the passed items in the consistency verification result to platform topics and categories to generate platform mapping results; synthesizing indexed data based on the platform mapping results and performing rule detection on its length, prohibited words, and format to generate rule detection results; and correcting violations based on the rule detection results to generate publishable indexed data.

[0129] The feedback module is used to collect online interactive feedback of the publishable indexing data, generate feedback feature vectors, and use the feedback feature vectors to generate backflow update parameters for updating the parameters of multimodal feature extraction and fusion processing and the mapping relationship of the controlled lexicon.

[0130] The evidence storage module is used to store the structured indexing result, the fragment index information and the reflow update parameters as binding objects by using the SHA-256 hash algorithm and the timestamp from NTP or local secure crystal oscillator, and generate log fingerprint records.

[0131] Based on the above disclosure, the present invention also provides an electronic device. For example... Figure 4 As shown, the electronic device of this embodiment includes at least one processor and at least one storage medium electrically connected to each other. The storage medium is electrically connected to the processor, wherein the storage medium stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0132] Based on the same inventive concept, the present invention also provides a storage medium storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0133] It should be noted that the electrical connections between the various units described above do not necessarily represent direct or indirect connections. Any indirect connection method can be applied to the embodiments of the present invention as long as it achieves the purpose of the present invention. The above descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the present invention.

[0134] This invention was applied to an online education platform. This platform possesses a massive library of teaching videos, covering courses ranging from basic science to cutting-edge technologies. A 10-minute (600-second) video on "Introduction to Quantum Computing" was selected as the subject of this analysis. The video has a resolution of 1920x1080, a frame rate of 30fps, and an audio sampling rate of 44.1kHz.

[0135] The system first acquires the video and audio streams. By reading the timestamps of the data packets within the MP4 container, it performs timestamp alignment to ensure that the audio-visual synchronization error is controlled within 40 milliseconds. Then, it demultiplexes the synchronized media data, separating the H.264 encoded video stream and the AAC encoded audio stream. Next, it decodes the video stream into a raw image frame sequence containing 18,000 frames (600 seconds * 30fps) and the audio stream into a PCM format audio sequence. The system performs visual feature change detection on the frame sequence, identifying scene transitions by calculating the structural similarity (SSIM) between adjacent frames. For example, at 02:15 in the video, the scene switches from a close-up of the lecturer to an animated demonstration. The SSIM value S between two adjacent frames drops sharply from 0.95 to 0.22, and the calculated visual change D = 1 - 0.22 = 0.78. This value exceeds the preset visual change threshold of 0.7, therefore the frame at 02:15 is identified as a keyframe and stored in the keyframe set. Simultaneously, the system performs speech energy analysis on the audio sequence, calculating the short-time energy E of approximately 20 milliseconds per frame. For effective speech activity detection, the linear short-time energy E is typically converted to decibel values. The calculation formula is: , This is a reference energy value, which can be set to 1 or a fixed value. (Speech segment) The values ​​are generally higher than -20dB, while in the silent or noisy range... The value is typically below -50 dB. Using this decibel threshold, the generated speech energy feature sequence can effectively distinguish between utterance and silence / noise regions. Finally, the frame sequence, audio sequence, keyframe set, and speech energy feature sequence are integrated into a structured media foundation data set.

[0136] The system analyzes basic media data to generate content rhythm pattern features. For example, in the video segment from 03:00 to 03:30, the lecturer is explaining the complex concept of "quantum entanglement" at a relatively fast pace. The ASR results show an average of 5.2 words per second, and the PPT was switched 3 times during this period. These switches were detected through a set of keyframes, resulting in a high content rhythm pattern feature value. Based on this, the system calculates the content complexity index C. Assuming that the information entropy H of the content rhythm pattern feature sequence is 0.85 and the mutation density Dt is 0.50 (i.e., 1.5 mutation points every 10 seconds) during this time period, and the weights w1 and w2 are both set to 0.5, then the content complexity index C = 0.5 * 0.85 + 0.5 * 0.50 = 0.675. Based on this indicator, the slice granularity is dynamically adjusted. Assuming a baseline duration T0 of 30 seconds and an adjustment coefficient k of 1.2, the slice driving parameter, i.e., the dynamic slice granularity threshold T = 30 / (1+1.2*0.675) = 30 / 1.81 ≈ 16.58 seconds. This indicates that for complex segments, the system tends to produce shorter, more refined slices.

[0137] The system uses the slice-driven parameter T to segment the video. First, it generates preliminary shot slices based on the keyframe set and performs boundary smoothing, merging small segments less than 1 second in length into adjacent segments. In parallel, it performs ASR and topic mutation detection on the audio sequence. By calculating the cosine distance of the BERT vectors of adjacent text blocks (every 10 seconds), it detects a topic shift from "qubit" to "quantum entanglement" at 03:00, with a cosine distance of 0.45, exceeding the threshold of 0.4. Therefore, a semantic slice point is set at this point. Subsequently, visual and semantic slice points are aligned. At 02:59, there is a visual slice point (PPT switch), with a time difference of less than 2 seconds from the semantic slice point at 03:00. The system merges these two, forming a strong boundary at 03:00. Based on these strong and weak boundaries and guided by the slice-driven parameter T, the system finally generates a set of knowledge unit segments. For example, the knowledge unit segment numbered KU-005 has a time range of 03:00-03:17. The system generates the following fragment index information: {UUID:"a1b2c3d4-...",start_time:180.0s,end_time:197.0s,frame_range:[5400,6010],subtitle_range:"Quantum entanglement is a peculiar quantum phenomenon..."}.

[0138] For the knowledge unit fragment KU-005, the system extracts its corresponding image sequence, sampling 17 frames per second, encoding with ResNet50 and using average pooling to generate a 2048-dimensional visual vector. Simultaneously, it extracts the ASR text "Quantum entanglement is a peculiar quantum phenomenon, when two particles are entangled..." within the subtitle range, and identifies the text "QuantumEntanglement" on the PPT in the image, encoding them into text vectors using BERT. Then, these three types of vectors are projected onto a unified 768-dimensional space to obtain Pv, ​​Pa, and Po. Through a gating and attention weighting module, since the core information of this fragment lies in the speech and PPT text, the model calculates the modal weights as follows: gv (visual) = 0.15, ga (speech) = 0.55, go (OCR) = 0.30. The final multimodal fusion representation vector M is generated by weighted summation M = 0.15Pv + 0.55Pa + 0.30*Po.

[0139] The system matches vector M against a controlled lexicon containing 100,000 specialized terms. After calculating cosine similarity, it finds that terms with similarities of 0.92 ("quantum entanglement") and 0.88 ("Bell's inequality") exceed the threshold of 0.85. The system then queries the knowledge graph to find the overarching concept of "quantum entanglement," "quantum information science." After merging, the system performs semantic similarity aggregation on the candidate tag set, clustering "entangled state" and "quantum entanglement" together. After filtering by a list of prohibited words, representative terms are extracted from each cluster, such as selecting "quantum entanglement," which has the highest similarity to vector M, to generate aligned candidate tags.

[0140] The system generates an evidence index for each candidate tag. For example, the evidence for the tag "quantum entanglement" is traced back to the captions of the segment 03:02-03:08 and the PPT keyframe containing the words "QuantumEntanglement" at 03:05. The system performs semantic consistency verification to confirm that the tag matches the evidence content. Then, the evidence coverage index R = Te / Ts is calculated. The total effective evidence duration Te for this tag is 8 seconds (6 seconds of captions + 2 seconds of keyframe duration), and the total segment duration Ts is 17 seconds, so R = 8 / 17 ≈ 0.47. This value is higher than the coverage threshold of 0.15, and the tag is retained. Finally, the system maps "quantum entanglement" to the platform's topic "#quantumphysics" and checks for format compliance, ultimately generating publishable indexed data.

[0141] After the video clip was released, its associated hashtag "#quantumphysics" garnered high click-through rates and user dwell time. The system collected these interactive feedback records, generated a normalized feedback index sequence, and encoded it as a positive feedback feature vector. This vector was used to incrementally update the upstream model. For example, through backpropagation, the gating parameters of the multimodal fusion module were fine-tuned at a learning rate of 0.00001, generating fusion parameter update values ​​such as increasing the weights of ga and go. Simultaneously, the controlled lexicon was corrected to improve the basic matching weight of the term "quantum entanglement," generating lexicon matching update values. These collectively constitute the backflow update parameters. Finally, the system used the SHA-256 hash algorithm and obtained a reliable timestamp generated by a Network Time Protocol (NTP) server or a local high-precision secure crystal oscillator. The initial structured indexing results, fragment index information, and the generated backflow update parameters were packaged and encrypted as a joint binding object to calculate a unique log fingerprint record.

[0142] Through the above embodiments, the present invention successfully and automatically divides a 10-minute teaching video into 18 independent knowledge unit segments, and generates an average of 3.5 highly relevant and interpretable indexes for each segment.

[0143] The above description is merely an exemplary embodiment of the present invention and should not be construed as limiting the scope of the invention. Any equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. Those skilled in the art will readily conceive of other embodiments of the invention upon considering the disclosure of the specification and practical truths. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or conventional techniques in the art not described herein.

Claims

1. A method for disseminating knowledge unit slices and interpretable indexing of short video or live stream content, characterized in that: The method includes: Acquire video stream data and audio data, and perform time synchronization and audio-visual separation processing on the video stream data and audio data to generate basic media data; Analyze the content rhythm pattern of the media basic data to generate slice driving parameters; The media base data is segmented based on the slice driving parameters to generate a knowledge unit fragment set, and fragment index information is generated for the knowledge unit fragment set. Multimodal feature extraction and fusion processing are performed on the knowledge unit fragment set to generate a multimodal fusion representation vector; The multimodal fusion representation vector is matched with a preset controlled lexicon to generate aligned candidate tags; The fragment index information is used to generate an evidence index set for the alignment candidate tags; this includes extracting time identifiers of frame ranges from the fragment index information to generate a frame identifier set; extracting interval identifiers of subtitle ranges from the fragment index information to generate a subtitle interval identifier set; associating the frame identifier set and the subtitle interval identifier set with the corresponding alignment candidate tags to generate tag evidence binding pairs; aggregating the tag evidence binding pairs to construct a traceable index structure and generate an evidence index set; The process involves performing semantic consistency verification and platform rule mapping on the multimodal fusion representation vector and the evidence index set to generate publishable indexed data. This includes: performing semantic consistency verification and interpretable reasoning on the multimodal fusion representation vector and the evidence index set to generate structured indexing results; calculating the coverage ratio of the evidence index set to each tag in the structured indexing results to generate an evidence coverage index; verifying the consistency between the evidence coverage index and the tag content to generate a consistency verification result; mapping the passed items in the consistency verification result to platform topics and categories to generate platform mapping results; synthesizing indexed data based on the platform mapping results and performing rule checks on its length, prohibited words, and format to generate rule detection results; and correcting violations based on the rule detection results to generate publishable indexed data. The online interactive feedback of the publishable index data is collected, a feedback feature vector is generated, and the feedback feature vector is used to generate backflow update parameters to update the multimodal feature extraction and fusion process and the controlled lexicon matching process. Using the SHA-256 hash algorithm and a timestamp derived from an NTP or local secure crystal oscillator, the structured indexing result, the fragment index information, and the reflow update parameters are used as binding objects for evidence storage, generating a log fingerprint record.

2. The knowledge unit slicing and interpretable indexing propagation method for short video or live stream content according to claim 1, characterized in that, The generated media basic data includes: Acquire video stream data and audio data, align timestamps, and generate synchronized media data; Perform a demultiplexing operation on the synchronous media data to generate audio-visual separated data; Decode the audio-visual separation data to generate a frame sequence and an audio sequence; Visual feature change detection is performed on the frame sequence to generate a keyframe set; The audio sequence is subjected to speech energy analysis to generate a speech energy feature sequence; The frame sequence, the audio sequence, the keyframe set, and the speech energy feature sequence are integrated to generate basic media data.

3. The knowledge unit slicing and interpretable indexing propagation method for short video or live stream content according to claim 2, characterized in that, The slice generation driving parameters: Analyze the switching frequency of frame sequences and the speech rate changes of audio sequences in the media basic data to generate content rhythm pattern features; Calculate the information entropy and mutation point density of the content rhythm pattern features to generate a content complexity index; The slice granularity threshold is dynamically adjusted based on the content complexity index to generate slice driving parameters.

4. The knowledge unit slicing and interpretable indexing propagation method for short video or live stream content according to claim 3, characterized in that, The process of generating a set of knowledge unit fragments and generating fragment index information for the set of knowledge unit fragments includes: The frame sequence of the media base data is segmented using the slice driving parameters to generate shot slice fragments; The lens slices are subjected to boundary smoothing processing to generate a smooth boundary sequence; Speech recognition and topic mutation detection are performed on the audio sequence of the media basic data to generate semantic segment fragments; Align the smooth boundary sequence with the temporal position of the semantic slice fragment to generate aligned fragment boundaries; Based on the alignment of fragment boundaries, fragments are merged to generate a set of knowledge unit fragments; A unique identifier is assigned to the knowledge unit fragment set and associated with the time position, frame range, and subtitle range to generate fragment index information.

5. The knowledge unit slicing and interpretable indexing propagation method for short video or live stream content according to claim 4, characterized in that, The generated multimodal fusion representation vector includes: Extract the image sequence corresponding to the frame range in the knowledge unit fragment set, and encode the visual features to generate a visual vector; Extract the text data corresponding to the subtitle range in the knowledge unit fragment set, and perform text feature encoding to generate speech recognition text data; The text regions in the image sequence are identified and encoded to generate optical character recognition text data. The visual vector, the speech recognition text data, and the optical character recognition text data are concatenated to generate a multimodal raw vector; The original multimodal vectors are subjected to gating and attention weighting calculations to generate a multimodal fusion representation vector.

6. The knowledge unit slicing and interpretable indexing propagation method for short video or live stream content according to claim 5, characterized in that, The generation of alignment candidate labels includes: Calculate the similarity between the multimodal fusion representation vector and the vectors of each entry in the preset controlled lexicon, and generate lexicon matching results; The matching results from the lexicon are queried for their parent concepts in a pre-defined knowledge graph of relationships between descriptive concepts, and ontology association results are generated. Merge the lexicon matching results with the ontology association results to generate a candidate tag set; Semantic similarity aggregation is performed on the candidate tag set to generate a synonym aggregation set; The non-compliant items in the synonym aggregation set are filtered out by a preset list of banned words to generate an available aggregation set; Representative items are extracted from the available aggregate set to generate alignment candidate labels.

7. The knowledge unit slicing and interpretable indexing propagation method for short video or live stream content according to claim 6, characterized in that, The step of generating backflow update parameters using the feedback feature vector includes: Collect user click, dwell, and appeal records corresponding to the publishable index data, and generate interaction feedback records; The interaction feedback records are normalized to generate a feedback index sequence; The correlation between the feedback index sequence and the platform mapping result is encoded and then dimensionality reduction is performed to generate a feedback feature vector; The feedback feature vector is used to incrementally update the gating and attention weighting parameters in the multimodal feature extraction and fusion process to generate updated fusion parameter values; The feedback feature vector is used to correct the dictionary matching result, and a dictionary matching update value is generated; The fusion parameter update value is merged with the dictionary matching update value to generate the backflow update parameter.

8. A knowledge unit slicing and interpretable indexing and dissemination system for short video or live stream content, characterized in that: The system includes: The data acquisition module is used to acquire video stream data and audio data, and to perform time synchronization and audio-visual separation processing on the video stream data and audio data to generate basic media data. The slice analysis module is used to analyze the content rhythm pattern of the media basic data and generate slice driving parameters; The data segmentation module is used to segment the media basic data based on the slice driving parameters, generate a set of knowledge unit fragments, and generate fragment index information for the set of knowledge unit fragments. The feature representation module is used to perform multimodal feature extraction and fusion processing on the knowledge unit fragment set to generate a multimodal fusion representation vector; The tag generation module is used to match the multimodal fusion representation vector with a preset controlled lexicon to generate aligned candidate tags; The evidence generation module is used to generate an evidence index set for the alignment candidate tags by referencing the segment index information; including extracting time identifiers of frame ranges from the segment index information to generate a frame identifier set; extracting interval identifiers of subtitle ranges from the segment index information to generate a subtitle interval identifier set; associating the frame identifier set and the subtitle interval identifier set with the corresponding alignment candidate tags to generate tag evidence binding pairs; aggregating the tag evidence binding pairs to construct a traceable index structure and generate the evidence index set; The verification and reasoning module is used to perform semantic consistency verification and platform rule mapping on the multimodal fusion representation vector and the evidence index set to generate publishable indexed data. This includes: performing semantic consistency verification and interpretable reasoning on the multimodal fusion representation vector and the evidence index set to generate structured indexing results; calculating the coverage ratio of the evidence index set to each tag in the structured indexing results to generate an evidence coverage index; verifying the consistency between the evidence coverage index and the tag content to generate a consistency verification result; mapping the passed items in the consistency verification result to platform topics and categories to generate platform mapping results; synthesizing indexed data based on the platform mapping results and performing rule detection on its length, prohibited words, and format to generate rule detection results; and correcting violations based on the rule detection results to generate publishable indexed data. The feedback module is used to collect online interactive feedback of the publishable indexing data, generate feedback feature vectors, and use the feedback feature vectors to generate backflow update parameters for updating the parameters of multimodal feature extraction and fusion processing and the mapping relationship of the controlled lexicon. The evidence storage module is used to store the structured indexing result, the fragment index information and the backflow update parameters as binding objects by using the SHA-256 hash algorithm and the timestamp from NTP or local secure crystal oscillator, and generate log fingerprint records.

9. A computer storage medium, characterized in that, The system stores one or more programs that, when executed, can implement the knowledge unit slicing and interpretable indexing propagation method for short video or live streaming content as described in any one of claims 1 to 8.

10. A device, characterized in that: It includes a processor, a communication interface, a memory, and a communication bus; the memory stores at least one program that can be loaded by the processor and executed from the computer storage medium as described in claim 9.

Images