A deep learning-based film and television knowledge base automatic updating method and system

Abstract

This invention discloses a method and system for automatically updating a film and television knowledge base based on deep learning, comprising the following steps: S1, synchronously collecting multimodal streaming data of film and television, extracting timestamps and metadata to generate an initial dataset; S2, performing spatiotemporal alignment and tension-aware calculations, and non-uniformly segmenting and filtering key plot fragment sets; S3, extracting multi-source feature vectors and dimensionally aligning them, constructing a heterogeneous topology graph, and outputting multi-dimensional fusion feature encoding; S4, inputting it into an improved TPLinker model, based on a subspace matrix mapping mechanism that separates text and cross-modal features, and outputting knowledge triples and confidence scores; S5, comparing with historical knowledge, performing spatiotemporal dual verification when logical mutual exclusion is detected, and generating update instructions; S6, calculating topological deformation forces according to the instructions and iteratively updating the edge weights of associated nodes, and executing automatic knowledge base updates. This invention effectively improves the accuracy of knowledge extraction and the stability of graph evolution.

Description

Technical Field

[0001] This invention relates to the field of cross-integration of artificial intelligence and film and television big data processing, and in particular to a method and system for automatic updating of film and television knowledge base based on deep learning. Background Technology

[0002] Multimodal deep learning and knowledge graph technologies, due to their powerful fusion and representation capabilities in unstructured data processing and symbolic reasoning, have been widely applied in recent years in fields such as film and television media asset management, intelligent content review, and personalized recommendation, becoming an important development direction for the digital and intelligent transformation of the film and television industry. However, in practical applications, the construction and updating of film and television knowledge bases face many challenges, such as heterogeneous data modalities, complex plot state evolution, and strong spatiotemporal context dependencies. The automated implementation of multimodal knowledge graphs is still constrained by many factors.

[0003] Most current film and television knowledge extraction methods rely on feature input from a single text modality, making it difficult to fully utilize multi-source information such as continuous video frames, audio spectra, and speech-to-text transcription, resulting in a lack of comprehensiveness in modeling complex plot states. Some systems only use fixed segmentation strategies to process long video data, ignoring the differences in information density brought about by multi-modal resonances such as sudden changes in facial micro-expressions, spectral energy jumps, and emotional polarity reversals, resulting in low efficiency in screening high-value key plot segments and limiting the accuracy of feature encoding.

[0004] Furthermore, most existing film and television knowledge update mechanisms are static overlays or simple additions, failing to perform spatiotemporal dual verification of the logical mutual exclusion between newly extracted knowledge and content already stored in traditional knowledge bases. This results in data noise or parallel worldview branches directly polluting the backbone network, making it difficult to adapt to the dynamic evolution of long film and television plots on time and space lines. Moreover, the lack of a smooth propagation and convergence mechanism based on topological deformation forces during knowledge injection can easily cause drastic fluctuations in the weights of local nodes, seriously affecting the practical value and topological stability of film and television knowledge bases in real long-tail scenarios.

[0005] Therefore, how to provide a method and system for automatically updating a film and television knowledge base based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a method and system for automatically updating a film and television knowledge base based on deep learning. This invention fully integrates key steps such as multimodal spatiotemporal alignment, plot tension perception calculation, improved TPLinker model extraction, spatiotemporal dual verification, and topological deformation force iteration. It constructs a dynamic evolution process for film and television knowledge with high-value plot segment selection, cross-modal entity association encoding, logical conflict resolution, and smooth convergence of network weights, achieving automatic construction and self-optimization of the knowledge graph in complex long-video environments. This invention innovatively constructs a subspace matrix mapping mechanism that separates text and cross-modal elements within the improved TPLinker model, effectively isolating feature interference between heterogeneous modalities and significantly improving the accuracy of cross-modal entity and relationship handshakes. This invention possesses advantages such as deep fusion of multimodal features, high confidence in knowledge extraction, rigorous spatiotemporal conflict verification, and strong adaptability in topological updates. It can significantly improve the accuracy of knowledge entry and the stability of graph evolution in complex film and television scenarios, thereby effectively solving problems such as limitations of single-modal representation, coarse handling of logical mutual exclusion, and easy collapse of knowledge base updates in existing methods.

[0007] An automatic update method for a film and television knowledge base based on deep learning according to an embodiment of the present invention includes the following steps: S1. Synchronously collect multimodal streaming data of film and television, extract global timestamps and basic scene metadata, and generate an initial film and television dataset; S2. Perform multimodal spatiotemporal alignment and plot tension perception calculation on the initial film and television dataset to generate a plot tension curve, and perform non-uniform segmentation on the initial film and television dataset to select a set of high-value key plot segments. S3. Extract the feature vectors of continuous frames, spectral features and speech-to-text from high-value key plot segments and align them with dimensions. Generate gating coefficients and construct a heterogeneous topology graph. Output multi-dimensional fusion feature encoding. S4. Input the multidimensional fusion feature encoding into the improved TPLinker model to obtain candidate knowledge triples and fact verification confidence. The improved TPLinker model includes an input embedding module, a heterogeneous Token Pair mapping module, an entity handshake decoding module, a cross-modal relationship handshake module, and a confidence verification module. The heterogeneous Token Pair mapping module constructs a subspace matrix mapping mechanism that separates text from cross-modal relationships. S5. Compare the candidate knowledge triples and the fact verification confidence with the target historical knowledge triples. When logical mutual exclusion is detected, perform timeline verification and spatial line verification, and generate triple evolution update instructions or branch isolation update instructions. S6. Calculate the topological deformation force generated by the newly injected triplet according to the triplet evolution update instruction or the branch isolation update instruction, and iteratively update the edge weights of the connected nodes to perform automatic updates of the film and television knowledge base.

[0008] Optionally, S1 specifically includes: S11. Extract continuous frame images from the video stream using the player's decoding protocol, extract image sequences at a preset frame rate, and use the short-time Fourier transform method to perform frequency domain conversion on the time domain signal of the audio stream to extract spectral features. S12. Call the automatic speech recognition interface to convert the audio stream into speech-to-text composed of text, read the system encoding time of the video stream and audio stream, calculate the time deviation value and fill in the deviation frame, generate the aligned global timestamp, parse the encapsulation file header information of the video stream, and extract the shooting scene identifier and lens position parameters as basic scene metadata. S13. Pack and bind continuous frame data, spectral features, speech-to-text transcription, global timestamps and basic scene metadata in chronological order to generate an initial film and television dataset.

[0009] Optionally, S2 specifically includes: S21. Perform face region localization on consecutive frames in the initial video dataset, calculate the displacement difference of facial key point coordinates in adjacent frames, mark frames with displacement difference greater than the preset mutation threshold as mutation points, and use the displacement difference corresponding to the mutation point as a numerical vector to extract facial micro-expression mutation features. S22. Calculate the sum of squares of amplitude of spectral features frame by frame along the time direction, calculate the difference between the sum of squares of amplitude of adjacent frames, mark the time when the difference is greater than the preset energy threshold as the energy jump point, and extract the energy jump point of spectral features. S23. Divide the speech-to-text into independent sentences according to punctuation marks, traverse the preset positive sentiment dictionary and negative sentiment dictionary, count the number of positive words and negative words in each sentence, subtract the number of negative words from the number of positive words to obtain the sentiment difference, and mark the reversal position as the sentiment polarity reversal point when the sentiment difference sign of adjacent sentences reverses from positive to negative or from negative to positive, and extract the sentiment polarity reversal point of the speech-to-text. S24. Add the energy jump points of facial micro-expression mutation features and spectral features at the same moment in chronological order with the emotional polarity reversal points of the speech-to-text to generate a plot tension curve. S25. Use a one-dimensional sliding window to slide along the plot tension curve, calculate the average value of the values ​​in each sliding window, and determine the interval where all values ​​in the window are greater than the average value of the adjacent windows as the peak interval. Perform non-uniform segmentation on the initial film and television dataset based on the start and end time of the peak interval. S26. Extract and package the continuous frame data, spectral features, and speech-to-text corresponding to the segmented peak intervals to select a set of high-value key plot segments.

[0010] Optionally, S3 specifically includes: S31. Divide the continuous frames of high-value key plot segments into image blocks of a preset size, perform weighted summation and pooling downsampling on the color channel values ​​of pixels in each image block, and extract visual feature vectors. S32. Divide the spectral features into segments according to the time window, calculate the statistical average and variance of the frequency amplitude in each segment, extract the auditory feature vector, convert each word in the speech-to-text into a word vector of a preset dimension and add them in word order, extract the text feature vector, and convert the three feature vectors to the preset dimension through linear mapping matrix multiplication to complete the dimension alignment. S33. Read the plot tension value of the high-value key plot fragment set at the corresponding moment in the plot tension curve, substitute the plot tension value into the exponential operation formula and divide by one plus the exponential operation result, calculate the output value between 0 and 1, and use the output value as the gating coefficient. S34. Multiply each value in the dimension-aligned visual feature vector, auditory feature vector, and text feature vector by the gating coefficient, and concatenate the three multiplied vectors by concatenating their first and last positions in sequence to generate a long vector. S35. Cut the long vector into equal parts according to the preset length, and treat each segment of the vector as a multimodal node. Connect the nodes belonging to different vector segments at the same time to construct cross-modal interaction edges. Connect the nodes of adjacent time points within the same vector segment to construct temporal edges. Combine them to generate a heterogeneous topology graph. S36. For each multimodal node in the heterogeneous topology graph, find the neighboring nodes that can be reached along the connected cross-modal interaction edges and temporal edges. Add the values ​​of the corresponding positions of the vectors of all the neighboring nodes found and divide by the total number of neighboring nodes to calculate the average vector. Add the average vector to the value of the corresponding position of the vector of the current node and perform neighboring node feature aggregation. S37. Perform a linear transformation on each value in the aggregated node vector, calculate the predicted value, calculate the absolute value of the difference between the predicted value and the historical true value as the error, propagate the error in reverse along the connection of the topology graph, update the edge weight by subtracting the product of the edge weight and the preset learning rate from each edge weight, iteratively execute the neighbor node feature aggregation and edge weight update until the error is lower than the preset convergence threshold, encode cross-modal entity association and action interaction information, and output multi-dimensional fusion feature encoding.

[0011] Optionally, S4 specifically includes: S41. Input the multi-dimensional fusion feature encoding into the input embedding module of the improved TPLinker model, and divide it into a text word vector sequence and a cross-modal feature vector sequence according to the sequence order. Obtain the total number of times each word in the text word vector sequence appears in the preset document library and divide it by the total number of documents in the preset document library to obtain the word frequency value. S42. Divide the total number of documents by the number of documents to obtain the inverse document frequency value, and multiply it with the word frequency value to obtain the word frequency inverse document frequency value. Add the corresponding word frequency inverse document frequency value and the word order number in the sentence to the value of each position of each word vector in the text word vector sequence to obtain the modality category to which each vector belongs in the cross-modal feature vector sequence. S43. Label the visual category as value 1 and the auditory category as value 2. Add the corresponding modal category label value to the first value of each vector in the cross-modal feature vector sequence, and concatenate it with the cross-modal feature vector sequence in the order of first and last to generate the total input embedding matrix. S44. Input the total input embedding matrix into the heterogeneous Token Pair mapping module to construct a subspace matrix mapping mechanism that separates text and cross-modal features. Copy the total input embedding matrix into two copies and replace all values ​​at the row and column positions that do not belong to the text word vector sequence and the cross-modal feature vector sequence with 0 respectively to construct the text subspace matrix and the cross-modal subspace matrix. S45. For the text subspace matrix, extract the vector in the i-th row and the vector in the j-th column, multiply the values ​​at the same index position in the two vectors and add them to obtain the first dot product value. S46. Fill the first dot product value into the i-th row and j-th column of the blank matrix. Iterate through all rows and columns and repeat the multiplication and addition operations to generate the text Token Pair matrix. For the cross-modal subspace matrix, extract the m-th row vector and the n-th column vector and perform the same multiplication and addition operations to generate the cross-modal Token Pair matrix. S47. Input the text Token Pair matrix and the cross-modal Token Pair matrix into the entity handshake decoding module, read the start word index number and end word index number corresponding to the pre-annotated entity words, locate the cell in the text Token Pair matrix where the row corresponding to the start word index number and the column corresponding to the end word index number intersect, modify the value in the cell to 1, and modify the value in all other cells in the text Token Pair matrix to 0. S48. Read the text word index number corresponding to the pre-labeled entity words and the cross-modal index number of the corresponding image region features. Perform the same localization modification operation as S47 in the cross-modal Token Pair matrix. Stack the two matrices after numerical modification in the top and bottom positions to generate an entity handshake feature map. S49. Input the entity handshake feature map into the cross-modal relationship handshake module for calculation and input it into the confidence verification module to output the candidate knowledge triples and the fact verification confidence.

[0012] Optionally, S49 specifically includes: S491. Input the entity handshake feature map into the cross-modal relationship handshake module, read the text word index number and cross-modal index number corresponding to the first entity and the second entity extracted in the entity handshake decoding stage, locate the cell in the text token pair matrix where the row corresponding to the text word index number of the first entity intersects with the column corresponding to the text word index number of the second entity, and locate the cell in the cross-modal token pair matrix where the row corresponding to the cross-modal index number of the first entity intersects with the column corresponding to the cross-modal index number of the second entity. S492. Read the numerical code corresponding to each relation category in the preset relation category dictionary table, and fill the numerical code into the intersection cell located in the text Token Pair matrix and the cross-modal Token Pair matrix in turn. Modify the values ​​in all other cells in the two matrices to 0. Stack the text Token Pair matrix and the cross-modal Token Pair matrix after filling in the numerical code in the top and bottom positions to generate a cross-modal relation handshake feature map. S493. Traverse all cell numbers in the cross-modal relationship handshake feature map, compare and find the number with the largest value, record the row index and column index of the largest number, search for the specific relationship text corresponding to the largest number in the preset relationship category dictionary table, and extract the relationship category information connecting the first entity and the second entity. S494. Input the entity boundary span information and the relationship category information connecting the first entity and the second entity into the confidence verification module. Extract the first text vector pointed to by the text word index number and the first cross-modal vector pointed to by the cross-modal index number corresponding to the first entity from the entity boundary span information. Extract the second text vector pointed to by the text word index number and the second cross-modal vector pointed to by the cross-modal index number corresponding to the second entity. S495. Add the values ​​at the same position in the first text vector and the first cross-modal vector to generate the first fusion vector. Add the values ​​at the same position in the second text vector and the second cross-modal vector to generate the second fusion vector. Multiply the value at each position in the first fusion vector with the value at the same position in the second fusion vector and sum all the products to obtain the cross-modal vector dot product value. S496. Locate the cell containing the largest numerical number recorded in the cross-modal relationship handshake feature map, read the original values ​​in the text token pair matrix and cross-modal token pair matrix before stacking, and add them to obtain the basic category score. Add the cross-modal vector dot product value to the basic category score to obtain the total score. Calculate the total score using the Sigmoid function and output the fact verification confidence score. S497. Extract the text word string pointed to by the text word index number corresponding to the first entity from the entity boundary span information as the head entity text, extract the text word string pointed to by the text word index number corresponding to the second entity as the tail entity text, and use the relation text in the relation category information connecting the first entity and the second entity as the relation predicate. S498. Concatenate the head entity text, relational predicate, and tail entity text in sequence using underscores to generate a string, which represents the candidate knowledge triplet representing the current plot. Connect the string of the candidate knowledge triplet with the calculated fact verification confidence value using commas, and output the candidate knowledge triplet and the fact verification confidence value.

[0013] Optionally, S5 specifically includes: S51. Input the candidate knowledge triples and the fact verification confidence into the film and television knowledge base, extract the head entity text, relational predicate and tail entity text in the candidate knowledge triples, and compare them character by character with the head entity text, relational predicate and tail entity text of all target historical knowledge triples stored in the film and television knowledge base. When the head entity text and relational predicate are found to be completely the same but the tail entity text is inconsistent, it is determined that logical mutual exclusion is detected. S52. When logical mutual exclusion is detected, read the global timestamp corresponding to the current candidate knowledge triplet and the global timestamp corresponding to the target historical knowledge triplet that has logical mutual exclusion. Subtract the previous timestamp value from the latter timestamp value to calculate the time difference. Determine whether the time difference is greater than the preset time threshold. If the time difference is greater than the preset time threshold, it is determined that the timeline verification has passed. If the time difference is less than or equal to the preset time threshold, it is determined that the timeline verification has not passed. S53. Extract the shooting scene identifier and camera position parameters from the basic scene metadata corresponding to the current candidate knowledge triplet. Extract the shooting scene identifier and camera position parameters from the basic scene metadata corresponding to the target historical knowledge triplet that has logical mutual exclusion. Compare the two sets of shooting scene identifiers with strings. Subtract the horizontal rotation angle value and vertical pitch angle value corresponding to the two sets of camera position parameters to calculate the angle difference. S54. Determine whether the angle difference is greater than the preset angle threshold. If the angle difference is greater than the preset angle threshold or the shooting scene labels are inconsistent, it is determined that the spatial line verification is passed. If the angle difference is less than or equal to the preset angle threshold and the shooting scene labels are consistent, it is determined that the spatial line verification is not passed. S55. When both timeline and spatial line checks are passed, the logical mutual exclusion is determined to be a normal change caused by the development of the plot, and a triplet evolution update instruction is generated. When either the timeline or spatial line check is not passed, the logical mutual exclusion is determined to be data noise or a branch of a parallel worldview, and a branch isolation update instruction is generated.

[0014] Optionally, S6 specifically includes: S61. When a triplet evolution update instruction is received, locate the target nodes corresponding to the head entity text and tail entity text of the candidate knowledge triplet in the topology network of the film and television knowledge base and establish new injection edges. When a branch isolation update instruction is received, create two isolated nodes in the topology network to store the head entity text and tail entity text respectively and establish new injection edges. Assign the fact verification confidence value corresponding to the candidate knowledge triplet to the new injection edge as the initial edge weight. S62. Calculate the sum of edge weights of all edges connected to the two ends of the newly injected edge. Divide the initial edge weight by the sum of edge weights to get the injection influence ratio, and multiply it by the initial edge weight to get the topological deformation force generated by the newly injected triplet. Propagate the topological deformation force outward along the edges of the graph network, find the first layer of associated nodes that are directly connected to the newly injected edge, and update the edge weight value by adding the topological deformation force value. S63. Continue searching along the edge to reach the next layer of associated nodes. Multiply the updated edge weight value of the current layer associated node by the preset attenuation coefficient to calculate the attenuation propagation force. Add the attenuation propagation force value to the edge weight value of the next layer associated node for updating. S64. Subtract the edge weight of the corresponding node calculated in the previous iteration from the edge weight of each node calculated in the current iteration to calculate the weight change. Determine whether the absolute value of the weight change of all nodes is less than the preset threshold. If they are all less than the preset threshold, it is determined that the weight change of all nodes in the network has converged. If any absolute value is greater than or equal to the preset threshold, return to continue to perform the operation of finding the next layer of related nodes and updating the edge weight. S65. When the weight change of all nodes in the network converges to the preset threshold, save the current edge weight values ​​of all nodes in the topology network and complete the automatic update of the film and television knowledge base.

[0015] An automatic update system for a film and television knowledge base based on deep learning according to an embodiment of the present invention includes: The data acquisition and initial construction module is used to synchronously acquire multimodal video streaming data and extract global timestamps and basic scene metadata from the multimodal video streaming data to generate an initial video dataset. The plot tension perception and segmentation module is used to perform multimodal spatiotemporal alignment and plot tension perception calculation on the initial film and television dataset, generate plot tension curves, and perform non-uniform segmentation on the initial film and television dataset to filter out a set of high-value key plot segments. The cross-modal feature encoding module is used to extract the feature vectors of continuous screen frames, spectral features and speech-to-text in the high-value key plot segments, align them in dimensions, generate gating coefficients and construct a heterogeneous topology graph, and output multi-dimensional fusion feature encoding. The cross-modal entity relation extraction module is used to input multi-dimensional fusion feature encoding into the improved TPLinker model to obtain candidate knowledge triples and fact verification confidence. The improved TPLinker model includes an input embedding module, a heterogeneous Token Pair mapping module, an entity handshake decoding module, a cross-modal relation handshake module, and a confidence verification module. The heterogeneous Token Pair mapping module constructs a subspace matrix mapping mechanism that separates text from cross-modal data. The spatiotemporal dual verification and conflict handling module is used to compare the candidate knowledge triples and the fact verification confidence with the target historical knowledge triples. When logical mutual exclusion is detected, timeline verification and spatial line verification are performed to generate triple evolution update instructions or branch isolation update instructions. The topology deformation and dynamic update module is used to calculate the topology deformation force generated by the newly injected triples according to the triple evolution update instruction or the branch isolation update instruction, and to iteratively update the edge weights of the connected nodes, thereby performing automatic updates of the film and television knowledge base.

[0016] The beneficial effects of this invention are: This invention addresses the challenges of heterogeneous modalities, uneven information density, and conflicting knowledge updates in film and television data by simultaneously acquiring multimodal streaming data and extracting global timestamps. It employs facial micro-expression mutations, spectral energy jumps, and emotional polarity reversals to calculate the plot tension curve. Based on peak intervals, it performs non-uniform segmentation to select a set of high-value key plot segments. By dimensional alignment and gating weighting of continuous frames, spectral features, and speech-to-text within the segment set, a heterogeneous topology graph is constructed, neighbor node features are aggregated, and a multidimensional fusion feature code is output. This multidimensional fusion feature code is input into an improved TPLinker model. Through a subspace matrix mapping mechanism that separates text and cross-modal data, text token pair matrices and cross-modal tokens are generated in independent subspaces. The Pair matrix, after entity handshake decoding and cross-modal relationship handshake calculation, outputs candidate knowledge triples and fact verification confidence scores. During the knowledge ingestion phase, combined with global timestamps and basic scene metadata, temporal and spatial line verifications are performed on logically mutually exclusive triples, generating triple evolution update instructions or branch isolation update instructions. Furthermore, based on the update instructions, the topological deformation force generated by the newly injected triples is calculated, decaying and propagating outwards along the graph network edges, iteratively updating the edge weights of associated nodes until the entire network converges. Ultimately, this achieves accurate extraction of cross-modal knowledge from the film and television knowledge base, spatiotemporal resolution of logical conflicts, and smooth evolution and update of the graph topology, effectively improving the accuracy of multimodal entity relationship extraction, the rigor of conflict knowledge handling, and the stability of the dynamic evolution of the knowledge network. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an automatic update method for a film and television knowledge base based on deep learning proposed in this invention; Figure 2 This is a structural diagram of an automatic update system for a film and television knowledge base based on deep learning proposed in this invention. Figure 3 This is a flowchart of the cross-modal knowledge extraction and spatiotemporal dual verification update process based on the improved TPLinker model proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figures 1-3 A method for automatically updating a film and television knowledge base based on deep learning includes the following steps: S1. Synchronously collect multimodal video streaming data, including continuous frames in the video stream, spectral features in the audio stream, and speech-to-text transcription, and extract the global timestamp and basic scene metadata of the multimodal video streaming data to generate an initial video dataset. S2. Perform multimodal spatiotemporal alignment and plot tension perception calculation on the initial film and television dataset. By jointly extracting the facial micro-expression mutation features, the energy jump points of the spectral features, and the emotional polarity reversal points of the speech-to-text in continuous frame images, a plot tension curve distributed over time is generated. Based on the peak interval of the plot tension curve, the initial film and television dataset is non-uniformly segmented to select a set of high-value key plot segments. S3. Extract the feature vectors of continuous frames, spectral features and speech-to-text in the high-value key plot segments and align them in dimensions. Generate gating coefficients based on the plot tension value determined by the plot tension curve through the activation function. Perform weighted splicing and construct a heterogeneous topology graph containing multimodal nodes and cross-modal interaction edges. Perform neighbor node feature aggregation and edge weight iterative update to encode cross-modal entity association and action interaction information, and output multi-dimensional fusion feature encoding. S4. Input the multi-dimensional fusion feature encoding into the improved TPLinker model to obtain candidate knowledge triples representing the current plot and fact verification confidence. The improved TPLinker model includes an input embedding module, a heterogeneous TokenPair mapping module, an entity handshake decoding module, a cross-modal relationship handshake module, and a confidence verification module. The heterogeneous TokenPair mapping module constructs a subspace matrix mapping mechanism that separates text from cross-modal relationships. S5. Compare the candidate knowledge triples and the fact verification confidence with the target historical knowledge triples stored in the film and television knowledge base. When logical mutual exclusion is detected, perform timeline verification by combining the global timestamp and perform spatial line verification by extracting spatial context features by combining the basic scene metadata. Based on the verification results, generate triple evolution update instructions or branch isolation update instructions for conflicting knowledge. S6. According to the triplet evolution update instruction or branch isolation update instruction, inject the candidate knowledge triplet into the topology network of the film and television knowledge base, calculate the topology deformation force generated by the newly injected triplet, and propagate the topology deformation force outward along the edge of the graph network. Iteratively update the edge weights of the associated nodes connected to it until the weight change of all nodes in the network converges to the preset threshold, and complete the automatic update of the film and television knowledge base.

[0020] This implementation significantly improves the accuracy of film and television knowledge extraction and the stability of graph evolution. By simultaneously collecting video, audio, and text stream data and extracting timestamps and scene metadata, a unified spatiotemporal benchmark alignment of multimodal data is achieved. Joint extraction of facial micro-expressions, audio energy jumps, and text emotional polarity generates a plot tension curve, which can intelligently identify plot climaxes and non-uniform segmentations, accurately remove redundant transitions, and efficiently filter out high-value key plot segments. Based on this, tension values ​​are used to generate gating coefficients to construct a heterogeneous topology graph, effectively encoding cross-modal entity relationships and outputting high-quality multi-dimensional fusion feature encodings. The encodings are input into an improved TPLinker model, which, through the construction of a subspace matrix mapping mechanism separating text and cross-modal relationships, significantly reduces mutual interference between heterogeneous features, accurately outputting knowledge triples and fact verification confidence scores. When new knowledge is detected to be logically mutually exclusive with historical data, a spatiotemporal dual verification mechanism is used to determine plot changes or parallel perspectives, generating evolution or isolation instructions. Finally, the topological deformation force is calculated according to the instructions and the edge weights of the entire network are iteratively updated until convergence. This completely avoids the "one mistake leads to a hundred mistakes" problem caused by traditional static coverage, and realizes the smooth self-evolution of the film and television knowledge base. It shows strong robustness and practical value when dealing with complex long-form dramas.

[0021] In this embodiment, S1 specifically includes: S11. Extract continuous frame images from the video stream using the player's decoding protocol, extract image sequences at a preset frame rate, and perform frequency domain conversion on the time domain signal of the audio stream using the short-time Fourier transform method to extract spectral features. The preset frame rate is 24 frames per second. S12. Call the automatic speech recognition interface to convert the audio stream into speech-to-text composed of text, read the system encoding time of the video stream and audio stream, calculate the time deviation value and fill in the deviation frame, generate the aligned global timestamp, parse the encapsulation file header information of the video stream, and extract the shooting scene identifier and lens position parameters as basic scene metadata. S13. Pack and bind continuous frame data, spectral features, speech-to-text transcription, global timestamps and basic scene metadata in chronological order to generate an initial film and television dataset.

[0022] In this embodiment, S2 specifically includes: S21. Perform face region localization on consecutive frames in the initial video dataset, calculate the displacement difference of facial key point coordinates in adjacent frames, mark frames with displacement difference greater than a preset mutation threshold as mutation points, use the displacement difference corresponding to the mutation point as a numerical vector, and extract facial micro-expression mutation features. The preset mutation threshold is 2.5 pixels. S22. Calculate the sum of squares of amplitude of spectral features frame by frame along the time direction, calculate the difference between the sum of squares of amplitude of adjacent frames, mark the time when the difference is greater than a preset energy threshold as an energy jump point, and extract the energy jump point of spectral features. The preset energy threshold is 3 times the average value of the sum of squares of amplitude of the previous 100 frames. S23. Divide the speech-to-text into independent sentences according to punctuation marks, traverse the preset positive sentiment dictionary and negative sentiment dictionary, count the number of positive words and negative words in each sentence, subtract the number of negative words from the number of positive words to obtain the sentiment difference, and mark the reversal position as the sentiment polarity reversal point when the sentiment difference sign of adjacent sentences reverses from positive to negative or from negative to positive, and extract the sentiment polarity reversal point of the speech-to-text. S24. The energy jump points of facial micro-expression mutation features and spectral features at the same moment are numerically added to the emotional polarity reversal points of the speech-to-text in chronological order to generate a plot tension curve that is distributed over time. S25. Use a one-dimensional sliding window to slide along the plot tension curve, calculate the average value of the values ​​in each sliding window, and determine the interval where all values ​​in the window are greater than the average value of the adjacent windows as the peak interval. Perform non-uniform segmentation on the initial film and television dataset based on the start and end time of the peak interval. S26. Extract and package the continuous frame data, spectral features, and speech-to-text corresponding to the segmented peak intervals to select a set of high-value key plot segments.

[0023] In this embodiment, S3 specifically includes: S31. Divide the continuous frame of the high-value key plot segment into image blocks of a preset size, perform weighted summation and pooling downsampling on the color channel values ​​of the pixels in each image block, and extract the visual feature vector. The preset size is 224×224. S32. Divide the spectral features into segments according to the time window, calculate the statistical average and variance of the frequency amplitude in each segment, extract auditory feature vectors, convert each word in the speech-to-text into a word vector of a preset dimension and add them in word order, extract text feature vectors, and convert the three feature vectors to the preset dimension through linear mapping matrix multiplication to complete dimension alignment. The preset dimension is 768 dimensions. S33. Read the plot tension value of the high-value key plot fragment set at the corresponding moment in the plot tension curve, substitute the plot tension value into the exponential operation formula and divide by one plus the exponential operation result, calculate the output value between 0 and 1, and use the output value as the gating coefficient. S34. Multiply each value in the dimension-aligned visual feature vector, auditory feature vector, and text feature vector by the gating coefficient, and concatenate the three multiplied vectors by concatenating their first and last positions in sequence to generate a long vector. S35. The long vector is truncated into equal parts according to a preset length. Each segment of the vector is treated as a multimodal node. Nodes belonging to different vector segments at the same time are connected in pairs to construct cross-modal interaction edges. Nodes at adjacent times within the same vector segment are connected sequentially to construct temporal edges. The nodes are combined to generate a heterogeneous topology graph containing multimodal nodes and cross-modal interaction edges. The preset length is 256. S36. For each multimodal node in the heterogeneous topology graph, find the neighboring nodes that can be reached along the connected cross-modal interaction edges and temporal edges. Add the values ​​of the corresponding positions of the vectors of all the neighboring nodes found and divide by the total number of neighboring nodes to calculate the average vector. Add the average vector to the value of the corresponding position of the vector of the current node and perform neighboring node feature aggregation. S37. Perform a linear transformation on each value in the aggregated node vector, calculate the predicted value, calculate the absolute value of the difference between the predicted value and the historical true value as the error, propagate the error in reverse along the connection of the topology graph, update the edge weight by subtracting the product of the edge weight and the preset learning rate from each edge weight, iteratively execute the neighbor node feature aggregation and edge weight update until the error is lower than the preset convergence threshold, encode cross-modal entity association and action interaction information, and output multi-dimensional fusion feature encoding, wherein the preset learning rate is 0.01 and the preset convergence threshold is 0.001.

[0024] In this embodiment, S4 specifically includes: S41. Input the multidimensional fusion feature code into the input embedding module of the improved TPLinker model. In the input embedding module, the multidimensional fusion feature code is divided into a text word vector sequence and a cross-modal feature vector sequence according to the sequence order. The total number of times each word in the text word vector sequence appears in the preset document library is obtained and divided by the total number of documents in the preset document library to obtain the word frequency value. S42. Divide the total number of documents by the number of documents containing the corresponding words to obtain the inverse document frequency value. Multiply the word frequency value by the inverse document frequency value to obtain the word frequency inverse document frequency value. Add the corresponding word frequency inverse document frequency value and the order number of the corresponding word in the sentence to the value of each position of each word vector in the text word vector sequence to obtain the modality category to which each vector belongs in the cross-modal feature vector sequence. S43. Label the visual category as value 1 and the auditory category as value 2. Add the corresponding modal category label value to the first value of each vector in the cross-modal feature vector sequence. Concatenate the processed text word vector sequence with the cross-modal feature vector sequence in the order of the first and last parts to generate the concatenated total input embedding matrix. S44. Input the total input embedding matrix into the heterogeneous Token Pair mapping module, construct a text and cross-modal separation subspace matrix mapping mechanism, copy the total input embedding matrix into two copies, traverse all rows and columns of the first total input embedding matrix, replace all values ​​corresponding to the row positions that do not belong to the text word vector sequence with 0 and replace all values ​​corresponding to the column positions that do not belong to the text word vector sequence with 0, and construct the text subspace matrix. S45. For the second total input embedding matrix, traverse all rows and columns, replace all values ​​corresponding to the row positions that do not belong to the cross-modal feature vector sequence with 0, and replace all values ​​corresponding to the column positions that do not belong to the cross-modal feature vector sequence with 0, construct the cross-modal subspace matrix, for the text subspace matrix, extract the i-th row vector and the j-th column vector, multiply the values ​​at the same index position in the two vectors, and add all products to obtain the first dot product value; S46. Fill the first dot product value into the i-th row and j-th column of the blank matrix. Iterate through all rows and columns and repeat the multiplication and addition operations to generate the text Token Pair matrix. For the cross-modal subspace matrix, extract the m-th row vector and the n-th column vector and perform the same multiplication and addition operations to generate the cross-modal Token Pair matrix. S47. Input the independent text Token Pair matrix and cross-modal Token Pair matrix into the entity handshake decoding module, read the start word index number and end word index number corresponding to the pre-annotated entity words, locate the cell in the text Token Pair matrix where the row corresponding to the start word index number and the column corresponding to the end word index number intersect, modify the value in the cell to 1, and modify the value in all other cells in the text Token Pair matrix to 0. S48. Read the text word index number corresponding to the pre-labeled entity words and the cross-modal index number of the corresponding image region features. Locate the cell in the cross-modal Token Pair matrix where the row corresponding to the text word index number intersects with the column corresponding to the cross-modal index number. Modify the value in the cell to 1. Modify the values ​​in all other cells in the cross-modal Token Pair matrix to 0. Stack the two matrices after modifying the values ​​to 1 in the top and bottom positions to generate an entity handshake feature map. Record the row number and column number of the cell with the value of 1 as the entity boundary span information. S49. Input the entity handshake feature map into the cross-modal relationship handshake module to calculate the entity boundary span information and the relationship category information connecting the first entity and the second entity, and input them into the confidence verification module to output the candidate knowledge triples and the fact verification confidence.

[0025] In this embodiment, S49 specifically includes: S491. Input the entity handshake feature map into the cross-modal relationship handshake module, read the text word index number and cross-modal index number corresponding to the first entity and the second entity extracted in the entity handshake decoding stage, locate the cell in the text token pair matrix where the row corresponding to the text word index number of the first entity intersects with the column corresponding to the text word index number of the second entity, and locate the cell in the cross-modal token pair matrix where the row corresponding to the cross-modal index number of the first entity intersects with the column corresponding to the cross-modal index number of the second entity. S492. Read the numerical code corresponding to each relation category in the preset relation category dictionary table, and fill the numerical code into the intersection cell located in the text Token Pair matrix and the cross-modal Token Pair matrix in turn. Modify the values ​​in all other cells in the two matrices to 0. Stack the text Token Pair matrix and the cross-modal Token Pair matrix after filling in the numerical code in the top and bottom positions to generate a cross-modal relation handshake feature map. S493. Traverse all cell numbers in the cross-modal relationship handshake feature map, compare and find the number with the largest value, record the row index and column index of the largest number, search for the specific relationship text corresponding to the largest number in the preset relationship category dictionary table, and extract the relationship category information connecting the first entity and the second entity. S494. Input the entity boundary span information and the relationship category information connecting the first entity and the second entity into the confidence verification module. Extract the first text vector pointed to by the text word index number and the first cross-modal vector pointed to by the cross-modal index number corresponding to the first entity from the entity boundary span information. Extract the second text vector pointed to by the text word index number and the second cross-modal vector pointed to by the cross-modal index number corresponding to the second entity. S495. Add the values ​​at the same position in the first text vector and the first cross-modal vector to generate the first fusion vector. Add the values ​​at the same position in the second text vector and the second cross-modal vector to generate the second fusion vector. Multiply the value at each position in the first fusion vector with the value at the same position in the second fusion vector and sum all the products to obtain the cross-modal vector dot product value. S496. Locate the cell containing the largest number recorded in the cross-modal relationship handshake feature map, read the original values ​​in the text token pair matrix and cross-modal token pair matrix before stacking, and add them to obtain the basic category score. Add the cross-modal vector dot product value to the basic category score to obtain the total score. Calculate the total score using the Sigmoid function and output a value between 0 and 1 as the confidence level for fact verification. S497. Extract the text word string pointed to by the text word index number corresponding to the first entity from the entity boundary span information as the head entity text, extract the text word string pointed to by the text word index number corresponding to the second entity as the tail entity text, and use the relation text in the relation category information connecting the first entity and the second entity as the relation predicate. S498. Concatenate the head entity text, relational predicate, and tail entity text in sequence using underscores to generate a string, which represents the candidate knowledge triplet representing the current plot. Connect the string of the candidate knowledge triplet with the calculated fact verification confidence value using commas, and output the candidate knowledge triplet and the fact verification confidence value.

[0026] This implementation achieves high-precision joint extraction and fact verification of cross-modal entity relationships by introducing an improved TPLinker model and subspace matrix mapping mechanism. The multi-dimensional fusion feature encoding is segmented into text and cross-modal sequences, and a total input embedding matrix is ​​generated by combining TF-IDF calculation, position indices, and modal category labels. By constructing separate subspace matrices for text and cross-modal relationships and performing row and column dot product operations in their respective independent spaces, mutual interference between heterogeneous features is effectively isolated, accurately generating text and cross-modal token pair matrices. Based on entity handshake and cross-modal relationship handshake mechanisms, feature maps are generated by locating boundary spans and filling in relationship numbers, directly anchoring complex associations. Finally, the fact verification confidence score is output by fusing the cross-modal vector dot product and category base scores through Sigmoid mapping, and knowledge triples are output by concatenating the head and tail entities and relationship predicates. This method significantly reduces entity boundary overflow and relationship mismatch problems caused by cross-modal noise, greatly improving the accuracy and reliability of knowledge extraction in complex film and television plots.

[0027] The improved TPLinker model of this invention is similar to the original TPLinker model in that both retain the core entity relationship joint extraction architecture based on Token Pair links. That is, both construct a handshake feature map in the form of a lower triangular matrix, and use the index numbers of the entity start and end words to locate the intersecting cells in the matrix to achieve entity span decoding. Both also adopt the method of stacking matrices to synchronously predict the boundary information of entities and relationships in the same feature space, avoiding the error accumulation problem caused by traditional pipeline extraction.

[0028] The difference lies in that this invention breaks through the limitation of the original TPLinker model, which directly concatenates textual and visual features into a single joint matrix for global interactive computation. It introduces a subspace physical isolation mechanism to construct a heterogeneous feature separation mapping system. Building upon the original model's direct computation of the global Token Pair matrix, this invention pre-emptively constructs the subspace matrix in steps S44 and S45. It duplicates the total input embedding matrix and performs a strict masking operation, forcibly setting the row and column values ​​that do not belong to the corresponding modality to zero, thereby constructing completely independent text subspace matrices and cross-modal subspace matrices. Subsequently, in steps S46 to S48, row and column vector dot products and handshake decoding are performed independently in these two isolated subspaces, rather than being uniformly calculated in the mixed matrix. Finally, in steps S49 to S498, entity boundaries and cross-modal relationships are independently extracted in their respective subspaces, and the fact verification confidence score is output by fusing the cross-modal vector dot product with the category base score.

[0029] Based on the above improvements, the beneficial effects of this invention are as follows: by constructing a subspace matrix mapping mechanism that separates text and cross-modal data, the improved TPLinker model can adaptively capture the contextual associations and cross-modal correspondences of homogeneous text in independent dimensions, breaking the limitations of feature interference and semantic ambiguity caused by the original model using a single joint space to process all modal features, and achieving pure decoding of Pair interactions. This design significantly enhances the model's depth of mining implicit cross-modal action interactions in complex film and television plots, and can more accurately locate the corresponding boundaries between image region features and text words. The separate parallel computing mechanism improves the accuracy of cross-modal entity recognition and relationship classification while effectively suppressing the propagation of irrelevant modal noise, enhancing the system's extraction robustness and the reliability of fact verification in multi-source heterogeneous film and television data environments.

[0030] In this embodiment, S5 specifically includes: S51. Input the candidate knowledge triples and the fact verification confidence into the film and television knowledge base, extract the head entity text, relational predicate and tail entity text in the candidate knowledge triples, and compare them character by character with the head entity text, relational predicate and tail entity text of all target historical knowledge triples stored in the film and television knowledge base. When the head entity text and relational predicate are found to be completely the same but the tail entity text is inconsistent, it is determined that logical mutual exclusion is detected. S52. When logical mutual exclusion is detected, read the global timestamp corresponding to the current candidate knowledge triplet and the global timestamp corresponding to the target historical knowledge triplet that has logical mutual exclusion. Subtract the previous timestamp value from the latter timestamp value to calculate the time difference. Determine whether the time difference is greater than a preset time threshold. If the time difference is greater than the preset time threshold, it is determined that the timeline verification has passed. If the time difference is less than or equal to the preset time threshold, it is determined that the timeline verification has failed. The preset time threshold is 300 seconds. S53. Extract the shooting scene identifier and camera position parameters from the basic scene metadata corresponding to the current candidate knowledge triplet. Extract the shooting scene identifier and camera position parameters from the basic scene metadata corresponding to the target historical knowledge triplet that has logical mutual exclusion. Compare the two sets of shooting scene identifiers with strings. Subtract the horizontal rotation angle value and vertical pitch angle value corresponding to the two sets of camera position parameters to calculate the angle difference. S54. Determine whether the angle difference is greater than the preset angle threshold. If the angle difference is greater than the preset angle threshold or the shooting scene labels are inconsistent, it is determined that the spatial line verification is passed. If the angle difference is less than or equal to the preset angle threshold and the shooting scene labels are consistent, it is determined that the spatial line verification is not passed. The preset angle threshold is 45 degrees. S55. When both timeline and spatial line checks are passed, the logical mutual exclusion is determined to be a normal change caused by the development of the plot, and a triplet evolution update instruction is generated. When either the timeline or spatial line check is not passed, the logical mutual exclusion is determined to be data noise or a branch of a parallel worldview, and a branch isolation update instruction is generated.

[0031] In this embodiment, S6 specifically includes: S61. When a triplet evolution update instruction is received, locate the target nodes corresponding to the head entity text and tail entity text of the candidate knowledge triplet in the topology network of the film and television knowledge base and establish new injection edges. When a branch isolation update instruction is received, create two isolated nodes in the topology network to store the head entity text and tail entity text respectively and establish new injection edges. Assign the fact verification confidence value corresponding to the candidate knowledge triplet to the new injection edge as the initial edge weight. S62. Calculate the sum of edge weights of all edges connected to the two ends of the newly injected edge. Divide the initial edge weight by the sum of edge weights to get the injection influence ratio, and multiply it by the initial edge weight to get the topological deformation force generated by the newly injected triplet. Propagate the topological deformation force outward along the edges of the graph network, find the first layer of associated nodes that are directly connected to the newly injected edge, and update the edge weight value by adding the topological deformation force value. S63. Continue searching along the edge to reach the next layer of associated nodes. Multiply the updated edge weight value of the current layer associated node by a preset attenuation coefficient to calculate the attenuation propagation force. Add the attenuation propagation force value to the edge weight value of the next layer associated node for updating. The preset attenuation coefficient is 0.5. S64. Subtract the edge weight of the corresponding node calculated in the previous iteration from the edge weight of each node calculated in the current iteration to calculate the weight change. Determine whether the absolute value of the weight change of all nodes is less than a preset threshold. If they are all less than the preset threshold, it is determined that the weight change of all nodes in the network has converged. If any absolute value is greater than or equal to the preset threshold, return to continue to perform the operation of finding the next layer of associated nodes and updating the edge weight. The preset threshold is 0.0001. S65. When the weight change of all nodes in the network converges to the preset threshold, save the current edge weight values ​​of all nodes in the topology network and complete the automatic update of the film and television knowledge base.

[0032] An automatic update system for a film and television knowledge base based on deep learning, comprising: The data acquisition and initial construction module is used to synchronously acquire multimodal video streaming data and extract global timestamps and basic scene metadata from the multimodal video streaming data to generate an initial video dataset. The plot tension perception and segmentation module is used to perform multimodal spatiotemporal alignment and plot tension perception calculation on the initial film and television dataset, generate plot tension curves, and perform non-uniform segmentation on the initial film and television dataset to filter out a set of high-value key plot segments. The cross-modal feature encoding module is used to extract the feature vectors of continuous screen frames, spectral features and speech-to-text in the high-value key plot segments, align them in dimensions, generate gating coefficients and construct a heterogeneous topology graph, and output multi-dimensional fusion feature encoding. The cross-modal entity relation extraction module is used to input multi-dimensional fusion feature encoding into the improved TPLinker model to obtain candidate knowledge triples and fact verification confidence. The improved TPLinker model includes an input embedding module, a heterogeneous Token Pair mapping module, an entity handshake decoding module, a cross-modal relation handshake module, and a confidence verification module. The heterogeneous Token Pair mapping module constructs a subspace matrix mapping mechanism that separates text from cross-modal data. The spatiotemporal dual verification and conflict handling module is used to compare the candidate knowledge triples and the fact verification confidence with the target historical knowledge triples. When logical mutual exclusion is detected, timeline verification and spatial line verification are performed to generate triple evolution update instructions or branch isolation update instructions. The topology deformation and dynamic update module is used to calculate the topology deformation force generated by the newly injected triples according to the triple evolution update instruction or the branch isolation update instruction, and to iteratively update the edge weights of the connected nodes, thereby performing automatic updates of the film and television knowledge base.

[0033] Example 1: To verify the feasibility of this invention in practice, it was applied to a national-level film and television big data and intelligent media asset management platform. This platform is responsible for the entry, review, cataloging, and knowledge graph construction of massive amounts of film and television dramas across the entire network, processing more than 450 hours of new film and television content daily, covering various types such as domestic long-form dramas, suspenseful short dramas, and imported overseas dramas. The construction of traditional film and television knowledge bases relies heavily on manual annotation and pure text entity extraction, which faces extremely serious knowledge conflicts and update lags when dealing with long-form dramas. Typical dilemmas include: as the plot progresses, the protagonist's identity, faction, and possessions will dynamically evolve, and traditional systems often directly overwrite old knowledge, leading to a break in spatiotemporal logic; at the same time, due to the lack of comprehensive understanding of visuals and sounds, when dialogue is ambiguous or satirical, pure text extraction is prone to generating erroneous knowledge triples and directly polluting the backbone database. Especially when dealing with complex plots involving suspense, mystery, and multiple timelines, traditional methods often result in fatal errors such as confused character relationships and conflicting ownership of key props, seriously affecting the accuracy of downstream intelligent question answering, plot recap, and personalized recommendation services.

[0034] In practical deployment, the method of this invention first acquires the video and audio streams of newly added episodes in real time through the platform's underlying decoding protocol. It then extracts spectral features using short-time Fourier transform and calls an automatic speech recognition interface to generate speech-to-text with a global timestamp, simultaneously parsing camera position parameters as basic scene metadata. The system then performs multimodal spatiotemporal alignment and plot tension perception calculations on the initial dataset, accurately identifying multimodal resonance points such as sudden changes in facial micro-expressions, energy jumps in background sounds, and reversals in the emotional polarity of dialogue, generating a plot tension curve. Based on the non-uniform segmentation mechanism of this curve, the system automatically filters out a large number of meaningless transition scenes, inputting only high-value key plot segments into subsequent models. In the core knowledge extraction stage, multidimensional fusion feature encoding is fed into an improved TPLinker model. This model constructs a subspace matrix mapping mechanism that separates text and cross-modal elements, completing entity handshakes and relational handshakes for text TokenPairs and cross-modal TokenPairs in an independent space isolated from interference, ultimately outputting candidate knowledge triples with factual verification confidence. When a newly extracted triplet is logically mutually exclusive with the historical database, the system does not blindly overwrite it. Instead, it initiates a dual spatiotemporal verification: comparing the global timestamp to determine if it's a normal plot development change, and comparing the difference between the shooting scene identifier and the camera angle to determine if it's a parallel perspective or scene switch. Based on the verification results, after generating evolution or isolation instructions, the system uses the confidence level of the new triplet as the initial edge weights, calculates the topological deformation force, and propagates it layer by layer in the graph network until the total network weights reach dynamic equilibrium, thus achieving smooth self-evolutionary updates to the knowledge base. During a three-month online testing period, the method of this invention was compared in parallel with the platform's existing pure text knowledge extraction and static overwrite update system based on traditional BERT-BiLSTM-CRF in the same environment. Four highly representative complex plot knowledge processing scenarios were selected for performance verification. Specific comparison data are shown in the table below: Table 1. Performance Comparison Data Between the Invention and Traditional Methods

[0035] Based on the comparative data shown in Table 1, it can be seen that the deep learning-based automatic update method for film and television knowledge base proposed in this invention has an overwhelming performance advantage over traditional plain text methods in terms of complex plot understanding and dynamic knowledge evolution. It has achieved a qualitative leap in core indicators such as knowledge evolution accuracy, extraction response speed, and logical conflict control.

[0036] In terms of knowledge evolution accuracy, this invention consistently maintains an extremely high level of over 94% in all four types of high-difficulty scenarios, while traditional methods, lacking cross-modal verification mechanisms, generally hover around 65% accuracy. Taking the highly representative scenario of "dynamic reversal of character factions" as an example, when dealing with plots such as undercover agents being exposed or turning to the dark side in suspense dramas, traditional systems rely solely on textual dialogue for judgment. Once they encounter deliberately misleading dialogue, they generate incorrect triples and directly overwrite the original positive identity labels, causing all subsequent graph queries related to the plot to become invalid, resulting in an accuracy of only 66.9%. This invention introduces facial micro-expression mutation features and audio energy jump points as cross-validation to accurately identify dialogue disguises. At the same time, when it detects the logical mutual exclusion of "the same person being both a policeman and a gangster", the system automatically retrieves the global timestamps and basic scene metadata of the preceding and following segments. It finds that the difference in the horizontal rotation angle of the camera position between the two exceeds the preset 45-degree threshold and the shooting scene identification is inconsistent. Therefore, it is determined to be a parallel narrative perspective rather than a data error. It decisively generates a branch isolation update instruction, establishes isolated nodes in the graph to save the gangster identity clues, avoids the pollution of the backbone network, and greatly improves the accuracy to 95.9%.

[0037] In terms of extraction and update response timeliness, the heterogeneous topology graph aggregation mechanism and the improved TPLinker matrix mapping mechanism of this invention demonstrate extremely high computational efficiency. Traditional methods exhibit exponentially increasing computational complexity when processing long sequences, with an average processing time exceeding 130 milliseconds. In contrast, this invention filters out a large number of low-value segments in advance through the plot tension curve, significantly reducing the amount of input data. At the same time, the subspace matrix mapping breaks down the massive joint computation into parallel operations, compressing the average extraction time for the four types of scenarios to less than 40 milliseconds, achieving a response speedup of nearly three times. This fully meets the platform's stringent timeliness requirements for real-time cataloging and cataloging of massive amounts of newly added dramas.

[0038] The advantages of this invention are more pronounced in controlling the temporal logic conflict rate and the cross-modal relationship missed sampling rate. Traditional methods, due to their static overlay updates, are prone to chain reactions once a misjudgment occurs, resulting in a persistently high temporal logic conflict rate, reaching as high as 22.8% in "multi-view parallel spatiotemporal interweaving" scenarios. Simultaneously, traditional methods completely fail to capture the cross-modal relationship between on-screen actions and dialogue; for example, actions such as a character handing a key prop to someone without explicit dialogue can lead to a cross-modal missed sampling rate generally exceeding 20%. This invention, relying on spatiotemporal dual verification and a smooth convergence mechanism based on topological deformation forces, comprehensively suppresses the temporal logic conflict rate to an extremely low level below 3.5%, and precisely anchors the correspondence between image region features and text words through a cross-modal Token Pair matrix, controlling the cross-modal missed sampling rate to around 3%. Overall, this invention, by deeply integrating multimodal perception and graph network dynamic evolution mechanisms, completely overcomes the long-standing industry pain point of "static rigidity and numerous errors" in film and television knowledge bases, providing a solid and reliable underlying knowledge engine for the next generation of intelligent media asset platforms.

[0039] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based automatic updating method for a movie and television knowledge base, characterized in that, Includes the following steps: S1. Synchronously collect multimodal streaming data of film and television, extract global timestamps and basic scene metadata, and generate an initial film and television dataset; S2. Perform multimodal spatiotemporal alignment and plot tension perception calculation on the initial film and television dataset to generate a plot tension curve, and perform non-uniform segmentation on the initial film and television dataset to select a set of high-value key plot segments. S3. Extract the feature vectors of continuous frames, spectral features and speech-to-text from high-value key plot segments and align them with dimensions. Generate gating coefficients and construct a heterogeneous topology graph. Output multi-dimensional fusion feature encoding. S4. Input the multidimensional fusion feature encoding into the improved TPLinker model to obtain candidate knowledge triples and fact verification confidence. The improved TPLinker model includes an input embedding module, a heterogeneous Token Pair mapping module, an entity handshake decoding module, a cross-modal relationship handshake module, and a confidence verification module. The heterogeneous Token Pair mapping module constructs a subspace matrix mapping mechanism that separates text from cross-modal relationships. S5. Compare the candidate knowledge triples and the fact verification confidence with the target historical knowledge triples. When logical mutual exclusion is detected, perform timeline verification and spatial line verification, and generate triple evolution update instructions or branch isolation update instructions. S6. Calculate the topological deformation force generated by the newly injected triplet according to the triplet evolution update instruction or the branch isolation update instruction, and iteratively update the edge weights of the connected nodes to perform automatic updates of the film and television knowledge base. 2.The method of claim 1, wherein, S1 specifically includes: S11. Extract continuous frame images from the video stream using the player's decoding protocol, extract image sequences at a preset frame rate, and use the short-time Fourier transform method to perform frequency domain conversion on the time domain signal of the audio stream to extract spectral features. S12. Call the automatic speech recognition interface to convert the audio stream into speech-to-text composed of text, read the system encoding time of the video stream and audio stream, calculate the time deviation value and fill in the deviation frame, generate the aligned global timestamp, parse the encapsulation file header information of the video stream, and extract the shooting scene identifier and lens position parameters as basic scene metadata. S13. Pack and bind continuous frame data, spectral features, speech-to-text, global timestamps and basic scene metadata in chronological order to generate an initial film and television dataset. 3.The method of claim 1, wherein, S2 specifically includes: S21. Perform face region localization on consecutive frames in the initial video dataset, calculate the displacement difference of facial key point coordinates in adjacent frames, mark frames with displacement difference greater than the preset mutation threshold as mutation points, and use the displacement difference corresponding to the mutation point as a numerical vector to extract facial micro-expression mutation features. S22. Calculate the sum of squares of amplitude of spectral features frame by frame along the time direction, calculate the difference between the sum of squares of amplitude of adjacent frames, mark the time when the difference is greater than the preset energy threshold as the energy jump point, and extract the energy jump point of spectral features. S23. Divide the speech-to-text into independent sentences according to punctuation marks, traverse the preset positive sentiment dictionary and negative sentiment dictionary, count the number of positive words and negative words in each sentence, subtract the number of negative words from the number of positive words to obtain the sentiment difference, and mark the reversal position as the sentiment polarity reversal point when the sentiment difference sign of adjacent sentences reverses from positive to negative or from negative to positive, and extract the sentiment polarity reversal point of the speech-to-text. S24. Add the energy jump points of facial micro-expression mutation features and spectral features at the same moment in chronological order with the emotional polarity reversal points of the speech-to-text to generate a plot tension curve. S25. Use a one-dimensional sliding window to slide along the plot tension curve, calculate the average value of the values ​​in each sliding window, and determine the interval where all values ​​in the window are greater than the average value of the adjacent windows as the peak interval. Perform non-uniform segmentation on the initial film and television dataset based on the start and end time of the peak interval. S26. Extract and package the continuous frame data, spectral features, and speech-to-text corresponding to the segmented peak intervals to select a set of high-value key plot segments.

4. The method for automatically updating a film and television knowledge base based on deep learning according to claim 1, characterized in that, S3 specifically includes: S31. Divide the continuous frames of high-value key plot segments into image blocks of a preset size, perform weighted summation and pooling downsampling on the color channel values ​​of pixels in each image block, and extract visual feature vectors. S32. Divide the spectral features into segments according to the time window, calculate the statistical average and variance of the frequency amplitude in each segment, extract the auditory feature vector, convert each word in the speech-to-text into a word vector of a preset dimension and add them in word order, extract the text feature vector, and convert the three feature vectors to the preset dimension through linear mapping matrix multiplication to complete the dimension alignment. S33. Read the plot tension value of the high-value key plot fragment set at the corresponding moment in the plot tension curve, substitute the plot tension value into the exponential operation formula and divide by one plus the exponential operation result, calculate the output value between 0 and 1, and use the output value as the gating coefficient. S34. Multiply each value in the dimension-aligned visual feature vector, auditory feature vector, and text feature vector by the gating coefficient, and concatenate the three multiplied vectors by concatenating their first and last positions in sequence to generate a long vector. S35. Cut the long vector into equal parts according to the preset length, and treat each segment of the vector as a multimodal node. Connect the nodes belonging to different vector segments at the same time to construct cross-modal interaction edges. Connect the nodes of adjacent time points within the same vector segment to construct temporal edges. Combine them to generate a heterogeneous topology graph. S36. For each multimodal node in the heterogeneous topology graph, find the neighboring nodes that can be reached along the connected cross-modal interaction edges and temporal edges. Add the values ​​of the corresponding positions of the vectors of all the neighboring nodes found and divide by the total number of neighboring nodes to calculate the average vector. Add the average vector to the value of the corresponding position of the vector of the current node and perform neighboring node feature aggregation. S37. Perform a linear transformation on each value in the aggregated node vector, calculate the predicted value, calculate the absolute value of the difference between the predicted value and the historical true value as the error, propagate the error in reverse along the connection of the topology graph, update the edge weight by subtracting the product of the edge weight and the preset learning rate from each edge weight, iteratively execute the neighbor node feature aggregation and edge weight update until the error is lower than the preset convergence threshold, encode cross-modal entity association and action interaction information, and output multi-dimensional fusion feature encoding.

5. The method for automatically updating a film and television knowledge base based on deep learning according to claim 1, characterized in that, S4 specifically includes: S41. Input the multi-dimensional fusion feature encoding into the input embedding module of the improved TPLinker model, and divide it into a text word vector sequence and a cross-modal feature vector sequence according to the sequence order. Obtain the total number of times each word in the text word vector sequence appears in the preset document library and divide it by the total number of documents in the preset document library to obtain the word frequency value. S42. Divide the total number of documents by the number of documents to obtain the inverse document frequency value, and multiply it with the word frequency value to obtain the word frequency inverse document frequency value. Add the corresponding word frequency inverse document frequency value and the word order number in the sentence to the value of each position of each word vector in the text word vector sequence to obtain the modality category to which each vector belongs in the cross-modal feature vector sequence. S43. Label the visual category as value 1 and the auditory category as value 2. Add the corresponding modal category label value to the first value of each vector in the cross-modal feature vector sequence, and concatenate it with the cross-modal feature vector sequence in the order of first and last to generate the total input embedding matrix. S44. Input the total input embedding matrix into the heterogeneous Token Pair mapping module to construct a subspace matrix mapping mechanism that separates text and cross-modal features. Copy the total input embedding matrix into two copies and replace all values ​​at the row and column positions that do not belong to the text word vector sequence and the cross-modal feature vector sequence with 0 respectively to construct the text subspace matrix and the cross-modal subspace matrix. S45. For the text subspace matrix, extract the vector in the i-th row and the vector in the j-th column, multiply the values ​​at the same index position in the two vectors and add them to obtain the first dot product value. S46. Fill the first dot product value into the i-th row and j-th column of the blank matrix. Iterate through all rows and columns and repeat the multiplication and addition operations to generate the text Token Pair matrix. For the cross-modal subspace matrix, extract the m-th row vector and the n-th column vector and perform the same multiplication and addition operations to generate the cross-modal Token Pair matrix. S47. Input the text Token Pair matrix and the cross-modal Token Pair matrix into the entity handshake decoding module, read the start word index number and end word index number corresponding to the pre-annotated entity words, locate the cell in the text Token Pair matrix where the row corresponding to the start word index number and the column corresponding to the end word index number intersect, modify the value in the cell to 1, and modify the value in all other cells in the text Token Pair matrix to 0. S48. Read the text word index number corresponding to the pre-labeled entity words and the cross-modal index number of the corresponding image region features. Perform the same localization modification operation as S47 in the cross-modal Token Pair matrix. Stack the two matrices after numerical modification in the top and bottom positions to generate an entity handshake feature map. S49. Input the entity handshake feature map into the cross-modal relationship handshake module for calculation and input it into the confidence verification module to output the candidate knowledge triples and the fact verification confidence.

6. The method for automatically updating a film and television knowledge base based on deep learning according to claim 5, characterized in that, Specifically, S49 includes: S491. Input the entity handshake feature map into the cross-modal relationship handshake module, read the text word index number and cross-modal index number corresponding to the first entity and the second entity extracted in the entity handshake decoding stage, locate the cell in the text token pair matrix where the row corresponding to the text word index number of the first entity intersects with the column corresponding to the text word index number of the second entity, and locate the cell in the cross-modal token pair matrix where the row corresponding to the cross-modal index number of the first entity intersects with the column corresponding to the cross-modal index number of the second entity. S492. Read the numerical code corresponding to each relation category in the preset relation category dictionary table, and fill the numerical code into the intersection cell located in the text Token Pair matrix and the cross-modal Token Pair matrix in turn. Modify the values ​​in all other cells in the two matrices to 0. Stack the text Token Pair matrix and the cross-modal Token Pair matrix after filling in the numerical code in the top and bottom positions to generate a cross-modal relation handshake feature map. S493. Traverse all cell numbers in the cross-modal relationship handshake feature map, compare and find the number with the largest value, record the row index and column index of the largest number, search for the specific relationship text corresponding to the largest number in the preset relationship category dictionary table, and extract the relationship category information connecting the first entity and the second entity. S494. Input the entity boundary span information and the relationship category information connecting the first entity and the second entity into the confidence verification module. Extract the first text vector pointed to by the text word index number and the first cross-modal vector pointed to by the cross-modal index number corresponding to the first entity from the entity boundary span information. Extract the second text vector pointed to by the text word index number and the second cross-modal vector pointed to by the cross-modal index number corresponding to the second entity. S495. Add the values ​​at the same position in the first text vector and the first cross-modal vector to generate the first fusion vector. Add the values ​​at the same position in the second text vector and the second cross-modal vector to generate the second fusion vector. Multiply the value at each position in the first fusion vector with the value at the same position in the second fusion vector and sum all the products to obtain the cross-modal vector dot product value. S496. Locate the cell containing the largest numerical number recorded in the cross-modal relationship handshake feature map, read the original values ​​in the text token pair matrix and cross-modal token pair matrix before stacking, and add them to obtain the basic category score. Add the cross-modal vector dot product value to the basic category score to obtain the total score. Calculate the total score using the Sigmoid function and output the fact verification confidence score. S497. Extract the text word string pointed to by the text word index number corresponding to the first entity from the entity boundary span information as the head entity text, extract the text word string pointed to by the text word index number corresponding to the second entity as the tail entity text, and use the relation text in the relation category information connecting the first entity and the second entity as the relation predicate. S498. Concatenate the head entity text, relational predicate, and tail entity text in sequence using underscores to generate a string, which represents the candidate knowledge triplet representing the current plot. Connect the string of the candidate knowledge triplet with the calculated fact verification confidence value using commas, and output the candidate knowledge triplet and the fact verification confidence value.

7. The method for automatically updating a film and television knowledge base based on deep learning according to claim 1, characterized in that, S5 specifically includes: S51. Input the candidate knowledge triples and the fact verification confidence into the film and television knowledge base, extract the head entity text, relational predicate and tail entity text in the candidate knowledge triples, and compare them character by character with the head entity text, relational predicate and tail entity text of all target historical knowledge triples stored in the film and television knowledge base. When the head entity text and relational predicate are found to be completely the same but the tail entity text is inconsistent, it is determined that logical mutual exclusion is detected. S52. When logical mutual exclusion is detected, read the global timestamp corresponding to the current candidate knowledge triplet and the global timestamp corresponding to the target historical knowledge triplet that has logical mutual exclusion. Subtract the previous timestamp value from the latter timestamp value to calculate the time difference. Determine whether the time difference is greater than the preset time threshold. If the time difference is greater than the preset time threshold, it is determined that the timeline verification has passed. If the time difference is less than or equal to the preset time threshold, it is determined that the timeline verification has not passed. S53. Extract the shooting scene identifier and camera position parameters from the basic scene metadata corresponding to the current candidate knowledge triplet. Extract the shooting scene identifier and camera position parameters from the basic scene metadata corresponding to the target historical knowledge triplet that has logical mutual exclusion. Compare the two sets of shooting scene identifiers with strings. Subtract the horizontal rotation angle value and vertical pitch angle value corresponding to the two sets of camera position parameters to calculate the angle difference. S54. Determine whether the angle difference is greater than the preset angle threshold. If the angle difference is greater than the preset angle threshold or the shooting scene labels are inconsistent, it is determined that the spatial line verification is passed. If the angle difference is less than or equal to the preset angle threshold and the shooting scene labels are consistent, it is determined that the spatial line verification is not passed. S55. When both timeline and spatial line checks are passed, the logical mutual exclusion is determined to be a normal change caused by the development of the plot, and a triplet evolution update instruction is generated. When either the timeline or spatial line check is not passed, the logical mutual exclusion is determined to be data noise or a branch of a parallel worldview, and a branch isolation update instruction is generated.

8. The method for automatically updating a film and television knowledge base based on deep learning according to claim 1, characterized in that, S6 specifically includes: S61. When a triplet evolution update instruction is received, locate the target nodes corresponding to the head entity text and tail entity text of the candidate knowledge triplet in the topology network of the film and television knowledge base and establish new injection edges. When a branch isolation update instruction is received, create two isolated nodes in the topology network to store the head entity text and tail entity text respectively and establish new injection edges. Assign the fact verification confidence value corresponding to the candidate knowledge triplet to the new injection edge as the initial edge weight. S62. Calculate the sum of edge weights of all edges connected to the two ends of the newly injected edge. Divide the initial edge weight by the sum of edge weights to get the injection influence ratio, and multiply it by the initial edge weight to get the topological deformation force generated by the newly injected triplet. Propagate the topological deformation force outward along the edges of the graph network, find the first layer of associated nodes that are directly connected to the newly injected edge, and update the edge weight value by adding the topological deformation force value. S63. Continue searching along the edge to reach the next layer of associated nodes. Multiply the updated edge weight value of the current layer associated node by the preset attenuation coefficient to calculate the attenuation propagation force. Add the attenuation propagation force value to the edge weight value of the next layer associated node for updating. S64. Subtract the edge weight of the corresponding node calculated in the previous iteration from the edge weight of each node calculated in the current iteration to calculate the weight change. Determine whether the absolute value of the weight change of all nodes is less than the preset threshold. If they are all less than the preset threshold, it is determined that the weight change of all nodes in the network has converged. If any absolute value is greater than or equal to the preset threshold, return to continue to perform the operation of finding the next layer of related nodes and updating the edge weight. S65. When the weight change of all nodes in the network converges to the preset threshold, save the current edge weight values ​​of all nodes in the topology network and complete the automatic update of the film and television knowledge base.

9. A deep learning-based automatic update system for a film and television knowledge base, comprising executing the deep learning-based automatic update method for a film and television knowledge base as described in any one of claims 1 to 8, characterized in that, include: The data acquisition and initial construction module is used to synchronously acquire multimodal video streaming data and extract global timestamps and basic scene metadata from the multimodal video streaming data to generate an initial video dataset. The plot tension perception and segmentation module is used to perform multimodal spatiotemporal alignment and plot tension perception calculation on the initial film and television dataset, generate plot tension curves, and perform non-uniform segmentation on the initial film and television dataset to filter out a set of high-value key plot segments. The cross-modal feature encoding module is used to extract the feature vectors of continuous screen frames, spectral features and speech-to-text in the high-value key plot segments, align them in dimensions, generate gating coefficients and construct a heterogeneous topology graph, and output multi-dimensional fusion feature encoding. The cross-modal entity relation extraction module is used to input multi-dimensional fusion feature encoding into the improved TPLinker model to obtain candidate knowledge triples and fact verification confidence. The improved TPLinker model includes an input embedding module, a heterogeneous Token Pair mapping module, an entity handshake decoding module, a cross-modal relation handshake module, and a confidence verification module. The heterogeneous Token Pair mapping module constructs a subspace matrix mapping mechanism that separates text from cross-modal data. The spatiotemporal dual verification and conflict handling module is used to compare the candidate knowledge triples and the fact verification confidence with the target historical knowledge triples. When logical mutual exclusion is detected, timeline verification and spatial line verification are performed to generate triple evolution update instructions or branch isolation update instructions. The topology deformation and dynamic update module is used to calculate the topology deformation force generated by the newly injected triples according to the triple evolution update instruction or the branch isolation update instruction, and to iteratively update the edge weights of the connected nodes, thereby performing automatic updates of the film and television knowledge base.

Images