135 results about "Subtitle" patented technology

The invention discloses a virtual digital human multimedia teaching interaction method and system and a storage medium, and the method comprises the steps: calculating an audio mouth shape time difference, a mouth shape subtitle time difference, an audio subtitle time difference and a video mouth shape time difference based on a same-window data group on the basis of a unified time aperture and a window, comparing the time differences with a time difference threshold value, and outputting a dislocation alarm result, unified constraint on four types of key alignment relationships is realized; when an alarm is triggered, selecting a degradation strategy according to a dislocation alarm result and generating an execution record, and summarizing the dislocation alarm result and the execution record to form an evidence chain data packet; when the cumulative number of times in the preset window number exceeds a diffusion blocking threshold value, recording output is switched into a video stream which retains audio and subtitles and does not overlap mouth shapes; and generating a correction release stream based on the evidence chain data packet, and replacing the recorded broadcast corresponding time slice and retaining the Hash verification value when the time sequence consistency index meets the release threshold, thereby realizing drivable treatment action, suppressible diffusion and replaceable correction.

The invention discloses a simultaneous interpretation method and system based on a large model and electronic equipment, and the method comprises the steps: extracting bilingual parallel corpora related to terms from professional resources based on a standardized professional dictionary, obtaining qualified corpora through data enhancement processing and manual screening, and constructing a multi-level corpus according to the levels of words, sentences and paragraphs; the method comprises the following steps: receiving an input audio stream in real time, extracting acoustic features through preprocessing, inputting a pre-established large-scale speech recognition model, and carrying out incremental decoding on the acoustic features in a sliding window mode; and calling a sentence boundary prediction network to judge a pause point, and outputting a text stream with a timestamp. Performing fine tuning on the large-scale speech recognition model by using a multi-level corpus, translating a text stream based on the fine-tuned large-scale speech recognition model, and constraining term translation according to a standardized professional dictionary; and synchronously displaying the audio output in the translation result and the subtitles. According to the scheme, the terminology recognition and translation accuracy is improved, and simultaneous interpretation delay is reduced.

The invention discloses a video subtitle erasing method, device and equipment and a storage medium, and relates to the field of digital image processing, and the method comprises the steps: detecting subtitles of a subtitle video to be erased, merging timestamps of the same subtitles in the subtitle video to be erased, and determining a subtitle fragment set and a subtitle-free fragment set; splitting the subtitle segment into independent shot segments by using a preset lens splitting algorithm, and analyzing video frames of the independent shot segments to obtain a first frame and a tail frame; determining a target reference frame based on the first frame and the tail frame, generating an expanded independent shot segment according to the target reference frame and the independent shot segment, and segmenting a target character mask; and generating an erased independent lens segment according to the expanded independent lens segment and the target character mask through a preset erasure algorithm, performing a preset post-processing optimization operation on the erased independent lens segment to obtain a target independent lens segment, and integrating the target independent lens segment and the subtitle-free segment set to generate a target video. According to the method and the device, the video subtitles can be accurately erased.

The present invention provides a foreign language teaching video generation method and apparatus. The foreign language teaching video generation method comprises: acquiring video subtitle text corresponding to a video to be processed; on the basis of the video subtitle text and a preset explanation generation rule, using a large language model to generate foreign language teaching explanation text of the video subtitle text; generating a corresponding explanation audio on the basis of the foreign language teaching explanation text; determining a time information relationship between the foreign language teaching explanation text and the explanation audio; on the basis of the time information relationship between the foreign language teaching explanation text and the explanation audio, generating a display style corresponding to the explanation audio; and on the basis of the explanation audio, the display style corresponding to the explanation audio, and the time information relationship between the explanation text and the explanation audio, generating an explanation video corresponding to the video to be processed. In the present invention, a target teaching video corresponding to a video to be processed can be automatically generated, and the generated teaching video is highly targeted, so that the personalized requirements of users can be met.

A system, a method and a process are for AI-powered generation and delivery of video clips. The processor is configured to: classify a first video content item including a first video file comprising video frames associated with timestamps and a first subtitle file comprising subtitle text associated with the timestamps, wherein the first video content item is classified with narrative classifiers by: executing a natural language processing (NLP) model with the first subtitle file as input, the NLP model including a dialogue analysis for identifying first narrative elements from dialogue included in the first subtitle file and associating the first narrative elements with first timestamps, executing an image recognition model with the first video file as input, the image recognition model including an object identification analysis for identifying second narrative elements from objects or persons portrayed in the video frames of the first video file and associating the second narrative elements with second timestamps, combining a first output of the NLP model with a second output of the image recognition model, and generating a first set of timestamps associated with the narrative classifiers; define one or more segments within the first video content item, each segment comprising a starting timestamp and an ending timestamp defining a duration and having one or more of the narrative classifiers associated therewith; and generate a video clip including one or more of the segments based on prioritization rules in which some narrative classifiers are associated with a priority for inclusion in the video clip, the one or more segments selected for inclusion in the video clip so that a combined duration of the one or more segments is less than a set time value, the set time value being less than a full duration of the first video content item.

The embodiment of the invention provides a video subtitle recognition method and device, equipment, a medium and a program product. The video subtitle recognition method comprises the steps that text content and text observation features of all video frames in a target video are acquired; subtitle recognition prompt information is constructed based on the text content and the text observation features, and the subtitle recognition prompt information is used for indicating a subtitle recognition model to determine subtitles in the text content according to the text observation features; based on the subtitle recognition prompt information, the subtitle recognition model is called to generate a subtitle recognition result, and the subtitle recognition result comprises a subtitle spatial-temporal feature and a subtitle text. The subtitle recognition prompt information is constructed based on the text content and the text observation characteristics of each video frame, so that the subtitle recognition model can perform context correlation analysis on the text content in combination with the text observation characteristics, and the text segments with correlation are recognized as subtitles in the continuous frames, so that the subtitle recognition accuracy is improved.

1. Name of the product in this design: Audio and video chat graphical user interface for electronic devices. 2. Purpose of this design: An electronic device. 3. The key design feature of this product is its graphical user interface. 4. The image or photo that best illustrates the design points: Design 1 Interface Change State Diagram 1. 5. Design 1 is designated as the basic design. 6. Purpose of the graphical user interface: for audio and video chat. 7. Description of the changing states of the graphical user interface: Clicking "Voice Call" in the main view of Design 1 will result in the interface changing state diagram 1 of Design 1. In the interface changing state diagram 1 of Design 1, the real-time collected voice information will be converted into subtitles for display, resulting in the interface changing state diagram 2 of Design 1. Clicking "Video Call" in the main view of Design 2 will bring up the Design 2 interface change state diagram 1. In the Design 2 interface change state diagram 1, the real-time collected voice information will be converted into subtitles for display, resulting in the Design 2 interface change state diagram 2. 8. Other situations requiring explanation: In each view, "XX" represents text or characters, and each view uses color blocks to represent variable content screens.

The invention provides an English movie and television reading difficulty grading method and system based on natural language processing, and relates to the technical field of English movie and television grading, and the method comprises the steps: firstly collecting multi-mode subtitle data of a target English movie and television resource, the data comprising language layer information and physical layer information; on the basis of language layer information, complexity features of a source language and comparison features of Chinese and English translation are extracted in parallel through a natural language processing technology, and meanwhile playing feature information of physical layer information is converted to obtain adaptive feature vectors; and then fusion feature representation corresponding to each fragment of the target English film and television resource is generated through a multi-modal cooperation mechanism, and after sequential dynamic modeling is carried out by a recurrent neural network, a modeling result is subjected to grading processing to obtain a difficulty grading result. The English film and television reading difficulty can be accurately and efficiently graded.

This invention discloses a video subtitle erasure method. It involves acquiring a fidelity stream and a computational stream of the video to be processed, where the computational stream consists of multiple computational frames. Each computational frame includes a target detection region. For each computational frame, the method detects whether a target subtitle to be erased exists within each target detection region. If so, it acquires an initial repair mask for the target subtitle. It then performs structural texture restoration on the initial repair masks to acquire corresponding target repair masks. Based on the fidelity stream, the initial repair masks, and the corresponding target repair masks, it obtains the subtitle-erased video. This method effectively improves the computational efficiency of acquiring the subtitle-erased video and enhances its quality.

This disclosure relates to a subtitle processing method and apparatus. The method includes: during the editing of a multimedia material segment, obtaining subtitle text corresponding to the audio and timestamp information of audio segments corresponding to each text element in the subtitle text through speech recognition; determining the material segment matching the text element in the multimedia material segment based on the timestamp information of the audio segment corresponding to each text element; and then synthesizing each text element with the matching material segment within the specified time to obtain a target multimedia material with a subtitle text appearing word by word in an animation effect. The solution of this disclosure can achieve a subtitle animation effect where the corresponding text subtitle appears when a certain word is spoken; furthermore, user input commands can automatically generate dynamic subtitles, simplifying user operation and improving user experience.

The application provides a display device and a subtitle language recognition method. The method can acquire a subtitle corresponding to a media video in response to a play instruction of the media video, extract visual texture features of the subtitle in a case where the subtitle is an image subtitle, perform language recognition according to the visual texture features to obtain a language recognition result, extract syntax topology features of the subtitle in a case where the subtitle is a text subtitle, perform language recognition according to the syntax topology features to obtain a language recognition result, write the language recognition result into metadata corresponding to the media video, control a display to play the media video and the subtitle, and display a language identifier corresponding to the subtitle according to the language recognition result in the metadata corresponding to the media video. The method recognizes a language by analyzing features of different languages in physical forms and syntax structures, so that the language identifier of the subtitle can be correctly displayed when the media video is played.

The invention relates to the technical field of computer vision and natural language processing, in particular to a barrier-free drama-oriented real-time sign language-subtitle-voice three-dimensional synchronous generation system. Comprising a multi-mode drama content collection module, a drama semantic and emotion deep analysis module, a three-dimensional emotional content generation module, a millisecond-level synchronous calibration module, a personalized demand adaptation module, a multi-terminal output module and a feedback iteration module which are linked in sequence. The multi-mode drama content acquisition module supports offline theaters and online live broadcast / recorded broadcast scenes and can acquire line audios, actor performance data and scene auxiliary data, the deep binding of role personalization, plot emotion, scene atmosphere and barrier-free content is realized for the first time, the pain points of'action stiffness and emotion missing 'of a general system are solved, and the system has the advantages of being high in practicability and high in practicability. The sign language / voice / subtitle drama adaptation degree is improved to 95% or above, the overall time delay is controlled within 100 ms through a synchronous calibration algorithm and is far lower than the standard of 200 ms in the industry, and the drama continuity is guaranteed.

The present disclosure relates to a cloud-based system designed to automate the packaging, validation, processing, and delivery of media content and its supporting items. The system processes video, audio, subtitles, artwork, and metadata according to predefined specifications, ensuring compliance with technical and qualitative requirements for various endpoints. The system performs validation, error correction, transcoding, file conversion, and packaging based on saved profiles or templates. By leveraging cloud-based workflows and optional human oversight, the system ensures efficient and accurate delivery of media content to any destination.

The invention provides a video translation method and device, and the method comprises the steps: determining a voice audio segment corresponding to each target subtitle text in a subtitle set of a target language through the subtitle set of the target language and an audio file of a to-be-translated video; then determining an emotion category of a voice audio segment corresponding to the target subtitle text and a long audio of a role to which the target subtitle text belongs; then, based on the target subtitle text, the duration of the target subtitle text, the emotion category of the voice audio fragment corresponding to the target subtitle text and the long audio of the role to which the target subtitle text belongs, generating a target voice audio fragment corresponding to the target subtitle text; and finally, generating a translated video based on the target human voice audio segments corresponding to all the target subtitle texts, thereby remarkably improving the quality and availability of automatic dubbing, and effectively improving the watching experience.

To help a computer game player in understanding a computer game, upon pausing the game, visual subtitles may be presented. In addition, or alternatively, Braille representing subtitles may be output as a series of vibrations on a touch pad of the controller. When the person's finger reaches the edge of the touch pad, a new series of Braille subtitles may be presented. Depending on where the player is in reading the subtitles and how fast the player reads them, the game video may be slowed down from normal speed.

The invention provides a video large-area static subtitle erasing method, device and equipment, and the method comprises the steps: carrying out the text region detection of each video frame in a to-be-processed video, and obtaining the text region of each video frame; screening each text region according to a preset error detection judgment rule, and rejecting the text region subjected to error detection to obtain a reserved text region; performing morphological contraction on each reserved text area to obtain a character area of each reserved text area; performing image completion on each character region according to an optical flow algorithm and other video frames to obtain a repair region without subtitles; and after each repair area is pasted back to the corresponding video frame, a processed video without subtitles is obtained. According to the method, the problems of high false detection rate and high erasing difficulty of the large-area static subtitles are effectively solved, and the erasing efficiency of the large-area static subtitles of the video is remarkably improved.

This invention discloses a method for generating subtitles for educational videos based on a large model, comprising the following steps: inputting original video data, segmenting frames and extracting visual features; performing multimodal feature fusion and decoding through a cross-attention mechanism and a Swing Transformer; and jointly optimizing model parameters during training using the aforementioned weighted combined loss function to generate video subtitles. This method for generating subtitles for educational videos, by fusing multimodal features, enables the model to more comprehensively understand the video content, significantly improving the matching degree between the generated subtitles and the original video's audio and scene, achieving high-accuracy subtitle output. Furthermore, by introducing language model perplexity as a readability loss, it significantly improves the fluency and naturalness of the generated subtitles, making the subtitle content easier to understand and read.

The embodiment of the invention provides a style image generation method and device, a storage medium and a computer program product. The method comprises the following steps: acquiring a video clip, wherein the video clip comprises a set of image frames and a plurality of subtitles; extracting a plurality of key frames from the set of image frames, wherein each key frame has an associated subtitle; generating a style image corresponding to each key frame in the plurality of key frames by using a trained style image generation model; and correspondingly adding the subtitles associated with each key frame to a predetermined position in the corresponding style image to generate a target style image. According to the method disclosed by the embodiment of the invention, an artificial intelligence (AI) technology can be utilized to automatically generate a set of style images for video clips such as a short play, so that the generation time and the manufacturing period of the set of style images can be greatly shortened, and the watching experience of more diversified contents is provided for a user.

Systems and methods are provided for applying attributes to subtitles. One example method includes accessing a subtitle file, wherein the subtitle file comprises one or more subtitles, and identifying an attribute to apply to at least a subset of the subtitles. The subtitle file is amended indicate an attribute to apply to at least a subset of the subtitles to create an amended subtitle file. At a computing device, the subtitles of the amended subtitle file are generated for display, wherein the attribute is applied to the subset of the subtitles.

The invention provides a subtitle processing method and system, electronic equipment and a storage medium, and is applied to the technical field of data processing. A plurality of sentences and original subtitle sentence durations thereof are generated according to a subtitle file; wherein the sentence is composed of at least one subtitle, and the original subtitle sentence duration is the sum of the original subtitle duration of all subtitles forming the sentence; translating the sentences to obtain translations of the sentences; based on the sentences and the duration of the original subtitle sentences, performing minimum deviation segmentation on translations of the sentences to obtain segmentation schemes of the sentences; wherein the segmentation scheme comprises a translation fragment of each subtitle corresponding to the sentence; according to the method and the device, the translation fragment of each subtitle is dubbed, and the dubbing rate of the dubbing of the translation fragment of each subtitle is adjusted based on the original subtitle duration of the subtitle, so that the aim of high-precision time sequence alignment is fulfilled under the conditions of avoiding sentence splitting and semantic incoherence and ensuring semantic integrity and a watching process.

Automatic captioning pipelines and methods for automatically annotating video data with subtitles, which can be obtained using automatic speech recognition (ASR). An automatic captioning pipeline with inputs of multimodal data scales up the dataset of high-quality video-caption pairs. The automatic captioning pipeline generates video-caption pairs by establishing and using a large video-language dataset along with an automatic captioning approach leveraging multimodal inputs, such as textual video description, subtitles, and individual video frames.

The invention provides a voice subtitle generation method and device, a computer readable storage medium and a computer program product, and the method comprises the steps: constructing a target database for a plurality of historical subtitles based on the plurality of historical subtitles of a user; wherein the target database comprises entity segmented words, attribute information of the entity segmented words and a timestamp of each entity segmented word; based on the target database, generating a target graph structure for each entity segmented word; wherein the target graph structure comprises an association relationship between segmented words; if the to-be-determined word exists in the target voice information of the user, for each candidate word corresponding to the to-be-determined word, determining a target word corresponding to the to-be-determined word from a plurality of candidate words based on the target graph structure and a plurality of pieces of semantic information determined from a plurality of historical subtitles; and correcting the to-be-determined word based on the target word, and generating a target subtitle for the target voice information. According to the invention, the accuracy of generating the voice subtitles can be improved.

Embodiments of the present disclosure disclose a video subtitle extraction method and device. The specific implementation of the method comprises: disassembling a target video into a plurality of image frames, and numbering each image frame in chronological order; determining an initial subtitle for each image frame respectively; dividing each initial subtitle into a plurality of subtitle groups based on the similarity between the initial subtitles of adjacent image frames; for a subtitle group containing a plurality of initial subtitles, determining a target subtitle in the subtitle group based on the number of repeated occurrences and the text length of each initial subtitle in the subtitle group, and setting the start time and end time of the target subtitle based on the numbering of the image frames corresponding to the subtitle group. The implementation can extract subtitles in a video coherently and accurately.

A system providing real-time subtitles during live performances, such as plays, operas and musicals, in a preferred language and to aid the hearing impaired. The display device may be AR or smart glas

The invention discloses a teaching content evaluation method and apparatus, an electronic device and a medium. The method comprises the steps of obtaining a subtitle file and a teaching outline text corresponding to teaching audio data; performing first segmentation processing on the subtitle file to obtain a plurality of teaching fragments, and vectorizing each teaching fragment into a first vector; performing second segmentation processing on the teaching outline text to obtain a plurality of key knowledge point fragments, vectorizing each key knowledge point fragment into a second vector, and storing the second vector into a vector database; for each first vector, performing multi-path retrieval in a vector database to obtain a candidate knowledge point fragment set, and determining a target knowledge point fragment corresponding to the first vector from the candidate knowledge point fragment set; and inputting the target knowledge point fragment and the teaching fragment into a discrimination model, and outputting a matching result of the teaching fragment and the target knowledge point fragment. And the matching result can accurately evaluate the coincidence degree between the teaching of the teacher and the teaching outline.

A scalable, automated timed text workflow system designed to optimize the generation and refinement of time-synchronized textual content for video is disclosed. Integrated Machine Translation (MT) models and advanced AI-driven tools such as Computer Vision, Generative AI, and Traditional AI automatically generate and refine timed text, including subtitles, closed captions (CC), and SDH (Subtitles for the Deaf and Hard of Hearing). Human review is coordinated through a workflow orchestration system when needed, offering flexibility and scalability for handling high-volume media processing across various platforms and formats.

The invention discloses a remote sensing image subtitle generation method and system. The method comprises the steps of obtaining a combined remote sensing image data set; for each remote sensing image in the combined remote sensing image data set, generating first annotation information of the remote sensing image based on a remote sensing image category, a target detection bounding box and a semantic segmentation mask, determining image complexity based on the first annotation information, and determining a subtitle generation mode according to the image complexity, the subtitle generation mode comprises generation of subtitles according to rules, generation of subtitles by a multi-modal large model and generation of subtitles by combining generation of subtitles according to rules and generation of subtitles by the multi-modal large model; generating subtitles of the remote sensing image based on a subtitle generation mode; all the image subtitle pairs form a remote sensing image-text pairing data set; and training a CLIP model based on the remote sensing image-text pairing data set. According to the method, the workload of manual labeling is effectively reduced, and the construction cost of the high-quality remote sensing image-text data set is reduced.

This invention discloses a subtitle extraction method based on video text merging, filtering, and classification. The method includes extracting frames from the video, using optical character recognition (OCR) to identify text within all video frames, obtaining a set of text boxes within the video; merging and filtering the text boxes based on text content, text box coordinates, and text appearance time; and using a machine learning-based subtitle classification model to predict whether each filtered text box is a subtitle, saving the text identified as subtitles and their location information as the subtitle information for the video. This method, by merging and filtering text boxes, can initially filter out most text that does not belong to the subtitle type. By constructing a machine learning subtitle classification model, the text box type can be further determined. This method does not require specifying the subtitle area and can address the problem of variable subtitle positions in existing internet videos.

1. Name of the product in this design: Main body of the conference all-in-one machine. 2. Purpose of this design: The entire product is used for remote video conferencing, simultaneous interpretation, overlaying subtitles and generating meeting minutes, etc. The part for which protection is sought is the main body of the all-in-one conferencing device. 3. The key design features of this product are the shape of the solid line drawing. 4. The image or photograph that best illustrates the design's key points: 3D view 1. 5. Other situations requiring explanation: Other explanation: Dashed lines represent parts that do not require protection.

The application provides an AI-based precise clipping method and system for recording and broadcasting videos with time subtitles, and is applied to the technical field of video clipping. By acquiring a video to be processed and extracting image frames, reference time information and a region of interest in the image frames are identified, a mapping relationship table of the reference time and the video playing time is constructed, a user interaction instruction is received and analyzed to obtain a target reference time period, a corresponding video segment playing time point is queried based on the mapping relationship table, and the video is cut based on the query result, thereby generating a target video segment, which has the advantages of realizing precise and efficient clipping of recording and broadcasting videos and supporting flexible user interaction modes.