Subtitle timestamp correction method and related device

Abstract

The embodiment of the invention provides a subtitle timestamp correction method and a related device, and the method specifically comprises the steps: obtaining a standard subtitle file which corresponds to a multimedia file and contains a reference timestamp, and a reference subtitle file which corresponds to the multimedia file and contains a standard timestamp; and correcting the reference timestamp in the standard subtitle file according to a text comparison result between the reference subtitle file and the standard subtitle file and the standard timestamp. By implementing the embodiment of the invention, the accuracy of the timestamps corresponding to the subtitles can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a subtitle timestamp correction method and related devices. Background Art

[0002] When processing multimedia content, it is usually necessary to manually adjust the timeline of subtitles to keep them consistent with the timeline of audio and video content. This makes it easier for users to browse multimedia content with matching subtitles and audio and video.

[0003] To improve multimedia content processing efficiency, related technologies typically use editing software or pre-trained speech recognition models to convert audio and video content into text, and then create subtitles for the multimedia content. However, these subtitles often mismatch the audio and video content, resulting in subtitles that are output too quickly or too slowly. Summary of the Invention

[0004] In view of this, multiple embodiments of the present application are dedicated to providing a subtitle timestamp correction method and related apparatus, which can improve the accuracy of the timestamp corresponding to the subtitles.

[0005] One embodiment of the present application provides a subtitle timestamp correction method, the method comprising: obtaining a standard subtitle file corresponding to a multimedia file and containing a reference timestamp, and a reference subtitle file corresponding to the multimedia file and containing a standard timestamp; and correcting the reference timestamp in the standard subtitle file based on a text comparison result between the reference subtitle file and the standard subtitle file, and the standard timestamp.

[0006] Optionally, the step of obtaining a reference subtitle file corresponding to the multimedia file and containing a standard timestamp includes: obtaining a standard audio file corresponding to the multimedia file; wherein the standard audio file has a time axis consistent with that of the multimedia file; and extracting subtitles from the standard audio file according to the time axis in the standard audio file to obtain a reference subtitle file containing a standard timestamp.

[0007] Optionally, the step of correcting the reference timestamp in the standard subtitle file based on the text comparison result between the reference subtitle file and the standard subtitle file, as well as the standard timestamp, includes: comparing the reference subtitle file and the standard subtitle file at a sentence granularity level to obtain a text comparison result; correcting the standard timestamp at a sentence granularity level based on the text comparison result and the reference timestamp at the sentence granularity; comparing the corrected standard timestamp with the reference timestamp at the sentence granularity level to obtain a timestamp comparison result; and correcting the reference timestamp in the standard subtitle file based on the timestamp comparison result.

[0008] Optionally, the text comparison result indicates that there are completely identical sentences in the standard subtitle file and the reference subtitle file; and the step of correcting the standard timestamp at the sentence granularity level according to the text comparison result and the reference timestamp of the sentence granularity includes: correcting the end timestamp of the corresponding sentence in the standard timestamp according to the end timestamp of the corresponding sentence in the standard subtitle file.

[0009] Optionally, the text comparison result indicates that there are non-completely consistent sentences in the standard subtitle file and the reference subtitle file; the step of correcting the standard timestamp at the sentence granularity level based on the text comparison result and the reference timestamp of the sentence granularity includes: determining the character position that causes the non-completely consistent result from the standard subtitle file; and correcting the standard timestamp at the sentence granularity level based on the correction logic corresponding to the character position; wherein different character positions correspond to different correction logics.

[0010] Optionally, the step of correcting the standard timestamp at the sentence granularity level according to the correction logic corresponding to the character position includes: when the character position is at the beginning or middle of a sentence, according to the corresponding correction logic, correcting the start and end timestamps of the corresponding sentence in the reference subtitle file to the start and end timestamps of the corresponding sentence in the standard subtitle file; or, when the character position is at the end of a sentence, according to the corresponding correction logic, correcting the end timestamp of the corresponding sentence in the reference subtitle file according to the start timestamp of the next sentence of the corresponding sentence in the standard subtitle file; or, when the character position includes the beginning and end of a sentence, and the standard subtitle file In the case that the previous sentence / next sentence of the corresponding sentence in the standard subtitle file is a sentence that is completely consistent with the reference subtitle file, according to the corresponding correction logic, the sentence start timestamp of the corresponding sentence in the reference subtitle file is corrected according to the sentence end timestamp of the previous sentence of the corresponding sentence in the standard subtitle file, and the sentence end timestamp of the corresponding sentence in the reference subtitle file is corrected according to the sentence start timestamp of the next sentence of the corresponding sentence in the standard subtitle file; or, in the case that the character position contains the sentence start and sentence end, and the previous sentence and the next sentence of the corresponding sentence in the standard subtitle file are not completely consistent with the reference subtitle file, an alarm is triggered.

[0011] Optionally, the step of correcting the reference timestamp in the standard subtitle file according to the timestamp comparison result includes: when the timestamp comparison result indicates that the timestamp error for the same sentence in the reference timestamp and the corrected standard timestamp is greater than a preset threshold, correcting the first and last timestamps of the corresponding sentence in the reference timestamp to the first and last timestamps of the corresponding sentence in the standard timestamp; wherein the timestamp error is determined based on at least one of a sentence start timestamp error and a sentence end timestamp error.

[0012] One embodiment of the present application further provides a subtitle timestamp correction device, comprising: a subtitle file acquisition module, configured to acquire a standard subtitle file corresponding to a multimedia file and containing a reference timestamp, and a reference subtitle file corresponding to the multimedia file and containing a standard timestamp; a timestamp correction module, configured to correct the reference timestamp in the standard subtitle file based on a text comparison result between the reference subtitle file and the standard subtitle file, and the standard timestamp.

[0013] One embodiment of the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores at least one computer program, and the processor loads and executes the at least one computer program to implement the aforementioned method.

[0014] One embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by a processor, the method as described above can be implemented.

[0015] One embodiment of the present application further provides a computer program product, which is used to implement the aforementioned method.

[0016] In multiple implementations provided in the present application, a reference subtitle file has a standard timestamp. By comparing the reference subtitle file with the standard subtitle file, the reference subtitle file can be corrected based on the text comparison results between the two and the standard subtitle file. The reference timestamp of the standard subtitle file can then be corrected using the corrected reference subtitle file. This can result in a standard subtitle file with a standard timestamp, thereby improving the timestamp accuracy of the standard subtitle file and avoiding the problem of over-fast / over-slow subtitle output in multimedia content using the standard subtitle file. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a system architecture for implementing a subtitle timestamp correction method provided in accordance with one embodiment of the present application.

[0018] Figure 2A schematic diagram of timestamp alignment in the case where completely identical sentences exist in a reference subtitle file and a standard subtitle file provided for one embodiment of the present application.

[0019] Figure 3 A schematic diagram of timestamp alignment in a case where there are incompletely consistent sentences in a reference subtitle file and a standard subtitle file provided for one embodiment of the present application.

[0020] Figure 4 A flowchart of a subtitle timestamp correction method provided in accordance with one embodiment of the present application.

[0021] Figure 5 A schematic diagram of the modules of a subtitle timestamp correction device provided in one embodiment of the present application.

[0022] Figure 6 A schematic diagram of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the information to be retrieved in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0024] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0025] Related technologies primarily rely on manual processing of multimedia content to ensure that subtitles align with the timeline of the audio and video content. Specifically, software identifies the text corresponding to the multimedia content as subtitles. Subtitles typically come with timestamps, but these software-generated timestamps can easily misalign with the audio and video. Fusion of such subtitles with audio and video can result in multimedia content that outputs subtitles too quickly or too slowly, impacting the user experience.

[0026] Generally, to avoid these issues, it's necessary to manually review and adjust subtitle timestamps so that each sentence in the subtitles corresponds to the corresponding segment in the audio or video, thus preventing the subtitles from being output too quickly or too slowly compared to the audio or video output speed. However, this approach is clearly inefficient and can significantly impact multimedia content production efficiency when processing a large number of subtitles.

[0027] Therefore, it is necessary to provide a subtitle timestamp correction method. The reference subtitle file has a standard timestamp. By comparing the reference subtitle file and the standard subtitle file, the reference subtitle file can be corrected based on the text comparison results of the two and the standard subtitle file. The reference timestamp of the standard subtitle file is then corrected using the corrected reference subtitle file. A standard subtitle file with a standard timestamp can be obtained, which improves the timestamp accuracy of the standard subtitle file and avoids the problem of subtitle output being too fast / too slow in multimedia content using the standard subtitle file.

[0028] See also Figure 1 . In multiple implementations provided in the present application, the subtitle timestamp correction method can be applied to a subtitle timestamp correction device. The subtitle timestamp correction device can be an electronic device with certain computing capabilities and network access capabilities. The electronic device can be a desktop computer, a laptop computer, a tablet computer, or a server. The electronic device can be connected to the server through the network. The server can be a distributed server, including multiple processors, memories, network communication modules, etc., which work together to achieve various functions. Alternatively, the server can also be a server cluster formed by several servers, which has higher computing and data processing capabilities. With the development of science and technology, the server can also be implemented using new forms of technical means, such as a new type of "server" based on quantum computing. Of course, in some implementations, the subtitle timestamp correction device can also be a program module running in an electronic device.

[0029] Specifically, the electronic device includes a processor, a memory, a display module, and a network access module for network communication. The network access module for network communication is used to obtain from the outside a standard subtitle file containing a reference timestamp corresponding to a multimedia file, and a reference subtitle file containing a standard timestamp corresponding to the multimedia file. The memory is used to store the standard subtitle file and the reference subtitle file. The processor is used to read the stored standard subtitle file and the reference subtitle file from the memory, and correct the reference timestamp in the standard subtitle file based on the text comparison result between the reference subtitle file and the standard subtitle file, as well as the standard timestamp. The memory is used to store the corrected standard subtitle file. The display module is used to read and display the corrected standard subtitle file in the memory.

[0030] One embodiment of this application provides an example application scenario for a subtitle timestamp correction method. This subtitle timestamp correction method can be applied to scenarios such as digital human video production, online education video production, and film and television post-processing. The following description uses the digital human video production scenario as an example.

[0031] In the digital human video production business, the large model generates the text "Artificial intelligence technology is changing our lives. It not only improves efficiency but also creates new possibilities" as input, and digital human technology is used to generate corresponding video content. A subtitle generation tool generates a standard subtitle file containing a reference timestamp corresponding to the video content.

[0032] Due to limitations of subtitle generation tools, the generated standard subtitle files containing reference timestamps may have timestamp deviations, causing the subtitles to be displayed too fast or too slow, and out of sync with the voice of the speaker in the video.

[0033] This application can use the audio-to-text service to generate a reference subtitle file containing standard timestamps for the video content. At the sentence level, the reference subtitle file and the standard subtitle file are compared sentence by sentence to obtain text comparison results. Among them, the text comparison of sentences is performed word by word.

[0034] See also Figure 2 If the text comparison result indicates that the Nth sentence of the reference subtitle file and the standard subtitle file are both "Artificial intelligence technology is changing our lives" (i.e., the two sentences are exactly the same), and N is a positive integer, then the end timestamp "01:01" of the sentence "Artificial intelligence technology is changing our lives" in the reference subtitle file is corrected to the end timestamp "01:00" of the sentence "Artificial intelligence technology is changing our lives" in the standard subtitle file.

[0035] See also Figure 3 . If the text comparison result indicates that the Nth sentences of the reference subtitle file and the standard subtitle file are "Still artificial intelligence technology is changing our lives" and "Artificial intelligence technology is changing our lives" respectively (that is, the first letters of the two sentences are inconsistent), then the end timestamp "01:01" of the sentence "Still artificial intelligence technology is changing our lives" in the reference subtitle file is corrected to the end timestamp "01:00" of the sentence "Artificial intelligence technology is changing our lives" in the standard subtitle file, and the beginning timestamp "00:03" of the sentence "Still artificial intelligence technology is changing our lives" in the reference subtitle file is corrected to the beginning timestamp "00:01" of the sentence "Artificial intelligence technology is changing our lives" in the standard subtitle file.

[0036] If the text comparison result indicates that the Nth sentences of the reference subtitle file and the standard subtitle file are "Artificial intelligence technology is changing our lives" and "Artificial intelligence technology is changing our lives" respectively (that is, the mid-sentence characters of the two sentences are inconsistent), then the end timestamp "01:01" of the sentence "Artificial intelligence technology is changing our lives" in the reference subtitle file is corrected to the end timestamp "01:00" of the sentence "Artificial intelligence technology is changing our lives" in the standard subtitle file, and the beginning timestamp "00:03" of the sentence "Artificial intelligence technology is changing our lives" in the reference subtitle file is corrected to the beginning timestamp "00:01" of the sentence "Artificial intelligence technology is changing our lives" in the standard subtitle file.

[0037] If the text comparison result indicates that the Nth sentences of the reference subtitle file and the standard subtitle file are "Artificial intelligence technology is changing the way we make fire" and "Artificial intelligence technology is changing our lives" respectively (i.e., the ending characters of the two sentences are inconsistent), then the sentence-start timestamp "01:02" of the next sentence "It not only improves efficiency" of "Artificial intelligence technology is changing our lives" in the standard subtitle file is subtracted from the preset value (such as 00:01) and used as the sentence-end timestamp "01:01" of the sentence "Artificial intelligence technology is changing our making fire" in the reference subtitle file.

[0038] If the text comparison results indicate that the Nth sentence in the reference subtitles file and the standard subtitles file are "Artificial intelligence technology is changing our fire-making" and "Artificial intelligence technology is changing our lives," respectively (i.e., the ending and beginning characters of the two sentences are different), and the previous / next sentence of "Artificial intelligence technology is changing our lives" in the standard subtitles file is consistent with the reference subtitles file, then the sentence-ending timestamp "01:02" of the next sentence of "Artificial intelligence technology is changing our lives" in the standard subtitles file, "It not only improves efficiency," is subtracted from the preset value (e.g., 00:01), and this is used as the sentence-ending timestamp "01:01" of the sentence in the reference subtitles file. Furthermore, the sentence-ending timestamp "00:05" of the previous sentence of "Artificial intelligence technology is changing our lives" in the standard subtitles file, "With the development of technology," is added to the preset value (e.g., 00:01), and this is used as the sentence-ending timestamp "00:06" of the sentence in the reference subtitles file.

[0039] If the text comparison results indicate that the Nth sentence in the reference subtitle file and the standard subtitle file are "Artificial intelligence technology is changing our lives" and "Artificial intelligence technology is changing our lives" respectively (i.e., the ending and beginning characters of the two sentences are inconsistent), and the previous and next sentences of "Artificial intelligence technology is changing our lives" in the standard subtitle file are inconsistent with those in the reference subtitle file, an alarm is triggered.

[0040] Furthermore, the standard timestamp and the revised reference timestamp can be compared at the sentence level to obtain a timestamp comparison result. If the timestamp comparison result indicates that the timestamp error (i.e., the start and end timestamp of the sentence "Artificial intelligence technology is changing our lives") between the standard subtitle file and the revised reference timestamp is greater than 00:01, the start and end timestamps of the sentence in the standard subtitle file will be corrected to the start and end timestamps of the sentence in the reference timestamp.

[0041] In this application scenario example, the reference subtitle file and the standard subtitle file can be compared, and based on the text comparison results of the two, the reference subtitle file can be corrected based on the standard subtitle file, and then the reference timestamp of the standard subtitle file can be corrected using the corrected reference subtitle file. A standard subtitle file with a standard timestamp can be obtained, which improves the timestamp accuracy of the standard subtitle file and avoids the problem of subtitle output being too fast / too slow in multimedia content using the standard subtitle file.

[0042] See also Figure 4 One embodiment of the present application provides a subtitle timestamp correction method. The subtitle timestamp correction method can be applied to a subtitle timestamp correction device. The subtitle timestamp correction method can include the following steps.

[0043] Step S110: obtaining a standard subtitle file containing a reference time stamp corresponding to the multimedia file, and a reference subtitle file containing a standard time stamp corresponding to the multimedia file.

[0044] Step S120: Correcting the reference timestamp in the standard subtitle file according to the text comparison result between the reference subtitle file and the standard subtitle file, and the standard timestamp.

[0045] In this embodiment, the subtitle timestamp correction device can use the reference subtitle file to correct the timestamp of the standard subtitle file, thereby realizing automatic monitoring and correction of the subtitle timestamp and improving the output synchronization of the subtitle and multimedia content.

[0046] Specifically, the subtitle timestamp correction device can obtain a standard subtitle file containing a reference timestamp corresponding to a multimedia file and a reference subtitle file containing a standard timestamp corresponding to a media file. The subtitles contained in the standard subtitle file are text information that can accurately express the multimedia content in the multimedia file, such as "...artificial intelligence technology is changing our lives..." The standard subtitle file contains at least one of the following content that can be used as subtitles: proper nouns, abbreviations, uppercase and lowercase characters, emoticons, etc. The reference subtitle file can be obtained by invoking an audio-to-text service for the multimedia file. Therefore, there is output synchronization between the standard timestamps of the subtitles in the reference subtitle file and the multimedia content. The multimedia content includes but is not limited to: audio, video, and animated graphics with sound. Compared with the standard subtitle file, the reference subtitle file has a more accurate timestamp, and the standard subtitle file has more accurate subtitles than the reference subtitle file. Subtitles can be a sequence of multiple ordered characters / symbols.

[0047] In this embodiment, the standard timestamp and the reference timestamp can be represented in the same or different ways. The standard timestamp and the reference timestamp can be sentence-granular timestamps. It is understood that the standard timestamp and the reference timestamp can also be word-granular timestamps, paragraph-granular timestamps, chapter-granular timestamps, etc. The specific granularity depends on actual needs and is not limited in this embodiment of the application.

[0048] In this embodiment, the subtitle timestamp correction device can compare the subtitles in the reference subtitle file and the standard subtitle file word by word according to the character order in the reference subtitle file and the standard subtitle file, thereby obtaining a text comparison result. In some embodiments, the text comparison result is at a word granularity, a sentence granularity, or a paragraph granularity. For example, if the text comparison result is at a sentence granularity, if both the reference subtitle file and the standard subtitle file contain multiple sentences, the text comparison result includes multiple sub-results, each corresponding to a sentence. The sub-results are used to indicate the degree of consistency between sentences of the same sequence from the reference subtitle file and the standard subtitle file, respectively. The degree of consistency can be expressed as: completely consistent or not completely consistent.

[0049] In this embodiment, the timestamp processing methods corresponding to different degrees of consistency can be the same or different. Based on the representation of the degree of consistency in the text comparison results, the subtitle timestamp correction device can use processing logic that matches the degree of consistency to process the standard timestamp based on the reference timestamp. The processed standard timestamp can then be used as the basis for correcting the reference timestamp.

[0050] In this embodiment, the reference subtitle file has a standard timestamp. By comparing the reference subtitle file and the standard subtitle file, the reference subtitle file can be corrected based on the text comparison results of the two and the standard subtitle file. The reference timestamp of the standard subtitle file is then corrected using the corrected reference subtitle file. A standard subtitle file with a standard timestamp can be obtained, which improves the timestamp accuracy of the standard subtitle file and avoids the problem of too fast / too slow subtitle output in multimedia content using the standard subtitle file.

[0051] In some embodiments, the subtitle timestamp correction device can obtain a standard audio file corresponding to the multimedia file; wherein the standard audio file has a time axis consistent with the multimedia file; according to the time axis in the standard audio file, subtitles are extracted from the standard audio file to obtain a reference subtitle file containing a standard timestamp.

[0052] In this embodiment, to avoid subtitle misalignment due to timeline deviation, the subtitle timestamp correction device can extract a standard audio file from a multimedia file (such as a digital human video) using an audio separation tool (e.g., FFmpeg). A standard audio file is an audio stream that retains the complete voice information and timeline information of the multimedia file. The timeline of the standard audio file is consistent with the timeline of the multimedia file, which ensures that the timestamps of the subtitle extraction are synchronized with the video / audio images. Furthermore, the subtitle timestamp correction device can call an audio-to-text service to convert the audio content into a reference subtitle file containing a standard timestamp. The reference subtitle file is consistent with the timeline of the standard audio file, ensuring that the start and end times of each word / sentence / segment in the subtitles accurately reflect the actual playback position of the audio / video.

[0053] In this embodiment, when applied to the digital human video production scenario, if the standard subtitle file causes timestamp deviation due to the intelligent matching error of the editing software, the standard timestamp of the standard subtitle file can be combined with the standard timestamp of the reference subtitle file to achieve timestamp correction of the standard subtitle file, thereby improving the synchronization between the subtitles and the digital human video, reducing manual intervention, and improving the degree of automation of correcting the subtitle timestamp.

[0054] In some embodiments, the subtitle timestamp correction device can compare the reference subtitle file and the standard subtitle file at the sentence granularity level to obtain a text comparison result; correct the standard timestamp at the sentence granularity level based on the text comparison result and the reference timestamp at the sentence granularity; compare the corrected standard timestamp and the reference timestamp at the sentence granularity level to obtain a timestamp comparison result; and correct the reference timestamp in the standard subtitle file based on the timestamp comparison result.

[0055] In this embodiment, to improve the accuracy of subtitle timestamp correction, the subtitle timestamp correction device can correct the standard timestamp at the sentence granularity level based on the reference timestamp, and then correct the reference timestamp based on the corrected standard timestamp, thereby improving the synchronization between the timestamps of the standard subtitle file and the multimedia content. Sentence granularity refers to a processing method that uses sentences as the smallest processing unit. During the timestamp correction process, sentences are divided based on the recognition of specific punctuation marks (e.g., periods, question marks) or semantic integrity. Specifically, correcting the standard timestamp at the sentence granularity level refers to correcting the end and / or beginning timestamps within the standard timestamp. The corrected standard timestamp is sequentially compared with the reference timestamp at each time point, resulting in a timestamp comparison result comprising various groups of timestamp comparison sub-results. Each group of timestamp comparison sub-results represents the consistency between two time points (e.g., 00:01 and 00:00) derived from the standard timestamp and the reference timestamp, respectively. Based on the consistency indicated by the timestamp comparison result, the reference timestamp can be corrected to ensure that the standard subtitle file has accurate timestamps.

[0056] In some embodiments, when the granularity corresponding to the standard timestamp (e.g., sentence granularity) and the granularity corresponding to the reference timestamp (e.g., character granularity) are inconsistent, only the time corresponding to the granularity of the standard timestamp in the reference timestamp can be modified. For example, only the timestamp in the character granularity timestamp that can be identified as the end of sentence timestamp / beginning of sentence timestamp is corrected, and the timestamp corresponding to any character in the sentence is ignored to improve the efficiency of timestamp correction.

[0057] In some embodiments, the text comparison result indicates that there are completely identical sentences in the standard subtitle file and the reference subtitle file; the subtitle timestamp correction device can correct the end timestamp of the corresponding sentence in the standard timestamp based on the end timestamp of the corresponding sentence in the standard subtitle file.

[0058] In this embodiment, to improve the matching between the standard timestamp and the reference timestamp, the subtitle timestamp correction device can, when detecting that the Nth sentence in the standard subtitle file and the reference subtitle file are completely identical, modify the standard timestamp of the corresponding sentence in the reference subtitle file (e.g., 00:03) to the reference timestamp of the end of the corresponding sentence in the standard subtitle file (e.g., 00:01), since the sentence end timestamp can directly reflect the end position of the current sentence. Where N is a positive integer.

[0059] In some embodiments, the text comparison result indicates that there are non-completely consistent sentences between the standard subtitle file and the reference subtitle file; the subtitle timestamp correction device can determine the character position that causes the non-completely consistent result from the standard subtitle file; and correct the standard timestamp at the sentence granularity level according to the correction logic corresponding to the character position; wherein different character positions correspond to different correction logics.

[0060] In this embodiment, in order to improve the accuracy of subtitle timestamp correction, when comparing the Nth sentences in the standard subtitle file and the reference subtitle file, if there are different characters in the Nth sentences of the two, it means that the Nth sentences of the two are not completely consistent sentences. The subtitle timestamp correction device can determine the character position of the sentence in the reference subtitle file that causes the non-complete consistency. Different character positions correspond to different correction logics to achieve accurate correction of the subtitle timestamp. The correction logic is used to indicate at least one timestamp correction execution scheme, for example, modifying the first and last timestamps in the standard timestamp to the first and last timestamps in the reference timestamp. In some embodiments, the character position includes but is not limited to: in the middle of the sentence, at the beginning of the sentence, and at the end of the sentence. Regardless of the character position, the standard timestamp is corrected so that the modified standard timestamp can be used as the basis for correcting the reference timestamp.

[0061] In some embodiments, the subtitle timestamp correction device may, when the character position is at the beginning or middle of a sentence, correct the start and end timestamps of the corresponding sentence in the reference subtitle file to the start and end timestamps of the corresponding sentence in the standard subtitle file according to the corresponding correction logic; or, when the character position is at the end of a sentence, correct the end timestamp of the corresponding sentence in the reference subtitle file according to the start timestamp of the next sentence of the corresponding sentence in the standard subtitle file according to the corresponding correction logic; or, when the character position includes both the beginning and end of a sentence and the previous / next sentence of the corresponding sentence in the standard subtitle file is completely consistent with the reference subtitle file, correct the start timestamp of the corresponding sentence in the reference subtitle file according to the end timestamp of the previous sentence of the corresponding sentence in the standard subtitle file, and correct the end timestamp of the corresponding sentence in the reference subtitle file according to the start timestamp of the next sentence of the corresponding sentence in the standard subtitle file according to the corresponding correction logic; or, when the character position includes both the beginning and end of a sentence and the previous and next sentences of the corresponding sentence in the standard subtitle file are not completely consistent with the reference subtitle file, trigger an alarm.

[0062] In this embodiment, in order to achieve personalized timestamp correction for different character positions, when traversing the subtitles in character order, if the characters that cause incomplete consistency appear at the beginning of the Nth sentence, the start and end timestamps of the Nth sentence in the reference subtitle file can be corrected to the start and end timestamps of the Nth sentence in the standard subtitle file. In this case, there is no need to continue traversing and comparing other characters in the Nth sentence, but to traverse and compare the characters in the N+1th sentence, which can save computing power. When traversing in character order and finding that the characters that cause incomplete consistency appear in the Nth sentence, the start and end timestamps of the Nth sentence in the reference subtitle file can be corrected to the start and end timestamps of the Nth sentence in the standard subtitle file. When the characters that cause incomplete consistency appear at the end of the Nth sentence, the start timestamp of the N+1th sentence in the standard subtitle file can be added to the preset time amount to obtain the end timestamp of the Nth sentence in the reference subtitle file. When there are characters at both the beginning and end of a sentence that cause inconsistencies, the beginning timestamp of the N+1th sentence in the standard subtitle file can be added to a preset time amount to obtain the ending timestamp of the Nth sentence in the reference subtitle file. Furthermore, the ending timestamp of the N-1th sentence in the standard subtitle file can be added to the preset time amount to obtain the beginning timestamp of the Nth sentence in the reference subtitle file. It is required that only if the N+1th sentence in the standard subtitle file and the N-1th sentence in the reference subtitle file are completely consistent, otherwise, no timestamp correction is performed and an alarm is triggered to prompt manual intervention.

[0063] In some embodiments, the subtitle timestamp correction device can correct the first and last timestamps of the corresponding sentence in the reference timestamp to the first and last timestamps of the corresponding sentence in the standard timestamp when the timestamp comparison result indicates that the timestamp error for the same sentence in the reference timestamp and the corrected standard timestamp is greater than a preset threshold; wherein the timestamp error is determined based on at least one of the sentence start timestamp error and the sentence end timestamp error.

[0064] In this embodiment, to improve the synchronization between the timestamps of the standard subtitle file and the multimedia content, the subtitle timestamp correction device can correct the start and end timestamps of the standard subtitle file to those of the reference subtitle file if the timestamps of the same sentence in the standard subtitle file and the reference subtitle file are greater than a preset threshold (e.g., 0.1s), thereby improving the accuracy of the timestamps in the standard subtitle file. The timestamp error is calculated as the weighted sum of the start timestamp error, the end timestamp error, and the start and end timestamp errors.

[0065] See also Figure 5. The embodiment of the present application also provides a subtitle timestamp correction device. The subtitle timestamp correction device may include: a subtitle file acquisition module, used to obtain a standard subtitle file containing a reference timestamp corresponding to a multimedia file, and a reference subtitle file containing a standard timestamp corresponding to the multimedia file; a timestamp correction module, used to correct the reference timestamp in the standard subtitle file based on the text comparison result between the reference subtitle file and the standard subtitle file, and the standard timestamp.

[0066] In this embodiment, the specific functions and effects achieved by the subtitle timestamp correction device can be explained in reference to other embodiments of the present application and will not be repeated here.

[0067] See also Figure 6 The present application also provides a computer device, comprising a memory and a processor, wherein the memory stores at least one computer program, and the processor loads and executes the at least one computer program to implement the aforementioned method.

[0068] The embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor implements the method as described above.

[0069] The embodiments of the present application further provide a computer program product comprising instructions, which implements the aforementioned method when executed by a processor.

[0070] The user information or user account information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, etc.) involved in multiple implementation methods of this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0071] It should be understood that the specific examples herein are only intended to help those skilled in the art better understand the embodiments of the present application, rather than to limit the scope of the present invention.

[0072] It can be understood that in the various implementation methods of this application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation method of this application.

[0073] It can be understood that the various embodiments described in this application can be implemented individually or in combination, and the embodiments of this application are not limited to this.

[0074] Unless otherwise indicated, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art in the technical field of the present application. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the scope of this application. The term "and / or" used in this application includes any and all combinations of one or more related listed items. The singular forms "a", "above", and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise.

[0075] It is understood that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-mentioned method embodiment can be completed by hardware integrated logic circuits in the processor or software instructions. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above-mentioned method.

[0076] It will be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0077] Those skilled in the art will appreciate that the units and algorithmic steps of each example described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the information to be retrieved. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0078] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and units can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0079] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0080] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0081] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0082] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the information to be retrieved in this application, or the part that contributes to the prior art, or the part of the information to be retrieved can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0083] The above description is merely a specific embodiment of the present application, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A subtitle timestamp correction method, characterized in that: The method comprises: Obtaining a standard subtitle file containing a reference timestamp corresponding to the multimedia file, and a reference subtitle file containing a standard timestamp corresponding to the multimedia file; The reference timestamp in the standard subtitle file is corrected according to a text comparison result between the reference subtitle file and the standard subtitle file, and the standard timestamp.

2. The method according to claim 1, characterized in that The step of obtaining a reference subtitle file containing a standard timestamp corresponding to the multimedia file includes: Obtaining a standard audio file corresponding to the multimedia file; wherein the standard audio file has a time axis consistent with that of the multimedia file; Subtitles are extracted from the standard audio file according to a time axis in the standard audio file to obtain a reference subtitle file containing a standard timestamp.

3. The method according to claim 1, characterized in that The step of correcting the reference timestamp in the standard subtitle file according to the text comparison result between the reference subtitle file and the standard subtitle file, and the standard timestamp, comprises: Comparing the reference subtitle file with the standard subtitle file at a sentence granularity level to obtain a text comparison result; Correcting the standard timestamp at the sentence granularity level according to the text comparison result and the reference timestamp at the sentence granularity; Comparing the corrected standard timestamp with the reference timestamp at the sentence granularity level to obtain a timestamp comparison result; The reference timestamp in the standard subtitle file is corrected according to the timestamp comparison result.

4. The method according to claim 3, characterized in that The text comparison result indicates that there are completely identical sentences in the standard subtitle file and the reference subtitle file; and the step of correcting the standard timestamp at the sentence granularity level according to the text comparison result and the reference timestamp at the sentence granularity includes: According to the end timestamp of the corresponding sentence in the standard subtitle file, the end timestamp of the corresponding sentence in the standard timestamp is corrected.

5. The method according to claim 3, characterized in that The text comparison result indicates that there are non-completely identical sentences between the standard subtitle file and the reference subtitle file; and the step of correcting the standard timestamp at the sentence granularity level according to the text comparison result and the reference timestamp at the sentence granularity includes: Determining character positions that result in non-completely consistent results from the standard subtitle file; The standard timestamp is corrected at a sentence granularity level according to a correction logic corresponding to the character position; wherein different character positions correspond to different correction logics.

6. The method according to claim 5, characterized in that The step of correcting the standard timestamp at a sentence granularity level according to correction logic corresponding to the character position includes: When the character position is at the beginning or middle of a sentence, the start and end timestamps of the corresponding sentence in the reference subtitle file are corrected to the start and end timestamps of the corresponding sentence in the standard subtitle file according to the corresponding correction logic; Alternatively, when the character position is at the end of a sentence, the end timestamp of the corresponding sentence in the reference subtitle file is corrected according to the start timestamp of the next sentence of the corresponding sentence in the standard subtitle file according to the corresponding correction logic; Alternatively, when the character position includes the beginning and end of a sentence, and the previous sentence / next sentence of the corresponding sentence in the standard subtitle file is completely consistent with the sentence in the reference subtitle file, according to the corresponding correction logic, the beginning timestamp of the corresponding sentence in the reference subtitle file is corrected according to the ending timestamp of the previous sentence of the corresponding sentence in the standard subtitle file, and the ending timestamp of the corresponding sentence in the reference subtitle file is corrected according to the beginning timestamp of the next sentence of the corresponding sentence in the standard subtitle file; Alternatively, an alarm is triggered when the character position includes the beginning and the end of a sentence, and the previous sentence and the next sentence of the corresponding sentence in the standard subtitle file are not completely consistent with those in the reference subtitle file.

7. The method according to claim 3, characterized in that The step of correcting the reference timestamp in the standard subtitle file according to the timestamp comparison result includes: When the timestamp comparison result indicates that the timestamp error for the same sentence in the reference timestamp and the corrected standard timestamp is greater than a preset threshold, the first and last timestamps of the corresponding sentence in the reference timestamp are corrected to the first and last timestamps of the corresponding sentence in the standard timestamp; wherein the timestamp error is determined based on at least one of the sentence start timestamp error and the sentence end timestamp error.

8. A subtitle timestamp correction device, characterized in that: The device comprises: A subtitle file acquisition module, configured to acquire a standard subtitle file containing a reference timestamp corresponding to a multimedia file, and a reference subtitle file containing a standard timestamp corresponding to the multimedia file; The timestamp correction module is used to correct the reference timestamp in the standard subtitle file according to the text comparison result between the reference subtitle file and the standard subtitle file and the standard timestamp.

9. A computer device, characterized in that: The computer device includes a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by a processor, it can implement the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that A computer program product for implementing the method according to any one of claims 1 to 7.

Images