Subtitle timestamp correction method and related apparatus

Abstract

The embodiment of the present application provides a subtitle timestamp correction method and related device, specifically comprising: obtaining a standard subtitle file corresponding to a multimedia file and containing reference timestamps, and a reference subtitle file corresponding to the multimedia file and containing standard timestamps; correcting the reference timestamps in the standard subtitle file according to a text comparison result between the reference subtitle file and the standard subtitle file and the standard timestamps. The embodiment of the present application can improve the accuracy of the timestamps corresponding to the subtitles.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of computer technology, and in particular, to a subtitle timestamp correction method and related device. BACKGROUND

[0002] In processing multimedia content, it is usually necessary to manually adjust the time axis of the subtitle so as to keep the subtitle consistent with the time axis of the audio and video content, so as to facilitate the user to browse the multimedia content in which the subtitle and the audio and video are matched.

[0003] In order to improve the processing efficiency of multimedia content, the related art usually uses editing software or a pre-trained speech recognition model to recognize the audio and video content into text, and makes the text as a subtitle to the multimedia content. However, the subtitle obtained by this way often has the problem of not matching the audio and video content, that is, the problem of too fast / too slow output of the subtitle. SUMMARY

[0004] Therefore, embodiments of the present application are devoted to providing a subtitle timestamp correction method and related device, which can improve the accuracy of the timestamp corresponding to the subtitle.

[0005] One embodiment of the present application provides a subtitle timestamp correction method, which comprises: obtaining a standard subtitle file corresponding to a multimedia file, the standard subtitle file comprising reference timestamps, and a reference subtitle file corresponding to the multimedia file, the reference subtitle file comprising standard timestamps; and correcting the reference timestamps in the standard subtitle file according to a text comparison result between the reference subtitle file and the standard subtitle file and the standard timestamps.

[0006] Optionally, the step of obtaining the reference subtitle file comprising the standard timestamps corresponding to the multimedia file comprises: obtaining a standard audio file corresponding to the multimedia file; wherein the standard audio file has a time axis consistent with the multimedia file; and extracting a subtitle from the standard audio file according to the time axis in the standard audio file to obtain the reference subtitle file comprising the standard timestamps.

[0007] Optionally, the step of correcting the reference timestamps in the standard subtitle file according to the text comparison result between the reference subtitle file and the standard subtitle file and the standard timestamps comprises: comparing the reference subtitle file and the standard subtitle file at a sentence granularity level to obtain a text comparison result; correcting the reference timestamps at the sentence granularity level according to the text comparison result and the standard timestamps at the sentence granularity level; comparing the corrected reference timestamps and the standard timestamps at the sentence granularity level to obtain a timestamp comparison result; and correcting the reference timestamps in the standard subtitle file according to the timestamp comparison result.

[0008] Optionally, the text comparison result indicates that there are completely consistent sentences in the standard subtitle file and the reference subtitle file; and the step of correcting the standard time stamp at the sentence granularity level according to the text comparison result and the standard time stamp at the sentence granularity level includes: correcting the end time stamp of the corresponding sentence in the reference time stamp according to the end time stamp of the corresponding sentence in the reference subtitle file.

[0009] Optionally, the text comparison result indicates that there are non-completely consistent sentences in the standard subtitle file and the reference subtitle file; and the step of correcting the standard time stamp at the sentence granularity level according to the text comparison result and the reference time stamp at the sentence granularity level includes: determining the character position in the standard subtitle file that leads to the non-completely consistent result; and correcting the reference time stamp at the sentence granularity level according to the correction logic corresponding to the character position; wherein different character positions correspond to different correction logics.

[0010] Optionally, the step of correcting the reference time stamp at the sentence granularity level according to the correction logic corresponding to the character position includes: in the case that the character position is the beginning of a sentence or in the middle of a sentence, correcting the beginning and end time stamps of the corresponding sentence in the standard subtitle file to the beginning and end time stamps of the corresponding sentence in the reference subtitle file according to the corresponding correction logic; or in the case that the character position is the end of a sentence, correcting the end time stamp of the corresponding sentence in the standard subtitle file according to the beginning time stamp of the next sentence of the corresponding sentence in the reference subtitle file according to the corresponding correction logic; or in the case that the character position includes the beginning and end of a sentence and the previous / next sentence of the corresponding sentence in the standard subtitle file is a sentence completely consistent with the reference subtitle file, correcting the beginning time stamp of the corresponding sentence in the standard subtitle file according to the end time stamp of the previous sentence of the corresponding sentence in the reference subtitle file and correcting the end time stamp of the corresponding sentence in the standard subtitle file according to the beginning time stamp of the next sentence of the corresponding sentence in the reference subtitle file according to the corresponding correction logic; or in the case that the character position includes the beginning and end of a sentence and the previous and next sentences of the corresponding sentence in the standard subtitle file are sentences non-completely consistent with the reference subtitle file, triggering an alarm.

[0011] Optionally, the step of correcting the reference time stamp in the standard subtitle file according to the time stamp comparison result comprises: in the case that the time stamp error of the same sentence in the corrected reference time stamp and the standard time stamp is greater than a preset threshold according to at least one of the beginning time stamp error and the end time stamp error, correcting the beginning time stamp and the end time stamp of the corresponding sentence in the reference time stamp to the beginning time stamp and the end time stamp of the corresponding sentence in the standard time stamp.

[0012] One embodiment of the present application further provides a subtitle time stamp correction device, which comprises: a subtitle file acquisition module, configured to acquire a standard subtitle file corresponding to a multimedia file and comprising reference time stamps, and a reference subtitle file corresponding to the multimedia file and comprising standard time stamps; and a time stamp correction module, configured to correct the reference time stamps in the standard subtitle file according to a text comparison result between the reference subtitle file and the standard subtitle file and the standard time stamps.

[0013] One embodiment of the present application further provides a computer device, which comprises a memory and a processor, and the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the method as described above.

[0014] One embodiment of the present application further provides a computer readable storage medium, which stores at least one computer program, and the at least one computer program is executed by a processor to implement the method as described above.

[0015] One embodiment of the present application further provides a computer program product, which is used to implement the method as described above.

[0016] In the embodiments provided in the present application, the reference subtitle file has standard time stamps, by comparing the reference subtitle file and the standard subtitle file, the reference subtitle file can be corrected based on the text comparison result and the standard subtitle file, and then the reference time stamps of the standard subtitle file can be corrected by using the corrected reference subtitle file, so that the standard subtitle file with standard time stamps is obtained, the time stamp accuracy of the standard subtitle file is improved, and the problem of too fast or too slow subtitle output of the multimedia content using the standard subtitle file can be avoided. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The system architecture schematic diagram for implementing the subtitle time stamp correction method is provided for one embodiment of the present application.

[0018] Figure 2The timestamp alignment diagram in the case of the completely consistent sentence in the reference subtitle file and the standard subtitle file provided for an embodiment of the present application.

[0019] Figure 3 The timestamp alignment diagram in the case of the inconsistent sentence in the reference subtitle file and the standard subtitle file provided for an embodiment of the present application.

[0020] Figure 4 The flowchart of the subtitle timestamp correction method provided for an embodiment of the present application.

[0021] Figure 5 The module diagram of the subtitle timestamp correction device provided for an embodiment of the present application.

[0022] Figure 6 The schematic diagram of the computer device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0023] The to-be-retrieved information in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0024] In the description of the embodiments of the present application, it should be understood that the terms "first", "second" are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0025] In the related art, the multimedia content is mainly processed by manual processing, so that the time axis of the subtitle and the audio and video content is kept consistent. Specifically, the text corresponding to the multimedia content is recognized by the related software as the subtitle, and the subtitle usually has a timestamp, but the timestamp is generated by the related software, and the problem of not corresponding to the audio and video easily exists. The fusion of such subtitles and audio and video will produce a multimedia content with too fast / slow subtitle output, affecting the user experience.

[0026] Generally speaking, in order to avoid the above-mentioned problems, the timestamp of the subtitle needs to be manually audited, and the timestamp needs to be manually adjusted, so that each sentence in the subtitle can correspond to the corresponding segment in the audio and video, avoiding the problem of too fast / slow subtitle output compared with the audio and video output speed. However, obviously, the efficiency of this way is relatively low, and when there are a large number of subtitles to be processed, the production efficiency of the multimedia content will be obviously affected.

[0027] Therefore, it is necessary to provide a subtitle timestamp correction method. The reference subtitle file has standard timestamps. By comparing the reference subtitle file and the standard subtitle file, the text comparison result of the two can be based on the standard subtitle file to correct the reference timestamp of the reference subtitle file. The corrected reference subtitle file can be used to correct the reference timestamp of the standard subtitle file. The standard subtitle file with standard timestamp can be obtained, which improves the timestamp accuracy of the standard subtitle file and avoids the problem of too fast / slow output of the multimedia content using the standard subtitle file.

[0028] Please refer to Figure 1 In the multiple embodiments provided in the present application, the subtitle timestamp correction method can be applied to a subtitle timestamp correction device. The subtitle timestamp correction device can be an electronic device with certain computing power and network access capability. The electronic device can be a desktop computer, a notebook computer, a tablet computer, or a server. The electronic device can connect to a server through a network. The server can be a distributed server, including multiple processors, memories, network communication modules, etc. to realize various functions. Alternatively, the server can also be a server cluster formed by several servers, which has higher computing and data processing capability. With the development of science and technology, the server can also use new forms of technical means to realize, such as a new type of "server" based on quantum computing. Of course, in some embodiments, the subtitle timestamp correction device can also be a program module running in the electronic device.

[0029] Specifically, the electronic device includes a processor, a memory, a display module, and a network access module for network communication. The network access module for network communication is used to obtain the standard subtitle file containing the reference timestamp corresponding to the multimedia file and the reference subtitle file containing the standard timestamp corresponding to the multimedia file from the outside. The memory is used to store the standard subtitle file and the reference subtitle file. The processor is used to read the standard subtitle file and the reference subtitle file from the memory, correct the reference timestamp in the standard subtitle file according to the text comparison result between the reference subtitle file and the standard subtitle file, and the standard timestamp. The memory is used to store the corrected standard subtitle file. The display module is used to read and display the corrected standard subtitle file in the memory.

[0030] An embodiment of the present application provides an application scenario example of a subtitle timestamp correction method. The subtitle timestamp correction method can be applied to digital human video production, online education video production, and film and television post-processing scenarios. The following takes the digital human video production scenario as an example for illustration.

[0031] In the digital human video production business, a large model generates a script "Artificial intelligence technology is changing our lives. It not only improves efficiency, but also creates new possibilities" as input, and uses digital human technology to generate video content corresponding to it. A standard subtitle file containing reference timestamps corresponding to the video content is generated by a subtitle generation tool.

[0032] Due to the limitations of the subtitle generation tool, the generated standard subtitle file containing reference timestamps may have timestamp deviations, resulting in too fast or too slow display of subtitles, which is out of sync with the speaker's voice in the video.

[0033] The present application can call an audio-to-text service to generate a reference subtitle file containing standard timestamps for video content. At the sentence granularity level, the reference subtitle file and the standard subtitle file are compared sentence by sentence to obtain a text comparison result. Among them, for the text comparison of a sentence, a word-by-word comparison method is used.

[0034] Please refer to Figure 2 If the text comparison result indicates that the Nth sentence of the reference subtitle file and the standard subtitle file is "Artificial intelligence technology is changing our lives" (i.e., the two sentences are completely consistent), and N is a positive integer, the end timestamp "01:00" of the sentence "Artificial intelligence technology is changing our lives" in the standard subtitle file is corrected to the end timestamp "01:01" of the sentence "Artificial intelligence technology is changing our lives" in the reference subtitle file.

[0035] Please refer to Figure 3 If the text comparison result indicates that the Nth sentence of the reference subtitle file and the standard subtitle file is "still artificial intelligence technology is changing our lives" and "artificial intelligence technology is changing our lives" respectively (i.e., the first word of the two sentences is inconsistent), the end timestamp "01:00" of the sentence "artificial intelligence technology is changing our lives" in the standard subtitle file is corrected to the end timestamp "01:01" of the sentence "still artificial intelligence technology is changing our lives" in the reference subtitle file, and the start timestamp "00:01" of the sentence "artificial intelligence technology is changing our lives" in the standard subtitle file is corrected to the start timestamp "00:03" of the sentence "still artificial intelligence technology is changing our lives" in the reference subtitle file.

[0036] If the text alignment result indicates that the Nth sentence of the reference subtitle file and the standard subtitle file are "Artificial intelligence technology is changing our life" and "Artificial intelligence technology is changing our life" respectively (i.e., the last word of the two sentences is inconsistent), the last timestamp "01:00" of the sentence "Artificial intelligence technology is changing our life" in the standard subtitle file is corrected to the last timestamp "01:01" of the sentence "Artificial intelligence technology is changing our life" in the reference subtitle file, and the first timestamp "00:01" of the sentence "Artificial intelligence technology is changing our life" in the standard subtitle file is corrected to the first timestamp "00:03" of the sentence "Artificial intelligence technology is changing our life" in the reference subtitle file.

[0037] If the text alignment result indicates that the Nth sentence of the reference subtitle file and the standard subtitle file are "Artificial intelligence technology is changing our life" and "Artificial intelligence technology is changing our life" respectively (i.e., the last word of the two sentences is inconsistent), the last timestamp "01:00" of the sentence "Artificial intelligence technology is changing our life" in the standard subtitle file is corrected to the last timestamp "01:01" of the sentence "Artificial intelligence technology is changing our life" in the reference subtitle file, and the first timestamp "00:01" of the sentence "Artificial intelligence technology is changing our life" in the standard subtitle file is corrected to the first timestamp "00:03" of the sentence "Artificial intelligence technology is changing our life" in the reference subtitle file.

[0038] If the text alignment result indicates that the Nth sentence of the reference subtitle file and the standard subtitle file are "Artificial intelligence technology is changing our life" and "Artificial intelligence technology is changing our life" respectively (i.e., the last word of the two sentences is inconsistent), the last timestamp "01:00" of the sentence "Artificial intelligence technology is changing our life" in the standard subtitle file is corrected to the last timestamp "01:01" of the sentence "Artificial intelligence technology is changing our life" in the reference subtitle file, and the first timestamp "00:01" of the sentence "Artificial intelligence technology is changing our life" in the standard subtitle file is corrected to the first timestamp "00:03" of the sentence "Artificial intelligence technology is changing our life" in the reference subtitle file.

[0039] If the text comparison result indicates that the Nth sentence of the reference subtitle file and the standard subtitle file are "still artificial intelligence technology is changing our life" and "artificial intelligence technology is changing our life" respectively (i.e., the end-of-sentence character and the beginning-of-sentence character are inconsistent between the two sentences), and the previous sentence and the next sentence of "still artificial intelligence technology is changing our life" in the reference subtitle file are inconsistent with the sentences of the standard subtitle file, an alarm is triggered.

[0040] Further, the standard timestamp and the corrected reference timestamp can be compared at the sentence granularity level to obtain a timestamp comparison result. If the timestamp comparison result indicates that the timestamp (i.e., the beginning-of-sentence timestamp / end-of-sentence timestamp) of "artificial intelligence technology is changing our life" in the standard timestamp and the corrected reference timestamp is greater than 00:01, the beginning-of-sentence timestamp and the end-of-sentence timestamp of the sentence in the standard subtitle file are corrected to the beginning-of-sentence timestamp and the end-of-sentence timestamp of the sentence in the standard timestamp.

[0041] In the application scenario example, the reference subtitle file and the standard subtitle file can be compared, and based on the text comparison result of the two, the reference subtitle file is corrected based on the standard subtitle file. The reference timestamp of the standard subtitle file is corrected using the corrected reference subtitle file, and a standard subtitle file with a standard timestamp is obtained, which improves the accuracy of the timestamp of the standard subtitle file and avoids the problem of too fast / too slow output of the multimedia content using the standard subtitle file.

[0042] Referring to Figure 4 One embodiment of the present application provides a subtitle timestamp correction method. The subtitle timestamp correction method can be applied to a subtitle timestamp correction device. The subtitle timestamp correction method can include the following steps.

[0043] Step S110: obtaining a standard subtitle file corresponding to a multimedia file and containing a reference timestamp, and a reference subtitle file corresponding to the multimedia file and containing a standard timestamp.

[0044] Step S120: correcting the reference timestamp in the standard subtitle file according to the text comparison result between the reference subtitle file and the standard subtitle file, and the standard timestamp.

[0045] In this embodiment, the subtitle timestamp correction device can correct the timestamp of the standard subtitle file using the reference subtitle file, thereby realizing automatic monitoring and correction of the subtitle timestamp and improving the output synchronization of the subtitle and the multimedia content.

[0046] Specifically, the subtitle timestamp correction apparatus can obtain a standard subtitle file corresponding to the multimedia file and containing reference timestamps, and a reference subtitle file corresponding to the multimedia file and containing standard timestamps. The subtitle contained in the standard subtitle file can accurately express the text information of the multimedia content in the multimedia file, such as “……Artificial intelligence technology is changing our lives……”. The standard subtitle file contains at least one of the following contents that can be used as subtitles: proper nouns, abbreviations, uppercase and lowercase characters, emoticons, etc. The reference subtitle file can be obtained by calling a speech-to-text service for the multimedia file. Therefore, there is output synchronization between the standard timestamps of the subtitles in the reference subtitle file and the multimedia content. The multimedia content includes but is not limited to audio, video, dynamic sound images, etc. The reference subtitle file has more accurate timestamps than the standard subtitle file, and the standard subtitle file has more accurate subtitles than the reference subtitle file. The subtitle can be a sequence of multiple ordered characters / symbols.

[0047] In this embodiment, the standard timestamps and the reference timestamps can use the same / different representation. The standard timestamps and the reference timestamps can be sentence-granularity timestamps. It can be understood that the standard timestamps and the reference timestamps can also be word-granularity timestamps, section-granularity timestamps, chapter-granularity timestamps, etc. The specific granularity depends on the actual requirements, and the embodiments of the present application do not limit this.

[0048] In this embodiment, the subtitle timestamp correction apparatus can compare the subtitles in the reference subtitle file and the standard subtitle file word by word according to the character order in the reference subtitle file and the standard subtitle file, to obtain a text comparison result. In some embodiments, the text comparison result is word-granularity / sentence-granularity / section-granularity. Taking the case of sentence-granularity text comparison result as an example, if the reference subtitle file and the standard subtitle file both contain multiple sentences, the text comparison result contains multiple sub-results, each of which corresponds to a sentence. The sub-result is used to represent the consistency degree of the same order of sentences from the reference subtitle file and the standard subtitle file, respectively. The consistency degree can be represented as: complete consistency, incomplete consistency.

[0049] In this embodiment, the corresponding timestamp processing methods can be the same or different for different consistency degrees. According to the representation of the text comparison result for the consistency degree, the subtitle timestamp correction apparatus can use a processing logic that matches the consistency degree to process the reference timestamps of the standard subtitle file according to the standard timestamps of the reference subtitle file. The processed standard subtitle file can be used as a correction basis for the reference subtitle file.

[0050] In the embodiment, the reference subtitle file has standard timestamps, by comparing the reference subtitle file and the standard subtitle file, the text comparison result of the two can be obtained, and the reference subtitle file is corrected based on the standard subtitle file, and then the reference timestamp of the standard subtitle file is corrected by using the corrected reference subtitle file, so that the standard subtitle file with standard timestamps is obtained, the timestamp accuracy of the standard subtitle file is improved, and the problem of too fast / slow subtitle output of the multimedia content using the standard subtitle file can be avoided.

[0051] In some embodiments, the subtitle timestamp correction device can obtain a standard audio file corresponding to the multimedia file; wherein the standard audio file has a time axis consistent with the multimedia file; and perform subtitle extraction on the standard audio file according to the time axis in the standard audio file to obtain a reference subtitle file containing standard timestamps.

[0052] In the embodiment, in order to avoid the problem of subtitle misplacement caused by time axis deviation, the subtitle timestamp correction device can extract a standard audio file from the multimedia file (such as a digital human video) by using an audio separation tool (such as FFmpeg). The standard audio file is an audio stream that retains the complete voice information and time axis information of the multimedia file, and the time axis of the standard audio file is consistent with that of the multimedia file, so that the timestamp of the extracted subtitle can be synchronized with the video / audio picture. Further, the subtitle timestamp correction device can call an audio-to-text service to convert the audio content into a reference subtitle file containing standard timestamps, wherein the reference subtitle file is consistent with the time axis of the standard audio file, so that the start and end times of each word / sentence / paragraph in the subtitle can accurately reflect the actual playing position of the audio / video.

[0053] In the embodiment, when applied to the digital human video production scene, if the standard subtitle file has timestamp deviation caused by intelligent matching error of the editing software, the timestamp of the standard subtitle file can be corrected by combining the standard timestamps of the reference subtitle file, so as to improve the synchronization of the subtitle and the digital human video, reduce manual intervention, and improve the automation degree of correcting the subtitle timestamp.

[0054] In some embodiments, the subtitle timestamp correction device can compare the reference subtitle file and the standard subtitle file at the sentence granularity level to obtain a text comparison result; correct the reference timestamp at the sentence granularity level according to the text comparison result and the standard timestamp at the sentence granularity level; compare the corrected reference timestamp and the standard timestamp at the sentence granularity level to obtain a timestamp comparison result; and correct the reference timestamp in the standard subtitle file according to the timestamp comparison result.

[0055] In the embodiment, in order to improve the correction accuracy of the subtitle timestamp, the subtitle timestamp correction device can correct the reference timestamp of the standard subtitle file at the sentence granularity level according to the standard timestamp of the reference subtitle file, and then correct the reference subtitle file according to the corrected standard subtitle file, so as to improve the synchronization of the timestamp of the standard subtitle file and the multimedia content. The sentence granularity refers to a processing manner taking a sentence as the minimum processing unit, and the division of a sentence is based on the recognition of a specific punctuation mark (such as a period, a question mark) or semantic integrity in the timestamp correction process. Specifically, correcting the reference timestamp at the sentence granularity level refers to correcting the end-of-sentence timestamp and / or the beginning-of-sentence timestamp in the reference timestamp. After sequentially comparing the corrected reference timestamp with the standard timestamp at each time point, a timestamp comparison result including a plurality of timestamp comparison sub-results can be obtained. Each timestamp comparison sub-result is used to represent the consistency of two time points (such as 00:01 and 00:00) from the standard timestamp and the reference timestamp, respectively. According to the consistency represented by the timestamp comparison result, the reference timestamp can be corrected, so that the standard subtitle file has accurate timestamps.

[0056] In some embodiments, when the granularity corresponding to the standard timestamp (such as sentence granularity) and the granularity corresponding to the reference timestamp (such as word granularity) are inconsistent, only the time corresponding to the granularity of the standard timestamp in the reference timestamp can be modified, for example, only the timestamp that can be identified as the end-of-sentence timestamp / beginning-of-sentence timestamp in the word granularity timestamp is corrected, and the timestamp corresponding to any character in the sentence is ignored, so as to improve the timestamp correction efficiency.

[0057] In some embodiments, the text comparison result indicates that there is a completely consistent sentence in the standard subtitle file and the reference subtitle file; and the subtitle timestamp correction device can correct the end-of-sentence timestamp of the corresponding sentence in the reference timestamp according to the end-of-sentence timestamp of the corresponding sentence in the standard subtitle file.

[0058] In the embodiment, in order to improve the matching between the standard timestamp and the reference timestamp, the subtitle timestamp correction device can correct the reference timestamp of the corresponding sentence in the standard subtitle file (such as 00:03) to the standard timestamp of the end of the corresponding sentence in the reference subtitle file (such as 00:01) when it is detected that the Nth sentence of the standard subtitle file and the reference subtitle file is completely consistent. The Nth sentence is a positive integer. The end-of-sentence timestamp can directly reflect the end position of the current sentence.

[0059] In some embodiments, the text comparison result indicates that the standard subtitle file and the reference subtitle file have non-identical sentences; the subtitle timestamp correction device can determine the character position in the standard subtitle file that causes the non-identical result; and correct the reference timestamp at the sentence level according to the correction logic corresponding to the character position; wherein different character positions correspond to different correction logics.

[0060] In the present embodiment, in order to improve the correction accuracy of the subtitle timestamp, when comparing the Nth sentence in the standard subtitle file and the reference subtitle file, if there are different characters in the Nth sentences of the two, it means that the Nth sentences of the two are non-identical sentences; the subtitle timestamp correction device can determine the character position in the reference subtitle file that causes the non-identical result; different character positions correspond to different correction logics to achieve accurate correction of the subtitle timestamp. The correction logic is used to indicate at least one timestamp correction execution scheme, for example, modifying the first and last timestamps in the reference timestamp to the first and last timestamps in the standard timestamp. In some embodiments, the character position includes but is not limited to the middle of the sentence, the beginning of the sentence, and the end of the sentence. Regardless of the character position, the reference timestamp is corrected so that the modified standard subtitle file can be used as the basis for correcting the reference subtitle file.

[0061] In some embodiments, the subtitle timestamp correction device can correct the first and last timestamps of the corresponding sentence in the standard subtitle file to the first and last timestamps of the corresponding sentence in the reference subtitle file according to the corresponding correction logic when the character position is the beginning of the sentence or the middle of the sentence; or correct the end timestamp of the corresponding sentence in the standard subtitle file according to the beginning timestamp of the next sentence of the corresponding sentence in the reference subtitle file according to the corresponding correction logic when the character position is the end of the sentence; or correct the beginning timestamp of the corresponding sentence in the standard subtitle file according to the end timestamp of the previous sentence of the corresponding sentence in the reference subtitle file according to the corresponding correction logic when the character position includes the beginning and the end, and the previous / next sentence of the corresponding sentence in the standard subtitle file is a sentence identical to the reference subtitle file; or correct the beginning timestamp of the corresponding sentence in the standard subtitle file according to the end timestamp of the previous sentence of the corresponding sentence in the reference subtitle file and correct the end timestamp of the corresponding sentence in the standard subtitle file according to the beginning timestamp of the next sentence of the corresponding sentence in the reference subtitle file according to the corresponding correction logic when the character position includes the beginning and the end, and the previous and next sentences of the corresponding sentence in the standard subtitle file are non-identical sentences to the reference subtitle file; or trigger an alarm.

[0062] In the embodiment, in order to realize the individualized timestamp correction for different character positions, when the first character of the Nth sentence in the standard subtitle file is inconsistent with the first character of the Nth sentence in the reference subtitle file, the first timestamp of the Nth sentence in the standard subtitle file can be corrected to the first timestamp of the Nth sentence in the reference subtitle file. In this case, the algorithm can be saved by not continuing to traverse and compare other characters in the Nth sentence, but traversing and comparing characters in the (N+1)th sentence. When the last character of the Nth sentence in the standard subtitle file is inconsistent with the last character of the Nth sentence in the reference subtitle file, the last timestamp of the Nth sentence in the standard subtitle file can be corrected to the last timestamp of the Nth sentence in the reference subtitle file. When the first character and the last character of the Nth sentence in the standard subtitle file are inconsistent with the first character and the last character of the Nth sentence in the reference subtitle file, respectively, the last timestamp of the (N+1)th sentence in the reference subtitle file can be added by a preset time amount to obtain the last timestamp of the Nth sentence in the standard subtitle file, and the first timestamp of the (N-1)th sentence in the reference subtitle file can be added by a preset time amount to obtain the first timestamp of the Nth sentence in the standard subtitle file. If the (N+1)th sentence in the standard subtitle file and the reference subtitle file is completely consistent or the (N-1)th sentence is completely consistent, the timestamp correction is not performed, but an alarm is triggered to prompt manual intervention.

[0063] In some embodiments, the subtitle timestamp correction device can correct the first and last timestamps of a corresponding sentence in the reference timestamp to the first and last timestamps of the corresponding sentence in the standard timestamp when the timestamp comparison result indicates that the timestamp error of the same sentence in the reference timestamp and the corrected standard timestamp is greater than a preset threshold. The timestamp error is determined according to at least one of the first timestamp error and the last timestamp error.

[0064] In the embodiment, in order to improve the synchronization between the timestamps of the standard subtitle file and the multimedia content, the subtitle timestamp correction device can correct the first and last timestamps of the standard subtitle file to the first and last timestamps of the reference subtitle file when the timestamps of the same sentence in the standard subtitle file and the reference subtitle file are greater than a preset threshold (e.g., 0.1s), so as to improve the accuracy of the timestamps in the standard subtitle file. The timestamp error is the first timestamp error / the last timestamp error / the weighted sum of the first timestamp error and the last timestamp error.

[0065] Please refer to Figure 5The application further provides a subtitle timestamp correction device. The subtitle timestamp correction device can comprise: a subtitle file acquisition module, configured to acquire a standard subtitle file corresponding to a multimedia file and comprising reference timestamps, and a reference subtitle file corresponding to the multimedia file and comprising standard timestamps; and a timestamp correction module, configured to correct the reference timestamps in the standard subtitle file according to a text comparison result between the reference subtitle file and the standard subtitle file and the standard timestamps.

[0066] In the embodiment, the specific functions and effects of the subtitle timestamp correction device can be explained in reference to other embodiments of the application, and will not be repeated here.

[0067] Please refer to Figure 6 The application further provides a computer device, comprising a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the method as described above.

[0068] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method as described above.

[0069] The application further provides a computer program product comprising instructions, and the computer program product is executed by a processor to implement the method as described above.

[0070] The user information or user account information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, etc.) involved in the embodiments of the application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the related data need to comply with the relevant legal provisions and standards of the relevant countries and regions, and provide corresponding operation portals for the user to choose authorization or rejection.

[0071] It can be understood that the specific examples herein are only to help those skilled in the art better understand the embodiments of the application, and do not limit the scope of the application.

[0072] It can be understood that in various embodiments of the application, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0073] It can be understood that the various embodiments described in the application can be implemented alone or in combination, and the embodiments of the application do not limit this.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The use of the terms "and / or" and "at least one of" includes any and all combinations of one or more of the associated listed items. As used herein, including the claims "a," "an," and "the" preceding a term or phrase generally indicates that the term or phrase is open-ended, and thus includes one or more of elements or features. As used herein, including the claims, the term "or" as used in the "at least one of A and / or B" and "at least one of A and B" includes "at least one of A, at least one of B, or at least one of both A and B.

[0075] It can be understood that the processor in the embodiments of the application can be an integrated circuit chip with a signal processing capability. In the implementation process, the steps of the above method embodiments can be completed by hardware integrated logic circuits in the processor or by instructions in the form of software. The processor mentioned above can be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the application can be implemented or executed. The general purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the application can be directly embodied as a hardware code processor for execution, or executed by a combination of hardware and software modules in the code processor. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory, and the processor reads the information in the memory, and combines the hardware to complete the steps of the above method.

[0076] It can be understood that the memory in the embodiments of the application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM). It should be noted that the memory of the system and method described herein is intended to include but not limited to these and any other suitable type of memory.

[0077] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the information to be retrieved. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0078] It is clear to those skilled in the art that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0079] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0080] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0081] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0082] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the part of the information to be searched, which is essential or contributes to the prior art, or part of the information to be searched can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0083] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A subtitle timestamp correction method characterized by, The method comprises: obtaining a standard subtitle file corresponding to a multimedia file and containing reference timestamps, and a reference subtitle file corresponding to the multimedia file and containing standard timestamps; comparing the reference subtitle file and the standard subtitle file at a sentence granularity level to obtain a text comparison result, wherein the text comparison result indicates that there are non-identical sentences in the standard subtitle file and the reference subtitle file; determining a character position in the standard subtitle file that causes the non-identical result; in a case where the character position is at the beginning or in the middle of a sentence, correcting the beginning and end timestamps of the corresponding sentence in the standard subtitle file to the beginning and end timestamps of the corresponding sentence in the reference subtitle file according to corresponding correction logic; in a case where the character position is at the end of a sentence, correcting the end timestamp of the corresponding sentence in the standard subtitle file according to the beginning timestamp of the next sentence of the corresponding sentence in the reference subtitle file according to corresponding correction logic; in a case where the character position includes the beginning and the end of a sentence, and the previous / next sentence of the corresponding sentence in the standard subtitle file is a sentence that is identical to the reference subtitle file, correcting the beginning timestamp of the corresponding sentence in the standard subtitle file according to the end timestamp of the previous sentence of the corresponding sentence in the reference subtitle file according to corresponding correction logic, and correcting the end timestamp of the corresponding sentence in the standard subtitle file according to the beginning timestamp of the next sentence of the corresponding sentence in the reference subtitle file according to corresponding correction logic; in a case where the character position includes the beginning and the end of a sentence, and the previous and next sentences of the corresponding sentence in the standard subtitle file are sentences that are non-identical to the reference subtitle file, triggering an alarm.

2. The method of claim 1, wherein, The step of obtaining the reference subtitle file corresponding to the multimedia file and containing standard timestamps comprises: obtaining a standard audio file corresponding to the multimedia file, wherein the standard audio file has a time axis consistent with the multimedia file; extracting subtitles from the standard audio file according to the time axis in the standard audio file to obtain the reference subtitle file containing standard timestamps.

3. The method of claim 1, wherein, The text comparison result indicates that there are identical sentences in the standard subtitle file and the reference subtitle file; and the step of correcting the reference timestamps at the sentence granularity level according to the text comparison result and the standard timestamps at the sentence granularity level comprises: correcting the end timestamp of the corresponding sentence in the reference timestamps according to the end timestamp of the corresponding sentence in the reference subtitle file.

4. The method of claim 1, wherein, Further comprising: comparing the corrected reference timestamps and the standard timestamps at the sentence granularity level to obtain a timestamp comparison result; in a case where the timestamp comparison result indicates that the timestamp error of the same sentence in the corrected reference timestamps and the standard timestamps is greater than a preset threshold, correcting the beginning and end timestamps of the corresponding sentence in the reference timestamps to the beginning and end timestamps of the corresponding sentence in the standard timestamps; wherein the timestamp error is determined according to at least one of the beginning timestamp error and the end timestamp error.

5. A subtitle timestamp correction apparatus characterized by comprising: The device comprises: The subtitle file obtaining module is configured to obtain a standard subtitle file corresponding to the multimedia file and containing reference time stamps, and a reference subtitle file corresponding to the multimedia file and containing standard time stamps. The timestamp correction module is configured to compare the reference subtitle file and the standard subtitle file at a sentence granularity level to obtain a text comparison result, wherein the text comparison result indicates that there are non-identical sentences in the standard subtitle file and the reference subtitle file; determine a character position in the standard subtitle file that causes the non-identical result; in a case where the character position is a beginning of a sentence or in a sentence, correct, according to a corresponding correction logic, a beginning time stamp and an ending time stamp of a corresponding sentence in the standard subtitle file to a beginning time stamp and an ending time stamp of the corresponding sentence in the reference subtitle file; in a case where the character position is an ending of a sentence, correct, according to a corresponding correction logic, the ending time stamp of the corresponding sentence in the standard subtitle file according to a beginning time stamp of a next sentence of the corresponding sentence in the reference subtitle file; in a case where the character position includes a beginning of a sentence and an ending of a sentence, and a previous sentence / next sentence of the corresponding sentence in the standard subtitle file is a sentence identical to the reference subtitle file, correct, according to a corresponding correction logic, the beginning time stamp of the corresponding sentence in the standard subtitle file according to an ending time stamp of the previous sentence of the corresponding sentence in the reference subtitle file, and correct the ending time stamp of the corresponding sentence in the standard subtitle file according to a beginning time stamp of the next sentence of the corresponding sentence in the reference subtitle file; and in a case where the character position includes a beginning of a sentence and an ending of a sentence, and the previous sentence and the next sentence of the corresponding sentence in the standard subtitle file are sentences non-identical to the reference subtitle file, trigger an alarm.

6. A computer device, comprising: The computer device includes a memory and a processor, the memory stores at least one computer program, the at least one computer program is loaded and executed by the processor to implement the method of any one of claims 1-4.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores at least one computer program, and the at least one computer program is executed by the processor to implement the method of any one of claims 1-4.

8. A computer program product, characterised in that, The computer program product includes a plurality of program instructions, and the plurality of program instructions are executed by the processor to implement the method of any one of claims 1-4.

Images