Subtitle text processing method and apparatus, and electronic device and storage medium

Abstract

Provided in the embodiments of the present disclosure are a subtitle text processing method and apparatus, and an electronic device and a storage medium. The method comprises: loading cached subtitle data of an initial video material, wherein the cached subtitle data comprises initial subtitle text of the initial video material and corresponding initial timeline information, and the initial timeline information represents the display time of each text character in the initial subtitle text within the initial video material; in response to a first editing operation on the initial video material, generating a first video segment; and on the basis of the cached subtitle data, locally generating on a terminal device a first subtitle segment corresponding to the first video segment.

Description

Subtitle text processing method and device, electronic equipment and storage medium

[0001] The present application claims priority to the Chinese patent application No. 202410876300.X, filed on July 1, 2024, and entitled "Subtitle text processing method and device, electronic equipment and storage medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] Embodiments of the present disclosure relate to the technical field of video editing, and in particular to a subtitle text processing method and device, electronic equipment and storage medium. BACKGROUND

[0003] Currently, in the application (Applicant, APP) and platform with video editing function, it provides the function of automatically generating video subtitles for users, that is, by recognizing the voice information in the video, the corresponding video subtitles are automatically generated, which solves the problem of manual input of subtitles by users. SUMMARY

[0004] Embodiments of the present disclosure provide a subtitle text processing method and device, electronic equipment and storage medium.

[0005] In a first aspect, embodiments of the present disclosure provide a subtitle text processing method, comprising:

[0006] After displaying a video editing draft, obtaining cache subtitle data of the video editing draft, wherein an initial video material is set on a video editing track of the video editing draft, the cache subtitle data includes initial subtitle text and corresponding initial time axis information of the initial video material, the initial subtitle text is obtained by performing voice recognition on audio of the initial video material, and the initial time axis information represents display time of text characters in the initial subtitle text in the initial video material; in response to a first editing operation on the initial video material, generating a corresponding first video segment; based on the cache subtitle data, generating a first subtitle segment corresponding to the first video segment, wherein the first subtitle segment is generated by adjusting the cache subtitle data based on modification content corresponding to the first editing operation.

[0007] In a second aspect, embodiments of the present disclosure provide a subtitle text processing device, comprising:

[0008] The cache module obtains cache subtitle data of the video editing draft after the video editing draft is displayed, wherein an initial video material is arranged on a video editing track of the video editing draft, the cache subtitle data comprises initial subtitle text of the initial video material and corresponding initial time axis information, the initial subtitle text is obtained by performing speech recognition on audio of the initial video material, and the initial time axis information represents display time of text characters in the initial subtitle text in the initial video material.

[0009] The interaction module generates a corresponding first video clip in response to a first editing operation on the initial video material.

[0010] The processing module generates a first subtitle clip corresponding to the first video clip based on the cache subtitle data, wherein the first subtitle clip is generated by adjusting the cache subtitle data based on modification content corresponding to the first editing operation.

[0011] In a third aspect, an electronic device is provided, comprising a processor and a memory.

[0012] The memory stores computer execution instructions.

[0013] The processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the subtitle text processing method as described in the first aspect and various possible designs of the first aspect.

[0014] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the subtitle text processing method as described in the first aspect and various possible designs of the first aspect is implemented.

[0015] In a fifth aspect, a computer program product is provided, comprising a computer program. When a processor executes the computer program, the subtitle text processing method as described in the first aspect and various possible designs of the first aspect is implemented. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0017] FIG. 1 is a kind of application scene diagram of the subtitle text processing method provided by the present disclosure.

[0018] FIG. 2 is a flow diagram of a method for processing subtitle text according to an embodiment of the present disclosure;

[0019] FIG. 3 is a diagram illustrating a process of generating a first subtitle segment according to an embodiment of the present disclosure;

[0020] FIG. 4 is a flow diagram of a specific implementation of step S103 in the embodiment shown in FIG. 2;

[0021] FIG. 5 is a flow diagram of a specific implementation of step S1031 in the embodiment shown in FIG. 4;

[0022] FIG. 6 is a diagram illustrating a process of deleting an editing operation according to an embodiment of the present disclosure;

[0023] FIG. 7 is a flow diagram of another specific implementation of step S1031 in the embodiment shown in FIG. 4;

[0024] FIG. 8 is a diagram illustrating a process of splitting an editing operation according to an embodiment of the present disclosure;

[0025] FIG. 9 is a flow diagram of another method for processing subtitle text according to an embodiment of the present disclosure;

[0026] FIG. 10 is a flow diagram of a specific implementation of step S208 in the embodiment shown in FIG. 9;

[0027] FIG. 11 is a block diagram of a device for processing subtitle text according to an embodiment of the present disclosure;

[0028] FIG. 12 is a diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure;

[0029] FIG. 13 is a diagram illustrating a hardware structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0032] After generating the video caption, when further modification of the video is needed, such as cutting, moving, etc., since the video content has changed, the previously generated video caption needs to be re-transcribed.

[0033] The caption generation scheme in the prior art needs to be implemented based on a large speech recognition model deployed on a server, so when editing the video, the video caption needs to be repeatedly transcribed, which causes the terminal device to have limited operation and increased editing time.

[0034] Embodiments of the present disclosure provide a caption text processing method and device, an electronic device and a storage medium to overcome the problem of limited operation and increased editing time of the terminal device in the video editing process.

[0035] The application scenario of the embodiments of the present disclosure is explained as follows:

[0036] FIG. 1 is an application scenario diagram of a caption text processing method provided by an embodiment of the present disclosure. The caption text processing method provided by the present disclosure can be applied in an application program (APP) with video editing function, such as a short video application program, a video editing application program, etc. More specifically, it can be applied in the application scenario of automatic synchronization of video caption in the video editing process. The execution subject of the present embodiment can be a terminal device running the above-mentioned application program with video editing function, or a server of a server corresponding to the above-mentioned application program, or other electronic devices with similar functions.

[0037] In some embodiments, the terminal device or the server can implement the subtitle text processing method provided in the present application by running various computer-executable instructions or computer programs. For example, the computer-executable instructions can be program-level commands, machine instructions, or software instructions. The computer program can be a native program in the operating system or a software module; it can be a local application program, i.e., a program that needs to be installed in the operating system to run, or it can be a small program embedded in any APP, i.e., a program running based on a browser environment. In summary, the above computer-executable instructions can be any form of instructions, and the above computer programs can be any form of application programs, modules, or plug-ins, and the specific implementation form can be configured as needed. Further, in some embodiments, the server can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud storage, cloud communication, cloud database, cloud computing, cloud function, network service, middleware service, domain name service, security service, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platform, wherein the cloud service can be an interactive processing service for terminal devices to call.

[0038] Referring to FIG. 1, for example, after running the application program with the video editing function (hereinafter referred to as application program) described above, the terminal device displays a video editing draft in the running application program, loads video materials to be edited in the video editing track (for example, Track_1 and Track_2 shown in the figure) of the video editing draft, and then automatically or based on user operation (for example, the operation of clicking the “Generate Subtitle” button in the figure), generates the video subtitle corresponding to the video materials (the content of the video subtitle is represented by “X” in the figure). After that, the terminal device responds to the user's editing operation to edit the video materials, for example, to crop the video materials, generate a video segment in the editing page, and then the terminal device automatically deletes the subtitle segment corresponding to the cropped part of the video content from the video subtitle (represented by “X” in the figure), forms the corresponding changed video subtitle, thereby realizing the function of automatic synchronization of video subtitles.

[0039] In the prior art, after generating a video subtitle in an application, when further modification of the video is needed, such as cropping, moving, etc., since the video content has changed, in order to synchronize the video content with the video subtitle, the terminal device needs to re-transcribe the video subtitle. In the prior art, the process of generating a subtitle needs to send video data (or audio data in the video data) to a server, and the speech recognition model deployed on the server is used to complete the transcription. Therefore, in one implementation, whenever the content of the video material on the video editing track changes, the terminal device sends the video data after the content changes to the server for processing. At the same time, on the terminal device side, the corresponding processing progress is displayed, and related operations are locked to prevent transcription errors. In this implementation, since the video data needs to be sent to the server for processing, it inevitably leads to an increase in processing time during editing, limited operations, and other problems, affecting the video editing efficiency. In another implementation, after editing the video, the content of the video subtitle is changed based on user triggering (such as re-clicking the "generate subtitle" button) to match the video content. This method can reduce the processing time and improve the video editing efficiency, but it may cause temporary inconsistency between the subtitle and the video content, affecting the video editing effect and user experience.

[0040] The embodiment of the present disclosure provides a subtitle text processing method to solve the above problems.

[0041] Referring to FIG. 2, FIG. 2 is a flowchart of a subtitle text processing method according to an embodiment of the present disclosure. The method of the present embodiment can be applied in a terminal device, and the subtitle text processing method comprises the following steps:

[0042] In step S101, after displaying a video editing draft, the cached subtitle data of the video editing draft is obtained, wherein the video editing track of the video editing draft is provided with an initial video material, the cached subtitle data includes initial subtitle text of the initial video material and corresponding initial time axis information, the initial subtitle text is obtained by performing speech recognition on the audio of the initial video material, and the initial time axis information represents the display time of the text characters in the initial subtitle text in the initial video material.

[0043] Exemplarily, referring to the application scenario diagram shown in FIG. 1, after a terminal device runs an application program, the terminal device enters a video editing interface through creating, loading a video draft and the like, and then loads a video material, i.e., an initial video material, for subsequent editing on a video editing track. The initial video material is a piece of video, which can be an original video material that is not processed after being shot, or a video material that is edited, and is not limited herein. Then, in a possible implementation, referring to the manner in which the application scenario diagram shown in FIG. 1, the terminal device identifies the initial video material in an automatic or manual manner through the application program, to obtain corresponding subtitle data. The step of identifying the initial video material can be completed by sending the initial video material to a server, and by a speech recognition model deployed on the server. Further, the subtitle data includes initial subtitle text of the initial video material and corresponding initial time axis information, the initial time axis information representing display time of each text character in the initial subtitle text in the initial video material. Then, the terminal device caches the subtitle data locally or loads the subtitle data into a memory, to form cached subtitle data, i.e., the step of loading the cached subtitle data of the initial video material. In another possible implementation, the cached subtitle data is pre-generated data corresponding to the initial video material, which can be pre-stored locally on the terminal device or on a server. After the terminal device loads the initial video material, the pre-generated cached subtitle data is downloaded locally or loaded into the memory, i.e., the step of loading the cached subtitle data of the initial video material is completed. At the same time, regardless of the implementation, the terminal device obtains the initial subtitle text corresponding to the initial video material. In a possible implementation, the terminal device displays the initial subtitle text in the video editing interface.

[0044] Further, the cached subtitle data loaded by the terminal device in the step is equivalent to caching of the initial subtitle text of the initial video material, and in combination with the data format unique to the cached subtitle data provided in this embodiment, the terminal device changes the initial subtitle text locally in a subsequent step, to generate subtitle text (a first subtitle segment) that matches the content of the changed video material (a first video segment).

[0045] Step S102: In response to a first editing operation on the initial video material, a corresponding first video segment is generated.

[0046] Step S103: Based on the cached subtitle data, a first subtitle segment corresponding to the first video segment is generated, wherein the first subtitle segment is generated by adjusting the cached subtitle data based on modification content corresponding to the first editing operation.

[0047] Further, after the terminal device loads the cached subtitle data, the user applies a first editing operation to the terminal device (video editing interface) according to specific editing requirements, to edit the initial video material, and further generate a first video segment. For example, by applying the first editing operation, a specific segment in the initial video material is deleted (the first video segment is a video material with a shorter time length), the initial video material is split and moved (the first video segment is at least two video materials with a shorter time length), and the like. Subsequently, as the video content of the initial video material changes, the corresponding subtitle content of the initial subtitle text also changes, that is, a first subtitle segment corresponding to the first video segment is generated, and the first subtitle segment is displayed in the video editing interface of the terminal device. For example, the first editing operation is a video cropping operation, and by the first editing operation, the initial video material is cropped into a first video segment with a shorter length. Correspondingly, the characters in the initial subtitle text corresponding to the cropped part are also deleted or marked, forming the first subtitle segment.

[0048] FIG. 3 is a schematic diagram of generating a first subtitle segment according to an embodiment of the present disclosure. As shown in FIG. 3, in the video editing interface, a video preview area and a subtitle area are included, wherein the video preview area is used to display the selected video material, and the subtitle area is used to display the subtitle text corresponding to the video material. First, after the terminal device loads the initial video material on the video editing track Track_1, the initial video material is displayed in the video preview area, for example, the initial video material is played or a certain video frame of the initial video material is displayed; and the corresponding initial subtitle text is displayed in the subtitle area. Specifically, for example, as shown in the figure, the initial video material is a video clip with a time length of 20 seconds, and the initial subtitle text corresponding thereto includes [word_1], [word_2] … [word_200] a total of 200 text characters. Subsequently, in response to the first editing operation on the initial video material, the terminal device crops the initial video material from the 2nd second to the 10th second (a total of 8 seconds) to generate a video clip with a time length of 12 seconds, that is, a first video segment. Correspondingly, the terminal device uses the cached subtitle data to generate a corresponding first subtitle segment, and the first subtitle segment includes [word_1] to [word_60], [word_161] to [word_200] a total of 100 text characters, that is, the text characters [word_61] to [word_160] corresponding to the segment content of the initial video material from the 2nd second to the 10th second are deleted.

[0049] Further, the first subtitle segment is generated based on the cached subtitle data. Specifically, the cached subtitle data includes initial subtitle text of the initial video material and corresponding initial time axis information. The initial subtitle text is an ordered set of text characters, such as a series of ordered Chinese characters, numbers, etc. Further, in one specific implementation, the initial time axis information can be a set of data pairs, such as [t1, t2], representing the start time and end data of a text character. Through the initial time axis information, the display time of each text character in the initial subtitle text can be described. In another possible implementation, the data pairs in the initial time axis information represent the offset value and duration value corresponding to the text character, respectively. The offset value represents the start timestamp of the speech segment corresponding to the text character in the initial video material, and the duration value represents the duration of the speech segment corresponding to the text character in the initial video material. Through this data format, the difference in the rate between segments can be smoothed, which can be applied to the modification and restoration of the subtitle text in the variable speed playback scenario, so that the cached subtitle data has higher usability.

[0050] Further, based on the cached subtitle data with the special data format, the display time corresponding to each text character can be obtained. Then, the display time of the modified content corresponding to the initial video material is mapped to the cached subtitle data, and the corresponding target cached information is obtained from the cached subtitle data. Finally, the first subtitle segment is generated based on the target cached information. Specifically, as shown in FIG. 4, the specific implementation of step S103 includes:

[0051] Step S1031: determining target cached information corresponding to the first video segment based on the cached subtitle data. The target cached information is used to represent the display time of the target text character. The target text character is the text character corresponding to the modified content triggered by the first editing operation in the initial video material.

[0052] Step S1032: generating the first subtitle segment corresponding to the first video segment locally in the terminal device according to the target cached information.

[0053] Based on the previous description, the target cached information is used to indicate the display time of the target text character corresponding to the modified content in the initial subtitle text, i.e., the display time of the text character corresponding to the modified content triggered by the editing operation in the initial video material. Then, a copy of the initial subtitle text can be created based on the cached subtitle data, and the corresponding target text character in the copy can be located and modified based on the target cached information, so as to generate the first subtitle segment.

[0054] Further, the above-mentioned manner of generating the first subtitle segment will be described in more detail in combination with a specific implementation manner of the first editing operation. In a possible implementation manner, the first editing operation includes a deletion editing operation, and the deletion editing operation is used to delete a target material segment of the initial video material. Correspondingly, as shown in FIG. 5, the specific implementation manner of step S1031 includes:

[0055] Step S1031A-1: Obtain a first display time period of a target material segment corresponding to the deletion editing operation.

[0056] Step S1031A-2: Obtain at least one target text character corresponding to the first display time period from the cached subtitle data.

[0057] Step S1031A-3: Generate target cache information according to a second display time period corresponding to the at least one target text character.

[0058] FIG. 6 is a schematic diagram of a deletion editing operation process provided by an embodiment of the present disclosure. The above-mentioned process will be described in combination with FIG. 6. As shown in FIG. 6, first, based on the deletion editing operation, a time period [T1:T2] corresponding to a target material segment in the initial video material is determined, that is, a first display time period. Then, based on the first display time period, a mapping query is performed on the cached subtitle data, to obtain target text characters covered by the first display time period, for example, text characters A, B and C shown in the figure, that is, target text characters. Then, the cached subtitle data is used to obtain a display time period corresponding to each of the text characters A, B and C, that is, a second display time period, for example, [t1, t2], [t3, t4] and [t5, t6]. Then, target cache information is generated based on the second display time period. In a subsequent step, according to the second display time period [t1, t2], [t3, t4] and [t5, t6] indicated by the target cache information, the target text characters A, B and C in the copy of the initial subtitle text are processed. Exemplarily, the specific processing manner includes: according to the target cache information, the target text characters in the initial subtitle text are processed to generate a mark character, the target text characters in the initial subtitle text are replaced by the mark character, to generate a first subtitle segment and corresponding timeline information. For example, as shown in the figure, the mark character can be a deletion character (that is, A, B and C shown in the figure) indicating that the deletion is completed. Through the mark character manner provided by the embodiment of the present disclosure, the deletion effect is indicated, and the content of the deleted target text character is retained, thereby facilitating subsequent recovery of the deleted target text character.

[0059] Further, while modifying the text content of the initial subtitle text, the timeline information corresponding to the initial subtitle text is modified through the second display time period described above, so as to generate the first subtitle segment corresponding to the first video segment, and generate the changed timeline information corresponding to the first subtitle segment, the changed timeline information representing the display time of the text characters in the first subtitle segment in the video segment. Since this process can be completed only by using the cached subtitle data cached locally in the terminal device, it is not necessary to send the audio data corresponding to the first changed video material to the server, thereby realizing the rapid and real-time generation of the first subtitle segment.

[0060] In another possible implementation, the first editing operation includes a split editing operation, and the split editing operation is used to split the initial video material into at least two video segments; correspondingly, as shown in FIG. 7, the specific implementation of step S1031 includes:

[0061] Step S1031B-1: obtaining the display time interval corresponding to each video segment.

[0062] Step S1031B-2: based on each display time interval, obtaining the corresponding text segment from the cached subtitle data.

[0063] Step S1031B-3: generating target cache information according to the third display time period corresponding to the text characters in each text segment.

[0064] FIG. 8 is a schematic diagram of a process of segment editing operation according to an embodiment of the present disclosure. The process will be described below in conjunction with FIG. 8. As shown in FIG. 8, first, based on the segment editing operation, the terminal device divides the initial video material P in the video editing track #1 into a video segment P1 and a video segment P2, and obtains the display time interval [T1:T2] and [T3:T4] corresponding to the video segment P1 and the video segment P2 respectively, where T2 and T3 can be the timestamps of adjacent frames. Then, based on the display time interval, the corresponding text segment Text_1 and text segment Text_2 are obtained by mapping and querying the cache subtitle data. Then, according to the third display time period corresponding to the text characters in each text segment, for example, the display time period [t1, t2], [t3, t4] corresponding to the text characters A, B in the text segment Text_1, the display time period [t5, t6], [t7, t8] corresponding to the text characters C, D in the text segment Text_2, the target cache information is generated. In the subsequent step, according to the third display time period [t1, t2], [t3, t4], [t5, t6], [t7, t8] indicated by the target cache information, the copy of the initial subtitle text is processed to form two pieces of subtitle text corresponding to the video segment P1 and the video segment P2 respectively, that is, the first subtitle segment, and the time axis information corresponding to the two pieces of subtitle text in the first subtitle segment is changed.

[0065] In yet another possible implementation, the first editing operation further comprises a moving editing operation, the moving editing operation being used to adjust a position of the initial video material in the video editing track and / or adjust a video editing track in which the initial video material is located, for example, before responding to the moving editing operation, the first initial video material is located in the video editing track #1 within the video editing interface, after responding to the moving editing operation, the first initial video material is moved to the video editing track #2. For another example, before responding to the moving editing operation, the first initial video material is located in a display time interval [T1:T2] in the video editing track #1 within the video editing interface, after responding to the moving editing operation, the first initial video material is moved to a display time interval [T3:T4] in the video editing track #1 or a display time interval [T3:T4] in the video editing track #2. In this case, since no content change occurs to the initial video material, the specific implementation of step S1031 comprises: determining, as the target cache information, display times corresponding to all text characters in the cache subtitle data. Correspondingly, after the position of the initial video material in the video editing track and / or the video editing track is changed in response to the moving editing operation, the terminal device generates the changed timeline information corresponding to the first subtitle clip according to the cache subtitle data, the changed timeline information representing display times of the text characters in the first subtitle clip in the first video clip, thereby realizing synchronization of the timeline information of the first subtitle clip after the moving editing operation. In the step of the embodiment, the synchronization of the subtitle text is realized by first obtaining the target cache information from the cache subtitle data and then generating the first subtitle clip based on the target cache information. Since the target cache information can be obtained from the cache subtitle data, the process does not need to access the server and does not need to re-identify the video material, thereby reducing the time consumption caused by communication with the server, data transmission and identification processing of the video material, and further realizing the purpose of providing the speed of video subtitle synchronization.

[0066] In the embodiment, the cache subtitle data of the initial video material is loaded, the cache subtitle data comprising initial subtitle text of the initial video material and corresponding initial timeline information, the initial timeline information representing display times of each text character in the initial subtitle text in the initial video material; the first video clip is generated in response to the first editing operation on the initial video material; and the first subtitle clip corresponding to the first video clip is generated locally at the terminal device based on the cache subtitle data. By loading the cache subtitle data corresponding to the initial video material and generating the corresponding first subtitle clip by modifying the corresponding initial subtitle text after the initial video material is modified, the corresponding subtitle text is obtained without re-identifying the modified first video clip, thereby improving the speed of generating video subtitles, reducing the time consumption of accessing the server, and improving the efficiency of video editing.

[0067] Referring to FIG. 9, FIG. 9 is a flowchart of a subtitle text processing method according to an embodiment of the present disclosure. The embodiment further includes the steps of generating cache subtitle data and recovering an editing operation on the basis of the embodiment shown in FIG. 2. The subtitle text processing method includes the following steps:

[0068] Step S201: Perform speech recognition on the initial video material to generate initial subtitle text.

[0069] Step S202: Obtain positioning information and playing time information of each text character in the initial subtitle text, wherein the positioning information includes a paragraph identifier and an intra-paragraph serial number of a paragraph to which the text character belongs, and the playing time information includes a start timestamp and an end timestamp of a corresponding speech segment of the text character in the initial video material.

[0070] Step S203: Generate cache subtitle data according to the positioning information and the playing time information of each text character in the initial subtitle text.

[0071] By way of example, in the embodiment, after the terminal device loads the initial video material, the terminal device generates initial subtitle text by performing speech recognition on the initial video material. This step can be implemented by deploying a small speech recognition model locally on the terminal device, or by sending the initial video material to a server and deploying a large speech recognition model with better performance on the server. The selection can be made according to requirements. Then, the terminal device obtains the recognition result of speech recognition on the initial video material, i.e., subtitle data containing the initial subtitle text. Then, the terminal device obtains the positioning information and the playing time information of each text character in the initial subtitle text from the subtitle data, wherein the positioning information includes a paragraph identifier and an intra-paragraph serial number of a paragraph to which the text character belongs. For example, the playing time information of text character A is [3, 2], indicating that text character A is in the 3rd row and the 2nd character in the initial subtitle text. The playing time information includes a start timestamp and an end timestamp of a corresponding speech segment of the text character in the initial video material. For example, the playing time information of text character B is [t1, t2], indicating that the start time of the speech segment corresponding to text character B is t1 and the end time is t2. That is, the positioning information is used to indicate the text position of each text character in the initial subtitle text, and the playing time information is used to indicate the pronunciation position of each text character in the initial video material.

[0072] Afterwards, the cache subtitle data is generated according to the positioning information and the playing time information, wherein the positioning information is used to indicate the text characters in the cache subtitle data, which is equivalent to the identification of the text characters in the cache subtitle data; and the playing time information is used to indicate the display time of the text characters in the cache subtitle data. Based on the introduction in the embodiment shown in FIG. 2, in a possible implementation manner, the cache subtitle data contains the offset value and the duration value corresponding to each text character in the initial subtitle text, and therefore, at least the playing time information needs to be converted in the process of generating the cache subtitle data. For example, the playing time information of the text character A is [t1, t2], and after the conversion of the text character A, the offset value and the duration value corresponding to the text character are generated, that is, [t0+det_1, dur_1], wherein t0 is the start time, which is 0 by default, det_1 is the offset value, which is calculated by the start time stamp t1 of the text character A; and dur_1 is the duration value, which is obtained by the difference between the start time stamp t1 and the end time stamp t2 of the text character A.

[0073] Step S204: After displaying the video editing draft, the cache subtitle data of the video editing draft is obtained, wherein the video editing track of the video editing draft is provided with the initial video material, the cache subtitle data includes the initial subtitle text of the initial video material and the corresponding initial time axis information, the initial subtitle text is obtained by performing speech recognition on the audio of the initial video material, and the initial time axis information represents the display time of the text characters in the initial subtitle text in the initial video material.

[0074] Step S205: In response to the first editing operation on the initial video material, the corresponding first video segment is generated.

[0075] Step S206: Based on the cache subtitle data, the first subtitle segment corresponding to the first video segment is generated, wherein the first subtitle segment is generated by adjusting the cache subtitle data based on the modification content corresponding to the first editing operation.

[0076] After the initial video material is generated, the terminal device loads the initial video material, and then responds to the first editing operation of the user to edit the initial video material, and generates the first subtitle segment corresponding to the first video segment locally in the terminal device. The specific implementation manner of the above steps has been introduced in the embodiment shown in FIG. 2, and will not be described herein.

[0077] Step S207: In response to the second editing operation on the first video segment, the second video segment is generated, wherein the second video segment is the processing result of the initial video material after at most N-1 sub-operations, and the second editing operation is a cancel operation of at least one sub-operation.

[0078] Step S208: based on the cached subtitle data, generating a second subtitle segment corresponding to the second video segment locally at the terminal device.

[0079] Further, after generating the first subtitle segment by responding to the first editing operation, it is equivalent to that the user performs an editing operation on the initial video material once, and then the user may further edit and adjust the video material (the first video segment) according to specific needs, that is, to apply a second editing operation, and the terminal device further generates a second video segment after responding to the second editing operation. In a possible implementation, the second editing operation is similar to the implementation of the first editing operation, for example, further segmentation, moving, clipping and the like of the first video segment, and the terminal device further generates a second subtitle segment corresponding to the second video segment locally at the terminal device based on the cached subtitle data after responding to the first editing operation. In this case, the implementation is similar to the implementation steps of responding to the first editing operation to generate the first subtitle segment as described in the foregoing embodiments, and will not be described herein.

[0080] In another possible implementation, the second editing operation is a cancel operation, that is, to cancel part (N) or all of the modified content in the first video segment triggered by the first editing operation, that is, the processing result after N-1 sub-operations, where N is an integer greater than or equal to 2. For example, the first editing operation includes a plurality of sub-operation steps, and specifically, for example, the first editing operation includes three sub-operation steps of segmentation, moving and clipping of the initial video material, and the terminal device randomly cancels the moving operation by responding to the second editing operation, that is, the cancel operation, while retaining the modified content generated by the segmentation operation and the clipping operation. Then, with the execution of the second editing operation, the terminal device processes the first subtitle segment based on the cached subtitle data to make the content of the first subtitle segment match the second video segment generated after the cancellation. That is, to restore the changes of the subtitle text generated by the target sub-operation in the first editing operation indicated by the second editing operation.

[0081] Compared with the prior art, in the embodiment, the random restoration of the subtitle text can be implemented based on the cached subtitle data, without the sequential restoration based on the draft cancel stack, thereby further improving the efficiency of the synchronous generation and adjustment of the subtitle text. Specifically, in a possible implementation, the cached subtitle data includes operation records corresponding to each sub-operation, and the operation record is used to represent the modified content of the initial video material. For example, as shown in FIG. 10, the specific implementation of step S208 includes:

[0082] Step S2081: obtaining the operation record corresponding to the sub-operation based on the cached subtitle data.

[0083] Step S2082: Based on the operation record, determine the target cache information corresponding to the second video clip.

[0084] Step S2083: According to the target cache information, generate the second subtitle clip corresponding to the second video clip locally at the terminal device.

[0085] Exemplarily, first, after receiving the second editing operation, the terminal device obtains, from the cached subtitle data, an operation record corresponding to a target sub-operation in the first editing operation indicated by the second editing operation, which needs to be recovered, the operation record containing a corresponding operation target and operation content, wherein the operation target can be represented by a paragraph identifier and an intra-paragraph serial number, and the operation content includes an editing operation type, such as a deletion editing operation, a splitting editing operation, a moving editing operation, etc., and for different types of operation types, it can also include corresponding operation parameters, such as the operation parameters of the moving editing operation, including the positions of the video material in the video editing track before and after moving, and / or the track identifier of the video editing track where the video material is located before and after moving, etc. Then, according to the above operation record, the target cache information corresponding to the second video clip is obtained from the cached subtitle data, wherein the target cache information is used to represent the display time of the target text character, and the target text character is the text character corresponding to the modification content triggered by the target sub-operation. The target cache information is similar to the target cache information corresponding to the first editing operation in the embodiment shown in FIG. 2, and reference can be made to the introduction of the target cache information in the embodiment shown in FIG. 2. After that, based on the target cache information, the first subtitle clip is modified, thereby generating the second subtitle clip corresponding to the second video clip locally at the terminal device, and the corresponding timeline information is also modified, which will not be described again.

[0086] In the above embodiment steps in this embodiment, through the initial subtitle text and the corresponding initial timeline information of the initial video material recorded in the cached subtitle data, and the operation record corresponding to each sub-operation, random recovery of multiple sub-operations is realized, without the need for sequential recovery based on the draft undo stack, thereby further improving the efficiency of synchronous generation and adjustment of the subtitle text.

[0087] In this embodiment, the implementation manners of steps S204 to S206 are the same as those of steps S101 to S103 in the embodiment shown in FIG. 2 of the present disclosure, which will not be described again.

[0088] Corresponding to the subtitle text processing method of the above embodiment, FIG. 11 is a structural block diagram of a subtitle text processing apparatus provided by an embodiment of the present disclosure. The method introduced in the above embodiment can be executed by the subtitle text processing apparatus. The apparatus can be implemented in the form of software and / or hardware. The apparatus can be integrated in an electronic device having a certain data processing function. The electronic device can include, but is not limited to, a mobile terminal having a large data processing capability, and a desktop computer, a supercomputer, and other fixed terminals having a large data processing capability.

[0089] For ease of illustration, only parts related to the embodiments of the present disclosure are shown. Referring to FIG. 11, the subtitle text processing apparatus 3 includes:

[0090] The cache module 31 loads the cached subtitle data of the initial video material. The cached subtitle data includes the initial subtitle text of the initial video material and corresponding initial time axis information. The initial time axis information represents the display time of each text character in the initial subtitle text in the initial video material.

[0091] The interaction module 32 is configured to generate a first video segment in response to a first editing operation on the initial video material.

[0092] The processing module 33 is configured to generate a first subtitle segment corresponding to the first video segment locally in the terminal device based on the cached subtitle data.

[0093] According to one or more embodiments of the present disclosure, the cached subtitle data includes an offset value and a duration value corresponding to each text character in the initial subtitle text. The offset value represents the starting timestamp of the speech segment corresponding to the text character in the initial video material, and the duration value represents the duration of the speech segment corresponding to the text character in the initial video material.

[0094] According to one or more embodiments of the present disclosure, the cache module 31 is further configured to perform speech recognition on the initial video material to generate the initial subtitle text, obtain the positioning information and the play time information of each text character in the initial subtitle text, wherein the positioning information includes the paragraph identifier of the paragraph to which the text character belongs and the serial number in the paragraph, and the play time information includes the starting timestamp and the ending timestamp of the speech segment corresponding to the text character in the initial video material, and generate the cached subtitle data according to the positioning information and the play time information of each text character in the initial subtitle text.

[0095] According to one or more embodiments of the present disclosure, the processing module 33 is specifically configured to: determine target cache information corresponding to the first video segment based on the cached subtitle data, the target cache information being used to represent display time corresponding to target text characters, the target text characters being text characters corresponding to modified content in the initial video material triggered based on the first editing operation; and generate a first subtitle segment corresponding to the first video segment locally at the terminal device according to the target cache information.

[0096] According to one or more embodiments of the present disclosure, the first editing operation includes a deletion editing operation, the deletion editing operation being used to delete a target material segment of the initial video material; and when the processing module 33 determines the target cache information corresponding to the first video segment based on the cached subtitle data, the processing module 33 is specifically configured to: obtain a first display time period of the target material segment corresponding to the deletion editing operation; obtain at least one target text character corresponding to the first display time period from the cached subtitle data; and generate the target cache information according to a second display time period corresponding to the at least one target text character.

[0097] According to one or more embodiments of the present disclosure, when the processing module 33 generates the first subtitle segment corresponding to the first video segment locally at the terminal device according to the target cache information, the processing module 33 is specifically configured to: process the target text characters in the initial subtitle text according to the target cache information to generate a mark character; and replace the target text characters in the initial subtitle text with the mark character to generate the first subtitle segment.

[0098] According to one or more embodiments of the present disclosure, the first editing operation includes a moving editing operation, the moving editing operation being used to adjust a position of the initial video material in a video editing track and / or adjust a video editing track in which the initial video material is located; and the processing module 33 is further configured to: generate change timeline information corresponding to the first subtitle segment according to the cached subtitle data, the change timeline information representing display time of text characters in the first subtitle segment in the video segment.

[0099] According to one or more embodiments of the present disclosure, the first editing operation includes a splitting editing operation, the splitting editing operation being used to split the initial video material into at least two video segments; and when the processing module 33 determines the target cache information corresponding to the first video segment based on the cached subtitle data, the processing module 33 is specifically configured to: obtain a display time interval corresponding to each video segment; obtain a text segment corresponding to each display time interval from the cached subtitle data; and generate the target cache information according to a third display time period corresponding to text characters in each text segment.

[0100] According to one or more embodiments of the present disclosure, the first editing operation includes N sub-operations, and the interaction module 32 is further configured to: in response to a second editing operation on the first video segment, generate a second video segment, where the second video segment is a processing result of the initial video material after at most N-1 sub-operations, and the second editing operation is a cancel operation of at least one sub-operation; and the processing module 33 is further configured to: based on the cached subtitle data, locally generate a second subtitle segment corresponding to the second video segment at the terminal device.

[0101] According to one or more embodiments of the present disclosure, the operation record corresponding to each sub-operation is included in the cached subtitle data, and the operation record is used to represent the modification content on the initial video material; and when the processing module 33 locally generates the second subtitle segment corresponding to the second video segment at the terminal device based on the cached subtitle data, the processing module 33 is specifically configured to: based on the cached subtitle data, obtain the operation record corresponding to the sub-operation; based on the operation record, determine target cached information corresponding to the second video segment; and according to the target cached information, locally generate the second subtitle segment corresponding to the second video segment at the terminal device.

[0102] The cache module 31, the interaction module 32, and the processing module 33 are connected in sequence. The subtitle text processing apparatus 3 provided in this embodiment can execute the technical solutions of the above method embodiments, and has similar implementation principles and technical effects, which will not be described here again.

[0103] FIG. 12 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 12, the electronic device 4 includes:

[0104] a processor 41, and a memory 42 connected with the processor 41 in communication;

[0105] The memory 42 stores computer execution instructions.

[0106] The processor 41 executes the computer execution instructions stored in the memory 42, so as to implement the subtitle text processing method in the embodiments shown in FIGS. 2-10.

[0107] Optionally, the processor 41 and the memory 42 are connected through a bus 43.

[0108] The related description can be understood by referring to the related description and effects of the steps in the embodiments corresponding to FIGS. 2-10, which will not be described here again.

[0109] The present disclosure provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. When the computer execution instructions are executed by a processor, the computer execution instructions are used to implement the subtitle text processing method provided in any one of the embodiments corresponding to FIGS. 2-10 of the present disclosure.

[0110] The embodiment of the present disclosure provides a computer program product, comprising a computer program, wherein the computer program is executed by a processor to implement the subtitle text processing method provided in any one of the embodiments corresponding to FIG. 2 to FIG. 10.

[0111] In order to implement the above-mentioned embodiments, the embodiment of the present disclosure further provides an electronic device.

[0112] Referring to FIG. 13, a structural diagram of an electronic device 900 suitable for implementing the embodiments of the present disclosure is shown, which can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDA), tablet computers (PAD), portable multimedia players (PMP), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in FIG. 13 is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0113] As shown in FIG. 13, the electronic device 900 can include a processing device (such as a central processor, a graphics processor, etc.) 901, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 902 or loaded into a random access memory (RAM) 903 from a storage device 908. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0114] In general, the following devices can be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 907 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 908 including, for example, a tape, a hard disk, and the like; and communication devices 909. The communication devices 909 can allow the electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. While FIG. 13 shows the electronic device 900 with various devices, it is understood that all of the illustrated devices are not required to be implemented or present. More or less devices can alternatively be implemented or present.

[0115] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 909, or installed from the storage devices 908, or installed from the ROM 902. When the computer program is executed by the processing devices 901, the above-described functions defined in the methods of the embodiments of the present disclosure are performed.

[0116] It should be noted that the computer-readable medium in the above disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0117] The computer-readable medium described above can be contained in the electronic device described above; or can exist separately and not be assembled into the electronic device.

[0118] The computer-readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0119] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0120] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block of the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or by combinations of dedicated hardware and computer instructions.

[0121] The units or modules involved in the embodiments of the present disclosure can be implemented by software or by hardware. In some cases, the name of the unit or module does not constitute a limitation on the unit itself.

[0122] The functions described in the foregoing description can be implemented in part or in whole by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0123] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0124] In a first aspect, according to one or more embodiments of the present disclosure, a subtitle text processing method is provided, comprising:

[0125] After displaying the video editing draft, cache subtitle data of the video editing draft is obtained, wherein initial video materials are arranged on a video editing track of the video editing draft, the cache subtitle data comprises initial subtitle text and corresponding initial time axis information of the initial video materials, the initial subtitle text is obtained by performing speech recognition on audio of the initial video materials, and the initial time axis information represents display time of text characters in the initial subtitle text in the initial video materials; in response to a first editing operation on the initial video materials, a corresponding first video segment is generated; and based on the cache subtitle data, a first subtitle segment corresponding to the first video segment is generated, wherein the first subtitle segment is generated by adjusting the cache subtitle data based on modification content corresponding to the first editing operation.

[0126] According to one or more embodiments of the present disclosure, the cache subtitle data contains an offset value and a duration value corresponding to each text character in the initial subtitle text, wherein the offset value represents a starting timestamp of a speech segment corresponding to the text character in the initial video materials, and the duration value represents a duration of the speech segment corresponding to the text character in the initial video materials.

[0127] According to one or more embodiments of the present disclosure, the method further comprises: performing speech recognition on the initial video material to generate initial subtitle text; obtaining positioning information and playing time information of each text character in the initial subtitle text, wherein the positioning information comprises a paragraph identifier and an intra-paragraph serial number of a paragraph to which the text character belongs, and the playing time information comprises a start timestamp and an end timestamp of a corresponding speech segment in the initial video material; and generating the cache subtitle data according to the positioning information and the playing time information of each text character in the initial subtitle text.

[0128] According to one or more embodiments of the present disclosure, the generating, based on the cache subtitle data, of the first subtitle segment corresponding to the first video segment at the terminal device comprises: obtaining target cache information corresponding to the first video segment based on the cache subtitle data, the target cache information being used to represent display time of target text characters corresponding to modified content in the initial video material triggered based on the first editing operation; and generating the first subtitle segment corresponding to the first video segment at the terminal device according to the target cache information.

[0129] According to one or more embodiments of the present disclosure, the first editing operation comprises a deletion editing operation used to delete a target material segment of the initial video material; and the obtaining, based on the cache subtitle data, of the target cache information corresponding to the first video segment comprises: obtaining a first display time period of the target material segment corresponding to the deletion editing operation; obtaining at least one target text character corresponding to the cache subtitle data according to the first display time period; and generating the target cache information according to a second display time period corresponding to the at least one target text character.

[0130] According to one or more embodiments of the present disclosure, the generating, based on the target cache information, of the first subtitle segment corresponding to the first video segment at the terminal device comprises: processing target text characters in the initial subtitle text according to the target cache information to generate mark characters; and replacing the target text characters in the initial subtitle text with the mark characters to generate the first subtitle segment.

[0131] According to one or more embodiments of the present disclosure, the first editing operation comprises a moving editing operation, the moving editing operation being used to adjust a position of the initial video material in a video editing track and / or adjust a video editing track in which the initial video material is located; and the method further comprises: generating, according to the cached subtitle data, change timeline information corresponding to the first subtitle segment, the change timeline information representing display time of text characters in the first subtitle segment in the video segment.

[0132] According to one or more embodiments of the present disclosure, the first editing operation comprises a splitting editing operation, the splitting editing operation being used to split the initial video material into at least two video segments; and the obtaining, based on the cached subtitle data, of the target cached information corresponding to the first video segment comprises: acquiring display time intervals corresponding to the video segments; based on the display time intervals, acquiring corresponding text segments from the cached subtitle data; and generating the target cached information according to third display time periods corresponding to text characters in the text segments.

[0133] According to one or more embodiments of the present disclosure, the first editing operation comprises N sub-operations, and the method further comprises: in response to a second editing operation for the first video segment, generating a second video segment, wherein the second video segment is a processing result of the initial video material after at most N-1 sub-operations, and the second editing operation is a cancel operation of at least one sub-operation; and based on the cached subtitle data, generating, locally at the terminal device, a second subtitle segment corresponding to the second video segment.

[0134] According to one or more embodiments of the present disclosure, the cached subtitle data comprises operation records corresponding to the sub-operations, the operation records being used to represent modification contents for the initial video material; and the generating, locally at the terminal device, of the second subtitle segment corresponding to the second video segment based on the cached subtitle data comprises: based on the cached subtitle data, acquiring operation records corresponding to the sub-operations; based on the operation records, obtaining target cached information corresponding to the second video segment; and according to the target cached information, generating, locally at the terminal device, the second subtitle segment corresponding to the second video segment.

[0135] In a second aspect, according to one or more embodiments of the present disclosure, a subtitle text processing apparatus is provided, comprising:

[0136] The cache module is configured to obtain cache subtitle data of the video editing draft after the video editing draft is displayed, wherein the video editing track of the video editing draft is provided with an initial video material, the cache subtitle data comprises initial subtitle text and corresponding initial time axis information of the initial video material, the initial subtitle text is obtained by performing voice recognition on audio of the initial video material, and the initial time axis information represents display time of a text character in the initial subtitle text in the initial video material.

[0137] The interaction module is configured to generate a corresponding first video clip in response to a first editing operation on the initial video material.

[0138] The processing module is configured to generate a first subtitle clip corresponding to the first video clip based on the cache subtitle data, wherein the first subtitle clip is generated by adjusting the cache subtitle data based on modification content corresponding to the first editing operation.

[0139] According to one or more embodiments of the present disclosure, the cache subtitle data comprises an offset value and a duration value corresponding to each text character in the initial subtitle text, wherein the offset value represents a starting timestamp of a voice clip corresponding to the text character in the initial video material, and the duration value represents a duration of the voice clip corresponding to the text character in the initial video material.

[0140] According to one or more embodiments of the present disclosure, the cache module is further configured to perform voice recognition on the initial video material to generate initial subtitle text, obtain positioning information and play time information of each text character in the initial subtitle text, wherein the positioning information comprises a paragraph identifier and a paragraph serial number of a paragraph to which the text character belongs, and the play time information comprises a starting timestamp and an ending timestamp of a voice clip corresponding to the text character in the initial video material, and generate the cache subtitle data based on the positioning information and the play time information of each text character in the initial subtitle text.

[0141] According to one or more embodiments of the present disclosure, the processing module is specifically configured to obtain target cache information corresponding to the first video clip based on the cache subtitle data, the target cache information is used to represent display time of a target text character, the target text character is a text character corresponding to modification content triggered based on the first editing operation in the initial video material, and generate the first subtitle clip corresponding to the first video clip locally in the terminal device based on the target cache information.

[0142] According to one or more embodiments of the present disclosure, the first editing operation comprises a deletion editing operation, and the deletion editing operation is used to delete a target material segment of the initial video material; when the processing module obtains the target cache information corresponding to the first video segment based on the cache subtitle data, the processing module is specifically configured to: obtain a first display time period of a target material segment corresponding to the deletion editing operation; obtain at least one target text character corresponding to the first display time period from the cache subtitle data; and generate the target cache information according to a second display time period corresponding to the at least one target text character.

[0143] According to one or more embodiments of the present disclosure, when the processing module generates the first subtitle segment corresponding to the first video segment locally at the terminal device according to the target cache information, the processing module is specifically configured to: process a target text character in the initial subtitle text according to the target cache information to generate a mark character; and replace the target text character in the initial subtitle text with the mark character to generate the first subtitle segment.

[0144] According to one or more embodiments of the present disclosure, the first editing operation comprises a moving editing operation, and the moving editing operation is used to adjust a position of the initial video material in a video editing track and / or adjust a video editing track in which the initial video material is located; and the processing module is further configured to: generate change time axis information corresponding to the first subtitle segment according to the cache subtitle data, and the change time axis information represents display time of a text character in the first subtitle segment in the video segment.

[0145] According to one or more embodiments of the present disclosure, the first editing operation comprises a splitting editing operation, and the splitting editing operation is used to split the initial video material into at least two video segments; when the processing module obtains the target cache information corresponding to the first video segment based on the cache subtitle data, the processing module is specifically configured to: obtain a display time interval corresponding to each of the video segments; obtain a text segment corresponding to each of the display time intervals from the cache subtitle data; and generate the target cache information according to a third display time period corresponding to a text character in each of the text segments.

[0146] According to one or more embodiments of the present disclosure, the first editing operation includes N sub-operations, and the interaction module is further configured to: in response to a second editing operation on the first video segment, generate a second video segment, where the second video segment is a processing result of the initial video material after at most N-1 sub-operations, and the second editing operation is a cancel operation of at least one sub-operation; and the processing module is further configured to: based on the cached subtitle data, locally generate, at the terminal device, a second subtitle segment corresponding to the second video segment.

[0147] According to one or more embodiments of the present disclosure, the cached subtitle data includes operation records corresponding to the sub-operations, and the operation records are used to represent modification contents on the initial video material; and when the processing module locally generates, at the terminal device, the second subtitle segment corresponding to the second video segment based on the cached subtitle data, the processing module is specifically configured to: based on the cached subtitle data, obtain the operation records corresponding to the sub-operations; based on the operation records, obtain target cached information corresponding to the second video segment; and according to the target cached information, locally generate, at the terminal device, the second subtitle segment corresponding to the second video segment.

[0148] In a third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, including: at least one processor and a memory.

[0149] The memory stores computer-executable instructions.

[0150] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the subtitle text processing method according to the first aspect and various possible designs of the first aspect.

[0151] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, and the computer-readable storage medium stores computer-executable instructions, when a processor executes the computer-executable instructions, the subtitle text processing method according to the first aspect and various possible designs of the first aspect is implemented.

[0152] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, and when a processor executes the computer program, the subtitle text processing method according to the first aspect and various possible designs of the first aspect is implemented.

[0153] The above description merely illustrates the preferred embodiments of the disclosure and a principle for applying the technologies. It is understood by those skilled in the art that the disclosed scope of the disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by the mutual replacement of the above-described features and the technical features with similar functions disclosed in the disclosure (but not limited to) can be formed.

[0154] Further, although operations are depicted in a particular, sequential order, this should not be understood as requiring or implying that the operations are performed in the order illustrated or sequentially. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, although specific implementation details are contained in the above discussion, these should not be construed as limiting the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0155] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A subtitle text processing method, comprising: obtaining cached subtitle data of a video editing draft after displaying the video editing draft, wherein an initial video material is arranged on a video editing track of the video editing draft, the cached subtitle data comprises initial subtitle text of the initial video material and corresponding initial timeline information, the initial subtitle text is obtained by performing speech recognition on audio of the initial video material, and the initial timeline information represents display time of a text character in the initial subtitle text in the initial video material; generating a corresponding first video segment in response to a first editing operation on the initial video material; generating a first subtitle segment corresponding to the first video segment based on the cached subtitle data, wherein the first subtitle segment is generated by adjusting the cached subtitle data based on modification content corresponding to the first editing operation.

2. The method of claim 1, wherein, The cached subtitle data comprises an offset value and a duration value corresponding to each text character in the initial subtitle text, wherein the offset value represents a start timestamp of a speech segment corresponding to the text character in the initial video material, and the duration value represents a duration of the speech segment corresponding to the text character in the initial video material.

3. The method of claim 2, further comprising: performing speech recognition on the initial video material to generate the initial subtitle text; obtaining positioning information and play time information of each text character in the initial subtitle text, wherein the positioning information comprises a paragraph identifier of a paragraph to which the text character belongs and an intra-paragraph serial number, and the play time information comprises a start timestamp and an end timestamp of a speech segment corresponding to the text character in the initial video material; generating the cached subtitle data according to the positioning information and the play time information of each text character in the initial subtitle text.

4. The method of claim 1, wherein, The generating of the first subtitle segment corresponding to the first video segment based on the cached subtitle data comprises: obtaining target cached information corresponding to the first video segment based on the cached subtitle data, the target cached information being used to represent display time of a target text character corresponding to modification content triggered by the first editing operation in the initial video material; generating the first subtitle segment corresponding to the first video segment locally on a terminal device according to the target cached information.

5. The method of claim 4, wherein, The first editing operation comprises a deletion editing operation, and the deletion editing operation is used to delete a target material segment of the initial video material. The obtaining of the target cached information corresponding to the first video segment based on the cached subtitle data comprises: obtaining a first display time period of the target material segment corresponding to the deletion editing operation; obtaining at least one target text character corresponding to the first display time period from the cached subtitle data; generating the target cached information according to a second display time period corresponding to the at least one target text character.

6. The method of claim 4, wherein, The first subtitle segment corresponding to the first video segment is generated locally at the terminal device according to the target cache information, and the first subtitle segment is generated locally at the terminal device according to the target cache information, and the first subtitle segment is generated locally at the terminal device according to the target cache information. According to the target cache information, the target text character in the initial subtitle text is processed to generate a mark character, and the mark character is a target text character with a text display special effect; The target text character in the initial subtitle text is replaced by the mark character to generate a first subtitle segment.

7. The method of claim 4, wherein, The first editing operation includes a moving editing operation, and the moving editing operation is used to adjust the position of the initial video material in the video editing track and / or adjust the video editing track where the initial video material is located. The method further comprises: According to the cache subtitle data, the change time axis information corresponding to the first subtitle segment is generated, and the change time axis information represents the display time of the text character in the first subtitle segment in the video segment.

8. The method of claim 4, wherein, The first editing operation includes a splitting editing operation, and the splitting editing operation is used to split the initial video material into at least two video segments. The target cache information corresponding to the first video segment is obtained based on the cache subtitle data, and the target cache information corresponding to the first video segment is obtained based on the cache subtitle data. Obtain the display time interval corresponding to each video segment; Based on each display time interval, the corresponding text segment is obtained from the cache subtitle data; According to the third display time period corresponding to the text character in each text segment, the target cache information is generated.

9. The method of claim 1, wherein, The first editing operation includes N sub-operations, and the method further comprises: In response to a second editing operation for the first video segment, a second video segment is generated, wherein the second video segment is a processing result of the initial video material after at most N-1 sub-operations, and the second editing operation is a cancel operation of at least one sub-operation; Based on the cache subtitle data, a second subtitle segment corresponding to the second video segment is generated.

10. The method of claim 9, wherein, The cache subtitle data includes operation records corresponding to each sub-operation, and the operation records are used to represent the modification content of the initial video material; The cache subtitle data includes operation records corresponding to each sub-operation, and the operation records are used to represent the modification content of the initial video material; The cache subtitle data includes operation records corresponding to each sub-operation, and the operation records are used to represent the modification content of the initial video material; The cache subtitle data includes operation records corresponding to each sub-operation, and the operation records are used to represent the modification content of the initial video material.

11. A subtitle text processing apparatus, comprising: A cache module, after displaying a video editing draft, obtains cache subtitle data of the video editing draft, wherein an initial video material is arranged on a video editing track of the video editing draft, the cache subtitle data includes an initial subtitle text of the initial video material and corresponding initial time axis information, the initial subtitle text is obtained by performing speech recognition on audio of the initial video material, and the initial time axis information represents display time of a text character in the initial subtitle text in the initial video material; ​ An interaction module, configured to generate a first video clip corresponding to the first editing operation on the initial video material; A processing module, configured to generate a first subtitle clip corresponding to the first video clip based on the buffered subtitle data, wherein the first subtitle clip is generated by adjusting the buffered subtitle data based on modified content corresponding to the first editing operation.

12. An electronic device comprising: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor executes the subtitle text processing method according to any one of claims 1-10.

13. A computer-readable storage medium, having computer-executable instructions stored therein, when the processor executes the computer-executable instructions, the subtitle text processing method according to any one of claims 1-10 is implemented.

14. A computer program product, comprising a computer program, when the processor executes the computer program, the subtitle text processing method according to any one of claims 1-10 is implemented.

Images