Subtitle processing method and device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a subtitle processing method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring a first subtitle file; extracting the time axis in each piece of subtitle information to obtain subtitle content in each piece of subtitle information; obtaining translation prompt information of the first subtitle file, and translating each subtitle content under the prompt of the translation prompt information to obtain the translated content of each subtitle content; and performing reconstruction processing on the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file. By adopting the embodiment of the invention, the translation accuracy and translation efficiency of subtitle translation can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to the field of artificial intelligence, and specifically to a subtitle processing method, a subtitle processing device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Subtitling is the process of converting content such as dialogue, narration, and lyrics in audio or video from one language to another.

[0003] Currently, subtitle translation suffers from poor accuracy and inefficiency. For example, translating individual subtitle lines separately and then reassembling them results in poor translation quality. Furthermore, when translating an entire subtitle file, the translation efficiency is less than 50%, significantly wasting translation resources and reducing translation efficiency. Therefore, improving the accuracy and efficiency of subtitle translation has become a hot topic of research. Summary of the Invention

[0004] The embodiments of the present application provide a subtitle processing method, apparatus, device, medium, and program product, which can improve the translation accuracy and efficiency of subtitle translation.

[0005] On the one hand, an embodiment of the present application provides a subtitle processing method, the method comprising:

[0006] Obtain a first subtitle file, the first subtitle file including N subtitle information, each subtitle information including subtitle content and a time axis corresponding to the subtitle content; N is a positive integer;

[0007] Extracting the time axis of each subtitle information to obtain the subtitle content of each subtitle information;

[0008] Obtaining translation prompt information of the first subtitle file, and translating each subtitle content under the prompt of the translation prompt information to obtain translation content of each subtitle content, where each subtitle content and its translation content are in different languages;

[0009] The translation content of each subtitle content and the time axis corresponding to each subtitle content are reconstructed to generate a second subtitle file.

[0010] On the other hand, an embodiment of the present application provides a subtitle processing device, which includes:

[0011] An acquiring unit, configured to acquire a first subtitle file, wherein the first subtitle file includes N subtitle information, each subtitle information including subtitle content and a time axis corresponding to the subtitle content; N is a positive integer;

[0012] A processing unit, configured to extract the time axis of each subtitle information to obtain the subtitle content of each subtitle information;

[0013] The processing unit is further configured to obtain translation prompt information of the first subtitle file and, under the guidance of the translation prompt information, translate each subtitle content to obtain translation content of each subtitle content, wherein each subtitle content and its translation content are in different languages;

[0014] The processing unit is further configured to reconstruct the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file.

[0015] In one implementation, the translation process is performed by calling a subtitle translation model; the translation prompt information defines specifications that the subtitle translation model needs to follow when performing the translation process; the specifications include at least one of the following:

[0016] Content format specification, which is used to instruct the subtitle translation model to translate the subtitle content in accordance with the content format of the subtitle content; and

[0017] Context summary specification: The context summary specification is used to instruct the subtitle translation model to generate context semantics for multiple consecutive subtitle contents and use the context semantics to guide the subtitle translation model to perform translation processing;

[0018] The terminology specification is used to indicate that the subtitle translation model obtains preset terms from the terminology library and performs the translation process on the subtitle content.

[0019] In one implementation, after the extraction process, a time axis in each subtitle information is obtained, and N time axes are arranged in chronological order; the processing unit is further configured to:

[0020] Obtain a window sliding strategy, where the window sliding strategy is used to represent a rule for grouping N subtitle contents based on N time axes;

[0021] Perform window sliding processing on N time axes according to the window sliding strategy to obtain at least one time axis group; each time axis group includes at least one time axis;

[0022] The subtitle contents corresponding to each time axis in each time axis group are respectively combined to obtain at least one subtitle group; each subtitle group includes at least one subtitle content.

[0023] In one implementation, the processing unit is configured to obtain translation prompt information of the first subtitle file, and translate each subtitle content under the guidance of the translation prompt information to obtain the translated content of each subtitle content, specifically for:

[0024] Obtaining initial prompt information of a first subtitle file; the initial prompt information defines content format specifications, context summary specifications, and terminology specifications;

[0025] Obtaining group information of a target subtitle group, wherein the group information includes the number of subtitle contents included in the target subtitle group; the target subtitle group is any one of the at least one subtitle group;

[0026] Add the target subtitle group's group information to the initial prompt information to obtain the target translation prompt information corresponding to the target subtitle group;

[0027] The subtitle translation model is called to translate each subtitle content in the target subtitle group under the prompt of the target translation prompt information to obtain the translation content of each subtitle content in the target subtitle group.

[0028] In one implementation, the processing unit is configured to perform window sliding processing on N time axes according to the window sliding strategy to obtain at least one time axis group, specifically to:

[0029] Obtain the time threshold and quantity threshold according to the window sliding strategy;

[0030] Starting from the first time axis among the N time axes, recursively calculate whether the time interval between adjacent time axes is greater than the time threshold;

[0031] If the i-th time interval is equal to or greater than the time threshold, the time axis before the i+1-th time axis among the N time axes is divided into a time axis group; i is an integer, and 1≤i≤N-1;

[0032] From the multiple ungrouped time axes in the N time axes, detecting whether each time axis meets a time threshold or a quantity threshold;

[0033] If there is a target time axis that meets the time threshold or the quantity threshold, the time axis that is located before the target time axis among the multiple ungrouped time axes in the N time axes is divided into one time axis group.

[0034] In one implementation, the processing unit is further configured to:

[0035] If there is no time axis that meets the time threshold or the quantity threshold among the multiple ungrouped time axes in the N time axes, the multiple ungrouped time axes in the N time axes are divided into one time axis group.

[0036] In one implementation, the window sliding strategy is used to indicate that the maximum time interval between adjacent time axes among N time axes is used as the time threshold. The process of determining the time threshold includes:

[0037] Calculate the time intervals between adjacent time axes in N time axes respectively to obtain N-1 time intervals;

[0038] The maximum time interval among the N-1 time intervals is determined as the time threshold.

[0039] In one implementation, the processing unit is configured to translate each subtitle content under the prompt of the translation prompt information, and when obtaining the translation content of each subtitle content, specifically to:

[0040] Calling the subtitle translation model to translate each subtitle content under the prompt of the translation prompt information to obtain the output content of the subtitle translation model;

[0041] Parse the output content in a format that matches the output content and obtain the parsing result;

[0042] If the parsing result indicates that the translation content of the target subtitle content fails to be parsed, the candidate translation tool is called to re-translate the target subtitle content to obtain a new translation content of the target subtitle content; the target subtitle content belongs to N subtitle contents;

[0043] The processing unit is used to reconstruct the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate the second subtitle file, specifically for:

[0044] Reconstructing the new translation content of the target subtitle content, the multiple successfully parsed translation contents among the N subtitle contents, and the time axis corresponding to each subtitle content to generate a second subtitle file.

[0045] In one implementation, the output content includes at least one subtitle translation result; the processing unit is configured to parse the output content for a content format, and upon obtaining the parsed result, specifically to:

[0046] Get at least one regular expression;

[0047] Use at least one regular expression to parse each subtitle translation result to obtain a parsed result; wherein,

[0048] If any subtitle translation result matches a regular expression, the parsing result corresponding to any subtitle translation result is parsed successfully, and any subtitle translation result is regarded as a translation content;

[0049] If any subtitle translation result does not match at least one regular expression, the parsing result corresponding to any subtitle translation result is parsing failure.

[0050] In one implementation, each subtitle content includes subtitle text and a number, where the number is used to represent the playback order of the subtitle text in the multimedia resource; the translation content includes translation text and a number;

[0051] The processing unit is used to reconstruct the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate the second subtitle file, specifically for:

[0052] Obtaining the time axis corresponding to each subtitle content from the N time axes according to the number included in the translation content corresponding to each subtitle content;

[0053] Combine the translation content of each subtitle content with the time axis corresponding to the subtitle content to obtain N translation information;

[0054] A second subtitle file is generated based on the N translation information.

[0055] In another aspect, an embodiment of the present application provides a computer device, comprising:

[0056] a processor adapted to execute a computer program;

[0057] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the subtitle processing method is implemented.

[0058] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded by a processor and executing the above-mentioned subtitle processing method.

[0059] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the above-mentioned subtitle processing method.

[0060] In an embodiment of the present application, the first subtitle file to be processed or translated includes N subtitle information, each subtitle information includes subtitle content and the time axis corresponding to the subtitle content, and N is a positive integer. Considering that the time axis corresponding to each subtitle content in the first subtitle file does not need to be translated, the time axis in each subtitle information is extracted to obtain the subtitle content in each subtitle information; in this way, the time axis is extracted before translation, and only the subtitle content needs to be translated, which significantly saves translation resources of software and hardware and improves translation efficiency. Further, it supports the use of translation prompt information constructed for the first subtitle file, so that each subtitle content is translated under the prompt of the translation prompt information; since the translation prompt information defines a variety of specifications that help improve translation quality and accuracy, the translation process is performed under the prompt of the translation prompt information, which can significantly improve the accuracy and quality of the translation content of the subtitle content. Furthermore, after the translation process obtains the translation content of each subtitle content, each translation content and the time axis corresponding to each subtitle content are reconstructed, and the second subtitle file can be quickly generated without tedious operations. It can be seen that the embodiment of the present application translates the subtitle content in the first subtitle file according to the prompt of the translation prompt information, and improves the accuracy and quality of the translation by means of the translation prompt information. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0062] Figure 1 This is a schematic diagram of the file structure of a subtitle file provided by an exemplary embodiment of the present application;

[0063] Figure 2 This is a schematic diagram of the architecture of a subtitle processing system provided by an exemplary embodiment of the present application;

[0064] Figure 3 This is a flowchart of a subtitle processing method provided by an exemplary embodiment of the present application;

[0065] Figure 4 This is a schematic diagram of extracting a time axis from subtitle information provided by an exemplary embodiment of the present application;

[0066] Figure 5 This is a schematic diagram of a subtitle translation model provided by an exemplary embodiment of the present application performing translation processing under the prompt of translation prompt information;

[0067] Figure 6 is a schematic diagram of a regular expression provided by an exemplary embodiment of the present application;

[0068] Figure 7 is a flowchart of another subtitle processing method provided by an exemplary embodiment of the present application;

[0069] Figure 8 This is a schematic diagram of a timeline-based subtitle content grouping method provided by an exemplary embodiment of the present application;

[0070] Figure 9 This is a schematic diagram of a time axis extraction based on a time threshold provided by an exemplary embodiment of the present application;

[0071] Figure 10 This is a schematic diagram of a time axis extraction based on a time threshold and a quantity threshold provided by an exemplary embodiment of the present application;

[0072] Figure 11 This is a flowchart of another subtitle processing method provided by an exemplary embodiment of the present application;

[0073] Figure 12 This is a structural diagram of a subtitle processing device provided by an exemplary embodiment of the present application;

[0074] Figure 13 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0076] In an embodiment of the present application, a subtitle processing solution is provided, specifically a subtitle translation solution based on artificial intelligence (AI) technology. The following is a brief introduction to the technical terms and related concepts involved in the subtitle processing solution, including:

[0077] (1)Subtitles.

[0078] Subtitles, or subtitle texts, are textual content to help users understand multimedia resources; for example, subtitle texts may only include text (such as the subtitle text "How long until the babysitter arrives"), or may include text and icons. Multimedia resources include, but are not limited to: film and television works (such as movies, TV series, short videos, stage plays, documentaries, and game videos, etc.) and music, etc.; depending on the type of multimedia resources, the subtitles of multimedia resources may vary. For example, subtitles for film and television works may include, but are not limited to: the lines (or dialogues) of the actors in the film and television works, narration, cast and crew, soundtracks for the film and television works, and task introductions for the film and television works. For another example, subtitles for music include the lyrics of the music.

[0079] Since subtitles are adapted to multimedia resources, the adaptation is reflected in the playback position and playback time of subtitles in multimedia resources, which need to match the resource content of multimedia resources (such as video images) to achieve synchronization of images and sounds. Therefore, the subtitles of multimedia resources have a special subtitle format, which is used to standardize or constrain the subtitle files corresponding to multimedia resources so that when the subtitle files are loaded, the subtitles can be adapted to the resource content of multimedia resources. Among them, the subtitle format of the subtitle file can be found in Figure 1 , assuming that the multimedia resource includes 3 subtitle texts or 3 lines of subtitle texts; the subtitle file of the multimedia resource includes subtitle information corresponding to each line of subtitle text, and any subtitle information includes subtitle content and the time axis corresponding to the subtitle content. Among them: ① Any subtitle content is composed of a subtitle text and a number corresponding to the subtitle text; the number corresponding to the subtitle text is used to represent the playback order of the subtitle text in the multimedia resource, such as subtitle text 1 numbered 1, subtitle text 2 numbered 2, and subtitle text 3 numbered 3. The playback order in the multimedia resource is: subtitle text 1 → subtitle text 2 → subtitle text 3. ② The time axis corresponding to any subtitle content is used to represent the playback time period of the subtitle content in the multimedia resource; by configuring the time axis for the subtitle content, it is ensured that the subtitle content can be played synchronously with the resource content in the multimedia resource. The time axis corresponding to the subtitle content consists of the start time and the end time, and the time format of the start time and the end time is "hours: minutes: seconds, milliseconds". For example, Figure 1 The start time of the middle time axis 101 is "00:00:10,500", which means that the subtitle text starts to be displayed at 10 seconds and 500 milliseconds of the video, and the end time is "00:00:11,100", which means that the subtitle text ends to be displayed at 11 seconds and 100 milliseconds of the video.

[0080] (2) Artificial intelligence.

[0081] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. Machine learning models within the AI field are network models derived through model training using machine learning. Machine learning is a multidisciplinary discipline that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. Machine learning typically encompasses techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, model-based learning, and deep learning (DL).

[0082] The embodiments of the present application mainly relate to large language models (LLMs) in the field of deep learning in machine learning technology. Large language models are general language processing models based on deep learning technology and massive parameter scale (usually billions to trillions); large language models have gradually become the core technology in the field of natural language processing (NLP), leading the technological innovation from traditional tasks to generative artificial intelligence. Through pre-training of large-scale parameters, large language models have achieved significant performance improvements in tasks such as text generation, question-answering systems, and machine translation, demonstrating strong generalization and context understanding capabilities. As the model scale of large language models continues to expand, from the initial millions of parameters to hundreds of billions or even trillions of parameters today, the performance of large language models has shown an approximately logarithmic linear growth trend with model scale, which has further promoted the widespread adoption of large language models in scientific research and industrial applications. Large language models have the following characteristics: strong generalization ability and multi-round result generation capabilities; strong comprehension ability, which can better understand the semantics of subtitle text; and content generation capabilities, which can accurately generate translation content corresponding to subtitle text.

[0083] Existing subtitle translation solutions include: line-by-line translation solution, file translation solution and large language model translation solution. Among them: ① In the line-by-line translation solution, the subtitle text in the subtitle file needs to be split into single lines, and then each line of subtitle text is input into the translation service, and the translation is completed by the translator and then the translated text is returned; but multiple subtitle texts are textual manifestations of coherent plot scenes. If the contextual semantics are not considered during the translation process, the accuracy of the subtitle translation will be greatly reduced. ② In the file translation solution, the entire subtitle file is transmitted to the service platform, and the platform side translates the content of the entire subtitle file and then returns the translated subtitle file; since only part of the content in the subtitle file needs to be translated, such as the timeline does not need to be translated, translating the entire subtitle file will result in a large amount of character space being occupied, thereby wasting translation resources and reducing the efficiency of translation (for example, only 20% of the total translated content is subtitle content that needs to be translated). ③ In large language model translation solutions, subtitle text or content is simply assembled into a JSON (JavaScript Object Notation, a Java data structure) array and input into the large language model for subtitle translation. However, large language models all process data based on the Transformer structure, and the Transformer structure often suffers from problems such as overfitting and hallucination when processing JSON arrays. Subtitles often overlap and some content is lost, resulting in unstable output and a significant waste of API (Application Programming Interface) resources (i.e., resources consumed by calling the large language model through the interface). In summary, existing subtitle translation solutions suffer from poor translation accuracy (such as mismatches between subtitles and video images, and inconsistencies between the content before and after translation) and low translation efficiency.

[0084] To improve the quality and efficiency of subtitle translation, the subtitle processing solution provided in the embodiment of the present application specifically proposes a subtitle translation solution based on a subtitle translation model (in the embodiment of the present application, a large language model or other model capable of implementing subtitle translation in combination with prompts), Prompt (prompts of the large language model) technology, and timeline extraction technology. This subtitle translation solution supports the construction of subtitle-specific translation prompt information (i.e., Prompt) for the first subtitle file (i.e., the subtitle file before subtitle translation). Under the prompt of this translation prompt information, the thinking and output of the subtitle translation model are guided and constrained, so that the subtitle translation model can better achieve subtitle translation.

[0085] Among them, the general implementation process of the subtitle processing solution provided by the embodiment of the present application may include: a computer device obtains a first subtitle file to be translated, the first subtitle file includes N subtitle information, each subtitle information includes subtitle content and the time axis corresponding to the subtitle content, N is a positive integer. To prevent the subtitle translation model from translating the time axis, the computer device will first extract the time axis in each subtitle information in the first subtitle file to obtain the subtitle content in each subtitle information, the subtitle content includes the subtitle text and number to be translated. Then, the computer device obtains the translation prompt information of the first subtitle file, the translation prompt information defines the specifications that the subtitle translation model needs to follow when performing translation processing; in this way, the computer device calls the subtitle translation model to translate each subtitle content under the prompt of the translation prompt information, which can ensure that the subtitle translation model implements the translation of the subtitle content according to the specification and outputs the translation content of each subtitle content; the translation content and the corresponding subtitle content are different in language (such as the subtitle content is Chinese and the translation content is English). Since the output of the subtitle translation model does not include a timeline, it is impossible to synchronize the translated content with the multimedia resources corresponding to the first subtitle file. Therefore, the computer device reconstructs the translated content of each subtitle content and the timeline corresponding to each subtitle content to generate a second subtitle file (i.e., the subtitle file after subtitle translation of the first subtitle file).

[0086] It can be seen that, on the one hand, the embodiment of the present application extracts the timeline from each subtitle information, so that the subtitle translation model only needs to translate the subtitle content, thereby reducing the resource consumption brought about by the timeline translation. On the other hand, according to the prompts of the translation prompt information with defined specifications, the subtitle translation model is guided to translate each subtitle content, so that the subtitle translation model can translate the subtitle content and output the results according to the specified specifications, which greatly improves the accuracy and efficiency of the subtitle translation. On the other hand, calling the AI model (i.e., the subtitle translation model) for subtitle translation and only translating the subtitle content significantly improves the translation efficiency.

[0087] The subtitle processing solution provided in the embodiments of the present application, as a solution for subtitle translation, can be applied to various subtitle translation scenarios that utilize subtitle translation models for subtitle translation. In other words, the subtitle processing solution provided in the embodiments of the present application can be applied to any subtitle translation scenario that requires the use of a subtitle translation model for subtitle translation; among which, the subtitle translation model classification model may be applicable to at least one of the following subtitle translation scenarios:

[0088] Video translation scenario: Video works often include dialogue subtitles between multiple characters, music subtitles and cast subtitles, etc. Each subtitle may correspond to an independent first subtitle file; for each first subtitle file, it is supported to first extract the time axis of each subtitle information in the first subtitle file, and then call the subtitle translation model to translate each subtitle content after the time axis is extracted under the prompt of the translation prompt information to obtain the translation content of each subtitle content; the translation content and the time axis corresponding to the corresponding subtitle content are reconstructed to quickly construct the second subtitle file after the subtitle translation. It is worth noting that according to the difference in the subtitle type corresponding to the first subtitle file, such as the first subtitle file is a dialogue subtitle, or a music subtitle, etc., the specifications in the translation prompt information corresponding to the first subtitle file will be different, but the specifications are all to guide the subtitle translation model to perform better subtitle translation, so as to achieve better translation effects for all types of subtitle files.

[0089] Music translation scenario: Musical works often contain lyrics, which can be displayed to help users understand the music content. In this scenario, the first subtitle file is the lyrics subtitles to be translated. Each line of lyrics in the lyrics subtitles is extracted from the time axis, and then the subtitle translation model is called to translate each line of lyrics after the time axis is extracted under the prompt of the translation prompt information to obtain the translated lyrics of each line of lyrics. The translated lyrics and the time axis corresponding to the corresponding lyrics are then reconstructed to obtain the second translated subtitle file - the translated lyrics subtitles.

[0090] It should be noted that the above description is only an exemplary product performance and subtitle translation scenario given in the embodiment of the present application, and does not limit the product performance and subtitle translation scenario of the subtitle processing solution provided in the embodiment of the present application. The subtitle processing solution provided in the embodiment of the present application can provide efficient, accurate and convenient subtitle translation services in various subtitle translation scenarios, and shows high value and practicality in various subtitle translation scenarios, which helps to improve the efficiency of subtitle translation. For the sake of convenience, the subtitle processing solution provided in the embodiment of the present application will be introduced by taking the case where the first subtitle file to be translated is a subtitle file corresponding to a multimedia resource, such as a multimedia resource being a video or audio, as an example, and will be specifically explained here.

[0091] Furthermore, the subtitle processing solution provided in the embodiment of the present application is integrated into a computer device, and specifically can be loaded and executed by the computer device in the form of an application service. The model data of the subtitle translation model is configured into the application service in a plug-in manner; wherein the model data of the subtitle translation model includes at least: a model name and a model storage address, so as to facilitate the rapid location of the subtitle translation model based on the model storage address and model name. Optionally, the subtitle processing solution is provided by an application, which refers to a computer program that has the function of performing subtitle translation according to the subtitle processing solution provided in the embodiment of the present application. According to the classification of the operation mode of the application, the application may include but is not limited to: downloading an installation package and installing the installation package to a client running in the terminal; a small program that does not require downloading an installation package, but is directly invoked and run in the client, and the small program serves as a subroutine of the client; a web (WorldWide Web, Global Wide Area Network) application opened by a browser running in the terminal; and so on. According to the classification of the functional type of the application, the application may include but is not limited to video applications, or audio applications, etc. Optionally, in addition to being provided by application services, the subtitle processing solution can also be provided by a plug-in; a plug-in is a component that can extend software functionality or add new features; a plug-in can be in the form of a computer program, or a component (such as a binary file, configuration file, script, or other type of data packet). The plug-in is designed to be compatible with the application (i.e., application program) or operating system and can be loaded and executed at runtime to implement the subtitle processing solution. The following will be explained using the example of the subtitle processing solution provided by a video application (such as a video client) in an embodiment of the present application, which is specifically explained here.

[0092] For ease of understanding, the following Figure 2 The subtitle processing system shown in the figure introduces the subtitle processing solution based on the subtitle translation model proposed in the embodiment of the present application. Figure 2 As shown, the subtitle processing system includes a terminal 201, a server 202, and a server 203. This embodiment of the present application does not limit the number and type of terminals 201, servers 202, and servers 203. Assume that the subtitle processing solution provided in this embodiment of the present application is integrated into a video client, which is installed and running in the terminal 201 to provide audio and video playback functions for users.

[0093] (1) Terminal 201 refers to a terminal device that can realize human-computer interaction with a user; for example, a user can watch a film or TV series through a video application running on the terminal 201, and convert the language of the subtitles of the film or TV series. The terminal device may include, but is not limited to, a smartphone (such as a smartphone running an Android system or a smartphone running an Internetworking Operating System (IOS)), a tablet computer, a laptop computer, a desktop computer, a vehicle-mounted device, a head-mounted device, a smart home appliance, a smart chat robot, and an aircraft. The embodiments of this application do not limit the type of terminal device, and this is explained here.

[0094] (2) Server 202 refers to a server corresponding to terminal 201, which is used to interact with terminal 201 for data to provide computing and application service support for terminal 201; specifically, it is a background service that provides application services for the video client running in terminal 201. Server 203 is a server deployed with a subtitle translation model. When the subtitle translation model in server 203 is called, server 203 provides computing resources for the subtitle translation model to help the subtitle translation model translate subtitles. Among them, server 202 and server 203 can be independent physical servers, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Among them, the terminal 201, server 202 and server 203 can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0095] The computer device that executes the subtitle processing solution provided in the embodiment of the present application can be any one of the terminal 201 and the server 202, or the computer device includes the terminal 201 and the server 202, or the computer device includes the terminal 201, the server 202 and the server 203. Figure 2 The subtitle processing system shown introduces the subtitle processing solution flow.

[0096] In a specific implementation, when the server 202 receives a subtitle translation request for subtitle translation of a first subtitle file corresponding to a multimedia resource, the server 203 obtains the first subtitle file in response to the subtitle translation request, such as obtaining it locally or from the terminal 201. The subtitle translation request may be generated by the terminal 201 based on a subtitle translation trigger condition and sent to the server 202; the subtitle translation trigger condition may include, but is not limited to: the terminal 201 receives a play operation performed by a user on a multimedia resource in a video client, and detects that the language commonly used in the user's current geographic location is different from the language of the subtitle content included in the first subtitle file corresponding to the multimedia resource, then the terminal 201 automatically generates a subtitle translation request, which is used to request that the subtitle content included in the first subtitle file be translated into a language commonly used in the user's current geographic location, so as to achieve automatic translation of the language type and enhance the user's experience of viewing multimedia resources. Alternatively, the terminal 201 receives a language switching operation from a user in a video client to switch the subtitle language of a multimedia resource from a first language to a second language, and the terminal 201 generates a subtitle translation request based on the language switching operation, which is used to request that the subtitle content included in the first subtitle file be switched from the first language to the second language.

[0097] Then, the server 202 extracts each of the N subtitle information included in the first subtitle file from the time axis to obtain the subtitle content in each subtitle information. In addition, the server 202 obtains the translation prompt information of the first subtitle file, which is pre-generated and cached, or generated in real time based on the number of subtitles of the subtitle content input into the subtitle translation model in each round. Furthermore, the server 202 calls the subtitle translation model in the server 203 according to the model name and model storage address through the plug-in subtitle translation model, and translates each subtitle content under the prompt of the translation prompt information to obtain the translation content of each subtitle content. Specifically, the server 202 inputs the translation prompt information and the subtitle content into the called subtitle translation model, so that the subtitle translation model performs the translation process and returns the model output result - the translation content of the subtitle content - to the server 202 through the server 203.

[0098] Finally, after server 202 obtains the translation of each subtitle content from server 203, it configures a timeline for each translation and generates a second subtitle file. This second subtitle file is sent by transmission server 202 to terminal 201, so that when terminal 201 loads the second subtitle file, the subtitles and video content can be synchronized or matched in the video client.

[0099] Based on the above Figure 2As can be seen from the subtitle processing system shown, the embodiment of the present application supports the plug-in of the model. Not only can it be configured to access various large language models for subtitle translation at any time to improve the accuracy and flexibility of subtitle translation, but the terminal also supports automatic judgment of whether subtitle language translation is required based on user positioning, providing adaptability between user positioning and the subtitle language type of multimedia resources, thereby enhancing the user experience of enjoying multimedia resources.

[0100] Based on the above brief introduction to the subtitle processing solution and subtitle processing system provided by the embodiments of the present application, the following points should be explained:

[0101] ① The above-mentioned embodiments of this application Figure 2 The system architecture shown is for the purpose of more clearly illustrating the technical solutions of the embodiments of the present application and does not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. In other words, Figure 2 The architectural diagram of the subtitle processing system shown is an exemplary architectural diagram; in actual applications, the number and distribution of computer devices included in the subtitle processing system may vary, and the embodiments of the present application do not limit the architectural diagram of the subtitle processing system.

[0102] For example, a computer program integrating a subtitle processing solution can be deployed in the terminal 201 of the subtitle processing system. In this way, the subtitle translation model can be called to perform subtitle translation through direct interaction between the terminal 201 and the server 203, without going through the server 202. For another example, in addition to being a single server, the server 203 in the subtitle processing system can also be a distributed system; the distributed system includes multiple servers, so that the subtitle translation task performed by the subtitle translation model can be split among different servers for execution; for example, N subtitle contents can be divided into multiple groups, and multiple servers can be called to translate the subtitle contents in each group in parallel, thereby significantly improving the efficiency of subtitle translation and reducing the load on a single server.

[0103] For example, server 202 and server 203 have the following Figure 2 In addition to the independent devices shown, the service capabilities of server 202 and server 203 can also be integrated into the same server. In this case, one server provides both application services and translation services. Figure 2 In the system shown, the server 202 that provides application services to the video client running in the terminal 201 and the server 203 that deploys the subtitle translation model are the same server.

[0104] ② The collection and processing of relevant data in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations. The acquisition of personal information requires the knowledge or consent of the individual subject (or the legal basis for obtaining the information), and subsequent data use and processing shall be carried out within the scope of authorization of laws and regulations and the subject of personal information. For example, when the embodiments of this application are applied to specific products or technologies, such as when translating the subtitle content included in the first subtitle file, the user's permission or consent must be obtained, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of the relevant region.

[0105] Based on the subtitle processing solution described above, the embodiment of the present application proposes a more detailed subtitle processing method. The subtitle processing method proposed in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0106] Figure 3 A flowchart of a subtitle processing method provided by an exemplary embodiment of the present application is shown; the subtitle processing method can be executed by a computer device, which can be Figure 2 The terminal and / or server in the system shown. The subtitle processing method may include but is not limited to steps S301-S304:

[0107] S301: Obtain a first subtitle file.

[0108] The first subtitle file is the subtitle file to be translated corresponding to the multimedia resource. For example, if the multimedia resource is a video, the first subtitle file corresponding to the video may be a subtitle file containing the lines to be translated, a subtitle file containing the cast and crew, or a background subtitle file used for the video introduction. For another example, if the multimedia resource is music, the first subtitle file corresponding to the music may be a subtitle file containing the lyrics to be translated. Given the wide variety of multimedia resource types, the embodiments of this application do not limit the specific subtitle type of the first subtitle file.

[0109] The first subtitle file includes N subtitle information (or N lines of subtitle information), each of which includes subtitle content and a corresponding timeline. The subtitle content is composed of subtitle text and a corresponding number for the subtitle text. In other words, a subtitle information in the first subtitle file can be represented as (number, timeline, subtitle text). For explanations of the number, subtitle text, and timeline included in the subtitle information, please refer to the aforementioned description and will not be repeated here.

[0110] In a specific implementation, the first subtitle file of the multimedia resource is pre-created and stored when the multimedia resource is created; when the computer device needs to perform subtitle translation on the first subtitle file corresponding to the multimedia resource, the first subtitle file can be directly obtained. For example, the computer device is a server, and the first subtitle file is stored in a dedicated file database, then the server retrieves the first subtitle file from the file database. For another example, if the computer device is a terminal, and the first subtitle file is stored in a dedicated file database, then the possible way for the terminal to obtain the first subtitle file is: the terminal sends a file acquisition request to the server, and the server obtains the first subtitle file from the file database in response to the file acquisition request, and returns the first subtitle file to the terminal. It should be understood that the way in which the computer device obtains the first subtitle file varies depending on the device type of the computer device and the storage method of the first subtitle file; the embodiment of the present application does not limit the specific way in which the computer device obtains the first subtitle file.

[0111] S302: Extracting the time axis of each subtitle information to obtain the subtitle content of each subtitle information.

[0112] As mentioned above, among the N subtitle information included in the first subtitle file, each subtitle information includes a time axis corresponding to the subtitle content, and the playback time of the subtitle content and the playback time of the multimedia resource can be synchronized through the time axis. Considering that the number of characters occupied by the time axis in the first subtitle file is large, and these time axes are invisible to the user, if the first subtitle file is directly translated into characters, the subtitle translation overhead will increase and the efficiency of the subtitle translation will be reduced. Based on this, the embodiment of the present application supports extracting the time axis from each subtitle information included in the first subtitle file before translating the subtitles of the first subtitle file. After the time axis in the subtitle information is extracted, only the subtitle content retained in each subtitle information can be subtitled; compared to directly translating each subtitle information, the subtitle translation overhead is significantly reduced and the efficiency of the subtitle translation is improved.

[0113] The extraction of the time axis in the subtitle information can be understood as extracting the time axis included in the subtitle information from the subtitle information; in this way, the subtitle information after the extraction also includes the subtitle content, specifically the subtitle text and the number. The extraction process also obtains the time axis in each subtitle information, and the N time axes in the N subtitle information are arranged in chronological order, specifically, they are sorted according to the playback order between the subtitle information to which the corresponding time axis belongs, to obtain a time axis sequence, and the time axis sequence is cached. For example, if Figure 4As shown, assuming that the first subtitle file includes subtitle information 1, subtitle information 2, and subtitle information 3; subtitle information 1 is represented as (number 1, timeline 1, subtitle text 1), subtitle information 2 is represented as (number 2, timeline 2, subtitle text 2), and subtitle information 3 is represented as (number 3, timeline 3, subtitle text 3). Then the timelines in the three subtitle information are extracted, and the three subtitle contents are subtitle content 1 (number 1, subtitle text 1), subtitle content 2 (number 2, subtitle text 2), and subtitle content 3 (number 3, subtitle text 3). The extraction process also obtains the timelines in the three subtitle information, and the three timelines are arranged in chronological order, and the obtained timeline sequence is: timeline 1, timeline 2, and timeline 3.

[0114] S303: Obtain translation prompt information of the first subtitle file, and translate each subtitle content under the prompt of the translation prompt information to obtain the translation content of each subtitle content.

[0115] Among them, the translation processing for the subtitle content is executed by calling the subtitle translation model. The translation prompt information of the first subtitle file refers to the prompt input to the subtitle translation model, which is used to guide the subtitle translation model to generate output related to the translation prompt information. Therefore, accurately designing subtitle-specific translation prompt information for subtitle translation is the key to the accurate translation of subtitle content by the subtitle translation model. The embodiment of the present application defines the specifications that the subtitle translation model needs to follow when translating the subtitle content in the translation prompt information; after the translation prompt information is input into the subtitle translation model, the subtitle translation model follows at least one specification defined in the translation prompt information, and translates each subtitle content to ensure that the translation content of the output subtitle content meets the specifications in the translation prompt information, thereby improving the translation quality and accuracy of the translation content.

[0116] Specifically, the specifications defined in the exclusive translation prompt information designed for subtitle translation in this embodiment of the application include at least one of the following: content format specifications, context summary specifications, terminology specifications, model role specifications, work content specifications, and work example specifications. The following describes the guiding role of each specification in the subtitle translation model; among them:

[0117] ① The content format specification is used to instruct the subtitle translation model to translate and process the subtitle content in accordance with the content format of the subtitle content. Specifically, by limiting the content format of the subtitle translation model's output content to the content format of the subtitle content (i.e., the content format of (number, subtitle content)), the time axis can be removed from the output content of the subtitle translation model, thereby effectively avoiding subtitle merging issues (such as incorrectly merging the translation content of multiple subtitles into one translation content, etc.) when the subtitle translation model fails to translate and process multiple subtitle information in accordance with the specified content format. This improves the accuracy of the one-to-one correspondence between the translation content output by the subtitle translation model and the subtitle content.

[0118] ② The context summary specification is used to instruct: the subtitle translation model to generate contextual semantics for multiple consecutive subtitle contents, and use the contextual semantics to guide the subtitle translation model in translation processing. In other words, the context summary specification can force the subtitle translation model to summarize the overall semantics expressed by multiple consecutive subtitle contents; for example, let the subtitle translation model summarize the picture scenes described by multiple consecutive subtitle contents (for example, if the multimedia resource is a film or TV series, the subtitle translation model summarizes the dialogue scenes or plot scenes based on multiple subtitle contents), or let the subtitle translation model summarize the summary information in multiple consecutive subtitle contents to achieve a summary of contextual semantics, etc. In this way, after the subtitle translation model generates contextual semantics for multiple consecutive subtitle contents under the prompt of the context summary specification, the contextual semantics assists the subtitle translation model in translating a single subtitle content by introducing contextual relevance, thereby improving the translation quality.

[0119] ③Terminology specification is used to indicate: the subtitle translation model obtains preset terms from the term library to translate the subtitle content. In detail, some professional terms in Chinese (such as slippery, beautiful, and not three-dimensional) have unique cultural connotations, expressions, and contexts; if they are translated into other languages (such as English) by relying solely on the subtitle translation model, it is difficult to translate accurately; for this reason, the embodiment of the present application supports the establishment of a term library, in which some term pairs (i.e., preset terms) are pre-set. A term pair includes a term 1 to be translated that belongs to the first language category, and a term 2 that belongs to the second language category after term 1 is translated. In other words, the term library can be understood as a database that caches professional terms corresponding to different language categories. Different term libraries can be established for translation needs of different languages (i.e., language categories); for example, a Chinese-English term library, an English-Chinese term library, a Chinese-Korean term library, etc.

[0120] ④ The model role specification is used to indicate the role played by the subtitle translation model in the translation process of subtitle content.

[0121] ⑤ The work content specification is used to standardize the work content that the subtitle translation model needs to perform during the translation process of subtitle content, such as intent slot extraction, intent recognition, and word segmentation.

[0122] ⑥ The working example specification is used to provide one or more translation processing examples that meet the subtitle translation requirements, so that the subtitle translation model can learn from one or more translation processing examples and thus have the ability to accurately translate the subtitle content.

[0123] It should be noted that the translation prompt information provided in the embodiment of the present application may include other specifications in addition to the above-mentioned points ①-⑥, and the embodiment of the present application does not limit this.

[0124] Furthermore, the following Figure 5 The exemplary implementation process of calling the subtitle translation model on the computer device to translate the subtitle content under the prompt of the translation prompt information to obtain the translated content of the subtitle content is introduced. Figure 5 As shown, the model data (model name and model storage address, etc.) of the aforementioned subtitle translation model is configured as a plug-in or application service in the computer device; therefore, when the computer device constructs the translation prompt information, it will call the subtitle translation model through the interface and according to the model data. The computer device inputs the subtitle content and translation prompt information as user input into the subtitle translation model. Among them, in order to ensure the accuracy of the user input, the embodiment of the present application also supports formatting the user input and then inputting it into the subtitle translation model; for example, converting the subtitle content and translation prompt information in the user input into a markup language format (eXtensible Markup Language, XML) can ensure that the translation prompt information and subtitle content will not produce format changes and content changes during the transmission process, thereby helping the subtitle translation model to read the correct user input.

[0125] The character translation model can quickly locate its role (i.e., the role of the subtitle translation model) under the guidance of the model role specification defined in the translation prompt information. For example, the model role specification defined in the translation prompt information is "You are a translation assistant who translates subtitles for content." Under the guidance of this model role specification, the subtitle translation model can locate its role as a translation assistant. When locating the role of the subtitle translation model, the character translation model can know the content of the work it needs to complete under the guidance of the work content specification defined in the translation prompt information. For example, the work content specification defined in the translation prompt information is "Translate subtitle content belonging to the first language into translation content belonging to the second language." Under the guidance of this work content specification, the subtitle translation model can clearly understand the content of the work it needs to perform.

[0126] The subtitle translation model also learns from the subtitle translation examples provided by the work example specifications defined in the translation prompt information. This helps the subtitle translation model clarify the task requirements (or work content) of the translation task, provides translation background information to help the subtitle translation model understand the translation context, and so on, which helps improve the subtitle translation model's translation capabilities. The translation prompt information also defines a context summary specification, which allows the subtitle translation model to summarize the contextual semantics expressed by multiple consecutive subtitles input. The summarized contextual semantics can assist the subtitle translation model in translating individual subtitles. For example, the contextual semantics generated by summarizing the contextual semantics of multiple consecutive subtitles is "The scene described by the multiple consecutive subtitles is: Character 1 and Character 2 playing badminton on the school playground." Based on the multiple specifications defined in the above translation prompt information, the subtitle translation model has learned some subtitle translation capabilities. Under the prompts of the content format specifications and terminology specifications defined in the translation prompt information, the subtitle translation model can query preset terms from the terminology library to assist in the translation process, and limit the content format of the output results (i.e., the translated content) according to the content format indicated by the content format specifications, such as the content format is (number, translated content), to avoid problems such as merging or overlapping of multiple translated contents.

[0127] It can be seen that the embodiment of the present application prompts the subtitle translation model to translate the subtitle content according to the specifications corresponding to multiple dimensions by designing translation prompt information specific to subtitles; compared with subtitle translation by a large language model without translation prompt information or subtitle translation by a large language model based on translation prompt information without defined multi-dimensional specifications, the subtitle translation model can significantly improve the translation quality and translation accuracy of the subtitle content.

[0128] S304: Reconstruct the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file.

[0129] Considering that the subtitle translation model may have problems such as overfitting and hallucination, resulting in unstable output content; for example, the output content may have formatting anomalies such as missing line breaks (that is, multiple lines of translated content are not wrapped and displayed, but merged and displayed on a single line), or content anomalies such as missing numbers and labels. In order to correctly parse the output content of the subtitle translation model to obtain translation content that conforms to the content format, the embodiments of the present application support content format parsing of the output content of the subtitle translation model, effectively improving the parsing accuracy of the translated content.

[0130] In a specific implementation, a computer device invokes a subtitle translation model and, under the guidance of translation prompts, translates each subtitle content to obtain the output content of the subtitle translation model. This output content may include at least one subtitle translation result, with at least one subtitle translation result displayed on a new line. A subtitle translation result may correspond to the translation content of a single subtitle content, or, due to issues such as model overfitting, may combine the translation content of multiple subtitle contents. To this end, the computer device performs content format analysis on the output content of the subtitle translation model to obtain a parsed result. Specifically, content format analysis is performed on each subtitle translation result to extract translation content that conforms to the content format.

[0131] In an embodiment of the present application, in order to improve the accuracy of parsing the output content of the subtitle translation model, the embodiment of the present application supports multi-rule-based regular expressions to implement content format parsing of the output content of the subtitle translation model. The so-called multi-rule regular expression refers to the use of multiple different regular expressions to parse the content format of the output content. Among them, regular expressions, or regular rules, are a tool for matching, searching, and replacing in character strings (such as subtitle translation results). In an embodiment of the present application, at least one regular expression can be configured that can match the translation content that conforms to the content format from the subtitle translation results. Specifically, the process of content parsing processing of the output content of the subtitle translation model by a computer device based on at least one regular expression can be seen in Figure 6 ;like Figure 6 As shown in the figure: at least one pre-configured regular expression is obtained; each subtitle translation result included in the output content is parsed using the at least one regular expression to obtain a parsing result. If any subtitle translation result matches a regular expression, the parsing result corresponding to the subtitle translation result is a successful parsing, and the subtitle translation result is used as the translation content of the subtitle content; if any subtitle translation result does not match any of the at least one regular expression, the parsing result corresponding to the subtitle translation result is a failed parsing.

[0132] The regular expression includes multiple characters (such as Chinese characters, English characters, symbols, punctuation marks, or numbers, etc.) or character strings. In this way, when the subtitle translation result is parsed using the regular expression, the multiple characters included in the regular expression are matched with the characters in the subtitle translation result in order. If both the characters and the character order match, it is determined that the subtitle translation result hits the regular expression, that is, the subtitle translation result is parsed successfully.

[0133] Furthermore, in order to ensure that the subtitle content that has not been parsed out for translation can be translated correctly, the embodiment of the present application also designs a fallback downgrade strategy, so that the subtitle content that failed to be parsed can be re-translated through the downgrade strategy, thereby ensuring the translation success rate for the first subtitle file. Among them, the downgrade strategy can include a model downgrade sub-strategy and a service downgrade sub-strategy; the model downgrade sub-strategy refers to calling other large language models with content generation capabilities other than the subtitle translation model to re-translate the target subtitle content; the so-called service downgrade sub-strategy refers to calling a service with content generation capabilities to re-translate the target subtitle content. Among them, there is a priority between the model downgrade sub-strategy and the service downgrade sub-strategy, such as giving priority to the model downgrade sub-strategy for re-translation processing, and when the translation result fails to parse the content format, re-translation processing is performed based on the service downgrade sub-strategy; it can ensure that the model downgrade sub-strategy with better translation quality is used first to improve the translation quality of the translated content.

[0134] In a specific implementation, if the parsing result indicates that the translation content of the target subtitle content has failed to be parsed, that is, there is translation content of the target subtitle content that has not been parsed from the output content (or subtitle translation result), the computer device calls the candidate translation tool to re-translate the target subtitle content to obtain the new translation content of the target subtitle content; the target subtitle content belongs to N subtitle contents and the number of target subtitle contents is one or more. In this case, the computer device will reconstruct the new translation content of the target subtitle content, the multiple translation contents that have been successfully parsed among the N subtitle contents, and the timeline corresponding to each subtitle content to generate a second subtitle file. Among them, the candidate translation tool mentioned above is a large language model with content generation capabilities other than the subtitle translation content defined in the model degradation sub-strategy, or a service with content generation capabilities defined in the service degradation sub-strategy, etc. The embodiment of the present application does not limit the tool type of the candidate translation tool. It is worth noting that, as mentioned above, the subtitle translation model is configured in a plug-in manner into the application service or plug-in that integrates the subtitle processing solution provided by the embodiment of the present application; therefore, the candidate translation tool can also be configured as a plug-in into the application service or plug-in to facilitate the computer device to quickly call the candidate translation tool to re-translate the target subtitle content. Among them, depending on the tool type of the candidate translation tool, the processing logic of the candidate translation tool for re-translating the target subtitle content is also different, but the re-translation process is intended to translate the target subtitle from one language type to another language type. The embodiment of the present application does not limit the specific implementation process of the re-translation process.

[0135] If the parsing result indicates that the translation content of the N subtitle contents is successfully parsed, each of the multiple subtitle translation results output by the subtitle translation model is directly used as the translation content, without calling the candidate translation tool for re-translation processing. In this case, the computer device reconstructs the N successfully parsed translation contents of the N subtitle contents and the timeline corresponding to each subtitle content to generate a second subtitle file.

[0136] As described above, the playback time of subtitles in the multimedia resource screen needs to be synchronized with the timeline of the multimedia resource screen content; therefore, after obtaining the translation content output for each subtitle content based on step S304, the embodiment of the present application also reconstructs the translation content of each subtitle content and the timeline corresponding to each subtitle content to generate a second subtitle file. The second subtitle file includes N subtitle information, each of the N subtitle information includes translation content and the timeline corresponding to the translation content; each translation content corresponds to the subtitle content corresponding to the corresponding timeline in the first subtitle file, thereby accurately translating the first subtitle file and generating the second subtitle file.

[0137] Among them, the process of reconstructing and generating the second subtitle file may include: as mentioned above, each subtitle content includes subtitle text and number, the subtitle text is the specific content that needs to be displayed on the display screen, and the number is used to represent the playback order of the subtitle text in the multimedia resource, specifically representing the playback order of the subtitle text in the multiple consecutive subtitle texts in the first subtitle file. Based on this, the computer device can obtain the time axis corresponding to each subtitle content from the N time axes obtained by the extraction process based on the number included in the translation content corresponding to each subtitle content (obtained directly through the translation process of the subtitle translation model, or translated through the candidate translation tool); then, the translation content of each subtitle content and the time axis corresponding to the corresponding subtitle content are combined to obtain N translation information; finally, the second subtitle file is generated based on the N translation information.

[0138] It should be noted that the above is introduced based on the example of indexing the time axis according to the number in the subtitle content. In other implementations, it is considered that the translation content of each subtitle content also includes the translation text and the number; since the number is the number 1, 2, 3... the subtitle translation model also translates the number as the number 1, 2, 3..., which can ensure that the number in the translation content and the number in the subtitle content are the same. Therefore, the embodiment of the present application also supports the computer device to obtain the time axis corresponding to each translation content from the N time axes obtained by the extraction process according to the number in each translation content; and then combine the translation content of each subtitle content and the time axis corresponding to each translation content to generate a second subtitle file.

[0139] In summary, on the one hand, in an embodiment of the present application, after obtaining the first subtitle file to be processed or translated, the N subtitle information included in the first subtitle file will be extracted from the time axis before translation; thereby ensuring that when the subsequent subtitle translation model performs translation processing, there is no need to translate the time axis, and only the subtitle content needs to be translated, which significantly saves software and hardware translation resources and improves translation efficiency. On the other hand, the embodiment of the present application constructs a translation prompt information exclusive to subtitles, and the translation prompt information defines a variety of specifications that help improve translation quality and accuracy, so that the subtitle translation model translates each subtitle content under the prompt of the translation prompt information; because the translation prompt information defines a variety of specifications that help improve translation quality and accuracy, the subtitle translation model performs translation processing under the prompt of the translation prompt information, which can significantly improve the accuracy and quality of the translation content of the subtitle content. On the other hand, after the translation processing obtains the translation content of each subtitle content, each translation content and the time axis corresponding to each subtitle content are reconstructed, and the rapid generation of the second subtitle file can be achieved without tedious operations, significantly improving the accuracy and quality of the translation content of the subtitle content.

[0140] Figure 7 A flowchart of another subtitle processing method provided by an exemplary embodiment of the present application is shown; the subtitle processing method can be executed by a computer device, which can be Figure 2 The terminal and / or server in the system shown. The subtitle processing method may include but is not limited to steps S701-S705:

[0141] S701: Obtain a first subtitle file.

[0142] S702: Extracting the time axis of each subtitle information to obtain the subtitle content of each subtitle information.

[0143] It should be noted that the specific implementation process shown in steps S701-S702 and Figure 3 The specific implementation process shown in steps S301-S302 in the illustrated embodiment is the same, and reference may be made to the relevant description of the specific implementation process shown in steps S301-S302, which will not be repeated here.

[0144] S703: Grouping N subtitle contents to obtain at least one subtitle group; each subtitle group includes at least one subtitle content.

[0145] After extracting N subtitle contents from N subtitle information based on the aforementioned step S702, the embodiment of the present application also supports splitting the N subtitle contents into subtitles corresponding to different screen scenes. In this way, the subtitle translation model can perform context summary based on the subtitle group, obtain the contextual semantics describing the screen scene corresponding to the subtitle group, and thus better translate the subtitle contents in the subtitle group, while controlling the length of the content input to the subtitle translation model while retaining semantic coherence, greatly improving the translation quality and translation efficiency of the subtitle content.

[0146] The embodiment of the present application supports grouping subtitle content based on the time axis to realize dividing the subtitle content into at least one subtitle group corresponding to at least one screen scene. For example, the general process of the computer device grouping subtitle content based on the time axis can be seen in Figure 8 ;like Figure 8 The steps of grouping subtitle content include but are not limited to s11-s13:

[0147] s11: Get the window sliding strategy.

[0148] s12: Perform window sliding processing on the N time axes according to the window sliding strategy to obtain at least one time axis group.

[0149] In steps s11-s12, the window sliding strategy, or context window sliding mechanism, is used to represent the rules for grouping N subtitle content based on N time axes. The embodiment of the present application does not limit the specific content of the window sliding strategy; the window sliding strategy includes but is not limited to: grouping N subtitle content based on N time axes according to a time threshold; or grouping N subtitle content based on N time axes according to a time threshold and a quantity threshold, etc. The following describes the process of performing window sliding processing according to these two window sliding strategies to obtain at least one time axis group; wherein:

[0150] (1) The window sliding strategy is to group N subtitle contents based on N time axes according to the time threshold.

[0151] Specifically, the computer device obtains a time threshold according to a window sliding strategy. The time threshold is a time period, such as 1 second 100 milliseconds. Then, starting from the first time axis of the N time axes, a recursive calculation is performed to determine whether the time interval between adjacent time axes is greater than the time threshold. The time interval between adjacent time axes is the time interval between the end time of the previous time axis and the start time of the next time axis. For example, if the end time of the first time axis is "00:00:37,700" and the start time of the second time axis is "00:00:60,700", then the time interval between the first and second time axes is 23 seconds. If the i-th time interval is greater than or equal to the time threshold, the time axis preceding the i+1-th time axis of the N time axes is grouped into a time axis group, specifically, the first time axis group. i is an integer, and 1≤i≤N-1. Then, starting from the first time axis (i.e., the i+1-th time axis) of at least one ungrouped time axis in the N subtitle content, a recursive calculation is performed to determine whether the time interval between adjacent time axes is greater than the time threshold. If a time axis meets the time threshold, specifically, the time interval between the time axis and the adjacent time axis before the time axis is greater than or equal to the time threshold, then the time axis from the i+1th time axis to the time axis before the time axis is divided into a time axis group, specifically, the second time axis group. Repeat the above steps until all time axes in the N time axes are grouped, or no ungrouped time axes in the N time axes meet the time threshold, then the remaining time axes are divided into a time axis group, such as the third time axis group.

[0152] Among them, the embodiments of the present application do not limit the method of determining the time threshold, including but not limited to: pre-set according to empirical values; or, determined according to the maximum gap algorithm; etc. Specifically, if the window sliding strategy indicates that the maximum time interval between adjacent time axes in N time axes is used as the time threshold, then the time threshold is determined according to the maximum gap algorithm. Among them, the process of determining the time threshold according to the maximum gap algorithm may include: respectively calculating the time intervals between adjacent time axes in N times, obtaining N-1 time intervals, and determining the maximum time interval in the N-1 time intervals as the time threshold; taking into account that the intervals between non-continuous picture scenes in film and television scenes are relatively long, the maximum time interval is used as the time threshold to achieve scene splitting of N subtitle contents, thereby improving the correctness of picture scene splitting, thereby contributing to the accuracy of subsequent subtitle translation.

[0153] For example, if Figure 9As shown, assuming N = 100, starting from the T1th time axis of the N time axes, recursively calculating the time interval between the Tith time axis and the Ti+1th time axis is equal to or greater than the time threshold, and the time intervals between adjacent time axes before the Tith time axis are all less than the time threshold, the first time axis group is divided into [T1th time axis, Tith time axis]. Then, starting from the Ti+1th time axis, recursively calculating the time interval between the Tjth time axis and the Tj+1th time axis is equal to or greater than the time threshold, and the time intervals between adjacent time axes from the Ti+1th time axis to the Tjth time axis are all less than the time threshold, the second time axis group is divided into [Ti+1th time axis, Tjth time axis]. Then, if there is no time interval between adjacent time axes from the Tj+1th time axis to the TNth time axis that is equal to or greater than the time threshold, the third time axis group is divided into [Tj+1th time axis, TNth time axis].

[0154] It should be noted that the above is introduced based on the example of grouping N subtitle contents based on N time axes according to the time threshold, and the time threshold used when dividing each time axis group is the same. However, when the time threshold is determined according to the maximum gap algorithm, and there is only one maximum time interval in the N-1 time intervals, after the first time axis group is obtained, all the remaining time axes can only be divided into one time axis group. Therefore, in order to improve the correctness of scene splitting, the embodiment of the present application also supports different time thresholds used when dividing each time axis group. For example, after the first time axis group is obtained, a new time threshold is recalculated from the remaining time axes in the N time axes according to the maximum gap algorithm, and then the second time axis is divided according to the new time threshold; this greatly improves the accuracy of scene splitting, thereby helping the subsequent subtitle translation model to translate more accurate translation content based on smaller picture scenes.

[0155] (2) The window sliding strategy is to group N subtitle contents based on N time axes according to the time threshold and quantity threshold.

[0156] like Figure 10As shown, the computer device obtains the time threshold according to the window sliding strategy; wherein, the method for determining the time threshold can refer to the relevant description of the method for determining the time threshold in the above implementation method (1), which will not be repeated here. Then, the computer device parses the start time and end time of the N time axes to facilitate the calculation of the time interval between adjacent time axes, and recursively calculates whether the time interval between adjacent time axes is greater than the time threshold starting from the first time axis of the N time axes; if the time interval between the current time axis and the previous time axis is less than the time threshold, the time interval between the next time axis and the next time axis is continued to be parsed. If it is detected that the i-th time interval is equal to or greater than the time threshold, the time axis before the i+1-th time axis in the N time axes is divided into a time axis group; i is an integer, and 1≤i≤N-1.

[0157] In the case where the time threshold used in each time axis group is the same, and the time threshold is determined by calculating the maximum gap algorithm for N time axes, there may not be (or may be) a time axis that meets the time threshold in the remaining time axes in the N time axes (that is, the time interval between the time axis and the adjacent time axis is less than the time threshold). In the case where there is no time axis that meets the time threshold in the remaining time axes from the N time axes, if all the remaining time axes in the N time axes are directly divided into one time axis group, then it may be because the time axis group contains too many time axes, causing greater computing pressure to the subtitle translation model, and because of the roughness of the scene splitting, resulting in poor translation quality. Based on this, the embodiment of the present application also supports the introduction of a quantity threshold, so that the time axis remaining in the N time axes continues to split the scene in combination with the time threshold and the quantity threshold; wherein, the quantity threshold can be understood as the number of multiple continuous time axes, and the number of time axes and the number of rows of corresponding subtitle content are the same. In short, after creating the first timeline group based on the time threshold, it can be understood as dividing the remaining timeline groups in the N timelines into (or storing them as) a new picture scene; then, the new picture scene can be further split into sub-scenes according to the time threshold and the quantity threshold.

[0158] The sub-scene splitting process may include traversing the time axis in the new screen scene to achieve sub-scene splitting; specifically, the process includes: detecting whether each time axis meets a time threshold or a quantity threshold from multiple ungrouped time axes in the N time axes (i.e., the remaining time axes except those divided into the first time axis group). If there is a target time axis that meets the time threshold or the quantity threshold, then the time axis before the target time axis in the multiple ungrouped time axes in the N time axes is stored as (or divided into) a new time axis group except the first time axis. Specifically, if it is detected that a certain time axis meets the time threshold (i.e., the time interval between the certain time axis and the adjacent previous time axis is equal to or greater than the time threshold), then the certain time axis is determined as the target time axis; if the certain time axis does not meet the time threshold but meets the quantity threshold (i.e., the number of time axes between the first ungrouped time axis in the N time axes and the certain time axis is equal to or greater than the quantity threshold), then the certain time axis is determined as the target time axis; if the certain time axis does not meet the quantity threshold, then the certain time axis cannot be determined as the target time axis, and the judgment continues on the next time axis.

[0159] Conversely, if none of the ungrouped timelines in the N timelines meet the time threshold or quantity threshold, the ungrouped timelines in the N timelines are grouped into one timeline group. In other words, if none of the ungrouped timelines in the N timelines have a target timeline with a large time interval, or if the number of ungrouped timelines is small, the ungrouped timelines in the N timelines can be grouped into one timeline group.

[0160] It should be noted that the aforementioned implementation method (1) and implementation method (2) are only specific implementation processes of the two window sliding strategies provided in the embodiment of the present application. It should be understood that the purpose of the embodiment of the present application to group N time axes is to split the picture scenes described by N subtitle contents, ensure the contextual semantic coherence of smaller picture scenes, and control the content length of the input subtitle translation model to improve the efficiency of subtitle translation. Therefore, as long as the window sliding strategy can achieve the grouping purpose of the embodiment of the present application, it falls within the protection scope of the embodiment of the present application.

[0161] s13: Combine the subtitle contents corresponding to each time axis in each time axis group to obtain at least one subtitle group.

[0162] After dividing the N time axes into at least one time axis group based on the aforementioned steps s11-s12, the embodiment of the present application further groups the N subtitle contents according to the at least one time axis group to obtain at least one subtitle group; each subtitle group includes at least one continuous subtitle content. For example, the N time axes are time axis 1, time axis 2, time axis 3, time axis 4, time axis 5, and time axis 6; if the first time axis group includes [time axis 1, time axis 2, time axis 3] and the second time axis group includes [time axis 4, time axis 5, time axis 6], then the N subtitle contents are grouped to obtain two subtitle groups, namely, the first subtitle group includes [subtitle content 1 corresponding to time axis 1, subtitle content 2 corresponding to time axis 2, subtitle content 3 corresponding to time axis 3], and the second subtitle group includes [subtitle content 4 corresponding to time axis 4, subtitle content 5 corresponding to time axis 5, subtitle content 6 corresponding to time axis 6].

[0163] S704: Obtain translation prompt information of the first subtitle file, and translate the subtitle content in each subtitle group according to the translation prompt information to obtain translation content of each subtitle content.

[0164] After the N subtitle contents are divided into at least one subtitle group according to the picture scene based on the aforementioned step S703, the embodiment of the present application supports the construction of personalized translation prompt information for each subtitle group, and calls the subtitle translation model to translate and process each subtitle content in the corresponding subtitle group under the prompt of the translation prompt information of the subtitle group, aiming to obtain the translation content of each subtitle content in the subtitle group translated according to a smaller picture scene. It should be noted that the process of the subtitle translation model translating and processing each subtitle content in the subtitle group under the prompt of the translation prompt information of the subtitle group is similar to the aforementioned Figure 3 The subtitle translation model described in step S303 in the illustrated embodiment translates each subtitle content in the first subtitle file under the prompt of the translation prompt information constructed for the first subtitle file. The implementation logic is the same and will not be described in detail here.

[0165] Among them, the specifications defined in the personalized translation prompt information built for each subtitle group, and Figure 3In the illustrated embodiment, the specifications defined in the translation prompt information constructed when the N subtitle contents are taken as a whole as described in step S303 are the same, and reference can be made to the aforementioned related description. The difference between the two is that the translation prompt information of the subtitle group includes the group information of the subtitle group (the group information of different subtitle groups is different), and the group information may include but is not limited to the number of lines of the subtitle content included in the subtitle group. In this way, the subtitle translation model, prompted by the number of lines of the translation prompt information of the subtitle group, can prompt the subtitle translation model to pay attention to the consistency between the number of subtitle translation results output and the number of lines, thereby avoiding the situation where the translation content of multiple subtitle contents is merged in the subtitle translation result, and improving the accuracy of the output content of the subtitle translation model.

[0166] Exemplarily, the process of constructing translation prompt information for a target subtitle group (any one of at least one subtitle group) and performing translation processing may include: a computer device obtains initial prompt information of a first subtitle file, wherein the initial prompt information defines content format specifications, context summary specifications, terminology specifications, work content specifications, work example specifications, and model role specifications. Then, the group information of the target subtitle group is obtained, which includes the number of subtitle contents included in the target subtitle group. Finally, the group information of the target subtitle group is added to the initial prompt information to obtain the target translation prompt information corresponding to the target subtitle group. In this case, the computer device calls the subtitle translation model to translate the subtitle contents in the target subtitle group under the prompt of the target translation prompt information, and obtains the translated content of the subtitle contents in the target subtitle group.

[0167] S705: Reconstruct the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file.

[0168] It should be noted that the specific implementation process shown in step S705 and Figure 3 The specific implementation process shown in step S304 in the illustrated embodiment is the same. Please refer to the relevant description of the specific implementation process shown in step S304, and no further details will be given here.

[0169] To better understand the complete solution process shown in the above steps S701-S705, Figure 11 This paper introduces the complete process of subtitle processing. Figure 11As shown in the following: ① Timeline Extraction: After a computer device obtains a first subtitle file, it first extracts the timeline of the N subtitle information in the first subtitle file, resulting in N sequentially arranged timelines and N sequentially arranged subtitle contents, with each of the N timelines corresponding to the N subtitle contents. ② Grouping: Based on the timeline, the N subtitle contents are divided into at least one subtitle group according to a window sliding strategy. Each subtitle group includes at least one continuous subtitle content, and the subtitle groups are also continuous. ③ Constructing Translation Prompt Information for Subtitle Groups: Constructing personalized translation prompt information for each subtitle group. ④ Translation Processing: Utilizing a translation subtitle model, the computer device performs translation processing on each subtitle content in each subtitle group, guided by the translation prompt information for each subtitle group, to obtain output content for each subtitle group, including at least one subtitle translation result. ⑤ Result Parsing: The output content is parsed using a multi-rule regular expression to obtain a parsing result. ⑥ Degradation Processing: If the parsing result indicates that the target subtitle content failed to be parsed, the target subtitle content is re-translated using a degrading strategy to obtain a new translation of the target subtitle content. ⑦ Timeline reconstruction: Based on the translation content of N subtitles and the timeline, the subtitle file is reconstructed to generate a second subtitle file.

[0170] In summary, on the one hand, the embodiment of the present application extracts the time axis before translation, and only needs to translate the subtitle content, which significantly saves the translation resources of software and hardware and improves translation efficiency. On the other hand, it supports the construction of translation prompt information exclusive to subtitles, and the translation prompt information defines a variety of specifications that help improve translation quality and accuracy. Therefore, the subtitle translation model performs translation processing under the prompt of the translation prompt information, which can significantly improve the accuracy and quality of the translation content of the subtitle content. On the other hand, before calling the subtitle translation model to translate the subtitle content, the embodiment of the present application supports grouping N subtitle contents based on the time axis, aiming to split the larger range of picture scenes described by the N subtitle contents into smaller picture scenes, so that the subtitle translation model can summarize the contextual semantics of the more accurate picture scenes, thereby assisting the translation of the subtitle content in the subtitle group based on the contextual semantics, significantly improving the translation accuracy of the subtitle content; and, since the number of subtitle contents included in the subtitle group input to the subtitle translation model is small each time, the translation efficiency of the subtitle translation model is improved.

[0171] The method of the embodiment of the present application is described in detail above. In order to facilitate the above-mentioned scheme of the embodiment of the present application to be better implemented, accordingly, the device of the embodiment of the present application is provided below. In the embodiment of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuit or memory) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit that includes the module or unit function.

[0172] Figure 12 A schematic diagram of a subtitle processing device provided by an exemplary embodiment of the present application is shown; the subtitle processing device can be used to perform Figure 3 and Figure 7 Some or all of the steps in the method embodiment shown. Figure 12 , the device includes the following units:

[0173] The acquiring unit 1201 is configured to acquire a first subtitle file, wherein the first subtitle file includes N subtitle information, each subtitle information including subtitle content and a time axis corresponding to the subtitle content; N is a positive integer;

[0174] The processing unit 1202 is configured to extract the time axis of each subtitle information to obtain the subtitle content of each subtitle information;

[0175] The processing unit 1202 is further configured to obtain translation prompt information of the first subtitle file and, under the guidance of the translation prompt information, translate each subtitle content to obtain a translation of each subtitle content, wherein each subtitle content and its translation are in a different language.

[0176] The processing unit 1202 is further configured to reconstruct the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file.

[0177] In one implementation, the translation process is performed by calling a subtitle translation model; the translation prompt information defines specifications that the subtitle translation model needs to follow when performing the translation process; the specifications include at least one of the following:

[0178] Content format specification, which is used to instruct the subtitle translation model to translate the subtitle content in accordance with the content format of the subtitle content; and

[0179] Context summary specification: The context summary specification is used to instruct the subtitle translation model to generate context semantics for multiple consecutive subtitle contents and use the context semantics to guide the subtitle translation model to perform translation processing;

[0180] The terminology specification is used to indicate that the subtitle translation model obtains preset terms from the terminology library and performs the translation process on the subtitle content.

[0181] In one implementation, after the extraction process, a time axis in each subtitle information is obtained, and N time axes are arranged in chronological order; the processing unit 1202 is further configured to:

[0182] Obtain a window sliding strategy, where the window sliding strategy is used to represent a rule for grouping N subtitle contents based on N time axes;

[0183] Perform window sliding processing on N time axes according to the window sliding strategy to obtain at least one time axis group; each time axis group includes at least one time axis;

[0184] The subtitle contents corresponding to each time axis in each time axis group are respectively combined to obtain at least one subtitle group; each subtitle group includes at least one subtitle content.

[0185] In one implementation, the processing unit 1202 is configured to obtain translation prompt information of the first subtitle file, and translate each subtitle content under the guidance of the translation prompt information. When obtaining the translation content of each subtitle content, the processing unit 1202 is specifically configured to:

[0186] Obtaining initial prompt information of a first subtitle file; the initial prompt information defines content format specifications, context summary specifications, and terminology specifications;

[0187] Obtaining group information of a target subtitle group, wherein the group information includes the number of subtitle contents included in the target subtitle group; the target subtitle group is any one of the at least one subtitle group;

[0188] Add the target subtitle group's group information to the initial prompt information to obtain the target translation prompt information corresponding to the target subtitle group;

[0189] The subtitle translation model is called to translate each subtitle content in the target subtitle group under the prompt of the target translation prompt information to obtain the translation content of each subtitle content in the target subtitle group.

[0190] In one implementation, the processing unit 1202 is configured to perform window sliding processing on N time axes according to the window sliding strategy to obtain at least one time axis group, specifically to:

[0191] Obtain the time threshold and quantity threshold according to the window sliding strategy;

[0192] Starting from the first time axis among the N time axes, recursively calculate whether the time interval between adjacent time axes is greater than the time threshold;

[0193] If the i-th time interval is equal to or greater than the time threshold, the time axis before the i+1-th time axis among the N time axes is divided into a time axis group; i is an integer, and 1≤i≤N-1;

[0194] From the multiple ungrouped time axes in the N time axes, detecting whether each time axis meets a time threshold or a quantity threshold;

[0195] If there is a target time axis that meets the time threshold or the quantity threshold, the time axis that is located before the target time axis among the multiple ungrouped time axes in the N time axes is divided into one time axis group.

[0196] In one implementation, the processing unit 1202 is further configured to:

[0197] If there is no time axis that meets the time threshold or the quantity threshold among the multiple ungrouped time axes in the N time axes, the multiple ungrouped time axes in the N time axes are divided into one time axis group.

[0198] In one implementation, the window sliding strategy is used to indicate that the maximum time interval between adjacent time axes among N time axes is used as the time threshold. The process of determining the time threshold includes:

[0199] Calculate the time intervals between adjacent time axes in N time axes respectively to obtain N-1 time intervals;

[0200] The maximum time interval among the N-1 time intervals is determined as the time threshold.

[0201] In one implementation, the processing unit 1202 is configured to translate each subtitle content under the prompt of the translation prompt information, and to obtain the translation content of each subtitle content by:

[0202] Calling the subtitle translation model to translate each subtitle content under the prompt of the translation prompt information to obtain the output content of the subtitle translation model;

[0203] Parse the output content in a format that matches the output content and obtain the parsing result;

[0204] If the parsing result indicates that the translation content of the target subtitle content fails to be parsed, the candidate translation tool is called to re-translate the target subtitle content to obtain a new translation content of the target subtitle content; the target subtitle content belongs to N subtitle contents;

[0205] The processing unit 1202 is configured to reconstruct the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file, specifically for:

[0206] Reconstructing the new translation content of the target subtitle content, the multiple successfully parsed translation contents among the N subtitle contents, and the time axis corresponding to each subtitle content to generate a second subtitle file.

[0207] In one implementation, the output content includes at least one subtitle translation result; the processing unit 1202 is configured to parse the output content for a content format, and upon obtaining the parsed result, specifically to:

[0208] Get at least one regular expression;

[0209] Use at least one regular expression to parse each subtitle translation result to obtain a parsed result; wherein,

[0210] If any subtitle translation result matches a regular expression, the parsing result corresponding to any subtitle translation result is parsed successfully, and any subtitle translation result is regarded as a translation content;

[0211] If any subtitle translation result does not match at least one regular expression, the parsing result corresponding to any subtitle translation result is parsing failure.

[0212] In one implementation, each subtitle content includes subtitle text and a number, where the number is used to represent the playback order of the subtitle text in the multimedia resource; the translation content includes translation text and a number;

[0213] The processing unit 1202 is configured to reconstruct the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file, specifically for:

[0214] Obtaining the time axis corresponding to each subtitle content from the N time axes according to the number included in the translation content corresponding to each subtitle content;

[0215] Combine the translation content of each subtitle content with the time axis corresponding to the subtitle content to obtain N translation information;

[0216] A second subtitle file is generated based on the N translation information.

[0217] According to one embodiment of the present application, Figure 12The various units in the subtitle processing device shown can be individually or completely combined into one or several other units to form a structure, or one (or some) of the units can be further divided into multiple functionally smaller units to form a structure, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the subtitle processing device can also include other units. In actual applications, these functions can also be implemented with the assistance of other units and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, the subtitle processing device can be executed by running on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM). Figure 3 and Figure 7 The computer program of each step involved in the corresponding method shown is constructed as follows Figure 12 The subtitle processing device shown in and the subtitle processing method of the embodiment of the present application are implemented. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0218] In an embodiment of the present application, the first subtitle file to be processed or translated includes N subtitle information, each subtitle information includes subtitle content and the time axis corresponding to the subtitle content, and N is a positive integer. Considering that the time axis corresponding to each subtitle content in the first subtitle file does not need to be translated, the time axis in each subtitle information is extracted to obtain the subtitle content in each subtitle information; in this way, the time axis is extracted before translation, and only the subtitle content needs to be translated, which significantly saves software and hardware translation resources and improves translation efficiency. Further, it supports the use of translation prompt information constructed for the first subtitle file, so that the subtitle translation model translates each subtitle content under the prompt of the translation prompt information; since the translation prompt information defines a variety of specifications that help improve translation quality and accuracy, the subtitle translation model performs translation processing under the prompt of the translation prompt information, which can significantly improve the accuracy and quality of the translation content of the subtitle content. Furthermore, after the translation process obtains the translation content of each subtitle content, each translation content and the time axis corresponding to each subtitle content are reconstructed, and the second subtitle file can be quickly generated without tedious operations. It can be seen that the embodiment of the present application calls the subtitle translation model to translate the subtitle content in the first subtitle file according to the prompt of the translation prompt information. Not only does it improve the translation accuracy and quality by relying on the translation prompt information, but it also significantly improves the translation efficiency by calling the AI (Artificial Intelligence) model for subtitle translation and only translating the subtitle content.

[0219] Figure 13 FIG2 shows a schematic diagram of a computer device provided by an exemplary embodiment of the present application. Figure 13 , the computer device includes a processor 1301, a communication interface 1302 and a computer-readable storage medium 1303. The processor 1301, the communication interface 1302 and the computer-readable storage medium 1303 can be connected via a bus or other means. The communication interface 1302 is used to receive and send data. The computer-readable storage medium 1303 can be stored in the memory of the computer device, the computer-readable storage medium 1303 is used to store computer programs, and the processor 1301 is used to execute the computer programs stored in the computer-readable storage medium 1303. The processor 1301 (or CPU) is the computing core and control core of the computer device, which is suitable for implementing one or more computer programs, and is specifically suitable for loading and executing one or more computer programs to implement corresponding method processes or corresponding functions.

[0220] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the processing system of the computer device. In addition, one or more computer programs suitable for being loaded and executed by the processor 1301 are also stored in the storage space. These computer programs can be one or more computer programs. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.

[0221] In one embodiment, the computer-readable storage medium stores one or more computer programs; the processor 1301 loads and executes the one or more computer programs stored in the computer-readable storage medium to implement the corresponding steps in the above-mentioned subtitle processing method embodiment; in a specific implementation, the one or more computer programs in the computer-readable storage medium are loaded by the processor 1301 and execute the steps of each embodiment of the present application; wherein, the steps of each embodiment of the present application can be referred to the relevant description of the aforementioned embodiments, and are not repeated here.

[0222] Based on the same inventive concept, the principles and beneficial effects of the computer device provided in the embodiment of the present application in solving the problem are similar to the principles and beneficial effects of the subtitle processing method in the method embodiment of the present application in solving the problem. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0223] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned subtitle processing method is implemented.

[0224] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0225] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs (one or more). When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer program can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer program can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. Available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, digital video disks (DVDs)), or semiconductor media (eg, solid state disks (SSDs)).

[0226] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A subtitle processing method, characterized in that: include: Obtaining a first subtitle file, wherein the first subtitle file includes N subtitle information, each of the subtitle information including subtitle content and a time axis corresponding to the subtitle content; N is a positive integer; Extracting the time axis of each subtitle information to obtain the subtitle content of each subtitle information; Obtaining translation prompt information of the first subtitle file, and translating each subtitle content under the prompt of the translation prompt information to obtain translation content of each subtitle content, where each subtitle content and its translation content are in a different language; The translation content of each subtitle content and the time axis corresponding to each subtitle content are reconstructed to generate a second subtitle file.

2. The method according to claim 1, wherein The translation process is performed by calling a subtitle translation model; the translation prompt information defines the specifications that the subtitle translation model needs to follow when performing the translation process; the specifications include at least one of the following: a content format specification, wherein the content format specification is used to instruct the subtitle translation model to perform the translation process according to the content format of the subtitle content; as well as, A context summary specification, wherein the context summary specification is used to instruct: the subtitle translation model to generate context semantics for a plurality of consecutive subtitle contents, and use the context semantics to guide the subtitle translation model to perform the translation process; A terminology specification is used to indicate that the subtitle translation model obtains preset terms from a term library to perform the translation process on the subtitle content.

3. The method according to claim 1 or 2, wherein: After the extraction process, a time axis in each of the subtitle information is obtained, and the N time axes are arranged in chronological order. After the time axis in each of the subtitle information is extracted to obtain the subtitle content in each of the subtitle information, the method further includes: Obtaining a window sliding strategy, where the window sliding strategy is used to represent a rule for grouping the N subtitle contents based on the N time axes; Performing window sliding processing on the N time axes according to the window sliding strategy to obtain at least one time axis group; each time axis group includes at least one time axis; The subtitle contents corresponding to the time axes in each time axis group are respectively combined to obtain at least one subtitle group; each subtitle group includes at least one subtitle content.

4. The method according to claim 3, wherein The step of obtaining translation prompt information of the first subtitle file and translating each subtitle content under the prompt of the translation prompt information to obtain translation content of each subtitle content includes: Obtaining initial prompt information of the first subtitle file; wherein the initial prompt information defines content format specifications, context summary specifications, and terminology specifications; Acquire group information of a target subtitle group, wherein the group information includes the number of subtitle contents included in the target subtitle group; the target subtitle group is any one of at least one of the subtitle groups; Adding the group information of the target subtitle group to the initial prompt information to obtain target translation prompt information corresponding to the target subtitle group; The subtitle translation model is called to translate each of the subtitle contents in the target subtitle group under the prompt of the target translation prompt information to obtain the translation content of each of the subtitle contents in the target subtitle group.

5. The method according to claim 3, wherein The performing window sliding processing on the N time axes according to the window sliding strategy to obtain at least one time axis group includes: Obtaining a time threshold and a quantity threshold according to the window sliding strategy; Starting from the first time axis among the N time axes, recursively calculating whether the time interval between adjacent time axes is greater than the time threshold; If the i-th time interval is equal to or greater than the time threshold, the time axis before the i+1-th time axis among the N time axes is divided into one of the time axis groups; i is an integer, and 1≤i≤N-1; detecting, from a plurality of ungrouped time axes in the N time axes, whether each of the time axes meets the time threshold or the quantity threshold; If there is a target time axis that meets the time threshold or the quantity threshold, the time axis that is located before the target time axis among the multiple ungrouped time axes in the N time axes is divided into one of the time axis groups.

6. The method according to claim 5, wherein The method further comprises: If there is no time axis that meets the time threshold or the quantity threshold among the multiple ungrouped time axes in the N time axes, the multiple ungrouped time axes in the N time axes are divided into one time axis group.

7. The method according to claim 5, wherein The window sliding strategy is used to indicate that the maximum time interval between adjacent time axes in the N time axes is used as the time threshold; the process of determining the time threshold includes: Calculating the time intervals between adjacent time axes in the N time axes respectively to obtain N-1 time intervals; The maximum time interval among the N-1 time intervals is determined as the time threshold.

8. The method according to claim 1, wherein The step of translating each of the subtitle contents under the prompt of the translation prompt information to obtain the translated content of each of the subtitle contents includes: Calling a subtitle translation model to translate each subtitle content under the prompt of the translation prompt information to obtain output content of the subtitle translation model; Performing content format analysis on the output content to obtain a parsing result; If the parsing result indicates that the translation content of the target subtitle content fails to be parsed, calling a candidate translation tool to re-translate the target subtitle content to obtain a new translation content of the target subtitle content; the target subtitle content belongs to the N subtitle contents; The step of reconstructing the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file includes: Reconstructing the new translation content of the target subtitle content, the multiple successfully parsed translation contents among the N subtitle contents, and the time axis corresponding to each subtitle content to generate a second subtitle file.

9. The method according to claim 8, wherein The output content includes at least one subtitle translation result; and the content format of the output content is parsed to obtain the parsing result, including: Get at least one regular expression; Each of the subtitle translation results is parsed using at least one of the regular expressions to obtain a parsed result; wherein, If any of the subtitle translation results matches a regular expression, the parsing result corresponding to any of the subtitle translation results is considered a successful parsing, and any of the subtitle translation results is used as a translation content; If any of the subtitle translation results does not match at least one of the regular expressions, the parsing result corresponding to any of the subtitle translation results is parsing failure.

10. The method according to claim 1 or 8, wherein Each of the subtitle contents includes subtitle text and a number, wherein the number is used to represent the playback order of the subtitle text in the multimedia resource; the translation content includes translation text and a number; The step of reconstructing the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file includes: Obtaining the time axis corresponding to each of the subtitle contents from the N time axes according to the serial number included in the translation content corresponding to each of the subtitle contents; Combining the translation content of each subtitle content with the time axis corresponding to the subtitle content to obtain N translation information; A second subtitle file is generated based on the N pieces of translation information.

11. A subtitle processing device, characterized in that: include: An acquiring unit, configured to acquire a first subtitle file, wherein the first subtitle file includes N subtitle information, each of which includes subtitle content and a time axis corresponding to the subtitle content; N is a positive integer; a processing unit, configured to extract the time axis of each subtitle information to obtain the subtitle content of each subtitle information; The processing unit is further configured to obtain translation prompt information of the first subtitle file, and translate each subtitle content under the guidance of the translation prompt information to obtain translation content of each subtitle content, wherein each subtitle content and its translation content are in a different language; The processing unit is further configured to reconstruct the translation content of each subtitle content and the time axis corresponding to each subtitle content to generate a second subtitle file.

12. A computer device, characterized in that: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the subtitle processing method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the subtitle processing method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the subtitle processing method according to any one of claims 1 to 10 is implemented.

Images