Subtitle display method, browser, storage medium and terminal device

Abstract

The embodiment of the present invention discloses a subtitle display method, a browser, a storage medium and a terminal device, which are applied to the field of information processing technology based on artificial intelligence. When the current web page process started by the browser main process determines that the existing subtitles of the target video need to be converted into new subtitles, it will obtain a subtitle conversion model, and complete the subtitle conversion through the subtitle conversion model, and transmit the subtitle event object to the browser main process. When the browser main process determines that the display time of the new subtitle has arrived during the display of the target video, it notifies the current web page process to display the new subtitle on the display interface of the target video. In this way, the subtitle conversion model obtained by the current web page process in the browser can realize the real-time conversion of the existing subtitles in the target video without the need for a special translation team to convert the subtitles, which meets the real-time needs of users to watch videos of various language types.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology based on artificial intelligence, and in particular to a subtitle display method, a browser, a storage medium and a terminal device. Background Art

[0002] Subtitles refer to non-visual content such as dialogues in videos such as TV dramas, movies, variety shows, news and animations displayed in text form. They also refer to the text processed in the post-production of film and television works, the commentary text that appears at the bottom of the movie screen or TV screen, as well as various texts, such as the film's title, cast list, lyrics, dialogues, explanatory words, as well as character introductions, place names and years.

[0003] Generally, the audio content included in the video is displayed in the form of subtitles, so that the display of the subtitle text is synchronized with the display of the audio content. The content of the subtitle text can be consistent with the audio content, or it can be a translated text of the audio content. For example, if the language of the audio content is English, the subtitle text can be the corresponding Chinese subtitle, which makes it easier for users who do not understand the audio content in the video to understand the video content.

[0004] At present, the Chinese subtitles corresponding to foreign-language videos are generally translated by specialized people and integrated into the video files later. However, due to issues such as manpower and copyright, the number of translations of foreign-language videos is limited and the distribution channels are restricted. As a result, viewers can only passively wait for the release of translated foreign-language videos, which cannot meet users' needs to understand foreign-language videos. Summary of the invention

[0005] The embodiment of the present invention provides a subtitle display method, a browser, a storage medium and a terminal device, which realizes real-time conversion of subtitles through a subtitle conversion model.

[0006] An embodiment of the present invention provides a subtitle display method, comprising:

[0007] The current web page process determines that existing subtitles in the target video are to be converted into new subtitles, and the first language type corresponding to the existing subtitles is different from the second language type corresponding to the new subtitles;

[0008] The current web page process obtains a subtitle conversion model;

[0009] When the existing subtitles are converted into new subtitles through the subtitle conversion model, the current web page process transmits the subtitle event object to the browser main process;

[0010] When the browser main process determines that the display time of the new subtitles has arrived according to the subtitle event object, the current web page process is notified to display the new subtitles on the display interface of the target video.

[0011] Another aspect of the present invention further provides a browser, including:

[0012] a conversion determination unit, configured to determine that existing subtitles in a target video are to be converted into new subtitles, wherein a first language type corresponding to the existing subtitles is different from a second language type corresponding to the new subtitles;

[0013] Model unit, used to obtain the subtitle conversion model;

[0014] A conversion unit, configured to convert the existing subtitles into new subtitles through the subtitle conversion model, transmit the subtitle event object to the browser main process, and notify the display unit to display the subtitles when the browser main process determines that the display time of the new subtitles has arrived according to the subtitle event object;

[0015] A display unit is used to display the new subtitles on the display interface of the target video.

[0016] Another aspect of the embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of computer programs, wherein the computer programs are suitable for being loaded by a processor and executing the subtitle display method as described in one aspect of the embodiment of the present invention.

[0017] Another aspect of the embodiment of the present invention further provides a terminal device, including a processor and a memory;

[0018] The memory is used to store multiple computer programs, and the computer programs are used to be loaded by the processor and execute the subtitle display method as described in one aspect of an embodiment of the present invention; the processor is used to implement each computer program in the multiple computer programs.

[0019] It can be seen that in the method of this embodiment, when the current web page process started by the browser main process determines that the existing subtitles of the target video need to be converted into new subtitles, it will obtain a subtitle conversion model, and complete the subtitle conversion through the subtitle conversion model, and transmit the subtitle event object to the browser main process. When the browser main process determines that the display time of the new subtitle has arrived during the display of the target video, it notifies the current web page process to display the new subtitle on the display interface of the target video. In this way, the subtitle conversion model obtained by the current web page process in the browser can realize the real-time conversion of the existing subtitles in the target video without the need for a special translation team to convert the subtitles, which meets the real-time needs of users to watch videos of various language types. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 is a schematic diagram of a subtitle display method provided by an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a word display switching method provided by an embodiment of the present invention;

[0023] Figure 3 is a flow chart of a method for training a subtitle conversion model in one embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of a subtitle display method in an application embodiment of the present invention;

[0025] Figure 5 is a flow chart of a subtitle display method in an application embodiment of the present invention;

[0026] Figure 6 is a schematic diagram of a distributed system to which a subtitle display method in another application embodiment of the present invention is applied;

[0027] Figure 7 is a schematic diagram of a block structure in another application embodiment of the present invention;

[0028] Figure 8 It is a schematic diagram of the logical structure of a browser provided by an embodiment of the present invention;

[0029] Fig. 9 It is a schematic diagram of the logical structure of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] The embodiment of the present invention provides a subtitle display method, which is mainly used in a browser, such as Figure 1 As shown, the browser can display the subtitles in the target video in the following way:

[0033] The current web page process determines that existing subtitles in a target video are to be converted into new subtitles, and a first language type corresponding to the existing subtitles is different from a second language type corresponding to the new subtitles; the current web page process obtains a subtitle conversion model; when the existing subtitles are converted into new subtitles by means of the subtitle conversion model, the current web page process transmits a subtitle event object to a browser main process; when the browser main process determines, based on the subtitle event object, that a display time of the new subtitles has arrived, the current web page process is notified to display the new subtitles on a display interface of the target video.

[0034] The above subtitle conversion model is a machine learning model based on artificial intelligence. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0035] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0036] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0037] In this way, the subtitle conversion model obtained through the current web page process in the browser can realize real-time conversion of existing subtitles in the target video without the need for a dedicated translation team to convert the subtitles, thereby meeting the real-time needs of users to watch videos in various languages.

[0038] An embodiment of the present invention provides a subtitle display method, which is mainly executed by a browser. The flow chart is as follows: Figure 2 As shown, including:

[0039] Step 101: the current web page process determines that existing subtitles in a target video need to be converted into new subtitles, and a first language type corresponding to the existing subtitles is different from a second language type corresponding to the new subtitles.

[0040] It can be understood that when a web page is opened through a browser, the browser main process can start a web page process (such as the current web page process), and the current web page process displays the web page, displays the target video on the web page, and displays the subtitles involved in the target video at the corresponding positions. During the display of the target video, the subtitle display process of this embodiment can be initiated.

[0041] In one case, after the target video is displayed, a subtitle switching interface is included on the display interface of the target video. When the user operates the subtitle switching interface, such as an interface for switching English subtitles to Chinese subtitles, the current web page process may determine that the existing subtitles in the target video need to be converted to new subtitles. In this case, during the display of the target video, the subtitle conversion is initiated by the user's operation on its display interface.

[0042] In another case, before displaying the target video, a user selection interface is displayed, the user selection interface includes a selection interface for switching subtitles, and in response to an operation on the selection interface for switching subtitles, the current web page process determines that the existing subtitles in the target video need to be converted into new subtitles. In this case, if the subtitle conversion is completed by the steps of this embodiment before displaying the target video, the current web page process can display the converted new subtitles while displaying the target video.

[0043] In other cases, before displaying the target video, the current web page process can determine whether subtitle conversion is required based on the user's historical access information in the current web page process. For example, in the user's historical access to the current web page process, when the ratio of historical accesses of the second language type corresponding to the new subtitles to all historical accesses is greater than a threshold, it is considered that the user is familiar with the text of the second language type, and it is determined that the existing subtitles need to be converted to new subtitles. The historical access can be access to information such as web pages or multimedia data.

[0044] Step 102: Obtain a subtitle conversion model.

[0045] The subtitle conversion model is a machine learning model that can be trained through a certain training method. In one case, the operating logic of the subtitle conversion model can be stored in the local system in advance, and is specifically used to convert text of one language type into text of another language type, and the content of the text of one language type is consistent with that of the text of another language type.

[0046] It should be noted that the subtitle conversion model can include multiple types, and the browser can preset at least one type of subtitle conversion model locally in advance. Any type of subtitle conversion model is used to convert text of one language type into text of another language type. Among them, when multiple types of subtitle conversion models are preset locally, it is specifically possible to directly request the video server or the server corresponding to the browser to obtain the information of the subtitle conversion model and store it in the local system; or directly obtain the subtitle conversion model through training samples, and then store its running logic locally. In this way, when the subtitle display process of this embodiment is initiated, the subtitle conversion model can be directly called, and specifically, the first type of subtitle conversion model in the preset at least one type of subtitle conversion model can be called, and the first type of subtitle conversion model is used to convert text of the first language type into text of the second language type.

[0047] In another case, when initiating the process of this embodiment, if the corresponding subtitle conversion model has not been preset in the local system, it is necessary to first request the video server or the server corresponding to the browser to obtain the information of the subtitle conversion model, and then execute the following step 103.

[0048] In this way, by presetting multiple types of subtitle conversion models, the existing subtitles in any video can be converted into subtitles in multiple languages, for example, the English subtitles in the video can be converted into Chinese subtitles, or other minority language subtitles, etc., thereby meeting the needs of different video viewing users. In addition, since the subtitle conversion model is trained based on certain training samples, a corresponding subtitle conversion model can be trained for the conversion between any two specific language types of texts, and the accuracy will be relatively high. For example, the training subtitle conversion model 1 is used to convert Chinese text into English text, the training subtitle conversion model 2 is used to convert English text into Chinese text, and the training subtitle conversion model 3 is used to convert Chinese text into other minority language texts, etc.

[0049] Step 103: When the existing subtitles are converted into new subtitles through the subtitle conversion model, the current web page process transmits the subtitle event object to the browser main process.

[0050] It should be noted that when the current web page process displays the target video, it can specifically obtain the resources of the target video and display them, wherein the resources of the target video may include each frame of the target video and audio data, and may also include a subtitle file, wherein the subtitle file includes the subtitle text of the existing subtitles and the subtitle display time, the subtitle text may include multiple existing subtitles, and the subtitle display time is the display time corresponding to each existing subtitle. When the current web page process displays the existing subtitles of the target video, it mainly directly reads the subtitle text of the target video and displays it on the display interface of the target video.

[0051] In the steps of this embodiment, the current web page process can directly read the subtitle text and input the subtitle text into the called subtitle conversion model, so that the subtitle conversion model can convert the existing subtitles in the subtitle text into new subtitles and output the text of the new subtitles, wherein the text of the new subtitles can include a new subtitle corresponding to each existing subtitle, that is, the existing subtitles correspond to the new subtitles one by one, and the display time of each existing subtitle is the display time of the corresponding new subtitles.

[0052] In this way, when the current web page process transmits the subtitle event object to the browser main process, the subtitle display time of the existing subtitles can also be transmitted to the browser main process, and the browser main process determines whether the display time of each new subtitle has arrived. When the browser main process receives the subtitle event object, it can start a timer. The timing duration of the timer can be the duration between the current time and the display time of the new subtitle. When the timing duration of the timer is reached, it can be determined that the display time of the new subtitle has arrived.

[0053] Step 104, when the browser main process determines that the display time of the new subtitle has arrived according to the subtitle event object, the current web page process is notified to display the new subtitle on the display interface of the target video. Furthermore, while the display interface of the target video displays the new subtitle, the current web page process can also display the corresponding existing subtitle, realizing the display of dual subtitles.

[0054] It should be noted that the current web page process can input the existing subtitles into the subtitle conversion model one by one, and the subtitle conversion model outputs the text of the new subtitles one by one, and the browser main process determines whether the display time of the new subtitles has been reached one by one.

[0055] It can be understood that the existing subtitles of the target video may include multiple existing subtitles, each existing subtitle corresponds to at least one frame of the target video, that is, the existing subtitle needs to be displayed in at least one frame of the target video, and the display time of each existing subtitle is the display time of at least one frame of the target video. Among them, if multiple adjacent existing subtitles correspond to a complete sentence, that is, the length of a complete sentence is relatively long, and it needs to be divided into multiple subtitles for display, and for texts of different language types, due to different grammars, the expression order of a complete sentence is also different. In this way, after the multiple existing subtitles are converted into multiple new subtitles, the display order of the multiple new subtitles is different from the display order of the multiple existing subtitles in the target video. Therefore, when displaying these multiple new subtitles, specifically:

[0056] The current web page process needs to first determine the display order of multiple new subtitles corresponding to multiple existing subtitles according to the grammar of the second language type, and then display the multiple new subtitles in the determined display order when the display time of the multiple new subtitles arrives.

[0057] It can be seen that in the method of this embodiment, when the current web page process started by the browser main process determines that the existing subtitles of the target video need to be converted into new subtitles, it will obtain a subtitle conversion model, and complete the subtitle conversion through the subtitle conversion model, and transmit the subtitle event object to the browser main process. When the browser main process determines that the display time of the new subtitle has arrived during the display of the target video, it notifies the current web page process to display the new subtitle on the display interface of the target video. In this way, the subtitle conversion model obtained by the current web page process in the browser can realize the real-time conversion of the existing subtitles in the target video without the need for a special translation team to convert the subtitles, which meets the real-time needs of users to watch videos of various language types.

[0058] refer to Figure 3 As shown, in the specific implementation process, the browser can implement the training of the above subtitle conversion model through the following steps, including:

[0059] Step 201, determining an initial subtitle conversion model.

[0060] Specifically, the browser can determine the multi-layer structure included in the initial subtitle conversion model, and determine the initial values ​​corresponding to the parameters in each layer of the structure. The parameters in each layer of the structure refer to the fixed parameters used in the calculation process of each layer of the initial subtitle conversion model, that is, the parameters that do not need to be assigned at any time with the adjustment of the later model, such as parameter scale, number of network layers, user vector length and other parameters.

[0061] The structure of the initial subtitle conversion model may include: a feature extraction module and a conversion module, wherein the feature extraction module is used for feature information of a text of one language type; the conversion module is used to convert a text of one language type into a text of another language type according to the feature information extracted by the feature extraction module, and the content of the text of one language type is consistent with that of the text of another language type.

[0062] Step 202: Acquire training samples, where the training samples include a plurality of sample pairs, each sample pair including a text of a first language type and a corresponding text of a second language type.

[0063] Step 203: convert the text of the first language type in the sample pair by using the initial subtitle conversion model to obtain the text of the second language type.

[0064] Step 204: Adjust the initial subtitle conversion model according to the text of the second language type obtained by the initial subtitle conversion model and the corresponding text of the second language type in the sample pair to obtain a final subtitle conversion model.

[0065] Specifically, the browser will first calculate the loss function related to the initial subtitle conversion model based on the second language type text obtained by the initial subtitle conversion model and the corresponding second language type text in the training sample. The loss function is used to indicate the error between the second language type text obtained by the initial subtitle conversion model and the actual second language type text in the training sample, such as the cross entropy loss function.

[0066] The training process of the subtitle conversion model is to minimize the value of the above error. The training process continuously optimizes the parameter values ​​of the parameters in the above initial subtitle conversion model through a series of mathematical optimization methods such as back propagation derivation and gradient descent, and minimizes the calculated value of the above loss function.

[0067] For example, when the function value of the calculated loss function is large, such as greater than a preset value, it is necessary to change the parameter value, such as reducing the weight value of a certain neuron connection, so that the function value of the loss function calculated according to the adjusted parameter value is reduced.

[0068] It should be noted that the above steps 201 to 204 are an adjustment of the parameter values ​​of the parameters in the initial subtitle conversion model by the text of the second language type obtained by the initial subtitle conversion model, and in actual applications, it is necessary to continuously loop through the above steps 203 to 204 until the adjustment of the parameter values ​​meets certain training stop conditions.

[0069] Therefore, after executing step 204 of the above embodiment, the browser still needs to determine whether the current adjustment of the parameter value meets the preset training stop condition. The preset training stop condition includes but is not limited to any one of the following conditions: the difference between the current adjusted parameter value and the last adjusted parameter value is less than a threshold, that is, the adjusted parameter value reaches convergence; and the number of parameter value adjustments is equal to the preset number of times, etc.

[0070] It is understandable that after the browser has trained the subtitle conversion model, it obtains the information of the subtitle conversion model in a specific format, where the specific format is specifically the format supported by the browser corresponding application, and stores the information of the subtitle conversion model in the specific format to the browser. In this way, multiple video applications (such as browsers and other applications that can play videos) can perform subtitle conversion in real time when displaying videos.

[0071] It should also be noted that the above-mentioned training of the subtitle conversion model is carried out through a browser. In other embodiments, the subtitle conversion model can be trained through a server and the information of the trained subtitle conversion model can be stored in the server. When the browser needs to call the subtitle conversion model, it can be downloaded from the server, which can save browser resources.

[0072] The following is a specific application example to illustrate the subtitle display method of the present invention. In this embodiment, the method mainly includes the following two parts:

[0073] (1) Training the subtitle conversion model through the server

[0074] Specifically, you can follow the above Figure 3 The method described is used to train the subtitle conversion model. In this embodiment, the subtitle conversion model from English text to Chinese text is used as an example to illustrate. In the specific training process: for the selection of sample pairs in the training samples, selection can be made from Chinese and English bilingual parallel corpora; for the determination of the initial subtitle conversion model, T5 can be selected to complete the conversion from English text to Chinese text.

[0075] In order to adapt to the processing of the browser and reduce the size of the binary file, the information of the trained subtitle conversion model (including the model's structural information and weight information, etc.) can be converted into information in the format of a directly interpreted scripting language (JavaScript, JS), and the information of the subtitle conversion model in the converted format is stored in a server (such as a cloud server), so that the browser can dynamically pull the information of the subtitle conversion model in the format corresponding to the browser as needed.

[0076] (2)Reference Figure 4 and Figure 5 As shown, the browser can achieve real-time subtitle conversion and display through the following steps:

[0077] Step 301, when a browser opens a web page, the browser main process will start a web page process (ie, the current web page process), and the current web page process will obtain a target video from a video server and display the target video in the displayed web page.

[0078] In step 302, the current web page process dynamically obtains the information of the subtitle conversion model from the cloud server and stores it in the local system. Specifically, the current web page process first downloads the weight list (including the storage address of the weight) of the subtitle conversion model, namely model.json, from the cloud server, and then issues an additional HyperText Transfer Protocol (HTTP) request to obtain the weight file fragment referenced in the model.json weight list; downloads the file of the subtitle conversion model, such as en2zh.tflite, which is used to convert English text into Chinese text.

[0079] Among them, the size of model.json and weight files are generally smaller than the typical cache file size limit, so the subtitle conversion model may be loaded and called faster when used subsequently.

[0080] Step 303: the current web page process loads the subtitle conversion model and displays a subtitle switching interface on the display interface of the target video, such as a switch button for switching from English subtitles to Chinese subtitles.

[0081] Step 304, when the user operates the subtitle switching interface and determines that the existing English subtitles in the target video need to be converted into Chinese subtitles, the current web page process will call the corresponding subtitle conversion model to convert the English subtitles in the target video into Chinese subtitles one by one through the subtitle conversion model.

[0082] Step 305, the current web page process transmits a subtitle event object to the browser main process, and also transmits the subtitle display time to the browser process. The subtitle event object is used to trigger the display of Chinese subtitles.

[0083] Step 306, when the browser main process determines that the display time of the Chinese subtitles has arrived according to the subtitle event object, it notifies the current web page process to display the Chinese subtitles in the screen of the target video.

[0084] It can be seen that in this embodiment, the subtitle conversion model called by the current web page process converts the existing subtitles in any video into new subtitles of another language type in real time, and displays them in real time on the display interface of the video. In this way, the conversion result of the subtitles can be obtained without requesting the server, and the subtitle data does not need to be uploaded, which can shorten the delay of the conversion and protect privacy. In practical applications, since the browser function can be realized in more applications, the subtitle display method of the embodiment of the present invention can be used in more applications, thereby realizing cross-platform real-time display of subtitles.

[0085] The following is another specific application example to illustrate the subtitle display method in the present invention. The browser in the embodiment of the present invention is mainly a distributed system 100, which may include a client 300 and multiple nodes 200 (any form of computing device in the access network, such as a server, a user terminal), and the client 300 and the node 200 are connected in the form of network communication.

[0086] Take the distributed system as the blockchain system as an example, see Figure 6 This is an optional structural diagram of the distributed system 100 provided by the embodiment of the present invention applied to the blockchain system, which is formed by multiple nodes 200 (any form of computing devices in the access network, such as servers and user terminals) and clients 300, and a peer-to-peer (P2P) network is formed between the nodes. The P2P protocol is an application layer protocol running on the Transmission Control Protocol (TCP) protocol. In a distributed system, any machine such as a server or a terminal can join and become a node, and the node includes a hardware layer, an intermediate layer, an operating system layer, and an application layer.

[0087] See also Figure 6 The functions of each node in the blockchain system shown include:

[0088] 1) Routing: a basic function of a node, used to support communication between nodes.

[0089] In addition to the routing function, the node can also have the following functions:

[0090] 2) Applications, which are deployed in the blockchain to implement specific businesses according to actual business needs, record data related to the implementation of functions to form record data, carry digital signatures in the record data to indicate the source of the task data, and send the record data to other nodes in the blockchain system for other nodes to add the record data to the temporary block when they successfully verify the source and integrity of the record data.

[0091] For example, the service implemented by the application includes the code for implementing the subtitle display function, and the subtitle display function mainly includes:

[0092] The current web page process determines that existing subtitles in a target video are to be converted into new subtitles, and a first language type corresponding to the existing subtitles is different from a second language type corresponding to the new subtitles; the current web page process obtains a subtitle conversion model; when the existing subtitles are converted into new subtitles by means of the subtitle conversion model, the current web page process transmits a subtitle event object to a browser main process; when the browser main process determines, based on the subtitle event object, that a display time of the new subtitles has arrived, the current web page process is notified to display the new subtitles on a display interface of the target video.

[0093] 3) Blockchain, including a series of blocks that are connected to each other in the order of their generation. Once a new block is added to the blockchain, it will not be removed. The block records the record data submitted by the nodes in the blockchain system.

[0094] See also Figure 7 An optional schematic diagram of the block structure provided by the embodiment of the present invention, each block includes the hash value of the transaction record stored in this block (the hash value of this block) and the hash value of the previous block, and each block is connected by the hash value to form a blockchain. In addition, the block can also include information such as the timestamp when the block was generated. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains relevant information for verifying the validity of its information (anti-counterfeiting) and generating the next block.

[0095] The embodiment of the present invention also provides a browser, the structure diagram of which is as follows Figure 8 As shown, it may specifically include:

[0096] The conversion determination unit 10 is used to determine that existing subtitles in the target video are to be converted into new subtitles, wherein the first language type corresponding to the existing subtitles is different from the second language type corresponding to the new subtitles.

[0097] The model unit 11 is used to call the subtitle conversion model when the conversion determination unit 10 determines to perform conversion.

[0098] The conversion unit 12 is used to transmit a subtitle event object to the browser main process when the subtitle conversion model obtained by the model unit 11 converts the existing subtitles into new subtitles. When the browser main process determines that the display time of the new subtitles has arrived according to the subtitle event object, it notifies the display unit 13 to display the subtitles.

[0099] The display unit 13 is used to display the new subtitles obtained by the conversion unit 12 on the display interface of the target video.

[0100] Among them, the conversion determination unit 10 is specifically used for, when the display interface of the target video includes: a subtitle switching interface; in response to the operation of the subtitle switching interface, determining that the existing subtitles in the target video are to be converted into new subtitles. Alternatively, the conversion determination unit 10 is specifically used for, before displaying the target video, displaying a user selection interface, the user selection interface including a selection interface for switching subtitles; in response to the operation of the selection interface for switching subtitles, determining that the existing subtitles in the target video are to be converted into new subtitles. Alternatively, the conversion determination unit 10 is specifically used for, when in the user's historical access, the ratio of historical accesses in which the language type of the accessed content is the second language type to all historical accesses is greater than a threshold, determining that the existing subtitles in the target video are to be converted into new subtitles.

[0101] The above-mentioned display unit 13 is specifically used to determine the display order of multiple new subtitles corresponding to the multiple existing subtitles according to the grammar of the second language type when the existing subtitles of the target video include multiple adjacent existing subtitles, and the multiple existing subtitles correspond to one sentence; when the display time of the multiple new subtitles arrives, display the multiple new subtitles in the display order.

[0102] Furthermore, the browser of this embodiment also includes:

[0103] The preset unit 14 is used to preset at least one type of subtitle conversion model locally, and any type of subtitle conversion model is used to convert text of one language type into text of another language type; thus, the above-mentioned model unit 11 is specifically used to call a first type of subtitle conversion model among the at least one type of subtitle conversion model preset by the preset unit 14, and the first type of subtitle conversion model is used to convert text of the first language type into text of the second language type.

[0104] The training unit 15 is used to determine an initial subtitle conversion model; determine a training sample, wherein the training sample includes a plurality of sample pairs, each sample pair includes a text of a first language type and a corresponding text of a second language type; convert the text of the first language type in the sample pair by the initial subtitle conversion model to obtain a text of a second language type; and adjust the initial subtitle conversion model according to the text of the second language type obtained by the initial subtitle conversion model and the corresponding text of the second language type in the sample pair to obtain a final subtitle conversion model. In this way, the above-mentioned presetting unit 14 can preset the information of the final subtitle conversion model obtained by the training unit 15 to the local.

[0105] It can be seen that in the browser of this embodiment, when the conversion determination unit 10 determines that the existing subtitles of the target video need to be converted into new subtitles, the model unit 11 will call the subtitle conversion model, and the conversion unit 12 completes the subtitle conversion through the subtitle conversion model. When the display time of the new subtitle arrives during the display of the target video, the display unit 13 displays the new subtitle on the display interface of the target video. In this way, the subtitle conversion model obtained in the browser can realize the real-time conversion of the existing subtitles in the target video without the need for a special translation team to convert the subtitles, which meets the real-time needs of users to watch videos of various language types.

[0106] The embodiment of the present invention further provides a terminal device, the structural diagram of which is shown in FIG. Fig. 9 As shown, the terminal device may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 20 (for example, one or more processors) and memory 21, and one or more storage media 22 (for example, one or more mass storage devices) storing application programs 221 or data 222. Among them, the memory 21 and the storage medium 22 can be short-term storage or permanent storage. The program stored in the storage medium 22 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the terminal device. Furthermore, the central processing unit 20 can be configured to communicate with the storage medium 22 to execute a series of instruction operations in the storage medium 22 on the terminal device.

[0107] Specifically, the application 221 stored in the storage medium 22 includes an application for subtitle display, and the program may include the conversion determination unit 10, the model unit 11, the conversion unit 12, the display unit 13, the preset unit 14 and the training unit 15 in the above-mentioned subtitle display device, which will not be described in detail here. Furthermore, the central processor 20 can be configured to communicate with the storage medium 22, and execute a series of operations corresponding to the application for subtitle display stored in the storage medium 22 on the terminal device.

[0108] The terminal device may also include one or more power supplies 23, one or more wired or wireless network interfaces 24, one or more input and output interfaces 25, and / or one or more operating systems 223, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, FreeBSD TM, etc.

[0109] The steps performed by the above browser in the above method embodiment can be based on the Fig. 9 The structure of the terminal device shown.

[0110] Another aspect of the embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of computer programs, wherein the computer programs are suitable for being loaded by a processor and executing the subtitle display method executed by the above-mentioned browser.

[0111] Another aspect of the embodiment of the present invention further provides a terminal device, including a processor and a memory;

[0112] The memory is used to store a plurality of computer programs, and the computer programs are used to be loaded by the processor and execute the subtitle display method executed by the above-mentioned browser; the processor is used to implement each computer program in the plurality of computer programs.

[0113] A person skilled in the art may understand that all or part of the steps in the various methods of the above-mentioned embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0114] The above is a detailed introduction to a subtitle display method, terminal, storage medium and terminal device provided by an embodiment of the present invention. Specific examples are used herein to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A subtitle display method, characterized in that: include: When, in the user's historical visits, the ratio of historical visits in which the language type of the accessed content is the second language type to all historical visits is greater than a threshold value, the current web page process determines that the existing subtitles in the target video are to be converted into new subtitles, and the first language type corresponding to the existing subtitles is different from the second language type corresponding to the new subtitles; The current web page process obtains a subtitle conversion model; When the existing subtitles are converted into new subtitles through the subtitle conversion model, the current web page process transmits the subtitle event object to the browser main process; When the browser main process determines that the display time of the new subtitles has arrived according to the subtitle event object, the current web page process is notified to display the new subtitles on the display interface of the target video.

2. The method according to claim 1, characterized in that The display interface of the target video includes: a subtitle switching interface; the determining to convert the existing subtitles in the target video into new subtitles specifically includes: The current web page process determines to convert existing subtitles in the target video into new subtitles in response to an operation on the subtitle switching interface.

3. The method according to claim 1, characterized in that The step of determining that the existing subtitles in the target video are to be converted into new subtitles specifically includes: The current web page process displays a user selection interface before displaying the target video, wherein the user selection interface includes a selection interface for switching subtitles; The current web page process determines to convert existing subtitles in the target video into new subtitles in response to an operation on the subtitle switching selection interface.

4. The method according to any one of claims 1 to 3, characterized in that Before obtaining the subtitle conversion model, the method further includes: At least one type of subtitle conversion model is preset locally, and any type of subtitle conversion model is used to convert text of one language type into text of another language type; The acquiring of the subtitle conversion model specifically includes: calling a first type of subtitle conversion model among the at least one type of subtitle conversion model, wherein the first type of subtitle conversion model is used to convert text of the first language type into text of a second language type.

5. The method according to any one of claims 1 to 3, characterized in that: The existing subtitles of the target video include a plurality of adjacent existing subtitles, and the plurality of existing subtitles correspond to one sentence; The current web page process displays the new subtitles on the display interface of the target video, specifically including: The current web page process determines, according to the grammar of the second language type, a display order of the multiple new subtitles corresponding to the multiple existing subtitles; When the display time of the multiple new subtitles arrives, the current web page process displays the multiple new subtitles according to the display order.

6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determine the initial subtitle conversion model; Determine a training sample, wherein the training sample includes a plurality of sample pairs, each sample pair including a text of a first language type and a corresponding text of a second language type; Converting the text of the first language type in the sample pair by the initial subtitle conversion model to obtain the text of the second language type; According to the text of the second language type obtained by the initial subtitle conversion model and the text of the second language type corresponding to the sample pair, the initial subtitle conversion model is adjusted to obtain a final subtitle conversion model.

7. A browser, characterized in that: include: A conversion determination unit, configured to determine that existing subtitles in a target video should be converted into new subtitles when a ratio of historical accesses in which the language type of accessed content is the second language type to all historical accesses in the current web page process during the user's historical accesses is greater than a threshold, wherein the first language type corresponding to the existing subtitles is different from the second language type corresponding to the new subtitles; Model unit, used to obtain the subtitle conversion model; A conversion unit, configured to convert the existing subtitles into new subtitles through the subtitle conversion model, transmit the subtitle event object to the browser main process, and notify the display unit to display the subtitles when the browser main process determines that the display time of the new subtitles has arrived according to the subtitle event object; A display unit is used to display the new subtitles on the display interface of the target video.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of computer programs, and the computer programs are suitable for being loaded by a processor and executing the subtitle display method according to any one of claims 1 to 6.

9. A terminal device, characterized in that: including a processor and a memory; The memory is used to store a plurality of computer programs, and the computer programs are used to be loaded by the processor and execute the subtitle display method according to any one of claims 1 to 6; the processor is used to implement each computer program in the plurality of computer programs.

Images