Subtitle rendering method, device, electronic device, storage medium and program product

Abstract

The present application provides a subtitle rendering method, device, electronic device, storage medium and program product; the method includes: obtaining a three-dimensional subtitle file corresponding to a three-dimensional target video, determining at least one subtitle frame based on the three-dimensional subtitle file, and each subtitle frame at least includes start and end time information; when playing the three-dimensional target video, obtaining the subtitle frame to be displayed, and obtaining the current playback time of the three-dimensional target video; when the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, determining the text view based on the subtitle frame to be displayed; encapsulating the text view onto a subtitle entity, and rendering the subtitle entity onto a screen entity, and the screen entity is used to display the three-dimensional target video. Through the present application, the rendered subtitles can have a three-dimensional effect, enrich the display form of the subtitles, and ensure the accurate synchronization of the subtitles and the three-dimensional video.

Description

Technical Field

[0001] The present application relates to the technical field of subtitle processing, and in particular to a subtitle rendering method, device, electronic device, storage medium and program product. Background Art

[0002] With the development of video technology, 3D (three-dimensional) video can be applied in many fields. When watching 3D video, subtitles are one of the important factors affecting the viewing experience.

[0003] In the related art, subtitles in 3D videos are generally embedded in video files and displayed on the video in a two-dimensional effect. In addition, the synchronization between the subtitles and the video is also poor, which affects the viewing experience of the 3D video. Summary of the invention

[0004] The embodiments of the present application provide a subtitle rendering method, device, electronic device, storage medium and program product, which can enable the rendered subtitles to have a three-dimensional effect, enrich the display form of the subtitles, and ensure the accurate synchronization of the subtitles and the three-dimensional video.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] An embodiment of the present application provides a subtitle rendering method, the method comprising: obtaining a three-dimensional subtitle file corresponding to a three-dimensional target video, determining at least one subtitle frame based on the three-dimensional subtitle file, each of the subtitle frames including at least start and end time information; when playing the three-dimensional target video, obtaining a subtitle frame to be displayed, and obtaining a current playback time of the three-dimensional target video; when the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, determining a text view based on the subtitle frame to be displayed; encapsulating the text view onto a subtitle entity, and rendering the subtitle entity onto a screen entity, the screen entity being used to display the three-dimensional target video.

[0007] An embodiment of the present application provides a subtitle rendering device, including: a subtitle frame determination module, used to obtain a three-dimensional subtitle file corresponding to a three-dimensional target video, and determine at least one subtitle frame based on the three-dimensional subtitle file, each of the subtitle frames including at least start and end time information; a playback time acquisition module, used to obtain a subtitle frame to be displayed when playing the three-dimensional target video, and obtain the current playback time of the three-dimensional target video; a text view determination module, used to determine a text view based on the subtitle frame to be displayed when the playback time is within the range of the start and end time information included in the subtitle frame to be displayed; and a rendering module, used to encapsulate the text view onto a subtitle entity, and render the subtitle entity onto a screen entity, the screen entity being used to display the three-dimensional target video.

[0008] In the above scheme, when the 3D subtitle file is an embedded subtitle file, the subtitle frame determination module is also used to obtain a multimedia file corresponding to the 3D target video; demultiplex the multimedia file to obtain an audio file, a video file and a 3D subtitle file of the 3D target video, the 3D subtitle file including at least one 3D subtitle package; and parse the at least one 3D subtitle package to obtain at least one subtitle frame.

[0009] In the above scheme, when the 3D subtitle file is an external subtitle file, the subtitle frame determination module is further used to demultiplex the 3D subtitle file to obtain at least one 3D subtitle package; and parse the at least one 3D subtitle package to obtain at least one subtitle frame.

[0010] In the above scheme, when the three-dimensional subtitle file includes a first subtitle file and a second subtitle file, the subtitle frame determination module is further used to parse the first subtitle file to determine at least one first subtitle frame; parse the second subtitle file to determine at least one second subtitle frame; for each first subtitle frame, merge the first subtitle frame and the second subtitle frame corresponding to the first subtitle frame to obtain the subtitle frame.

[0011] In the above scheme, the subtitle frame determination module is also used to merge the first subtitle frame and the second subtitle frame to obtain the subtitle frame when the first start and end time information in the first subtitle frame is the same as the second start and end time information in the second subtitle frame; when the first start and end time information in the first subtitle frame is different from the second start and end time information in the second subtitle frame and there is an intersection, split at least one of the first subtitle frame and the second subtitle frame based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information to obtain at least three subtitle subframes; merge two subtitle subframes with the same start and end time information among the at least three subtitle subframes to obtain a merged subtitle subframe; splice the subtitle subframes with different start and end time information among the at least three subtitle subframes and the merged subtitle subframe in time order to obtain the subtitle frame.

[0012] In the above scheme, when the first start time is greater than the second start time, and the first end time is greater than the second end time; the subtitle frame determination module is also used to split the first subtitle frame to obtain the first subtitle subframe based on the first start time and the second end time, and split the second subtitle frame to obtain the second subtitle subframe; based on the first end time and the second end time, split the first subtitle frame to obtain the third subtitle subframe; based on the first start time and the second start time, split the second subtitle frame to obtain the fourth subtitle subframe.

[0013] In the above scheme, when the first start time is greater than the second start time and the first end time is less than the second end time; the subtitle frame determination module is also used to split the second subtitle frame to obtain the fifth subtitle subframe based on the first start time and the second start time; split the second subtitle frame to obtain the sixth subtitle subframe based on the first start time and the first end time; split the second subtitle frame to obtain the seventh subtitle subframe based on the first end time and the second end time; and determine the first subtitle frame as the eighth subtitle subframe.

[0014] In the above scheme, when the first start time is greater than the second start time and the first end time is equal to the second end time, or when the first start time is equal to the second start time and the first end time is less than the second end time; the subtitle frame determination module is also used to split the second subtitle frame based on the first start time and the first end time to obtain a ninth subtitle subframe and a tenth subtitle subframe; and determine the first subtitle frame as the eleventh subtitle subframe.

[0015] In the above scheme, the subtitle rendering device also includes a synchronization module, which is used to determine the time difference between the start time in the start and end time information and the playback time when the playback time is not within the range of the start and end time information; after the time difference, determine the text view based on the subtitle frame to be displayed; encapsulate the text view onto the subtitle entity, and render the subtitle entity onto the screen entity.

[0016] In the above scheme, the subtitle frame also includes position information, at least one subtitle text and format information of each subtitle text; the text view determination module is also used to determine the display area based on the position information in the subtitle frame to be displayed; each subtitle text in the subtitle frame to be displayed is displayed in the display area according to the format information of the subtitle text, so as to obtain the text view.

[0017] In the above scheme, the rendering module is also used to obtain the first spatial coordinates and first size information of the screen entity; determine the second spatial coordinates of the subtitle entity based on the first spatial coordinates, the first size information and the position information of the text view; render the screen entity based on the first spatial coordinates, and render the subtitle entity based on the second spatial coordinates.

[0018] In the above scheme, the first subtitle frame corresponds to a first language, and the second subtitle frame corresponds to a second language; the text view determination module is also used to obtain the subtitle language of the subtitles to be displayed in the three-dimensional target video in response to the received language selection request; when the subtitle language is the first language, the text view is determined based on the first subtitle frame in the subtitle frames to be displayed; when the subtitle language is the second language, the text view is determined based on the second subtitle frame in the subtitle frames to be displayed; when the subtitle language is the first language and the second language, the text view is determined based on the subtitle frame to be displayed.

[0019] In the above scheme, the rendering module is also used to obtain first color information of the image area displaying the subtitle entity on the screen entity; obtain second color information of the text view in the subtitle entity; determine the similarity between the first color information and the second color information; when the similarity is greater than or equal to a preset similarity threshold, determine a target color from a preset color library based on the first color information; and render the subtitle entity based on the target color.

[0020] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0021] A memory for storing computer executable instructions or computer programs;

[0022] The processor is used to implement the subtitle rendering method provided in the embodiment of the present application when executing the computer executable instructions or computer programs stored in the memory.

[0023] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing a subtitle rendering method provided in an embodiment of the present application when executed by a processor.

[0024] An embodiment of the present application provides a computer program product, including a computer program or a computer executable instruction. When the computer program or the computer executable instruction is executed by a processor, the subtitle rendering method provided in the embodiment of the present application is implemented.

[0025] The embodiments of the present application have the following beneficial effects:

[0026] When playing a three-dimensional target video, when the current playback time of the three-dimensional target video is within the range of the start and end time information included in the subtitle frame to be displayed, the subtitles in the subtitle frame to be displayed are rendered and displayed, and the start and end time information of the subtitle frame to be displayed is accurately matched with the playback time of the three-dimensional target video, thereby ensuring the synchronization accuracy of the subtitles and the video content. Compared with the method of querying the current playback time by polling in the related art and selecting the subtitle frame that can be displayed according to the current playback time, in the embodiment of the present application, the subtitle frame to be displayed is first obtained, and then the current playback time of the three-dimensional target video is obtained. When the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, the subtitle frame to be displayed can be rendered. There is no need to query the current playback time at fixed intervals, unifying the subtitle and video playback processes, saving the resources required for subtitle rendering, and being able to more accurately control the display of subtitles. In addition, the embodiment of the present application determines the text view based on the subtitle frame to be displayed, encapsulates the text view into the subtitle entity, and combines it with the screen entity that displays the three-dimensional target video. Since the subtitle entity can have a three-dimensional effect, the subtitle displayed on the three-dimensional target video after rendering can also have a three-dimensional effect, enriching the display form of the subtitle. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the architecture of the subtitle rendering system provided in an embodiment of the present application;

[0028] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0029] Figure 3 This is a schematic diagram of the process of the subtitle rendering method provided in the embodiment of the present application. Figure 1 ;

[0030] Figure 4 is a schematic diagram of a subtitle frame provided by an embodiment of the present application;

[0031] Figure 5 This is a schematic diagram of the process of the subtitle rendering method provided in the embodiment of the present application. Figure 2 ;

[0032] Figure 6 This is a schematic diagram of the process of the subtitle rendering method provided in the embodiment of the present application. Figure 3 ;

[0033] Figure 7 This is a schematic diagram of the process of the subtitle rendering method provided in the embodiment of the present application. Figure 4 ;

[0034] Figure 8 This is a schematic diagram of the process of the subtitle rendering method provided in the embodiment of the present application. Figure 5 ;

[0035] Fig. 9This is a schematic diagram of the process of the subtitle rendering method provided in the embodiment of the present application. Figure 6 ;

[0036] Fig.10 is another optional flowchart of the subtitle rendering method provided in an embodiment of the present application;

[0037] Fig.11 is a schematic diagram of a video playback interface provided in an embodiment of the present application;

[0038] Fig.12 It is a schematic diagram of displaying Chinese subtitles provided by an embodiment of the present application;

[0039] Fig.13 It is a process architecture diagram of a subtitle rendering system provided by an embodiment of the present application;

[0040] Fig.14 is a schematic diagram of a conversation analysis provided by an embodiment of the present application;

[0041] Fig.15 This is a schematic diagram of the structure of a subtitle package obtained by parsing a dialogue provided in an embodiment of the present application;

[0042] Fig.16 It is a schematic diagram of the process of merging two subtitle frames provided in an embodiment of the present application;

[0043] Fig.17 It is a schematic diagram of a process of splitting and merging two subtitle frames provided in an embodiment of the present application;

[0044] Fig.18 is a schematic diagram of the origin of the spatial coordinate system provided in an embodiment of the present application;

[0045] Fig.19 is a schematic diagram of coordinates of vertices of a screen entity provided in an embodiment of the present application;

[0046] Fig. 20 is a schematic diagram of a scene hierarchy structure in an augmented reality application provided in an embodiment of the present application;

[0047] Fig.21 It is a schematic diagram of the hierarchical relationship between subtitle attachments, subtitle entities and screen entities in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0049] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0050] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0051] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0052] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0053] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0054] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0055] 1) Subtitle frame: It is the basic unit of video subtitles and is text with timestamp information and format. In a video or animation, a subtitle frame is the text content displayed in a frame with start and end time information.

[0056] 2) External subtitles: refers to subtitle files stored separately from video files. Users can choose whether to load these subtitles as needed. There are generally two types of subtitles in videos. One is embedded subtitles, also called hard subtitles, which means compressing subtitles and video and audio content into the same file. When playing, the video screen naturally has subtitle information; while external subtitles, also called soft subtitles, separate subtitles from video files. When playing, there are no subtitles in the video screen itself, but subtitles are rendered separately on various terminals (iOS, Android, visionOS, etc.). Users can freely choose different subtitles when watching videos, such as Mandarin, English, Cantonese, etc., or even a combination of multiple types of subtitles, such as Mandarin and English at the same time.

[0057] 3) Embedded subtitles: These are subtitles that are embedded directly into the video stream and are part of the video. They cannot be turned on or off separately. This type of subtitle is often used to ensure that the subtitles are always synchronized with the video content.

[0058] 4) Multimedia file: A data file that contains multiple types of media such as audio, video, images, and text.

[0059] 5) Subtitle package: usually refers to a collection of files containing subtitle data and related metadata, which can be a single file (such as .srt or .sub file) or a group of files. Subtitle packages enable users to choose different language versions or styles according to their personal preferences.

[0060] 6) RealityKit: is a game engine-like library for building augmented reality (AR) and mixed reality (MR) applications, including physically based rendering (PBR), physics engine, and entity component system (ECS) design. It can be used for 3D rendering in AR, running logic writing, etc.

[0061] 7) Entity: It is the basic building block of the scene and can be understood as any object in the real world or virtual world, such as a 3D model, light source, camera, or any other object that can be presented in an augmented reality (AR) or virtual reality (VR) environment.

[0062] 8) Subtitle rendering: refers to the technical process of adding the content of an external subtitle file to the video playback process so that the subtitles can be displayed synchronously with the video content. This usually involves steps such as parsing the subtitle file format and determining the subtitle display time and position. Rendering here refers to the process of converting the text information that is transmitted after the subtitle file is synchronized with the audio and video into an image and displaying it on the screen. Implementing subtitle rendering involves many aspects, including text rendering, position and style settings, animation effects, etc.

[0063] 9) Embedded subtitle file: refers to the subtitle information encoded together with the video content and stored in the same multimedia file.

[0064] 10) External subtitle file: a subtitle file stored separately from multimedia files.

[0065] In order to better understand the subtitle rendering method provided in the embodiments of the present application, the problems faced by the subtitle rendering method in the related art are first explained.

[0066] Augmented reality devices have super spatial computing and binocular 4K display capabilities, which can greatly improve the viewing experience of 3D videos. In related technologies, when 3D videos are played through various players on augmented reality devices, subtitles are embedded in the video file, not rendered through external subtitles. Users cannot freely choose the display of subtitles through the player, which affects the viewing experience. For the external subtitle rendering technology of ordinary 2D videos, external subtitles are generally used for synchronous transmission: the subtitle playback is separated from the audio and video playback, and the audio and video playback and subtitle playback are two sets of processes. Subtitles have their own separate display control module. The subtitle display control module will start a thread. The thread will query the current playback time every certain period of time, such as 100ms, select the subtitle frame that can be displayed according to the current playback time, and then send it to the rendering module for rendering. The rendering process is generally targeted at different platforms, and the rendering software of different platforms, such as AndroidTextView or Canvas, iOS UILabel or Core Graphics, Windows TextBlock or Direct2D, etc., will set the text to the corresponding view (View) to achieve rendering, and the style and position of the view can be dynamically adjusted. The subtitle rendering method in the related art mainly faces the following problems:

[0067] 1. Each platform has its own rendering solution (which is highly coupled with platform characteristics) and is not suitable for enhanced display devices. In addition, the current platform rendering solutions are all based on 2D drawing, and watching 3D movies in the giant screen theater mode in the living room naturally requires corresponding 3D subtitle rendering to experience a more perfect 3D effect. Therefore, the current subtitle rendering solution can no longer match the spatial rendering properties of enhanced display devices, that is, there is a lack of 3D subtitle rendering methods, which affects the viewing experience of 3D movies.

[0068] 2. In the related art, subtitle processing and audio and video processing are two sets of processing flows, which increases the complexity of the architecture and the cost of understanding. At the same time, because the subtitle display control module polls the playback time, the synchronization accuracy of subtitles and videos is low. In order to ensure the real-time display of subtitles, the query time needs to be set relatively short. Many unnecessary queries lead to increased performance consumption and waste of system resources.

[0069] Based on the problems existing in the related technologies, the embodiments of the present application provide a subtitle rendering method, apparatus, device, computer-readable storage medium and computer program product, which can enable the rendered subtitles to have a three-dimensional effect and ensure the accurate synchronization of the subtitles with the three-dimensional video, thereby improving the viewing experience.

[0070] In the subtitle rendering method provided in the embodiment of the present application, first, a three-dimensional subtitle file corresponding to a three-dimensional target video is obtained, and at least one subtitle frame is determined based on the three-dimensional subtitle file, and each subtitle frame includes at least start and end time information; then, when playing the three-dimensional target video, a subtitle frame to be displayed is obtained, and the current playback time of the three-dimensional target video is obtained; when the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, a text view is determined based on the subtitle frame to be displayed; finally, the text view is encapsulated on a subtitle entity, and the subtitle entity is rendered on a screen entity, and the screen entity is used to display the three-dimensional target video.

[0071] The following describes an exemplary application of a subtitle rendering device provided in an embodiment of the present application, which is an electronic device for implementing a subtitle rendering method. The electronic device provided in an embodiment of the present application can be implemented as various types of terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart TV, and a vehicle-mounted terminal, or it can be implemented as a server. Among them, the server can be an independent physical server, or it can be a server cluster or a distributed system composed of multiple physical servers, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present application. Below, an exemplary application when the subtitle rendering device is implemented as a terminal or a server will be described.

[0072] See also Figure 1 , Figure 1 : is a schematic diagram of the architecture of the subtitle rendering system provided in the embodiment of the present application. In order to perform the subtitle rendering operation, a subtitle rendering application can be provided. For example, the subtitle rendering application can be an application dedicated to subtitle rendering in a video, or it can be a functional module in other applications (such as a subtitle rendering module in a video playback application, etc.). The subtitle rendering system 100 of the embodiment of the present application at least includes a terminal 400, a network 300 and a server 200, wherein the server 200 is a server of the subtitle rendering application. The server 200 can constitute the subtitle rendering device of the embodiment of the present application, that is, the subtitle rendering method of the embodiment of the present application is implemented by the server 200. The terminal 400 is connected to the server 200 via the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two.

[0073] See also Figure 1, the user can perform interactive operations on the client of the subtitle rendering application through the terminal 400, and the interactive operations can be, for example, click to play the video operation, select the operation of rendering subtitles, etc. After receiving the interactive operation of the user, the client sends the subtitle rendering request to the server 200 through the network 300. After receiving the subtitle rendering request, the server 200 responds to the subtitle rendering request sent by the terminal, obtains the three-dimensional subtitle file corresponding to the three-dimensional target video, and determines at least one subtitle frame based on the three-dimensional subtitle file, and each subtitle frame includes at least start and end time information; when playing the three-dimensional target video, the server 200 obtains the subtitle frame to be displayed and obtains the current playback time of the three-dimensional target video; when the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, the server 200 determines the text view based on the subtitle frame to be displayed; the server 200 encapsulates the text view to the subtitle entity, and renders the subtitle entity to the screen entity, and the screen entity is used to display the three-dimensional target video. The server 200 can also send the rendered subtitle entity and screen entity to the terminal 400, and the terminal 400 displays the subtitle entity and the screen entity to present the three-dimensional target video with three-dimensional subtitles.

[0074] In some embodiments, the terminal 400 may also perform the subtitle rendering method of the embodiment of the present application, that is, after the user performs an interactive operation on the client of the subtitle rendering application through the terminal 400, the terminal 400 obtains the 3D subtitle file corresponding to the 3D target video in response to the interactive operation, and determines at least one subtitle frame based on the 3D subtitle file, each subtitle frame at least including start and end time information; when playing the 3D target video, the terminal 400 obtains the subtitle frame to be displayed and obtains the current playback time of the 3D target video; when the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, the terminal 400 determines the text view based on the subtitle frame to be displayed; the terminal 400 encapsulates the text view on the subtitle entity, and renders the subtitle entity on the screen entity, and the screen entity is used to display the 3D target video. The terminal 400 displays the subtitle entity and the screen entity to present the 3D target video with the 3D subtitle.

[0075] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 2 The electronic device shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0076] The processor 410 may be an integrated circuit chip having signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0077] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0078] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0079] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiment of the present application is intended to include any suitable type of memory.

[0080] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0081] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0082] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless LAN (WiFi), and Universal Serial Bus (USB), etc.;

[0083] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0084] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0085] In some embodiments, the subtitle rendering device provided in the embodiments of the present application can be implemented in software. Figure 2 A subtitle rendering device 455 stored in a memory 450 is shown, which may be software in the form of a program or a plug-in, and includes the following software modules: a subtitle frame determination module 4551, a play time acquisition module 4552, a text view determination module 4553, and a rendering module 4554. These modules are logical, and thus may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0086] In other embodiments, the subtitle rendering device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the subtitle rendering method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.

[0087] The subtitle rendering method provided in the embodiment of the present application will be described in combination with the exemplary application and implementation of the server provided in the embodiment of the present application.

[0088] The following describes the subtitle rendering method provided by the embodiment of the present application. As mentioned above, the electronic device implementing the subtitle rendering method of the embodiment of the present application may be a terminal, a server, or a combination of the two. Therefore, the execution subject of each step will not be repeatedly described below.

[0089] Figure 3This is a schematic diagram of the process of the subtitle rendering method provided in the embodiment of the present application. Figure 1 , will combine Figure 3 The steps shown are explained as Figure 3 As shown, the subtitle rendering method is described by taking the execution subject of the server as an example, and the method includes the following steps 101 to 104:

[0090] In step 101, a 3D subtitle file corresponding to a 3D target video is obtained, and at least one subtitle frame is determined based on the 3D subtitle file.

[0091] Each subtitle frame includes at least start and end time information, and the subtitle frame may also include subtitle text.

[0092] Here, the three-dimensional target video refers to video content containing three-dimensional spatial information, which has a high sense of reality and immersion and can simulate a visual experience close to the real world. The three-dimensional target video can be generated in a variety of ways, such as by professional three-dimensional modeling software, by shooting with a dedicated three-dimensional camera, or by post-processing and converting ordinary two-dimensional video. Taking the scene of an augmented reality device playing a video as an example, the three-dimensional target video can be a 3D movie played by an augmented reality device. A three-dimensional subtitle file refers to a subtitle file used in conjunction with the video content of a three-dimensional target video. The subtitle file may include multiple lines of subtitles sorted from top to bottom according to the time when the subtitles appear in the three-dimensional target video. A subtitle frame refers to the basic unit in a three-dimensional subtitle file, that is, each independent segment containing subtitle text and its related attributes. Each subtitle frame may include subtitle text and start and end time information, and the subtitle frame may also include at least one of position information and format information of the subtitle text. The start and end time information is used to indicate the time point when the subtitle text starts to be displayed and stops to be displayed. The subtitle text is the specific text information to be displayed. The position information can be used to indicate the position, orientation, and other information of the subtitle text in the three-dimensional space. The subtitle text in the subtitle frame can be at least one. Exemplary, Figure 4 is a schematic diagram of a subtitle frame provided by an embodiment of the present application. Figure 4 , the subtitle frame includes two subtitle texts ( Figure 4 The text fragment in the text fragment), the start and end time information from 0ms to 10.5ms ( Figure 4 The timestamp in the text) and the display position of the two subtitle texts, and the corresponding format of each subtitle text.

[0093] In some embodiments, when the 3D subtitle file is an embedded subtitle file, step 101 can be implemented in the following manner: first, a multimedia file corresponding to the 3D target video is obtained; then, the multimedia file is demultiplexed to obtain an audio file, a video file and a 3D subtitle file of the 3D target video, and the 3D subtitle file includes at least one 3D subtitle package; finally, at least one 3D subtitle package is parsed to obtain at least one subtitle frame.

[0094] Here, the embedded subtitle file refers to the subtitle information encoded together with the video content of the three-dimensional target video and stored in the same multimedia file. Multimedia files are data files used to store three-dimensional target videos, and may include multiple media types such as audio, video, and text. A multimedia file containing an embedded subtitle file may be read from a specified path. For example, a multimedia file of "XXX movie" may be selected through a video playback application, and the video playback application reads and loads the multimedia file based on the storage path of the multimedia file of "XXX movie". Demultiplexing refers to the operation of separating composite multimedia streams (such as video, audio, and subtitles) into separate streams. A demultiplexing tool may be used to demultiplex multimedia files to obtain audio files, video files, and three-dimensional subtitle files of the three-dimensional target video. An audio file is a file used to store sound information in a three-dimensional target video, and may include various sound contents such as music, voice, and sound effects. A video file is a file used to store a continuous image sequence (video stream) in a three-dimensional target video. A three-dimensional subtitle file may include at least one three-dimensional subtitle package, and each three-dimensional subtitle package corresponds to a subtitle frame. A three-dimensional subtitle package is a data packet that stores or transmits subtitle information, and may include original text parsed from a three-dimensional subtitle file. The original text includes the code form of data such as subtitle text, start and end time information, and position information. The subtitle frame is the subtitle displayed within the start and end time information range of the three-dimensional target video, which is used to indicate when the subtitle text in this subtitle frame should be displayed, as well as the display time length and display position. The subtitle package needs to be parsed into subtitle frames in order to correctly display the subtitles in the video player. For each three-dimensional subtitle package, parse the three-dimensional subtitle package to obtain the subtitle frame corresponding to the three-dimensional subtitle package. Parse the original text in the three-dimensional subtitle package to obtain the subtitle text, start and end time information, and position information in the subtitle frame. For example, the subtitle package is "00:02→00:04, X1, X2, Y1, Y2, "I like you", color=red". The original text is 00:02→00:04, X1, X2, Y1, Y2, "I like you", color=red. The subtitle package is parsed to obtain a subtitle frame containing the start and end time information of 00:02→00:04, the position information of (X1, Y1) and (X2, Y2), the subtitle text of "I like you", and the format information of the subtitle text being red.

[0095] The embodiment of the present application separates the embedded subtitle file from the multimedia file through demultiplexing processing, thereby solving the problem of inconvenient processing of embedded three-dimensional subtitle files, improving the efficiency of subtitle extraction and processing, and providing users with a better viewing experience.

[0096] In some embodiments, when the 3D subtitle file is an external subtitle file, step 101 can be implemented in the following manner: first, demultiplexing the 3D subtitle file to obtain at least one 3D subtitle package; then, parsing the at least one 3D subtitle package to obtain at least one subtitle frame.

[0097] Here, the multimedia file of the three-dimensional target video may not contain an embedded subtitle file. In this case, an external subtitle file is required to render the three-dimensional subtitles on the three-dimensional target video. The external subtitle file is a subtitle file stored separately from the multimedia file. The multimedia file of the three-dimensional target video and the external subtitle file of the three-dimensional target video can be read from the specified path. For example, a multimedia file of "XXX movie" can be selected through a video playback application, and the video playback application reads and loads the multimedia file based on the storage path of the multimedia file of "XXX movie", and then plays the three-dimensional target video. At the same time, an external subtitle file can be selected through the video playback application, and a demultiplexing tool can be used to demultiplex the external subtitle file to obtain at least one three-dimensional subtitle package, and each three-dimensional subtitle package corresponds to a subtitle frame. For each three-dimensional subtitle package, the three-dimensional subtitle package is parsed to obtain the subtitle frame corresponding to the three-dimensional subtitle package.

[0098] The embodiment of the present application obtains subtitle frames by demultiplexing and parsing external subtitle files, thereby improving the efficiency of subtitle extraction and processing, and providing users with three-dimensional subtitles when the three-dimensional target video itself does not have subtitles, thereby providing a better viewing experience.

[0099] In some embodiments, when the 3D subtitle file includes a first subtitle file and a second subtitle file, see Figure 5 In step 101, determining at least one subtitle frame based on the three-dimensional subtitle file can be achieved by following steps 1011 to 1013, which are described in detail below.

[0100] In step 1011, a first subtitle file is parsed to determine at least one first subtitle frame.

[0101] Here, the first subtitle file and the second subtitle file can be subtitle files of the same type or different types. The same type means that the first subtitle file and the second subtitle file are both embedded subtitle files or both external subtitle files. For example, the first subtitle file is an embedded subtitle file, and the specific implementation process of parsing the first subtitle file to determine at least one first subtitle frame in step 1011 can refer to the embodiment of step 101 when the three-dimensional subtitle file is an embedded subtitle file, which will not be repeated here.

[0102] In step 1012, the second subtitle file is parsed to determine at least one second subtitle frame.

[0103] For example, the second subtitle file is an external subtitle file. The specific implementation process of parsing the second subtitle file to determine at least one second subtitle frame in step 1012 can refer to the embodiment of step 101 when the three-dimensional subtitle file is an external subtitle file, which will not be repeated here.

[0104] In step 1013, for each first subtitle frame, the first subtitle frame and the second subtitle frame corresponding to the first subtitle frame are merged to obtain a subtitle frame.

[0105] Here, it should be noted that, for different subtitle files of the same three-dimensional target video, the division rules of subtitle frames are the same. That is, the number of first subtitle frames obtained by parsing the first subtitle file is the same as the number of second subtitle frames obtained by parsing the second subtitle file, and each first subtitle frame has a corresponding second subtitle frame. For each first subtitle frame, the second subtitle frame corresponding to the first subtitle frame can be determined from multiple second subtitle frames based on the number of lines of the first subtitle frame in the first subtitle file. For example, if the first subtitle frame is in the third line of the first subtitle file, the second subtitle frame parsed in the third line of the second subtitle file is the second subtitle frame corresponding to the first subtitle frame. The subtitle text in the first subtitle frame and the subtitle text in the second subtitle frame corresponding to the first subtitle frame may have the same meaning. Exemplarily, the subtitle text in the first subtitle frame is "I love you", and the subtitle text in the second subtitle frame is "I love you". Alternatively, the subtitle text in the second subtitle frame corresponding to the first subtitle frame may also be an annotation or additional information of the subtitle text in the first subtitle frame. The position information of the subtitle text in the first subtitle frame may be different from the position information of the subtitle text in the second subtitle frame. For example, "I love you" is displayed at the bottom of the 3D target video, and "I love you" can be displayed at the next line of "I love you". Therefore, the first subtitle frame and the second subtitle frame corresponding to the first subtitle frame can be merged to obtain a subtitle frame.

[0106] The embodiment of the present application can achieve richer subtitle display effects, such as displaying bilingual subtitles, providing additional information or annotations, etc. by fusing the first subtitle frame and the second subtitle frame of the first subtitle file and the second subtitle file.

[0107] In some embodiments, see Figure 6 In step 1013, the first subtitle frame and the second subtitle frame corresponding to the first subtitle frame are merged to obtain a subtitle frame. This can be achieved by following steps 10131 to 10135, which are described in detail below.

[0108] In step 10131, it is determined whether the first start and end time information in the first subtitle frame is the same as the second start and end time information in the second subtitle frame.

[0109] Among them, when the first start and end time information in the first subtitle frame is the same as the second start and end time information in the second subtitle frame, enter step 10132; when the first start and end time information in the first subtitle frame is different from the second start and end time information in the second subtitle frame and there is an intersection, enter step 10133.

[0110] In step 10132, the first subtitle frame and the second subtitle frame are merged to obtain a subtitle frame.

[0111] Here, merging the first subtitle frame and the second subtitle frame means retaining the first start and end time information or retaining the second start and end time information, merging the subtitle texts, position information, and format information in the first subtitle frame and the second subtitle frame to obtain a subtitle frame. For example, the first subtitle frame includes the first start and end time information: 0-10.5ms, the first position information, the subtitle text a and the format information of the subtitle text a, and the subtitle text b and the format information of the subtitle text b, and the second subtitle frame includes the second start and end time information: 0-10.5ms, the second position information, the subtitle text c and the format information of the subtitle text c, and the subtitle text d and the format information of the subtitle text d, then the first subtitle frame and the second subtitle frame are merged to obtain a subtitle frame including: start and end time information: 0-10.5ms, the first position information, the format information of the subtitle text a and the subtitle text a, the format information of the subtitle text b and the subtitle text b, the second position information, the format information of the subtitle text c and the subtitle text c, and the format information of the subtitle text d and the subtitle text d.

[0112] In step 10133, based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information, at least one of the first subtitle frame and the second subtitle frame is split to obtain at least three subtitle subframes.

[0113] Here, when the first start and end time information in the first subtitle frame is different from the second start and end time information in the second subtitle frame, the first start and end time information in the first subtitle frame and the second start and end time information in the second subtitle frame corresponding to the first subtitle frame must have an intersection. The first start time and the first end time in the first start and end time information can be obtained, and the second start time and the second end time in the second start and end time information can be obtained. For example, if the first start and end time information is 3-12ms, and the second start and end time information is 0-10ms, then the first start and end time information is different from the second start and end time information and there is an intersection, the first start time in the first start and end time information is 3ms, the first end time is 12ms, the second start time in the second start and end time information is 0ms, and the second end time is 10ms. Based on the first start time and the first end time, and the second start time and the second end time, at least one of the first subtitle frame and the second subtitle frame can be split to obtain at least three subtitle subframes. There are two subtitle subframes in the at least three subtitle subframes that have the same start and end time information.

[0114] In some embodiments, when the first start time is greater than the second start time, and the first end time is greater than the second end time, the step 10133 splits at least one of the first subtitle frame and the second subtitle frame based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information, to obtain at least three subtitle subframes, which can be achieved in the following manner: first, based on the first start time and the second end time, split the first subtitle frame to obtain the first subtitle subframe, and split the second subtitle subframe from the second subtitle frame; then, based on the first end time and the second end time, split the first subtitle frame to obtain the third subtitle subframe; finally, based on the first start time and the second start time, split the second subtitle frame to obtain the fourth subtitle subframe.

[0115] For example, if the first start and end time information is 3-12ms, and the second start and end time information is 0-10ms, then the first start and end time information is different from the second start and end time information and there is an intersection, the first start time in the first start and end time information is 3ms, the first end time is 12ms, the second start time in the second start and end time information is 0ms, and the second end time is 10ms. At this time, the first start time is greater than the second start time, and the first end time is greater than the second end time. The intersection of the first start and end time information and the second start and end time information is 3-10ms, that is, from the first start time to the second end time. Therefore, the first subtitle subframe can be split from the first subtitle frame, and the start and end time information of the first subtitle subframe is 3-10ms from the first start time to the second end time. The second subtitle subframe can be split from the second subtitle frame, and the start and end time information of the second subtitle subframe is 3-10ms from the first start time to the second end time. Based on the first end time and the second end time, the third subtitle subframe is split from the first subtitle frame, and the start and end time information of the third subtitle subframe is 10-12ms. Based on the first start time and the second start time, the fourth subtitle subframe is split from the second subtitle frame, and the start and end time information of the fourth subtitle subframe is 0-3ms. It should be noted that all the above splitting processes only split the start and end time information, and the subtitle text, position information, format information, etc. remain unchanged.

[0116] In some embodiments, when the first start time is greater than the second start time, and the first end time is less than the second end time, the step 10133 splits at least one of the first subtitle frame and the second subtitle frame based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information, to obtain at least three subtitle subframes, which can be achieved in the following manner: first, based on the first start time and the second start time, split the second subtitle frame to obtain the fifth subtitle subframe; and based on the first start time and the first end time, split the second subtitle frame to obtain the sixth subtitle subframe; then, based on the first end time and the second end time, split the second subtitle frame to obtain the seventh subtitle subframe; finally, determine the first subtitle frame as the eighth subtitle subframe.

[0117] For example, if the first start and end time information is 3-8ms, and the second start and end time information is 0-10ms, then the first start and end time information is different from the second start and end time information and there is an intersection, the first start time in the first start and end time information is 3ms, the first end time is 8ms, the second start time in the second start and end time information is 0ms, and the second end time is 10ms. At this time, the first start time is greater than the second start time, and the first end time is less than the second end time. The intersection of the first start and end time information and the second start and end time information is 3-8ms, that is, from the first start time to the first end time. Therefore, the second subtitle frame is split, and the start and end time information of the fifth subtitle subframe obtained is from the second start time to the first start time 0-3ms. The start and end time information of the sixth subtitle subframe is from the first start time to the first end time 3-8ms. The start and end time information of the seventh subtitle subframe is from the first end time to the second end time 8-10ms. The first subtitle frame is determined to be the eighth subtitle subframe. The start and end time information of the sixth subtitle subframe is the same as that of the eighth subtitle subframe.

[0118] In some embodiments, when the first start time is greater than the second start time and the first end time is equal to the second end time, or when the first start time is equal to the second start time and the first end time is less than the second end time, the step 10133 of splitting at least one of the first subtitle frame and the second subtitle frame based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information to obtain at least three subtitle subframes can be achieved in the following way: first, based on the first start time and the first end time, splitting the second subtitle frame to obtain a ninth subtitle subframe and a tenth subtitle subframe; then, determining the first subtitle frame as the eleventh subtitle subframe.

[0119] For example, if the first start and end time information is 3-10ms, and the second start and end time information is 0-10ms, then the first start time is greater than the second start time, and the first end time is equal to the second end time. The intersection of the first start and end time information and the second start and end time information is 3-10ms, that is, from the first start time to the first end time. The first start time to the first end time in the second subtitle frame can be split to obtain a ninth subtitle subframe, and the remaining subframe after removing the ninth subtitle subframe from the second subtitle frame is the tenth subtitle subframe. The start and end time information of the ninth subtitle subframe is 3-10ms from the first start time to the first end time. The start and end time information of the tenth subtitle subframe is 0-3ms from the second start time to the first start time. The first subtitle frame is determined as the eleventh subtitle subframe.

[0120] Alternatively, if the first start and end time information is 0-3ms and the second start and end time information is 0-10ms, then the first start time is equal to the second start time, and the first end time is less than the second end time. The intersection of the first start and end time information and the second start and end time information is 0-3ms, that is, from the first start time to the first end time. The first start time to the first end time in the second subtitle frame can be split to obtain the ninth subtitle subframe, and the subframe remaining after removing the ninth subtitle subframe from the second subtitle frame is the tenth subtitle subframe. The start and end time information of the ninth subtitle subframe is 0-3ms from the first start time to the first end time. The start and end time information of the tenth subtitle subframe is 3-10ms from the first end time to the second end time. The first subtitle frame is determined as the eleventh subtitle subframe.

[0121] In the embodiment of the present application, when the first start and end time information is different from the second start and end time information and there is an intersection, at least one of the first subtitle frame and the second subtitle frame is effectively split based on the first start time and the first end time, and the second start time and the second end time, to obtain at least three subtitle subframes, so that two subtitle subframes with the same start and end time information in the at least three subtitle subframes can be directly merged. Therefore, the embodiment of the present application improves the connectivity of the subtitle frames on the timeline and improves the richness and synchronization of the subtitle content through precise time matching and content splitting.

[0122] In step 10134, two subtitle subframes with the same start and end time information among the at least three subtitle subframes are merged to obtain a merged subtitle subframe.

[0123] In the embodiment of the present application, when the first start time is greater than the second start time, and the first end time is greater than the second end time, taking the first start and end time information as 3-12ms and the second start and end time information as 0-10ms as an example, the first subtitle subframe and the second subtitle subframe have the same start and end time information of 3-10ms. The first subtitle subframe and the second subtitle subframe can be merged according to the method in the embodiment of step 10132 to obtain a merged subtitle subframe, which will not be described in detail here.

[0124] When the first start time is greater than the second start time, and the first end time is less than the second end time, taking the first start and end time information as 3-8ms and the second start and end time information as 0-10ms as an example, the sixth subtitle subframe and the eighth subtitle subframe have the same start and end time information of 3-8ms. The sixth subtitle subframe and the eighth subtitle subframe can be merged according to the method in the embodiment of step 10132 to obtain a merged subtitle subframe, which will not be described in detail here.

[0125] When the first start time is greater than the second start time, and the first end time is equal to the second end time, taking the first start and end time information as 3-10ms and the second start and end time information as 0-10ms as an example, the ninth subtitle subframe and the eleventh subtitle subframe have the same start and end time information of 3-10ms. When the first start time is equal to the second start time, and the first end time is less than the second end time, taking the first start and end time information as 0-3ms and the second start and end time information as 0-10ms as an example, the ninth subtitle subframe and the eleventh subtitle subframe have the same start and end time information of 0-3ms. The ninth subtitle subframe and the eleventh subtitle subframe can be merged according to the method in the embodiment of step 10132 to obtain a merged subtitle subframe, which will not be repeated here.

[0126] In step 10135, the subtitle subframes with different start and end time information among at least three subtitle subframes and the merged subtitle subframes are spliced ​​in time sequence to obtain subtitle frames.

[0127] In the embodiment of the present application, when the first start time is greater than the second start time, and the first end time is greater than the second end time, taking the first start and end time information as 3-12ms and the second start and end time information as 0-10ms as an example, the start and end time information of the merged subtitle subframe is 3-10ms, the start and end time information of the third subtitle subframe is 10-12ms, and the start and end time information of the fourth subtitle subframe is 0-3ms. In time order, the fourth subtitle subframe, the merged subtitle subframe, and the third subtitle subframe can be spliced ​​to obtain a subtitle frame.

[0128] When the first start time is greater than the second start time, and the first end time is less than the second end time, taking the first start and end time information as 3-8ms and the second start and end time information as 0-10ms as an example, the start and end time information of the merged subtitle subframe is 3-8ms. The start and end time information of the fifth subtitle subframe is 0-3ms, and the start and end time information of the seventh subtitle subframe is 8-10ms. In time order, the fifth subtitle subframe, the merged subtitle subframe, and the seventh subtitle subframe can be spliced ​​to obtain a subtitle frame.

[0129] When the first start time is greater than the second start time, and the first end time is equal to the second end time, taking the first start and end time information as 3-10ms and the second start and end time information as 0-10ms as an example, the start and end time information of the merged subtitle subframe is 3-10ms. The start and end time information of the tenth subtitle subframe is 0-3ms. In time order, the tenth subtitle subframe can be spliced ​​with the merged subtitle subframe to obtain a subtitle frame.

[0130] When the first start time is equal to the second start time, and the first end time is less than the second end time, taking the first start and end time information as 0-3ms and the second start and end time information as 0-10ms as an example, the start and end time information of the merged subtitle subframe is 0-3ms. The start and end time information of the tenth subtitle subframe is 3-10ms. In time order, the merged subtitle subframe can be spliced ​​with the tenth subtitle subframe to obtain a subtitle frame.

[0131] The embodiment of the present application ensures the synchronization of subtitles and video content by accurately matching the start and end time information of the first subtitle frame and the second subtitle frame. When the time information of the first subtitle frame and the second subtitle frame is completely consistent, the richness of the subtitle content can be increased by merging, such as merging bilingual subtitles, providing additional information or annotations, etc. When the start and end time information of the first subtitle frame and the second subtitle frame overlap, the subtitle frames are split and reorganized to ensure the temporal coherence and accuracy of the subtitles, and a more flexible subtitle display effect can be achieved, such as displaying different subtitle content in different time periods.

[0132] It should be noted that the above embodiment is illustrated by taking the first subtitle file and the second subtitle file as an example. In actual application scenarios, multiple subtitle files may also be included. The process of fusing multiple subtitle files can refer to the fusing process of the first subtitle file and the second subtitle file, and will not be repeated.

[0133] In step 102, when playing the three-dimensional target video, a subtitle frame to be displayed is obtained, and the current playing time of the three-dimensional target video is obtained.

[0134] Here, a video playback application can be started to play the three-dimensional target video. At the start time of the playback, for example, 0 ms, the first subtitle frame is obtained from multiple subtitle frames in chronological order as the subtitle frame to be displayed. The current playback time of the three-dimensional target video is obtained as 0 ms. Assuming that the start and end time information of the third subtitle frame among the multiple subtitle frames is 10-20 ms, at this time, the third subtitle frame has been rendered as the subtitle frame to be displayed, then the fourth subtitle frame is determined as the subtitle frame to be displayed, and the current playback time of the three-dimensional target video is 21 ms.

[0135] In step 103, when the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, a text view is determined based on the subtitle frame to be displayed.

[0136] Here, the text view is a view component used to display subtitle text. In the 3D rendering process, the entity Entity is an object in the scene, and the content of the object comes from components (Components) or attachments (Attachment), where attachments (Attachment) provide a method to attach additional content to the entity, which can be models, textures, sounds, etc. When the playback time is within the start and end time information range included in the subtitle frame to be displayed, the subtitle rendering can be achieved by filling the subtitle text into the attachment (Attachment) in the form of a text view, and then attaching the attachment (Attachment) to the subtitle entity. The playback time is within the start and end time information range included in the subtitle frame to be displayed, which means that the playback time is greater than or equal to the start time in the start and end time information, and less than or equal to the end time in the start and end time information.

[0137] In some embodiments, the subtitle frame further includes position information, at least one subtitle text, and format information of each subtitle text. Figure 7 , determining the text view based on the subtitle frame to be displayed in step 103 can be implemented by following steps 1031 to 1032, which are described in detail below.

[0138] In step 1031, a display area is determined based on position information in the subtitle frame to be displayed.

[0139] Here, the position information may be the coordinate information of the display area. The coordinate information may be the coordinates of the upper left vertex and the lower right vertex of the display area, or the relative coordinates of other positions of the display area relative to the three-dimensional target video. Exemplarily, the position information includes coordinate information: (X1, Y1) and (X2, Y2), X1=100, X2=200, Y1=100, Y2=200, then the display area is a rectangular area with the coordinates of the upper left vertex being (100, 100) and the coordinates of the lower right vertex being (200, 200). It should be noted that the embodiment of the present application does not limit the shape of the display area, for example, it may also be a diamond, a circle, etc.

[0140] In step 1032, each subtitle text in the subtitle frame to be displayed is displayed in the display area according to the format information of the subtitle text, so as to obtain a text view.

[0141] Here, for each subtitle text in the subtitle frame to be displayed, the format information of the subtitle text may include at least one of the font, font size, color, etc. The format information of the subtitle text is obtained, and the subtitle text is displayed in the display area according to the format information of the subtitle text to obtain a text view.

[0142] The embodiment of the present application determines the text view through the preset position information and format information in the subtitle frame, and can freely set the position of the subtitles on the video after rendering, as well as the form of subtitle presentation, to improve the viewing effect.

[0143] In some embodiments, the first subtitle frame corresponds to the first language, and the second subtitle frame corresponds to the second language. Determining the text view based on the subtitle frame to be displayed in step 103 can also be implemented in the following manner: first, in response to the received language selection request, the subtitle language of the subtitle to be displayed in the three-dimensional target video is obtained; then, when the subtitle language is the first language, the text view is determined based on the first subtitle frame in the subtitle frame to be displayed; or, when the subtitle language is the second language, the text view is determined based on the second subtitle frame in the subtitle frame to be displayed; or, when the subtitle language is the first language and the second language, the text view is determined based on the subtitle frame to be displayed.

[0144] Here, the subtitle frame to be displayed includes a fused first subtitle frame and a second subtitle frame. The first subtitle frame corresponds to a first language, the second subtitle frame corresponds to a second language, and the first language and the second language are different languages. For example, the first language is Chinese and the second language is English. The user can perform a language selection operation through the terminal, such as selecting Chinese, and then send a language selection request that the subtitle language of the subtitle to be displayed is Chinese to the server. Determine the text view based on the first subtitle frame in the subtitle frame to be displayed. The user can perform a language selection operation through the terminal, such as selecting English, and then send a language selection request that the subtitle language of the subtitle to be displayed is English to the server. Determine the text view based on the second subtitle frame in the subtitle frame to be displayed. Alternatively, the user can perform a language selection operation through the terminal, select Chinese and English, and then send a language selection request that the subtitle language of the subtitle to be displayed is Chinese and English to the server. At this time, the text view is determined based on the subtitle frame to be displayed, and the subtitle text in the first subtitle frame is displayed in the display area corresponding to the position information in the first subtitle frame, and the subtitle text in the second subtitle frame is displayed in the display area corresponding to the position information in the second subtitle frame.

[0145] The embodiment of the present application determines the text view according to the language selected by the user, and can support multi-language subtitles, thereby improving the user experience. The user can select the first language or the second language as the subtitle language as needed, which enhances the flexibility of the viewing experience. In addition, when the user chooses to display the first language and the second language at the same time, the subtitles in the two languages ​​can be viewed at the same time, enriching the subtitle display effect of the video.

[0146] In step 104, the text view is encapsulated on the subtitle entity, and the subtitle entity is rendered on the screen entity.

[0147] The screen entity is used to display the three-dimensional target video.

[0148] Here, after determining the text view of the subtitle frame to be displayed, the text view can be filled into the attachment, and then the attachment can be encapsulated into the subtitle entity to achieve the rendering of the subtitle. The screen entity is an object used to display a three-dimensional target video. For example, the screen entity can be set to a huge 300-inch screen. Set the position of the subtitle entity below the screen entity and add it as a child entity of the screen entity. The screen entity and the subtitle entity will together form a complete viewing scene.

[0149] When playing a three-dimensional target video, when the current playback time of the three-dimensional target video is within the range of the start and end time information included in the subtitle frame to be displayed, the subtitles in the subtitle frame to be displayed are rendered and displayed, and the start and end time information of the subtitle frame to be displayed is accurately matched with the playback time of the three-dimensional target video, thereby ensuring the synchronization accuracy of the subtitles and the video content. Compared with the method of querying the current playback time by polling in the related art and selecting the subtitle frame that can be displayed according to the current playback time, in the embodiment of the present application, the subtitle frame to be displayed is first obtained, and then the current playback time of the three-dimensional target video is obtained. When the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, the subtitle frame to be displayed can be rendered. There is no need to query the current playback time at fixed intervals, unifying the subtitle and video playback processes, saving the resources required for subtitle rendering, and being able to more accurately control the display of subtitles. In addition, the embodiment of the present application determines the text view based on the subtitle frame to be displayed, encapsulates the text view into the subtitle entity, and combines it with the screen entity that displays the three-dimensional target video. Since the subtitle entity can have a three-dimensional effect, the subtitle displayed on the three-dimensional target video after rendering can also have a three-dimensional effect, enriching the display form of the subtitle.

[0150] In some embodiments, see Figure 8 , rendering the subtitle entity onto the screen entity in step 104 can be implemented by following steps 1041 to 1043, which are described in detail below.

[0151] In step 1041, first space coordinates and first size information of the screen entity are obtained.

[0152] Here, first determine the spatial coordinate system. Taking the augmented reality scene as an example, the origin of the spatial coordinate system is (0, 0, 0), and each point in the 3D space can be represented by (x, y, z) coordinates. The actual position of the origin of the spatial coordinate system is not limited in this embodiment of the application. For example, it can be the sole position of the object wearing the augmented display device. The first spatial coordinate of the screen entity can be the center coordinate of the screen entity, and the first spatial coordinate can also include the coordinate of the upper left corner vertex of the screen entity. This embodiment of the application does not limit this. The first size information includes the width (w) and height (h) of the screen entity. Exemplarily, the first spatial coordinate of the screen entity is the coordinate of the upper left corner vertex of the screen entity (0, 0, 0), and the first size information is that the width and height are both 300 inches.

[0153] In step 1042, the second spatial coordinates of the subtitle entity are determined based on the first spatial coordinates, the first size information, and the position information of the text view.

[0154] Here, the second spatial coordinates of the subtitle entity may include the coordinates of the upper left corner vertex and the lower right corner vertex of the subtitle entity. Assume that the position information of the text view is the coordinates of the upper left corner (50, 50) and the lower right corner (100, 100), and the position of the text view on the screen is a rectangular area. The first spatial coordinate of the screen entity is (0, 0, 0), and the first size information is that the width and height are both 300 inches. If the rectangular area where the text view is located does not exceed the display range of the screen entity constructed based on the first spatial coordinates and the first size information, the second spatial coordinates of the subtitle entity can be determined to be (50, 50, 0) and (100, 100, 0). If the rectangular area where the text view is located does not exceed the display range of the screen entity constructed based on the first spatial coordinates and the first size information, the text view can be scaled by the default ratio to obtain the second spatial coordinates of the subtitle entity. Alternatively, the distance between the subtitle entity and the screen entity on the spatial Z axis can be preset, for example, 20, and the second spatial coordinates of the subtitle entity are (50, 50, 20) and (100, 100, 20).

[0155] In step 1043, the screen entity is rendered based on the first spatial coordinates, and the subtitle entity is rendered based on the second spatial coordinates.

[0156] For example, if the first spatial coordinate is the upper left vertex coordinate (0, 0, 0), and the first size information is that the width and height are both 300 inches, then a 300-inch screen entity can be rendered according to the first spatial coordinate as a huge screen to play the three-dimensional target video. The second spatial coordinates of the subtitle entity are (50, 50, 20) and (100, 100, 20), then the subtitle entity can be rendered in a rectangular area composed of the upper left vertex coordinate (50, 50) and the lower right vertex coordinate (100, 100) at a position 20 inches in front of the screen entity according to the second spatial coordinate to display three-dimensional subtitles. The embodiment of the present application can also set the material, shape, tilt angle, light and shadow and other three-dimensional rendering effects of the subtitle entity.

[0157] The embodiment of the present application renders the screen entity by the first spatial coordinate and renders the subtitle entity based on the second spatial coordinate, so as to achieve accurate adaptation of the positions of the screen entity and the subtitle entity, improve the stereoscopic display effect of the screen entity and the subtitle entity, and make the subtitles displayed on the rendered three-dimensional target video also have a three-dimensional effect, thereby creating a more realistic and immersive viewing experience for users.

[0158] In some embodiments, rendering the subtitle entity onto the screen entity in step 104 can also be achieved in the following manner: first, obtaining first color information of an image area on the screen entity displaying the subtitle entity; then, obtaining second color information of a text view in the subtitle entity; and determining the similarity between the first color information and the second color information; then, when the similarity is greater than or equal to a preset similarity threshold, determining a target color from a preset color library based on the first color information; and finally, rendering the subtitle entity based on the target color.

[0159] Here, the image area on the screen entity where the subtitle entity is displayed can be determined based on the second spatial coordinates of the subtitle entity. The first color information of the image area is obtained, and the second color information of the text view can be the display color of the subtitle text. The embodiment of the present application does not limit the method for representing color information. For example, the values ​​of the three channels of RGB (Red, Green, Blue) can be used for representation, and the value range of each channel is usually from 0 to 255. The first color information can be the result of weighted averaging of the RGB values ​​of multiple pixels in the image area. The first color information and the second color information can be encoded respectively to obtain a first vector representation and a second vector representation. The similarity between the first vector representation and the second vector representation is calculated, for example, the cosine similarity is used as the similarity between the first color information and the second color information. The preset similarity threshold is a boundary value used to indicate whether the first color information and the second color information are similar, resulting in unclear display of the subtitles. The embodiment of the present application does not limit the value of the similarity threshold, and it can be set based on experience and actual scenarios. When the similarity is greater than or equal to a preset similarity threshold, the color of the subtitle entity is close to the color of the image in the three-dimensional target video, and the subtitle is not clearly displayed. The similarity between the first color information and other color information in the preset color library can be determined, and a color with a similarity less than the similarity threshold is selected as the target color. The subtitle text in the subtitle entity is rendered based on the target color.

[0160] The embodiment of the present application compares the similarity of the first color information of the image area displaying the subtitle entity on the screen entity and the second color information of the text view in the subtitle entity to ensure that the subtitle color has sufficient contrast with the background color, thereby improving the readability of the subtitles. When the similarity is greater than or equal to a preset similarity threshold, the target color is automatically selected to render the subtitle entity, thereby improving the readability and aesthetics of the subtitles.

[0161] In some embodiments, see Fig. 9 After step 102, the subtitle rendering method provided in the embodiment of the present application can also be implemented through the following steps 201 to 203, which are described in detail below.

[0162] In step 201, when the playing time is not within the range of the start and end time information, the time difference between the start time and the playing time in the start and end time information is determined.

[0163] For example, the playing time is 15ms, the start and end time information of the subtitle frame to be displayed is 18-20ms, and the playing time is not within the start and end time information range. At this time, it can be determined that the time difference between the start time and the playing time in the start and end time information is 18-15=3ms.

[0164] In step 202, after the time difference is determined, a text view is determined based on the subtitle frame to be displayed.

[0165] Continuing with the above example, after an interval of 3ms, the text view can be directly determined based on the subtitle frame to be displayed. The specific process of determining the text view based on the subtitle frame to be displayed can refer to the embodiment of the above step 103, which will not be repeated here.

[0166] In step 203, the text view is encapsulated on the subtitle entity, and the subtitle entity is rendered on the screen entity.

[0167] Here, the specific process of encapsulating the text view into the subtitle entity and rendering the subtitle entity onto the screen entity can refer to the embodiment of the above step 104, which will not be repeated here. The text view encapsulated in the subtitle entity changes with time. For example, the start and end time information of the subtitle frame to be displayed is 18-20ms. At the 20th ms, the text view of the subtitle frame to be displayed is displayed. At the next moment, a new subtitle frame to be displayed is obtained again, and when the playback time is within the start and end time information range of the new subtitle frame to be displayed, the text view of the subtitle frame to be displayed is re-determined.

[0168] After obtaining the subtitle frame to be displayed, the embodiment of the present application compares the start and end time information of the subtitle frame to be displayed with the current playback time of the three-dimensional target video. When the playback time is not within the range of the start and end time information, it will wait for the wake-up mechanism until the time difference between the start time and the playback time in the start and end time information is reached, and then try to retry rendering again, instead of using polling to query the current playback time. This can more accurately control the display of subtitles and improve the synchronization between subtitles and videos.

[0169] Fig.10 is another optional flow chart of the subtitle rendering method provided in the embodiment of the present application, such as Fig.10 As shown, the method includes the following steps 301 to 309:

[0170] Step 301: The terminal receives an interactive operation from a user.

[0171] Here, interactive operations include clicking to play the video, selecting to render subtitles, etc.

[0172] Step 302: The terminal generates a subtitle rendering request in response to the interactive operation.

[0173] Step 303: The terminal sends a subtitle rendering request to the server.

[0174] Step 304: the server obtains a 3D subtitle file corresponding to the 3D target video in response to the subtitle rendering request sent by the terminal, and determines at least one subtitle frame based on the 3D subtitle file.

[0175] Here, each subtitle frame includes at least start and end time information. The specific process of determining at least one subtitle frame based on the three-dimensional subtitle file can refer to step 101 in the above embodiment, and will not be repeated.

[0176] Step 305: The server obtains the subtitle frame to be displayed and obtains the current play time of the 3D target video.

[0177] Here, the specific process of obtaining the subtitle frame to be displayed and obtaining the current playback time of the three-dimensional target video can refer to step 102 in the above embodiment, and will not be repeated.

[0178] Step 306: When the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, the server determines a text view based on the subtitle frame to be displayed.

[0179] Here, the specific process of determining the text view based on the subtitle frame to be displayed can refer to step 103 in the above embodiment, and will not be repeated here.

[0180] Step 307: The server encapsulates the text view onto the subtitle entity, and renders the subtitle entity onto the screen entity.

[0181] Here, the screen entity is used to display the 3D target video. The text view is encapsulated on the subtitle entity, and the subtitle entity is rendered on the screen entity. The specific process of using the screen entity to display the 3D target video can refer to step 104 in the above embodiment, and will not be repeated.

[0182] Step 308: The server sends the rendered subtitle entity and screen entity to the terminal.

[0183] Step 309: The terminal displays the subtitle entity and the screen entity to present the 3D target video with the 3D subtitle.

[0184] In the embodiment of the present application, the server determines whether to render the subtitle frame to be displayed according to the playback time, thereby improving the synchronization between the subtitle and the video content, and can generate a subtitle rendering request according to the user's choice, thereby enhancing the flexibility of subtitle display. After the server renders the subtitle entity and the screen entity, since the subtitle entity can have a three-dimensional effect, the subtitle displayed on the rendered three-dimensional target video can also have a three-dimensional effect, creating a more realistic and immersive viewing experience for users.

[0185] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0186] It should be noted that the embodiment of the present application is described by taking the scenario of an enhanced display device playing a three-dimensional video as an example, but those skilled in the art, based on their understanding of the following text, can apply the subtitle rendering method provided in the embodiment of the present application to any scenario of three-dimensional subtitle rendering, such as virtual reality (VR) applications, augmented reality (AR) applications, online education platforms, game development, multi-language subtitle support, and other scenarios.

[0187] The embodiment of the present application proposes a 3D external subtitle rendering system suitable for enhanced display devices, and implements a solution for synchronous audio and video rendering of external soft subtitles. It adopts a processing flow that is completely consistent with audio and video, and divides the processing of subtitles into five steps: demultiplexing, parsing, post-processing, synchronization, transmission, and rendering. In the post-processing module, the frames of multiple subtitles with exactly the same timestamps are merged, and the frames of multiple subtitles with corresponding contents and overlapping but not completely consistent timestamps are first split and then merged. The merged subtitle frames are sent to the audio and video synchronizer for synchronization, thereby realizing the synchronization of multiple subtitles with audio and video, and the synchronous display of multiple subtitles. After being processed synchronously with the audio and video, the text information of the subtitles is dynamically revealed according to the time range of subtitle display and disappearance. First, based on RealityKit technology, the subtitle text is abstracted into a text view Text and the font color size and other attributes of the subtitle are set. Then the text view is encapsulated into the attachment Attachment, and then the subtitle attachment Attachment is bound to the SubtitleEntity (subtitle entity). Through the subtitle entity, the subtitle's scaling size, spatial coordinates (3D coordinates), geometric shape and material and other attributes can be set, so that the appearance of the subtitle and the spatial position relative to the video can be flexibly defined. Finally, the subtitle entity is added to the carrier (RealityViewContent) of the application that carries the 3D scene and rendered. It should be noted that the display of the video here is also added to the carrier that carries the 3D scene with the screen entity (VideoEntity) as the carrier, that is, the screen entity and the subtitle entity, etc., as basic building block objects, together constitute the viewing scene of the carrier that carries the 3D scene. In addition, there is also a hierarchy between them (usually the subtitles need to be above the screen).

[0188] The embodiments of the present application can solve the following technical problems: unify the subtitle playback process and the audio and video playback process, and pass the subtitle frame through the audio and video synchronization module at the same time, so there is no need to use polling to query the current playback time, and the timing of subtitle exposure can be controlled more accurately. Based on RealityKi technology, the video player has opened up all links from decoding to rendering under augmented reality, expanded the playback capability to the immersive space, and created a giant screen theater with visual effects of over 300 inches, a colorful vision, immersive video and other features for users, leading users into a new era of audio-visual experience. In the giant screen theater scene in the living room, watching 3D movies will also get an unprecedented new experience. The video player has carried out full-link support and optimization of 3D technology on augmented reality wearable devices, which not only supports the playback of 3D videos on augmented reality wearable devices, but also supports the rendering of 3D subtitles. 3D subtitles are an important feature of showing the dialogue between characters in the video or describing the content of the video in text, allowing users to experience a more complete 3D effect in the giant screen theater. Big-budget 3D movies combined with 3D subtitles make the special effects scenes in Hollywood blockbusters seem to jump out of the screen, vivid and lifelike.

[0189] Fig.11 is a schematic diagram of a video playback interface provided in an embodiment of the present application. Fig.11 When the user turns on the enhanced display device and enters the "giant screen theater environment" mode to watch 3D videos, he can select the subtitle list on the playback control bar. The user can select any subtitle in the subtitle list to view the corresponding subtitle information in the video. Fig.12 It is a schematic diagram of displaying Chinese subtitles provided in an embodiment of the present application.

[0190] Fig.13 is a flow chart of the subtitle rendering system provided by the embodiment of the present application. Fig.13 , the subtitle rendering system may include a demultiplexing module 501, a subtitle parsing module 502, an audio decoding module 503, a video decoding module 504, an audio post-processing module 505, a video post-processing module 506, a subtitle post-processing module 507, a synchronization module 508, an audio rendering module 509, a video rendering module 510 and a subtitle rendering module 511. The process of the subtitle rendering system includes five steps: demultiplexing, decoding (parsing), post-processing, synchronization, and rendering. The subtitle rendering process may include two subtitles: subtitles obtained by parsing multimedia files and subtitles obtained by parsing appearance files. Multimedia files can be understood as audio and video files, in which audio and video are merged together, and subtitles may also be included. Plug-in files refer to extracting subtitles from the video separately to form an independent file, similar to a text file. Multimedia files may also not include subtitles.

[0191] The demultiplexing module 501 can parse out file packets one by one from a local file or a network stream. For example, the demultiplexing module 501 can demultiplex an audio packet (corresponding to the audio file in the above embodiment), a video packet (corresponding to the video file in the above embodiment) and multiple subtitle packets (corresponding to the 3D subtitle packets in the above embodiment) from a multimedia file. The demultiplexing module 501 can demultiplex an external file (corresponding to the external subtitle file in the above embodiment) to obtain multiple subtitle packets. Fig.14 , the demultiplexing module 501 can parse the three dialogues into three subtitle packages. A complete line of subtitles is a subtitle package. The structure of the parsed subtitle package can be seen in Fig.15 .

[0192] The subtitle packet is input into the subtitle parsing module 502 in chronological order, and the subtitle text in the subtitle file is sorted from top to bottom according to the time when the subtitle appears. The subtitle parsing module 502 parses the subtitle packet into the corresponding subtitle frame. The audio decoding module 503 can decode the audio packet into the corresponding audio frame, and the video decoding module 504 can decode the video packet into the corresponding video frame. Taking the subtitle packet as "00:02→00:04, X1, X2, Y1, Y2, color=red" as an example, the timestamp (corresponding to the start and end time information in the above embodiment) can be parsed to be 00:02 to 00:04, the position information is (X1, Y1) and (X2, Y2) and the format of each text segment, such as the color is red, etc. The subtitle packet is generally a text segment, and can also include two text segments, and the formats are set separately.

[0193] When playing multiple subtitles, in order to ensure real-time synchronization of the two subtitles, the subtitle post-processing module 507 needs to merge the subtitle frames with the same timestamp of the two subtitles, so that the two subtitle frames will be sent to the subtitle rendering module 511 for synchronous display. In actual situations, the subtitle frame timestamps of the two subtitles may be inconsistent. For example, for Chinese and English subtitles, although the content corresponds, due to language differences and different subtitle producers, an English subtitle frame may last longer than a Chinese subtitle frame. In this case, the subtitle post-processing module 507 needs to split the frames before merging. The subtitle post-processing module 507 splits and merges the subtitle frames of multiple subtitles. Even if there are multiple subtitle files, they usually correspond. For example, the first line of a subtitle file corresponds to the first line of subtitle file 2. Each subtitle file is read in turn, and the subtitle frames at the corresponding position are split and merged line by line. See. Fig.16 In the first case, if the timestamps of the two subtitle frames of the two subtitles arriving at the subtitle post-processing module 507 are completely consistent, the text segments of the two subtitle frames (corresponding to the first subtitle frame and the second subtitle frame in the above embodiment) are directly merged. Fig.17In the second case, although the two subtitle frames of the two channels of subtitles arriving at the subtitle post-processing module 507 have corresponding contents, their timestamps are not completely consistent, and they need to be split and then the frames with the same timestamps are merged. The timestamps of the two subtitle frames are (00:03.000, 00:12.000), i.e., from 3 seconds to 12 seconds, and the timestamps of the other subtitle frame are (00:00.000, 00:10.000), i.e., from 0 seconds to 10 seconds. The overlapping part of these two subtitle frames is 3 seconds to 10 seconds, so the first subtitle frame is split into two time periods: (00:03.000, 00:10.000) and (00:10.000, 00:12.000), and the second subtitle frame is split into two time periods: (00:00.000, 00:03.000) and (00:03.000, 00:10.000). After splitting, the subtitle frames with the same timestamp can be merged.

[0194] The synchronization module 508 is responsible for synchronizing the subtitle and audio time. After the subtitle frame (corresponding to the subtitle frame to be displayed in the above embodiment) arrives at the synchronization module 508, it is compared with the time point at which the audio and video are currently played (corresponding to the playback time in the above embodiment). If the current playback time point is within the range of the subtitle frame timestamp, the subtitle frame will continue to be sent to the subtitle rendering module 511, otherwise it will continue to wait until the playback time point reaches the subtitle frame time range. For example: the subtitle starts to display for 15 seconds. When trying to display the subtitle in the 10th second, it is found that there are still 5 seconds left, so it will keep waiting (similar to the waiting wake-up mechanism of a lock), and try again after 5 seconds instead of polling at intervals. Audio and video are also rendered separately, that is, the audio rendering module 505 is responsible for rendering the sound, and the video rendering module 506 is responsible for rendering the picture.

[0195] In 3D rendering, spatial computing is a core concept that involves technologies and methods for perceiving, understanding, and interacting in three-dimensional space. Spatial computing enables devices to understand and respond to positions, postures, and movements in the physical world, thereby providing a more natural and immersive user experience. In three-dimensional space, the coordinate system of any spatial element is different from the familiar 2D plane, see Fig.18 , the origin of its coordinate system is at the feet of the user wearing the augmented display device. Fig.19It is a coordinate diagram of the vertices of the screen entity provided in the embodiment of the present application. Based on this three-dimensional coordinate system, it can be found that: any view element has its own three-dimensional coordinates (x, y, z), and any view element can be arranged at any position in the immersive space, such as the screen and blue sphere in the figure. The shape, material, etc. of the view element can be defined arbitrarily, such as a square screen and a spherical blue sphere, which are all 3D forms. If you want to modify the size of the view element, the distance from the wearer, etc., you can also adjust it arbitrarily. For example, the smaller the z value of the screen in the figure is, the farther the screen is from the wearer, and vice versa.

[0196] The 3D rendering framework (RealityKit) supports many features such as mapping, lighting, shadows, spatial audio, 3D animation, particle systems, collision detection, etc. It also supports adding 2D controls (SwiftUI) to 3D models. Therefore, RealityKit can be considered a powerful tool for spatial computing. It provides a high-level application programming interface (API) for processing spatial perception, spatial understanding and spatial interaction, making it easier for developers to build AR and VR applications. The 3D rendering framework includes entities, components, systems, meshes, materials, image based lighting, shadows, foveation, dynamic content scaling, spatial audio, video, 3D models, reality views, attached views, animations, particle emitters, collision detection, physics, anchoring, portals, Reality Composer Pro, universal scene description (USD) and material X.

[0197] Entities are basic building blocks used to represent objects in the scene, such as screens and subtitles in theater mode. Each entity can customize its appearance (geometry, material, size, etc.), spatial coordinates, behavior (e.g., position: the position of the entity in three-dimensional space, scale: the scale of the entity, gestures: the gesture interaction behavior of the entity, such as scaling, rotation, etc., events: event processing of the entity, such as click events, collision events, etc., rendering: the material properties of the entity, such as color, texture, visibility control of the entity, such as hiding or showing.) and other properties. For example, the screen entity can be set to a huge 300-inch screen, and the screen entity can also be set to mirror reflection material capabilities, so as to ensure that the video has the ability to reflect screen light when playing, and enhance the immersive experience. In addition, entities can be organized into hierarchical structures to form a parent-child or brother relationship, so as to build more complex scenes. For example, the screen and subtitles together form the user's viewing scene through superimposed layers.

[0198] For example, you can create a plane mesh as a screen and set a black material. Then add the screen entity to the anchor point in the scene. Create a text mesh as a subtitle and set a white material. Then set the position of the subtitle entity below the screen entity and add it as a child entity of the screen entity. Through this hierarchical structure, the screen and subtitle entities will form a complete viewing scene together. The screen entity is the parent entity and the subtitle entity is the child entity.

[0199] Fig. 20 Schematic diagram of the scene hierarchy structure in the augmented reality application provided by the embodiment of the present application. Fig. 20 , view (ARView) is the top-level node, representing the entire scene view. Scene (Scene) This is a direct child node, representing a scene in the view. Anchor entity (Anchor entity) is the two child nodes under the scene, representing the anchor entities in the scene. Anchor entities are used to locate virtual objects in the real world. Entity (Entity) is a child node under each anchor entity, representing a general entity. An entity can be any type of object, virtual or real. Model entity (Model entity) is another child node under each anchor entity, usually used to load and display 3D models. In summary, Fig. 20 It shows the hierarchy from view to scene to anchor entity and finally to entity or model entity. Each anchor entity can contain multiple entities or model entities, and there can be deeper entity nodes under the entity node.

[0200] Entities are objects in the scene, and the content of the objects comes from components or attachments. Attachments provide a way to attach additional content to entities, which can be models, textures, sounds, etc. In the subtitles scenario, the use of attachments is just right. By filling the subtitle text into the attachment in the form of a text view, and then attaching the attachment to the entity, the subtitle rendering is achieved. In addition, when the content of the subtitle changes, the text view in the attachment will also be refreshed in real time to ensure the synchronization of the subtitles with the audio and video. The subtitle entity is the final carrier of the subtitle. Before the attachment of the subtitle object is added, the subtitle cannot be displayed, unless the subtitle is embedded in the video and played as the video plays.

[0201] In the embodiment of the present application: the subtitle attachment is responsible for rendering the subtitle information and attaching it to the subtitle entity, and then adding the subtitle entity to the 3D space, and the hierarchy between the subtitle entity and the screen entity can be achieved by setting different spatial coordinate systems, thereby ensuring that the display of the video picture and subtitle information has a more spatial sense of hierarchy. Fig.21 Schematic diagram of the hierarchical relationship between the subtitle attachment, subtitle entity and screen entity in the embodiment of the present application, such as Fig.21 As shown, the subtitle attachment 601 is encapsulated in the subtitle entity 602, and the subtitle entity 602 is a child entity of the screen entity 603. Through this hierarchical structure, the screen entity 603 and the subtitle entity 602 will together form a complete 3D video viewing scene.

[0202] The embodiment of the present application unifies the subtitle playback process and the audio and video playback process, and at the same time, the subtitle frames also pass through the audio and video synchronization module, so there is no need to use polling to query the current playback time, and the display of subtitles can be controlled more accurately. The embodiment of the present application first synchronizes the transmission of the subtitle text with the audio and video, abstracts the subtitle text into a text view and sets the font, color, size, line width, number of lines, bold and other attributes of the subtitle, then encapsulates the text view into a subtitle attachment, and then binds the subtitle attachment to the subtitle entity. The subtitle entity is used to set the subtitle based on the video's scaling size, 3D coordinates, geometric shape and material attributes. This is the key to the subtitles having a 3D effect when watching a 3D movie. Finally, the subtitle entity is added to the application's carrier that carries the application's 3D elements and rendered.

[0203] The embodiment of the present application realizes the synchronization, transmission and rendering of subtitles with audio and video, but the subtitle information currently transmitted is in the form of the simplest string text. The appearance of the subtitles that the user finally sees is set in a fixed style during the rendering stage, such as a fixed color of white. The limitation here is that the appearance of the subtitles cannot change dynamically with the change of the color of the video. For example, white subtitles cannot be seen clearly when the color of the video is white. Therefore, the embodiment of the present application can also add rich text information of the subtitles to the subtitle package, that is, the external subtitle file itself has attributes such as color, size, thickness, etc., which can be directly set according to the read attribute value during the rendering stage. Rich text is a text that can contain information in multiple formats. It not only contains plain text content, but also contains multiple elements such as fonts and colors. Rich text can provide richer and more diverse text display effects. The format of rich text usually consists of a series of tags and attributes, which define the structure, style and content of the text. For example: <strong> Red bold text< / strong> :<spanstyle="color:red;"> , can give special subtitle style modification according to the context of the current video screen color, context, etc. The embodiment of the present application can also perform color recognition of the video screen for the image area rendered with subtitles. For example, if the color is similar to the color of the default subtitles, the subtitle color is changed to another color to highlight the subtitle information and enhance the user's overall viewing experience.

[0204] It is understandable that in the embodiments of the present application, related data such as user information is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards.

[0205] The following is a description of an exemplary structure of a subtitle rendering device 455 provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the subtitle rendering device 455 of the memory 450 may include: a subtitle frame determination module 4551, which is used to obtain a three-dimensional subtitle file corresponding to the three-dimensional target video, and determine at least one subtitle frame based on the three-dimensional subtitle file, each subtitle frame including at least start and end time information; a playback time acquisition module 4552, which is used to obtain the subtitle frame to be displayed when playing the three-dimensional target video, and obtain the current playback time of the three-dimensional target video; a text view determination module 4553, which is used to determine the text view based on the subtitle frame to be displayed when the playback time is within the range of the start and end time information included in the subtitle frame to be displayed; a rendering module 4554, which is used to encapsulate the text view onto the subtitle entity, and render the subtitle entity onto the screen entity, and the screen entity is used to display the three-dimensional target video.

[0206] In some embodiments, when the 3D subtitle file is an embedded subtitle file, the subtitle frame determination module 4551 is also used to obtain a multimedia file corresponding to the 3D target video; demultiplex the multimedia file to obtain an audio file, a video file and a 3D subtitle file of the 3D target video, the 3D subtitle file including at least one 3D subtitle package; and parse and process at least one 3D subtitle package to obtain at least one subtitle frame.

[0207] In some embodiments, when the 3D subtitle file is an external subtitle file, the subtitle frame determination module 4551 is further used to demultiplex the 3D subtitle file to obtain at least one 3D subtitle package; and parse at least one 3D subtitle package to obtain at least one subtitle frame.

[0208] In some embodiments, when the three-dimensional subtitle file includes a first subtitle file and a second subtitle file, the subtitle frame determination module 4551 is further used to parse the first subtitle file to determine at least one first subtitle frame; parse the second subtitle file to determine at least one second subtitle frame; for each first subtitle frame, merge the first subtitle frame and the second subtitle frame corresponding to the first subtitle frame to obtain a subtitle frame.

[0209] In some embodiments, the subtitle frame determination module 4551 is also used to merge the first subtitle frame and the second subtitle frame to obtain a subtitle frame when the first start and end time information in the first subtitle frame is the same as the second start and end time information in the second subtitle frame; when the first start and end time information in the first subtitle frame is different from the second start and end time information in the second subtitle frame and there is an intersection, split at least one of the first subtitle frame and the second subtitle frame based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information to obtain at least three subtitle subframes; merge two subtitle subframes with the same start and end time information among the at least three subtitle subframes to obtain a merged subtitle subframe; splice the subtitle subframes with different start and end time information among the at least three subtitle subframes and the merged subtitle subframe in chronological order to obtain a subtitle frame.

[0210] In some embodiments, when the first start time is greater than the second start time, and the first end time is greater than the second end time; the subtitle frame determination module 4551 is also used to split the first subtitle frame to obtain the first subtitle subframe based on the first start time and the second end time, and split the second subtitle frame to obtain the second subtitle subframe; split the first subtitle frame to obtain the third subtitle subframe based on the first end time and the second end time; split the second subtitle frame to obtain the fourth subtitle subframe based on the first start time and the second start time.

[0211] In some embodiments, when the first start time is greater than the second start time, and the first end time is less than the second end time; the subtitle frame determination module 4551 is also used to split the second subtitle frame to obtain the fifth subtitle subframe based on the first start time and the second start time; split the second subtitle frame to obtain the sixth subtitle subframe based on the first start time and the first end time; split the second subtitle frame to obtain the seventh subtitle subframe based on the first end time and the second end time; and determine the first subtitle frame as the eighth subtitle subframe.

[0212] In some embodiments, when the first start time is greater than the second start time and the first end time is equal to the second end time, or when the first start time is equal to the second start time and the first end time is less than the second end time; the subtitle frame determination module 4551 is also used to split the second subtitle frame based on the first start time and the first end time to obtain a ninth subtitle subframe and a tenth subtitle subframe; and determine the first subtitle frame as the eleventh subtitle subframe.

[0213] In some embodiments, the subtitle rendering device 455 also includes a synchronization module, which is used to determine the time difference between the start time and the playback time in the start and end time information when the playback time is not within the range of the start and end time information; after the interval time difference, determine the text view based on the subtitle frame to be displayed; encapsulate the text view onto the subtitle entity, and render the subtitle entity onto the screen entity.

[0214] In some embodiments, the subtitle frame also includes position information, at least one subtitle text and format information of each subtitle text; the text view determination module 4553 is also used to determine the display area based on the position information in the subtitle frame to be displayed; each subtitle text in the subtitle frame to be displayed is displayed in the display area according to the format information of the subtitle text to obtain a text view.

[0215] In some embodiments, the rendering module 4554 is further used to obtain the first spatial coordinates and first size information of the screen entity; determine the second spatial coordinates of the subtitle entity based on the first spatial coordinates, the first size information and the position information of the text view; render the screen entity based on the first spatial coordinates, and render the subtitle entity based on the second spatial coordinates.

[0216] In some embodiments, the first subtitle frame corresponds to the first language, and the second subtitle frame corresponds to the second language; the text view determination module 4553 is also used to obtain the subtitle language of the subtitles to be displayed in the three-dimensional target video in response to the received language selection request; when the subtitle language is the first language, the text view is determined based on the first subtitle frame in the subtitle frames to be displayed; when the subtitle language is the second language, the text view is determined based on the second subtitle frame in the subtitle frames to be displayed; when the subtitle language is the first language and the second language, the text view is determined based on the subtitle frames to be displayed.

[0217] In some embodiments, the rendering module 4554 is also used to obtain first color information of an image area displaying a subtitle entity on a screen entity; obtain second color information of a text view in the subtitle entity; determine the similarity between the first color information and the second color information; when the similarity is greater than or equal to a preset similarity threshold, determine a target color from a preset color library based on the first color information; and render the subtitle entity based on the target color.

[0218] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer readable storage medium, and the processor executes the computer executable instruction, so that the electronic device performs the subtitle rendering method described in the embodiment of the present application.

[0219] The present application embodiment provides a computer-readable storage medium in which computer executable instructions or computer programs are stored. When the computer executable instructions or computer programs are executed by a processor, the processor will be caused to execute the subtitle rendering method provided by the present application embodiment, for example, Figure 3 The subtitle rendering method is shown.

[0220] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0221] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0222] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0223] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0224] In summary, the embodiment of the present application unifies the subtitle playback process and the audio and video playback process and structure, improves the consistency, understandability and maintainability of the architecture design, avoids the waste of system resources caused by polling-style query of playback time, and can more accurately synchronize subtitles with audio and video, while also providing users with the ability to arbitrarily select subtitles in various languages. In terms of subtitle rendering, ordinary text subtitles are rendered into a spatial view with 3D effects, which greatly improves the 3D viewing experience.

[0225] The above are only embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A subtitle rendering method, characterized in that: The method comprises: Acquire a 3D subtitle file corresponding to the 3D target video, and determine at least one subtitle frame based on the 3D subtitle file, each of the subtitle frames including at least start and end time information; When playing the three-dimensional target video, obtaining a subtitle frame to be displayed, and obtaining a current playing time of the three-dimensional target video; When the playback time is within the range of the start and end time information included in the subtitle frame to be displayed, determining a text view based on the subtitle frame to be displayed; Encapsulating the text view onto a subtitle entity, and rendering the subtitle entity onto a screen entity, wherein the screen entity is used to display the three-dimensional target video; When the 3D subtitle file includes a first subtitle file and a second subtitle file, the determining at least one subtitle frame based on the 3D subtitle file includes: Parsing the first subtitle file to determine at least one first subtitle frame, parsing the second subtitle file to determine at least one second subtitle frame; For each first subtitle frame, when the first start and end time information in the first subtitle frame is different from the second start and end time information in the second subtitle frame and there is an intersection, based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information, at least one of the first subtitle frame and the second subtitle frame is split to obtain at least three subtitle subframes; Merging two subtitle subframes with the same start and end time information among the at least three subtitle subframes to obtain a merged subtitle subframe; The subtitle subframes with different start and end time information among the at least three subtitle subframes and the merged subtitle subframe are spliced ​​in time sequence to obtain the subtitle frame.

2. The method according to claim 1, characterized in that When the 3D subtitle file is an embedded subtitle file, the step of obtaining the 3D subtitle file corresponding to the 3D target video and determining at least one subtitle frame based on the 3D subtitle file includes: Acquire a multimedia file corresponding to the three-dimensional target video; Demultiplexing the multimedia file to obtain an audio file, a video file and a 3D subtitle file of the 3D target video, wherein the 3D subtitle file includes at least one 3D subtitle package; The at least one three-dimensional subtitle package is parsed to obtain at least one subtitle frame.

3. The method according to claim 1, characterized in that: When the 3D subtitle file is an external subtitle file, the step of obtaining the 3D subtitle file corresponding to the 3D target video and determining at least one subtitle frame based on the 3D subtitle file includes: Demultiplexing the 3D subtitle file to obtain at least one 3D subtitle package; The at least one 3D subtitle package is parsed to obtain at least one subtitle frame.

4. The method according to claim 1, characterized in that The method further comprises: For each first subtitle frame, when the first start and end time information in the first subtitle frame is the same as the second start and end time information in the second subtitle frame, the first subtitle frame and the second subtitle frame are merged to obtain the subtitle frame.

5. The method according to claim 1, characterized in that When the first start time is greater than the second start time, and the first end time is greater than the second end time; The method of splitting at least one of the first subtitle frame and the second subtitle frame based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information, to obtain at least three subtitle subframes, includes: Based on the first start time and the second end time, split the first subtitle frame to obtain a first subtitle subframe, and split the second subtitle frame to obtain a second subtitle subframe; Based on the first end time and the second end time, splitting the first subtitle frame to obtain a third subtitle subframe; Based on the first start time and the second start time, a fourth subtitle subframe is obtained by splitting the second subtitle frame.

6. The method according to claim 1, characterized in that When the first start time is greater than the second start time, and the first end time is less than the second end time; The method of splitting at least one of the first subtitle frame and the second subtitle frame based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information, to obtain at least three subtitle subframes, includes: Splitting the second subtitle frame to obtain a fifth subtitle subframe based on the first start time and the second start time; Splitting the second subtitle frame to obtain a sixth subtitle subframe based on the first start time and the first end time; Splitting the second subtitle frame to obtain a seventh subtitle subframe based on the first end time and the second end time; The first subtitle frame is determined as an eighth subtitle subframe.

7. The method according to claim 1, characterized in that When the first start time is greater than the second start time and the first end time is equal to the second end time, or when the first start time is equal to the second start time and the first end time is less than the second end time; The method of splitting at least one of the first subtitle frame and the second subtitle frame based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information, to obtain at least three subtitle subframes, includes: Based on the first start time and the first end time, split the second subtitle frame to obtain a ninth subtitle subframe and a tenth subtitle subframe; The first subtitle frame is determined as the eleventh subtitle subframe.

8. The method according to claim 1, characterized in that The method further comprises: When the playing time is not within the range of the start and end time information, determining the time difference between the start time in the start and end time information and the playing time; After the time difference is separated, determining a text view based on the subtitle frame to be displayed; The text view is encapsulated on a subtitle entity, and the subtitle entity is rendered on a screen entity.

9. The method according to any one of claims 1 to 8, characterized in that: The subtitle frame further includes position information, at least one subtitle text and format information of each subtitle text; The determining of the text view based on the subtitle frame to be displayed comprises: Determine a display area based on position information in the subtitle frame to be displayed; Each of the subtitle texts in the subtitle frame to be displayed is displayed in the display area according to the format information of the subtitle text to obtain the text view.

10. The method according to claim 9, characterized in that The rendering of the subtitle entity onto the screen entity comprises: Acquire first spatial coordinates and first size information of the screen entity; Determining a second spatial coordinate of the subtitle entity based on the first spatial coordinate, the first size information, and the position information of the text view; The screen entity is rendered based on the first spatial coordinates, and the subtitle entity is rendered based on the second spatial coordinates.

11. The method according to claim 1, characterized in that: The first subtitle frame corresponds to a first language, and the second subtitle frame corresponds to a second language; and determining a text view based on the subtitle frame to be displayed includes: In response to the received language selection request, acquiring a subtitle language of subtitles to be displayed in the three-dimensional target video; When the subtitle language is the first language, determining a text view based on a first subtitle frame in the subtitle frames to be displayed; When the subtitle language is the second language, determining a text view based on a second subtitle frame in the subtitle frames to be displayed; When the subtitle language is the first language and the second language, a text view is determined based on the subtitle frame to be displayed.

12. The method according to any one of claims 1 to 8, characterized in that: The rendering of the subtitle entity onto the screen entity comprises: Acquire first color information of an image area on the screen entity displaying the subtitle entity; Obtaining second color information of the text view in the subtitle entity; determining a similarity between the first color information and the second color information; When the similarity is greater than or equal to a preset similarity threshold, determining a target color from a preset color library based on the first color information; The subtitle entity is rendered based on the target color.

13. A subtitle rendering device, characterized in that: The device comprises: A subtitle frame determination module is used to obtain a three-dimensional subtitle file corresponding to a three-dimensional target video, determine at least one subtitle frame based on the three-dimensional subtitle file, and each of the subtitle frames includes at least start and end time information; when the three-dimensional subtitle file includes a first subtitle file and a second subtitle file, parse the first subtitle file to determine at least one first subtitle frame, and parse the second subtitle file to determine at least one second subtitle frame; for each first subtitle frame, when the first start and end time information in the first subtitle frame is different from the second start and end time information in the second subtitle frame and there is an intersection, split at least one of the first subtitle frame and the second subtitle frame based on the first start time and the first end time in the first start and end time information, and the second start time and the second end time in the second start and end time information, to obtain at least three subtitle subframes; merge two subtitle subframes with the same start and end time information among the at least three subtitle subframes to obtain a merged subtitle subframe; splice the subtitle subframes with different start and end time information among the at least three subtitle subframes and the merged subtitle subframe in time order to obtain the subtitle frame; A playback time acquisition module, used to acquire a subtitle frame to be displayed and acquire a current playback time of the three-dimensional target video when playing the three-dimensional target video; A text view determination module, configured to determine a text view based on the subtitle frame to be displayed when the playback time is within the start and end time information range included in the subtitle frame to be displayed; A rendering module is used to encapsulate the text view into a subtitle entity, and render the subtitle entity onto a screen entity, wherein the screen entity is used to display the three-dimensional target video.

14. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; The processor is configured to implement the subtitle rendering method according to any one of claims 1 to 12 when executing the computer executable instructions or computer programs stored in the memory.

15. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the subtitle rendering method according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the subtitle rendering method according to any one of claims 1 to 12 is implemented.

Images