Display apparatus and subtitle display method

Abstract

Some embodiments of this application illustrate a display device and a subtitle display method. The method includes: acquiring first content, first display characteristics, a first display start time, and a first display end time of a first subtitle segment; controlling the display to display the first content according to the first display characteristics when the first display start time is reached; acquiring second content, second display characteristics, and a second display start time of a second subtitle segment; calculating the difference between the second display start time and the first display end time; if the difference is less than a preset value, controlling the display to display the second content according to the second display characteristics when the first display end time is reached. In the embodiments of this application, when the display time interval between two adjacent segments is short, blank content can be omitted, and the content of the next subtitle segment can be displayed directly, avoiding subtitle flickering and uneven content, thus improving the user experience.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a method for displaying subtitles. Background Technology

[0002] TTML (Timed Text Markup Language) was adopted as the subtitle standard by the ATSC 3.0 broadcast system due to its good compatibility, simple configuration, and convenient synchronization. However, TTML subtitles suffer from several drawbacks. Small subtitle segments, numerous transmission segments, and short segment intervals, coupled with insufficient hardware graphics performance, can lead to flickering and uneven content display during rapid multi-segment transmission and display, resulting in a poor user experience. For example, when using TTML for multi-segment display, segment A needs to be displayed first, followed by a blank space, and then segment B. The process from displaying the blank space to re-displaying segment B requires a significant amount of time. If the time difference between the start time of segment B and the end time of segment A is less than the time taken from displaying the blank space to displaying segment B, segment B will be delayed, resulting in a sudden short blank space in the subtitle. If the content of segment A and segment B is identical, it will cause subtitle flickering, leading to a poor user experience. Summary of the Invention

[0003] Some embodiments of this application provide a display device and a subtitle display method. When the display time interval between two adjacent segments is short, blank content can be omitted, and the content of the next subtitle segment can be displayed directly, avoiding subtitle flickering and uneven content, thus improving the user experience.

[0004] In a first aspect, some embodiments of this application provide a display device, including:

[0005] monitor;

[0006] The controller is configured as follows:

[0007] Obtain the first content, first display characteristics, first display start time, and first display end time of the first subtitle segment;

[0008] When the first display startup time is reached, the display is controlled to display the first content according to the first display characteristics;

[0009] Obtain the second content, second display characteristics, and second display start time of the second subtitle segment;

[0010] Calculate the difference between the second display start time and the first display end time;

[0011] If the difference is less than a preset value, when the first display end time is reached, the display is controlled to display the second content according to the second display characteristics.

[0012] In some embodiments, the controller's action of controlling the display to display second content according to the second display characteristics upon reaching the first display end time is further configured to:

[0013] Determine whether the first content and the second content are the same;

[0014] If the first content is the same as the second content, then when the first display end time is reached, the display is controlled to display the second content according to the second display characteristics.

[0015] In some embodiments, the controller is configured to:

[0016] If the first content is different from the second content, then when the first display end time is reached, the first content is cleared, and the display is controlled to display the second content according to the second display characteristics.

[0017] In some embodiments, the controller is configured to:

[0018] If the first content is different from the second content, then determine whether the first content and the second content are related.

[0019] If the first content is related to the second content, when the first display end time is reached, the display is controlled to display the second content according to the second display characteristics;

[0020] If the first content and the second content are not related, then when the first display end time is reached, the first content is cleared, and the display is controlled to display the second content according to the second display characteristics.

[0021] In some embodiments, the controller's function of determining whether the first content and the second content are associated is further configured to:

[0022] Determine whether the first content ends with a sentence-final punctuation mark;

[0023] If the first content ends with a sentence-final punctuation mark, then the first content and the second content are determined to be unrelated.

[0024] If the first content does not end with a sentence punctuation mark, then the first content and the second content are determined to be related.

[0025] In some embodiments, the controller's action of controlling the display to display second content according to the second display characteristics upon reaching the first display end time is further configured to:

[0026] Determine whether the first content and the second content are the same;

[0027] If the first content is the same as the second content, then determine whether the first display characteristic is the same as the second display characteristic;

[0028] If the first display characteristic is the same as the second display characteristic, when the first display end time is reached, control the display to display the second content according to the second display characteristic;

[0029] If the first display characteristic is different from the second display characteristic, then when the first display end time is reached, the first content is cleared, and the display is controlled to display the second content according to the second display characteristic.

[0030] In some embodiments, the controller is configured to:

[0031] If the difference is greater than or equal to a preset value, the first content is cleared when the first display end time is reached;

[0032] When the second display startup time is reached, the display is controlled to display the second content according to the second display characteristics.

[0033] Secondly, some embodiments of this application provide a subtitle display method, including:

[0034] Obtain the first content, first display characteristics, first display start time, and first display end time of the first subtitle segment;

[0035] When the first display startup time is reached, the display is controlled to display the first content according to the first display characteristics;

[0036] Obtain the second content, second display characteristics, and second display start time of the second subtitle segment;

[0037] Calculate the difference between the second display start time and the first display end time;

[0038] If the difference is less than a preset value, when the first display end time is reached, the display is controlled to display the second content according to the second display characteristics.

[0039] In some embodiments, the step of controlling the display to display second content according to the second display characteristics when the first display end time is reached includes:

[0040] Determine whether the first content and the second content are the same;

[0041] If the first content is the same as the second content, then when the first display end time is reached, the display is controlled to display the second content according to the second display characteristics.

[0042] In some embodiments, the method further includes:

[0043] If the first content is different from the second content, then when the first display end time is reached, the first content is cleared, and the display is controlled to display the second content according to the second display characteristics.

[0044] Some embodiments of this application provide a display device and a subtitle display method. The display interval (difference) between two adjacent subtitle segments is determined by acquiring their display start and end times. This difference is compared with a preset value, which is the time required for the hardware to transition from displaying a blank space to displaying the next subtitle segment within that timeframe. If the difference is smaller than the preset value, it indicates that the hardware cannot complete the transition from displaying a blank space to displaying the next subtitle segment within that timeframe. Therefore, in this case, the blank space operation can be cancelled, and the next subtitle segment can be displayed directly, avoiding subtitle flickering and uneven content, thus improving the user experience. Attached Figure Description

[0045] Figure 1 An operational scenario between a display device and a control device according to some embodiments is illustrated;

[0046] Figure 2 A hardware configuration block diagram of a control device 100 according to some embodiments is shown;

[0047] Figure 3 A hardware configuration block diagram of a display device 200 according to some embodiments is shown;

[0048] Figure 4 A software configuration diagram of a display device 200 according to some embodiments is shown;

[0049] Figure 5 A flowchart illustrating the internal workings of a TTML module according to some embodiments is shown;

[0050] Figure 6 A schematic diagram illustrating a first type of subtitle display effect according to some embodiments is shown;

[0051] Figure 7 A schematic diagram illustrating a second type of subtitle display effect according to some embodiments is shown;

[0052] Figure 8 A system architecture diagram of a display device 200 according to some embodiments is shown;

[0053] Figure 9 An interaction diagram related to a TTML module is shown according to some embodiments;

[0054] Figure 10 A schematic diagram of a live streaming application settings interface provided according to some embodiments is shown;

[0055] Figure 11 A schematic diagram of a live streaming application subtitle settings interface is shown according to some embodiments;

[0056] Figure 12 A flowchart illustrating the internal workings of another TTML module, according to some embodiments, is shown;

[0057] Figure 13 A flowchart illustrating the internal workings of yet another TTML module provided according to some embodiments is shown;

[0058] Figure 14 A flowchart illustrating the execution steps of a subtitle display control module according to some embodiments is shown;

[0059] Figure 15 A schematic diagram of a user interface for personalized caption display settings is shown according to some embodiments;

[0060] Figure 16 A schematic diagram illustrating a third type of subtitle display effect according to some embodiments is shown;

[0061] Figure 17 A schematic diagram illustrating a fourth type of subtitle display effect according to some embodiments is shown;

[0062] Figure 18 A schematic diagram illustrating a fifth type of subtitle display effect according to some embodiments is shown;

[0063] Figure 19 A schematic diagram illustrating a sixth type of subtitle display effect according to some embodiments is shown;

[0064] Figure 20 A schematic diagram illustrating a seventh type of subtitle display effect provided according to some embodiments is shown. Detailed Implementation

[0065] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0066] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0067] The terms "first," "second," and "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0068] The term “comprising” and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0069] The display device provided in this application can have various implementation forms, such as a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figure 1 and Figure 2 This is one specific embodiment of the display device of this application.

[0070] Figure 1 This is a schematic diagram illustrating the operational scenario between the display device and the control unit according to the embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0071] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0072] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, an application running on the smart device may be used to control the display device 200.

[0073] In some embodiments, the display device may receive instructions not through the aforementioned smart devices or control devices, but through touch or gestures.

[0074] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0075] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0076] Figure 2 An exemplary block diagram of the configuration of the control device 100 according to an exemplary embodiment is shown. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0077] like Figure 3 The display device 200 includes at least one of the following: a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0078] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first to an nth interface for input / output.

[0079] The display 260 includes a display screen assembly for presenting images, a driving assembly for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0080] The display 260 can be an LCD display, an OLED display, or a projection display, and can also be a projection device and a projection screen.

[0081] The display 260 also includes a touch screen, which is used to receive input control commands from the user's fingers by sliding or clicking on the touch screen.

[0082] The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of the following: a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0083] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote control).

[0084] Detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0085] The external device interface 240 may include, but is not limited to, one or more of the following: High Definition Multimedia Interface (HDMI), analog or high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0086] The tuner / demodulator 210 receives broadcast television signals via wired or wireless means, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals.

[0087] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0088] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the monitor 260, the controller 250 can execute operations related to the object selected by the user command.

[0089] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0090] Users can input commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, users can input commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0091] A "user interface" is the medium through which an application or operating system interacts and exchanges information with the user. It converts information from its internal form to a form that the user can accept. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0092] like Figure 4 As shown, the display device system is divided into three layers, from top to bottom: the application layer, the middleware layer, and the hardware layer.

[0093] The application layer mainly includes commonly used applications on TVs, as well as the application framework. The commonly used applications are mainly browser-based applications, such as HTML5 apps, and native apps.

[0094] An application framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interface for these functions (toolbar, status bar, menu, dialog box).

[0095] Native apps can support online or offline access, push notifications, or access to local resources.

[0096] The middleware layer includes various television protocols, multimedia protocols, and system components. Middleware can use the basic services (functions) provided by system software to connect different parts of application systems or different applications on the network, achieving resource sharing and function sharing.

[0097] The hardware layer mainly includes the HAL interface, hardware, and drivers. The HAL interface is a unified interface for all TV chips, with the specific logic implemented by each chip. The drivers mainly include: audio drivers, display drivers, Bluetooth drivers, camera drivers, Wi-Fi drivers, USB drivers, HDMI drivers, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.), and power drivers.

[0098] TTML is an XML-based temporal text markup language. It is designed for use in global captioning and subtitle delivery applications, simplifying interoperability and maintaining consistency and compatibility with other captioning file formats. Due to its high compatibility, simple configuration, and convenient synchronization, the ATSC 3.0 broadcasting system made a breakthrough by introducing TTML as its captioning standard.

[0099] In some embodiments, the flowchart inside the TTML module is as follows: Figure 5 As shown. The TTML module includes CC_Interface (Closed Caption_Interface, subtitle interface), CC_Parser (subtitle syntax analyzer), CC_Process (subtitle process), and CC_Display_Control (subtitle display control module).

[0100] After receiving the command to start TTML display at the subtitle interface, the DM (DataManager, data management module) sends the acquired first subtitle segment data to the subtitle syntax analysis program. The subtitle syntax analysis program parses the first subtitle segment data to obtain relevant information, including the subtitle content, display start time, display end time, and display characteristics. Display characteristics include the subtitle font type, font size, font transparency, font background, and subtitle position. These display characteristics can be set by default or customized by the user. The subtitle syntax analysis program preprocesses the parsed first subtitle segment data and sends it to the subtitle process, which then sends the preprocessed data to the subtitle display control module. The purpose of preprocessing is to organize the relevant information of the first subtitle segment into a unified data structure and place it in a queue for easy transmission and retrieval of subtitle data. After receiving the preprocessed first subtitle segment data, the subtitle display control module performs operations related to the display of the first subtitle segment content. When the display end time of the first subtitle segment is reached, the display of the first subtitle segment content ends, displaying blank content.

[0101] The data management module sends the acquired second subtitle segment data to the subtitle syntax analysis program. After parsing the data, the program obtains relevant information about the second subtitle segment. The analysis program then preprocesses the parsed data and sends it to the subtitle process, which in turn sends the preprocessed data to the subtitle display control module. When the display start time for the second subtitle segment is reached, the subtitle display control module performs operations related to displaying the subtitle segment content. When the display end time for the second subtitle segment is reached, the display of the subtitle segment content is stopped.

[0102] However, TTML subtitles, due to their small fragment size, numerous transmitted fragments, and short fragment intervals, coupled with insufficient hardware graphics performance, can lead to subtitle flickering and uneven content display during rapid multi-fragment transmission and display, resulting in a poor user experience. Specifically, due to hardware limitations, there must be at least a 40ms interval between displaying blank content and re-displaying the second subtitle fragment. If the difference between the end time of the first subtitle fragment's display and the start time of the second subtitle fragment's display is greater than or equal to 40ms, then the display of the first subtitle fragment, blank content, and the second subtitle fragment can be completed normally within the specified time. Figure 6 As shown, the content of the first subtitle segment is "Beijing", and the display time period is 1000ms-3000ms. The content of the second subtitle segment is "Shanghai", and the display time period is 3100ms-5100ms. That is, the blank display time is 100ms. So the effect of displaying "Beijing" → displaying blank space → displaying "Shanghai" can be completed according to the original time.

[0103] If the difference between the end time of the first subtitle segment and the start time of the second subtitle segment is less than 40ms, then... Figure 7 As shown, the first subtitle segment contains "Beijing" and is displayed for a period of 1000ms-3000ms. The second subtitle segment contains "Shanghai" and is displayed for a period of 3001ms-5000ms, meaning there is a 1ms blank time. The original intention was to achieve... Figure 7 The effect shown in effect 1 is possible, but due to hardware limitations, displaying blank content within 1ms is not feasible; it can only achieve the effect shown in effect 2. Figure 7 The effect shown in Effect 2 is as follows: the second subtitle segment should have appeared at 3001ms, but it only appeared at 3040ms. The blank content displayed between 3001ms and 3040ms will cause the subtitle to pause and create an uneven display experience for the user.

[0104] If the first subtitle segment and the second subtitle segment are in a contextual relationship, for example, if the content of the first subtitle segment and the content of the second subtitle segment are a single sentence, the display effect is that a small blank space suddenly appears between the first half and the second half of the sentence, that is, a pause in the sentence.

[0105] If the content of the first subtitle segment is the same as that of the second subtitle segment, for example, the subtitle will flash once on the screen. If multiple segments have the same content, the same content will flash differently on the screen.

[0106] In summary, TTML subtitles may exhibit some issues that affect the display quality due to hardware performance limitations and unreasonable sentence segmentation design, leading to a poor user experience.

[0107] To address the above technical problems, this application provides a display device 200. The structure and functions of each part of the display device 200 can be found in the above embodiments. Furthermore, based on the display device 200 shown in the above embodiments, this embodiment further improves some functions of the display device 200.

[0108] Figure 8 This is a system architecture diagram of display device 200. The system architecture of display device 200 includes an application (APP) layer, a service layer, and a hardware (HAL) layer. The application layer includes the live TV application (Live TV) and the TV input channel (TvInput). TTML subtitles are displayed on top of the Live TV screen. The application layer is used to interact with the TTML module to complete TTML display, stop display, and set and switch display characteristics.

[0109] The middleware layer includes a service layer. The service layer is the logical layer of TTML. The DFB (DIRECT_FB, drawing engine module) in the same layer serves as the drawing service, used to draw TTML subtitles. The TTML services in the service layer include DM (DataManager, data management module), GFXHAL (GRAPHICS Hardware astract layer, graphics hardware abstraction module), and the TTML module. The data management module is the TTML data aggregation layer, used to distribute raw TTML data. The graphics hardware abstraction module serves as the interface layer for TTML subtitle display features, managing the drawing of various TTML lines, text, and rectangles. The TTML module implements TTML subtitle data parsing, data analysis, prediction, data processing, and display features.

[0110] The hardware layer is the source layer for TTML display data; all TTML subtitle fragment data originates from the hardware layer. The hardware layer includes the Demux (demultiplexing module), a hardware module of the main chip used to filter various types of data. It can filter out TTML subtitle fragment data from the bitstream and send the TTML subtitle fragment data to the data management module.

[0111] Figure 9 This is an interaction diagram related to the TTML module. The TTML module includes a subtitle interface, a subtitle syntax analyzer, a subtitle process, and a subtitle display control module. The system interaction logic is as follows: LiveTv serves as the source of system command settings, controlling the display and attributes of TTML subtitles. The data management module serves as the source of TTML subtitle data, used to input subtitle segment data into the TTML module. The Player serves as the source of the TTML synchronization clock, responsible for providing TTML synchronization time. The graphics hardware abstraction module serves as the drawing interface layer, implementing the TTML content drawing and display characteristics, and is drawn by the drawing engine module. The performance of this part is greatly affected by hardware.

[0112] In some embodiments, the user inputs a command to open a live streaming application, which is used to play live television programs, particularly ATSC 3.0 programs. After the live streaming application is opened, the TTML subtitle function is enabled by default. The display device 200 receives audio and video data and subtitle segment data, processes the audio and video data, plays it, and displays the content of the subtitle segments on the corresponding video data.

[0113] In some embodiments, after a user opens the live streaming application, the TTML subtitle function is not enabled by default. Instead, the user can choose to enable or disable the TTML subtitle function by setting a TTML subtitle function switch. For example, when a user opens the live streaming application and presses the menu button on the control device 100, the following is displayed: Figure 10 The user interface shown includes a subtitle control 101. It receives user input indicating that the subtitle control 101 is selected and displays the following... Figure 11 The user interface shown. Figure 11 The user interface includes a subtitle function control 111, which allows the user to control the TTML subtitle function to be turned on or off. When the user's command to turn on the TTML subtitle function is received, the subtitle interface starts the TTML subtitle display.

[0114] In some embodiments, such as Figure 12 As shown, after the subtitle interface receives the command to enable the TTML subtitle function, the demultiplexing module sends the first subtitle segment data filtered from the bitstream to the data management module. The data management module then sends the first subtitle segment data to the subtitle syntax analysis program. After parsing the first subtitle segment data, the subtitle syntax analysis program obtains the relevant information of the first subtitle segment. The subtitle syntax analysis program preprocesses the parsed first subtitle segment data and sends it to the subtitle process, which then sends the preprocessed first subtitle segment data to the subtitle display control module. The subtitle display control module then performs subsequent operations.

[0115] During the display of the first subtitle segment, the demultiplexing module sends the second subtitle segment data, filtered from the bitstream, to the data management module. The data management module then sends the second subtitle segment data to the subtitle syntax analysis program. The subtitle syntax analysis program parses the second subtitle segment data to obtain its relevant information. The subtitle syntax analysis program preprocesses the parsed second subtitle segment data and sends it to the subtitle process, which in turn sends the preprocessed second subtitle segment data to the subtitle display control module. The subtitle display control module then performs subsequent operations.

[0116] In some embodiments, such as Figure 13 As shown, after the subtitle interface receives the command to enable the TTML subtitle function, the demultiplexing module sends the first and second subtitle segment data filtered from the bitstream together to the data management module. The data management module then sends the first and second subtitle segment data to the subtitle syntax analysis program. The subtitle syntax analysis program parses the first and second subtitle segment data to obtain their relevant information. The subtitle syntax analysis program preprocesses the parsed first and second subtitle segment data and sends it to the subtitle process. The subtitle process then sends the preprocessed first and second subtitle segment data to the subtitle display control module. The subtitle display control module performs subsequent operations.

[0117] The controller includes a subtitle display control module, such as... Figure 14 As shown, the subtitle display control module performs the following steps:

[0118] Step S1401: Obtain the first content, first display characteristics, first display start time, and first display end time of the first subtitle segment;

[0119] The subtitle display control module can obtain the first content, first display characteristics, first display start time, and first display end time of the first subtitle segment from the pre-processed first subtitle segment data sent by the subtitle process. The first display characteristics include the display position, font type, font size, transparency, and subtitle background of the first content. The display position of the first content can be obtained from parsing the first subtitle segment data, while the font type, font size, transparency, and subtitle background can be obtained from the Live TV user settings file. If the display characteristics obtained from parsing the first subtitle segment data differ from the user-set display characteristics, one can be used. For example, if the font type obtained from parsing the first subtitle segment data differs from the user-set font type, either the font type obtained from parsing the first subtitle segment data or the user-set font type can be used.

[0120] Users can choose to open the display settings interface in the live streaming application settings interface, such as... Figure 11 As shown. Figure 11 The user interface also includes a caption display personalization control 112. The user selects the caption display personalization control 112 and presses the confirmation button on the control device 100 to display, for example... Figure 15 The user interface shown. Figure 15 The user interface includes a font type control 151, a font size control 152, a transparency control 153, and a subtitle background color control 154. Users can modify the display characteristics of the subtitles by selecting the corresponding controls.

[0121] Step S1402: When the first display startup time is reached, control the display to display the first content according to the first display characteristics;

[0122] The subtitle display control module sends the first content and first display characteristics of the first subtitle segment to the graphics hardware abstraction module. The graphics hardware abstraction module sends the first content and first display characteristics of the first subtitle segment to the drawing engine module, and the drawing engine module draws the first content based on the first display characteristics.

[0123] When the first display startup time is reached, the display 260 is controlled to display the first content drawn by the drawing engine module according to the first display characteristics at the position corresponding to the current playback interface.

[0124] It's important to note that the player periodically synchronizes its time with the subtitle display control module. Specifically, at preset intervals, the player sends its current time to the subtitle display control module. The subtitle display control module then calculates the difference between its recorded current time and the time sent by the player. If the difference is greater than a preset synchronization value, the module adjusts its recorded time accordingly to ensure synchronization between the player and the subtitle display control module, thus ensuring that the audio and video data played by the player corresponds to the subtitles. If the difference is less than or equal to the preset synchronization value, no time synchronization is performed.

[0125] Step S1403: Obtain the second content, second display characteristics, second display start time, and second display end time of the second subtitle segment;

[0126] The subtitle display control module can obtain the second content, second display characteristics, second display start time, and second display end time of the second subtitle segment from the pre-processed second subtitle segment data. The second display characteristics include the display position, font type, font size, transparency, and subtitle background of the second content.

[0127] Step S1404: Calculate the difference between the second display start time and the first display end time;

[0128] Step S1405: Determine whether the difference is less than a preset value;

[0129] The default value is the time required for the hardware to process the display of blank space and the display of the next subtitle segment.

[0130] If the difference is greater than or equal to a preset value, then step S1406 is executed: when the display end time of the first subtitle segment is reached, the first content is cleared;

[0131] The subtitle display control module sends the instruction to clear the first content to the graphics engine module through the graphics hardware abstraction module, and the graphics engine module clears the first content, that is, displays blank content.

[0132] Step S1407: Control the display to display the second content according to the second display characteristics.

[0133] The subtitle display control module sends the second content and second display characteristics of the second subtitle segment to the graphics hardware abstraction module. The graphics hardware abstraction module then sends the second content and second display characteristics of the second subtitle segment to the drawing engine module, which draws the content. When the second display start time is reached, the display 260 is controlled to display the second content drawn by the drawing engine module according to the second display characteristics at the position corresponding to the current playback interface.

[0134] In some embodiments, the third content, third display characteristics, third display start time, and third display end time of the third subtitle segment are obtained;

[0135] Calculate the difference between the third display start time and the second display end time, and determine the time to display the third subtitle segment content based on the difference.

[0136] In some embodiments, if the difference is less than a preset value, when the display end time of the first subtitle segment is reached, the display is controlled to display the second content according to the second display characteristics.

[0137] The subtitle display control module sends the second content and second display characteristics of the second subtitle segment to the graphics hardware abstraction module. The graphics hardware abstraction module then sends the second content and second display characteristics of the second subtitle segment to the drawing engine module, which draws the content. When the display end time of the first subtitle segment is reached, the display 260 is controlled to display the second content drawn by the drawing engine module according to the second display characteristics at the position corresponding to the current playback interface.

[0138] For example, such as Figure 16 As shown, if the first subtitle segment contains information about Beijing and the display time is 1000ms-3000ms, and the second subtitle segment contains information about Shanghai and the display time is 3100ms-5100ms, the difference is 3100-3000=100ms. Since 100ms > 40ms, the content of the second subtitle segment will be displayed at 3100ms. Figure 16 As shown in effect 2, the subtitle display effect is as follows: display "Beijing" → display blank → display "Shanghai".

[0139] If the first subtitle segment contains "Beijing" and is displayed for 1000ms-3000ms, and the second subtitle segment contains "Shanghai" and is displayed for 3001ms-5000ms, with a default value of 40ms, the difference is 3001 - 3000 = 1ms. Since 1ms < 40ms, the second subtitle segment will be displayed at 3000ms. Figure 16 As shown in Effect 1, the subtitle display effect is: displaying "Beijing" → displaying "Shanghai".

[0140] In the above embodiments, when the difference is less than a preset value, simply removing the display of blank content will cause some display features of TTML, such as panning display and sentence segmentation display, to be damaged.

[0141] In some embodiments, if the difference is less than a preset value, step S1408 is executed: determine whether the first content and the second content are the same;

[0142] In some embodiments, if the first content is the same as the second content, then when the first display end time is reached, the display 260 is controlled to display the second content drawn by the drawing engine module according to the second display characteristics at the position corresponding to the current playback interface.

[0143] If the first content is different from the second content, when the display end time of the first subtitle segment is reached, the subtitle display control module will send the instruction to clear the first content through the graphics hardware abstraction module to the graphics engine module, and the graphics engine module will clear the first content, that is, display blank content.

[0144] When the display start time of the second subtitle segment is reached, the display 260 is controlled to display the second content drawn by the drawing engine module according to the second display characteristics at the position corresponding to the current playback interface.

[0145] For example, such as Figure 17 As shown, the first subtitle segment contains the word "Beijing" and is displayed for a period of 1000ms-3000ms. The second subtitle segment contains the word "Beijing" and is displayed for a period of 3001ms-5000ms, with a default value of 40ms. The difference is 3001 - 3000 = 1ms. Since 1ms < 40ms and both the first and second subtitle segments contain the word "Beijing," the second subtitle segment is displayed at 3000ms. Figure 17 As shown in Effect 1, the subtitle display effect is: displaying "Beijing" → displaying "Beijing".

[0146] The first subtitle segment displays "Beijing" for a period of 1000ms-3000ms, and the second subtitle segment displays "Shanghai" for a period of 3001ms-5000ms, with a default value of 40ms. The difference is 3001 - 3000 = 1ms. Since 1ms < 40ms, but the content of the first and second subtitle segments is different, a blank space is displayed at 3000ms, and the content of the second subtitle segment is displayed at 3040ms. Figure 17 As shown in effect 2, the subtitle display effect is as follows: display "Beijing" → display blank → display "Shanghai".

[0147] This application embodiment uses the time when hardware performance is limited as the dividing point. When it is determined that the difference between adjacent subtitle segments is less than a preset value, it also adds a judgment on whether the displayed content is the same. Only when the content of adjacent subtitle segments is the same, there is no need to insert extra blank display. When the content of adjacent subtitle segments is different, the effect of displaying blank is still retained to ensure that the TTML sentence segmentation display and other effects are not destroyed.

[0148] In some embodiments, if the first content is the same as the second content, then step S1409 is executed: determine whether the first display characteristic is the same as the second display characteristic;

[0149] Display characteristics include display position, font type, font size, transparency, and subtitle background. The first and second display characteristics being identical can mean that all information such as display position, font type, font size, transparency, and subtitle background is the same. Alternatively, the first and second display characteristics can be identical if at least one of these information is the same. For example, if the display position is the same in both the first and second display characteristics, it can be tentatively determined that the first and second display characteristics are identical.

[0150] If the first display characteristic is the same as the second display characteristic, then step S1410 is executed: when the first display end time is reached, the display is controlled to display the second content according to the second display characteristic.

[0151] If the first display characteristic is different from the second display characteristic, then step S1406 is executed, that is, when the display end time of the first subtitle segment is reached, the subtitle display control module sends the instruction to clear the first content to the drawing engine module through the graphics hardware abstraction module, and the drawing engine module clears the first content, that is, displays blank content.

[0152] When the display start time of the second subtitle segment is reached, the display 260 is controlled to display the second content drawn by the drawing engine module according to the second display characteristics at the position corresponding to the current playback interface.

[0153] For example, such as Figure 18 As shown, the first subtitle segment contains the word "Beijing" and is displayed for a period of 1000ms-3000ms. The second subtitle segment contains the word "Beijing" and is displayed for a period of 3001ms-5000ms, with a default value of 40ms. The difference is 3001 - 3000 = 1ms. Since 1ms < 40ms and the first and second subtitle segments are displayed in the same position, the content of the second subtitle segment is displayed at 3000ms. Figure 18 As shown in Effect 1, the subtitle display effect is: displaying "Beijing" → displaying "Beijing".

[0154] The first subtitle segment displays "Beijing" for a period of 1000ms-3000ms. The second subtitle segment displays "Beijing" for a period of 3001ms-5000ms, with a default value of 40ms. The difference is 3001 - 3000 = 1ms. Since 1ms < 40ms, but the first and second subtitle segments are displayed at different positions, a blank space is displayed at 3000ms, and the content of the second subtitle segment is displayed at 3040ms. Figure 18 As shown in effect 2, the subtitle display effect is as follows: display "Beijing" → display blank → display "Beijing".

[0155] In some embodiments of this application, when adjacent subtitle segments have the same content, a judgment is added on whether the display characteristics are the same. Only when the display characteristics of adjacent subtitle segments are the same, there is no need to insert extra blank display. When the display characteristics of adjacent subtitle segments are different, the effect of display blank is still retained to ensure that the effects such as TTML panning display are not destroyed.

[0156] In some embodiments, if the first content is different from the second content, then step S1411 is executed: determine whether the first content and the second content are related;

[0157] In some embodiments, the step of determining whether the first content and the second content are related includes:

[0158] Determine whether the first content ends with a sentence-final punctuation mark;

[0159] If the first content ends with a sentence-final punctuation mark, then the first content and the second content are determined to be unrelated.

[0160] If the first content does not end with a sentence punctuation mark, then the first content and the second content are determined to be related.

[0161] Punctuation marks at the end of a sentence include periods, question marks, exclamation marks, and ellipses. Punctuation marks within a sentence include commas, pauses, dashes, semicolons, colons, double quotation marks, and single quotation marks.

[0162] For example, if the first content obtained is "The weather is so nice today!", "Where are you planning to go?", "I've finished my tasks for today," or "There are pandas, tigers, lions...", and the first content ends with ".", "?", "!", or "...", then it is determined that the first content and the second content are not related. If the first content obtained is "I just saw a child," "What you're doing is like throwing an egg against a rock—", "Xiao Hong said:", "Today we'll mainly talk about the following three points:", "The first point is to pay attention to hygiene;", or "The key to creating urban civilization," and the first content ends with ",", "—", ":", ";", or other text, then it is determined that the first content and the second content are related.

[0163] In some embodiments, if the first content and the second content are associated, that is, the association flag of the first content and the second content is set to 1; if the first content and the second content are not associated, the association flag of the first content is set to 0.

[0164] The step of determining whether the first content and the second content are related includes:

[0165] Determine whether the association flag of the first subtitle segment is 1.

[0166] If the association flag of the first subtitle segment is 1, then it is determined that the first content and the second content are associated.

[0167] If the association flag of the first subtitle segment is not 1, then it is determined that the first content and the second content are not associated.

[0168] If the first content is related to the second content, then step S1410 is executed, that is, when the display end time of the first subtitle segment is reached, the display 260 is controlled to display the second content drawn by the drawing engine module according to the second display characteristics at the position corresponding to the current playback interface.

[0169] If the first content and the second content are not related, then step S1406 is executed, that is, when the display end time of the first subtitle segment is reached, the subtitle display control module sends the instruction to clear the first content to the drawing engine module through the graphics hardware abstraction module, and the drawing engine module clears the first content, that is, displays blank content.

[0170] When the display start time of the second subtitle segment is reached, the display 260 is controlled to display the second content drawn by the drawing engine module according to the second display characteristics at the position corresponding to the current playback interface.

[0171] For example, such as Figure 19 As shown, the first subtitle segment reads "I just saw a child," with a display time period of 1000ms-3000ms. The second subtitle segment reads "She was walking a cute little dog.", with a display time period of 3001ms-5000ms. The default value is 40ms. The difference is 3001-3000=1ms. Since 1ms < 40ms and the content of the first and second subtitle segments is related, the content of the second subtitle segment is displayed at 3000ms. Figure 19 As shown in Effect 1, the subtitle display effect is as follows: "I just saw a child," → "She was walking a cute little dog."

[0172] The first subtitle segment displays "Today's weather is sunny.", with a display time period of 1000ms-3000ms. The second subtitle segment displays "Tomorrow's weather is sunny.", with a display time period of 3001ms-5000ms. The default value is 40ms. The difference is 3001-3000=1ms. Since 1ms < 40ms, but the content of the first and second subtitle segments is not related, a blank space is displayed at 3000ms, and the content of the second subtitle segment is displayed at 3040ms. Figure 19 As shown in effect 2, the subtitle display effect is as follows: display "Today's weather is sunny." → display blank → display "Tomorrow's weather is sunny."

[0173] In some embodiments of this application, when the content of adjacent segments is different, a judgment on the correlation between the content of adjacent segments is added. Only when the content of adjacent subtitle segments is related, there is no need to insert extra blank display. When the content of adjacent subtitle segments is not related, the effect of displaying blank is still retained, so as to ensure that the TTML sentence segmentation and other effects are not destroyed.

[0174] In some embodiments, "when the first display end time is reached, control the display to display the second content according to the second display characteristics" can be replaced with "when the second display start time is reached, control the display to display the second content according to the second display characteristics".

[0175] For example, the first subtitle segment contains "Beijing" and is displayed for a period of 1000ms-3000ms. The second subtitle segment contains "Shanghai" and is displayed for a period of 3001ms-5000ms, with a default value of 40ms. The difference is 3001 - 3000 = 1ms. Since 1ms < 40ms, the content of the second subtitle segment is displayed at 3001ms. Figure 20 As shown, the subtitle display effect is: displaying "Beijing" → displaying "Shanghai".

[0176] Some embodiments of this application provide a subtitle display method applicable to a display device, which includes a display and a controller. The controller is configured to: acquire first content, first display characteristics, and display end time of a first subtitle segment; control the display to display the first content according to the first display characteristics; acquire second content, second display characteristics, and display start time of a second subtitle segment; calculate the difference between the display start time and the display end time; if the difference is less than a preset value, control the display to display the second content according to the second display characteristics when the display end time is reached. When the display time interval between two adjacent segments is short, blank content can be omitted, and the content of the next segment can be displayed directly, avoiding subtitle flickering and uneven content smoothness, thus improving the user experience.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0178] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that, include: monitor; The controller is configured as follows: Obtain the first content, first display characteristics, first display start time, and first display end time of the first subtitle segment; When the first display startup time is reached, the display is controlled to display the first content according to the first display characteristics; Obtain the second content, second display characteristics, and second display start time of the second subtitle segment; Calculate the difference between the second display start time and the first display end time; If the difference is less than a preset value, when the first display end time is reached, it is determined whether the first content and the second content are the same; If the first content is the same as the second content, then when the first display end time is reached, the display is controlled to display the second content according to the second display characteristics; If the first content is different from the second content, then determine whether the first content and the second content are related. If the first content is related to the second content, when the first display end time is reached, the display is controlled to display the second content according to the second display characteristics; If the first content and the second content are not related, then when the first display end time is reached, the first content is cleared, and the display is controlled to display the second content according to the second display characteristics.

2. The display device according to claim 1, characterized in that, The controller is configured as follows: If the first content is different from the second content, then when the first display end time is reached, the first content is cleared, and the display is controlled to display the second content according to the second display characteristics.

3. The display device according to claim 1, characterized in that, The controller performs a judgment on whether the first content and the second content are related, and is further configured to: Determine whether the first content ends with a sentence-final punctuation mark; If the first content ends with a sentence-final punctuation mark, then the first content and the second content are determined to be unrelated. If the first content does not end with a sentence-final punctuation mark, then the first content and the second content are determined to be related.

4. The display device according to claim 1, characterized in that, The controller, upon reaching the first display end time, controls the display to show second content according to the second display characteristics, and is further configured to: Determine whether the first content and the second content are the same; If the first content is the same as the second content, then determine whether the first display characteristic is the same as the second display characteristic; If the first display characteristic is the same as the second display characteristic, when the first display end time is reached, control the display to display the second content according to the second display characteristic; If the first display characteristic is different from the second display characteristic, then when the first display end time is reached, the first content is cleared, and the display is controlled to display the second content according to the second display characteristic.

5. The display device according to claim 1, characterized in that, The controller is configured as follows: If the difference is greater than or equal to a preset value, the first content is cleared when the first display end time is reached; When the second display startup time is reached, the display is controlled to display the second content according to the second display characteristics.

6. A method for displaying subtitles, characterized in that, include: Obtain the first content, first display characteristics, first display start time, and first display end time of the first subtitle segment; When the first display startup time is reached, the display is controlled to display the first content according to the first display characteristics; Obtain the second content, second display characteristics, and second display start time of the second subtitle segment; Calculate the difference between the second display start time and the first display end time; If the difference is less than a preset value, at the first display end time... Determine whether the first content and the second content are the same; If the first content is the same as the second content, then when the first display end time is reached, the display is controlled to display the second content according to the second display characteristics; If the first content is different from the second content, then determine whether the first content and the second content are related. If the first content is related to the second content, when the first display end time is reached, the display is controlled to display the second content according to the second display characteristics; If the first content and the second content are not related, then when the first display end time is reached, the first content is cleared, and the display is controlled to display the second content according to the second display characteristics.

7. The method according to claim 6, characterized in that, The method further includes: If the first content is different from the second content, then when the first display end time is reached, the first content is cleared, and the display is controlled to display the second content according to the second display characteristics.

Images