Subtitle display method and device and storage medium

Abstract

The invention discloses a subtitle display method and device and a storage medium, and the method comprises the steps: pre-rendering a first state image and a second state image of a to-be-displayed subtitle, enabling the first state image to be stacked above the second state image, enabling the first state image to be in a subtitle non-playing state, and enabling the second state image to be in a subtitle playing state; the second state image is in a subtitle playing state; presetting a covering object, wherein the covering object can cover the upper-layer image so as to display the lower-layer image; and obtaining a display progress, and moving the covering along the subtitle display direction according to the display progress, so that the first state image is covered along with the display progress and the second state image is displayed along with the display progress. According to the method and the device, the amount of rendered data during subtitle display can be reduced, and the real-time requirement on rendering can be reduced, so that subtitle display blockage can be effectively avoided, and the subtitle display fluency is improved.

Description

Technical Field

[0001] The present application relates to the field of multimedia technologies, and particularly relates to a subtitle display method, apparatus, and storage medium. Background Art

[0002] When displaying high-definition data, real-time rendering of images is required. Since the amount of data processed during image rendering is huge, real-time rendering is a key factor affecting the display effect. The real-time rendering method is likely to cause the display interface to freeze, affecting the user experience. For example, when displaying lyrics on a large-sized multi-screen, the displayed lyrics are high-definition lyrics, and the lyrics need to be rendered synchronously with the audio playback progress, that is, the lyrics corresponding to the current playback position of the audio need to be rendered in real time each time it is played. Therefore, high requirements are imposed on both the software and hardware of the device. When the required rendering speed of the lyrics is high or the device performance is low, it will cause the lyrics interface to freeze, and even the playback may be unsmooth.

[0003] Therefore, how to effectively avoid subtitle freezes such as lyrics has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0004] In view of the above problems, the present application provides a subtitle display method for solving the technical problem that real-time rendering of subtitles is prone to freezing.

[0005] To achieve the above object, the inventor provides a subtitle display method, including the following steps:

[0006] Pre-render a first state image and a second state image of the subtitle to be displayed, and stack the first state image above the second state image; the first state image is the state where the subtitle has not been played, and the second state image is the state where the subtitle has been played;

[0007] Preset an occluder, and the occluder can cover the upper-layer image to display the lower-layer image;

[0008] Obtain the display progress, and move the occluder along the subtitle display direction according to the display progress, so that the first state image is covered as the display progress advances and the second state image is displayed as the display progress advances.

[0009] Further, the subtitle is the lyrics of a song, and the display progress is synchronized with the song playback progress.

[0010] Further, before the first state image and the second state image of the subtitle to be displayed are pre-rendered, the following step is further included:

[0011] Obtain the lyrics to be displayed, and divide the lyrics into multiple lyric segments according to the display sequence, and each lyric segment is a subtitle to be displayed;

[0012] When displaying the current lyric segment, the next described lyric segment is pre-rendered.

[0013] Furthermore, the subtitles are displayed on a multi-screen formed by splicing two or more display screens.

[0014] Furthermore, it also includes selecting an editable area through a transparent canvas in the video, and the first state image and the second state image are displayed within the editable area.

[0015] Furthermore, the text in the first state image and the second state image is different in any one or a combination of two or more of font, color, size, and brightness.

[0016] Furthermore, it also includes the steps of presetting a first buffer, a second buffer, and a third buffer. The first buffer is used to cache the rendering data of the first state image and the second state image; the second buffer is used to cache the image data generated during the movement of the cover; the third buffer is used to cache the image data sent from the second buffer and display the image data.

[0017] To solve the above technical problems, the present invention also provides another technical solution:

[0018] A subtitle display device, comprising:

[0019] A pre-rendering module, used to pre-render a first state image and a second state image of the subtitle to be displayed, and stack the first state image above the second state image; the first state image is the state where the subtitle has not been played, and the second state image is the state where the subtitle has been played;

[0020] A covering module, used to preset a cover, and the cover can cover the upper-layer image to display the lower-layer image;

[0021] A display module, used to obtain the display progress, move the cover along the subtitle display direction according to the display progress, so that the first state image is covered as the display progress, and the second state image is displayed as the display progress.

[0022] Furthermore, the subtitles are displayed on a multi-screen formed by splicing two or more display screens; the subtitles are the lyrics of the song, and the display progress is synchronized with the song playing progress.

[0023] To solve the above technical problems, the present invention also provides another technical solution:

[0024] A computer-readable storage medium stores a computer program thereon, and when the program is executed by a processor, the subtitle display method described in any one of the above technical solutions is implemented.

[0025] Different from the prior art, the subtitle display method of the above technical solution pre-renders the first state image and the second state image of the subtitle to be displayed, and preset a covering object, which can cover the upper image to display the lower image. By moving the covering object along with the display progress to cover the first state image, the second state image of the subtitle can be displayed, so as to achieve the real-time rendering effect of the subtitle. In this technical solution, only the first state image and the second state image need to be rendered, and the amount of data to be rendered is small. And since the first state image and the second state image are pre-rendered, subtitle display jams can be effectively avoided, and the fluency of subtitle display can be improved.

[0026] The above relevant description of the invention content is only an overview of the technical solution of this application. In order to enable those of ordinary skill in the art to more clearly understand the technical solution of this application, and then can be implemented according to the content recorded in the text of the specification and the drawings, and in order to make the above objects, other objects, features and advantages of this application more easily understood, the following will be described in conjunction with the specific implementation manners and drawings of this application. Brief Description of the Drawings

[0027] The drawings are only used to illustrate the principles, implementation manners, applications, features, effects, etc. of the specific implementation manners of the present invention and other related contents, and should not be considered as a limitation to this application.

[0028] In the drawings of the specification:

[0029] Figure 1 is a flowchart of the subtitle display method described in the specific implementation manner;

[0030] Figure 2 is a schematic diagram of the covering object covering the first state image described in the specific implementation manner;

[0031] Figure 3 is a specific flowchart of the subtitle display device described in the specific implementation manner;

[0032] Figure 4 is a schematic diagram of the subtitle display device described in the specific implementation manner;

[0033] Figure 5 is a schematic diagram of the computer-readable storage medium described in the specific implementation manner;

[0034] The reference numerals involved in the above drawings are explained as follows:

[0035] 1. First state image; 2. Second state image; 3. Covering object; Specific Embodiments

[0036] To describe in detail the possible application scenarios, technical principles, specific implementable solutions, achievable objectives and effects of this application, etc., the following will be described in detail with reference to the specific examples listed and in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of this application, so they are only examples and cannot be used to limit the protection scope of this application.

[0037] Referring to "embodiments" herein means that the specific features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of this application. The term "embodiment" that appears in various positions in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or relevance to other embodiments. In principle, in this application, as long as there is no technical contradiction or conflict, the various technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0038] Unless otherwise defined, the meanings of the technical terms used herein are the same as those commonly understood by those skilled in the technical field to which this application belongs; the use of the relevant terms herein is only for describing specific embodiments and is not intended to limit this application.

[0039] In the description of this application, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: there is A, there is B, and there is both A and B at the same time. In addition, the character " / " in this article generally represents an "or" logical relationship between the associated objects before and after.

[0040] In this application, terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, primary or secondary, or order relationship between these entities or operations.

[0041] Without more limitations, in this application, the open expressions such as "including", "comprising", "having" or other similar expressions used in the statements are intended to cover non-exclusive inclusion. These expressions do not exclude that there may be other elements in the process, method or product including the said elements, so that the process, method or product including a series of elements may not only include those defined elements, but also include other elements not explicitly listed, or also include elements inherent to this process, method or product.

[0042] Similar to the understanding in the "Examination Guidelines", in this application, expressions such as "greater than", "less than", "exceeding" are understood to exclude the present number; expressions such as "above", "below", "within" are understood to include the present number. In addition, in the description of the embodiments of this application, the meaning of "multiple" is two or more (including two), and similar expressions related to "many" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically defined.

[0043] In the description of the embodiments of this application, the spatially related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "perpendicular", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the specific embodiment or the accompanying drawings, and is only for the convenience of describing the specific embodiments of this application or for the reader to understand, rather than indicating or implying that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, so it cannot be understood as a limitation to the embodiments of this application.

[0044] Unless otherwise clearly specified or limited, in the description of the embodiments of this application, the terms "installed", "connected", "joined", "fixed", "set", etc. should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integral setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be directly connected, or indirectly connected through an intermediate medium; it can be the communication inside two components or the interaction relationship between two components. For those skilled in the art to which this application pertains, the specific meanings of the above terms in the embodiments of this application can be understood according to specific circumstances.

[0045] This embodiment provides a subtitle display method. This subtitle display method can be applied to various types of display devices to display subtitles, especially suitable for large-size display screens with high requirements for subtitle display clarity, such as applied to large-size multi-screen displays formed by splicing multiple display screens. Since the data processing volume during high-definition subtitle rendering is large and the real-time requirement is high, the software and hardware requirements for the device are high. In the prior art, phenomena such as stuttering are likely to occur during the display of this type of subtitle. In this embodiment, by pre-rendering the state images before and after subtitle playback, and then simulating the real-time rendering effect of subtitles by moving the cover, the data volume involved in image rendering is reduced, and the real-time requirement for rendering is lowered, thereby effectively avoiding subtitle stuttering and improving subtitle playback fluency.

[0046] Please refer to Figure 1 、 OneA subtitle display method, comprising the following steps:

[0047] S101. Pre-render a first state image and a second state image of the subtitle to be displayed, and stack the first state image above the second state image; the first state image is the state where the subtitle has not been played, and the second state image is the state where the subtitle has been played;

[0048] S102. Preset a cover, and the cover can cover the upper-layer image to display the lower-layer image;

[0049] S103. Obtain the display progress, and move the cover along the direction of subtitle display according to the display progress, so that the first state image is covered as the display progress progresses and the second state image is displayed as the display progress progresses.

[0050] In step S101, the image in the state where the subtitle to be displayed has been played (i.e., the second state image) and the image in the state where it has not been played (the first state image) are rendered through image rendering technology. And when the first state image is stacked above the second state image, the second state image is blocked by the first state image, so only the image in the first state is displayed, that is, only the image in the state where the subtitle has not been played is displayed. Among them, in order to highlight the current playing progress of the subtitle and facilitate distinguishing the played part and the unplayed part in the subtitle, the text in the first state image and the second state image is different in any one or a combination of two or more of font, color, size, and brightness. For example, the text in the first state image is rendered with high brightness and a specific color, while the text in the second state image is rendered in a gray style.

[0051] In step S102, the cover is not a specific style of layer or image. It only functions to cover the first state image, making the covered part transparent, so that the image at the corresponding position of the second state image below can be displayed.

[0052] As Figure 2 shown, in step S103, the first state image 1 is stacked above the second state image 2. When the cover 3 moves along the direction of dynamic subtitle playback (i.e., Figure 2In the direction indicated by the arrow, the cover can completely block the played part of the first-state image. The first-state image 1 is gradually blocked. At this time, the image of the blocked part of the first-state image 1 becomes transparent and is not displayed, and the image of the second-state image 2 can be displayed through the first-state image. That is, the second-state image 2 directly below the cover 3 is spliced with the unblocked image in the first-state image 1 to form a display effect. For example, when the right edge of the cover 3 moves to the middle of the first-state image 1, the left part of the subtitle is in the played state and the right part of the subtitle is in the unplayed state in the display effect. And with the display progress, the proportion of the played state on the left side of the subtitle will be more and more, while the proportion of the unplayed state on the right side of the subtitle will be less and less.

[0053] In this embodiment, it is not necessary to render the subtitle in real time according to the display progress. Instead, the state before and after the subtitle is played is pre-rendered, and then the cover is moved to gradually transition the subtitle from the unplayed state to the played state, so that the real-time rendering effect can be simulated in the display effect. In this embodiment, only the first-state image and the second-state image of each subtitle need to be rendered, and the amount of rendered data is small. And since the first-state image and the second-state image are pre-rendered, subtitle display stuttering can be effectively avoided and the subtitle display fluency can be improved.

[0054] In one embodiment, the subtitle is the lyrics of a song, the display progress is the playing progress of the song, and the display progress is synchronized with the song playing progress. In other embodiments, the subtitle is not limited to the lyrics, and the subtitle can also be the lines of the dialogue in the video, and the display progress can be the playing progress of the video, etc.

[0055] In some embodiments, before pre-rendering the first-state image and the second-state image of the subtitle to be displayed, the following steps are further included:

[0056] Obtain the lyrics to be displayed, and divide the lyrics into multiple lyric segments according to the display order. Each lyric segment is a subtitle to be displayed;

[0057] When the current lyric segment is displayed, pre-render the next lyric segment.

[0058] Since the duration of a song is usually long, when playing a song, the lyrics are displayed synchronously for each sentence. Therefore, when cutting the lyrics, the lyrics can be split into multiple lyric segments according to the playback time points of the audio corresponding to each sentence of the lyrics. Each lyric segment is each display subtitle in the above embodiments. Therefore, according to the song playback progress, each lyric segment is rendered and displayed one by one. Among them, in this embodiment, when displaying the current song segment, the next lyric segment to be displayed is pre-rendered, so that the corresponding first state image and second state image can be rendered before the next song segment is displayed. When the next song segment is displayed, only the moving speed of the cover needs to be controlled, thus greatly reducing the real-time requirement for lyric rendering.

[0059] In some embodiments, the subtitle display method further includes selecting an editable area through a transparent canvas in the video, and the first state image and the second state image are displayed within the editable area.

[0060] In this embodiment, through the transparent canvas, an editable area that can be freely edited can be directly selected in the MV of the song, and then the subtitle is displayed within the editable area, and real-time rendering is achieved by moving the cover, so that the subtitle is superimposed and displayed on the MV. In the prior art, the lyrics of the song and the MV are rendered simultaneously, that is, when making the MV, according to the song playback, the lyrics are superimposed on the MV. This way of lyric processing has poor flexibility and a large amount of modification work. In this embodiment, the lyrics can be made after the MV is made through the transparent canvas, greatly improving the flexibility of lyric processing.

[0061] In some embodiments, the subtitle display method further includes the steps of: presetting a first buffer, a second buffer, and a third buffer. The first buffer is used to cache the rendering data of the first state image and the second state image; the second buffer is used to cache the image data generated during the movement of the cover; the third buffer is used to cache the image data sent from the second buffer and display the image data.

[0062] Among them, the first buffer, the second buffer, and the third buffer are different storage areas in the video memory and can be used for high-speed access of image data. In this embodiment, through the first buffer, the second buffer, and the third buffer, multi-level buffering can be achieved, so that the processes of rendering the first state image and the second state image, moving the cover, and displaying the subtitle can all be performed independently without waiting for whether other execution processes have been completed. Therefore, the real rendering time can be further reduced, the rendering efficiency can be improved, and the problem of playback jamming can be avoided, providing a better user experience.

[0063] Such as Figure 3As shown, it is a specific flowchart for lyric playback in a multi-screen. It includes the following steps:

[0064] S301. Obtain the lyric text to be displayed; after obtaining the lyric text, sort it in chronological order.

[0065] S302. Split the lyrics; according to the time points of each lyric sentence, split the lyric text into multiple lyric segments.

[0066] S303. Lyric rendering; according to the display, obtain the lyric text segment that is about to be sung, and render this lyric segment into a first state image (i.e., an unplayed state image) and a second state image (i.e., a played state image). The first state image is rendered using gray font, and the second state image is rendered using highlighting and specific colors.

[0067] S304. Set up a transparent canvas; select a subtitle display area at the middle and lower position of the song's MV through the transparent canvas;

[0068] S305. Move the cover; during the song playback, the above-mentioned rendering thread reads the song progress status: first, place the first state image (unplayed state picture) on the upper layer, the second state image (played state picture) on the middle layer, and the transparent canvas on the bottom layer, and stack these three layers on the MV interface for display. After the song playback starts, first display the first state image on the upper layer. According to the current playback progress, move the cover to cover the sung lyric area in the first state image on the upper layer, and display the picture area of the sung lyrics in the second state image on the lower layer. Therefore, the un-sung song area in the first state image and the sung song area in the second state image on the middle layer are displayed on the display interface.

[0069] As Figure 2 shown, as the singing progress continues, continuously move the cover until the song in the unread state picture on the upper layer is completely covered.

[0070] The specific covering method can be: use a moving cover to convert the sung song area in the first state image on the upper layer into the sung song area in the second state image on the middle layer. For example, the lyric color in the first state image on the upper layer is white. By moving the cover to cover the alpha channel of the sung song area in the first state image on the upper layer, the picture area of the sung song in the second state image on the middle layer is displayed. Specifically, the function of the cover can be understood as a mask in PS. Through the mask, the display effect of the area can be switched. The area that has passed through becomes a played state picture, and the area that has not passed through is an unread state picture.

[0071] S306. Determine whether the subtitle has reached the end (i.e., whether the subtitle has ended). If it has not reached the end, jump to step S303 to loop the above steps. If it has reached the end, it means that the lyrics of the current song have been played, and jump to step S307 to end.

[0072] As Figure 4 shown, in another embodiment, a subtitle display device 400 is provided. The subtitle display device 400 includes: a pre-rendering module 401, a covering module 402, and a display module 403. Among them, the pre-rendering module 401 is used to pre-render a first state image and a second state image of the subtitle to be displayed, and stack the first state image above the second state image; the first state image is the state where the subtitle has not been played, and the second state image is the state where the subtitle has been played.

[0073] The covering module 402 is used to preset a covering object, and the covering object can cover the upper-layer image to display the lower-layer image.

[0074] The display module 402 is used to obtain the display progress, and move the covering object along the direction of subtitle display according to the display progress, so that the first state image is covered as the display progress progresses and the second state image is displayed as the display progress progresses.

[0075] Among them, when the first state image is stacked above the second state image, the second state image is blocked by the first state image. Therefore, only the image of the first state is displayed, that is, only the image when the subtitle has not been played is displayed. The text in the first state image and the second state image is different in any one or a combination of two or more of font, color, size, and brightness. For example, the text in the first state image is rendered with high brightness and a specific color, while the text in the second state image is rendered in a gray style.

[0076] In the covering module 402, the covering object is not a specific style of layer or image. It only functions to cover the first state image, making the covered part transparent, so that the image at the corresponding position of the second state image below can be displayed.

[0077] As Figure 2 shown, in step S103, the first state image 1 is stacked above the second state image 2. When the covering object 3 moves along the direction of dynamic subtitle playback (i.e., Figure 2In the direction indicated by the arrow, the covering can completely block the played part of the first-state image). The first-state image 1 is gradually blocked. At this time, the image of the blocked part of the first-state image 1 disappears and becomes transparent, and the image of the second-state image 2 can be displayed through the first-state image. That is, the second-state image 2 directly below the covering 3 is spliced with the unblocked image in the first-state image 1 to form a display effect. For example, when the right edge of the covering 3 moves to the middle of the first-state image 1, the display effect is that the left part of the subtitle is in the played state and the right part of the subtitle is in the unplayed state. And as the display progresses, the proportion of the played state on the left side of the subtitle will be more and more, while the proportion of the unplayed state on the right side of the subtitle will be less and less.

[0078] In this embodiment, it is not necessary to render the subtitle in real time according to the display progress. Instead, the state before and after the subtitle is played is pre-rendered, and then the covering is moved to gradually transition the subtitle from the unplayed state to the played state, so that the display effect can simulate the effect of real-time rendering. In this embodiment, only the first-state image and the second-state image need to be rendered for each subtitle, and the amount of rendered data is small. And since the first-state image and the second-state image are pre-rendered, it is possible to effectively avoid subtitle display stuttering and improve subtitle display fluency.

[0079] In one embodiment, the subtitle is displayed on a multi-screen formed by splicing two or more display screens; the subtitle is the lyrics of a song, and the display progress is the playing progress of the song, and the display progress is synchronized with the song playing progress. In other embodiments, the subtitle is not limited to lyrics, and the subtitle can also be the lines of dialogue in a video, and the display progress can be the playing progress of the video, etc.

[0080] As Figure 5 shown, in one embodiment, a computer-readable storage medium 500 is provided. A computer program is stored on the computer-readable storage medium 500, and when the program is executed by a processor, it implements the subtitle display method described in any of the above embodiments.

[0081] Finally, it should be noted that although the above embodiments have been described in the text and drawings of the specification of the present application, the patent protection scope of the present application cannot be limited thereby. Any technical solutions obtained by equivalent structure or equivalent process substitution or modification using the content recorded in the text and drawings of the specification of the present application based on the essential concept of the present application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the patent protection scope of the present application.

Claims

1. A subtitle display method, characterized in that, Including the following steps: Pre-render a first state image and a second state image of the subtitle to be displayed, stack the first state image above the second state image, where the first state image is the state where the subtitle has not been played, and the second state image is the state where the subtitle has been played; Preset a cover, which can cover the upper-layer image to display the lower-layer image; Obtain the display progress, and move the cover along the direction of subtitle display according to the display progress, so that the first state image is covered with the display progress and the second state image is displayed with the display progress.

2. The subtitle display method according to claim 1, characterized in that, The subtitle is the lyrics of a song, and the display progress is synchronized with the song playing progress.

3. The subtitle display method according to claim 2, characterized in that, Before pre-rendering the first state image and the second state image of the subtitle to be displayed, it further includes the steps of: Obtain the lyrics to be displayed, and divide the lyrics into multiple lyric segments according to the display order, and each lyric segment is a subtitle to be displayed; When displaying the current lyric segment, pre-render the next lyric segment.

4. The subtitle display method according to claim 1, characterized in that, The subtitle is displayed on a multi-screen, and the multi-screen is formed by splicing two or more display screens.

5. The subtitle display method according to claim 1, characterized in that, It further includes selecting an editable area through a transparent canvas in the video, and the first state image and the second state image are displayed in the editable area.

6. The subtitle display method according to claim 1, characterized in that, The text in the first state image and the second state image is different in any one or a combination of two or more of font, color, size, and brightness.

7. The subtitle display method according to claim 1, characterized in that, It further includes the steps of: presetting a first buffer, a second buffer, and a third buffer, where the first buffer is used to cache the rendering data of the first state image and the second state image; the second buffer is used to cache the image data generated during the movement of the cover; the third buffer is used to cache the image data sent from the second buffer and display the image data.

8. A subtitle display device, characterized in that, Including: A pre-rendering module, which is used to pre-render a first state image and a second state image of the subtitle to be displayed, and stack the first state image above the second state image; the first state image is the state where the subtitle has not been played, and the second state image is the state where the subtitle has been played; A covering module, which is used to preset a cover, and the cover can cover the upper-layer image to display the lower-layer image; A display module, which is used to obtain the display progress, and move the cover along the direction of subtitle display according to the display progress, so that the first state image is covered with the display progress and the second state image is displayed with the display progress.

9. The subtitle display device according to claim 8, characterized in that, The subtitle is displayed on a multi-screen, and the multi-screen is formed by splicing two or more display screens; the subtitle is the lyrics of a song, and the display progress is synchronized with the song playing progress.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that, When the program is executed by a processor, it implements the subtitle display method according to any one of claims 1-7.

Images