Subtitle display method and device, equipment and storage medium

Abstract

The invention relates to a subtitle display method and device, equipment and a storage medium. In the scheme, firstly, to-be-displayed target subtitle picture data at the current moment is obtained; the target subtitle picture data is picture data converted through a subtitle text to be displayed; then covering the target caption picture data to a corresponding display position of the pixel buffer area, converting the covered pixel buffer area into a sample buffer area, and displaying the sample buffer area on a caption interface view through a sample buffer display layer; visibly, when the external subtitles are displayed, the subtitle text is converted into the picture data and covers the pixel buffer area, then the picture data is converted into the sample buffer display layer, the external subtitles can be displayed through the sample buffer display layer, and due to the fact that the sample buffer display layer is built in the system, the development cost and the development difficulty can be reduced; moreover, the mode that the subtitles are displayed through the sample buffer display layer is adopted, and scene limitation existing when the plug-in subtitles are displayed can be avoided.

Description

Technical Field

[0001] The present application relates to the technical field of subtitle display, and in particular to a subtitle display method, device, equipment and storage medium. Background Art

[0002] Subtitles refer to non-visual content such as dialogues in television, movies, and stage works displayed in text form. They are very important information in audio and video playback and play a vital role in fully and accurately understanding the audio and video content. Subtitles generally exist in two forms, namely: hard-pressed subtitles and external subtitles. Compared with hard-pressed subtitles, external subtitles have the advantages of flexibility, easy replacement, and easy multi-language expansion. Therefore, external subtitles are currently widely used.

[0003] At present, when displaying external subtitles on the iOS system (mobile operating system), the text content of the external subtitles can be directly displayed through the iOS system standard UI (UserInterface, user interface) control UITextView (interface text box control) / UITextField (interface text box control). However, since this method uses the iOS system standard UI controls and is tightly coupled with the UI layer, the processing of external subtitles is generally located in the underlying core layer of the player, which will lead to insufficient flexibility in the design of the player software architecture, and this method will be extremely complicated when processing certain specific subtitle effects. In addition, in the related technology, the application layer can also use a graphics rendering engine to display the converted external subtitle image data, but the technical threshold of this method is relatively high, the development difficulty is relatively large, and this method still has the situation that it cannot work in some scenarios.

[0004] Therefore, how to reduce the development difficulty of external subtitle display and avoid the scene restrictions when displaying external subtitles is a problem that technical personnel in this field need to solve. Summary of the invention

[0005] The present application provides a subtitle display method, device, equipment and storage medium to reduce the development difficulty of external subtitle display and avoid scene restrictions when displaying external subtitles.

[0006] In a first aspect, the present application provides a subtitle display method, comprising:

[0007] Acquire target subtitle image data to be displayed at the current moment; wherein the target subtitle image data is image data converted from the subtitle text to be displayed;

[0008] Determining a display position of the target subtitle picture data;

[0009] Overwriting the target subtitle picture data to the display position of the pixel buffer, and converting the overwritten pixel buffer into a sample buffer;

[0010] The sample buffer is displayed in a subtitle interface view through a sample buffer display layer.

[0011] Optionally, before obtaining the target subtitle image data to be displayed at the current moment, the method further includes:

[0012] Create a subtitle interface view for external subtitles;

[0013] The layer type of the subtitle interface view is a sample buffer display layer, and the size of the subtitle interface view is the same as the size of the video interface view.

[0014] Optionally, before obtaining the target subtitle image data to be displayed at the current moment, the method further includes:

[0015] The target subtitle text is converted into target subtitle image data through a font rendering module; wherein the target subtitle text is the subtitle text to be displayed at the current moment.

[0016] Optionally, the step of converting the target subtitle text into target subtitle image data by means of a font rendering module includes:

[0017] Divide the subtitle text to be displayed at the current moment into target subtitle texts;

[0018] Through the font rendering module, each target subtitle text is converted into corresponding target subtitle image data.

[0019] Optionally, before obtaining the target subtitle image data to be displayed at the current moment, the method further includes:

[0020] Create a fully transparent pixel buffer; the pixel buffer is CVPixelBuffer, and the size of the pixel buffer is the same as the size of the video interface view.

[0021] Optionally, determining a display position of the target subtitle picture data, and overwriting the target subtitle picture data to the display position of a pixel buffer includes:

[0022] Determine a display position corresponding to each target subtitle picture data; the display position is the display position of each target subtitle picture data in the current playback window;

[0023] Each target subtitle picture data is overwritten to a display position corresponding to each target subtitle picture data in the pixel buffer.

[0024] Optionally, displaying the sample buffer in a subtitle interface view through a sample buffer display layer includes:

[0025] The sample buffer is sent to AVSampleBufferDisplayLayer, and the sample buffer is displayed in a subtitle interface view through the AVSampleBufferDisplayLayer; wherein the AVSampleBufferDisplayLayer is the sample buffer display layer.

[0026] In a second aspect, the present application provides a subtitle display device, comprising:

[0027] An acquisition module, used for acquiring target subtitle image data to be displayed at the current moment; wherein the target subtitle image data is image data converted from the subtitle text to be displayed;

[0028] A determination module, used to determine the display position of the target subtitle picture data;

[0029] An overlay module, used for overlaying the target subtitle picture data to the display position of the pixel buffer;

[0030] A first conversion module, used for converting the covered pixel buffer into a sample buffer;

[0031] The display module is used to display the sample buffer in the subtitle interface view through the sample buffer display layer.

[0032] In a third aspect, the present application provides an electronic device, including:

[0033] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the above-mentioned subtitle display method of the present application through the computer program.

[0034] In a fourth aspect, the present application further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used to execute the steps of the above-mentioned subtitle display method of the present application.

[0035] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the present application provides a subtitle display solution. In the present solution, firstly, the target subtitle image data to be displayed at the current moment is obtained; the target subtitle image data is the image data converted by the subtitle text to be displayed; the display position of the target subtitle image data is determined, the target subtitle image data is overwritten to the display position of the pixel buffer, and the overwritten pixel buffer is converted into a sample buffer, and the sample buffer is displayed in the subtitle interface view through the sample buffer display layer; it can be seen that when displaying external subtitles, the present application converts the subtitle text into image data and overwrites it in the pixel buffer, and then converts it into a sample buffer display layer, and the subtitles can be displayed through the sample buffer display layer. Since the sample buffer display layer is built-in to the system, the development cost and difficulty can be reduced; and the present application displays subtitles through the sample buffer display layer, which can also avoid the scene restrictions existing when displaying external subtitles. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0038] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0039] Figure 1 A flowchart of a subtitle display method provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of another subtitle display method provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of another subtitle display method provided in an embodiment of the present application;

[0042] Figure 4 A flowchart of a specific subtitle display method provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of the structure of a subtitle display device provided in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In the related art, subtitles generally exist in two forms, namely: hard-compressed subtitles and external subtitles. Among them, hard-compressed subtitles refer to equating subtitle information with audio and video content during audio and video production, and the two are compressed and encoded together into audio and video files. After the audio and video production is completed, if the subtitle information needs to be changed, it is necessary to re-produce it together with the audio and video content, which is not flexible enough. External subtitles refer to subtitles in which the subtitle information is separated from the audio and video content and exists in an independent form. Common external subtitle file types include srt (SubRip Text, subtitle file format) files, vtt (Versatile Text Track, subtitle file format) files, etc. Compared with hard-compressed subtitles, external subtitles have the advantages of flexibility, easy replacement, and easy multi-language expansion. Therefore, external subtitles are currently widely used.

[0046] There are two common ways to display external subtitles on iOS:

[0047] Method 1: Directly display the text content of external subtitles through the iOS system standard UI controls UITextView / UITextField.

[0048] Method 2: Convert the subtitle text into image data through the font rendering module, and then display the subtitle image through the graphics rendering engine; the data graphics rendering engine can be OpenGLES (OpenGL for Embeded System, open graphics library) / Metal (graphics rendering framework) and the like.

[0049] Of the two methods mentioned above, method 2 is the most common. Among them, method 1 will lead to inflexible design of the player software architecture. In addition, method 1 will be extremely complicated when processing certain specific subtitle effects. For example, if a subtitle contains text of different colors or sizes, a more complex method is required to display the specific subtitle. The technical threshold of method 2 is relatively high, the development difficulty is relatively large, and method 2 may not work in some scenarios, for example, in the background picture-in-picture scene of iOS18 beta1 system, the system prohibits the application layer from performing display operations through graphics rendering engines such as OpenGLES / Metal.

[0050] Therefore, in an embodiment of the present application, a subtitle display method, apparatus, device and storage medium are provided. This solution can be applied to audio and video playback applications of the iOS system. After converting the external subtitle text into image data through the font rendering module, this solution uses the AVSampleBufferDisplayLayer provided by the iOS system itself to display the external subtitle image data on the screen. This solution can solve many limitations and problems of the above-mentioned methods 1 and 2, for example: it can reduce the development difficulty and development cost of the external subtitle display, avoid the scene restrictions that exist when displaying external subtitles, etc.

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0052] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0053] See also Figure 1 , Figure 1 A flowchart of a subtitle display method provided in an embodiment of the present application is provided, and the method specifically comprises the following steps:

[0054] S101, obtaining target subtitle image data to be displayed at the current moment; the target subtitle image data is image data converted from the subtitle text to be displayed;

[0055] In the present application, the subtitles to be displayed are external subtitles. Since the subtitle information of the external subtitles and the audio and video content exist in an independent form, the present application only describes the display scheme of the external subtitles of audio and video, and the audio and video of the external subtitles are displayed in the video interface view through the original scheme, and its specific display scheme is not specifically limited here.

[0056] In this embodiment, since the external subtitles displayed by the audio and video at each moment are different, this application needs to convert the subtitle text to be displayed at each moment into the target subtitle image data. Among them, each moment in this application is the moment when the external subtitles need to be displayed, for example: the subtitles to be displayed at the 1st minute and 30th second of the video are "AAA", and the subtitles do not need to be displayed from the 1st minute and 31st second to the 1st minute and 40th second of the video, and the subtitles to be displayed at the 1st minute and 41st second of the video are "BBB". Then, the moments in this application are 1st minute and 30th second and 1st minute and 41st second respectively. The subtitle text to be displayed at the 1st minute and 30th second is "AAA", and the subtitle text to be displayed at the 1st minute and 41st second is "BBB".

[0057] Furthermore, since the present application needs to convert the subtitle text to be displayed into target subtitle image data, the present application does not need to consider the specific subtitle effect of the subtitle text when displaying external subtitles. For example, if a subtitle text contains text of different colors or sizes, it is only necessary to convert the subtitle text into the corresponding subtitle image data. No additional processing operations are required to display a specific subtitle effect. For the purpose of distinction, the present application refers to the moment when the external subtitle needs to be displayed as the current moment, and the subtitle image data that needs to be displayed at the current moment as the target subtitle image data.

[0058] S102, determining the display position of the target subtitle picture data;

[0059] In the present application, the display position is the display position of the corresponding external subtitles in the current playback window, and each target subtitle picture data has a corresponding display position, such as: the target subtitle picture data includes target subtitle picture data 1, target subtitle picture data 2...target subtitle picture data n, where n is the total number of target subtitle picture data, and the display position includes: display position 1 of target subtitle picture data 1, display position 2 of target subtitle picture data 2...display position n of target subtitle picture data n.

[0060] Furthermore, the display position can be set according to the preset display parameters. For example, by setting the display parameters, the external subtitles are displayed at the bottom center of the video screen to ensure that the user can easily read the subtitle content. The display position in this application can also be customized according to the user's needs. For example, the original display position of the external subtitles is at the bottom center of the video screen. The user can modify the display parameters to change the display position from the bottom center of the video screen to the top center of the video screen.

[0061] S103, overwriting the target subtitle picture data to the display position of the pixel buffer, and converting the overwritten pixel buffer into a sample buffer;

[0062] The pixel buffer in this application is a fully transparent pixel buffer. In the iOS system, the pixel buffer can be CVPixelBuffer, which is an object in the CoreVideo (a framework for processing and managing video data) framework, used to represent image pixel data. Therefore, after obtaining the target subtitle image data, this application needs to overwrite the target subtitle image data to the corresponding display position of the pixel buffer.

[0063] This application overwrites the target subtitle image data to the pixel buffer. After obtaining the overwritten pixel buffer, it is also necessary to convert the overwritten pixel buffer into a sample buffer. In the iOS system, the sample buffer is CMSampleBuffer, which is an object in the Core Media framework. CMSampleBuffer is used to represent audio and video sample data. This application converts CVPixelBuffer to CMSampleBuffer, which can convert pixel data into standard audio and video sample data for subsequent processing.

[0064] S104. Display the sample buffer in the subtitle interface view through the sample buffer display layer.

[0065] This application converts the overwritten pixel buffer into a sample buffer, which can be displayed in the subtitle interface view through the sample buffer display layer. The sample buffer display layer in the iOS system is specifically AVSampleBufferDisplayLayer, which is used to display compressed video frames or uncompressed video frames. Therefore, in this application, the image data of the external subtitles is specifically displayed through AVSampleBufferDisplayLayer, which can make full use of the functions provided by the iOS system itself, with low development difficulty and good compatibility. The subtitle interface view is the external subtitle UIView.

[0066] From the above, it can be seen that when displaying external subtitles, the present application converts the subtitle text into image data and overlays it on the pixel buffer, and then converts it into a sample buffer display layer. The subtitles can be displayed through the sample buffer display layer. Since the sample buffer display layer is built-in to the system, the development cost and difficulty can be reduced. Moreover, the present application displays subtitles through the sample buffer display layer, which can also avoid the scene limitations that exist when displaying external subtitles.

[0067] See also Figure 2 , Figure 2 A flowchart of another subtitle display method provided in an embodiment of the present application, the method specifically comprises the following steps:

[0068] S201, creating a subtitle interface view for external subtitles; wherein the layer type of the subtitle interface view is a sample buffer display layer, and the size of the subtitle interface view is the same as the size of the video interface view;

[0069] S202, creating a fully transparent pixel buffer; the pixel buffer is CVPixelBuffer, and the size of the pixel buffer is the same as the size of the video interface view;

[0070] S203, obtaining target subtitle image data to be displayed at the current moment; the target subtitle image data is image data converted from the subtitle text to be displayed;

[0071] S204, determining the display position of the target subtitle picture data;

[0072] S205, overwriting the target subtitle picture data to the display position of the pixel buffer, and converting the overwritten pixel buffer into a sample buffer;

[0073] S206, sending the sample buffer to AVSampleBufferDisplayLayer, and displaying the sample buffer in the subtitle interface view through AVSampleBufferDisplayLayer; wherein AVSampleBufferDisplayLayer is the sample buffer display layer.

[0074] In this application, it is necessary to create a subtitle interface view UIView for external subtitles before displaying the subtitles, and the layer type of the subtitle interface view is

[0075] AVSampleBufferDisplayLayer, in this application, the subtitle UIView of the external subtitle can be recorded as V, and the size of V is WxH, where W is the width of V and H is the height of V; the size of the subtitle UIView is the same as the size of the video UIView.

[0076] In addition, this application needs to pre-create a fully transparent pixel buffer, which can be a CVPixelBuffer. This application records the fully transparent CVPixelBuffer as CVP. The size of CVP is the same as the size of the video UIView, that is, the size of CVP is WxH.

[0077] After obtaining the target subtitle image data to be displayed at the current moment, this application needs to overwrite the target subtitle image data to the corresponding display position of the pixel buffer CVP, and convert the overwritten pixel buffer CVP into a sample buffer CMSampleBuffer. In this application, the sample buffer CMSampleBuffer is recorded as CMS. After obtaining the converted CMS, this application sends the CMS to AVSampleBufferDisplayLayer, through

[0078] AVSampleBufferDisplayLayer finally displays the external subtitles on view V.

[0079] At present, the internal implementation of the player on the iOS system is to display video frames or pictures through the OpenGLES or Metal graphics engine. At the same time, the iOS system itself also provides AVSampleBufferDisplayLayer for displaying compressed video frames or uncompressed video frames. Since the external subtitle text is converted into image data, it is essentially equivalent to an uncompressed video frame. Therefore, in this application, AVSampleBufferDisplayLayer is directly used to display the external subtitle image data. In this way, the functions provided by the iOS system itself can be fully utilized, and the development difficulty is low and the compatibility is good.

[0080] From the above, it can be seen that before displaying subtitles, the present application needs to pre-create a subtitle interface view, and the type of the subtitle interface view is a sample buffer display layer, so that the sample buffer display layer directly displays the sample buffer in the subtitle interface view to achieve effective display of subtitles; and before displaying subtitles, the present application needs to pre-create a fully transparent pixel buffer, and the size of the pixel buffer is the same as the size of the video interface view, so that the target subtitle image data is overlaid on the pixel buffer to achieve the display of external subtitles. In addition, the present application displays subtitles through AVSampleBufferDisplayLayer, which can also avoid the scene restrictions that exist when displaying external subtitles, for example: external subtitles can also be supported in picture-in-picture scenarios.

[0081] See also Figure 3 , Figure 3 A flowchart of another subtitle display method provided in an embodiment of the present application, the method specifically comprises the following steps:

[0082] S301, creating a subtitle interface view for external subtitles; wherein the layer type of the subtitle interface view is a sample buffer display layer, and the size of the subtitle interface view is the same as the size of the video interface view;

[0083] S302, creating a fully transparent pixel buffer; the pixel buffer is CVPixelBuffer, and the size of the pixel buffer is the same as the size of the video interface view;

[0084] S303, converting the target subtitle text into target subtitle image data through a font rendering module; wherein the target subtitle text is the subtitle text to be displayed at the current moment;

[0085] S304, obtaining target subtitle image data to be displayed at the current moment;

[0086] S305, determining a display position corresponding to each target subtitle picture data; the display position is a display position of each target subtitle picture data in the current playback window;

[0087] S306, overwriting each target subtitle picture data to the display position corresponding to each target subtitle picture data in the pixel buffer;

[0088] S307, converting the covered pixel buffer into a sample buffer;

[0089] S308, sending the sample buffer to AVSampleBufferDisplayLayer, and displaying the sample buffer in the subtitle interface view through AVSampleBufferDisplayLayer; wherein AVSampleBufferDisplayLayer is the sample buffer display layer.

[0090] In this application, the target subtitle text can be converted into the target subtitle image data through the font rendering module. The font rendering module can be freetype (open source font engine library), CoreText (text rendering engine) / CoreGraphic (drawing engine), etc., which is not specifically limited here. In actual application, the appropriate font rendering module can be selected according to the system type.

[0091] In some embodiments of the present application, the process of converting target subtitle text into target subtitle image data through a font rendering module includes: dividing the subtitle text to be displayed at the current moment into target subtitle texts; and converting each target subtitle text into corresponding target subtitle image data through the font rendering module.

[0092] Among them, when the present application generates target subtitle image data, the subtitle text to be displayed at the current moment can be divided into multiple target subtitle texts according to the pre-set division rules. The division rules record how to divide the target subtitle text in different situations. Each target subtitle text after division needs to be converted into different target subtitle image data through the font rendering module. The division rules can be customized. For example, the division rules include: single-line text division rules and multi-line text division rules, that is, if the subtitle text to be displayed at the current moment only needs one line of text to be displayed, then the subtitle text to be displayed can be used as a target subtitle text. If the subtitle text to be displayed at the current moment requires multiple lines of text to be displayed, then each line of subtitle text can be used as a target subtitle text, that is, the target subtitle text corresponding to each line of subtitle text is obtained.

[0093] For example: if there is a first line of subtitle text and a second line of subtitle text, the first line of subtitle text is used as target subtitle text 1, and the second line of subtitle text is used as target subtitle text 2. The target subtitle text 1 obtains target subtitle image data 1 through the font rendering module, and the target subtitle text 2 obtains target subtitle image data 2 through the font rendering module.

[0094] Furthermore, the present application converts the target subtitle text into the target subtitle image data through the font rendering module. This operation can be to convert all the target subtitle texts of the audio and video into the target subtitle image data in advance, so that the corresponding target subtitle image data can be directly obtained at each subsequent moment; or only the target subtitle text to be displayed at the current moment can be converted into the target subtitle image data. Both methods are acceptable and are not specifically limited here.

[0095] After the present application obtains the target subtitle image data to be displayed at the current moment, since the number of target subtitle image data can be multiple, the present application can determine the display position corresponding to each target subtitle image data, which is the display position of each target subtitle image data in the current playback window; for example: the target subtitle image data includes target subtitle image data 1, target subtitle image data 2, and target subtitle image data 3, then the display positions include: display position 1 of target subtitle image data 1, display position 2 of target subtitle image data 2, and display position 3 of target subtitle image data 3.

[0096] After determining the display position of each target subtitle image data, each target subtitle image data can be overwritten to the display position corresponding to each target subtitle image data in the pixel buffer; for example: overwrite target subtitle image data 1 at display position 1 in the pixel buffer, overwrite target subtitle image data 2 at display position 2 in the pixel buffer, and overwrite target subtitle image data 3 at display position 3 in the pixel buffer. Then convert the overwritten pixel buffer into a sample buffer, and display the sample buffer in the subtitle interface view through AVSampleBufferDisplayLayer.

[0097] It can be seen that in this embodiment, the target subtitle image data is converted specifically through the font rendering module, and during the conversion, the letter text to be displayed at the current moment can be divided into multiple target subtitle texts, so as to be converted into multiple target subtitle image data, thereby improving the flexibility of subtitle text conversion. In addition, since the number of target subtitle image data to be displayed can be multiple, the present application needs to determine the display position corresponding to each target subtitle image, and overwrite each target subtitle image data to the corresponding display position of the pixel buffer, so as to realize the simultaneous display of multiple target subtitle image data.

[0098] See also Figure 4 , Figure 4 A flowchart of a specific subtitle display method provided in an embodiment of the present application. Figure 4 It can be seen that if there are multiple external subtitle texts to be displayed at the current moment, they are respectively recorded as T1, T2...Tn, and T1, T2...Tn are respectively converted into RGBA image data through the font rendering module to obtain external subtitle RGBA (color coding system) image P1, external subtitle RGBA image P2...external subtitle RGBA image Pn, and the converted RGBA image data are respectively recorded as: P1, P2,..., Pn, and correspondingly, the display positions of the external subtitles in the current playback window are respectively recorded as L1, L2,..., Ln.

[0099] Then, the contents of the subtitle image data P1, P2, ..., Pn are respectively overlaid on the corresponding positions L1, L2, ..., Ln in the fully transparent CVPixelBuffer (CVP, pixel buffer) to obtain the synthesized external subtitle CVPixelBuffer, which is converted into an external subtitle CMSampleBuffer (CMS, sample buffer), and the external subtitle CMSampleBuffer is displayed on V (external subtitle UIView) through AVSampleBufferDisplayLayer (sample buffer display layer).

[0100] From the above, it can be seen that after converting the subtitle text into RGBA image data, this application does not need to use the OpenGLES / Metal graphics engine for display, but converts it into CMSampleBuffer and uses the system's own AVSampleBufferDisplayLayer for display. This method can reduce development costs, because AVSampleBufferDisplayLayer is provided by the iOS system itself and has powerful functions; and this method has good compatibility and can solve the limitation of the iOS system background picture-in-picture scene on the application layer using OpenGLES / Metal for drawing.

[0101] See also Figure 5 , Figure 5 A schematic diagram of the structure of a subtitle display device provided in an embodiment of the present application, the device specifically comprises:

[0102] The acquisition module 11 is used to acquire the target subtitle image data to be displayed at the current moment; wherein the target subtitle image data is the image data converted by the subtitle text to be displayed;

[0103] A determination module 12, used to determine the display position of the target subtitle picture data;

[0104] An overlay module 13, used to overlay the target subtitle picture data to the display position of the pixel buffer;

[0105] A first conversion module 14, configured to convert the covered pixel buffer into a sample buffer;

[0106] The display module 15 is used to display the sample buffer in the subtitle interface view through the sample buffer display layer.

[0107] As an optional embodiment, the subtitle display device further includes:

[0108] The first creation module is used to create a subtitle interface view of an external subtitle;

[0109] The layer type of the subtitle interface view is a sample buffer display layer, and the size of the subtitle interface view is the same as the size of the video interface view.

[0110] As an optional embodiment, the subtitle display device further includes:

[0111] The second conversion module is used to convert the target subtitle text into the target subtitle picture data through the font rendering module before acquiring the target subtitle picture data to be displayed at the current moment; wherein the target subtitle text is the subtitle text to be displayed at the current moment.

[0112] As an optional embodiment, the second conversion module includes:

[0113] A division unit, used for dividing the subtitle text to be displayed at the current moment into target subtitle texts;

[0114] The conversion unit is used to convert each target subtitle text into corresponding target subtitle image data through a font rendering module.

[0115] As an optional embodiment, the subtitle display device further includes:

[0116] The second creation module is used to create a fully transparent pixel buffer; the pixel buffer is CVPixelBuffer, and the size of the pixel buffer is the same as the size of the video interface view.

[0117] As an optional embodiment, the determination module is specifically used to: determine a display position corresponding to each target subtitle picture data; the display position is a display position of each target subtitle picture data in the current playback window;

[0118] The covering module is specifically used to cover each target subtitle picture data to the display position corresponding to each target subtitle picture data in the pixel buffer.

[0119] As an optional embodiment, the display module is specifically used for:

[0120] The sample buffer is sent to AVSampleBufferDisplayLayer, and the sample buffer is displayed in a subtitle interface view through the AVSampleBufferDisplayLayer; wherein the AVSampleBufferDisplayLayer is the sample buffer display layer.

[0121] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0122] See also Figure 6 , Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, the electronic device specifically includes:

[0123] The processor 21, the memory 22 and the computer program stored in the memory 22 and executable on the processor 21, the processor 21 executes the steps of the subtitle display method described in any of the above method embodiments through the computer program.

[0124] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0125] The memory 22 may include one or more computer-readable storage media, which may be non-transitory. The memory 22 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 22 is at least used to store the following computer program 221, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps in the subtitle display method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 22 may also include an operating system 222 and data 223, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 222 may include Windows, Unix, Linux, etc.

[0126] In some embodiments, the electronic device may further include a display screen 23 , an input / output interface 24 , a communication interface 25 , a sensor 26 , a power source 27 , and a communication bus 28 .

[0127] certainly, Figure 6 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of the present application. In actual applications, the electronic device may include Figure 6 More or fewer components than shown, or combinations of certain components.

[0128] In another exemplary embodiment, a computer storage medium is also provided, and when the program instructions are executed by a processor, the steps of the subtitle display method described in any of the above method embodiments are implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0129] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0130] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0131] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A subtitle display method, characterized in that: include: Acquire target subtitle image data to be displayed at the current moment; wherein the target subtitle image data is image data converted from the subtitle text to be displayed; Determining a display position of the target subtitle picture data; Overwriting the target subtitle picture data to the display position of the pixel buffer, and converting the overwritten pixel buffer into a sample buffer; The sample buffer is displayed in a subtitle interface view through a sample buffer display layer.

2. The subtitle display method according to claim 1, characterized in that: Before obtaining the target subtitle picture data to be displayed at the current moment, the method further includes: Create a subtitle interface view for external subtitles; The layer type of the subtitle interface view is a sample buffer display layer, and the size of the subtitle interface view is the same as the size of the video interface view.

3. The subtitle display method according to claim 1, characterized in that: Before obtaining the target subtitle picture data to be displayed at the current moment, the method further includes: The target subtitle text is converted into target subtitle image data through a font rendering module; wherein the target subtitle text is the subtitle text to be displayed at the current moment.

4. The subtitle display method according to claim 3, characterized in that: The target subtitle text is converted into target subtitle picture data by the font rendering module, including: Divide the subtitle text to be displayed at the current moment into target subtitle texts; Through the font rendering module, each target subtitle text is converted into corresponding target subtitle image data.

5. The subtitle display method according to claim 1, characterized in that: Before obtaining the target subtitle picture data to be displayed at the current moment, the method further includes: Create a fully transparent pixel buffer; the pixel buffer is CVPixelBuffer, and the size of the pixel buffer is the same as the size of the video interface view.

6. The subtitle display method according to claim 5, characterized in that: Determining a display position of the target subtitle picture data, and overwriting the target subtitle picture data to the display position of the pixel buffer, comprises: Determine a display position corresponding to each target subtitle picture data; the display position is the display position of each target subtitle picture data in the current playback window; Each target subtitle picture data is overwritten to a display position corresponding to each target subtitle picture data in the pixel buffer.

7. The subtitle display method according to any one of claims 1 to 6, characterized in that: The step of displaying the sample buffer in the subtitle interface view through the sample buffer display layer includes: The sample buffer is sent to AVSampleBufferDisplayLayer, and the sample buffer is displayed in a subtitle interface view through the AVSampleBufferDisplayLayer; wherein the AVSampleBufferDisplayLayer is the sample buffer display layer.

8. A subtitle display device, characterized in that: include: An acquisition module, used for acquiring target subtitle image data to be displayed at the current moment; wherein the target subtitle image data is image data converted from the subtitle text to be displayed; A determination module, used to determine the display position of the target subtitle picture data; An overlay module, used for overlaying the target subtitle picture data to the display position of the pixel buffer; A first conversion module, used for converting the covered pixel buffer into a sample buffer; The display module is used to display the sample buffer in the subtitle interface view through the sample buffer display layer.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the subtitle display method described in any one of claims 1 to 7 of the present application through the computer program.

10. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, and the computer executable instructions are used to execute the steps of the subtitle display method described in any one of claims 1 to 7 of the present application.

Images