Subtitle image generation method, device, electronic device and storage medium

Abstract

The disclosed embodiment discloses a subtitle image generation method, device, electronic device and storage medium. The method comprises: obtaining initial attribute values ​​of each subtitle level in the subtitles to be displayed, wherein the subtitle levels are divided based on different types of display content in the subtitles to be displayed; obtaining subtitle texts of the subtitles to be displayed, and for each subtitle level, performing text typeset drawing based on the subtitle text and the initial attribute values ​​to obtain an initial subtitle layer of the subtitle level at an initial time point; obtaining target attribute values ​​of each subtitle level, and performing image transformation on the corresponding initial subtitle layer based on the target attribute values ​​to obtain a target subtitle layer of the subtitle level at a target time point; superimposing multiple target subtitle layers to generate a target subtitle image of the subtitles to be displayed at the target time point; the disclosed embodiment can enrich the animation effect of the subtitles to be displayed.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a subtitle image generation method, device, electronic device and storage medium. Background Art

[0002] In audio and video playback, subtitles can provide text information corresponding to the audio and video content, helping viewers to better understand the plot and dialogue of the film. Subtitles with animation can further highlight key information and enhance the atmosphere of the film, which plays a very important role in improving the viewing experience. At present, all displayed content in traditional subtitle animations will present the same animation effect, resulting in a relatively simple animation effect of subtitle animations. Summary of the invention

[0003] The following is a summary of the subject matter of the detailed description of the present disclosure. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present disclosure provide a subtitle image generation method, device, electronic device, and storage medium, which can enrich the animation effect of subtitles to be displayed.

[0005] On the one hand, an embodiment of the present disclosure provides a method for generating a subtitle image, comprising:

[0006] Acquire initial attribute values ​​of each subtitle level in the subtitles to be displayed, wherein the subtitle levels are divided based on different types of display content in the subtitles to be displayed, and the initial attribute values ​​are attribute values ​​of the subtitle attributes of the subtitle levels at an initial time point;

[0007] Acquire the subtitle text of the subtitle to be displayed, and for each of the subtitle levels, perform text layout and drawing based on the subtitle text and the initial attribute value to obtain an initial subtitle layer of the subtitle level at the initial time point;

[0008] Obtaining target attribute values ​​of each of the subtitle levels, performing image transformation on the corresponding initial subtitle layer based on the target attribute values, and obtaining a target subtitle layer of the subtitle level at a target time point, wherein the target attribute value is an attribute value of the subtitle attribute at the target time point, and the target time point is a time point after the initial time point;

[0009] The multiple target subtitle layers are superimposed to generate a target subtitle image of the subtitle to be displayed at the target time point.

[0010] On the other hand, the embodiment of the present disclosure further provides a subtitle image generating device, including:

[0011] An acquisition module, used to acquire initial attribute values ​​of each subtitle level in the subtitles to be displayed, wherein the subtitle levels are divided based on different types of display content in the subtitles to be displayed, and the initial attribute values ​​are attribute values ​​of the subtitle attributes of the subtitle levels at an initial time point;

[0012] A typesetting and drawing module, used for obtaining the subtitle text of the subtitle to be displayed, and for each subtitle level, performing text typesetting and drawing based on the subtitle text and the initial attribute value to obtain an initial subtitle layer of the subtitle level at the initial time point;

[0013] An image transformation module, used to obtain a target attribute value of each of the subtitle levels, and perform image transformation on the corresponding initial subtitle layer based on the target attribute value to obtain a target subtitle layer of the subtitle level at a target time point, wherein the target attribute value is an attribute value of the subtitle attribute at the target time point, and the target time point is a time point after the initial time point;

[0014] The image generation module is used to superimpose multiple target subtitle layers to generate a target subtitle image of the subtitle to be displayed at the target time point.

[0015] Furthermore, the above image transformation module is specifically used for:

[0016] Determining an attribute change amount between the target attribute value and the corresponding initial attribute value;

[0017] Based on the attribute change amount, the corresponding initial subtitle layer is subjected to image transformation to obtain a target subtitle layer of the subtitle level at a target time point.

[0018] Further, the subtitle attribute includes a color attribute, the target attribute value includes a target color value of the color attribute, the attribute change includes a color change of the color attribute, and the image transformation module is specifically used for:

[0019] When the color change amount indicates that the subtitle level has a color change, the color value of the corresponding non-transparent pixel in the initial subtitle layer is transformed into the target color value to obtain a target subtitle layer of the subtitle level at a target time point.

[0020] Furthermore, the subtitle attribute includes a geometric attribute, the attribute change includes a geometric change of the geometric attribute, and the image transformation module is specifically used for:

[0021] Determine affine transformation parameters based on the geometric variation, and construct an affine transformation matrix according to the affine transformation parameters;

[0022] Based on the affine transformation matrix, the corresponding initial subtitle layer is affine transformed to obtain a target subtitle layer of the subtitle level at a target time point.

[0023] Furthermore, the subtitle attribute includes a mask attribute, the target attribute value includes a target mask value of the mask attribute, and the image transformation module is specifically used for:

[0024] Based on the target mask value, determining a mask area in the corresponding initial subtitle layer;

[0025] In the corresponding initial subtitle layer, the color values ​​of the non-transparent pixels in the mask area are transformed into preset mask color values ​​to obtain a target subtitle layer of the subtitle level at a target time point.

[0026] Furthermore, the mask attribute includes a color change ratio attribute, the target mask value includes a target color change ratio value of the color change ratio attribute, and the above-mentioned image transformation module is specifically used for:

[0027] Obtaining the layer width of the target subtitle layer, and determining the width threshold according to the product of the target color change ratio value and the layer width;

[0028] In the corresponding initial subtitle layer, an area whose horizontal coordinate is smaller than the width threshold is determined as a mask area.

[0029] Further, the mask attribute includes a plurality of mask coordinate attributes, the target mask value includes target mask coordinates of each of the mask coordinate attributes, and the above-mentioned image transformation module is specifically used for:

[0030] Determining a visible area in the corresponding initial subtitle layer according to each of the target mask coordinates, wherein the target mask coordinates are located at an area boundary of the visible area;

[0031] In the corresponding initial subtitle layer, an area outside the visible area is determined as a mask area.

[0032] Furthermore, the acquisition module is specifically used for:

[0033] Acquire a dynamic attribute set of the subtitles to be displayed, and divide the dynamic attribute set into an equivalent attribute set and a non-equivalent attribute set, wherein the equivalent attribute set includes subtitle attributes for realizing animation effects through image transformation, and the non-equivalent attribute set includes subtitle attributes for which animation effects cannot be realized through image transformation;

[0034] When the non-equivalent attribute set is an empty set, initial attribute values ​​of each subtitle level in the subtitle to be displayed are obtained from the equivalent attribute set.

[0035] Furthermore, the subtitle attributes in the equivalent attribute set include color attributes of each subtitle level, and the acquisition module is specifically used for:

[0036] When the non-equivalent attribute set is an empty set, for each of the subtitle levels, the color attribute corresponding to the current subtitle level in the equivalent attribute set is kept unchanged, and the color attributes corresponding to the remaining subtitle levels in the equivalent attribute set are adjusted to transparent, so as to obtain a level attribute set corresponding to the current subtitle level;

[0037] From each of the level attribute sets, the initial attribute value of the corresponding subtitle level in the subtitle to be displayed is obtained.

[0038] Furthermore, the hierarchical attribute set also includes an attribute change function corresponding to the initial attribute value, and the above-mentioned image transformation module is specifically used for:

[0039] Obtaining the target time point;

[0040] The target time point is respectively input into the attribute change function in each of the hierarchical attribute sets for calculation, so as to obtain the target attribute value of each of the subtitle hierarchies.

[0041] Furthermore, the above image transformation module is specifically used for:

[0042] Inputting the target time point into the attribute change function in each of the hierarchical attribute sets respectively for calculation, to obtain reference attribute values ​​of each of the subtitle hierarchies;

[0043] Inputting the subtitle text into a large language model for emotion recognition to obtain target emotion information of the subtitle text;

[0044] The target emotion information and the reference attribute value are spliced ​​and input into a regression model for regression to obtain target attribute values ​​of each subtitle level.

[0045] Furthermore, the multiple subtitle levels include a text level, a border level, a shadow level and a background level, and the above-mentioned image generation module is specifically used for:

[0046] The plurality of target subtitle layers are sequentially superimposed in the order of the background level, the shadow level, the frame level and the text level to generate a target subtitle image of the subtitle to be displayed at the target time point.

[0047] On the other hand, an embodiment of the present disclosure further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned subtitle image generation method when executing the computer program.

[0048] On the other hand, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned subtitle image generation method.

[0049] On the other hand, the embodiment of the present disclosure further provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned subtitle image generation method.

[0050] The disclosed embodiments include at least the following beneficial effects: by obtaining the initial attribute values ​​of the subtitle attributes of each subtitle level at the initial time point, and obtaining the subtitle text of the subtitle to be displayed, since the subtitle level is divided based on different types of display content in the subtitle to be displayed, it is possible to perform text layout and drawing based on the subtitle text and the initial attribute values, and obtain the initial subtitle layer of the subtitle level at the initial time point, thereby realizing the division of subtitle layers for different display contents. Then, based on the target attribute value of the subtitle attribute at the target time point, the initial subtitle layer is transformed into an image to obtain the target subtitle layer of the subtitle level at the target time point, and then multiple target subtitle layers are superimposed to generate the target subtitle image of the subtitle to be displayed at the target time point. In the image transformation, the image transformation process of the corresponding initial subtitle layer is controlled by each target attribute value, so that the refined control of each subtitle level is realized, so that different types of display content in the subtitle to be displayed can be finely controlled to enrich the animation effect of the subtitle to be displayed; in addition, for multiple frame images of the subtitle to be displayed, the initial subtitle layer can be regarded as a layer in the initial frame image displayed by the subtitle to be displayed at the initial time point, and the target subtitle layer can be regarded as a layer in the target frame image displayed by the subtitle to be displayed at the target time point. In the subtitle animation rendering process, it is only necessary to perform text layout drawing when generating the initial subtitle layer, and there is no need to perform text layout drawing when generating the target subtitle layer. Instead, the target subtitle layer is obtained by image transformation based on the initial subtitle layer, which can avoid frequent text layout drawing with large computational complexity, thereby improving the rendering efficiency of the subtitle animation and improving the real-time and smoothness of the subtitle animation.

[0051] Other features and advantages of the present disclosure will be set forth in the following description, and in part will be apparent from the description, or may be learned by practicing the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide further understanding of the technical solution of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solution of the present disclosure and do not constitute a limitation on the technical solution of the present disclosure.

[0053] Figure 1 A schematic diagram of an optional implementation environment provided for an embodiment of the present disclosure;

[0054] Figure 2 A schematic diagram of an optional flow chart of a method for generating a subtitle image provided by an embodiment of the present disclosure;

[0055] Figure 3 A schematic diagram of a first optional process for determining a target subtitle layer provided in an embodiment of the present disclosure;

[0056] Figure 4 A second optional flow chart of determining a target subtitle layer provided in an embodiment of the present disclosure;

[0057] Figure 5 A third optional flow chart of determining a target subtitle layer provided in an embodiment of the present disclosure;

[0058] Figure 6 A fourth optional flow chart of determining a target subtitle layer provided in an embodiment of the present disclosure;

[0059] Figure 7 A fifth optional flow chart of determining a target subtitle layer provided in an embodiment of the present disclosure;

[0060] Figure 8 A schematic diagram of an optional process of mask transformation provided in an embodiment of the present disclosure;

[0061] Fig. 9 A schematic diagram of an optional grouping of a dynamic attribute set provided in an embodiment of the present disclosure;

[0062] Fig.10 A schematic diagram of an optional process for determining a target attribute value provided in an embodiment of the present disclosure;

[0063] Fig.11 A schematic diagram of an optional architecture of a subtitle image generation method provided in an embodiment of the present disclosure;

[0064] Fig.12 A schematic diagram of an optional structure of a subtitle image generating device provided in an embodiment of the present disclosure;

[0065] Fig.13 A partial structural block diagram of a terminal provided in an embodiment of the present disclosure;

[0066] Fig.14A partial structural block diagram of a server provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0068] It should be noted that in various specific embodiments of the present disclosure, when it comes to the need to perform relevant processing based on data related to the characteristics of the target object such as the target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. Among them, the target object can be a user. In addition, when the embodiment of the present disclosure needs to obtain the attribute information of the target object, the separate permission or separate consent of the target object will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the separate permission or separate consent of the target object, the necessary target object-related data used to enable the normal operation of the embodiment of the present disclosure will be obtained.

[0069] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0070] To facilitate understanding of the technical solution provided by the embodiments of the present disclosure, some key terms used in the embodiments of the present disclosure are explained here:

[0071] Subtitles: refers to the display of non-visual content such as dialogues in television, film, and stage works in text form. It also refers to the text processed in the later stage of film and television works. The subtitles of dialogues in film and television works generally appear at the bottom of the screen, while the subtitles of drama works may be displayed on the sides or above the stage. Its function is to display the voice content of the program in the form of subtitles, which can help audiences with poor hearing understand the content of the program. In addition, subtitles can also be used to translate foreign language programs, so that audiences who do not understand the foreign language can hear the original soundtrack and understand the content of the program at the same time. Therefore, in audio and video playback, subtitles can provide text information corresponding to the audio and video content to help the audience better understand the plot and dialogue of the film.

[0072] At present, all displayed contents in traditional subtitle animations will present the same animation effect, resulting in a relatively simple animation effect of the subtitle animation.

[0073] Based on this, the embodiments of the present disclosure provide a subtitle image generation method, device, electronic device and storage medium, which can enrich the animation effects of subtitles to be displayed.

[0074] Reference Figure 1 , Figure 1 A schematic diagram of an optional implementation environment provided for an embodiment of the present disclosure, the implementation environment includes a terminal 101 and a server 102, wherein the terminal 101 and the server 102 are connected via a communication network.

[0075] Exemplarily, the server 102 may obtain the subtitle information input by the terminal 101, wherein the subtitle information is information related to the subtitles to be displayed, for example, it may include information related to subtitle attributes, subtitle text, subtitle display time point, etc. The initial attribute values ​​of each subtitle level in the subtitles to be displayed are obtained from the subtitle information, wherein the subtitle level is divided based on different types of display content in the subtitles to be displayed, and the initial attribute value is the attribute value of the subtitle attribute of the subtitle level at the initial time point; the subtitle text of the subtitle to be displayed is obtained from the subtitle information, and for each subtitle level, the text layout and drawing are performed based on the subtitle text and the initial attribute value to obtain the initial subtitle layer of the subtitle level at the initial time point; the target attribute value of each subtitle level is obtained, and the corresponding initial subtitle layer is image-transformed based on the target attribute value to obtain the target subtitle layer of the subtitle level at the target time point, wherein the target attribute value is the attribute value of the subtitle attribute at the target time point, and the target time point is a time point after the initial time point; multiple target subtitle layers are superimposed to generate a target subtitle image of the subtitle to be displayed at the target time point; the server 102 sends the target subtitle image to the terminal 101.

[0076] Server 102 obtains the initial attribute values ​​of the subtitle attributes of each subtitle level at the initial time point, and obtains the subtitle text of the subtitle to be displayed. Since the subtitle level is divided based on different types of display content in the subtitle to be displayed, it is possible to perform text layout and drawing based on the subtitle text and the initial attribute values, and obtain the initial subtitle layer of the subtitle level at the initial time point, thereby realizing the division of subtitle layers for different display contents. Then, based on the target attribute value of the subtitle attribute at the target time point, the initial subtitle layer is transformed into an image to obtain the target subtitle layer of the subtitle level at the target time point, and then multiple target subtitle layers are superimposed to generate the target subtitle image of the subtitle to be displayed at the target time point. In the image transformation, the image transformation process of the corresponding initial subtitle layer is controlled by each target attribute value, so that the refined control of each subtitle level is realized, so that different types of display content in the subtitle to be displayed can be finely controlled to enrich the animation effect of the subtitle to be displayed; in addition, for multiple frame images of the subtitle to be displayed, the initial subtitle layer can be regarded as a layer in the initial frame image displayed by the subtitle to be displayed at the initial time point, and the target subtitle layer can be regarded as a layer in the target frame image displayed by the subtitle to be displayed at the target time point. In the subtitle animation rendering process, it is only necessary to perform text layout drawing when generating the initial subtitle layer, and there is no need to perform text layout drawing when generating the target subtitle layer. Instead, the target subtitle layer is obtained by image transformation based on the initial subtitle layer, which can avoid frequent text layout drawing with large computational complexity, thereby improving the rendering efficiency of the subtitle animation and improving the real-time and smoothness of the subtitle animation.

[0077] Server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In addition, server 102 may also be a node server in a blockchain network.

[0078] The terminal 101 may be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, an intelligent wearable device, a vehicle-mounted terminal, etc., but is not limited thereto. The terminal 101 and the server 102 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present disclosure.

[0079] Reference Figure 2 , Figure 2An optional flow chart of a subtitle image generation method provided in an embodiment of the present disclosure is provided. The subtitle image generation method can be executed by a server, or by a terminal, or by a server and a terminal. The subtitle image generation method includes but is not limited to the following steps 201 to 204.

[0080] Step 201: Obtain the initial attribute value of each subtitle level in the subtitle to be displayed.

[0081] Among them, the subtitles to be displayed refer to subtitles that need to be displayed on the video. The video can be obtained from sources such as movies, animations, and photographic works, and the embodiments of the present disclosure are not limited here. Since subtitles are usually displayed continuously for a period of time, the subtitle animation of the subtitles to be displayed can be displayed in multiple video frames. Displaying the subtitles to be displayed in the video is equivalent to rendering corresponding subtitle images in multiple video frames, and each subtitle image is a frame image of the subtitle animation.

[0082] Among them, the subtitle level is divided based on different types of display content in the subtitles to be displayed. The subtitles may include multiple types of display content, for example, the subtitles include text, borders, shadows, and backgrounds. The text of the subtitles refers to the text content displayed on the video screen. The text of the subtitles may include dialogues, narrations, or other information. The border of the subtitles refers to the lines surrounding the text, and the border is used to increase the readability of the text. The shadow of the subtitles refers to the shadow effect added below or around the text, and the shadow is used to increase the three-dimensional sense of the subtitles. The background of the subtitles refers to the color block or pattern behind the text, and the background is used to enhance the readability of the subtitles. Different types of display content in the subtitles to be displayed are divided into different subtitle levels, for example, the background of the subtitles is divided into the first subtitle level, the shadow of the subtitles is divided into the second subtitle level, the border of the subtitles is divided into the third subtitle level, and the text of the subtitles is divided into the fourth subtitle level.

[0083] It should be noted that subtitles can have multiple subtitle attributes. Subtitle attributes refer to attributes used to describe the characteristics of subtitles. Dynamic effects can be added to subtitles to obtain corresponding subtitle animations. Since subtitle animation is a process in which certain subtitle attributes change over time, that is, the subtitle attributes of the subtitles have at least one dynamic attribute, the subtitle attributes can be divided into static attributes and dynamic attributes according to whether animation effects can be achieved. Static attributes refer to attributes that cannot achieve animation effects, and dynamic attributes refer to attributes used to achieve animation effects.

[0084] For example, static properties may include font name property, font size property, bold property, italic property, underline property, and strikethrough property, etc.; dynamic properties may include text color property, character spacing property, border width property, border color property, shadow distance property, shadow color property, background color property, edge blur property, rotation property, scaling property, shearing property, position property, color change ratio property, and mask coordinate property, etc.;

[0085] Among them, the attribute value type of the font name attribute is a string; the attribute value types of the font size attribute, character spacing attribute, border width attribute, shadow distance attribute, edge blur attribute, rotation attribute, scaling attribute, shearing attribute, position attribute, and color change ratio attribute are all floating point numbers; the attribute value types of the text color attribute, border color attribute, shadow color attribute, and background color attribute are all color values; the attribute value types of the bold attribute, italic attribute, underline attribute, and strikethrough attribute are all Boolean values; the attribute value type of the mask coordinate attribute is coordinates.

[0086] Specifically, subtitle attributes can be represented by a tuple. Taking dynamic attributes as an example, the expression formula of subtitle attributes in subtitles is as follows:

[0087]

[0088] in, It refers to subtitle attributes, It refers to The initial value of the subtitle attribute when it starts to change. It refers to The final value of the subtitle attribute when it stops changing. It refers to The time point at which the subtitle attribute starts to change. It refers to The end time point when the subtitle attribute ends changing. It refers to The transition curve expression of the subtitle attribute change is equivalent to the attribute change function. is a set of identifiers for subtitle attributes. , , , as well as All of them can be obtained from the subtitle information input by relevant personnel.

[0089] It is worth noting that for dynamic properties, a basic animation effect corresponds to a dynamic property. For example, the y-axis scaling factor of the subtitle changes from 1 to 1.5 in a linear transition from 4 seconds to 6 seconds, which can be expressed as , at the start time of subtitle display To the time when the change started The attribute values ​​of the subtitle attributes are kept at the initial values ​​of the attributes. To the end time of subtitle display The attribute values ​​of the subtitle attributes are kept as the final values ​​of the attributes, and the start time point is displayed. And display the end time point Belongs to the subtitle display time point, display start time point And display the end time point It can be obtained from the subtitle information entered by relevant personnel.

[0090] In addition, for static attributes, static attributes can be regarded as special cases of dynamic attributes. Therefore, static attributes can use the same expression formula as dynamic attributes. The expression formula of static attributes usually meets the following conditions: , as well as , It refers to the time when the subtitles start to be displayed. It refers to the end time point of subtitle display, that is, the display start time point is the change start time point, and the display end time point is the change end time point. The initial value of the attribute is the same as the final value of the attribute, which is equivalent to the attribute value of the static attribute remaining unchanged during the subtitle display.

[0091] Among them, the initial attribute value is the attribute value of the subtitle attribute of the subtitle level at the initial time point. For any subtitle attribute, when the representation formula of the subtitle attribute is not an empty set, it means that the subtitle has the subtitle attribute. The types of subtitle attributes possessed by the subtitle can be one or more. The types of subtitle attributes at each subtitle level can be the same as the types of subtitle attributes possessed by the subtitle. Since different subtitle levels are used to represent different types of subtitle content, there may be differences in the attribute values ​​of some subtitle attributes of different subtitle levels. Specifically, the number of initial attribute values ​​is the same as the number of types of subtitle attributes at the subtitle level. Therefore, it is necessary to obtain the attribute value of each subtitle attribute of the subtitle level at the initial time point. Assuming that there are 10 types of subtitle attributes at the subtitle level, it is necessary to obtain the attribute values ​​of these 10 subtitle attributes at the initial time point.

[0092] Specifically, when the first frame image among multiple subtitle images to be displayed is selected as the initial frame image, the initial time point is the playback time point corresponding to the first frame image, that is, the initial time point is the display start time point of the subtitles. When other frame images to be displayed are selected as the initial frame image, the initial time point is the playback time point corresponding to the selected frame image. Therefore, the initial time point is specifically the playback time point of the initial frame image. When the initial frame image is the first frame image, the subtitle attributes are set to Determine the initial property value.

[0093] Based on this, since different subtitle levels correspond to different types of display content in the subtitles to be displayed, obtaining the initial attribute values ​​of each subtitle level is equivalent to obtaining the initial attribute values ​​of different types of display content in the subtitles to be displayed, thereby realizing separate processing of different types of display content in the subtitles to be displayed, and subsequently being able to accurately and personalized adjust the subtitle effects of different types of display content in the subtitles to be displayed, thereby improving the quality and richness of the subtitle effects of the subtitles to be displayed.

[0094] Step 202: Obtain the subtitle text of the subtitle to be displayed, and for each subtitle level, perform text layout and drawing based on the subtitle text and the initial attribute value to obtain the initial subtitle layer of the subtitle level at the initial time point.

[0095] Among them, subtitle text refers to the text content displayed on the video screen, and the subtitle text can be obtained from the subtitle information input by relevant personnel. For the subtitle level where the subtitle text is located, since the type corresponding to the subtitle level is the text type, the pixels at the location of the text content in the initial subtitle layer are non-transparent pixels, and the pixels at other locations are transparent pixels, which can ensure that the initial subtitle layer can display the text content. Transparent pixels refer to pixels whose color values ​​are transparent color values, and transparent pixels are invisible; for other subtitle levels, since the type corresponding to the subtitle level is not the text type, the pixels at the location of the text content in the initial subtitle layer are transparent pixels, which can ensure that the initial subtitle layer will not display the text content, but will only display the display content of the corresponding type of the subtitle level.

[0096] It should be noted that different subtitle layers usually contain different display contents. By superimposing multiple subtitle layers, the display contents of each subtitle layer can be combined and displayed in the same picture. Therefore, by superimposing the initial subtitle layers of each subtitle level, the initial frame image displayed by the subtitles to be displayed at the initial time point can be obtained.

[0097] Based on this, the initial subtitle layer is used to display the corresponding type of display content. The initial subtitle layer is obtained by typesetting and drawing the subtitle text and the initial attribute values. Through appropriate text typesetting and drawing, the clarity and readability of the content displayed by the subtitles to be displayed at the initial time point can be effectively improved. The text typesetting and drawing process usually has a large amount of calculation. Especially when the subtitle animation involves more dynamically changing subtitle attributes, the text typesetting and drawing process is often time-consuming. By only typeset and draw the subtitle image of the subtitle to be displayed at the initial time point, it is possible to avoid frequent text typesetting and drawing with a large amount of calculation, thereby improving the rendering efficiency of the subtitle animation and improving the real-time and smoothness of the subtitle animation.

[0098] Specifically, text layout drawing is performed based on the subtitle text and the initial attribute value. Specifically, the subtitle text and the initial attribute value can be input into the text layout drawing function, and then the initial subtitle layer is generated; the text layout drawing function can be a function provided in the relevant technology or a function adaptively optimized based on the current video scene, which is not limited in the embodiment of the present disclosure; the determination formula of the initial subtitle layer is as follows:

[0099]

[0100] in, For the The initial subtitle layer corresponding to the subtitle level, Draw functions for text layout, For subtitle text, For the A set of initial attribute values ​​of all subtitle attributes corresponding to a subtitle level.

[0101] Exemplarily, in text typesetting and drawing, in addition to the initial attribute values ​​of dynamic attributes, the attribute values ​​of static attributes are also needed. When a certain type of subtitle attribute is an empty set, the attribute value of the subtitle attribute is usually set to the preset attribute value in the preset style. Specifically, the attribute values ​​of attributes such as font size, text alignment, line spacing, character spacing, interface size of the display interface, and the subtitle text can be input into the text line breaking algorithm, and the text is typeset through the text line breaking algorithm to achieve a reasonable layout of the subtitle text, and then the image is drawn based on the text typesetting result to obtain the initial subtitle layer; the font size belongs to the subtitle attribute, the text alignment, line spacing, character spacing, interface size of the display interface can be obtained by pre-setting or real-time input, and the text line breaking algorithm can adopt a greedy algorithm, a dynamic programming algorithm, etc., which is not limited to the embodiments of the present disclosure.

[0102] Step 203: Obtain target attribute values ​​of each subtitle level, perform image transformation on the corresponding initial subtitle layer based on the target attribute values, and obtain the target subtitle layer of the subtitle level at the target time point.

[0103] Among them, the target attribute value is the attribute value of the subtitle attribute at the target time point, and the target time point is the time point after the initial time point. Usually, when the initial time point is the display start time point of the subtitle, the target time point is other display time points of the subtitle. The number of target time points can be multiple, so the target subtitle layer can also be multiple. For example, all display time points after the display start time point of the subtitle can be determined as target time points; the initial time point can also be other display time points of the subtitle, which is not limited in the embodiments of the present disclosure.

[0104] Specifically, the initial subtitle layer is a two-dimensional digital matrix composed of pixels, and the image is also a two-dimensional digital matrix composed of pixels, so the initial subtitle layer can essentially be regarded as a special image. Therefore, the initial subtitle layer can be transformed to obtain a new subtitle layer to add dynamic effects to the subtitles, such as color transformation or affine transformation of the initial subtitle layer.

[0105] Based on this, by obtaining the target attribute value of the subtitle attribute at the target time point, the corresponding initial subtitle layer can be transformed based on the target attribute value to obtain the target subtitle layer of the subtitle level at the target time point. Therefore, the target subtitle layer is not obtained by text typesetting and drawing, but is obtained by image transformation based on the initial subtitle layer. Obtaining the target subtitle layer through image transformation with less computational complexity can effectively improve the generation efficiency of the target subtitle layer.

[0106] Step 204: Overlay multiple target subtitle layers to generate a target subtitle image of the subtitles to be displayed at a target time point.

[0107] Among them, each target subtitle layer is used to display the display content of the subtitles to be displayed at the target time point. Different target subtitle layers display different types of display content. Superimposing multiple target subtitle layers is equivalent to superimposing different types of display content, which can generate a target subtitle image that combines different types of display content, which is equivalent to obtaining the target frame image of the subtitles to be displayed at the target time point.

[0108] Based on this, by obtaining the initial attribute values ​​of the subtitle attributes of each subtitle level at the initial time point, and obtaining the subtitle text of the subtitle to be displayed, since the subtitle level is divided based on different types of display content in the subtitle to be displayed, it is possible to perform text layout and drawing based on the subtitle text and the initial attribute values, and obtain the initial subtitle layer of the subtitle level at the initial time point, thereby realizing the division of subtitle layers for different display contents. Then, based on the target attribute value of the subtitle attribute at the target time point, the initial subtitle layer is transformed into an image to obtain the target subtitle layer of the subtitle level at the target time point, and then multiple target subtitle layers are superimposed to generate the target subtitle image of the subtitle to be displayed at the target time point. In the image transformation, the image transformation process of the corresponding initial subtitle layer is controlled by each target attribute value, so that the refined control of each subtitle level is realized, so that different types of display content in the subtitle to be displayed can be finely controlled to enrich the animation effect of the subtitle to be displayed; in addition, for multiple frame images of the subtitle to be displayed, the initial subtitle layer can be regarded as a layer in the initial frame image displayed by the subtitle to be displayed at the initial time point, and the target subtitle layer can be regarded as a layer in the target frame image displayed by the subtitle to be displayed at the target time point. In the subtitle animation rendering process, it is only necessary to perform text layout drawing when generating the initial subtitle layer, and there is no need to perform text layout drawing when generating the target subtitle layer. Instead, the target subtitle layer is obtained by image transformation based on the initial subtitle layer, which can avoid frequent text layout drawing with large computational complexity, thereby improving the rendering efficiency of the subtitle animation and improving the real-time and smoothness of the subtitle animation.

[0109] Specifically, in the subtitle animation rendering process, it is only necessary to perform an initial text layout drawing on the subtitles to obtain the initial subtitle layers corresponding to each subtitle level, and then perform image transformation on the initial subtitle layers to obtain the target subtitle layers. Finally, the target subtitle images are generated by superimposing multiple target subtitle layers, thus realizing the dynamic transformation process of converting subtitle animation into subtitle images, which can improve rendering efficiency.

[0110] In one possible implementation, an image transformation is performed on the corresponding initial subtitle layer based on the target attribute value to obtain a target subtitle layer of the subtitle level at a target time point. Specifically, the attribute change amount between the target attribute value and the corresponding initial attribute value is determined; based on the attribute change amount, an image transformation is performed on the corresponding initial subtitle layer to obtain a target subtitle layer of the subtitle level at the target time point.

[0111] Based on this, the attribute change amount is first determined according to the target attribute value and the initial attribute value, and then the corresponding initial subtitle layer is transformed based on the attribute change amount to obtain the target subtitle layer of the subtitle level at the target time point. Since the initial attribute value is the attribute value of the subtitle attribute of the subtitle level at the initial time point, and the target attribute value is the attribute value of the subtitle attribute at the target time point, the attribute change amount can be used to determine whether the attribute value of the subtitle attribute changes when the target time point is reached and the specific change amount, which is equivalent to indicating the difference between the subtitle attribute values ​​corresponding to the initial time point and the target time point through the attribute change amount. Using the attribute change amount as a reference factor for transforming the initial subtitle layer can improve the reliability of the image transformation and further improve the accuracy of the image transformation result, thereby obtaining an accurate target subtitle layer.

[0112] In one possible implementation, reference Figure 3 , Figure 3 A first optional flow chart for determining a target subtitle layer is provided for an embodiment of the present disclosure, wherein the subtitle attribute includes a color attribute, the target attribute value includes a target color value of the color attribute, the attribute change includes a color change of the color attribute, and the corresponding initial subtitle layer is subjected to image transformation based on the attribute change to obtain a target subtitle layer of the subtitle level at a target time point. Specifically, when the color change indicates that the subtitle level has undergone a color change, the color value of the non-transparent pixel point in the corresponding initial subtitle layer is transformed into the target color value to obtain the target subtitle layer of the subtitle level at the target time point.

[0113] Among them, the color attribute is used to indicate the color status of the display content of the corresponding type of the subtitle level. The color attribute can be regarded as an inline attribute that only acts on the subtitle level of the part to be displayed. The inline attribute can be recorded as , the color change is the difference between the target color value and the initial color value, the target color value is the color value of the color attribute at the target time point, and the initial color value is the color value of the color attribute at the initial time point.

[0114] Among them, the non-transparent pixels in the initial subtitle layer belong to the part that needs to be displayed, and the transparent pixels in the initial subtitle layer belong to the part that does not need to be displayed. The non-transparent pixels in the initial subtitle layer can be used to determine the corresponding type of display content in the subtitles to be displayed. For example, for the subtitle level where the subtitle text is located, the text content of the subtitle is determined by the non-transparent pixels in the initial subtitle layer.

[0115] Specifically, the value of the color attribute can be expressed as an integer. When the color change indicates that the subtitle level has undergone a color change, the color change is not 0. When the color change indicates that the subtitle level has not undergone a color change, the color change is 0. Therefore, by determining whether the color change corresponding to each subtitle level is 0, it is possible to accurately determine whether the color change indicates that the subtitle level has undergone a color change.

[0116] Based on this, image transformation includes color transformation. For any subtitle level, when the color change indicates that the subtitle level has changed color, the initial subtitle layer representing the subtitle level needs to undergo color transformation. Specifically, the color value of the non-transparent pixel in the corresponding initial subtitle layer is transformed into a target color value, which is equivalent to transforming the initial color value into a target color value to obtain the target subtitle layer of the subtitle level at the target time point. After determining the subtitle animation of the subtitle to be displayed through the initial subtitle layer and the target subtitle layer, during the display process of the subtitle to be displayed, the dynamic effect produced by the color transformation can increase the visual sense of movement, improve the efficiency of information transmission, and enhance the audience's viewing experience.

[0117] Taking the case where the image transformation only includes color transformation, the formula for determining the target subtitle layer is:

[0118]

[0119] in, For the subtitle level at the target time point The corresponding target subtitle layer, For the The initial subtitle layer corresponding to the subtitle level at the initial time point, For the The color attributes corresponding to the subtitle level are at the target time point The target color value, is a color conversion function, which is used to convert the color value of the non-transparent pixel in the initial subtitle layer into the target color value.

[0120] Furthermore, the color value of the pixel in the target subtitle layer can be expressed as:

[0121]

[0122] in, For the The pixel points in the target subtitle layer corresponding to the subtitle level At target time The color value of For the The pixel points in the initial subtitle layer corresponding to the subtitle level The color value at the initial time point, Refers to non-transparent pixels, the transparent color value is 0, For the The color attributes corresponding to the subtitle level are at the target time point The target color value.

[0123] In one possible implementation, reference Figure 4 , Figure 4 A second optional flow chart for determining a target subtitle layer is provided in an embodiment of the present disclosure, wherein subtitle attributes include geometric attributes, and attribute changes include geometric changes of the geometric attributes. Based on the attribute changes, the corresponding initial subtitle layer is subjected to image transformation to obtain a target subtitle layer of the subtitle level at a target time point. Specifically, affine transformation parameters may be determined based on the geometric changes, and an affine transformation matrix may be constructed based on the affine transformation parameters; based on the affine transformation matrix, the corresponding initial subtitle layer is subjected to affine transformation to obtain a target subtitle layer of the subtitle level at a target time point.

[0124] The geometric attributes are used to indicate the geometric state of the display content of the corresponding type of the subtitle level. For example, the geometric state may include position, size, shape, etc. The geometric attributes can be regarded as paragraph attributes that act on the entire subtitle to be displayed. The paragraph attributes can be recorded as , the geometric change is the difference between the target geometric value and the initial geometric value. The target geometric value is the geometric value of the geometric attribute at the target time point, and the initial geometric value is the geometric value of the geometric attribute at the initial time point.

[0125] Specifically, affine transformations may generally involve multiple types of geometric variations. For example, geometric attributes may include rotation attributes, scaling attributes, shearing attributes, and position attributes. Geometric variations may include rotation variations of rotation attributes, scaling variations of scaling attributes, shearing variations of shearing attributes, and movement variations of position attributes. Specifically, affine transformation parameters are first determined based on the geometric variations, and corresponding affine transformation parameters can be determined for each type of geometric variation.

[0126] Based on this, image transformation includes affine transformation. Affine transformation refers to transforming the position, size or shape of the subtitle layer. When the subtitle attributes include geometric attributes, that is, when the geometric attributes are not an empty set, the subtitle layers corresponding to all subtitle levels need to be affine transformed. An affine transformation matrix is ​​constructed according to the affine transformation parameters, and then based on the affine transformation matrix, the corresponding initial subtitle layer is affine transformed to obtain the target subtitle layer of the subtitle level at the target time point. Specifically, the target subtitle layer is obtained based on the product of the affine transformation matrix and the matrix corresponding to the initial subtitle layer. After the subtitle animation of the subtitle to be displayed is determined by the initial subtitle layer and the target subtitle layer, the dynamic effect produced by the affine transformation can increase the visual appeal during the display of the subtitles to be displayed.

[0127] Taking the image transformation including only affine transformation as an example, the formula for determining the target subtitle layer is:

[0128]

[0129] in, , For the subtitle level at the target time point The corresponding target subtitle layer, For the The initial subtitle layer corresponding to the subtitle level at the initial time point, For the The set of geometric changes of all geometric attributes corresponding to the subtitle level, For the The set of target geometry values ​​for all geometry attributes corresponding to the subtitle level. For the The set of initial geometric values ​​of all geometric attributes corresponding to the subtitle level, For the The target geometry value of the geometry attribute, For the The initial geometry values ​​of the geometry attributes, is an affine transformation function, which is used to transform the pixels in the initial subtitle layer to new coordinates to obtain the target subtitle layer. The affine transformation function can be specifically used to construct an affine transformation matrix according to the affine transformation parameters, and to perform an affine transformation on the corresponding initial subtitle layer based on the affine transformation matrix.

[0130] Furthermore, the pixels in the target subtitle layer can be expressed as:

[0131]

[0132] in, For the The pixel points in the target subtitle layer corresponding to the subtitle level At target time The color value of For the The pixel points in the initial subtitle layer corresponding to the subtitle level The color value at the initial time point is equivalent to the pixel point in the initial subtitle layer Transform to new coordinates .

[0133] Specifically, for any type of affine transformation, the coordinate transformation formula is as follows:

[0134]

[0135] in, is the coordinate of the pixel in the initial subtitle layer, is the coordinate of the pixel point in the target subtitle layer. is the reference transformation matrix, are affine transformation parameters, is the horizontal scaling factor, is the shear factor in the horizontal direction, is the horizontal translation, is the shear factor in the vertical direction, is the scaling factor in the vertical direction, is the vertical translation.

[0136] It should be noted that when the transformation type of the affine transformation is a scaling transformation, the scaling change is used to determine and ; When the transformation type of the affine transformation is a shear transformation, the shear change is determined by and ; When the transformation type of the affine transformation is a move transformation, the shear change is determined by and ; When the transformation type of the affine transformation is a rotation transformation, the rotation change is determined by , , and .

[0137] Therefore, when multiple types of affine transformations are required, it is necessary to first determine the reference transformation matrix corresponding to each type of affine transformation, and then multiply each reference transformation matrix in turn to obtain the affine transformation matrix, and finally multiply the affine transformation matrix with the matrix corresponding to the initial subtitle layer to obtain the target subtitle layer.

[0138] In a possible implementation, the subtitle attributes include a mask attribute, the target attribute value includes a target mask value of the mask attribute, and the corresponding initial subtitle layer is subjected to image transformation based on the target attribute value to obtain a target subtitle layer. Specifically, based on the target mask value, a mask area is determined in the corresponding initial subtitle layer; in the corresponding initial subtitle layer, the color value of the non-transparent pixel point in the mask area is transformed into a preset mask color value to obtain the target subtitle layer of the subtitle level at the target time point.

[0139] Among them, the mask attribute is used to indicate the mask element of the display content of the corresponding type of the subtitle level. The mask attribute can be regarded as a special attribute that does not directly act on the subtitle to be displayed. The special attribute can be recorded as , the target mask value is the mask value of the mask attribute at the target time point, the target mask value is used to determine the mask area of ​​the subtitles to be displayed at the target time point, the mask area is a specific area in the initial subtitle layer where the display effect is controlled, and the display effect controlled in the mask area can be to transform the color value of the non-transparent pixel into the mask color value, wherein the mask color value can be preset as a transparent color value or a non-transparent color value, which is not limited in the embodiments of the present disclosure.

[0140] Based on this, the image transformation includes mask transformation, which refers to the transformation of mask attributes. When the subtitle attributes include mask attributes, that is, when the mask attributes are not an empty set, the subtitle layers corresponding to all subtitle levels need to undergo mask transformation. Based on the target mask value, the mask area is determined in the corresponding initial subtitle layer, and then in the corresponding initial subtitle layer, the color value of the transparent pixel in the initial subtitle layer remains unchanged, which can ensure that the display content of the corresponding type of the subtitle level remains unchanged, and the color value of the non-transparent pixel in the mask area is transformed into the mask color value to obtain the target subtitle layer, thereby achieving accurate control of the display effect of the mask area. After the subtitle animation of the subtitle to be displayed is determined by the initial subtitle layer and the target subtitle layer, during the display process of the subtitle to be displayed, the mask transformation will cause the masked area of ​​the display content in the subtitle to be displayed to gradually increase, decrease or move. By processing part of the display content in the subtitle to be displayed through the dynamically changing mask area, the key part of the subtitle can be highlighted, thereby increasing the visual appeal, improving the efficiency of information transmission and improving the audience's viewing experience.

[0141] In one possible implementation, reference Figure 5 , Figure 5A third optional flow chart for determining a target subtitle layer is provided for an embodiment of the present disclosure, wherein the mask attribute includes a color change ratio attribute, the target mask value includes a target color change ratio value of the color change ratio attribute, and the mask area is determined in the corresponding initial subtitle layer based on the target mask value. Specifically, the layer width of the target subtitle layer is obtained, and the width threshold is determined according to the product of the target color change ratio value and the layer width; in the corresponding initial subtitle layer, the area whose horizontal coordinate is less than the width threshold is determined as the mask area.

[0142] Among them, the mask transformation includes color mask transformation, the mask attribute includes the color change ratio attribute corresponding to the color mask transformation, the target color change ratio value is the attribute value of the color change ratio attribute at the target time point, and the layer width of the target subtitle layer is specifically the distance between the left boundary of the target subtitle layer and the right boundary of the target subtitle layer.

[0143] Based on this, the width threshold is determined by multiplying the target color change ratio value by the layer width, and then the area in the initial subtitle layer whose horizontal coordinate is less than the width threshold is determined as the mask area, and the horizontal coordinate of each pixel in the mask area is less than the width threshold; when the target color change ratio value is larger, the width threshold is larger, and the mask area is larger; conversely, when the target color change ratio value is smaller, the width threshold is smaller, and the mask area is smaller. After determining the target color change ratio value of the subtitle to be displayed at the target time point, the mask area can be accurately determined by the target color change ratio value, so that the color value of the non-transparent pixel point located in the mask area is transformed into the mask color value, which is usually a non-transparent color value, and can achieve a dynamic effect of changing the color of the display content of the subtitle to be displayed word by word, thereby achieving a rich visual effect, and can further highlight the uncolored part of the subtitle to be displayed, as well as the junction between the colored part and the uncolored part, thereby increasing the visual appeal, improving the efficiency of information transmission and improving the audience's viewing experience.

[0144] Specifically, under normal circumstances, the horizontal coordinate of the pixel point on the left border in the initial subtitle layer can be set to 0, and the positive direction of the x-axis is horizontal to the right. Therefore, the horizontal coordinates of the pixel points at other positions are greater than 0, and the masked area is located to the left of the non-masked area; the horizontal coordinate of the pixel point on the left border in the initial subtitle layer can also be set to other values. Assuming that the horizontal coordinate of the pixel point on the left border is the first preset value, at this time, the first preset value can be regarded as the offset of the area, and the product of the target color change ratio value and the layer width plus the first preset value is required to obtain the width threshold.

[0145] Exemplarily, by setting the color change ratio attribute, a karaoke effect can be added to the subtitles to be displayed, that is, at a certain display time point, the first half of the subtitles to be displayed displays one color, and the second half of the subtitles to be displayed displays another color.

[0146] Taking the case where the image transformation only includes the color mask transformation, the formula for determining the pixels in the target subtitle layer is:

[0147]

[0148] in, For the subtitle level at the target time point The corresponding pixel in the target subtitle layer The color value of For the The pixel point in the initial subtitle layer corresponding to the subtitle level at the initial time point The color value of is the coordinate of the pixel point, is the mask color value, is the horizontal coordinate of the pixel point in the initial subtitle layer, is the layer width, is the target color change ratio value, is the width threshold.

[0149] In one possible implementation, reference Figure 6 , Figure 6 A fourth optional flow chart for determining a target subtitle layer is provided for an embodiment of the present disclosure, wherein a mask attribute includes a plurality of mask coordinate attributes, a target mask value includes target mask coordinates of each mask coordinate attribute, and a mask area is determined in a corresponding initial subtitle layer based on the target mask value. Specifically, a visible area may be determined in the corresponding initial subtitle layer according to each target mask coordinate; in the corresponding initial subtitle layer, an area outside the visible area is determined as a mask area.

[0150] The mask transformation includes a visibility mask transformation, the mask attribute includes a visibility mask attribute corresponding to the visibility mask transformation, the target mask coordinates are attribute values ​​of the mask coordinate attributes at a target time point, and the target mask coordinates are located at the region boundary of the visible region.

[0151] Specifically, the shape of the visible area is a rectangle, a circle, an ellipse, etc., which is not limited in the embodiments of the present disclosure. Taking the shape of the visible area as a rectangle as an example, the number of mask coordinate attributes can be two, wherein the target mask coordinate of one mask coordinate attribute is the coordinate of the upper left corner of the visible area, and the target mask coordinate of the other mask coordinate attribute is the coordinate of the upper right corner of the visible area; in addition, the number of mask coordinate attributes can also be other numbers, and the target mask coordinate is also the coordinate of other positions in the visible area, which is not limited in the embodiments of the present disclosure.

[0152] Based on this, the visible area is determined in the corresponding initial subtitle layer through each target mask coordinate, and then the area outside the visible area is determined as the mask area, so that the color value of the non-transparent pixel point in the mask area is transformed into a mask color value. The mask color value can be a transparent color value, which can achieve a dynamic effect of displaying the display content of the subtitles to be displayed word by word, thereby achieving a rich visual effect, and can further highlight the display part of the subtitles to be displayed, thereby increasing visual appeal, improving information transmission efficiency and improving the audience's viewing experience.

[0153] Taking the case where the image transformation only includes the visibility mask transformation as an example, the formula for determining the pixels in the target subtitle layer is:

[0154]

[0155] in, For the subtitle level at the target time point The corresponding pixel in the target subtitle layer The color value of For the The pixel point in the initial subtitle layer corresponding to the subtitle level at the initial time point The color value of is the coordinate of the pixel point, is the first target mask coordinate, is the second target mask coordinate. At this time, the visible area is a matrix, the first target mask coordinate is located at the upper left corner of the visible area, the second target mask coordinate is located at the lower right corner of the visible area, and the transparent color value is 0.

[0156] In one possible implementation, reference Figure 7 , Figure 7 A fifth optional flow chart for determining a target subtitle layer is provided in an embodiment of the present disclosure. Image transformation may include one or more of color transformation, affine transformation, and mask transformation. During the image transformation process, it is necessary to determine in turn whether color transformation, affine transformation, and mask transformation are required. The corresponding transformation is performed when necessary, which can improve the flexibility and reliability of image transformation.

[0157] It should be noted that when the image transformation includes multiple transformations, the above-mentioned step of determining the target subtitle layer needs to be adjusted; the following is a detailed description taking the image transformation including color transformation, affine transformation and mask transformation as an example.

[0158] First, the color values ​​of the non-transparent pixels in the corresponding initial subtitle layer are transformed into the target color values ​​to obtain the target subtitle layer of the subtitle level at the target time point, which can be adjusted as follows: the color values ​​of the non-transparent pixels in the corresponding initial subtitle layer are transformed into the target color values ​​to obtain the first subtitle layer of the subtitle level at the target time point;

[0159] Then, based on the affine transformation matrix, the corresponding initial subtitle layer is affine transformed to obtain the target subtitle layer of the subtitle level at the target time point, which can be adjusted as follows: based on the affine transformation matrix, the corresponding first subtitle layer is affine transformed to obtain the second subtitle layer of the subtitle level at the target time point;

[0160] Then, in the corresponding initial subtitle layer, the color value of the non-transparent pixel points in the mask area is transformed into the preset mask color value to obtain the target subtitle layer of the subtitle level at the target time point. It can be adjusted as follows: in the corresponding second subtitle layer, the color value of the non-transparent pixel points in the mask area is transformed into the preset mask color value to obtain the target subtitle layer of the subtitle level at the target time point.

[0161] It is worth noting that reference Figure 8 , Figure 8 An optional flow chart of a mask transformation provided for an embodiment of the present disclosure, the mask transformation may include one or more of a color mask transformation and a visibility mask transformation. During the mask transformation process, it is necessary to determine in turn whether a color mask transformation and a visibility mask transformation are required, and the corresponding transformation is performed when necessary, which can improve the flexibility and reliability of the mask transformation.

[0162] It should be noted that when the mask transformation includes multiple transformations, it is necessary to make further adjustments to the above-mentioned step of determining the target subtitle layer. The following is a detailed description taking the example that the mask transformation includes both the color mask transformation and the visibility mask transformation.

[0163] First, when the color mask is transformed, the first mask area needs to be determined based on the target color change ratio value. The mask color value corresponding to the color mask transformation can be set to the color value ; In the corresponding initial subtitle layer, the color value of the non-transparent pixel point in the mask area is transformed into the preset mask color value, and the target subtitle layer of the subtitle level at the target time point is obtained, which can be adjusted as follows: In the corresponding second subtitle layer, the color value of the non-transparent pixel point in the first mask area is transformed into the color value , get the third subtitle layer of the subtitle level at the target time point;

[0164] Then, when the visibility mask is transformed, it is necessary to determine the second mask area based on the target mask coordinate value, and the mask color value corresponding to the color mask transformation can be set to a transparent color value; in the corresponding initial subtitle layer, the color value of the non-transparent pixel point in the mask area is transformed into a preset mask color value to obtain the target subtitle layer of the subtitle level at the target time point, which can be adjusted as follows: in the corresponding third subtitle layer, the color value of the non-transparent pixel point in the second mask area is transformed into a transparent color value to obtain the target subtitle layer of the subtitle level at the target time point.

[0165] Based on this, when the image transformation includes color transformation, affine transformation, color mask transformation and visibility mask transformation at the same time, the formula for determining the first subtitle layer can be specifically:

[0166]

[0167] in, For the subtitle level at the target time point The corresponding first subtitle layer, For the The initial subtitle layer corresponding to the subtitle level at the initial time point, For the color attribute at the target time point The target color value, is a color conversion function, which is used to convert the color value of the non-transparent pixel in the initial subtitle layer into the target color value.

[0168] Furthermore, the color value of the pixel in the first subtitle layer can be specifically expressed as:

[0169]

[0170] in, For the The pixel points in the first subtitle layer corresponding to the subtitle level At target time The color value of For the The pixel points in the initial subtitle layer corresponding to the subtitle level The color value at the initial time point, Refers to non-transparent pixels, the transparent color value is 0, For the The color attributes corresponding to the subtitle level are at the target time point The target color value.

[0171] Then, the formula for determining the second subtitle layer can be specifically:

[0172]

[0173] in, , For the subtitle level at the target time point The corresponding second subtitle layer, For the subtitle level at the target time point The corresponding first subtitle layer, For the The set of geometric changes of all geometric attributes corresponding to the subtitle level, For the The set of target geometry values ​​for all geometry attributes corresponding to the subtitle level. For the The set of initial geometric values ​​of all geometric attributes corresponding to the subtitle level, For the The target geometry value of the geometry attribute, For the The initial geometry values ​​of the geometry attributes, is an affine transformation function, which is used to transform the pixel points in the first subtitle layer to new coordinates to obtain the second subtitle layer.

[0174] Furthermore, the pixel points in the second subtitle layer can be specifically expressed as:

[0175]

[0176] in, For the The pixel points in the second subtitle layer corresponding to the subtitle level At target time The color value of For the The pixel points in the first subtitle layer corresponding to the subtitle level The color value at the initial time point is equivalent to the pixel in the first subtitle layer. Transform to new coordinates The affine transformation function can be specifically used to construct an affine transformation matrix according to the affine transformation parameters, and to perform an affine transformation on the corresponding first subtitle layer based on the affine transformation matrix.

[0177] Then, the formula for determining the pixel points in the third subtitle layer can be specifically:

[0178]

[0179] in, For the subtitle level at the target time point The corresponding pixel in the third subtitle layer The color value of For the The pixel points in the second subtitle layer corresponding to the subtitle level At target time The color value of is the coordinate of the pixel point, is the mask color value, is the horizontal coordinate of the pixel point in the second subtitle layer, is the layer width, is the target color change ratio value, is the width threshold.

[0180] Then, the formula for determining the pixel points in the target subtitle layer can be specifically as follows:

[0181]

[0182] in, For the subtitle level at the target time point The corresponding pixel in the target subtitle layer The color value of For the subtitle level at the target time point The corresponding pixel in the third subtitle layer The color value of is the coordinate of the pixel point, is the first target mask coordinate, is the second target mask coordinate. At this time, the visible area is a matrix, the first target mask coordinate is located at the upper left corner of the visible area, the second target mask coordinate is located at the lower right corner of the visible area, and the transparent color value is 0.

[0183] In a possible implementation, the initial attribute values ​​of each subtitle level in the subtitles to be displayed are obtained. Specifically, a dynamic attribute set of the subtitles to be displayed is obtained, and the dynamic attribute set is divided into an equivalent attribute set and a non-equivalent attribute set; when the non-equivalent attribute set is an empty set, the initial attribute values ​​of each subtitle level in the subtitles to be displayed are obtained from the equivalent attribute set.

[0184] The dynamic attribute set includes subtitle attributes for realizing animation effects, the equivalent attribute set includes subtitle attributes for realizing animation effects through image transformation, the non-equivalent attribute set includes subtitle attributes that cannot realize animation effects through image transformation, the subtitle attributes included in the dynamic attribute set are all dynamic attributes, the subtitle attributes included in the equivalent attribute set are all equivalent attributes, the subtitle attributes included in the non-equivalent attribute set are all non-equivalent attributes, and the non-equivalent attribute set can be recorded as .

[0185] For example, reference Fig. 9 , Fig. 9 A schematic diagram of an optional grouping of a dynamic attribute set provided in an embodiment of the present disclosure.

[0186] Among them, the equivalent attribute set may include the above-mentioned color attributes, geometric attributes and mask attributes, the color attributes may include text color attributes, border color attributes, shadow color attributes and background color attributes, the geometric attributes include rotation attributes, scaling attributes, shearing attributes and position attributes, the mask attributes include color change ratio attributes and mask coordinate attributes, the non-equivalent attribute set may specifically include character spacing attributes, border width attributes, shadow distance attributes and edge blur attributes, in addition, the dynamic attribute set may also include other subtitle attributes, which are not limited in the embodiments of the present disclosure.

[0187] It is worth noting that when the color attribute of the subtitle to be displayed is an empty set, that is, the text color attribute, border color attribute, shadow color attribute and background color attribute are all empty sets, the subtitle to be displayed can be treated as a whole level, recorded as , at this time, there is no need to split the subtitles to be displayed into multiple subtitle levels, and the preset color values ​​in the preset style can be used to configure the colors of various types of display content in the subtitles to be displayed; and when the color attribute of the subtitles to be displayed is not an empty set, that is, any of the text color attribute, border color attribute, shadow color attribute, and background color attribute is not an empty set, the subtitles to be displayed need to be split into multiple subtitle levels, so the list of subtitle levels is as follows:

[0188]

[0189] in, is the level of the subtitle level, For one of the subtitle levels, For the overall level, The subtitle level corresponding to the background of the subtitle. The subtitle level corresponding to the subtitle shadow. The subtitle level corresponding to the subtitle border. The subtitle level corresponding to the subtitle text. is the color attribute, Is an empty set.

[0190] Based on this, when the non-equivalent attribute set is an empty set, the attribute value representing the non-equivalent attribute will not change. The attribute value of the non-equivalent attribute is usually set to the preset attribute value in the preset style. At this time, the initial attribute value of each subtitle level in the subtitle to be displayed is obtained from the equivalent attribute set. The initial attribute value belongs to the attribute value of the dynamic attribute. In the subsequent subtitle animation rendering process, only the dynamic effect corresponding to the equivalent attribute needs to be added. Therefore, when generating the target subtitle layer, there is no need to perform text layout and drawing. Instead, the target subtitle layer is obtained through image transformation based on the initial subtitle layer, which can avoid frequent text layout and drawing with large computational complexity, thereby improving the rendering efficiency of the subtitle animation and improving the real-time and smoothness of the subtitle animation.

[0191] In addition, since non-equivalent attributes are closely related to glyphs, when the non-equivalent attribute set is not an empty set, the attribute value representing the non-equivalent attribute will change. Since image transformation cannot effectively handle the dynamic transformation of glyphs, image transformation is no longer used in the subsequent subtitle animation rendering process. Instead, the subtitle attributes of the subtitles to be displayed at each display time point are obtained. Then, for each display time point, text layout and drawing are performed based on the subtitle text and the subtitle attributes at the display time point to obtain the real subtitle image at the display time point, which is equivalent to re-layouting and drawing each frame of the subtitle animation for the subtitles to be displayed. The real subtitle images at each display time point can be used to generate a subtitle animation with dynamically changing glyphs, thereby ensuring the quality of the subtitle animation.

[0192] Specifically, the initial attribute values ​​can be divided into attribute values ​​of dynamic attributes and attribute values ​​of static attributes. In addition to obtaining the dynamic attribute set of the subtitles to be displayed, it is also necessary to obtain the static attribute set of the subtitles to be displayed. The subtitle attributes included in the static attribute set are all static attributes. When the static attribute set is an empty set, there is no need to obtain the initial attribute values ​​of each subtitle level in the subtitles to be displayed from the static attribute set. When the static attribute set is not an empty set, it is still necessary to obtain the initial attribute values ​​of each subtitle level in the subtitles to be displayed from the static attribute set. The initial attribute values ​​belong to the attribute values ​​of the static attributes.

[0193] In a possible implementation, the subtitle attributes in the equivalent attribute set include color attributes of each subtitle level. When the non-equivalent attribute set is an empty set, the initial attribute values ​​of each subtitle level in the subtitles to be displayed are obtained from the equivalent attribute set. Specifically, when the non-equivalent attribute set is an empty set, for each subtitle level, the color attribute corresponding to the current subtitle level in the equivalent attribute set is kept unchanged, and the color attributes corresponding to the remaining subtitle levels in the equivalent attribute set are adjusted to transparent to obtain the level attribute set corresponding to the current subtitle level; and the initial attribute value of the corresponding subtitle level in the subtitles to be displayed is obtained from each level attribute set.

[0194] Among them, adjusting the color attribute to transparent is equivalent to adjusting the color value of the color attribute to a transparent color value. The color value can be represented by an integer. For example, the transparent color value can be specifically 0. Each subtitle level has a corresponding color attribute. For example, assuming that the number of subtitle levels is four, the type corresponding to the first subtitle level is the background of the subtitle, the type corresponding to the second subtitle level is the shadow of the subtitle, the type corresponding to the third subtitle level is the border of the subtitle, and the type corresponding to the fourth subtitle level is the text of the subtitle. Then the color attribute corresponding to the first subtitle level can be the background color attribute, the color attribute corresponding to the second subtitle level can be the shadow color attribute, the color attribute corresponding to the third subtitle level can be the border color attribute, and the color attribute corresponding to the fourth subtitle level can be the text color attribute.

[0195] Based on this, when the non-equivalent attribute set is an empty set, the hierarchical attribute set corresponding to each subtitle level is generated through the equivalent attribute set. Specifically, the color attribute corresponding to the current subtitle level is kept unchanged in the equivalent attribute set, and the color attributes corresponding to the remaining subtitle levels in the equivalent attribute set are adjusted to transparent, and then the initial attribute value of the subtitle level corresponding to the subtitle to be displayed is obtained from the hierarchical attribute set, so that the initial attribute values ​​of different subtitle levels are only different in the initial color values, while other attribute values ​​are the same. Subsequently, when the initial subtitle layer is generated, it can be achieved that in the initial subtitle layer of each subtitle level, only the color value of the display content of the corresponding type of the current subtitle level is retained, and the color values ​​of other types of display content are all transparent, which can ensure that the initial subtitle layer will only display the display content of the corresponding type of the current subtitle level.

[0196] Specifically, the subtitle attribute set of the subtitle to be displayed may include a static attribute set and a dynamic attribute set. The static attribute set includes subtitle attributes that cannot achieve animation effects, and the dynamic attribute set includes subtitle attributes for achieving animation effects. For example, the subtitle level The corresponding hierarchical attribute set construction formula is as follows:

[0197]

[0198] in, Subtitle level The corresponding set of hierarchical attributes, is a collection of subtitle attributes. is a set of identifiers for subtitle attributes. For all but the subtitle level A collection of color attributes corresponding to other subtitle levels except , is the identifier of the background color property, is the identifier of the border color attribute, is the identifier of the shadow color attribute, means The Color attributes, the value of the color attribute is expressed as an integer, is the transparent color value, means After adjustment color attributes, The first element in The The initial value of the color attribute is the transparent color value. The second element in The The final color value of the color attribute is the transparent color value. It refers to the time when the subtitles start to be displayed. It refers to the end time of subtitle display. The fifth element in The The color value of the color attribute remains the transparent color value during subtitle display.

[0199] It should be noted that in During the construction of , when the static attribute set is an empty set and the non-equivalent attribute set is an empty set, Specifically, it is an equivalent attribute set, which realizes that the dynamic attribute of text color is retained on the basis of the equivalent attribute set, and the background color attribute, shadow color attribute and border color attribute are all regarded as transparent static attributes, so that Excluding static attributes, the attribute value of the static attribute can be set to the default attribute value in the preset style by default; and when the static attribute set is not an empty set, the subtitle attribute set Includes static properties so that Static properties are also included.

[0200] It is worth noting that the The construction formula is similar to the construction formula for the subtitle level corresponding to the background of the subtitle Construct a hierarchical attribute set , is the subtitle level corresponding to the subtitle shadow Construct a hierarchical attribute set , the subtitle level corresponding to the subtitle border Construct a hierarchical attribute set .

[0201] In a possible implementation, the hierarchical attribute set also includes an attribute change function corresponding to the initial attribute value, and the target attribute value of each subtitle level is obtained, which can be specifically obtained by obtaining the target time point; the target time point is input into the attribute change function in each hierarchical attribute set for calculation to obtain the target attribute value of each subtitle level.

[0202] Among them, each subtitle attribute in the hierarchical attribute set has a corresponding attribute change function, which refers to a transition curve expression of the subtitle attribute change. The independent variable of the attribute change function is the time point, and the dependent variable of the attribute change function is the attribute value. For example, the attribute change function can be a linear function, a quadratic function, or an exponential function, etc., which is not limited in the embodiment of the present disclosure. The formula for determining the target attribute value is as follows:

[0203]

[0204] in, For the The set of target attribute values ​​of all subtitle attributes corresponding to the subtitle level, for Middle subtitle attributes at the target time point The target attribute value of for Middle The attribute change function of the subtitle attribute, for The subtitle attributes, For the A collection of all subtitle attributes corresponding to a subtitle level.

[0205] Specifically, the attribute change function may be located within a multi-tuple corresponding to the subtitle attribute. The attribute change function may be determined in a variety of ways. For example, relevant personnel may directly input a specific attribute change function. For another example, relevant personnel may input multiple sample data combinations, wherein the sample data combination includes a sample attribute value and a sample time point corresponding to the sample attribute value, and the sample time point is a display time point of the subtitle. Interpolation operations are then performed based on the multiple sample data combinations to determine the attribute change function. The disclosed embodiments do not limit the method for determining the attribute change function.

[0206] Based on this, since each subtitle attribute in the hierarchical attribute set has a corresponding attribute change function, by inputting the target time point into the attribute change function for calculation, the accurate target attribute value of each subtitle attribute can be obtained, thereby improving the reliability of the image transformation process.

[0207] In one possible implementation, reference Fig.10 , Fig.10 An optional flow chart for determining target attribute values ​​provided in an embodiment of the present disclosure, wherein target time points are respectively input into attribute change functions in attribute sets of each level for calculation, to obtain target attribute values ​​of each subtitle level. Specifically, target time points can be respectively input into attribute change functions in attribute sets of each level for calculation, to obtain reference attribute values ​​of each subtitle level; subtitle text is input into a large language model for emotion recognition, to obtain target emotion information of the subtitle text; target emotion information and reference attribute values ​​are spliced ​​and input into a regression model for regression, to obtain target attribute values ​​of each subtitle level.

[0208] Among them, the target emotional information can represent the emotional atmosphere of the subtitle text. For example, the emotional atmosphere may include happiness, sadness, disappointment, and surprise, etc. The target emotional information can specifically be one of multiple emotional identifiers. Different emotional identifiers are used to indicate corresponding emotional information. For example, emotional identifiers may include 1, 2, and 3, etc.

[0209] Among them, the Large Language Model (LLM) is a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text; the large language model generally uses a recurrent neural network (RNN) or its variants, such as a long short-term memory network (LSTM) and a gated recurrent unit (GRU), to capture contextual information in text sequences, thereby achieving tasks such as natural language text generation, language model evaluation, text classification, and sentiment analysis; in the field of natural language processing, large language models have been widely used, such as speech recognition, machine translation, automatic summarization, dialogue systems, intelligent question and answer, etc.

[0210] Specifically, a large language model can be trained using multiple training samples related to the emotion recognition task, so that the large language model can learn to predict the emotion of the input text, thereby improving the prediction accuracy of the target emotion information in the reasoning stage. A regression model can be trained using multiple training samples related to the attribute value prediction task, so that the regression model can learn the connection between input and output, thereby improving the prediction accuracy of the target attribute value in the reasoning stage.

[0211] Based on this, the target time point is input into the attribute change function for calculation, and the reference attribute value of each subtitle attribute can be obtained. Then the subtitle text is input into the large language model, and the target emotional information of the subtitle text is predicted by the large language model; then the target emotional information and the reference attribute value are concatenated and input into the regression model for regression. The regression model can predict the target attribute value of each subtitle level, which is equivalent to optimizing the attribute value of the subtitle attribute through the target emotional information of the subtitle text, so that the target attribute value can be more in line with the emotional atmosphere of the subtitle text, which can further enhance the display effect and expressiveness of the subtitle.

[0212] Exemplarily, when the emotional atmosphere is happy, under the action of the regression model, the color value in the target attribute value is a color value indicating warm tones; when the emotional atmosphere is sad, under the action of the regression model, the color value in the target attribute value is a color value indicating cold tones; when the emotional atmosphere is surprised, under the action of the regression model, the target attribute value is obtained by adjusting the geometric value in the reference attribute value, so that the geometric change amount is increased, which can highlight the display content of the subtitles to be displayed.

[0213] In one possible implementation, multiple subtitle levels include a text level, a border level, a shadow level, and a background level. Multiple target subtitle layers are superimposed to generate a target subtitle image of the subtitles to be displayed at a target time point. Specifically, multiple target subtitle layers can be superimposed in sequence according to the order of background level, shadow level, border level, and text level to generate a target subtitle image of the subtitles to be displayed at the target time point.

[0214] Specifically, the order of the background level, the shadow level, the border level and the text level is the superposition order, and multiple target subtitle layers are superimposed in sequence according to the order of the background level, the shadow level, the border level and the text level, so that multiple target subtitle layers can be superimposed in sequence from bottom to top according to the superposition order;

[0215] Among them, the overlay order is used to indicate the display priority of each target subtitle layer. Since the display priority of the target subtitle layer located above is higher than that of the target subtitle layer located below, the target subtitle layer located above will cover the target subtitle layer located below. If the target subtitle layer located above is not completely transparent, the target subtitle layer located above can partially or completely cover the target subtitle layer located below.

[0216] Based on this, the display content of the subtitles to be displayed may include the text, borders, shadows and background of the subtitles. The display content corresponding to the text level is the text of the subtitles, the display content corresponding to the border level is the border of the subtitles, the display content corresponding to the shadow level is the shadow of the subtitles, and the display content corresponding to the background level is the background of the subtitles. Since among the various types of display content of the subtitle image, the display priority of the subtitle text is greater than the display priority of the subtitle border, the display priority of the subtitle border is greater than the display priority of the subtitle shadow, and the display priority of the subtitle shadow is greater than the display priority of the subtitle background, it is necessary to superimpose them in the order of the background level, the shadow level, the border level and the text level, so that each type of display content can be appropriately displayed, thereby achieving the expected subtitle visual effect and improving the readability of the subtitles to be displayed.

[0217] It can be seen that the subtitle image method provided by the embodiments of the present disclosure can be applied to a variety of scenarios.

[0218] For example, in a video processing scenario, a video to be processed and subtitle information of the video to be processed are obtained, and then the subtitle information is processed by the subtitle image method provided by the embodiment of the present disclosure, which can efficiently generate target subtitle images for each video frame in the video to be processed, and then each target subtitle image is added to the corresponding video frame to obtain the target video, which can effectively improve the efficiency of adding subtitles to the video.

[0219] For another example, in an audio processing scenario, subtitle information of the audio to be processed is obtained, and then the subtitle information is processed by the subtitle image method provided by the embodiment of the present disclosure, which can efficiently generate target subtitle images for each playback time point in the audio to be processed. Each target subtitle image can then be used as a video frame at the corresponding playback time point, and each target subtitle image can be combined in the order of the playback time points to generate a subtitle video, which can effectively improve the generation efficiency of the subtitle video. The subtitle video can provide text information to groups with limited hearing or who cannot use audio.

[0220] The complete process of the subtitle image generation method is described in detail below.

[0221] Reference Fig.11 , Fig.11A schematic diagram of an optional architecture of a subtitle image generation method provided in an embodiment of the present disclosure.

[0222] First, obtain the subtitle information of the subtitles to be displayed, and determine whether the subtitle attributes of the subtitle information include dynamic attributes. When the subtitle attributes of the subtitle information do not include dynamic attributes, text layout and drawing can be performed according to the subtitle information to obtain a static image, and then the static image is used as the subtitle image of the subtitles to be displayed, and the subtitle image is output.

[0223] Then, when the subtitle attributes of the subtitle information include dynamic attributes, a dynamic attribute set of the subtitle to be displayed can be obtained from the subtitle information, and the dynamic attribute set can be divided into an equivalent attribute set and a non-equivalent attribute set, wherein the equivalent attribute set includes subtitle attributes for realizing animation effects through image transformation, and the non-equivalent attribute set includes subtitle attributes for which animation effects cannot be realized through image transformation;

[0224] Then, when the non-equivalent attribute set is an empty set, for each subtitle level, the color attribute corresponding to the current subtitle level in the equivalent attribute set is kept unchanged, and the color attributes corresponding to the remaining subtitle levels in the equivalent attribute set are adjusted to transparent, so as to obtain the level attribute set corresponding to the current subtitle level;

[0225] Then, the initial attribute value of the corresponding subtitle level in the subtitle to be displayed is obtained from each level attribute set, wherein the subtitle level is divided based on different types of display content in the subtitle to be displayed, and the initial attribute value is the attribute value of the subtitle attribute of the subtitle level at the initial time point. The multiple subtitle levels include a text level, a border level, a shadow level, and a background level, so as to realize the subtitle level splitting of the subtitle to be displayed;

[0226] Then, the subtitle text of the subtitle to be displayed is obtained, and for each subtitle level, text layout and drawing are performed based on the subtitle text and the initial attribute value to obtain the initial subtitle layer of the subtitle level at the initial time point, thereby realizing the initial frame layout and drawing.

[0227] Then, the target time point is obtained; the target time point is input into the attribute change function in each level attribute set for calculation, and the target attribute value of each subtitle level is obtained, thereby realizing dynamic attribute interpolation.

[0228] Then, the subtitle image transformation is performed, which can be divided into the following transformation cases:

[0229] When the subtitle attribute includes a color attribute, the target attribute value includes a target color value of the color attribute, and the attribute change includes a color change of the color attribute, when the color change indicates that a color change occurs at the subtitle level, the color value of the non-transparent pixel in the corresponding initial subtitle layer is transformed into the target color value to obtain a target subtitle layer of the subtitle level at a target time point, wherein the target time point is a time point after the initial time point.

[0230] Alternatively, when the subtitle attributes include geometric attributes and the attribute changes include geometric changes of the geometric attributes, the affine transformation parameters are determined based on the geometric changes, and an affine transformation matrix is ​​constructed according to the affine transformation parameters; based on the affine transformation matrix, the corresponding initial subtitle layer is affine transformed to obtain the target subtitle layer of the subtitle level at the target time point.

[0231] Alternatively, when the mask attribute includes a color change ratio attribute and the target mask value includes a target color change ratio value of the color change ratio attribute, the layer width of the target subtitle layer is obtained, and the width threshold is determined according to the product of the target color change ratio value and the layer width; in the corresponding initial subtitle layer, the area whose horizontal coordinate is less than the width threshold is determined as the mask area; in the corresponding initial subtitle layer, the color value of the non-transparent pixel point in the mask area is transformed into a preset mask color value to obtain the target subtitle layer of the subtitle level at the target time point.

[0232] Alternatively, when the mask attribute includes multiple mask coordinate attributes and the target mask value includes the target mask coordinates of each mask coordinate attribute, the visible area is determined in the corresponding initial subtitle layer according to each target mask coordinate, wherein the target mask coordinate is located at the area boundary of the visible area; in the corresponding initial subtitle layer, the area outside the visible area is determined as the mask area; in the corresponding initial subtitle layer, the color value of the non-transparent pixel point in the mask area is transformed into a preset mask color value to obtain the target subtitle layer of the subtitle level at the target time point.

[0233] Then, multiple target subtitle layers are superimposed in sequence according to the order of background layer, shadow layer, border layer and text layer to generate a target subtitle image of the subtitles to be displayed at the target time point, thus realizing the fusion of subtitle layers; finally, the target subtitle image is output.

[0234] Based on this, by obtaining the initial attribute values ​​of the subtitle attributes of each subtitle level at the initial time point, and obtaining the subtitle text of the subtitle to be displayed, since the subtitle level is divided based on different types of display content in the subtitle to be displayed, it is possible to perform text layout and drawing based on the subtitle text and the initial attribute values, and obtain the initial subtitle layer of the subtitle level at the initial time point, thereby realizing the division of subtitle layers for different display contents. Then, based on the target attribute value of the subtitle attribute at the target time point, the initial subtitle layer is transformed into an image to obtain the target subtitle layer of the subtitle level at the target time point, and then multiple target subtitle layers are superimposed to generate the target subtitle image of the subtitle to be displayed at the target time point. In the image transformation, the image transformation process of the corresponding initial subtitle layer is controlled by each target attribute value, so that the refined control of each subtitle level is realized, so that different types of display content in the subtitle to be displayed can be finely controlled to enrich the animation effect of the subtitle to be displayed; in addition, for multiple frame images of the subtitle to be displayed, the initial subtitle layer can be regarded as a layer in the initial frame image displayed by the subtitle to be displayed at the initial time point, and the target subtitle layer can be regarded as a layer in the target frame image displayed by the subtitle to be displayed at the target time point. In the subtitle animation rendering process, it is only necessary to perform text layout drawing when generating the initial subtitle layer, and there is no need to perform text layout drawing when generating the target subtitle layer. Instead, the target subtitle layer is obtained by image transformation based on the initial subtitle layer, which can avoid frequent text layout drawing with large computational complexity, thereby improving the rendering efficiency of the subtitle animation and improving the real-time and smoothness of the subtitle animation.

[0235] It is to be understood that, although the steps in the above-mentioned various flow charts are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in the present embodiment, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above-mentioned flow charts can include a plurality of steps or a plurality of stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0236] Reference Fig.12 , Fig.12 This is an optional structural diagram of a subtitle image generating device provided in an embodiment of the present disclosure. The subtitle image generating device 1200 includes:

[0237] The acquisition module 1201 is used to acquire the initial attribute value of each subtitle level in the subtitle to be displayed, wherein the subtitle level is divided based on different types of display content in the subtitle to be displayed, and the initial attribute value is the attribute value of the subtitle attribute of the subtitle level at an initial time point;

[0238] The typesetting and drawing module 1202 is used to obtain the subtitle text of the subtitle to be displayed, and for each subtitle level, perform text typesetting and drawing based on the subtitle text and the initial attribute value to obtain an initial subtitle layer of the subtitle level at an initial time point;

[0239] The image transformation module 1203 is used to obtain the target attribute value of each subtitle level, and perform image transformation on the corresponding initial subtitle layer based on the target attribute value to obtain the target subtitle layer of the subtitle level at the target time point, wherein the target attribute value is the attribute value of the subtitle attribute at the target time point, and the target time point is a time point after the initial time point;

[0240] The image generation module 1204 is used to superimpose multiple target subtitle layers to generate a target subtitle image of the subtitles to be displayed at a target time point.

[0241] Furthermore, the image transformation module 1203 is specifically used for:

[0242] Determine the attribute change between the target attribute value and the corresponding initial attribute value;

[0243] Based on the attribute change amount, the corresponding initial subtitle layer is transformed to obtain a target subtitle layer at the target time point.

[0244] Further, the subtitle attribute includes a color attribute, the target attribute value includes a target color value of the color attribute, and the attribute change includes a color change of the color attribute. The above-mentioned image transformation module 1203 is specifically used for:

[0245] When the color change amount indicates that the subtitle level has a color change, the color value of the non-transparent pixel in the corresponding initial subtitle layer is transformed into a target color value to obtain a target subtitle layer of the subtitle level at a target time point.

[0246] Furthermore, the subtitle attributes include geometric attributes, and the attribute changes include geometric changes of the geometric attributes. The above-mentioned image transformation module 1203 is specifically used for:

[0247] Determine affine transformation parameters based on the geometric variation, and construct an affine transformation matrix according to the affine transformation parameters;

[0248] Based on the affine transformation matrix, the corresponding initial subtitle layer is affine transformed to obtain the target subtitle layer of the subtitle level at the target time point.

[0249] Further, the subtitle attribute includes a mask attribute, and the target attribute value includes a target mask value of the mask attribute. The above-mentioned image transformation module 1203 is specifically used for:

[0250] Based on the target mask value, determine the mask area in the corresponding initial subtitle layer;

[0251] In the corresponding initial subtitle layer, the color values ​​of the non-transparent pixels in the mask area are transformed into preset mask color values ​​to obtain a target subtitle layer of the subtitle level at a target time point.

[0252] Further, the mask attribute includes a color change ratio attribute, and the target mask value includes a target color change ratio value of the color change ratio attribute. The above-mentioned image transformation module 1203 is specifically used for:

[0253] Get the layer width of the target subtitle layer, and determine the width threshold according to the product of the target color change ratio value and the layer width;

[0254] In the corresponding initial subtitle layer, the area whose horizontal coordinate is less than the width threshold is determined as the mask area.

[0255] Furthermore, the mask attribute includes a plurality of mask coordinate attributes, and the target mask value includes the target mask coordinates of each mask coordinate attribute. The above-mentioned image transformation module 1203 is specifically used for:

[0256] Determine the visible area in the corresponding initial subtitle layer according to each target mask coordinate, wherein the target mask coordinate is located at the region boundary of the visible area;

[0257] In the corresponding initial subtitle layer, an area outside the visible area is determined as a mask area.

[0258] Furthermore, the acquisition module 1201 is specifically used for:

[0259] Acquire a dynamic attribute set of the subtitles to be displayed, and divide the dynamic attribute set into an equivalent attribute set and a non-equivalent attribute set, wherein the equivalent attribute set includes subtitle attributes for realizing animation effects through image transformation, and the non-equivalent attribute set includes subtitle attributes for which animation effects cannot be realized through image transformation;

[0260] When the non-equivalent attribute set is an empty set, the initial attribute values ​​of each subtitle level in the subtitle to be displayed are obtained from the equivalent attribute set.

[0261] Furthermore, the subtitle attributes in the equivalent attribute set include color attributes of each subtitle level. The acquisition module 1201 is specifically used for:

[0262] When the non-equivalent attribute set is an empty set, for each subtitle level, the color attribute corresponding to the current subtitle level in the equivalent attribute set is kept unchanged, and the color attributes corresponding to the remaining subtitle levels in the equivalent attribute set are adjusted to transparent, so as to obtain the level attribute set corresponding to the current subtitle level;

[0263] The initial attribute value of the corresponding subtitle level in the subtitle to be displayed is obtained from each level attribute set respectively.

[0264] Furthermore, the hierarchical attribute set also includes an attribute change function corresponding to the initial attribute value. The above-mentioned image transformation module 1203 is specifically used for:

[0265] Get the target time point;

[0266] The target time point is input into the attribute change function in each level attribute set for calculation to obtain the target attribute value of each subtitle level.

[0267] Furthermore, the image transformation module 1203 is specifically used for:

[0268] Input the target time point into the attribute change function in each level attribute set for calculation, and obtain the reference attribute value of each subtitle level;

[0269] Input the subtitle text into the large language model for emotion recognition to obtain the target emotion information of the subtitle text;

[0270] The target emotional information and the reference attribute values ​​are concatenated and input into the regression model for regression to obtain the target attribute values ​​of each subtitle level.

[0271] Furthermore, the multiple subtitle levels include a text level, a border level, a shadow level, and a background level. The above-mentioned image generation module 1204 is specifically used for:

[0272] Multiple target subtitle layers are sequentially superimposed in the order of background layer, shadow layer, border layer and text layer to generate a target subtitle image of the subtitle to be displayed at a target time point.

[0273] The above-mentioned subtitle image generating device 1200 and the subtitle image generating method are based on the same inventive concept, by obtaining the initial attribute values ​​of the subtitle attributes of each subtitle level at the initial time point, and obtaining the subtitle text of the subtitle to be displayed. Since the subtitle level is divided based on different types of display content in the subtitle to be displayed, it is possible to perform text layout and drawing based on the subtitle text and the initial attribute value, and obtain the initial subtitle layer of the subtitle level at the initial time point, thereby realizing the division of subtitle layers for different display contents. Then, based on the target attribute value of the subtitle attribute at the target time point, the initial subtitle layer is transformed into an image to obtain the target subtitle layer of the subtitle level at the target time point, and then multiple target subtitle layers are superimposed to generate the target subtitle image of the subtitle to be displayed at the target time point. In the image transformation, the image transformation process of the corresponding initial subtitle layer is controlled by each target attribute value, so that the refined control of each subtitle level is realized, so that different types of display content in the subtitle to be displayed can be finely controlled to enrich the animation effect of the subtitle to be displayed; in addition, for multiple frame images of the subtitle to be displayed, the initial subtitle layer can be regarded as a layer in the initial frame image displayed by the subtitle to be displayed at the initial time point, and the target subtitle layer can be regarded as a layer in the target frame image displayed by the subtitle to be displayed at the target time point. In the subtitle animation rendering process, it is only necessary to perform text layout drawing when generating the initial subtitle layer, and there is no need to perform text layout drawing when generating the target subtitle layer. Instead, the target subtitle layer is obtained by image transformation based on the initial subtitle layer, which can avoid frequent text layout drawing with large computational complexity, thereby improving the rendering efficiency of the subtitle animation and improving the real-time and smoothness of the subtitle animation.

[0274] The electronic device for executing the above-mentioned subtitle image generation method provided by the embodiment of the present disclosure may be a terminal. Fig.13 , Fig.13 This is a partial structural block diagram of a terminal provided in an embodiment of the present disclosure, and the terminal includes: a camera assembly 1310, a first memory 1320, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (Wi-Fi) module 1370, a first processor 1380, and a first power supply 1390. Those skilled in the art can understand that Fig.13 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0275] The camera assembly 1310 can be used to capture images or videos. Optionally, the camera assembly 1310 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and the VR (Virtual Reality) shooting function or other fusion shooting functions.

[0276] The first memory 1320 may be used to store software programs and modules. The first processor 1380 executes various functional applications and data processing of the terminal by running the software programs and modules stored in the first memory 1320 .

[0277] The input unit 1330 may be used to receive input digital or character information and generate key signal input related to the terminal's settings and function control. Specifically, the input unit 1330 may include a touch panel 1331 and other input devices 1332 .

[0278] The display unit 1340 may be used to display input information or provided information and various menus of the terminal. The display unit 1340 may include a display panel 1341 .

[0279] The audio circuit 1360 , the speaker 1361 , and the microphone 1362 may provide an audio interface.

[0280] The first power source 1390 may be alternating current, direct current, a disposable battery, or a rechargeable battery.

[0281] The number of sensors 1350 may be one or more, and the one or more sensors 1350 include but are not limited to: acceleration sensors, gyroscope sensors, pressure sensors, optical sensors, etc. Among them:

[0282] The acceleration sensor can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor can be used to detect the components of gravity acceleration on the three coordinate axes. The first processor 1380 can control the display unit 1340 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor. The acceleration sensor can also be used for collecting game or user motion data.

[0283] The gyroscope sensor can detect the body direction and rotation angle of the terminal, and the gyroscope sensor can cooperate with the acceleration sensor to collect the user's 3D actions on the terminal. The first processor 1380 can implement the following functions based on the data collected by the gyroscope sensor: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0284] The pressure sensor can be set in the side frame of the terminal and / or the lower layer of the display unit 1340. When the pressure sensor is set in the side frame of the terminal, the user's holding signal of the terminal can be detected, and the first processor 1380 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor. When the pressure sensor is set in the lower layer of the display unit 1340, the first processor 1380 controls the operability controls on the UI interface according to the user's pressure operation on the display unit 1340. The operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0285] The optical sensor is used to collect the ambient light intensity. In one embodiment, the first processor 1380 can control the display brightness of the display unit 1340 according to the ambient light intensity collected by the optical sensor. Specifically, when the ambient light intensity is high, the display brightness of the display unit 1340 is increased; when the ambient light intensity is low, the display brightness of the display unit 1340 is decreased. In another embodiment, the first processor 1380 can also dynamically adjust the shooting parameters of the camera assembly 1310 according to the ambient light intensity collected by the optical sensor.

[0286] In this embodiment, the first processor 1380 included in the terminal can execute the subtitle image generating method of the previous embodiment.

[0287] The electronic device for executing the above-mentioned subtitle image generation method provided in the embodiment of the present disclosure may also be a server. Fig.14 , Fig.14 Partial structural block diagram of a server provided in an embodiment of the present disclosure. The server may have relatively large differences due to different configurations or performances, and may include one or more second processors 1410 and a second memory 1430, and one or more storage media 1440 (e.g., one or more mass storage devices) storing application programs 1443 or data 1442. Among them, the second memory 1430 and the storage medium 1440 may be short-term storage or persistent storage. The program stored in the storage medium 1440 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the second processor 1410 may be configured to communicate with the storage medium 1440 to execute a series of instruction operations in the storage medium 1440 on the server.

[0288] The server may also include one or more second power supplies 1420, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1460, and / or one or more operating systems 1441, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0289] The second processor 1410 in the server may be configured to execute the subtitle image generating method.

[0290] The embodiments of the present disclosure further provide a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the subtitle image generation method of each of the aforementioned embodiments.

[0291] The embodiment of the present disclosure also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device implements the above-mentioned subtitle image generation method.

[0292] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0293] It should be understood that in the present disclosure, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0294] It should be understood that in the description of the embodiments of the present disclosure, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood as not including the number, and above, below, within, etc. are understood as including the number.

[0295] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0296] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0297] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0298] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0299] It should also be understood that the various implementations provided in the embodiments of the present disclosure can be combined arbitrarily to achieve different technical effects.

[0300] The above is a specific description of the preferred implementation of the present disclosure, but the present disclosure is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions under the shared conditions without violating the spirit of the present disclosure. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present disclosure.

Claims

1. A method for generating a subtitle image, characterized in that: include: Acquire initial attribute values ​​of each subtitle level in the subtitle to be displayed, wherein the subtitle level is divided based on different types of display content in the same subtitle to be displayed, and the different types of display content include subtitle text of the subtitle to be displayed and display content for decorating the subtitle text, and the initial attribute value is the attribute value of the subtitle attribute of the subtitle level at the initial time point; Acquire the subtitle text, and for each of the subtitle levels, perform text layout and drawing based on the subtitle text and the initial attribute value to obtain an initial subtitle layer of the subtitle level at the initial time point; Obtaining target attribute values ​​of each of the subtitle levels, performing image transformation on the corresponding initial subtitle layer based on the target attribute values, and obtaining a target subtitle layer of the subtitle level at a target time point, wherein the target attribute value is an attribute value of the subtitle attribute at the target time point, and the target time point is a time point after the initial time point; The multiple target subtitle layers are superimposed to generate a target subtitle image of the subtitle to be displayed at the target time point.

2. The subtitle image generation method according to claim 1, characterized in that: The step of performing image transformation on the corresponding initial subtitle layer based on the target attribute value to obtain a target subtitle layer of the subtitle level at a target time point includes: Determining an attribute change amount between the target attribute value and the corresponding initial attribute value; Based on the attribute change amount, the corresponding initial subtitle layer is subjected to image transformation to obtain a target subtitle layer of the subtitle level at a target time point.

3. The subtitle image generation method according to claim 2, characterized in that: The subtitle attribute includes a color attribute, the target attribute value includes a target color value of the color attribute, the attribute change includes a color change of the color attribute, and performing image transformation on the corresponding initial subtitle layer based on the attribute change to obtain a target subtitle layer of the subtitle level at a target time point includes: When the color change amount indicates that the subtitle level has a color change, the color value of the corresponding non-transparent pixel in the initial subtitle layer is transformed into the target color value to obtain a target subtitle layer of the subtitle level at a target time point.

4. The method for generating a subtitle image according to claim 2, wherein: The subtitle attribute includes a geometric attribute, the attribute change includes a geometric change of the geometric attribute, and performing image transformation on the corresponding initial subtitle layer based on the attribute change to obtain a target subtitle layer of the subtitle level at a target time point, including: Determine affine transformation parameters based on the geometric variation, and construct an affine transformation matrix according to the affine transformation parameters; Based on the affine transformation matrix, the corresponding initial subtitle layer is affine transformed to obtain a target subtitle layer of the subtitle level at a target time point.

5. The subtitle image generation method according to claim 1, characterized in that: The subtitle attribute includes a mask attribute, the target attribute value includes a target mask value of the mask attribute, and performing image transformation on the corresponding initial subtitle layer based on the target attribute value to obtain a target subtitle layer of the subtitle level at a target time point includes: Based on the target mask value, determining a mask area in the corresponding initial subtitle layer; In the corresponding initial subtitle layer, the color values ​​of the non-transparent pixels in the mask area are transformed into preset mask color values ​​to obtain a target subtitle layer of the subtitle level at a target time point.

6. The subtitle image generation method according to claim 5, characterized in that: The mask attribute includes a color change ratio attribute, the target mask value includes a target color change ratio value of the color change ratio attribute, and determining a mask area in the corresponding initial subtitle layer based on the target mask value includes: Obtaining the layer width of the target subtitle layer, and determining the width threshold according to the product of the target color change ratio value and the layer width; In the corresponding initial subtitle layer, an area whose horizontal coordinate is smaller than the width threshold is determined as a mask area.

7. The method for generating a subtitle image according to claim 5, wherein: The mask attribute includes a plurality of mask coordinate attributes, the target mask value includes target mask coordinates of each of the mask coordinate attributes, and determining the mask area in the corresponding initial subtitle layer based on the target mask value includes: Determining a visible area in the corresponding initial subtitle layer according to each of the target mask coordinates, wherein the target mask coordinates are located at an area boundary of the visible area; In the corresponding initial subtitle layer, an area outside the visible area is determined as a mask area.

8. The method for generating a subtitle image according to claim 1, wherein: The obtaining of the initial attribute values ​​of each subtitle level in the subtitle to be displayed includes: Acquire a dynamic attribute set of the subtitles to be displayed, and divide the dynamic attribute set into an equivalent attribute set and a non-equivalent attribute set, wherein the equivalent attribute set includes subtitle attributes for realizing animation effects through image transformation, and the non-equivalent attribute set includes subtitle attributes for which animation effects cannot be realized through image transformation; When the non-equivalent attribute set is an empty set, initial attribute values ​​of each subtitle level in the subtitle to be displayed are obtained from the equivalent attribute set.

9. The method for generating a subtitle image according to claim 8, characterized in that: The subtitle attributes in the equivalent attribute set include color attributes of each subtitle level, and when the non-equivalent attribute set is an empty set, obtaining the initial attribute value of each subtitle level in the subtitle to be displayed from the equivalent attribute set includes: When the non-equivalent attribute set is an empty set, for each of the subtitle levels, the color attribute corresponding to the current subtitle level in the equivalent attribute set is kept unchanged, and the color attributes corresponding to the remaining subtitle levels in the equivalent attribute set are adjusted to transparent, so as to obtain a level attribute set corresponding to the current subtitle level; From each of the level attribute sets, the initial attribute value of the corresponding subtitle level in the subtitle to be displayed is obtained.

10. The subtitle image generation method according to claim 9, characterized in that: The hierarchical attribute set also includes an attribute change function corresponding to the initial attribute value, and the step of obtaining the target attribute value of each subtitle hierarchical level includes: Acquire the target time point; The target time point is respectively input into the attribute change function in each of the hierarchical attribute sets for calculation, so as to obtain the target attribute value of each of the subtitle hierarchies.

11. The method for generating a subtitle image according to claim 10, wherein: The step of inputting the target time point into the attribute change function in each of the hierarchical attribute sets for calculation to obtain the target attribute value of each subtitle level includes: Inputting the target time point into the attribute change function in each of the hierarchical attribute sets respectively for calculation, to obtain reference attribute values ​​of each of the subtitle hierarchies; Inputting the subtitle text into a large language model for emotion recognition to obtain target emotion information of the subtitle text; The target emotion information and the reference attribute value are spliced ​​and input into a regression model for regression to obtain target attribute values ​​of each subtitle level.

12. The method for generating a subtitle image according to claim 1, wherein: The multiple subtitle levels include a text level, a border level, a shadow level, and a background level. The multiple target subtitle layers are superimposed to generate a target subtitle image of the subtitle to be displayed at the target time point, including: The plurality of target subtitle layers are sequentially superimposed in the order of the background level, the shadow level, the frame level and the text level to generate a target subtitle image of the subtitle to be displayed at the target time point.

13. A subtitle image generating device, characterized in that: include: An acquisition module, used to acquire initial attribute values ​​of each subtitle level in the subtitle to be displayed, wherein the subtitle level is divided based on different types of display content in the same subtitle to be displayed, and the different types of display content include subtitle text of the subtitle to be displayed and display content for decorating the subtitle text, and the initial attribute value is the attribute value of the subtitle attribute of the subtitle level at an initial time point; A typesetting and drawing module, used for obtaining the subtitle text, and performing text typesetting and drawing for each subtitle level based on the subtitle text and the initial attribute value, to obtain an initial subtitle layer of the subtitle level at the initial time point; An image transformation module, used to obtain a target attribute value of each of the subtitle levels, and perform image transformation on the corresponding initial subtitle layer based on the target attribute value to obtain a target subtitle layer of the subtitle level at a target time point, wherein the target attribute value is an attribute value of the subtitle attribute at the target time point, and the target time point is a time point after the initial time point; The image generation module is used to superimpose multiple target subtitle layers to generate a target subtitle image of the subtitle to be displayed at the target time point.

14. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the subtitle image generating method according to any one of claims 1 to 12 is implemented.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the subtitle image generating method according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the subtitle image generating method according to any one of claims 1 to 12 is implemented.

Images