Font generation and text display method, device, medium and computing device

Abstract

The embodiments of the present disclosure provide a font generation and text display method, apparatus, medium and computing device. The font generation method includes: obtaining the text content and style information of a multimedia resource, the style information is used to characterize the resource style of the multimedia resource; generating a style font matching the resource style according to the style information, and the text content is used to be displayed according to the style font. The text display method includes: obtaining the text content and style font of a multimedia resource, the style font is generated according to the style information of the multimedia resource and matches the resource style of the multimedia resource characterized by the style information; and displaying the text content according to the style font. This method can make the display effect of the text content have a high degree of fit with the resource style of the multimedia resource, presenting a text content display effect that is highly consistent with the resource style to the user, and helping to improve the user's viewing experience.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of display technology, and more specifically, embodiments of the present disclosure relate to a font generation and text display method, apparatus, medium, and computing device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] Multimedia resources usually include text content that needs to be displayed, such as lyrics of music, subtitles of videos, etc. In the process of playing the above multimedia resources, the above text content is often displayed in a fixed form.

[0004] Related technologies use background adjustment or dynamic special effects to display. Taking the lyrics display process as an example, the background or dynamic special effects are pre-set for the lyrics, and the above background is adjusted during the song playback, or the lyrics are displayed in a dynamic manner according to the above dynamic special effects, so as to present a dynamic lyrics display effect to the user. Summary of the invention

[0005] Although the above-mentioned solutions of the related technology can present flexible text display effects to a certain extent, they only achieve simple background or special effects adjustments, and different text contents are displayed in the same preset font. Therefore, the final display effect has a low degree of fit with the resource style, and it is difficult to present the style of the multimedia resource itself through the above-mentioned display effect. The viewing experience needs to be improved.

[0006] Therefore, an improved text display solution is needed to improve the fit between the text content display effect and the resource style, so that the text content display effect is more in line with the style of the multimedia resource itself.

[0007] In this context, embodiments of the present disclosure are intended to provide a font generation and text display method, apparatus, medium, and computing device.

[0008] In a first aspect of the embodiments of the present disclosure, a font generation method is provided, the method comprising:

[0009] Acquire text content and style information of a multimedia resource, wherein the style information is used to characterize a resource style of the multimedia resource;

[0010] A style font matching the resource style is generated according to the style information, and the text content is used to be displayed according to the style font.

[0011] Optionally, the acquiring of the style information of the multimedia resource includes:

[0012] The multimedia resource is input into the style recognition model to obtain the style information of the multimedia resource output by the style recognition model.

[0013] Optionally, the style recognition model is trained in the following manner:

[0014] Acquire sample resources, any of which is added with a corresponding style tag;

[0015] The sample resources and their corresponding style tags are used as training samples to train the style recognition model.

[0016] Optionally, generating a style font matching the resource style according to the style information includes:

[0017] A standard style font corresponding to the style information is determined as a style font matching the resource style.

[0018] Optionally, generating a style font matching the resource style according to the style information includes:

[0019] Determine a standard style font corresponding to the style information;

[0020] Get the user style font that matches the user's handwriting style;

[0021] A fusion style font is generated according to the standard style font and the user style font to serve as a style font matching the resource style.

[0022] Optionally, obtaining a user style font that matches the user's handwriting style includes:

[0023] Using a pre-acquired natural handwriting font as a user style font that matches the handwriting style of the user, the natural handwriting font being determined based on a font material manually written by the user; or

[0024] The pre-generated synthetic handwriting font is used as a user style font that matches the handwriting style of the user, and the synthetic handwriting font is generated according to the natural handwriting font and a basic standard style font that is not in the handwriting style.

[0025] Optionally, the using the pre-generated synthetic handwriting font as a user style font that conforms to the handwriting style of the user includes:

[0026] A synthetic handwriting font corresponding to the text content is selected from a pre-generated synthetic handwriting font library as a user style font that conforms to the handwriting style of the user, and each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the natural handwriting font and a basic standard style font of a non-handwriting style.

[0027] Optionally, the synthetic handwriting font is generated in the following manner:

[0028] The natural handwriting font and the basic standard style font are input into a handwriting font generation model to obtain the synthetic handwriting font output by the handwriting font generation model.

[0029] Optionally, the handwriting font generation model is trained in the following manner:

[0030] Selecting at least one sample style character from a plurality of handwritten standard style font libraries, and selecting at least one sample content character;

[0031] Determine the sample label words corresponding to the sample content words in each handwriting standard style font library, wherein any sample content word and each corresponding sample label word have the same font structure;

[0032] Generate training samples according to the sample style characters, sample content characters and sample label characters, wherein any training sample includes any sample content character and any corresponding sample style character; wherein the sample label of any training sample is the sample label character corresponding to any sample content character in the handwriting standard style font library to which any sample style character belongs;

[0033] The handwriting font generation model is trained using the training samples.

[0034] Optionally, the handwritten font generation model includes a style feature extraction network, a content feature extraction network, a style transfer network, and a font content matching network, and the handwritten font generation model is trained using any one of the training samples, including:

[0035] The style feature extraction network extracts the sample style feature of any sample style word and inputs it into the style transfer network, and the content feature extraction network extracts the sample content feature of any sample content word and inputs it into the style transfer network;

[0036] The font content matching network determines the structural offset between the sample predicted font output by the style transfer network and the sample label word in any one of the training samples, and inputs a feature map representing the structural offset into the style transfer network.

[0037] Optionally, the style feature extraction network inputs the sample style feature into the style transfer network, including:

[0038] The style feature extraction network inputs the feature graphs of the sample style features into multiple structural levels in the style transfer network respectively.

[0039] Optionally, generating a fusion style font according to the standard style font and the user style font includes:

[0040] respectively extracting standard style feature points in the standard style font and user style feature points in the user style font;

[0041] Identifying target user style feature points among the user style feature points that respectively match the standard style feature points;

[0042] The standard style feature points are moved toward the corresponding target user style feature points, and the font formed by the moved standard style feature points is used as the fusion style font.

[0043] Optionally, determining a standard style font corresponding to the style information includes:

[0044] According to the mapping relationship between the style information and the candidate standard style fonts, the standard style font corresponding to the style information is queried.

[0045] Optionally, the candidate standard style fonts are obtained from a preset standard style font library, and the standard style fonts include:

[0046] At least one handwriting style in a handwritten standard style font; and / or,

[0047] At least one basic standard style font that is not handwritten.

[0048] Optionally, also include:

[0049] In response to a font acquisition request initiated by the client, the style font is returned to the client, so that the client displays the text content according to the style font.

[0050] Optional,

[0051] In the case where the multimedia resource includes audio, the style information of the audio includes at least one of the following: music style, instrument information, paragraph information, rhythm information, strength information, and emotional information of the text content;

[0052] In the case where the multimedia resource includes a video, the style information of the video includes: emotional information of picture content and / or emotional information of the text content.

[0053] In a second aspect of the embodiments of the present disclosure, a text display method is provided, the method comprising:

[0054] Acquire text content and style fonts of a multimedia resource, wherein the style fonts are generated according to style information of the multimedia resource and match the resource style of the multimedia resource represented by the style information;

[0055] The text content is displayed in the style font.

[0056] Optional,

[0057] Also included: displaying a font style control in a playback interface of the multimedia resource;

[0058] The acquiring of the style font of the multimedia resource includes: acquiring the style font of the multimedia resource in response to the font style control being triggered.

[0059] Optionally, obtaining the style font of the multimedia resource includes:

[0060] receiving a style font of a multimedia resource sent by a server, wherein the style font is generated by the server according to the style information; or,

[0061] The style information of the multimedia resource is acquired, and the style font is generated according to the style information.

[0062] Optionally, the style font includes a standard style font corresponding to the style information, and the standard style font is obtained by querying according to a mapping relationship between the style information and candidate standard style fonts.

[0063] Optionally, the style font includes a fusion style font, and the step of acquiring the style font of the multimedia resource includes:

[0064] Determine a natural handwriting font from the font material manually written by the user, and submit the natural handwriting font to the server;

[0065] The fusion style font returned by the server is received, where the fusion style font is generated by the server according to the natural handwriting font and a standard style font corresponding to the style information.

[0066] Optionally, determining a natural handwritten font from a font material manually written by a user includes:

[0067] receiving or photographing an image recording a user-style font manually written by a user, and identifying the user-style font from the image; and / or,

[0068] The user-style fonts manually written by the user are collected using the screen or electronic drawing board of the terminal device.

[0069] Optionally, also include:

[0070] Acquire a synthetic handwriting font library returned by the server, wherein each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the natural handwriting font and a basic standard style font of a non-handwriting style;

[0071] The text content of other multimedia resources is displayed according to each synthetic handwriting font in the synthetic handwriting font library.

[0072] Optional,

[0073] The obtaining of the style font of the multimedia resource comprises: obtaining a plurality of style fonts matching the style of the resource;

[0074] The displaying of the text content in accordance with the style font comprises: determining a target style font from the multiple style fonts according to a preset rule, or, in response to a selection operation for the multiple style fonts, determining a selected first style font as a target style font; and displaying the text content in accordance with the target style font.

[0075] Optionally, also include:

[0076] In response to a switching operation on the target style font, determining a second style font after switching;

[0077] The text content is displayed according to the second style font.

[0078] Optionally, displaying the text content in the style font includes:

[0079] According to the playback progress of the multimedia resource, the text content corresponding to the current playback progress is displayed in the style font.

[0080] Optionally, displaying the text content in the style font includes:

[0081] Determine the style fonts corresponding to each content segment in the text content;

[0082] Display the corresponding content fragments according to different style fonts.

[0083] Optionally, displaying the text content in the style font includes:

[0084] In the case where the multimedia resource is a song, displaying the lyrics of the song in the lyrics display area of ​​the song;

[0085] In the case where the multimedia resource is a video, the subtitles, bullet screen and / or special effect text of the video are displayed in the video interface of the video.

[0086] In a third aspect of the embodiments of the present disclosure, a font generation device is provided, the method comprising:

[0087] An acquisition module, used to acquire text content and style information of a multimedia resource, wherein the style information is used to characterize the resource style of the multimedia resource;

[0088] A generating module is used to generate a style font matching the resource style according to the style information, and the text content is used to be displayed according to the style font.

[0089] Optionally, the acquisition module is further used for:

[0090] The multimedia resource is input into the style recognition model to obtain the style information of the multimedia resource output by the style recognition model.

[0091] Optionally, the method further includes a first training module, which is used to:

[0092] Acquire sample resources, any of which is added with a corresponding style tag;

[0093] The sample resources and their corresponding style tags are used as training samples to train the style recognition model.

[0094] Optionally, the generating module is further used for:

[0095] A standard style font corresponding to the style information is determined as a style font matching the resource style.

[0096] Optionally, the generating module is further used for:

[0097] Determine a standard style font corresponding to the style information;

[0098] Get the user style font that matches the user's handwriting style;

[0099] A fusion style font is generated according to the standard style font and the user style font to serve as a style font matching the resource style.

[0100] Optionally, the generating module is further used for:

[0101] Using a pre-acquired natural handwriting font as a user style font that matches the handwriting style of the user, the natural handwriting font being determined based on a font material manually written by the user; or

[0102] The pre-generated synthetic handwriting font is used as a user style font that matches the handwriting style of the user, and the synthetic handwriting font is generated according to the natural handwriting font and a basic standard style font that is not in the handwriting style.

[0103] Optionally, the generating module is further used for:

[0104] A synthetic handwriting font corresponding to the text content is selected from a pre-generated synthetic handwriting font library as a user style font that conforms to the handwriting style of the user, and each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the natural handwriting font and a basic standard style font of a non-handwriting style.

[0105] Optionally, a synthetic font generation module is also included, for:

[0106] The natural handwriting font and the basic standard style font are input into a handwriting font generation model to obtain the synthetic handwriting font output by the handwriting font generation model.

[0107] Optionally, a second training module is also included, for:

[0108] Selecting at least one sample style character from a plurality of handwritten standard style font libraries, and selecting at least one sample content character;

[0109] Determine the sample label words corresponding to the sample content words in each handwriting standard style font library, wherein any sample content word and each corresponding sample label word have the same font structure;

[0110] Generate training samples according to the sample style characters, sample content characters and sample label characters, wherein any training sample includes any sample content character and any corresponding sample style character; wherein the sample label of any training sample is the sample label character corresponding to any sample content character in the handwriting standard style font library to which any sample style character belongs;

[0111] The handwriting font generation model is trained using the training samples.

[0112] Optionally, the handwritten font generation model includes a style feature extraction network, a content feature extraction network, a style transfer network and a font content matching network, and the second training module is further used for:

[0113] The style feature extraction network extracts the sample style feature of any sample style word and inputs it into the style transfer network, and the content feature extraction network extracts the sample content feature of any sample content word and inputs it into the style transfer network;

[0114] The font content matching network determines the structural offset between the sample predicted font output by the style transfer network and the sample label word in any one of the training samples, and inputs a feature map representing the structural offset into the style transfer network.

[0115] Optionally, the second training module is further used for:

[0116] The style feature extraction network inputs the feature graphs of the sample style features into multiple structural levels in the style transfer network respectively.

[0117] Optionally, the generating module is further used for:

[0118] respectively extracting standard style feature points in the standard style font and user style feature points in the user style font;

[0119] Identifying target user style feature points among the user style feature points that respectively match the standard style feature points;

[0120] The standard style feature points are moved toward the corresponding target user style feature points, and the font formed by the moved standard style feature points is used as the fusion style font.

[0121] Optionally, the generating module is further used for:

[0122] According to the mapping relationship between the style information and the candidate standard style fonts, the standard style font corresponding to the style information is queried.

[0123] Optionally, the candidate standard style fonts are obtained from a preset standard style font library, and the standard style fonts include:

[0124] At least one handwriting style in a handwritten standard style font; and / or,

[0125] At least one basic standard style font that is not handwritten.

[0126] Optionally, also include:

[0127] The font returning module includes returning the style font to the client in response to a font acquisition request initiated by the client, so that the client can display the text content according to the style font.

[0128] Optional,

[0129] In the case where the multimedia resource includes audio, the style information of the audio includes at least one of the following: music style, instrument information, paragraph information, rhythm information, strength information, and emotional information of the text content;

[0130] In the case where the multimedia resource includes a video, the style information of the video includes: emotional information of picture content and / or emotional information of the text content.

[0131] In a fourth aspect of the embodiments of the present disclosure, a text display device is provided, and the method includes:

[0132] An acquisition module, used for acquiring text content and style fonts of multimedia resources, wherein the style fonts are generated according to style information of the multimedia resources and match the resource style of the multimedia resources represented by the style information;

[0133] A display module is used to display the text content according to the style font.

[0134] Optional,

[0135] It also includes a control display module, which is used to display font style controls in the playback interface of the multimedia resource;

[0136] The acquisition module is further used for acquiring the style font of the multimedia resource in response to the font style control being triggered.

[0137] Optionally, the acquisition module is further used for:

[0138] receiving a style font of a multimedia resource sent by a server, wherein the style font is generated by the server according to the style information; or,

[0139] The style information of the multimedia resource is acquired, and the style font is generated according to the style information.

[0140] Optionally, the style font includes a standard style font corresponding to the style information, and the standard style font is obtained by querying according to a mapping relationship between the style information and candidate standard style fonts.

[0141] Optionally, the style font includes a fusion style font, and the acquisition module is further used for:

[0142] Determine a natural handwriting font from the font material manually written by the user, and submit the natural handwriting font to the server;

[0143] The fusion style font returned by the server is received, where the fusion style font is generated by the server according to the natural handwriting font and a standard style font corresponding to the style information.

[0144] Optionally, the acquisition module is further used for:

[0145] receiving or photographing an image recording a user-style font manually written by a user, and identifying the user-style font from the image; and / or,

[0146] The user-style fonts manually written by the user are collected using the screen or electronic drawing board of the terminal device.

[0147] Optionally, also include:

[0148] A font library acquisition module, used for acquiring a synthetic handwriting font library returned by the server, wherein each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the basic standard style fonts of the natural handwriting font and non-handwriting style;

[0149] The other text display module is used to display the text content of other multimedia resources according to each synthetic handwriting font in the synthetic handwriting font library.

[0150] Optional,

[0151] The acquisition module is also used to: acquire multiple style fonts matching the resource style;

[0152] The display module is further used to: determine a target style font from the multiple style fonts according to a preset rule, or, in response to a selection operation for the multiple style fonts, determine a selected first style font as a target style font; and display the text content according to the target style font.

[0153] Optionally, also include:

[0154] A switching response module, configured to determine a second style font after switching in response to a switching operation on the target style font;

[0155] The second font display module is used to display the text content in the second style font.

[0156] Optionally, the display module is also used for:

[0157] According to the playback progress of the multimedia resource, the text content corresponding to the current playback progress is displayed in the style font.

[0158] Optionally, the display module is also used for:

[0159] Determine the style fonts corresponding to each content segment in the text content;

[0160] Display the corresponding content fragments according to different style fonts.

[0161] Optionally, the display module is also used for:

[0162] In the case where the multimedia resource is a song, displaying the lyrics of the song in the lyrics display area of ​​the song;

[0163] In the case where the multimedia resource is a video, the subtitles, bullet screen and / or special effect text of the video are displayed in the video interface of the video.

[0164] In a fifth aspect of the embodiments of the present disclosure, a medium is provided on which a computer program is stored. When the computer program is executed by a processor, the font generation method described in any embodiment of the first aspect or the text display method described in any embodiment of the second aspect is implemented.

[0165] In a sixth aspect of the embodiments of the present disclosure, there is provided a computing device, including:

[0166] processor;

[0167] a memory for storing processor-executable instructions;

[0168] The processor implements the font generation method described in any embodiment of the first aspect or the text display method described in any embodiment of the second aspect by running the executable instructions.

[0169] According to the font generation and text display method of the embodiment of the present disclosure, the server can obtain the text content of the multimedia resource and the style information used to characterize the resource style of the multimedia resource, and then generate a style font matching the resource style according to the style information. Correspondingly, the client can obtain the text content and style font of the multimedia resource, and display the text content according to the style font.

[0170] By adopting the above method, a style font that matches the resource style of the multimedia resource can be generated. It can be understood that the text content displayed according to the above style font also matches the resource style of the multimedia resource. It can be seen that this scheme generates a style font that matches the resource style of the multimedia resource, and displays the text content of the multimedia resource according to the style font, so that the display effect of the text content has a high degree of fit with the resource style of the multimedia resource, that is, the display effect of the text content is highly matched with the resource style of the multimedia resource, thereby presenting the user with a text content display effect that is highly consistent with the resource style, which helps to improve the user's text content viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0171] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0172] Figure 1 A schematic diagram of the structure of a resource consumption platform is shown schematically;

[0173] Figure 2 It is a schematic diagram of the association relationship of related concepts involved in the present disclosure;

[0174] Figure 3is a flow chart of a font generation method provided by an exemplary embodiment;

[0175] Figure 4 is a schematic diagram of a standard style font provided by an exemplary embodiment;

[0176] Figure 5 It is a schematic diagram of a mapping relationship between a standard style font and a music style provided by an exemplary embodiment;

[0177] Figure 6 is a schematic diagram of a sample handwritten standard style font provided by an exemplary embodiment;

[0178] Figure 7 is a structural schematic diagram of a handwriting font generation model provided by an exemplary embodiment;

[0179] Figure 8 is a schematic diagram of the working principle of a classifier provided by an exemplary embodiment;

[0180] Fig. 9 is a schematic diagram of a font fusion process provided by an exemplary embodiment;

[0181] Fig.10 is a flowchart of a text display method provided by an exemplary embodiment;

[0182] Fig.11 is a schematic diagram of a medium provided by an exemplary embodiment;

[0183] Fig.12 is a block diagram of a font generation device provided by an exemplary embodiment;

[0184] Fig.13 is a block diagram of a text display device provided by an exemplary embodiment;

[0185] Fig.14 is a schematic diagram of a computing device provided by an exemplary embodiment.

[0186] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0187] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0188] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0189] According to an embodiment of the present disclosure, a font generation and text display method, apparatus, medium and computing device are proposed.

[0190] In this document, any number of elements in the drawings is used for illustration rather than limitation, and any naming is used only for distinction and does not have any limiting meaning.

[0191] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY OF THE INVENTION

[0193] The inventors have discovered that, although the background adjustment or dynamic special effects scheme adopted by the related art can present a flexible text display effect to a certain extent, it only achieves a simple background or special effects adjustment, and different text contents are displayed in the same preset font. Therefore, the final display effect has a low degree of fit with the resource style, and it is difficult to present the style of the multimedia resource itself through the above-mentioned display effect. The viewing experience needs to be improved.

[0194] In order to solve the above problems, the present disclosure provides a font generation and text display method, apparatus, medium and computing device. According to the font generation and text display method of the embodiment of the present disclosure, the server can obtain the text content of the multimedia resource and the style information used to characterize the resource style of the multimedia resource, and then generate a style font matching the resource style according to the style information. Correspondingly, the client can obtain the text content and style font of the multimedia resource, and display the text content according to the style font.

[0195] By adopting the above method, a style font that matches the resource style of the multimedia resource can be generated. It can be understood that the text content displayed according to the above style font also matches the resource style of the multimedia resource. It can be seen that this scheme generates a style font that matches the resource style of the multimedia resource, and displays the text content of the multimedia resource according to the style font, so that the display effect of the text content has a high degree of fit with the resource style of the multimedia resource, that is, the display effect of the text content is highly matched with the resource style of the multimedia resource, thereby presenting the user with a text content display effect that is highly consistent with the resource style, which helps to improve the user's text content viewing experience.

[0196] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0197] Application Scenario Overview

[0198] It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0199] Figure 1 FIG. 1 is a schematic diagram of a resource consumption platform provided by an exemplary embodiment. Figure 1 As shown, the system may include a network 10, a server 11, and several electronic devices, such as a mobile phone 12, a mobile phone 13, and a mobile phone 14.

[0200] The mobile phones 12-14 are just examples of electronic devices that can be used by users. In fact, users can obviously also use electronic devices such as tablet devices, laptops, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smart watches, etc.), etc.; the client running in any of the above electronic devices (such as the client of the development client or application program described below) can be pre-installed on the electronic device so that the client can be started and run on the electronic device.

[0201] The server 11 may be a physical server including an independent host, or may be a virtual server or cloud server carried by a host cluster, etc., and one or more embodiments of the present disclosure are not limited thereto. The network 10 may include various types of wired or wireless networks.

[0202] In the resource consumption platform, the server running in the server 11 can cooperate with the corresponding client running in the mobile phones 12-14 to implement resource management and consumption solutions. For example, if the resource is music, the client can obtain the music file from the server of the music platform, and then play the audio, display the lyrics, etc., so that the user can listen to the music and view the lyrics. Alternatively, if the resource is a video, the client can obtain the video file from the server of the video platform, and then play the video, display the subtitles and / or the bullet screen, etc., so that the user can watch the video and view the subtitles and / or the bullet screen.

[0203] In this embodiment, the resource consumption platform can not only generate fonts and display text, but also serve as an integrated functional platform for many other functions, such as training style recognition models, querying standard style fonts, generating fusion style fonts, training handwriting font generation models, extracting style feature points of fonts, switching style fonts, displaying content fragments, etc., and one or more embodiments of the present disclosure are not limited to this.

[0204] In the technical solution of one or more embodiments of the present disclosure, the server or client may implement a font generation solution to generate a style font corresponding to the resource style of the multimedia resource; further, the client may implement a text display solution to display the text content of the multimedia resource in accordance with the above-mentioned style font, thereby presenting to the user a display effect in which the font of the displayed text conforms to the resource style of the current multimedia resource, thereby enhancing the user's resource viewing experience.

[0205] According to the font generation and text display method of the embodiment of the present disclosure, the server can obtain the text content of the multimedia resource and the style information used to characterize the resource style of the multimedia resource, and then generate a style font matching the resource style according to the style information. Correspondingly, the client can obtain the text content and style font of the multimedia resource, and display the text content according to the style font.

[0206] The text content of the multimedia resources described in the embodiments of the present disclosure is resource content in text form, such as lyrics, barrage, subtitles, flower characters, comments, etc., and the expression form of this type of resource content can be text. The embodiments of the present disclosure do not limit the language form of the above-mentioned text content, such as Chinese, English, Korean, Japanese and other forms. In addition, the font described in the embodiments of the present disclosure, that is, the character shape, or glyph, is the external expression form (appearance style) of the text, which is used to reflect the appearance characteristics of the text. Displaying text according to a certain font, that is, displaying text according to the glyph represented by the font, makes the appearance style of the displayed text conform to the appearance characteristics of the font.

[0207] Considering that the embodiments of the present disclosure involve many font-related concepts, in order to facilitate a clear and detailed description of the present solution, the following first combines Figure 2 The relationship diagram shown introduces the relevant concepts involved in this solution.

[0208] Style font: A font that matches the resource style of a multimedia resource and can be used to display text content. The style font described in the embodiment of the present disclosure may include two types: a standard style font and a fusion style font.

[0209] Standard style font: a standard font that already exists in the related art, and the present embodiment can use the font as a preset font. The standard style font described in the present embodiment can include two types: handwritten standard style font and basic standard style font.

[0210] Basic standard style fonts: standard fonts that are not handwritten in style, such as Songti, Kaiti, Times New Roman, Cambria, etc.

[0211] Handwritten standard fonts: handwritten standard fonts, such as New Handwriting, Zhao An, Fangzheng Xiangji Pure Steel Plate, Copperplate, Spencerian, etc.

[0212] Fusion style font: A font generated by fusing the standard style font and the user style font. This font can reflect the font features of both the standard style font and the user style font.

[0213] User style font: a font that can reflect the user's own handwriting style. The user style font described in the embodiment of the present disclosure may include two types: natural handwriting font and synthetic handwriting font.

[0214] Natural handwriting: A handwriting font that is created when the user writes the text manually.

[0215] Synthetic handwriting font: A font generated by synthesizing basic standard style fonts and natural handwriting fonts. This font can simultaneously reflect the font features of basic standard style fonts and natural handwriting fonts, that is, it incorporates the handwriting features of its own handwriting style on the basis of the font features of basic standard style fonts.

[0216] Among them, the above-mentioned handwritten standard style fonts and basic standard style fonts can constitute a standard style font library; handwritten standard style fonts can constitute a handwritten standard style font library. In addition, sample style characters, sample content characters and sample label characters can be selected from the handwritten standard style font library to form training samples for training the handwritten font generation model. Based on the handwritten font generation model, the basic standard style fonts and any user's manually written natural handwritten fonts can be used to generate the user's synthetic handwritten fonts, and these synthetic handwritten fonts can constitute the user's corresponding synthetic handwritten font library.

[0217] Exemplary Methods

[0218] It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0219] refer to Figure 3 , Figure 3 A flowchart of a font generation method according to an embodiment of the present disclosure is schematically shown. The method can be applied to a server of a resource consumption platform or a font generation device associated with the resource consumption platform. The method can include the following steps S302-S304.

[0220] Step S302: Acquire text content and style information of a multimedia resource, wherein the style information is used to characterize the resource style of the multimedia resource.

[0221] The multimedia resources described in the embodiments of the present disclosure may be any multimedia resource maintained by a resource consumption platform. Among them, the server side obtaining the text content and style information of the multimedia resource may be triggered by any client side. For example, any client side may initiate a consumption request for any multimedia resource to the resource consumption platform, so that, on the one hand, the server side of the resource consumption platform may return any multimedia resource and its text content to the client side in response to the request; on the other hand, it may start to trigger the generation of a style font for the resource. Alternatively, the server side obtaining the text content and style information of the multimedia resource may also be triggered by a preset task. For example, the background staff of the resource consumption platform may create a style font generation task for any multimedia resource, and the task may be executed regularly by the server side of the resource consumption platform. Thus, the server side may start to trigger the generation of a style font for the resource at the corresponding moment.

[0222] Among them, the server of the resource consumption platform can trigger the self-generation of the style font or call the font generation device to generate the style font, that is, the execution subject of the font generation method described in the embodiment of the present disclosure can be the server of the resource consumption platform or the font generation device associated with the server. For example, the server of the resource consumption platform can obtain the text content and style information of the multimedia resource, and use the text generation method described in the embodiment of the present disclosure to generate a style font for the resource, and then provide the style font to the client consuming the resource (such as any client that initiates the consumption request) for displaying the text content. For another example, the server of the resource consumption platform can also call the font generation device, and the latter uses the text generation method described in the embodiment of the present disclosure to generate a style font for the resource, and then provide the style font to the client consuming the resource or provide it to the client consuming the resource through the server of the resource consumption platform, so that the client uses the style font to display the text content.

[0223] In view of the fact that the resource consumption platform maintains multimedia resources and their textual contents, the font generator can obtain the multimedia resources and their textual contents from the resource consumption platform. For example, in the case where the font generator is the server of the resource consumption platform, the server can determine the multimedia resources of the style font to be generated from the locally maintained multimedia resources, and obtain the resources and their textual contents. For another example, in the case where the font generator is a font generation device associated with the resource consumption platform, the device can determine the multimedia resources of the style font to be generated from the multimedia resources maintained by the server of the resource consumption platform, and obtain the resources and their textual contents. Of course, the multimedia resources and their textual contents can also be actively sent to the latter by the server of the resource consumption platform, and the embodiments of the present disclosure are not limited to this. The following embodiments of the present disclosure are described by taking the execution subject of the font generation method (i.e., the aforementioned font generator) as the server of the resource consumption platform as an example.

[0224] After obtaining the multimedia resource for the style font to be generated, the server can further obtain the style information of the resource. Among them, the style information of the multimedia resource can be in multiple forms, and the embodiment of the present disclosure does not limit this. For example, in the case where the multimedia resource is audio, the style information of the audio may include at least one of the following: style (such as country music, jazz, rock, heavy metal and other song styles), instrument information (such as piano, guitar, violin and other instruments contained in the audio), paragraph information (such as 0-40s as the main song, 40s-50s as the interlude, 50s-1min10s as the main song, etc.), rhythm information (such as compact, loose, etc.), strength information (such as stress, light tone, etc.), emotional information of the text content (such as cheerful, melancholy, general, etc.). For another example, in the case where the multimedia resource is a video, the style information of the video may include: emotional information of the picture content, and may also include emotional information of the text content, wherein the above emotional information can be used to characterize the emotional tendency of the video. Of course, the style information may also include the above two types of emotional information at the same time, or may also be other forms of information. No more details.

[0225] The server can obtain the style information of the resource in a variety of ways. For example, the server can query the style information of the multimedia resource from the resource attribute information of the multimedia resource maintained locally. This method is simple and efficient, but may cause the query to fail due to the lack of style information, or the subsequent style fonts may not match the resource style of the multimedia resource well due to inaccurate style information.

[0226] To avoid the above defects, the server can train a style recognition model for multimedia resources (training phase), and use the model to obtain the style information of the multimedia resources (inference phase). For example, in the training phase, the server can first obtain multiple sample resources for training, and any sample resource can be added with a corresponding style tag. Furthermore, the server can use the sample resources and their corresponding style tags as training samples to train the style recognition model. It can be understood that the training of the style recognition model based on the above sample resources and style tags belongs to supervised training. In the reasoning phase, the multimedia resources can be input into the trained style recognition model to obtain the style information of the multimedia resource output by the style recognition model.

[0227] As mentioned above, the style information of any multimedia resource may be related to the resource type of the resource, and the resource may also have style information of multiple dimensions. Therefore, the server can generate a style recognition model for any style information of any type of resource, and generate corresponding style recognition models for various style information, so that for the multimedia resource, the corresponding style information can be obtained using the above-mentioned style recognition models. The following takes the style of songs maintained by the music platform as an example to describe the training stage and reasoning stage of the corresponding style recognition model.

[0228] During the training phase, the server may first collect song data of sample songs in a variety of different styles. The song data of any sample song may include the audio file and style tag of the song. Among them, the audio file may be in the format of .WAV, .MP3, .AAC, etc., and the lyrics file may be in the format of LRC, TRC, KRC, etc., which is not limited by the embodiments of the present disclosure. The styles may include pop, rock, folk, electronic, dance, rap, light music, jazz, country, R&B, classical, ethnic, British, metal, punk, blues, reggae, ancient style, etc. It should be noted that any style may include multiple sample songs, that is, any style may contain song data of multiple sample songs. At the same time, any sample song may also belong to multiple styles at the same time, that is, the sample song may have multiple style tags. The audio file of any sample song and any of its style tags may constitute a training sample. Among them, the genre label of any sample song can be added by the song consumers on the music platform when consuming the song; or, considering the uneven levels of consumers, it can also be added by professional labelers with certain music appreciation abilities, and the embodiments of the present disclosure do not limit this.

[0229] When the song data of the sample song is obtained, the server can use an appropriate audio feature extraction algorithm to extract features from the audio file of the sample song. For example, the mel spectrum extraction algorithm can be used to extract the audio feature vector of the sample song, wherein an audio feature vector can be extracted from the audio file of a sample song. Furthermore, the audio feature vector can be used as an input parameter of a style recognition model, and the model can be used to classify the audio data in combination with the global information and local information of the entire audio. Among them, the global information and local information of any sample song may include the average pitch and local pitch, average beat and local beat of the sample song, etc., which will not be repeated. Among them, the above-mentioned style recognition model can be built using the Transformer algorithm. Specifically, the attention mechanism can be used inside the model. Or the above-mentioned style recognition model may also include the aforementioned audio feature extraction algorithm, which will not be repeated.

[0230] Furthermore, the server can compare the above classification results with the genre label of the sample song, and adjust the model parameters of the style recognition model (such as the specific parameters of the Transformer algorithm) according to the deviation obtained from the comparison result, and use other sample songs to continue the next round of training for the adjusted model until the model loss corresponding to the model output result meets the preset stopping condition or the training number reaches the training condition, and the training is stopped to obtain a trained genre recognition model.

[0231] During the inference stage, the server can first obtain the audio file of the song for which the style font is to be generated, and extract the audio feature vector of the song through the aforementioned audio feature extraction algorithm, and then input the audio feature vector into the model as an input parameter of the style recognition model, and obtain the style of the song output by the model.

[0232] Through the above method, the server can train the genre recognition model and identify the genre of any song based on the trained model. Obviously, the training phase and the reasoning phase of the model are separated from each other, so the training phase can be completed in advance, that is, the genre recognition model can be pre-trained. Therefore, when the server needs to generate a style font for any song, the pre-trained genre recognition model can be directly called to identify the genre of the song. Of course, in addition to genre, the server can also pre-train a style recognition model corresponding to any style information of any other type of multimedia resource, and when it is necessary to generate a style font for any type of multimedia resource, the pre-trained style recognition model can be directly used to identify the style information of the multimedia resource.

[0233] It is understandable that the style information of multimedia resources is identified by the pre-trained style recognition model, which only requires the resource data of the multimedia resources themselves, without their related attribute information. Therefore, even in the absence of style information, the style information of multimedia resources can be identified by the model to ensure the smooth generation of subsequent style fonts. In addition, by using the style recognition model to identify the style information of multimedia resources, it is also helpful to realize the batch recognition of the style information of multimedia resources, so that the server can batch generate the style fonts of multimedia resources in the background in a silent manner, which helps to improve the efficiency of style font generation.

[0234] The style information of the multimedia resource obtained in the above manner can be used to characterize the resource style of the multimedia resource. In other words, the resource style of the multimedia resource can be reflected through its style information. The text generation method described in the embodiment of the present disclosure is used to generate a style text that matches the style of the resource (i.e., the style text of the multimedia resource), so that when the text content of the multimedia resource is displayed according to the style text, the display effect of the text content is consistent with the resource style of the resource.

[0235] Step S304: generating a style font that matches the resource style according to the style information, and the text content is used to be displayed according to the style font.

[0236] When the style information of the multimedia resource is obtained, the server can generate a style font that matches the style of the resource according to the style information. Specifically, the server can generate the style font in a variety of ways.

[0237] In one embodiment, the server may determine a standard style font corresponding to the style information of the multimedia resource as a style font matching the style of the resource. The standard style font is a font included in a font library in the related art, which can be divided into two categories: basic standard style font and handwritten standard style font. Figure 4 As shown in the figure, the basic standard style font is a standard font in a non-handwritten style, such as "Tengxiang Qinyuanjian-W2", "Level 3 Suxian Simplified", "Level 3 Jianhei Simplified-Medium", etc.; the handwritten standard style font is a standard font in a handwritten style, such as "Level 3 Xiangxiang Style", "Nuomi Tuanjian" corresponding to Nuomi Tuanjian, "Tengxiang Bubble" corresponding to Bubblejian font, etc. Of course, the above-mentioned basic standard style fonts and handwritten standard style fonts are not limited to Figure 4 The specific form shown may also be other non-handwriting or handwriting-style fonts.

[0238] The style of any of the above standard style fonts can be used to reflect the resource style of multimedia resources. For example, the strokes of "Songti" vary in thickness, and are generally thin horizontally and thick vertically, with decorative parts at the end (i.e., "character feet" or "serifs"). The strokes of dots, left strokes, right strokes, and hooks have sharp tips. They belong to serif fonts and are often used for typesetting text in books, magazines, and newspapers. Therefore, for multimedia resources with a relatively plain and formal style, Songti can be used as its style font. For another example, the Nuomi Tuan Jian font corresponding to "Nuomi Tuan Jian" is soft and cute, and can be used as the style font corresponding to nursery rhymes or children's animations, which will not be elaborated.

[0239] The server may maintain a mapping relationship between style information and candidate standard style fonts. The mapping relationship may be set by the administrator of the resource consumption platform in a background manner, or may be obtained by the server from other devices or platforms. The embodiments of the present disclosure are not limited to this. Figure 5 As shown, different music genres such as pop, rock, folk, ancient style, and rap correspond to different styles of candidate standard style fonts. Of course, any music genre can also correspond to multiple candidate standard style fonts, and each candidate standard style font can be pre-sorted according to the degree of matching with the music genre.

[0240] Based on the above mapping relationship, the server can query the standard style font corresponding to the style information of the multimedia resource, and the font is used as the style font of the multimedia resource. That is, the server can query the candidate standard style font corresponding to the style information of the multimedia resource in the mapping relationship to serve as the standard style font corresponding to the style information of the multimedia resource. Figure 5 For example, when the multimedia resource is a song and its genre is rock, the candidate standard style font "Hanyi Pili Simplified" can be used as the standard style font corresponding to the genre of the song, and then the font can be used as the style font of the song. Of course, when rock corresponds to multiple fonts, the font that matches rock the most can be directly used as the standard style font of the song; or, according to the intensity of rock in the genre of the song, a candidate standard style font with a corresponding degree of match can be selected as the standard style font of the song, which will not be repeated. This method can simply and quickly determine the style font of the multimedia resource, which helps to shorten the waiting time of users (i.e., users represented by the client) who consume the multimedia resources and enhance the user's resource consumption experience.

[0241] In addition, the candidate standard style fonts can be obtained from a preset standard style font library, and the standard style fonts can include at least one handwritten standard style font; and / or at least one non-handwritten basic standard style font. Figure 4As shown, no further details are given.

[0242] In another embodiment, in order to meet the user's personalized font display needs, the server can generate a fusion style font that can reflect the user's own handwriting style as the style font of the multimedia resource. For example, the server can determine the standard style font corresponding to the style information of the multimedia resource, and obtain the user style font that matches the user's handwriting style, and then generate a fusion style font based on the standard style font and the user style font as a style font that matches the style of the resource. Among them, the server can determine the standard style font corresponding to the style information of the multimedia resource in the manner described in the aforementioned embodiment, which will not be repeated here.

[0243] The user style font that conforms to the user's handwriting style can be a natural handwriting font or a synthetic handwriting font. For the above different handwriting fonts, the server can obtain them in different ways. For example, the server can pre-acquire a natural handwriting font and use the font as a user style font that conforms to the user's handwriting style, wherein the natural handwriting font can be determined based on the font material manually written by the user. The user can manually write the font material on the surface of non-electronic materials such as paper, and the terminal device where the client is located can collect the written font material by scanning or taking pictures; or the user can also manually write the font material on the surface of electronic devices such as electronic drawing boards and touch screens provided by the above terminal devices, and the terminal device can detect the above font material. Furthermore, after the user logs in to the resource consumption platform, the client can provide the font material collected or detected by the terminal device to the server. Thereafter, the server can extract the user's natural handwriting font based on the font material, and use the font as a user style font that conforms to the user's handwriting style. Obviously, because the above font material is manually written by the user, the user style font obtained in the above manner can fully reflect the user's own handwriting style.

[0244] For another example, the server may also use a pre-generated synthetic handwriting font as a user style font that matches the handwriting style of the user, and the synthetic handwriting font is generated based on the natural handwriting font and the basic standard style font of the non-handwriting style. Among them, the above-mentioned basic standard style font is one form of standard style font, so the basic standard style font can reflect the resource style of the multimedia resource to a certain extent, and the natural handwriting font can reflect the user's own handwriting style, so the synthetic handwriting font generated based on the natural handwriting font and the basic standard style font of the non-handwriting style can simultaneously match the resource style of the multimedia resource and the user's own handwriting style, thereby presenting a better text display effect to the user.

[0245] Among them, the server can pre-generate a synthetic handwriting font library for the user, and each synthetic handwriting font in the font library can be generated by the server based on the natural handwriting font and the basic standard style font of the non-handwriting style. Thus, the server can select the synthetic handwriting font corresponding to the text content in the pre-generated synthetic handwriting font library as the user style font that conforms to the handwriting style of the user. In this way, a synthetic handwriting font library for the user can be generated, and each synthetic handwriting font in the font library can be reused by the server multiple times, and directly obtaining synthetic handwriting fonts from the font library can also improve the efficiency of obtaining user style fonts, thereby helping to improve the efficiency of generating style fonts.

[0246] In fact, in addition to being a consumer of the multimedia resources, the above-mentioned user can also be a producer of the multimedia resources, such as the singer, copyright owner or UP host corresponding to the song. In the case where the above-mentioned user is the producer of the multimedia resources, the synthetic handwriting font can display to the user a font that conforms to the handwriting style of the producer. In addition, the above-mentioned synthetic handwriting font can also be generated according to the handwriting standard style font of the natural handwriting font and the handwriting style, which will not be repeated here.

[0247] In addition, the server can generate the synthetic handwriting font through a handwriting font generation model. For example, the server can input the natural handwriting font and the basic standard style font into the handwriting font generation model to obtain the synthetic handwriting font output by the handwriting font generation model. In this way, for any user, the server can batch generate multiple synthetic handwriting fonts corresponding to the user; and for multiple users, the server can batch generate at least one synthetic handwriting font corresponding to each user, which not only helps to reduce the user's waiting time, but also can improve the generation efficiency of style fonts to a certain extent. In particular, when multiple users consume multiple different multimedia resources in a short period of time, the above-mentioned batch generation method can simplify the processing flow of the server and ensure the generation efficiency of synthetic handwriting fonts.

[0248] The server may pre-train the handwriting font generation model in the following manner. For example, the server may select at least one sample style word and at least one sample content word from a plurality of handwriting standard style font libraries. Figure 6As shown, there are four handwritten standard style font libraries corresponding to four fonts: A "beautiful cursive script", B "natural cursive script", C "graceful script", and D "letter font", and each of the above handwritten standard style font libraries contains multiple handwritten standard style fonts of the corresponding style. At this time, the server can select at least one handwritten standard style font from each of the above handwritten standard style font libraries as a sample style character. It can be understood that the above sample style characters are used to construct training samples. Therefore, in order to make the trained handwritten font generation model have strong adaptability, the number of the handwritten standard style font libraries and the number of sample style characters selected from each handwritten standard style font library can be set appropriately more. In addition, it is worth noting that multiple sample content characters can be selected, and each sample content character and each sample style character can be different characters, and any sample content character should have corresponding characters in the handwritten standard style font library to which each sample style character belongs.

[0249] In addition, the server can determine the sample label characters corresponding to each sample content character in each handwritten standard style font library, and any sample content character and its corresponding sample label characters have the same font structure. It can be understood that each sample content character is used to represent the content of the character, such as which radicals it contains, which letters it contains and their order, etc., rather than representing the specific structure and external manifestation form of the character. The sample label character corresponding to any sample content character in any handwritten standard style font library is the same character as the sample content character (i.e., has the same content); in fact, any sample content character and its corresponding sample label characters are all the same character. Still taking Figure 6 as an example, assume that the server selects the character "新" as a certain sample content character. Then the server can select at least one sample style character from the handwritten standard style font libraries corresponding to the four fonts A, B, C, and D above, and use the character "新" contained in each handwritten standard style font library as the sample label character corresponding to each sample style character selected from this font library. For example, for the handwritten standard style font library corresponding to A "beautiful cursive script", the characters "漂", "来", "永" in it can be selected as the sample style characters corresponding to the sample content character "新", and the character "新" in this font library can be used as the sample label character corresponding to each of the three sample style characters "漂", "来", "永". The process of selecting sample style characters and sample label characters in the handwritten standard style font libraries of B, C, D and other fonts is similar to the above, and will not be elaborated one by one.

[0250] Further, the server can generate training samples based on the sample style characters, sample content characters, and sample label characters. Among them, any training sample can include any sample content character and its corresponding sample style character; the sample label of this training sample can be the sample label character corresponding to the any sample content character in the handwritten standard style font library to which the any sample style character belongs. For example, for the sample content character "新", it can form a training sample with the sample style character "来" in the handwritten standard style font library corresponding to A "beautiful handwriting style", and "新" in this font library can be used as the sample label of this training sample.

[0251] Using the training samples obtained through the above process, the server can train the handwritten font generation model. It can be understood that since the above training samples are labeled with sample labels, the training of the handwritten font generation model is a supervised training process. In this way, it can be ensured that the synthesized handwritten fonts output by the trained handwritten font generation model fully reflect the user's handwriting style. The following combines Figure 7 to describe the model structure and its training process.

[0252] As Figure 7 shown, the handwritten font generation model can include a style feature extraction network, a content feature extraction network, a style transfer network, and a content matching network. For any training sample containing a sample style character and a sample content character, on the one hand, the content feature extraction network can extract the sample content features of the any sample content character and input these features into the style transfer network. Among them, the sample content features can include features such as stroke type, intersection position, letter content, and letter order. The above sample content features can include at least one content feature vector, and these content feature vectors can form a content feature map, so that the content feature extraction network can input the content feature map into the style transfer network. On the other hand, the style feature extraction network can extract the sample style features of the sample style character in this training sample and input these features into the style transfer network. Among them, the sample style features can include features such as stroke curvature, stroke aspect ratio, and angle. The above sample style features can include at least one style feature vector, and these style feature vectors can form a style feature map, so that the style feature extraction network can input the style feature map into the style transfer network.

[0253] Among them, considering that the style transfer network usually includes at least an input layer, an output layer, and at least one intermediate layer, the style feature extraction network can input the feature map of the sample style features into multiple structural levels in the style transfer network (such as Figure 7As shown by the three arrows between the medium-style feature map and the style transfer network, this is done to avoid, as much as possible, the gradual disappearance of the sample style features as the number of levels increases during training, and to ensure the persistence of the sample style features.

[0254] Furthermore, the content matching network can determine the structural offset between the sample predicted font output by the style transfer network and the sample label character in any of the training samples, and input the feature map representing the structural offset into the style transfer network. As Figure 7 shown, for a training sample composed of the sample content character "新" and the sample style character "饭" in a handwriting style, the "新" is input into the content feature extraction network, and the "饭" is input into the style feature extraction network. The final output character of the model is the character "新" that incorporates the handwriting style corresponding to "饭".

[0255] Furthermore, it is also possible to predict, through a classifier, the corresponding handwritten standard style character in the handwritten standard style font library for the output character of the model. As Figure 8 shown, the classifier consists of a multi-classification network and an output result judgment module. The character "新" output by the handwritten font generation model can be used as the input to the multi-classification network, and the output of this network is the probability value of the character "新" being each handwritten standard style character in the handwritten standard style font library. Among them, the handwritten standard style character corresponding to the maximum probability value can be used as the handwritten standard style character corresponding to this character. For example, when the probability value of the 6th handwritten standard style character is the maximum value of 0.8, it can be determined that the character "新" output by the model is the 6th handwritten standard style character. Thus, the server can adjust the model parameters of the handwritten font generation model according to the deviation between the above maximum probability value and the sample signature character of the sample, and further use other training samples for training until it stops.

[0256] Through the above method, it is possible to use handwritten standard style fonts in multiple handwriting styles as training samples to train the handwritten font generation model. When generating a synthetic handwritten font based on the basic standard style font and the natural handwritten font using the trained handwritten font, the basic standard style font can be input into the content feature extraction model of the model, and the natural handwritten font can be input into the style feature extraction model of the model, thereby obtaining the synthetic handwritten font output by the model, which can reflect the user's handwriting style on the basis of the basic standard style.

[0257] Among them, the style transfer network and the content feature matching module can be connected in a way similar to negative feedback. As Figure 7As shown, the output of the style transfer network serves as one of the inputs to the content feature matching module, and the output of the content feature matching module serves as one of the inputs to the style transfer network. Through the above connection method, the output of the content feature matching module is a feature map containing the above structural offsets, and the style transfer network can adjust the font structure of the output characters according to the above structural offsets, such as adjusting the position of radicals, the angle of strokes, adding or deleting strokes, etc., so that the content of the output characters is consistent with the content of the input content characters - avoiding abnormal situations such as inputting character A and outputting character B (or even non-characters).

[0258] In the case where the standard style font corresponding to the style information of the multimedia resource obtained through the foregoing embodiment method is expected to conform to the user's handwritten style of the user style font, the server can fuse the two by means of feature point movement to generate a fused style font. For example, the server can respectively extract the standard style feature points in the standard style font and the user style feature points in the user style font, then identify the target user style feature points in the user style feature points that respectively match each standard style feature point, and finally move each standard style feature point toward the corresponding target user style feature point, and use the font formed by the moved standard style feature points as the fused style font.

[0259] As Fig. 9 shown in the character "大", the server can use the global affine transformation and / or local affine transformation techniques described in related technologies to extract the standard style feature points in the standard style font, as shown by the black dots in the figure; and extract the user style feature points in the user style font, as shown by the white dots in the figure. Furthermore, the server can identify the target user style feature points in the user style feature points that respectively match each standard style feature point. For example, the white dot closest to each black dot can be used as the target user style feature point corresponding to each black dot, or the target user style feature point can also be determined according to other rules. Further, each standard style feature point can be moved toward the corresponding target user style feature point. As Fig. 9 shown, assuming that the white dot Pu corresponding to the black dot Ps is the target user style feature point, then Ps can be moved to Pu (equivalent to replacing Pu with Ps), or Ps can also be moved toward Pu to the midpoint (or other proportional positions) of the two. After each standard style feature point is moved respectively, the server can generate a new font according to the moved standard style feature points to serve as the fused style font. Obviously, the fused style font generated by the above method can not only fit the resource style of the multimedia resource but also reflect the user's handwritten style.

[0260] So far, the introduction of the style font production process is completed. The style font generated based on the above embodiment can be used by the client to display the text content of the multimedia resource. For example, the service end can return the style font to the client in response to the font acquisition request initiated by the client, so that the client can display the text content according to the style font. It can be understood that, in the case where the above-mentioned user is a producer user of multimedia resources (such as a singer), the above-mentioned style text can be provided to the consumer client used by each consumer user (such as an ordinary user of a resource consumption platform) who consumes the multimedia resource, so that each consumer client can display the text content of the multimedia resource according to the style font. Of course, the above-mentioned style font can also be provided to the producer client used by the producer user, so that the producer client can display the text content that conforms to its own handwriting style to the producer user. In the case where the above-mentioned user is a consumer user of multimedia resources, the above-mentioned style font can be provided to the consumer client used by the consumer user or other consumer users, so as to display the text content of multimedia resources, which will not be repeated.

[0261] refer to Fig.10 , Fig.10 A flowchart of a text display method according to an embodiment of the present disclosure is schematically shown. The method can be applied to a client of a resource consumption platform, such as a production client used by a producer of multimedia resources, or a consumer client used by a common consumer of multimedia resources. The following embodiments of the present disclosure are described using a consumer client as an example. The method can include the following steps S1002-S1004.

[0262] Step S1002 : acquiring text content and style fonts of a multimedia resource, wherein the style fonts are generated according to style information of the multimedia resource and match the resource style of the multimedia resource represented by the style information.

[0263] In the disclosed embodiment, the client can realize the function of displaying the text content of the resource in a style font matching the multimedia resource. For the sake of brevity, this function is referred to as the style text display function below. Steps S1002-S1004 are the specific implementation process of this function.

[0264] In one embodiment, in order to fully respect the user's willingness to use, the client can set the style text display function as a user optional option, that is, the user decides whether to turn on the function. For example, the client can display a font style control in the playback interface of the multimedia resource, and the control can be triggered by the user. Furthermore, the client can obtain the style font of the multimedia resource in response to the font style control being triggered. It can be understood that the font style control is triggered by the user, which indicates that the user wants to turn on the above-mentioned style text display function, so the function can be implemented in response to the trigger, that is, the style font of the multimedia resource can be obtained. Of course, if the user does not trigger the font style control, it means that the user does not want to turn on the above-mentioned style text display function, so the text content of the multimedia resource can be displayed in a preset font, or the text font can be displayed in other ways in the relevant technology, which will not be repeated.

[0265] Of course, if the user does not trigger the font style control, it may be because the user is not aware of the above-mentioned style text display function, especially within a period of time after the function is newly launched. Therefore, in order to encourage users to try to use this function, the client can display a reminder message for the above-mentioned style text display function to the user when the font style control has never been triggered or has not been triggered for a certain period of time, so that the user knows about the function and then triggers the font style control to try to use the function.

[0266] In addition, the above-mentioned font style control can be in the form of a button, a slider, etc., and the embodiments of the present disclosure are not limited to this. Moreover, the client can also display the above-mentioned font style control in the client setting interface, so that the user's setting for the font style control can be applied to various multimedia resources displayed by the client. It can be understood that when the font style control is displayed to the user, the following embodiments of implementing the style text display function should be implemented when the control is triggered.

[0267] As mentioned above, the font generation method described in the embodiment of the present disclosure can be implemented on the server or client side. Accordingly, the client side can obtain the style font of the multimedia resource generated by the above method in different ways. For example, in the case where the style font is generated by the server side, the client side can receive the style font of the multimedia resource sent by the server side, and the style font is generated by the server side according to the style information. The process of the server side generating the style font according to the style information can be referred to in Figure 3The records of the corresponding aforementioned embodiments are not repeated here. For another example, when the style font is generated by the client, the client can obtain the style information of the multimedia resource and generate the style font according to the style information. The specific process of the client generating the style font according to the style information can also be found in the records of the aforementioned embodiments, but some of the aforementioned embodiments involving the interaction between the server and the client should not be applied to the client, which is hereby explained.

[0268] In one embodiment, the client may also maintain a mapping relationship between style information and candidate standard style fonts, and query the candidate standard style fonts corresponding to the style information of the multimedia resource based on the mapping relationship, and then use the queried font as the style font of the multimedia resource. Of course, the above-mentioned standard style fonts may also be queried by the server and sent to the client. It can be seen that the style font may include a standard style font corresponding to the style information of the multimedia resource, and the standard style font may be queried based on the mapping relationship between the style information and the candidate standard style fonts.

[0269] In another embodiment, the style font may include a fusion style font, so that the server can determine the natural handwriting font from the font material manually written by the user, and submit the natural handwriting font to the server, so that the server generates a fusion style font according to the natural handwriting font and the standard style font corresponding to the style information, and then can receive the fusion style font returned by the server. Among them, the client can receive or shoot an image of a user style font manually written by the user, and identify the user style font from the image. Taking shooting as an example, the user can manually write font materials on the surface of non-electronic materials such as paper, so that the client can drive the terminal device where it is located to collect the written font materials by scanning or taking pictures. Alternatively, the client can also use the screen or electronic drawing board of the terminal device to collect the user style font manually written by the user. Taking the screen as an example, the user can manually write font materials on the surface of the above screen, so that the client can drive the terminal device to detect the above font materials. Further, the client can identify the user's natural handwriting font based on the collected or detected font materials, and submit the font to the server. Obviously, because the above-mentioned font materials are manually written by users, the fusion style font generated based on the standard style font and the natural handwriting font extracted from the font material can fully reflect the user's own handwriting style.

[0270] In another embodiment, the server can also generate multiple synthetic handwriting fonts based on the natural handwriting font and the basic standard style font of the non-handwriting style, and the multiple synthetic handwriting fonts can constitute a synthetic handwriting font library, and then the server can return the font library to the client. After obtaining the synthetic handwriting font library returned by the client, the client can display the text content of the resource according to each synthetic handwriting font in the synthetic handwriting font library in the process of displaying other multimedia resources (outside the multimedia resources). For example, if the client uploads the natural handwriting font of the user corresponding to itself to the server in the process of displaying the song Song1, the fusion style font generated by the server according to the standard style font and the natural handwriting font, and displays the lyrics of the song Song1 according to the fusion style font returned by the server. In addition, the server can also generate a synthetic handwriting font library corresponding to the user based on the basic standard style font and the natural handwriting font, and return the font library to the client. Thereafter, the client can directly use the synthetic handwriting font in the font library to display the lyrics of Song2 in the process of displaying the song Song2. Of course, in order to ensure that the display effect of the lyrics of the song Song2 is consistent with the style of the song, the client can also use the synthetic handwriting fonts in the above-mentioned synthetic handwriting font library to display the lyrics of Song2 when it is determined that the style of the song Song2 is the same as that of the song Song1. In other words, for the synthetic handwriting font library returned by the server, if each synthetic handwriting font in the font library is generated according to the natural handwriting font of a certain user and a standard style font matching the style of a certain resource, then the synthetic handwriting font library can be used to display the text content of the multimedia resources consumed by the user that is consistent with the style of the resource; and the synthetic handwriting font library should avoid being used to display the text content of the multimedia resources consumed by the user that is inconsistent with the style of the resource, or should also avoid being used to display the text content of multimedia resources consumed by other users.

[0271] Step S1004: display the text content in the style font.

[0272] When the text content of the multimedia resource and the style font matching the resource style of the resource are acquired, the client may display the text content according to the style font.

[0273] In one embodiment, the client can obtain multiple style fonts that match the resource style, and display the text content according to the target style font. The client can determine the target style font in multiple ways. For example, the client can determine the target style font among the multiple style fonts according to a preset rule. The preset rule can be: determine the style font with the highest degree of matching with the resource style among the multiple style fonts as the target style font, so as to present the best style matching effect to the user as much as possible. Alternatively, the style font that is displayed most times by the user among the multiple style fonts can also be determined as the target style font, so as to make the display effect of the style font conform to the user's usage habits as much as possible on the basis of style matching.

[0274] For another example, the client may also display preview styles of the multiple style fonts to the user, so that the user can learn the expected display effects of the multiple style fonts based on the preview styles, and then select the style font that interests the user the most as the target style font. Accordingly, the client may determine the first style font selected as the target style font in response to the selection operation for the multiple style fonts, and then display the text content according to the target style font. In this way, on the basis of style matching, the display effect of the style font can be made to meet the user's personal wishes as much as possible, that is, to meet the user's personalized display needs.

[0275] Furthermore, in the process of the client displaying the text content according to the target style font, the user may want to adjust the current display effect, that is, there is a need to switch the display effect. In this regard, the user can perform a switching operation on the target style font. Accordingly, the client can determine the second style font after switching in response to the above switching operation, and display the text content according to the second style font. In this way, the client can switch to other style fonts to display the text content if the user is not satisfied with the display effect of the target style font, thereby further satisfying the user's switching demand for display effects.

[0276] In one embodiment, to ensure that the display progress of the text content is consistent with the playback progress of the multimedia resource, the client may display the text content corresponding to the current playback progress in the style font according to the playback progress of the multimedia resource.

[0277] In addition, the resource style of the multimedia resource may not be fixed, such as different tones in different paragraphs of a song, different tones in different chapters of a movie, etc. For such multimedia resources, in order to present a better display effect to the user, the display mode of the text content can be adjusted according to the content fragments of the multimedia resource. For example, the client can determine the style fonts corresponding to each content fragment in the text content, and display the corresponding content fragments according to each style font. For example, for the song Song3, if the lead vocal tone of the verse part is relatively soft, the harmony of the interlude part is gradually passionate, and the main harmony of the chorus part is high-pitched, the client can respectively determine the style fonts corresponding to the verse, interlude and chorus parts, and display the corresponding lyrics fragments in the verse, interlude and chorus parts according to the corresponding style fonts. Of course, in the case where the above-mentioned multimedia resource is a video, the above-mentioned content fragment is a video fragment, and the style font corresponding to any video fragment can match the screen content of the video fragment. In this way, not only can the display effect of the text content be guaranteed to match the current content segment in real time, but the display effect can also change with the playback progress of the multimedia resource, presenting a dynamic style of text display effect, which helps to improve the user's text viewing experience.

[0278] Specifically, different types of multimedia resources usually have text content in different forms. In this regard, the client can display the text content in a corresponding manner. For example, when the multimedia resource is a song, the client can display the lyrics of the music in the lyrics display area of ​​the song. For another example, when the multimedia resource is a video, the subtitles, bullet screen and / or special effect text (such as flower characters, etc.) of the video are displayed in the video interface of the video. The embodiments of the present disclosure do not limit the specific form of the above text content.

[0279] According to the font generation and text display method of the embodiment of the present disclosure, the server can obtain the text content of the multimedia resource and the style information used to characterize the resource style of the multimedia resource, and then generate a style font matching the resource style according to the style information. Correspondingly, the client can obtain the text content and style font of the multimedia resource, and display the text content according to the style font.

[0280] By adopting the above method, a style font that matches the resource style of the multimedia resource can be generated. It can be understood that the text content displayed according to the above style font also matches the resource style of the multimedia resource. It can be seen that this scheme generates a style font that matches the resource style of the multimedia resource, and displays the text content of the multimedia resource according to the style font, so that the display effect of the text content has a high degree of fit with the resource style of the multimedia resource, that is, the display effect of the text content is highly matched with the resource style of the multimedia resource, thereby presenting the user with a text content display effect that is highly consistent with the resource style, which helps to improve the user's text content viewing experience.

[0281] Exemplary Media

[0282] After introducing the method of the exemplary embodiment of the present disclosure, next, refer to Fig.11 A medium according to an exemplary embodiment of the present disclosure is described.

[0283] In this exemplary embodiment, the above method can be implemented by a program product, such as a portable compact disk read-only memory (CD-ROM) and including program code, which can be run on a device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, the readable medium 110 can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device.

[0284] The program product may use any combination of one or more readable media. The readable medium 110 may be a readable signal medium or a readable medium. The readable medium 110 may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0285] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0286] The program code contained on the readable medium 110 may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RE, etc., or any suitable combination of the foregoing.

[0287] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0288] Exemplary Devices

[0289] After introducing the medium of the exemplary embodiment of the present disclosure, next, reference is made to Fig.12 The device of the exemplary embodiment of the present disclosure is described. Regarding the following device, the specific manner in which each functional module performs an operation and the specific functions implemented after performing the operation have been described in detail in the aforementioned embodiments of the font generation and text display method, and will not be elaborated here.

[0290] Fig.12 A block diagram of a font generation device according to an embodiment of the present disclosure is schematically shown. The device may include:

[0291] An acquisition module 1201 is used to acquire text content and style information of a multimedia resource, wherein the style information is used to characterize the resource style of the multimedia resource;

[0292] The generating module 1202 is used to generate a style font matching the resource style according to the style information, and the text content is used to be displayed according to the style font.

[0293] Optionally, the acquisition module 1201 is further used for:

[0294] The multimedia resource is input into the style recognition model to obtain the style information of the multimedia resource output by the style recognition model.

[0295] Optionally, the method further includes a first training module 1203, which is used to:

[0296] Acquire sample resources, any of which is added with a corresponding style tag;

[0297] The sample resources and their corresponding style tags are used as training samples to train the style recognition model.

[0298] Optionally, the generating module 1202 is further used for:

[0299] A standard style font corresponding to the style information is determined as a style font matching the resource style.

[0300] Optionally, the generating module 1202 is further used for:

[0301] Determine a standard style font corresponding to the style information;

[0302] Get the user style font that matches the user's handwriting style;

[0303] A fusion style font is generated according to the standard style font and the user style font to serve as a style font matching the resource style.

[0304] Optionally, the generating module 1202 is further used for:

[0305] Using a pre-acquired natural handwriting font as a user style font that matches the handwriting style of the user, the natural handwriting font being determined based on a font material manually written by the user; or

[0306] The pre-generated synthetic handwriting font is used as a user style font that matches the handwriting style of the user, and the synthetic handwriting font is generated according to the natural handwriting font and a basic standard style font that is not in the handwriting style.

[0307] Optionally, the generating module 1202 is further used for:

[0308] A synthetic handwriting font corresponding to the text content is selected from a pre-generated synthetic handwriting font library as a user style font that conforms to the handwriting style of the user, and each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the natural handwriting font and a basic standard style font of a non-handwriting style.

[0309] Optionally, a synthetic font generation module 1204 is also included, which is used to:

[0310] The natural handwriting font and the basic standard style font are input into a handwriting font generation model to obtain the synthetic handwriting font output by the handwriting font generation model.

[0311] Optionally, a second training module 1205 is further included, which is used to:

[0312] Selecting at least one sample style character from a plurality of handwritten standard style font libraries, and selecting at least one sample content character;

[0313] Determine the sample label words corresponding to the sample content words in each handwriting standard style font library, wherein any sample content word and each corresponding sample label word have the same font structure;

[0314] Generate training samples according to the sample style characters, sample content characters and sample label characters, wherein any training sample includes any sample content character and any corresponding sample style character; wherein the sample label of any training sample is the sample label character corresponding to any sample content character in the handwriting standard style font library to which any sample style character belongs;

[0315] The handwriting font generation model is trained using the training samples.

[0316] Optionally, the handwritten font generation model includes a style feature extraction network, a content feature extraction network, a style transfer network and a font content matching network, and the second training module 1205 is further used for:

[0317] The style feature extraction network extracts the sample style feature of any sample style word and inputs it into the style transfer network, and the content feature extraction network extracts the sample content feature of any sample content word and inputs it into the style transfer network;

[0318] The font content matching network determines the structural offset between the sample predicted font output by the style transfer network and the sample label word in any one of the training samples, and inputs a feature map representing the structural offset into the style transfer network.

[0319] Optionally, the second training module 1205 is further used for:

[0320] The style feature extraction network inputs the feature graphs of the sample style features into multiple structural levels in the style transfer network respectively.

[0321] Optionally, the generating module 1202 is further used for:

[0322] extracting standard style feature points in the standard style font and user style feature points in the user style font respectively;

[0323] Identifying target user style feature points among the user style feature points that respectively match the standard style feature points;

[0324] The standard style feature points are moved toward the corresponding target user style feature points, and the font formed by the moved standard style feature points is used as the fusion style font.

[0325] Optionally, the generating module 1202 is further used for:

[0326] According to the mapping relationship between the style information and the candidate standard style fonts, the standard style font corresponding to the style information is queried.

[0327] Optionally, the candidate standard style fonts are obtained from a preset standard style font library, and the standard style fonts include:

[0328] At least one handwriting style in a handwritten standard style font; and / or,

[0329] At least one basic standard style font that is not handwritten.

[0330] Optionally, also include:

[0331] The font returning module 1206 includes returning the style font to the client in response to the font acquisition request initiated by the client, so that the client can display the text content according to the style font.

[0332] Optionally, when the multimedia resource includes audio, the style information of the audio includes at least one of the following: music style, instrument information, paragraph information, rhythm information, strength information, and emotional information of the text content;

[0333] In the case where the multimedia resource includes a video, the style information of the video includes: emotional information of picture content and / or emotional information of the text content.

[0334] Fig.13 A block diagram of a text display device according to an embodiment of the present disclosure is schematically shown. The device may include:

[0335] The acquisition module 1301 is used to acquire the text content and style font of the multimedia resource, wherein the style font is generated according to the style information of the multimedia resource and matches the resource style of the multimedia resource represented by the style information;

[0336] The display module 1302 is used to display the text content according to the style font.

[0337] Optional,

[0338] It also includes a control display module 1303, which is used to display font style controls in the playback interface of the multimedia resource;

[0339] The acquisition module 1301 is further used for: in response to the font style control being triggered, acquiring the style font of the multimedia resource.

[0340] Optionally, the acquisition module 1301 is further used for:

[0341] receiving a style font of a multimedia resource sent by a server, wherein the style font is generated by the server according to the style information; or,

[0342] The style information of the multimedia resource is acquired, and the style font is generated according to the style information.

[0343] Optionally, the style font includes a standard style font corresponding to the style information, and the standard style font is obtained by querying according to a mapping relationship between the style information and candidate standard style fonts.

[0344] Optionally, the style font includes a fusion style font, and the acquisition module 1301 is further used for:

[0345] Determine a natural handwriting font from the font material manually written by the user, and submit the natural handwriting font to the server;

[0346] The fusion style font returned by the server is received, where the fusion style font is generated by the server according to the natural handwriting font and a standard style font corresponding to the style information.

[0347] Optionally, the acquisition module 1301 is further used for:

[0348] receiving or photographing an image recording a user-style font manually written by a user, and identifying the user-style font from the image; and / or,

[0349] The user-style fonts manually written by the user are collected using the screen or electronic drawing board of the terminal device.

[0350] Optionally, also include:

[0351] A font library acquisition module 1304 is used to acquire a synthetic handwriting font library returned by the server, wherein each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the natural handwriting font and a basic standard style font of a non-handwriting style;

[0352] The other text display module 1305 is used to display the text content of other multimedia resources according to each synthetic handwriting font in the synthetic handwriting font library.

[0353] Optionally, the acquisition module 1301 is further used to: acquire multiple style fonts matching the resource style;

[0354] The display module 1302 is further configured to: determine a target style font from the multiple style fonts according to a preset rule, or, in response to a selection operation for the multiple style fonts, determine a selected first style font as a target style font; and display the text content according to the target style font.

[0355] Optionally, the method further includes: a switching response module 1306, configured to determine a second style font after switching in response to a switching operation on the target style font;

[0356] The second font display module 1307 is used to display the text content in the second style font.

[0357] Optionally, the display module 1302 is further used for:

[0358] According to the playback progress of the multimedia resource, the text content corresponding to the current playback progress is displayed in the style font.

[0359] Optionally, the display module 1302 is further used for:

[0360] Determine the style fonts corresponding to each content segment in the text content;

[0361] Display the corresponding content fragments according to different style fonts.

[0362] Optionally, the display module 1302 is further used for:

[0363] In the case where the multimedia resource is a song, displaying the lyrics of the song in the lyrics display area of ​​the song;

[0364] In the case where the multimedia resource is a video, the subtitles, bullet screen and / or special effect text of the video are displayed in the video interface of the video.

[0365] Exemplary Computing Devices

[0366] After introducing the method, medium and apparatus of the exemplary embodiments of the present disclosure, next, reference is made to Fig.14 A computing device according to an exemplary embodiment of the present disclosure is described.

[0367] Fig.14 The computing device 140 shown is merely an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0368] like Fig.14As shown, the computing device 140 is in the form of a general computing device. The components of the computing device 140 may include but are not limited to: at least one processing unit 1401, at least one storage unit 1402, and a bus 1403 connecting different system components (including the processing unit 1401 and the storage unit 1402).

[0369] The bus 1403 includes a data bus, a control bus, and an address bus.

[0370] The storage unit 1402 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 14021 and / or a cache memory 14022 , and may further include a readable medium in the form of a non-volatile memory, such as a read-only memory (ROM) 14023 .

[0371] The storage unit 1402 may also include a program / utility 14025 having a set (at least one) of program modules 14024, such program modules 14024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0372] The computing device 140 may also communicate with one or more external devices 1404 (eg, keyboards, pointing devices, etc.).

[0373] Such communication may be performed via input / output (I / O) interface 1405. In addition, computing device 140 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via network adapter 1406. Fig.14 As shown, network adapter 1406 communicates with other modules of computing device 140 via bus 1403. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with computing device 140, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0374] It should be noted that although several units / modules or sub-units / modules of the font generation device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present disclosure, the features and functions of two or more units / modules described above may be embodied in one unit / module. Conversely, the features and functions of one unit / module described above may be further divided to be embodied by multiple units / modules.

[0375] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0376] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A font generation method, comprising: Acquire text content and style information of a multimedia resource, wherein the style information is used to characterize a resource style of the multimedia resource; Generating a style font matching the resource style according to the style information, including: determining a standard style font corresponding to the style information, acquiring a user style font that matches the user's handwriting style, and generating a fusion style font according to the standard style font and the user style font as a style font matching the resource style, and the text content is used to be displayed according to the style font.

2. According to the method of claim 1, the step of obtaining the style information of the multimedia resource comprises: The multimedia resource is input into the style recognition model to obtain the style information of the multimedia resource output by the style recognition model.

3. The method according to claim 2, wherein the style recognition model is trained in the following manner: Acquire sample resources, any of which is added with a corresponding style tag; The sample resources and their corresponding style tags are used as training samples to train the style recognition model.

4. According to the method of claim 1, the step of obtaining a user-style font that matches the user's handwriting style comprises: Using a pre-acquired natural handwriting font as a user style font that matches the handwriting style of the user, the natural handwriting font being determined based on a font material manually written by the user; or, The pre-generated synthetic handwriting font is used as a user style font that matches the handwriting style of the user, and the synthetic handwriting font is generated according to the natural handwriting font and a basic standard style font that is not in the handwriting style.

5. The method according to claim 4, wherein the step of using the pre-generated synthetic handwriting font as a user-style font that conforms to the handwriting style of the user comprises: A synthetic handwriting font corresponding to the text content is selected from a pre-generated synthetic handwriting font library as a user style font that conforms to the handwriting style of the user, and each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the natural handwriting font and a basic standard style font of a non-handwriting style.

6. The method according to claim 4, wherein the synthetic handwriting font is generated by: The natural handwriting font and the basic standard style font are input into a handwriting font generation model to obtain the synthetic handwriting font output by the handwriting font generation model.

7. The method according to claim 6, wherein the handwriting font generation model is trained in the following manner: Selecting at least one sample style character from a plurality of handwritten standard style font libraries, and selecting at least one sample content character; Determine the sample label words corresponding to the sample content words in each handwriting standard style font library, wherein any sample content word and each corresponding sample label word have the same font structure; A training sample is generated according to the sample style word, the sample content word and the sample label word, wherein any training sample includes any sample content word and any corresponding sample style word; wherein The sample label of any training sample is the sample label word corresponding to any sample content word in the handwriting standard style font library to which any sample style word belongs; The handwriting font generation model is trained using the training samples.

8. The method according to claim 7, wherein the handwriting font generation model comprises a style feature extraction network, a content feature extraction network, a style transfer network and a font content matching network, and the handwriting font generation model is trained using any one of the training samples, comprising: The style feature extraction network extracts the sample style feature of any sample style word and inputs it into the style transfer network, and the content feature extraction network extracts the sample content feature of any sample content word and inputs it into the style transfer network; The font content matching network determines the structural offset between the sample predicted font output by the style transfer network and the sample label word in any one of the training samples, and inputs a feature map representing the structural offset into the style transfer network.

9. The method according to claim 8, wherein the style feature extraction network inputs the sample style feature into the style transfer network, comprising: The style feature extraction network inputs the feature graphs of the sample style features into multiple structural levels in the style transfer network respectively.

10. The method according to claim 4, wherein generating a fusion style font according to the standard style font and the user style font comprises: respectively extracting standard style feature points in the standard style font and user style feature points in the user style font; Identifying target user style feature points among the user style feature points that respectively match the standard style feature points; The standard style feature points are moved toward the corresponding target user style feature points, and the font formed by the moved standard style feature points is used as the fusion style font.

11. The method according to any one of claims 4 to 10, wherein determining the standard style font corresponding to the style information comprises: According to the mapping relationship between the style information and the candidate standard style fonts, the standard style font corresponding to the style information is queried.

12. The method according to claim 11, wherein the candidate standard style fonts are obtained from a preset standard style font library, and the standard style fonts include: At least one handwritten standard style font in handwriting style; and / or, At least one basic standard style font that is not handwritten.

13. The method according to claim 1, further comprising: In response to a font acquisition request initiated by the client, the style font is returned to the client, so that the client displays the text content according to the style font.

14. The method according to claim 1, In the case where the multimedia resource includes audio, the style information of the audio includes at least one of the following: music style, instrument information, paragraph information, rhythm information, strength information, and emotional information of the text content; In the case where the multimedia resource includes a video, the style information of the video includes: Emotional information of the picture content and / or emotional information of the text content.

15. A text display method, comprising: Acquire text content and style fonts of a multimedia resource, wherein the style fonts are generated according to style information of the multimedia resource and match the resource style of the multimedia resource represented by the style information; The style font includes a fusion style font, and the acquiring of the style font of the multimedia resource includes: determining a natural handwriting font from a font material manually written by a user, and submitting the natural handwriting font to a server; receiving a fusion style font returned by the server, wherein the fusion style font is generated by the server according to the natural handwriting font and a standard style font corresponding to the style information; The text content is displayed in the style font.

16. The method according to claim 15, Also includes: Displaying a font style control in a playback interface of the multimedia resource; The acquiring of the style font of the multimedia resource includes: acquiring the style font of the multimedia resource in response to the font style control being triggered.

17. The method according to claim 15, wherein obtaining the style font of the multimedia resource comprises: Receiving a style font of a multimedia resource sent by a server, wherein the style font is generated by the server according to the style information; or, The style information of the multimedia resource is acquired, and the style font is generated according to the style information.

18. The method according to claim 17, wherein determining the natural handwriting font from the font material manually written by the user comprises: receiving or photographing an image recording a user-style font manually written by a user, and recognizing the user-style font from the image; and / or, The user-style fonts manually written by the user are collected using the screen or electronic drawing board of the terminal device.

19. The method according to claim 17, further comprising: Acquire a synthetic handwriting font library returned by the server, wherein each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the natural handwriting font and a basic standard style font of a non-handwriting style; The text content of other multimedia resources is displayed according to each synthetic handwriting font in the synthetic handwriting font library.

20. The method according to claim 15, The step of obtaining the style font of the multimedia resource includes: Acquire multiple font styles that match the resource style; The displaying of the text content in accordance with the style font comprises: determining a target style font from the multiple style fonts according to a preset rule, or, in response to a selection operation for the multiple style fonts, determining a selected first style font as a target style font; and displaying the text content in accordance with the target style font.

21. The method according to claim 20, further comprising: In response to a switching operation on the target style font, determining a second style font after switching; The text content is displayed according to the second style font.

22. The method according to claim 15, wherein displaying the text content in the style font comprises: According to the playback progress of the multimedia resource, the text content corresponding to the current playback progress is displayed in the style font.

23. The method according to claim 15, wherein displaying the text content in the style font comprises: Determine the style fonts corresponding to each content segment in the text content; Display the corresponding content fragments according to different style fonts.

24. The method according to claim 15, wherein displaying the text content in the style font comprises: In the case where the multimedia resource is a song, displaying the lyrics of the song in the lyrics display area of ​​the song; In the case where the multimedia resource is a video, the subtitles, bullet screen and / or special effect text of the video are displayed in the video interface of the video.

25. A font generation device, comprising: An acquisition module, used to acquire text content and style information of a multimedia resource, wherein the style information is used to characterize the resource style of the multimedia resource; A generation module is used to generate a style font matching the resource style according to the style information, including: determining a standard style font corresponding to the style information, obtaining a user style font that matches the user's handwriting style, and generating a fusion style font according to the standard style font and the user style font as a style font matching the resource style, and the text content is used to be displayed according to the style font.

26. The device according to claim 25, wherein the acquisition module is further configured to: The multimedia resource is input into the style recognition model to obtain the style information of the multimedia resource output by the style recognition model.

27. The apparatus according to claim 26, further comprising a first training module for: Acquire sample resources, any of which is added with a corresponding style tag; The sample resources and their corresponding style tags are used as training samples to train the style recognition model.

28. The apparatus according to claim 27, wherein the generating module is further configured to: Using a pre-acquired natural handwriting font as a user style font that matches the handwriting style of the user, the natural handwriting font being determined based on a font material manually written by the user; or The pre-generated synthetic handwriting font is used as a user style font that conforms to the handwriting style of the user, and the synthetic handwriting font is generated according to the natural handwriting font and a basic standard style font that is not in the handwriting style.

29. The apparatus according to claim 28, wherein the generating module is further configured to: A synthetic handwriting font corresponding to the text content is selected from a pre-generated synthetic handwriting font library as a user style font that conforms to the handwriting style of the user, and each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the natural handwriting font and a basic standard style font of a non-handwriting style.

30. The apparatus according to claim 28, further comprising a synthetic font generation module, configured to: The natural handwriting font and the basic standard style font are input into a handwriting font generation model to obtain the synthetic handwriting font output by the handwriting font generation model.

31. The apparatus according to claim 30, further comprising a second training module, for: Selecting at least one sample style character from a plurality of handwritten standard style font libraries, and selecting at least one sample content character; Determine the sample label words corresponding to the sample content words in each handwriting standard style font library, wherein any sample content word and each corresponding sample label word have the same font structure; A training sample is generated according to the sample style word, the sample content word and the sample label word, wherein any training sample includes any sample content word and any corresponding sample style word; wherein The sample label of any training sample is the sample label word corresponding to any sample content word in the handwriting standard style font library to which any sample style word belongs; The handwriting font generation model is trained using the training samples.

32. The apparatus according to claim 31, wherein the handwritten font generation model comprises a style feature extraction network, a content feature extraction network, a style transfer network and a font content matching network, and the second training module is further used for: The style feature extraction network extracts the sample style feature of any sample style word and inputs it into the style transfer network, and the content feature extraction network extracts the sample content feature of any sample content word and inputs it into the style transfer network; The font content matching network determines the structural offset between the sample predicted font output by the style transfer network and the sample label word in any one of the training samples, and inputs a feature map representing the structural offset into the style transfer network.

33. The apparatus according to claim 32, wherein the second training module is further configured to: The style feature extraction network inputs the feature graphs of the sample style features into multiple structural levels in the style transfer network respectively.

34. The apparatus according to claim 28, wherein the generating module is further configured to: respectively extracting standard style feature points in the standard style font and user style feature points in the user style font; Identifying target user style feature points among the user style feature points that respectively match the standard style feature points; The standard style feature points are moved toward the corresponding target user style feature points, and the font formed by the moved standard style feature points is used as the fusion style font.

35. The apparatus according to any one of claims 28 to 34, wherein the generating module is further configured to: According to the mapping relationship between the style information and the candidate standard style fonts, the standard style font corresponding to the style information is queried.

36. The apparatus according to claim 35, wherein the candidate standard style fonts are obtained from a preset standard style font library, and the standard style fonts include: At least one handwritten standard style font in handwriting style; and / or, At least one basic standard style font that is not handwritten.

37. The apparatus of claim 25, further comprising: The font returning module includes returning the style font to the client in response to a font acquisition request initiated by the client, so that the client can display the text content according to the style font.

38. The device according to claim 25, In the case where the multimedia resource includes audio, the style information of the audio includes at least one of the following: music style, instrument information, paragraph information, rhythm information, strength information, and emotional information of the text content; In the case where the multimedia resource includes a video, the style information of the video includes: Emotional information of the picture content and / or emotional information of the text content.

39. A text display device, comprising: An acquisition module, used for acquiring text content and style fonts of multimedia resources, wherein the style fonts are generated according to style information of the multimedia resources and match the resource style of the multimedia resources represented by the style information; The style font includes a fusion style font, and the acquisition module is used to: determine a natural handwriting font from a font material manually written by a user, and submit the natural handwriting font to a server; receive a fusion style font returned by the server, the fusion style font is generated by the server according to the natural handwriting font and a standard style font corresponding to the style information; A display module is used to display the text content according to the style font.

40. The device according to claim 39, It also includes a control display module, which is used to display font style controls in the playback interface of the multimedia resource; The acquisition module is further used for acquiring the style font of the multimedia resource in response to the font style control being triggered.

41. The apparatus according to claim 39, wherein the acquisition module is further configured to: receiving a style font of a multimedia resource sent by a server, wherein the style font is generated by the server according to the style information; or, The style information of the multimedia resource is acquired, and the style font is generated according to the style information.

42. The apparatus according to claim 41, wherein the acquisition module is further configured to: receiving or photographing an image recording a user-style font manually written by a user, and identifying the user-style font from the image; and / or, The user-style fonts manually written by the user are collected using the screen or electronic drawing board of the terminal device.

43. The apparatus of claim 41, further comprising: A font library acquisition module, used for acquiring a synthetic handwriting font library returned by the server, wherein each synthetic handwriting font in the synthetic handwriting font library is generated by the server according to the basic standard style fonts of the natural handwriting font and non-handwriting style; The other text display module is used to display the text content of other multimedia resources according to each synthetic handwriting font in the synthetic handwriting font library.

44. The device according to claim 39, The acquisition module is also used to: acquire multiple style fonts matching the resource style; The display module is further used to: determine a target style font from the multiple style fonts according to a preset rule, or, in response to a selection operation for the multiple style fonts, determine a selected first style font as a target style font; and display the text content according to the target style font.

45. The apparatus of claim 44, further comprising: A switching response module, configured to determine a second style font after switching in response to a switching operation on the target style font; The second font display module is used to display the text content in the second style font.

46. ​​The device according to claim 39, wherein the display module is further configured to: According to the playback progress of the multimedia resource, the text content corresponding to the current playback progress is displayed in the style font.

47. The device according to claim 39, wherein the display module is further used for: Determine the style fonts corresponding to each content segment in the text content; Display the corresponding content fragments according to different style fonts.

48. The apparatus according to claim 39, wherein the display module is further configured to: In the case where the multimedia resource is a song, displaying the lyrics of the song in the lyrics display area of ​​the song; In the case where the multimedia resource is a video, the subtitles, bullet screen and / or special effect text of the video are displayed in the video interface of the video.

49. A medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 24 is implemented.

50. A computing device comprising: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 24 by running the executable instructions.

Images