Text generation method, device, electronic device and storage medium

Abstract

The disclosed embodiment provides a method, device, electronic device and storage medium for generating text, the method comprising: obtaining text to be displayed and a pre-selected target style type; converting the text to be displayed into a target text corresponding to the target style type; wherein the target text is pre-generated and / or generated in real time based on a style type conversion model; and displaying the target text on a target display interface. The technical solution provided by the disclosed embodiment not only provides a concise and efficient text design solution, but also avoids the problems of low efficiency, high cost and inability to accurately obtain the desired font in the traditional manual design process.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of artificial intelligence technology, and in particular to a text generation method, device, electronic device and storage medium. Background Art

[0002] At present, in the process of designing a set of Chinese characters with a unique style, relevant developers often need to pay a lot of time, material and manpower costs.

[0003] At the same time, due to the large differences between different styles of Chinese characters, even a highly professional designer may find it difficult to obtain the desired style of font after manually designing and modifying the Chinese characters many times. Summary of the invention

[0004] The embodiments of the present disclosure provide a text generation method, device, electronic device and storage medium, which not only provide a concise and efficient text design solution, but also avoid the problems of low efficiency, high cost and inability to accurately obtain the desired font in the traditional manual design process.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for generating text, the method comprising:

[0006] Get the text to be displayed and the pre-selected target style type;

[0007] Converting the to-be-displayed text into target text corresponding to the target style type; wherein the target text is pre-generated and / or generated in real time based on a style type conversion model;

[0008] The target text is displayed on a target display interface.

[0009] In a second aspect, the present disclosure also provides a text generation device, the device comprising:

[0010] A style type determination module is used to obtain the text to be displayed and the pre-selected target style type;

[0011] A target text determination module, used for converting the to-be-displayed text into a target text corresponding to the target style type; wherein the target text is pre-generated and / or generated in real time based on a style type conversion model;

[0012] The text display module is used to display the target text on the target display interface.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the text generation method as described in any of the embodiments of the present disclosure.

[0017] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the text generation method as described in any one of the embodiments of the present disclosure.

[0018] The technical solution of the disclosed embodiment first obtains the text to be displayed and a pre-selected target style type, then converts the text to be displayed into target text of the target style type, wherein the target text is pre-generated and / or generated in real time based on a style type conversion model, and finally displays the target text on a target display interface. By introducing an artificial intelligence model to generate fonts of a specific style, not only a concise and efficient text design solution is provided, but also the problems of low efficiency, high cost and inability to accurately obtain the desired font in the traditional manual design process are avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A flowchart of a text generation method provided in the first embodiment of the present disclosure;

[0021] Figure 2 A flowchart of a text generation method provided in Embodiment 2 of the present disclosure;

[0022] Figure 3 This is an overall network structure diagram of the style type conversion model provided in the second embodiment of the present disclosure;

[0023] Figure 4 A flowchart of a method for generating text provided in Embodiment 3 of the present disclosure;

[0024] Figure 5 The font feature extraction sub-model to be trained provided by the third embodiment of the present disclosure;

[0025] Figure 6 The trained font feature extraction sub-model provided in the third embodiment of the present disclosure;

[0026] Figure 7A flowchart of a text generation method provided in Embodiment 4 of the present disclosure;

[0027] Figure 8 This is a structural block diagram of a text generation device provided in Embodiment 5 of the present disclosure;

[0028] Fig. 9 This is a structural diagram of an electronic device provided in Embodiment 6 of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0034] Embodiment 1

[0035] Figure 1The flowchart of a text generation method provided in Embodiment 1 of the present disclosure. This embodiment is applicable to the situation of designing existing text to obtain a desired font. This method can be executed by a text generation device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, or a server, etc.

[0036] Before introducing the technical solution, an exemplary description of the application scenario can be given first. This technical solution can be applied to any scenario where specific style types of text need to be generated. For example, when a user finds that the style type corresponding to a certain character or multiple characters meets their expectations, based on the solution of this embodiment, any Chinese character can be presented in the above style type; or, on the basis of having obtained some text handwriting of a certain user, based on the solution of this embodiment, a computer font library in the user's exclusive handwriting style type can be quickly generated for the user.

[0037] As Figure 1 , the method of this embodiment includes:

[0038] S110. Obtain the text to be displayed and the pre-selected target style type.

[0039] Among them, the text to be displayed can be one or more characters written by the user, or can also be text that can be displayed on a display device. For example, it can be characters written by the user through a handwriting board or relevant applications in the computer. Correspondingly, after the user writes one or more characters, the computer can obtain these characters and determine them as the text to be displayed. It can be understood that in the actual application process, an image containing the user's handwritten text can also be recognized, and then the recognized text can be used as the text to be displayed. For example, after the user writes the character "永" on a writing board, it can be photographed and the image can be uploaded to the system. After the system recognizes the image, it can obtain the character "永" written by the user and then use it as the text to be displayed.

[0040] In this embodiment, the text to be displayed can also be text that has been designed in the computer and has been assigned a specific instruction sequence, such as the text in the existing simplified Chinese character library or traditional Chinese character library in the computer. It can be understood that based on the specific instruction sequence, the system can at least describe the glyph of the text and display it on the associated display device. Exemplarily, when the user inputs "yong" through the pinyin input method on the computer and selects a Chinese character corresponding to the pronunciation in the result list (such as the character "永"), the computer can obtain the internal code of this character (such as the internal code of the character "永") in the existing simplified Chinese character library and determine the text with the glyph corresponding to this internal code as the text to be displayed.

[0041] In this embodiment, after obtaining the text to be displayed, it is also necessary to determine the pre-selected target style type. The target style type is the text style type that the user expects to obtain. For example, for Chinese characters, the style type can be Songti, Kaiti, Heiti, etc. that have obtained the corresponding copyright. Of course, in actual application, the text style type expected by the user can be a font similar to his or her own writing style. In this case, the target style type is a style type similar to the user's handwriting.

[0042] It is understandable that different styles of characters have different stroke styles and structures. For example, the same Chinese character has different stroke thicknesses and roundnesses in different styles, and the strokes are also matched, arranged, and combined differently. For different users' handwriting styles, the differences in text styles will be further magnified.

[0043] In this embodiment, the user can select the target style type based on the style type selection control pre-developed in the system. For example, for Chinese characters, the drop-down menu of the corresponding style type selection control can include the copyrighted Songti and Kaiti styles, as well as the handwriting of user A and the handwriting of user B, etc.

[0044] S120: Convert the text to be displayed into target text corresponding to the target style type.

[0045] In this embodiment, when the system obtains the text to be displayed and determines the corresponding target style type, the text to be displayed can be converted to obtain the target text of the target style type. This process can be understood as converting text of one stroke style and structure into another stroke style and structure.

[0046] Specifically, the target text can be converted into the target text based on the style type conversion model. Among them, the style type conversion model can be a pre-trained convolutional neural network model, the input of the model is the text to be displayed and the target style type, and correspondingly, the output of the model is the target text. Exemplarily, when it is determined that the copyrighted Song font character input by the user based on the input method is the text to be displayed, and the pre-selected target style type is determined to be "user A handwriting", the copyrighted Song font "Yong" character and the information associated with the target style type can be input into the style type conversion model. After model processing, the "Yong" character similar to the handwriting of user A can be obtained, and the character is determined as the target text. It can be understood that when the text style type desired by the user is a font similar to his own writing style, the above-mentioned text processing process based on the style type conversion model is essentially a process of imitating the user's writing habits (handwriting) to generate the target text corresponding to the text to be displayed.

[0047] In actual application, the target text is pre-generated and / or generated in real time based on the style type conversion model. That is to say, the system can use the style type conversion model to process the text to be displayed in real time, so as to generate the corresponding target text; it can also use the style type conversion model to pre-process the existing multiple texts in the character library, so as to obtain the corresponding texts of multiple style types. Furthermore, a mapping table representing the association relationship is constructed based on the text in the existing character library and the corresponding texts of multiple style types. When the text to be displayed is determined from the existing character library and the target style type is determined, the corresponding target text can be directly determined by looking up the table for calling, which further optimizes the efficiency of text generation.

[0048] S130: Display the target text on the target display interface.

[0049] In this embodiment, after the target text is determined based on the style type conversion model, the system can at least describe and present the target text based on the model output result. It can be understood that the system can at least determine the image information corresponding to the target text based on the output of the style type conversion model, and display it on the target display interface. Among them, the target display interface can be a visual interface associated with the system, which can at least call and display the image information corresponding to the target text.

[0050] It should be noted that in actual application, after the target text is determined, the target text can be exported in the form of a related image file, or the related image file can be sent to the user's corresponding client; when there are multiple target texts obtained by conversion, a specific font library can be constructed for these texts, that is, a set of image sources is generated based on the image information of the target text, and the image source is associated with the internal code corresponding to the text, so that it can be used directly by the user as a font of the target style type in the subsequent process. It can be understood that this processing method provides a simple and efficient way for users to quickly generate a text font library similar to their own handwriting.

[0051] The technical solution of this embodiment first obtains the text to be displayed and the pre-selected target style type, then converts the text to be displayed into target text of the target style type, wherein the target text is pre-generated and / or generated in real time based on the style type conversion model, and finally displays the target text on the target display interface. By introducing an artificial intelligence model to generate fonts of a specific style, not only a concise and efficient text design solution is provided, but also the problems of low efficiency, high cost and inability to accurately obtain the desired font in the traditional manual design process are avoided.

[0052] Embodiment 2

[0053] Figure 2The flowchart of a method for generating text provided in the second embodiment of the present disclosure is as follows. On the basis of the above-mentioned embodiment, a style type conversion model is constructed based on the font feature extraction sub-model, the decoupling model, the feature splicing sub-model and the feature processing sub-model. The features of the text are determined by introducing a variety of artificial intelligence algorithms, providing users with an efficient and intelligent font generation method. The target text corresponding to the text to be displayed is directly determined from the target text package, further improving the efficiency of text generation. The specific implementation method can be found in the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.

[0054] like Figure 2 As shown, the method specifically comprises the following steps:

[0055] S210: When it is detected that the text to be displayed is edited, determine a target style type selected from a style type list.

[0056] In this embodiment, the system can detect the user's input in the text box. When it is detected that the user is editing text in the text box, the corresponding text can be obtained from the existing font library as the text to be displayed. At the same time, according to the user's touch operation on the style type selection control, the corresponding style type list is displayed. It can be understood that the list includes at least one style type, such as user A's handwriting, user B's handwriting, etc. Since the style type conversion model needs to be used to process the text to be displayed in the subsequent process, it can be understood that the style type list includes style types corresponding to the style type conversion model. Furthermore, the target style type can be determined based on the user's selection result in the list, that is, the font expected by the user can be determined.

[0057] S220: Convert the text to be displayed into target text corresponding to the target style type.

[0058] In the process of converting the to-be-displayed text into the target text, optionally, the target text consistent with the to-be-displayed text is obtained from a target text package corresponding to the target style type.

[0059] Specifically, after determining the target style type, the system can determine the target text package according to the identifier of the style type. The target text package is generated after converting each character into the target font based on the style type conversion model. It can be understood that based on the style type conversion model, the system converts each character in the existing font library into a character of the corresponding style type in advance, and obtains the relevant data of these characters (such as character identifiers, image information and corresponding machine codes), thereby constructing a target text package based on the relevant data of the converted characters, and at the same time, the target text package is associated with the corresponding style type in the style type list, for example, the target text package corresponds to "user A handwriting" in the style type list.

[0060] Furthermore, after the target text package is determined, the target text consistent with the text to be displayed can be obtained in the target text package based on the relevant data of the text to be displayed. That is to say, the target text with the same content as the text to be displayed but different style type (such as stroke style, spacing structure) can be obtained from the target text package.

[0061] The advantage of this processing method is that when the text to be displayed and the target style type are determined, the corresponding target text can be retrieved from the target text package, further improving the efficiency of text generation.

[0062] In actual application, when the user selects the target style type in the style type list, it may happen that the system has not pre-built a target text package for the font based on the style type conversion model. In this case, the system can also directly input the text to be displayed into the style type conversion model to obtain the target text corresponding to the target font. Figure 3 The overall network structure diagram of the style type conversion model shown in the figure provides a detailed explanation of the process of generating the target text.

[0063] See also Figure 3 In this embodiment, the style type conversion model includes a first font feature extraction sub-model, a second font feature extraction sub-model, a first decoupling model connected to the first font feature extraction sub-model, a second decoupling model connected to the second font feature extraction sub-model, a feature splicing sub-model connected to the first decoupling model and the second decoupling model, and a feature processing sub-model.

[0064] Among them, the first font feature extraction sub-model and the second feature extraction sub-model have the same model structure and are used to determine the text features of each character. Specifically, the text features include style type features and text content features. It can be understood that the text features include features reflecting the stroke order and structure of the character font (i.e., style type features), and also include features reflecting the meaning or identification information corresponding to the character in the computer (i.e., text content features). Therefore, the first font feature extraction sub-model and the second font feature extraction sub-model can also be used as multi-modal feature extractors for text.

[0065] Specifically, based on the first feature extraction sub-model, the first decoupled text feature of the text to be displayed is determined, and based on the second feature extraction sub-model, the second decoupled text feature of the target style text is determined. It can be understood that the first font feature extraction sub-model can be used to determine the style type feature and text content feature of the text to be displayed (i.e., the first decoupled text feature), and the second font feature extraction sub-model can be used to determine the style type feature and text content feature of any character belonging to the same style type as the target character (i.e., the second decoupled text feature). In the actual application process, any character belonging to the same style type as the target character can be used as the target style text. It can be understood that the text type of the target style text is consistent with the target style type.

[0066] Take Figure 3 as an example. After inputting the text to be displayed into the first font feature extraction sub-model for processing, the computer can determine that the text is the character '永' under the stroke order and structure of the Song typeface with copyright. When the target style type is 'User A's handwriting', in order to obtain the character '永' in this font, the existing character '春' written by User A can be input into the second font feature extraction sub-model, and the computer can determine that the text is the character '春' under the stroke order and structure of User A's handwriting.

[0067] In this embodiment, the decoupling model is used to perform decoupling processing on the text features extracted by the font feature extraction sub-model to distinguish the style type features and text content features. Specifically, based on the first decoupling model, the first decoupled text feature is processed to obtain the to-be-displayed style type and to-be-displayed content feature of the text to be displayed; and based on the second decoupling model, the second decoupled text feature is processed to obtain the target style type and target content feature of the target style text. It can be understood that after processing the text to be displayed based on the first decoupling model, the style type feature of the decoupled text to be displayed is used as the to-be-displayed style type feature, and the text content feature of the text to be displayed is used as the to-be-displayed content feature; at the same time, after processing the target style text based on the second decoupling model, the style type feature of the decoupled target style text is used as the target style type feature, and the text content feature of the target style text is used as the target content feature.

[0068] Continue to refer to Figure 3 When the first font feature extraction sub-model determines that the text to be displayed is the Song Dynasty style "Yong" character with copyright obtained, the corresponding first decoupling model can be used to decouple the style type feature and the text content feature of this character, and obtain the features of this character under the stroke order and structure of the Song Dynasty style with copyright obtained, as well as the features corresponding to the meaning or identification information of this character; when the second font feature extraction sub-model determines that the text to be displayed is the "Chun" character handwritten by user A, the corresponding second decoupling model can also be used to decouple the style type feature and the text content feature of this character, and obtain the features of this character under the stroke order and structure of user A's handwriting style, as well as the features corresponding to the meaning or identification information of this character.

[0069] In this embodiment, the feature splicing sub-model is used to splice and process the text features extracted by the decoupling model to obtain the corresponding text style features. Specifically, based on the feature splicing model, the content features to be displayed and the target style type are obtained, and the text style features corresponding to the text to be displayed are obtained. It can be understood that based on the text content features of the text to be displayed and the style type features of the target style text, the text style features corresponding to the text to be displayed are spliced.

[0070] Continue to refer to Figure 3 After the first decoupling model and the second decoupling model respectively decouple the multi-modal features of the "Yong" character and the "Chun" character, the feature splicing sub-model can select the text content feature of the "Yong" character and the style type feature of the "Chun" character from the decoupled features. Further, the above two features are spliced to obtain the features for generating the "Yong" character in the handwriting style type of user A.

[0071] In this embodiment, the feature processing sub-model is used to process the text style features to obtain the target text of the text to be displayed in the target style type, which can be a Convolutional Neural Networks (CNN) model. Specifically, based on the feature processing sub-model, the text style features are processed to obtain the target text corresponding to the text to be displayed in the target style type.

[0072] Continue to refer to Figure 3 When the feature splicing sub-model outputs the feature vector for generating the "Yong" character in the handwriting style type of user A, the CNN model can be used to process it, so as to output the image information of the "Yong" character that can be called and displayed by the computer.

[0073] S230. Display the target text on the target display interface.

[0074] The technical solution of this embodiment constructs a style type conversion model based on the font feature extraction sub-model, the decoupling model, the feature splicing sub-model and the feature processing sub-model, and determines the characteristics of the text by introducing a variety of artificial intelligence algorithms, providing users with an efficient and intelligent font generation method; directly determining the target text corresponding to the text to be displayed from the target text package, further improving the text generation efficiency.

[0075] Embodiment 3

[0076] Figure 4 This is a flowchart of a text generation method provided in the third embodiment of the present disclosure. On the basis of the above-mentioned embodiment, based on the first training sample, at least two sub-models of font feature extraction to be trained in the style type conversion model are trained. Further, based on the first preset loss function and the second preset loss function, the sub-models are respectively optimized in parameters. Finally, the decoding module is removed to obtain a multimodal feature extractor in the style type conversion model. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.

[0077] like Figure 4 As shown, the method specifically comprises the following steps:

[0078] S310, training to obtain at least two font feature extraction sub-models in the style type conversion model.

[0079] It should be noted that before generating the target text based on the style type conversion model, it is necessary to first train at least two font feature extraction sub-models in the model. It can be understood that at least one font feature extraction sub-model is trained to extract the style type features of the text (such as stroke order and structure), and at least one font feature extraction sub-model is also trained to extract the text content features of the text (such as text meaning and text logo). Figure 5 The font feature extraction sub-models to be trained are shown, and the process of training at least two font feature extraction sub-models is described in detail.

[0080] In order to train at least two font feature extraction sub-models, it is necessary to first obtain a first training sample set. It can be understood that in actual application, in order to improve the accuracy of the model, as many and rich training samples as possible can be obtained to construct a training sample set.

[0081] Specifically, the first training sample set includes multiple first training samples. Each first training sample includes a theoretical text image and theoretical text strokes corresponding to the first training text, as well as masked text strokes that mask some of the theoretical text strokes. It can be understood that the theoretical text image is a picture of a Chinese character presented in a specific font, and the theoretical text strokes are information reflecting the theoretical writing order of each stroke of the Chinese character. At the same time, in order to enable the computer to understand Chinese character features from a deep perspective of Chinese character writing, it is also necessary to select some content from the theoretical text strokes for masking (mask) processing, that is, to mask some strokes of the Chinese character so that they do not participate in the subsequent processing of the font feature extraction sub-model. It can be understood that after masking some strokes of the theoretical text strokes, the masked text strokes corresponding to the Chinese character are obtained.

[0082] Take Figure 5 as an example. When determining that the character "永" is the first training text, the text image corresponding to this character in a specific font is the theoretical text image, and the five strokes and their order that make up the character "永" are the theoretical text strokes. Further, perform masking processing on the theoretical text strokes, that is, after masking the first, second, and fourth strokes of the five strokes of the character "永", the masked text strokes corresponding to the character "永" are obtained.

[0083] Further, for each first training sample, input the theoretical text image and masked text strokes in the current first training sample into the font feature extraction sub-model to be trained, and obtain the actual text image and predicted text strokes corresponding to the current first training sample. Continuing to refer to Figure 5 Figure, after inputting the picture reflecting the style of the character "永" presented in a specific font and the masked text strokes with the first, second, and fourth strokes masked into the corresponding font feature extraction sub-model to be trained, the text image output by the model and the complete text strokes predicted by the model for the character "永" can be obtained.

[0084] In the process of determining the actual text image as described above, optionally, extract the image features corresponding to the theoretical text image and perform compression processing on the image features to obtain the first feature to be used; process the feature vector corresponding to the masked text strokes to obtain the second feature to be used; perform feature interaction on the first feature to be used and the second feature to be used to obtain the text image features corresponding to the first feature to be used and the actual stroke features corresponding to the second feature to be used.

[0085] Continuing to refer to Figure 5, after the image features corresponding to the character "永" are extracted based on the CNN model, the extracted image features can be compressed based on the Transformer model to obtain the first feature to be used. Similarly, the feature vectors of the masked text strokes can be processed based on the Transformer model to obtain the second feature to be used. Further, cross-attention processing is performed on the first feature to be used and the second feature to be used to achieve feature interaction between the text image information and the text stroke information, and thus the text image features corresponding to the character "永" and the actual stroke features of the character "永" can be obtained.

[0086] It should be noted that the sub-model for extracting font features to be trained includes a decoding module, that is Figure 5 the Decoder module shown. Based on this, after obtaining the above text image features and actual stroke features, based on the actual stroke features, predicted text strokes are obtained, and based on the decoding module, the text image features are decoded to obtain the actual text image. Continuing to refer to Figure 5 , after obtaining the text image features of the character "永" and its actual stroke features, its predicted strokes can be obtained. Further, based on the Decoder module, the text image features of the character "永" are decoded, and the actual text image corresponding to the character "永" output by the sub-model for extracting font features to be trained is obtained.

[0087] It can be understood that in this embodiment, the process of inputting each first training sample into the sub-model for extracting font features to be trained and obtaining the predicted text strokes and actual text images corresponding to the characters in the samples is a process of enabling the computer to understand the character features from a deep perspective of Chinese character writing.

[0088] In the process of training at least two sub-models for extracting font features, the optimization of model parameters is also involved. Optionally, the actual text image and the theoretical text image are subjected to loss processing based on the first preset loss function in the sub-model for extracting features to be trained, and the predicted text strokes and the theoretical text strokes are subjected to loss processing based on the second preset loss function, so as to correct the model parameters in the sub-model for extracting font features to be trained according to the obtained loss values; the convergence of the first preset loss function and the second preset loss function is used as the training target to obtain the sub-model for extracting font features to be used.

[0089] In this embodiment, the parameters in the feature extraction submodel to be trained can be corrected based on the first preset loss function. Here, the first preset loss function for a font feature extraction submodel to be trained is used as an example for explanation. Specifically, based on a font feature extraction submodel to be trained, after obtaining multiple groups of actual text images and theoretical text images for multiple characters in the training sample set, multiple corresponding loss values ​​can be determined; further, when the model parameters in the submodel are corrected using multiple loss values ​​and the first preset loss function, the training error of the loss function, that is, the loss parameter, can be used as a condition for detecting whether the loss function has reached convergence at present, such as whether the training error is less than the preset error or whether the error change trend tends to be stable, or whether the current number of iterations is equal to the preset number. If the convergence condition is detected, such as the training error of the loss function is less than the preset error, or the error change trend tends to be stable, it indicates that the training of the font feature extraction submodel to be trained is completed, and the iterative training can be stopped at this time. If it is detected that the convergence condition is not reached at present, the actual text images and theoretical text images corresponding to other characters can be further obtained to continue training the model until the training error of the loss function is within the preset range. When the training error of the loss function reaches convergence, the trained font feature extraction sub-model to be trained can be used as the font feature extraction sub-model to be used. That is, at this time, after inputting the theoretical text image of a certain text into the font feature extraction sub-model to be used, the actual text image corresponding to the text can be obtained.

[0090] For the feature extraction sub-model to be trained for processing text strokes, the model parameters can be modified in the same manner as above based on the second preset loss function and multiple groups of predicted text strokes and theoretical text strokes, and the embodiments of the present disclosure will not be repeated here.

[0091] In this embodiment, after the training of at least two to-be-trained font feature extraction sub-models is completed and the corresponding to-be-used font feature extraction sub-models are obtained, the parameters in the model can be frozen to provide high-quality feature information for subsequent text processing processes.

[0092] At the same time, in order to insert the font feature extraction sub-model to be used into the overall model network structure, the font feature extraction sub-model to be used needs to be eliminated to obtain the font feature extraction sub-model. Optionally, when the font feature extraction sub-model to be trained includes a decoding module, the decoding module in the font feature extraction sub-model to be used is eliminated to obtain the font feature extraction sub-model in the style type conversion model. Figure 6As shown, after inputting any Chinese character into the font feature extraction sub-model, the sub-model can process the style type features and text content features of the Chinese character, and then obtain the multimodal features of the Chinese character, such as the stroke order, frame structure, text meaning or text logo of the Chinese character in the current font. Those skilled in the art should understand that for the font feature extraction sub-model after removing the decoding module, the feature map associated with the text before inputting the decoding module is the output of the font feature extraction sub-model; at the same time, the two-dimensional feature map corresponding to each convolution layer in the CNN model is used as the input of the decoupling model in the subsequent processing process, which can retain more spatial information.

[0093] S320: Obtain the text to be displayed and the pre-selected target style type.

[0094] S330: Convert the text to be displayed into target text corresponding to the target style type.

[0095] S340: Display the target text on the target display interface.

[0096] The technical solution of this embodiment trains at least two font feature extraction sub-models to be trained in the style type conversion model based on the first training sample. Furthermore, the parameters of the sub-models are optimized based on the first preset loss function and the second preset loss function respectively. Finally, the decoding module is eliminated to obtain a multimodal feature extractor in the style type conversion model.

[0097] Embodiment 4

[0098] Figure 7 This is a flow chart of a method for generating text provided in the fourth embodiment of the present disclosure. On the basis of the above-mentioned embodiment, after the font feature extraction sub-model is trained, the style type conversion model is trained based on the second training sample set to obtain a trained style type conversion model; during the training process, at least three preset loss functions are used to optimize the parameters in the model, further reducing the error rate of the target text generated by the model. The specific implementation method can be found in the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.

[0099] like Figure 7 As shown, the method specifically comprises the following steps:

[0100] S410: Train and obtain at least two font feature extraction sub-models in the style type conversion model.

[0101] S420: Train and obtain a style type conversion model.

[0102] In this embodiment, after the training of at least two font feature extraction sub-models is completed, that is, after the multimodal feature extractor in the style type conversion model is obtained, the style type conversion model needs to be trained.

[0103] During the training process, it is first necessary to obtain a second training sample set; wherein the second training sample set includes multiple second training samples, the second training samples include two groups of sub-data to be processed and calibration data, the first group of sub-data to be processed includes a second text image and a second text stroke order corresponding to the text to be trained; the second group of sub-data to be processed includes a third text image and a third text stroke order of the target style type; the calibration data is a fourth text image corresponding to the second text image under the target style type.

[0104] Exemplarily, the first group of sub-data to be processed may include a plurality of copyrighted Song-style characters, and correspondingly, the second character image reflects the effect of these characters in the copyrighted Song-style type, and the second character stroke order represents the stroke order used when these characters are written in the copyrighted Song-style type. It can be understood that the second group of sub-data to be processed may include characters in another font, and correspondingly, the third character image and the third stroke order can also reflect the effect and stroke order of these characters in another font style type, and the embodiments of the present disclosure will not be repeated here.

[0105] After obtaining the second training sample set, further, for each second training sample, the current second training sample is input into the style type conversion model to be trained to obtain the actual text image corresponding to the current second training sample; wherein the style type conversion model to be trained includes a first font feature extraction sub-model, a second feature extraction sub-model, a first decoupling model to be trained, a second decoupling model to be trained, a feature splicing sub-model to be trained, and a feature processing sub-model to be trained. Those skilled in the art should understand that for each of the above-mentioned models to be trained, although the parameters in the model have not been trained, they can still achieve the effects introduced in the second embodiment of the present disclosure to a certain extent.

[0106] Optionally, based on the first font feature extraction sub-model, the second text image and the second text stroke order in the current training sample are processed to obtain the second decoupled text feature of the second text image; and, based on the second font feature extraction sub-model, the third text image and the third text stroke order in the current training sample are processed to obtain the third decoupled text feature of the third text image; based on the first to-be-trained decoupling model, the second decoupled text feature is decoupled to obtain the second style type feature and the second text content feature of the second text image; and, based on the second to-be-trained decoupling model, the third decoupled text feature is decoupled to obtain the third style type feature and the third text content feature of the third text image; based on the to-be-trained feature splicing model, the third style type feature and the second text content feature are spliced to obtain the actual text image corresponding to the current second training sample.

[0107] Taking Figure 3 as an example, when the text image and the text stroke order of the character "永" are used as the second text image and the second text stroke order, they can be input into the multimedia feature extractor (i.e., the trained first font feature extraction sub-model), so as to obtain the second decoupled text feature reflecting the style type feature and the text content feature of the character "永"; when the text image and the writing stroke order of the character "春" are used as the third text image and the third text stroke order, they are also input into the multimedia feature extractor, so as to obtain the third decoupled text feature reflecting the style type feature and the text content feature of the character "春".

[0108] Furthermore, by using the corresponding decoupling network to decouple the second decoupled text feature and the third decoupled text feature respectively, the style type feature and the text content feature of the character "永" can be distinguished, and the style type feature and the text content feature of the character "春" can be distinguished.

[0109] Finally, based on the to-be-trained feature splicing model, the text content feature of the character "永" is spliced with the style type feature of the character "春", that is, the actual text image of the character "永" is obtained. It can be understood that when the model is not trained yet, the character "永" in the actual text image can present the style of the font to which the character "春" belongs to a certain extent. Only after the model is trained, the obtained actual text image will fully present the target style type. It can be understood that the style type corresponding to the style type conversion model matches the target style type in the second group of to-be-processed sub-data.

[0110] Based on at least three preset loss functions in the style type conversion model to be trained, loss processing is performed on the actual text image and the fourth text image, so as to correct the model parameters of the first decoupling model to be trained, the second decoupling model to be trained, the feature splicing sub-model to be trained and the feature processing sub-model to be trained in the style type conversion model to be trained according to the obtained loss values; the convergence of at least three preset loss functions is taken as the training goal to obtain the style type conversion model.

[0111] In actual application, the three preset loss functions may include the reconstruction loss function (Rec Loss), the stroke loss function (Stroke Order Loss) and the adversarial loss function (Adv Loss). Specifically, for the reconstruction loss function, the function is used to intuitively constrain whether the network output meets expectations; for the stroke loss function, a self-designed recurrent neural network (RNN) that can predict stroke order information can be pre-trained, where the number of nodes in the RNN is the maximum number of strokes of Chinese characters, and the features predicted by each node are combined together through a connection function to form a stroke order feature matrix. For the stroke order loss, it can be obtained by calculating the loss value between the actual text image corresponding to the second training sample generated by the network and the stroke order feature matrix of the fourth text image under the target style type. Through the processing of the stroke order loss function, the error rate of the target text obtained in the text generation process can be greatly reduced; for the adversarial loss function, the discriminator structure corresponding to the conditional generative adversarial network (Auxiliary Classifier GAN, ACGAN) based on the auxiliary classifier can be adopted. Specifically, the discriminator not only judges the authenticity of the font finally generated by the model (that is, the font in the actual text image corresponding to the second training sample), but also classifies the type of the finally generated font. By deploying the discriminator in the model, the error rate of the target text obtained by the model is further reduced.

[0112] S430: Obtain the text to be displayed and the pre-selected target style type.

[0113] S440: Convert the text to be displayed into target text corresponding to the target style type.

[0114] S450: Display the target text on the target display interface.

[0115] According to the technical solution of this embodiment, after the font feature extraction sub-model is trained, the style type conversion model is trained based on the second training sample set to obtain a trained style type conversion model; during the training process, at least three preset loss functions are used to optimize the parameters in the model, thereby further reducing the error rate of the target text generated by the model.

[0116] Embodiment 5

[0117] Figure 8 This is a structural block diagram of a text generation device provided in Embodiment 5 of the present disclosure, which can execute the text generation method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. Figure 8 As shown, the device specifically includes: a style type determination module 510 , a target text determination module 520 and a text display module 530 .

[0118] The style type determination module 510 is used to obtain the text to be displayed and the pre-selected target style type.

[0119] The target text determination module 520 is used to convert the to-be-displayed text into a target text corresponding to the target style type; wherein the target text is pre-generated and / or generated in real time based on a style type conversion model.

[0120] The text display module 530 is used to display the target text on the target display interface.

[0121] Optionally, the style type determination module 510 is further configured to determine a target style type to be selected from a style type list when editing of the text to be displayed is detected; wherein the style type list includes style types corresponding to the style type conversion model.

[0122] Optionally, the target text determination module 520 is further used to obtain target text consistent with the text to be displayed from a target text package corresponding to the target style type; wherein the target text package is generated after converting each text into a target font based on the style type conversion model; or, the text to be displayed is input into the style type conversion model to obtain the target text corresponding to the target font.

[0123] On the basis of the above-mentioned technical solutions, the style type conversion model includes a first font feature extraction sub-model, a second font feature extraction sub-model, a first decoupling model connected to the first font feature extraction sub-model, a second decoupling model connected to the second font feature extraction sub-model, a feature splicing sub-model connected to the first decoupling model and the second decoupling model, and a feature processing sub-model; wherein the first font feature extraction sub-model and the second feature extraction sub-model have the same model structure, and are used to determine the text features of each character, and the text features include style type features and text content features; the decoupling model is used to decouple the text features extracted by the font feature extraction sub-model to distinguish between style type features and text content features; the feature splicing sub-model is used to splice the text features extracted by the decoupling model to obtain corresponding text style features; the feature processing sub-model is used to process the text style features to obtain the target text of the text to be displayed under the target style type.

[0124] Optionally, the target text determination module 520 is also used to determine the first to-be-decoupled text feature of the text to be displayed based on the first feature extraction sub-model, and to determine the second to-be-decoupled text feature of the target style text based on the second feature extraction sub-model; wherein the text type of the target style text is consistent with the target style type; based on the first decoupling model, the first to-be-decoupled text feature is processed to obtain the to-be-displayed style type and to-be-displayed content feature of the text to be displayed; and based on the second decoupling model, the second to-be-decoupled text feature is processed to obtain the target style type and target content feature of the target style text; based on the feature splicing model, the content feature to be displayed and the target style type are acquired to obtain the text style feature corresponding to the text to be displayed; based on the feature processing sub-model, the text style feature is processed to obtain the target text corresponding to the text to be displayed under the target style type.

[0125] Based on the above technical solutions, the text generation device also includes a font feature extraction sub-model training module.

[0126] The font feature extraction sub-model training module is used to train and obtain the at least two font feature extraction sub-models in the style type conversion model.

[0127] Based on the above technical solutions, the font feature extraction sub-model training module includes a first training sample set acquisition unit, a first training sample processing unit, a first correction unit, a font feature extraction sub-model determination unit to be used, and a font feature extraction sub-model determination unit.

[0128] The first training sample set acquisition unit is used to acquire the first training sample set; wherein the first training sample set includes multiple first training samples, each of which includes a theoretical text image and theoretical text strokes corresponding to the first training text, and a mask text stroke of the mask part of the theoretical text strokes.

[0129] The first training sample processing unit is used to input the theoretical text image and mask text strokes in the current first training sample into the font feature extraction sub-model to be trained for each first training sample, so as to obtain the actual text image and predicted text strokes corresponding to the current first training sample.

[0130] The first correction unit is used to perform loss processing on actual text images and theoretical text images based on the first preset loss function in the feature extraction sub-model to be trained, and to perform loss processing on the predicted text strokes and theoretical text strokes based on the second preset loss function, so as to correct the model parameters in the font feature extraction sub-model to be trained according to the obtained loss values.

[0131] The font feature extraction sub-model determination unit to be used is used to take the convergence of the first preset loss function and the second preset loss function as a training target to obtain the font feature extraction sub-model to be used.

[0132] The font feature extraction sub-model determining unit is used to obtain the font feature extraction sub-model by eliminating the font feature extraction sub-model to be used.

[0133] On the basis of the above technical solutions, the font feature extraction sub-model to be trained includes a decoding module.

[0134] Optionally, the first training sample processing unit is also used to extract image features corresponding to the theoretical text picture, and compress the image features to obtain a first feature to be used; obtain a second feature to be used by processing the feature vector corresponding to the mask text stroke; obtain a text image feature corresponding to the first feature to be used and an actual stroke feature corresponding to the second feature to be used by performing feature interaction on the first feature to be used and the second feature to be used; obtain the predicted text strokes based on the actual stroke features, and obtain the actual text picture by decoding the text image features based on the decoding module.

[0135] Optionally, the font feature extraction sub-model determination unit is further used to remove the decoding module in the font feature extraction sub-model to be used, so as to obtain the font feature extraction sub-model in the style type conversion model.

[0136] Based on the above technical solutions, the text generation device also includes a style type conversion model training module.

[0137] The style type conversion model training module is used to train and obtain the style type conversion model.

[0138] On the basis of the above technical solutions, the style type conversion model training module includes a second training sample set acquisition unit, a second training sample processing unit, a second correction unit and a style type conversion model determination unit.

[0139] A second training sample set acquisition unit is used to acquire a second training sample set; wherein the second training sample set includes multiple second training samples, the second training samples include two groups of sub-data to be processed and calibration data, the first group of sub-data to be processed includes a second text image and a second text stroke order corresponding to the text to be trained; the second group of sub-data to be processed includes a third text image and a third text stroke order of a target style type; the calibration data is a fourth text image corresponding to the second text image under the target style type.

[0140] The second training sample processing unit is used to input the current second training sample into the style type conversion model to be trained for each second training sample, so as to obtain an actual text image corresponding to the current second training sample; wherein the style type conversion model to be trained includes a first font feature extraction sub-model, a second feature extraction sub-model, a first decoupling model to be trained, a second decoupling model to be trained, a feature splicing sub-model to be trained and a feature processing sub-model to be trained.

[0141] The second correction unit is used to perform loss processing on the actual text image and the fourth text image based on at least three preset loss functions in the style type conversion model to be trained, so as to correct the model parameters of the first decoupling model to be trained, the second decoupling model to be trained, the feature splicing sub-model to be trained, and the feature processing sub-model to be trained in the style type conversion model to be trained according to the obtained loss values.

[0142] The style type conversion model determination unit is used to take the convergence of the at least three preset loss functions as a training target to obtain the style type conversion model.

[0143] Optionally, the second training sample processing unit is also used to process the second text image and the second text stroke order in the current training sample based on the first font feature extraction sub-model to obtain the second text feature to be decoupled of the second text image; and, based on the second font feature extraction sub-model, process the third text image and the third text stroke order in the current training sample to obtain the third text feature to be decoupled of the third text image; based on the first decoupling model to be trained, decouple the second text feature to be decoupled to obtain the second style type feature and the second text content feature of the second text image; and, based on the second decoupling model to be trained, decouple the third text feature to be decoupled to obtain the third style type feature and the third text content feature of the third text image; and, based on the feature splicing model to be trained, splice the third style type feature and the second text content feature to obtain the actual text image corresponding to the current second training sample.

[0144] On the basis of the above technical solutions, the style type corresponding to the style type conversion model matches the target style type in the second group of sub-data to be processed.

[0145] The technical solution provided in this embodiment first obtains the text to be displayed and the pre-selected target style type, then converts the text to be displayed into target text of the target style type, wherein the target text is pre-generated and / or generated in real time based on the style type conversion model, and finally displays the target text on the target display interface. By introducing an artificial intelligence model to generate fonts of a specific style, not only a concise and efficient text design solution is provided, but also the problems of low efficiency, high cost and inability to accurately obtain the desired font in the traditional manual design process are avoided.

[0146] The text generation device provided in the embodiments of the present disclosure can execute the text generation method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0147] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0148] Embodiment 6

[0149] Fig. 9 This is a schematic diagram of the structure of an electronic device provided by Embodiment 6 of the present disclosure. Fig. 9 , which shows an electronic device (eg, Fig. 9The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0150] like Fig. 9 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 606 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An edit / output (I / O) interface 605 is also connected to the bus 604.

[0151] Typically, the following devices may be connected to the I / O interface 605: an editing device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0152] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 606, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0153] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0154] The electronic device provided by the embodiment of the present disclosure and the text generation method provided by the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0155] Embodiment 7

[0156] An embodiment of the present disclosure provides a computer storage medium on which a computer program is stored. When the program is executed by a processor, the text generation method provided in the above embodiment is implemented.

[0157] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0158] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0159] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0160] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0161] Get the text to be displayed and the pre-selected target style type;

[0162] Converting the to-be-displayed text into target text corresponding to the target style type; wherein the target text is pre-generated and / or generated in real time based on a style type conversion model;

[0163] The target text is displayed on a target display interface.

[0164] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0165] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0166] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0167] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0168] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0169] According to one or more embodiments of the present disclosure, [Example 1] provides a text generation method, the method comprising:

[0170] Get the text to be displayed and the pre-selected target style type;

[0171] Converting the to-be-displayed text into target text corresponding to the target style type; wherein the target text is pre-generated and / or generated in real time based on a style type conversion model;

[0172] The target text is displayed on a target display interface.

[0173] According to one or more embodiments of the present disclosure, [Example 2] provides a text generation method, further comprising:

[0174] Optionally, when it is detected that the text to be displayed is edited, a target style type is selected from a style type list;

[0175] The style type list includes style types corresponding to the style type conversion model.

[0176] According to one or more embodiments of the present disclosure, [Example 3] provides a text generation method, further comprising:

[0177] Optionally, a target text that is consistent with the text to be displayed is obtained from a target text package corresponding to the target style type; wherein the target text package is generated by converting each text into a target font based on the style type conversion model; or

[0178] The text to be displayed is input into the style type conversion model to obtain the target text corresponding to the target font.

[0179] According to one or more embodiments of the present disclosure, [Example 4] provides a text generation method, further comprising:

[0180] Optionally, the style type conversion model includes a first font feature extraction sub-model, a second font feature extraction sub-model, a first decoupling model connected to the first font feature extraction sub-model, a second decoupling model connected to the second font feature extraction sub-model, a feature splicing sub-model connected to the first decoupling model and the second decoupling model, and a feature processing sub-model;

[0181] Among them, the model structure of the first font feature extraction submodel and the second feature extraction submodel is the same, and is used to determine the text features of each character, and the text features include style type features and text content features; the decoupling model is used to decouple the text features extracted by the font feature extraction submodel to distinguish style type features from text content features; the feature splicing submodel is used to splice the text features extracted by the decoupling model to obtain corresponding text style features; the feature processing submodel is used to process the text style features to obtain the target text of the text to be displayed under the target style type.

[0182] According to one or more embodiments of the present disclosure, [Example 5] provides a text generation method, further comprising:

[0183] Optionally, a first to-be-decoupled text feature of the to-be-displayed text is determined based on a first feature extraction sub-model, and a second to-be-decoupled text feature of the target style text is determined based on a second feature extraction sub-model; wherein the text type of the target style text is consistent with the target style type;

[0184] Processing the first to-be-decoupled text features based on the first decoupling model to obtain the to-be-displayed style type and to-be-displayed content features of the to-be-displayed text; and processing the second to-be-decoupled text features based on the second decoupling model to obtain the target style type and target content features of the target style text;

[0185] Acquire the to-be-displayed content features and the target style type based on the feature splicing model to obtain text style features corresponding to the to-be-displayed text;

[0186] The text style features are processed based on the feature processing sub-model to obtain the target text corresponding to the to-be-displayed text under the target style type.

[0187] According to one or more embodiments of the present disclosure, [Example 6] provides a text generation method, further comprising:

[0188] Optionally, the at least two font feature extraction sub-models in the style type conversion model are obtained by training;

[0189] The at least two font feature extraction sub-models in the style type conversion model obtained by training include:

[0190] Acquire a first training sample set; wherein the first training sample set includes a plurality of first training samples, each of which includes a theoretical character image and theoretical character strokes corresponding to a first training character, and masked character strokes of the masked theoretical character strokes;

[0191] For each first training sample, the theoretical text image and masked text strokes in the current first training sample are input into the font feature extraction sub-model to be trained, so as to obtain the actual text image and predicted text strokes corresponding to the current first training sample;

[0192] Performing loss processing on the actual text image and the theoretical text image based on the first preset loss function in the feature extraction sub-model to be trained, and performing loss processing on the predicted text strokes and the theoretical text strokes based on the second preset loss function, so as to modify the model parameters in the font feature extraction sub-model to be trained according to the obtained loss values;

[0193] Taking the convergence of the first preset loss function and the second preset loss function as a training target, and obtaining a font feature extraction sub-model to be used;

[0194] The font feature extraction sub-model is obtained by eliminating the font feature extraction sub-model to be used.

[0195] According to one or more embodiments of the present disclosure, [Example 7] provides a text generation method, further comprising:

[0196] Optionally, the font feature extraction sub-model to be trained includes a decoding module;

[0197] Extracting image features corresponding to the theoretical text image, and compressing the image features to obtain first features to be used;

[0198] By processing the feature vector corresponding to the strokes of the mask text, a second feature to be used is obtained;

[0199] By performing feature interaction on the first feature to be used and the second feature to be used, a text image feature corresponding to the first feature to be used and an actual stroke feature corresponding to the second feature to be used are obtained;

[0200] Based on the actual stroke features, the predicted text strokes are obtained, and based on the decoding module, the text image features are decoded to obtain the actual text image.

[0201] According to one or more embodiments of the present disclosure, [Example 8] provides a text generation method, further comprising:

[0202] Optionally, the decoding module in the font feature extraction sub-model to be used is removed to obtain the font feature extraction sub-model in the style type conversion model.

[0203] According to one or more embodiments of the present disclosure, [Example 9] provides a text generation method, further comprising:

[0204] Optionally, training to obtain the style type conversion model;

[0205] The training to obtain the style type conversion model includes:

[0206] Acquire a second training sample set; wherein the second training sample set includes a plurality of second training samples, the second training samples include two groups of sub-data to be processed and calibration data, the first group of sub-data to be processed includes a second character image and a second character stroke order corresponding to the character to be trained; the second group of sub-data to be processed includes a third character image and a third character stroke order of a target style type; the calibration data is a fourth character image corresponding to the second character image under the target style type;

[0207] For each second training sample, the current second training sample is input into the style type conversion model to be trained to obtain an actual text image corresponding to the current second training sample; wherein the style type conversion model to be trained includes a first font feature extraction sub-model, a second feature extraction sub-model, a first decoupling model to be trained, a second decoupling model to be trained, a feature splicing sub-model to be trained, and a feature processing sub-model to be trained;

[0208] Performing loss processing on the actual text image and the fourth text image based on at least three preset loss functions in the style type conversion model to be trained, so as to correct model parameters of the first decoupling model to be trained, the second decoupling model to be trained, the feature splicing sub-model to be trained, and the feature processing sub-model to be trained in the style type conversion model to be trained according to the obtained loss values;

[0209] The convergence of the at least three preset loss functions is taken as a training goal to obtain the style type conversion model.

[0210] According to one or more embodiments of the present disclosure, [Example 10] provides a text generation method, further comprising:

[0211] Optionally, based on the first font feature extraction sub-model, the second character image and the second character stroke order in the current training sample are processed to obtain a second character feature to be decoupled of the second character image; and based on the second font feature extraction sub-model, the third character image and the third character stroke order in the current training sample are processed to obtain a third character feature to be decoupled of the third character image;

[0212] Based on the first decoupling model to be trained, decoupling the second text feature to be decoupled to obtain a second style type feature and a second text content feature of the second text image; and

[0213] Based on the second decoupling model to be trained, decoupling the third text feature to be decoupled to obtain a third style type feature and a third text content feature of the third text image;

[0214] The third style type feature and the second text content feature are spliced ​​based on the feature splicing model to be trained to obtain an actual text image corresponding to the current second training sample.

[0215] According to one or more embodiments of the present disclosure, [Example 11] provides a text generation method, further comprising:

[0216] Optionally, the style type corresponding to the style type conversion model matches the target style type in the second group of sub-data to be processed.

[0217] According to one or more embodiments of the present disclosure, [Example 12] provides a text generation device, including:

[0218] A style type determination module is used to obtain the text to be displayed and the pre-selected target style type;

[0219] A target text determination module, used for converting the to-be-displayed text into a target text corresponding to the target style type; wherein the target text is pre-generated and / or generated in real time based on a style type conversion model;

[0220] The text display module is used to display the target text on the target display interface.

[0221] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0222] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0223] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for generating text, characterized in that: include: Get the text to be displayed and the pre-selected target style type; Converting the to-be-displayed text into target text corresponding to the target style type; wherein the target text is pre-generated and / or generated in real time based on a style type conversion model; the style type conversion model includes a first font feature extraction sub-model and a second font feature extraction sub-model; Displaying the target text on a target display interface; The method further includes: obtaining a first training sample set; wherein the first training sample set includes a plurality of first training samples, each of which includes a theoretical character image and theoretical character strokes corresponding to a first training character, and masked character strokes of a masked portion of the theoretical character strokes; For each first training sample, the theoretical text image and masked text strokes in the current first training sample are input into the font feature extraction sub-model to be trained, so as to obtain the actual text image and predicted text strokes corresponding to the current first training sample; Performing loss processing on the actual text image and the theoretical text image based on the first preset loss function in the feature extraction sub-model to be trained, and performing loss processing on the predicted text strokes and the theoretical text strokes based on the second preset loss function, so as to modify the model parameters in the font feature extraction sub-model to be trained according to the obtained loss values; Taking the convergence of the first preset loss function and the second preset loss function as a training target, and obtaining a font feature extraction sub-model to be used; The font feature extraction sub-model is obtained by eliminating the font feature extraction sub-model to be used.

2. The method according to claim 1, characterized in that The step of obtaining the text to be displayed and the pre-selected target style type includes: When it is detected that the text to be displayed is being edited, determining a target style type selected from a style type list; The style type list includes style types corresponding to the style type conversion model.

3. The method according to claim 1, characterized in that The step of converting the to-be-displayed text into a target text corresponding to the target style type includes: Obtaining target text consistent with the text to be displayed from a target text package corresponding to the target style type; wherein the target text package is generated by converting each text into a target font based on the style type conversion model; or, The text to be displayed is input into the style type conversion model to obtain the target text corresponding to the target font.

4. The method according to any one of claims 1 to 3, characterized in that: The style type conversion model also includes a first decoupling model connected to the first font feature extraction sub-model, a second decoupling model connected to the second font feature extraction sub-model, a feature splicing sub-model connected to the first decoupling model and the second decoupling model, and a feature processing sub-model; Among them, the model structure of the first font feature extraction submodel and the second font feature extraction submodel is the same, and is used to determine the text features of each character, and the text features include style type features and text content features; the decoupling model is used to decouple the text features extracted by the font feature extraction submodel to distinguish style type features from text content features; the feature splicing submodel is used to splice the text features extracted by the decoupling model to obtain corresponding text style features; the feature processing submodel is used to process the text style features to obtain the target text of the text to be displayed under the target style type.

5. The method according to claim 4, characterized in that The target text of the target style type is determined based on the style type conversion model, including: Determining a first to-be-decoupled text feature of the to-be-displayed text based on the first feature extraction sub-model, and determining a second to-be-decoupled text feature of the target style text based on the second feature extraction sub-model; wherein the text type of the target style text is consistent with the target style type; Processing the first to-be-decoupled text features based on the first decoupling model to obtain the to-be-displayed style type and to-be-displayed content features of the to-be-displayed text; and processing the second to-be-decoupled text features based on the second decoupling model to obtain the target style type and target content features of the target style text; Acquire the to-be-displayed content features and the target style type based on the feature splicing model to obtain text style features corresponding to the to-be-displayed text; The text style features are processed based on the feature processing sub-model to obtain the target text corresponding to the to-be-displayed text under the target style type.

6. The method according to claim 1, characterized in that The sub-model for extracting the character features to be trained includes a decoding module, and the theoretical character image and masked character strokes in the current first training sample are input into the sub-model for extracting the character features to be trained to obtain the actual character image and predicted character strokes corresponding to the current first training sample, including: Extracting image features corresponding to the theoretical text image, and compressing the image features to obtain first features to be used; By processing the feature vector corresponding to the strokes of the mask text, a second feature to be used is obtained; By performing feature interaction on the first feature to be used and the second feature to be used, a text image feature corresponding to the first feature to be used and an actual stroke feature corresponding to the second feature to be used are obtained; Based on the actual stroke features, the predicted text strokes are obtained, and based on the decoding module, the text image features are decoded to obtain the actual text image.

7. The method according to claim 6, characterized in that The method of obtaining the font feature extraction sub-model by eliminating the font feature extraction sub-model to be used includes: The decoding module in the font feature extraction sub-model to be used is removed to obtain the font feature extraction sub-model in the style type conversion model.

8. The method according to claim 1, characterized in that Also includes: Training to obtain the style type conversion model; The training to obtain the style type conversion model includes: Acquire a second training sample set; wherein the second training sample set includes a plurality of second training samples, the second training samples include two groups of sub-data to be processed and calibration data, the first group of sub-data to be processed includes a second character image and a second character stroke order corresponding to the character to be trained; the second group of sub-data to be processed includes a third character image and a third character stroke order of a target style type; the calibration data is a fourth character image corresponding to the second character image under the target style type; For each second training sample, the current second training sample is input into the style type conversion model to be trained to obtain an actual text image corresponding to the current second training sample; wherein the style type conversion model to be trained includes a first font feature extraction sub-model, a second feature extraction sub-model, a first decoupling model to be trained, a second decoupling model to be trained, a feature splicing sub-model to be trained, and a feature processing sub-model to be trained; Performing loss processing on the actual text image and the fourth text image based on at least three preset loss functions in the style type conversion model to be trained, so as to correct model parameters of the first decoupling model to be trained, the second decoupling model to be trained, the feature splicing sub-model to be trained, and the feature processing sub-model to be trained in the style type conversion model to be trained according to the obtained loss values; The convergence of the at least three preset loss functions is taken as a training goal to obtain the style type conversion model.

9. The method according to claim 8, characterized in that The step of inputting the current second training sample into the style type conversion model to be trained to obtain an actual text image corresponding to the current second training sample includes: Based on the first font feature extraction sub-model, the second character image and the second character stroke order in the current second training sample are processed to obtain a second character feature to be decoupled of the second character image; and based on the second font feature extraction sub-model, the third character image and the third character stroke order in the current second training sample are processed to obtain a third character feature to be decoupled of the third character image; Based on the first decoupling model to be trained, decoupling the second text feature to be decoupled to obtain a second style type feature and a second text content feature of the second text image; and Based on the second decoupling model to be trained, decoupling the third text feature to be decoupled to obtain a third style type feature and a third text content feature of the third text image; The third style type feature and the second text content feature are spliced ​​based on the feature splicing model to be trained to obtain an actual text image corresponding to the current second training sample.

10. The method according to claim 8, characterized in that The style type corresponding to the style type conversion model matches the target style type in the second group of sub-data to be processed.

11. A text generation device, characterized in that: include: A style type determination module is used to obtain the text to be displayed and the pre-selected target style type; A target text determination module, used for converting the to-be-displayed text into a target text corresponding to the target style type; wherein the target text is pre-generated and / or generated in real time based on a style type conversion model; the style type conversion model includes a first font feature extraction sub-model and a second font feature extraction sub-model; A text display module, used for displaying the target text on a target display interface; The text generation device also includes a font feature extraction sub-model training module; wherein the font feature extraction sub-model training module includes: a first training sample set acquisition unit, used to acquire a first training sample set; wherein the first training sample set includes multiple first training samples, each of which includes a theoretical text image and theoretical text strokes corresponding to the first training text, and mask text strokes of the masked part of the theoretical text strokes; a first training sample processing unit, used to input the theoretical text image and mask text strokes in the current first training sample into the font feature extraction sub-model to be trained for each first training sample, and obtain the actual text image and predicted text corresponding to the current first training sample. character strokes; a first correction unit, used to perform loss processing on actual character images and theoretical character images based on a first preset loss function in the feature extraction submodel to be trained, and to perform loss processing on the predicted character strokes and theoretical character strokes based on a second preset loss function, so as to correct the model parameters in the font feature extraction submodel to be trained according to the obtained loss values; a determination unit for the font feature extraction submodel to be used, used to take the convergence of the first preset loss function and the second preset loss function as a training target to obtain the font feature extraction submodel to be used; a font feature extraction submodel determination unit, used to obtain the font feature extraction submodel by eliminating the font feature extraction submodel to be used.

12. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the text generation method as described in any one of claims 1-10.

13. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to execute the text generation method as described in any one of claims 1 to 10 when executed by a computer processor.

Images