Font generation method, device, electronic equipment and system
By acquiring standard stroke sequences and handwritten character images, and utilizing style feature extraction models and decoders, personalized fonts are generated, solving the problem that existing technologies cannot generate personalized fonts and achieving more accurate expression of writing style and enhanced personalization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- IFLYTEK CO LTD
- Filing Date
- 2022-12-28
- Publication Date
- 2026-06-05
AI Technical Summary
In the existing technology, electronic devices cannot generate personalized fonts, and users cannot use their own handwritten fonts or favorite fonts to communicate and interact.
By acquiring the standard stroke sequence and handwritten character image of the text to be generated, a style feature extraction model and decoder are used to extract and fuse writing style features and text content features to generate a personalized font.
It enables the generation of personalized fonts, improves the accuracy of handwritten image writing style characteristics, makes the generated fonts better reflect the user's writing style, and enhances the degree of personalization.
Smart Images
Figure CN115880701B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text processing technology, and in particular to a font generation method, apparatus, electronic device, and system. Background Technology
[0002] With the development of computer technology, various electronic devices are gradually becoming more intelligent, bringing convenience to people's lives. In the process of human-computer interaction, electronic devices can transmit information via text.
[0003] In related technologies, text information can be input via keyboard or handwriting. However, regardless of the method, the font used is a standard input method font, such as Song, Kai, or Lishu, and users cannot use their own handwritten fonts or preferred fonts. With the development of electronic device technology, people's demand for personalization and differentiation is increasing. More and more people want to use personalized fonts for communication on electronic devices. Therefore, how to generate personalized fonts is a pressing technical problem that needs to be solved. Summary of the Invention
[0004] This invention provides a font generation method, apparatus, electronic device, and system to solve the problem of low personalization of fonts in the prior art and to realize the generation of personalized fonts.
[0005] This invention provides a font generation method, comprising:
[0006] Obtain the standard stroke sequence of the text to be generated, and extract the first text content feature of the standard stroke sequence;
[0007] A first single-character image is obtained from a handwritten character image, and the first single-character image is input into a style feature extraction model to obtain a first writing style feature output by the style feature extraction model; the style feature extraction model is used to characterize the writing style features of the text.
[0008] Extract the second text content feature from the first single character image, and remove the second text content feature from the first writing style feature to obtain the second writing style feature with content information removed;
[0009] The first text content feature and the second writing style feature are input into the decoder to obtain the first target style text corresponding to the text to be generated, which is output by the decoder. The decoder is used to fuse the text content feature and the writing style feature.
[0010] According to a font generation method provided by the present invention, the style feature extraction model is trained based on the following steps:
[0011] Obtain the first single-character sample image from the handwritten character sample images;
[0012] Anchor images, positive samples, and negative samples are determined from the first single-character sample images to obtain a contrastive learning sample set; wherein the anchor images and the positive samples are single-character sample images belonging to the same writer, and are different from the writers of the negative samples;
[0013] The first initial neural network model is trained using the contrastive learning sample set to obtain the style feature extraction model.
[0014] According to a font generation method provided by the present invention, the decoder is trained based on the following steps:
[0015] A second single-character sample image is obtained, and the second single-character sample image is input into the style feature extraction model to obtain the sample writing style features output by the style feature extraction model;
[0016] Obtain the standard stroke sequence of a single character in the second single-character sample image to obtain a standard stroke sequence sample, and extract the sample text content features of the standard stroke sequence sample.
[0017] The initial decoder is trained based on the sample writing style features and the sample text content features to obtain the decoder.
[0018] According to a font generation method provided by the present invention, the step of training an initial decoder based on the sample handwriting style features and the sample text content features to obtain the decoder includes:
[0019] The initial decoder and the style feature extraction model are jointly trained based on the sample writing style features and the sample text content features to obtain the decoder;
[0020] The style feature extraction model is updated during joint training.
[0021] According to a font generation method provided by the present invention, the step of extracting the second text content features of the first single-character image includes:
[0022] The first single-character image is input into the content recognition model to obtain the second text content feature output by the content recognition model;
[0023] The content recognition model is obtained by training a second initial neural network model based on the third single-character sample image and the label data corresponding to the third single-character sample image.
[0024] A font generation method according to the present invention further includes:
[0025] Obtain the second single-character image from the reference character image, and input the second single-character image into the twin model of the style feature extraction model to obtain the third writing style feature output by the twin model;
[0026] The third handwriting style feature is weighted and averaged with the second handwriting style feature to obtain the fourth handwriting style feature;
[0027] The first text content feature and the fourth writing style feature are input into the decoder to obtain the second target style text corresponding to the text to be generated output by the decoder. The second target style text is represented by the fused writing style of the handwritten character image and the reference character image.
[0028] The present invention also provides a font generation apparatus, comprising:
[0029] The first acquisition module is used to acquire the standard stroke sequence of the text to be generated;
[0030] The first feature extraction module is used to extract the first text content features of the standard stroke sequence;
[0031] The second acquisition module is used to acquire the first single character image from the handwritten character image;
[0032] The second feature extraction module is used to input the first single-character image into the style feature extraction model to obtain the first writing style feature output by the style feature extraction model; the style feature extraction model is used to characterize the writing style features of the text.
[0033] The third feature extraction module is used to extract the second text content feature of the first single character image and remove the second text content feature from the first writing style feature to obtain the second writing style feature with content information removed.
[0034] The decoding module is used to input the first text content feature and the second writing style feature into the decoder to obtain the first target style text corresponding to the text to be generated output by the decoder. The decoder is used to fuse the text content feature and the writing style feature.
[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the font generation method as described above.
[0036] A font generation device according to the present invention further includes a camera and / or a touch screen communicatively connected to the processor;
[0037] The camera is used to capture images of handwritten characters;
[0038] The touchscreen is used to receive handwriting input and, in response to the handwriting input, generate a handwritten character image.
[0039] The present invention also provides a font generation system, including the electronic device described above and an image acquisition device and / or handwriting screen communicatively connected to the electronic device;
[0040] The image acquisition device is used to acquire handwritten text images under the control of the electronic device;
[0041] The handwriting screen is used to receive handwriting input, generate a handwritten character image in response to the handwriting input, and send the handwritten character image to the electronic device.
[0042] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the font generation method as described above.
[0043] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the font generation methods described above.
[0044] The font generation method, apparatus, electronic device, and system provided by this invention first acquire the standard stroke sequence of the character to be generated and extract the first text content feature of the standard stroke sequence; then, acquire the first single-character image from the handwritten character image and input the first single-character image into a style feature extraction model used to characterize the writing style features of the text, obtaining the first writing style feature output by the style feature extraction model; next, extract the second text content feature of the first single-character image and remove the second text content feature from the first writing style feature, obtaining the second writing style feature with content information removed; then, input the first text content feature and the second writing style feature into a decoder used to fuse the text content feature and the writing style feature, obtaining the first target style text corresponding to the character to be generated output by the decoder. In this way, the generated first target style text will contain the writing style of the handwritten character image and can be represented by the writing style of the handwritten character image, thereby realizing the generation of personalized fonts. Moreover, in the font generation process, the text content feature is removed from the writing style feature of the handwritten character image, avoiding interference from the text content feature, improving the accuracy of the writing style feature of the handwritten character image, making the generated first target style text better reflect the writing style of the handwritten character, further improving the degree of personalization. Attached Figure Description
[0045] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0046] Figure 1 This is one of the flowcharts illustrating the font generation method provided in this embodiment of the invention;
[0047] Figure 2 This is a flowchart illustrating the training method of the style feature extraction model in an embodiment of the present invention;
[0048] Figure 3 This is a schematic diagram illustrating the training principle of the style feature extraction model in this embodiment of the invention;
[0049] Figure 4 This is one of the flowcharts illustrating the training method of the decoder in an embodiment of the present invention;
[0050] Figure 5 This is a schematic diagram of the training principle of the decoder in an embodiment of the present invention;
[0051] Figure 6 This is the second flowchart illustrating the training method of the decoder in this embodiment of the invention;
[0052] Figure 7 This is a second flowchart illustrating the font generation method provided in this embodiment of the invention;
[0053] Figure 8 This is a schematic diagram illustrating the principle of personalized font generation that integrates personal writing style with other writing styles in an embodiment of the present invention;
[0054] Figure 9 This is a schematic diagram of the font generation device provided in an embodiment of the present invention;
[0055] Figure 10 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention;
[0056] Figure 11 This is a schematic diagram of the font generation system provided in an embodiment of the present invention. Detailed Implementation
[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0058] It should be noted that the serial numbers assigned to the objects described in this invention, such as "first" and "second", are only used to distinguish the objects being described and do not have any sequential or technical meaning.
[0059] The following is combined with Figures 1-8 The font generation method of the present invention is described below. This font generation method can be applied to electronic devices such as terminal devices or servers. Terminal devices may include mobile phones, computers, in-vehicle devices, tablet computers, wearable devices, smart home devices, etc.; servers may include independent servers, cluster servers, or cloud servers, etc. This font generation method can also be applied to a font generation device installed in electronic devices such as terminal devices or servers, which can be implemented through software, hardware, or a combination of both. The following description uses the application of this font generation method to an electronic device as an example.
[0060] Figure 1 An exemplary schematic diagram of one of the font generation methods provided in an embodiment of the present invention is shown below, with reference to... Figure 1 As shown, the font generation method may include the following steps 110 to 140.
[0061] Step 110: Obtain the standard stroke sequence of the text to be generated, and extract the first text content feature of the standard stroke sequence.
[0062] A standard stroke sequence is a set of coordinate points formed by digitizing standard characters. For example, each stroke in a standard character can be separated into standard strokes, sorted in the correct order, and each standard stroke can be digitized to form a set of coordinate points with a directional order, thus obtaining a standard stroke sequence.
[0063] The text to be generated is the text for which a personalized font needs to be created. The electronic device can directly read the standard stroke sequence of the text to be generated from a character library, input this standard stroke sequence into a content extractor, and obtain the first text content feature output by the content extractor. This content extractor is used to extract the text content features of the standard stroke sequence. For example, this content extractor can be a Long Short-Term Memory (LSMT) encoder, which can encode the input standard stroke sequence into a text content feature vector.
[0064] Step 120: Obtain the first single character image from the handwritten character image, and input the first single character image into the style feature extraction model to obtain the first writing style feature output by the style feature extraction model.
[0065] Electronic devices can acquire handwritten images from images stored in local or external storage devices; or, electronic devices can capture handwritten text on paper or other writing media by their own camera or a camera device connected to them, and obtain handwritten images; or, electronic devices can also receive handwritten input from users on their touch screen or a handwriting screen connected to them, and generate handwritten images.
[0066] After acquiring a handwritten character image, the electronic device can perform single-character detection on the handwritten character image to obtain a first single-character image. Then, the first single-character image is input into a style feature extraction model. The style feature extraction model extracts the writing style features of the characters in the first single-character image and outputs a first writing style feature. The electronic device acquires the first writing style feature.
[0067] The style feature extraction model is used to extract the writing style features of text, and it can be trained by comparative learning on single-character sample images. Single-character detection in the images can be performed using object detection algorithms.
[0068] Step 130: Extract the second text content features from the first single character image, and remove the second text content features from the first writing style features to obtain the second writing style features with content information removed.
[0069] After an electronic device acquires the first single-character image from a handwritten text image, it can extract the text content features of the first single-character image using a content recognition model to obtain the second text content features. The content recognition model is used to identify the content features of the text in the single-character image. The second text content features characterize the content information of the first single-character image, that is, they represent what text information the first single-character image expresses.
[0070] For example, extracting the second text content features of the first single-character image may include: inputting the first single-character image into a content recognition model to obtain the second text content features output by the content recognition model; wherein, the content recognition model is obtained by training a second initial neural network model based on the third single-character sample image and the label data corresponding to the third single-character sample image.
[0071] The second initial neural network model may include, but is not limited to, convolutional neural networks (CNN), recurrent neural networks (RNN), backpropagation neural networks, or deep neural networks (DNN).
[0072] After obtaining the second text content feature of the first single character image, the first writing style feature can be decoupled from the content, and the second text content feature can be removed from the first writing style feature to obtain the second writing style feature with content information removed.
[0073] Since the goal of the content recognition model is to identify the content of individual characters, the features extracted by this model are highly correlated with the image content but unrelated to the writing style. Therefore, subtracting the second text content feature from the first writing style feature extracted in step 120 yields the writing style feature removed from the content information, i.e., the second writing style feature. Because this second writing style feature removes the interference of content information, it can more accurately represent the writing style information of the handwritten character image, improving the accuracy and precision of writing style feature extraction.
[0074] Step 140: Input the first text content feature and the second writing style feature into the decoder to obtain the first target style text corresponding to the text to be generated output by the decoder.
[0075] The decoder fuses text content features and handwriting style features, assigning handwriting style characteristics to the text content features so that the text corresponding to the text content features is displayed in the handwriting style represented by the handwriting style features. In this way, the first target style text corresponding to the text to be generated will be represented in the handwriting style of a handwritten image, enhancing the personalized style of the font. Generating personalized fonts makes information dissemination more vivid and intuitive. Compared to standardized fonts, personalized fonts can render text with a unique handwriting style, more flexibly expressing the writer's style and emotions; moreover, the visual effect of personalized fonts also allows the writer to receive more attention on various social media platforms.
[0076] For example, the decoder may include an LSTM decoder or other trained sequence-to-sequence model decoder.
[0077] The font generation method provided in this invention first obtains the standard stroke sequence of the character to be generated and extracts the first text content feature of the standard stroke sequence; then, it obtains the first single character image from the handwritten character image and inputs the first single character image into a style feature extraction model used to characterize the writing style features of the text, obtaining the first writing style feature output by the style feature extraction model; next, it extracts the second text content feature of the first single character image and removes the second text content feature from the first writing style feature, obtaining the second writing style feature with content information removed; then, it inputs the first text content feature and the second writing style feature into a decoder used to fuse the text content feature and the writing style feature, obtaining the first target style text corresponding to the character to be generated output by the decoder. In this way, the generated first target style text will contain the writing style of the handwritten character image and can be represented by the writing style of the handwritten character image, thereby realizing the generation of personalized fonts. Moreover, in the font generation process, the text content feature is removed from the writing style feature of the handwritten character image, avoiding interference from the text content feature, improving the accuracy of the writing style feature of the handwritten character image, making the generated first target style text better reflect the writing style of the handwritten character, further improving the personalization of the font.
[0078] With the font generation method provided by the embodiments of the present invention, users only need to write a small amount of handwritten text, such as one or two sentences, and the electronic device can generate any text content containing the user's personal writing style based on a small number of personal handwritten text images.
[0079] based on Figure 1 The font generation method corresponding to the embodiment, Figure 2 An exemplary flowchart illustrating the training method for a style feature extraction model is provided. Figure 2 As shown, the training method for the style feature extraction model may include the following steps 210 to 230.
[0080] Step 210: Obtain the first single-character sample image from the handwritten character sample images.
[0081] The handwritten sample images are selected from images of text written by different writers. Image acquisition devices, such as cameras, can be used to capture images of text written by different people to obtain handwritten sample images. Then, single character images can be detected from these handwritten sample images to obtain the first single character sample image. The first single character sample image includes text written in various styles by different people.
[0082] Step 220: Determine the anchor image, positive sample, and negative sample from the first single-character sample image to obtain the contrastive learning sample set.
[0083] Among them, the anchor point image and the positive sample are single-character sample images belonging to the same writer, and are written by a different writer than the negative sample.
[0084] For example, Figure 3 An exemplary diagram illustrating the training principle of a style feature extraction model is shown, combined with... Figure 3 As shown, handwritten character sample images A and B come from two different writers. Single-character detection is performed on handwritten character sample images A and B to obtain the first single-character sample image. Next, a single-character image is randomly selected from the first single-character sample image as an anchor image; for example, the selected anchor image is single-character image 3 from handwritten character sample image B. Then, a single-character image, also from handwritten character sample image B, is randomly selected from the first single-character sample image as a positive sample, and a single-character image from handwritten character sample image A is selected as a negative sample; for example, the selected positive sample is single-character image 2, and the negative sample is single-character image 1. Single-character images 1, 2, and 3 form a triplet. In this way, multiple triplets can be determined from the first single-character sample image, forming a contrastive learning sample set.
[0085] Step 230: Perform contrastive learning training on the first initial neural network model based on the contrastive learning sample set to obtain the style feature extraction model.
[0086] The initial neural network model can include, but is not limited to, neural network models such as CNN, RNN, or DNN.
[0087] Combination Figure 3 After obtaining the contrastive learning sample set, it is input into the first initial neural network model for feature extraction. For each triplet, a triplet loss function is used for contrastive learning training to minimize the distance between the feature representations of the anchor image and positive samples, while maximizing the distance between the feature representations of the anchor image and negative samples. This training process yields a style feature extraction model. The training process can be based on a large number of samples, improving the style feature extraction model's ability to represent handwriting style features.
[0088] After obtaining the style feature extraction model, a decoder can be trained based on the style feature extraction model. Figure 4 An exemplary schematic diagram of one of the training methods for the decoder in an embodiment of the present invention is shown, with reference to... Figure 4 As shown, the decoder can be trained through the following steps 410 to 430.
[0089] Step 410: Obtain the second single-character sample image and input the second single-character sample image into the style feature extraction model to obtain the sample writing style features output by the style feature extraction model.
[0090] In the decoder training stage, the second single-character sample image can be taken from handwritten character images written by different writers. The single-character detection can be performed on the handwritten character images written by different writers to obtain the second single-character sample image. The electronic device obtains the second single-character sample image and inputs the second single-character sample image into the style feature extraction model. The writing style feature of the second single-character sample image is extracted through the style feature extraction model to obtain the sample writing style feature output by the style feature extraction model. The style feature extraction model can be the one trained by using Figure 2 the method of the corresponding embodiment.
[0091] Step 420: Obtain the standard stroke sequence of the single character in the second single-character sample image to obtain the standard stroke sequence sample, and extract the sample text content feature of the standard stroke sequence sample.
[0092] In the decoder training stage, the second single-character sample image can be a small number of single-character images. For example, Figure 5 An exemplary schematic diagram of the training principle of the decoder is shown. Combining Figure 5 as shown, taking the single-character image 4 in the single-character sample image as an example, assuming that the single character in the single-character image 4 is "个", the standard stroke sequence of "个" can be obtained, and the standard stroke sequence is input into the content extractor to obtain the text content feature corresponding to the character "个" output by the content extractor. The content extractor therein can be the content extractor described in the above step 110.
[0093] It can be understood that for each single character in the single-character sample image, the same method as that of the character "个" can be used for processing.
[0094] Step 430: Train the initial decoder based on the sample writing style feature and the sample text content feature to obtain the decoder.
[0095] The initial decoder can include, for example, an initial LSMT network model or other sequence-to-sequence models. Correspondingly, the decoder obtained after training the initial decoder can include an LSMT decoder or other sequence-to-sequence model decoders. Among them, the LSMT decoder can decode the input feature vector into sequence data for output.
[0096] For example, combining Figure 5, the single - character image 4 can be input into the style feature extraction model to obtain the writing style features of the single character "个" output by the style feature extraction model. After the standard stroke sequence of "个" is subjected to content extraction by the content extractor, the text content features corresponding to the character "个" are obtained. Then, taking this writing style feature as the sample writing style feature and this text content feature as the sample text content feature, input them into the initial decoder, train the initial decoder, and predict the stroke sequence corresponding to the character "个" in the single - character image 4 through the initial decoder, that is, predict the personalized font of "个". In this way, through the same stroke sequence prediction training on a small number of characters written by multiple different people, a trained decoder can be obtained.
[0097] Figure 4 The method corresponding to the embodiment can train the decoder based on the trained style feature extraction model. In another optional embodiment, it is also possible to Figure 2 use the style feature extraction model trained according to the method corresponding to the embodiment as the pre - trained model for style feature extraction. In the decoder training stage, call this pre - trained model and jointly train the initial decoder and this pre - trained model according to the method Figure 4 corresponding to the embodiment.
[0098] Based on this, training the initial decoder based on the sample writing style features and sample text content features to obtain a decoder can include: jointly training the initial decoder and the style feature extraction model based on the sample writing style features and sample text content features to obtain a decoder; where the style feature extraction model is updated during the joint training process.
[0099] In this way, the style feature extraction model can be further adjusted during the joint training process to improve the personalization degree of the font. Moreover, Figure 2 using the style feature extraction model trained according to the method corresponding to the embodiment as the pre - trained model for style feature extraction can be applied to any scenario that requires style feature extraction. In these scenarios, this pre - trained model can be directly called, or with the help of this pre - trained model, adaptive retraining for the current scenario can be carried out to quickly obtain the model required for the current scenario, improving the efficiency of model training, and having strong portability and adaptability.
[0100] It can be understood that in Figure 1 the font generation method corresponding to the embodiment, the style feature extraction model used can be the style feature extraction model trained according to the method Figure 2 corresponding to the embodiment, or the style feature extraction model obtained after joint training in the decoder training stage.
[0101] Based on Figure 4In one example embodiment of the method corresponding to the embodiments, the content information of the writing style features of the second single-character sample image can be removed before it is used for the training of the decoder. Figure 6 This is an exemplary second flowchart illustrating the training method of the decoder in an embodiment of the present invention. (Refer to...) Figure 6 As shown, the decoder can be trained through the following steps 610 to 640.
[0102] Step 610: Obtain the second single-character sample image and input the second single-character sample image into the style feature extraction model to obtain the first sample writing style feature output by the style feature extraction model.
[0103] Step 620: Extract the first sample text content features from the second single character image, and remove the first sample text content features from the first sample writing style features to obtain the second sample writing style features with content information removed.
[0104] Step 630: Obtain the standard stroke sequence of the single character in the second single character sample image, obtain the standard stroke sequence sample, and extract the second sample text content features of the standard stroke sequence sample.
[0105] Step 640: Train the initial decoder based on the handwriting style features and text content features of the second sample to obtain the decoder.
[0106] In this way, during the training phase of the decoder, by removing the text content features from the writing style features of the second single-character sample image, the influence of the content information of the single character in the second single-character sample image on the writing style can be avoided, thereby improving the accuracy and precision of writing style feature extraction, which in turn improves the decoding effect of the trained decoder and further enhances the personalization of the generated font.
[0107] Based on the methods of the above embodiments, in one example embodiment of the present invention, personal style can also be integrated with other styles (such as the style of beautiful characters) to form a personalized font that integrates personal style and other styles, thus forming a richer personalized font library.
[0108] Based on this Figure 7 This is an exemplary second flowchart illustrating the font generation method provided in an embodiment of the present invention. (Refer to...) Figure 7 As shown, the font generation method may include the following steps 710 to 730.
[0109] Step 710: Obtain the second single-character image from the reference character image, and input the second single-character image into the twin model of the style feature extraction model to obtain the third writing style feature output by the twin model.
[0110] Imagery for reference refers to text images in styles other than one's own handwriting. It can be images of text written in an attractive style by others, whose writing style can be borrowed and integrated into one's own handwriting. For example, an electronic device can take a picture of attractively written text by another person using a camera to obtain an imagery for reference; alternatively, it can download attractively written text images that can be referenced from the internet or retrieved from a storage device.
[0111] Once the reference character image is obtained, the electronic device can perform single-character detection on the reference character image to obtain the second single-character image in the reference character image. Then, the second single-character image is input into the twin model of the style feature extraction model to obtain the third writing style feature output by the twin model. The third writing style feature can characterize the writing style of the reference character image.
[0112] Among them, the twin model is the same model as the style feature extraction model, with the same model structure and parameters, and can achieve the same function as the style feature extraction model. In other words, the twin model is a style feature extraction model.
[0113] Step 720: Take a weighted average of the third handwriting style feature and the second handwriting style feature to obtain the fourth handwriting style feature.
[0114] The second writing style feature can be obtained according to step 130 above, that is, extracting the second text content feature of the first single character image in the handwritten character image, and removing the second text content feature from the first writing style feature of the first single character image to obtain the second writing style feature with content information removed.
[0115] After extracting the third writing style feature from the second single-character image in the reference character image, the third writing style feature is weighted and averaged with the second writing style feature to obtain the fourth writing style feature. At this point, the fourth writing style feature simultaneously includes the writing style of the individual's handwritten character image and the writing style of the reference character image.
[0116] Step 730: Input the first text content feature and the fourth writing style feature into the decoder to obtain the second target style text corresponding to the text to be generated output by the decoder.
[0117] The first text content feature is the text content feature extracted from the standard stroke sequence of the text to be generated, which can be obtained according to step 110 above.
[0118] After obtaining the first text content feature and the fourth writing style feature, the electronic device inputs these features into a decoder for feature fusion, resulting in the second target style text corresponding to the text to be generated, as output by the decoder. At this point, the obtained second target style text is represented using a fused writing style of the handwritten image and the referenced image.
[0119] based on Figure 7 In one example embodiment, the font generation method of the corresponding embodiment can further extract the text content features of the second single character image in the reference character image to obtain the third text content features; remove the third text content features from the third writing style features to obtain the fifth writing style features with content information removed; perform a weighted average of the fifth writing style features and the second writing style features to obtain the fourth writing style features; and then input the first text content features and the fourth writing style features into the decoder to obtain the second target style text corresponding to the text to be generated output by the decoder.
[0120] In this way, by removing text content features from the third writing style features, the influence of the content information of individual characters in the second character image on the writing style can be avoided, improving the accuracy and precision of writing style feature extraction, and thus improving the personalization of the second target style text corresponding to the text to be generated.
[0121] based on Figure 7 The font generation method in the corresponding embodiment, taking an LSTM decoder as an example, Figure 8 An exemplary diagram illustrates the principle of personalized font generation that integrates personal handwriting style with other handwriting styles. (Refer to...) Figure 8 As shown, individual handwritten characters and referenced characters are detected separately. The resulting individual character images are then input into a style feature extraction model for handwriting style feature extraction. Simultaneously, text content features are extracted from each individual character image. Content decoupling is then performed: the text content features of the first individual character image are subtracted from the handwriting style features of the first individual character image, and the text content features of the second individual character image are subtracted from the handwriting style features of the second individual character image. These result in individual handwriting style features and referenced character handwriting style features. These two sets of handwriting style features are then weighted and averaged, and the resulting weighted average handwriting style features are input into an LSTM decoder. For the personalized character to be generated (i.e., the character to be generated), text content features are extracted using a content extractor and input into the LSTM decoder. The LSTM decoder then fuses the received text content features and handwriting style features using an autoregressive approach, decoding and outputting the personalized stroke sequence corresponding to the character to be generated, forming a personalized font.
[0122] In this way, the generated personalized fonts contain both personal writing styles and the writing styles of borrowed characters, which enhances the personalization of the fonts and enables the creation of a richer personalized font library.
[0123] The font generation apparatus provided by the present invention will be described below. The font generation apparatus described below can be referred to in correspondence with the font generation method described above.
[0124] Figure 9 An exemplary schematic diagram of the font generation device provided in an embodiment of the present invention is shown, with reference to... Figure 9 As shown, the font generation device 900 may include: a first acquisition module 910, used to acquire a standard stroke sequence of the text to be generated; a first feature extraction module 920, used to extract a first text content feature from the standard stroke sequence; a second acquisition module 930, used to acquire a first single character image from a handwritten character image; a second feature extraction module 940, used to input the first single character image into a style feature extraction model to obtain a first writing style feature output by the style feature extraction model, wherein the style feature extraction model is used to characterize the writing style features of the text; a third feature extraction module 950, used to extract a second text content feature from the first single character image and remove the second text content feature from the first writing style feature to obtain a second writing style feature with content information removed; and a decoding module 960, used to input the first text content feature and the second writing style feature into a decoder to obtain a first target style text corresponding to the text to be generated output by the decoder.
[0125] In one example embodiment, the font generation device 900 may further include: a first training module for training a style feature extraction model. Specifically, the first training module may include: an acquisition unit for acquiring a first single-character sample image from handwritten character sample images; a determination unit for determining anchor images, positive samples, and negative samples from the first single-character sample images to obtain a contrastive learning sample set, wherein the anchor images and positive samples are single-character sample images belonging to the same writer, and are different from the writers of the negative samples; and a first training unit for performing contrastive learning training on a first initial neural network model based on the contrastive learning sample set to obtain a style feature extraction model.
[0126] In one example embodiment, the font generation device 900 may further include: a second training module for training the decoder. Specifically, the second training module may include: a first feature extraction unit for acquiring a second single-character sample image and inputting the second single-character sample image into a style feature extraction model to obtain sample writing style features output by the style feature extraction model; a second feature extraction unit for acquiring the standard stroke sequence of a single character in the second single-character sample image to obtain standard stroke sequence samples and extracting sample text content features of the standard stroke sequence samples; and a second training unit for training the initial decoder based on the sample writing style features and sample text content features to obtain the decoder.
[0127] In one example embodiment, the second training unit may be specifically used to jointly train the initial decoder and the style feature extraction model based on the sample writing style features and sample text content features to obtain the decoder; wherein the style feature extraction model is updated during the joint training process.
[0128] In one example embodiment, the third feature extraction module 950 can be specifically used to input the first single-character image into the content recognition model to obtain the second text content features output by the content recognition model; wherein, the content recognition model is obtained by training the second initial neural network model based on the third single-character sample image and the label data corresponding to the third single-character sample image.
[0129] In one example embodiment, the font generation device 900 may further include: a third acquisition module for acquiring a second single-character image from a reference character image; a fourth feature extraction module for inputting the second single-character image into a twin model of a style feature extraction model to obtain a third writing style feature output by the twin model; and a calculation module for performing a weighted average of the third writing style feature and the second writing style feature to obtain a fourth writing style feature. The decoding module 960 may further be used to input the first text content feature and the fourth writing style feature into a decoder to obtain a second target style text corresponding to the text to be generated output by the decoder, wherein the second target style text is represented by a fused writing style of the handwritten character image and the reference character image.
[0130] Figure 10 An example is a schematic diagram of the structure of an electronic device, such as... Figure 10As shown, the electronic device may include a processor 1010, a communication interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communication interface 1020, and the memory 1030 can communicate with each other through the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute the font generation method provided in the above-described method embodiments. This method may include, for example, obtaining a standard stroke sequence of the text to be generated and extracting a first text content feature of the standard stroke sequence; obtaining a first single character image from a handwritten character image and inputting the first single character image into a style feature extraction model to obtain a first writing style feature output by the style feature extraction model; the style feature extraction model is used to characterize the writing style feature of the text; extracting a second text content feature from the first single character image and removing the second text content feature from the first writing style feature to obtain a second writing style feature with content information removed; inputting the first text content feature and the second writing style feature into a decoder to obtain a first target style text corresponding to the text to be generated output by the decoder, wherein the decoder is used to fuse the text content feature and the writing style feature.
[0131] In one example embodiment, the electronic device may further include a camera and / or a touchscreen communicatively connected to the processor 1010. The camera can be used to capture images of handwritten characters; the touchscreen can be used to receive handwritten input and, in response to the handwritten input, generate images of handwritten characters.
[0132] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0133] This invention also provides a font generation system. Figure 11 An exemplary schematic diagram of the font generation system provided in an embodiment of the present invention is shown, with reference to... Figure 11As shown, the font generation system may include an electronic device 1110 and an image acquisition device 1120 and / or a handwriting screen 1130 that are communicatively connected to the electronic device 1110.
[0134] Among them, electronic device 1110 can be as follows: Figure 10 The electronic device described in the corresponding embodiment; the image acquisition device 1120 can be used to acquire handwritten character images under the control of the electronic device 1110; the handwriting screen 1130 can be used to receive handwriting input, generate handwritten character images in response to handwriting input, and send the handwritten character images to the electronic device 1110.
[0135] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the font generation method provided in the above-described method embodiments. This method may include, for example,: obtaining a standard stroke sequence of the text to be generated and extracting a first text content feature of the standard stroke sequence; obtaining a first single character image from a handwritten character image and inputting the first single character image into a style feature extraction model to obtain a first writing style feature output by the style feature extraction model; the style feature extraction model is used to characterize the writing style feature of the text; extracting a second text content feature from the first single character image and removing the second text content feature from the first writing style feature to obtain a second writing style feature with content information removed; inputting the first text content feature and the second writing style feature into a decoder to obtain a first target style text corresponding to the text to be generated output by the decoder; the decoder is used to fuse the text content feature and the writing style feature.
[0136] In another aspect, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the font generation method provided in the above-described method embodiments. This method may include, for example,: acquiring a standard stroke sequence of the text to be generated and extracting a first text content feature from the standard stroke sequence; acquiring a first single-character image from a handwritten character image and inputting the first single-character image into a style feature extraction model to obtain a first writing style feature output by the style feature extraction model; the style feature extraction model being used to characterize the writing style features of the text; extracting a second text content feature from the first single-character image and removing the second text content feature from the first writing style feature to obtain a second writing style feature with content information removed; inputting the first text content feature and the second writing style feature into a decoder to obtain a first target style text corresponding to the text to be generated output by the decoder, wherein the decoder is used to fuse the text content feature and the writing style feature.
[0137] For example, a computer-readable storage medium may include a non-transitory computer-readable storage medium.
[0138] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A font generation method, characterized in that, include: Obtain the standard stroke sequence of the text to be generated, and extract the first text content feature of the standard stroke sequence; The standard stroke sequence is a set of coordinate points with directional order formed by digitizing standard characters; A first single-character image is obtained from a handwritten character image, and the first single-character image is input into a style feature extraction model to obtain a first writing style feature output by the style feature extraction model; the style feature extraction model is used to characterize the writing style features of the text. Extract the second text content feature from the first single character image, and remove the second text content feature from the first writing style feature to obtain the second writing style feature with content information removed; The first text content feature and the second handwriting style feature are input into the decoder to obtain the first target style text corresponding to the text to be generated, which is output by the decoder. The decoder is used to fuse the text content feature and the handwriting style feature. The style feature extraction model is trained based on the following steps: Obtain the first single-character sample image from the handwritten character sample images; Anchor images, positive samples, and negative samples are determined from the first single-character sample images to obtain a contrastive learning sample set; the anchor images and the positive samples are single-character sample images belonging to the same writer, but different from the writers of the negative samples; The first initial neural network model is trained by contrastive learning based on the contrastive learning sample set to obtain the style feature extraction model. The contrastive learning training aims to minimize the distance between the feature representations of the anchor image and the positive sample, and maximize the distance between the feature representations of the anchor image and the negative sample.
2. The font generation method according to claim 1, characterized in that, The decoder is trained based on the following steps: A second single-character sample image is obtained, and the second single-character sample image is input into the style feature extraction model to obtain the sample writing style features output by the style feature extraction model; Obtain the standard stroke sequence of a single character in the second single-character sample image to obtain a standard stroke sequence sample, and extract the sample text content features of the standard stroke sequence sample. The initial decoder is trained based on the sample writing style features and the sample text content features to obtain the decoder.
3. The font generation method according to claim 2, characterized in that, The process of training the initial decoder based on the sample writing style features and the sample text content features to obtain the decoder includes: The initial decoder and the style feature extraction model are jointly trained based on the sample writing style features and the sample text content features to obtain the decoder; The style feature extraction model is updated during joint training.
4. The font generation method according to claim 1, characterized in that, The extraction of the second text content features from the first single-character image includes: The first single-character image is input into the content recognition model to obtain the second text content feature output by the content recognition model; The content recognition model is obtained by training a second initial neural network model based on the third single-character sample image and the label data corresponding to the third single-character sample image.
5. The font generation method according to any one of claims 1 to 4, characterized in that, Also includes: Obtain the second single-character image from the reference character image, and input the second single-character image into the twin model of the style feature extraction model to obtain the third writing style feature output by the twin model; The third handwriting style feature is weighted and averaged with the second handwriting style feature to obtain the fourth handwriting style feature; The first text content feature and the fourth writing style feature are input into the decoder to obtain the second target style text corresponding to the text to be generated output by the decoder. The second target style text is represented by the fused writing style of the handwritten character image and the reference character image.
6. A font generation device, characterized in that, include: The first acquisition module is used to acquire the standard stroke sequence of the text to be generated; The standard stroke sequence is a set of coordinate points with directional order formed by digitizing standard characters; The first feature extraction module is used to extract the first text content features of the standard stroke sequence; The second acquisition module is used to acquire the first single character image from the handwritten character image; The second feature extraction module is used to input the first single-character image into the style feature extraction model to obtain the first writing style feature output by the style feature extraction model; the style feature extraction model is used to characterize the writing style features of the text. The third feature extraction module is used to extract the second text content feature of the first single character image and remove the second text content feature from the first writing style feature to obtain the second writing style feature with content information removed. The decoding module is used to input the first text content feature and the second writing style feature into the decoder to obtain the first target style text corresponding to the text to be generated, which is output by the decoder. The decoder is used to fuse the text content feature and the writing style feature. The style feature extraction model is trained based on the following steps: Obtain the first single-character sample image from the handwritten character sample images; Anchor images, positive samples, and negative samples are determined from the first single-character sample images to obtain a contrastive learning sample set; the anchor images and the positive samples are single-character sample images belonging to the same writer, but different from the writers of the negative samples; The first initial neural network model is trained by contrastive learning based on the contrastive learning sample set to obtain the style feature extraction model. The contrastive learning training aims to minimize the distance between the feature representations of the anchor image and the positive sample, and maximize the distance between the feature representations of the anchor image and the negative sample.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the font generation method as described in any one of claims 1 to 5.
8. The electronic device according to claim 7, characterized in that, It also includes a camera and / or touchscreen that are communicatively connected to the processor; The camera is used to capture images of handwritten characters; The touchscreen is used to receive handwriting input and, in response to the handwriting input, generate a handwritten character image.
9. A font generation system, characterized in that, Includes the electronic device as described in claim 7 and an image acquisition device and / or handwriting screen communicatively connected to the electronic device; The image acquisition device is used to acquire handwritten text images under the control of the electronic device; The handwriting screen is used to receive handwriting input, generate a handwritten character image in response to the handwriting input, and send the handwritten character image to the electronic device.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the font generation method as described in any one of claims 1 to 5.
11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the font generation method as described in any one of claims 1 to 5.