Font generation method and device for minority language

Abstract

The embodiment of the invention provides a small language font generation method, which comprises the following steps: obtaining a reference character and a to-be-processed character, the reference character being a non-small language character of a reference font, and the to-be-processed character being a small language character; inputting the reference characters and the to-be-processed characters into a decoder of a font generation model, extracting character style features from the reference characters, and extracting character structure features from the to-be-processed characters; the character style features and the character structure features are fused through an encoder, so that the character to be processed learns the font style of the reference character, a small language font corresponding to the character to be processed is generated, and the font generation model is obtained based on a heterogeneous aggregation expert model. According to the technical scheme, cross-language font generation can be achieved, the small language font generation efficiency is improved, and the problem that a font library of a low-resource small language is limited or does not exist is solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of network model technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for generating fonts in a minority language. Background Art

[0002] In order to improve the recognition accuracy of text in minority languages ​​by text recognition models, a large amount of sample data of text in minority languages ​​is needed to train the text recognition model. Therefore, how to obtain a large amount of training sample data becomes crucial. Sample data for minority languages ​​includes a variety of fonts, which are often obtained by collecting existing data of minority languages ​​or manually designed. Since there are relatively few people who understand minority languages, and the amount of data and existing fonts of minority languages ​​themselves is also small, it is quite difficult to collect sample data for minority languages. Moreover, the efficiency of manually designing fonts for minority languages ​​is relatively low, and the labor cost is very high.

[0003] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for generating fonts for a minority language to solve or alleviate one or more of the technical problems raised above.

[0005] One aspect of an embodiment of the present application provides a method for generating a font for a minority language, the method comprising: Obtain a reference character and a character to be processed, wherein the reference character is a non-minority language character of a reference font, and the character to be processed is a minority language character; Input the reference character and the character to be processed into a decoder of a font generation model, extract text style features from the reference character, and extract text structure features from the character to be processed; The text style features and the text structure features are fused through an encoder so that the character to be processed learns the font style of the reference character and generates a small language font corresponding to the character to be processed. The font generation model is obtained based on a heterogeneous aggregation expert model.

[0006] Optionally, extracting text style features from the reference characters and extracting text structure features from the characters to be processed include: Decomposing the reference character and the character to be processed into stroke dimension codes; Based on the stroke dimension code, the text structure features of the character to be processed and the text style features of the reference character are analyzed.

[0007] Optionally, the fusing of the text style feature and the text structure feature so that the character to be processed learns the font style of the reference character and generates a small language font corresponding to the character to be processed includes: According to the character structure feature of the character to be processed, finding the strokes of the reference character corresponding to the character structure feature; The text style feature corresponding to the stroke is obtained, and a small language font corresponding to the character to be processed is generated by simulating the text style feature.

[0008] Optionally, the method further comprises: In the font generation model, the text structure features and text style features of each character are decoupled through a heterogeneous aggregation expert model. The heterogeneous aggregation expert model adopts a dual-branch structure, wherein the first branch processes the channel information of the character, and the second branch processes the spatial information of the character, and aggregates the channel information output by the first branch and the spatial information output by the second branch.

[0009] Optionally, the font generation model is trained based on multilingual sample data, and the multilingual sample data includes the language corresponding to the reference character, but does not include a minority language corresponding to the character to be processed.

[0010] Optionally, the training process of the font generation model includes: Acquire multilingual sample data, where the multilingual sample data is character samples of non-minority languages; Training the font generation model according to the multilingual sample data; The model effect is evaluated by at least one loss function to obtain the finally trained font generation model.

[0011] Optionally, evaluating the model effect by at least one loss function includes: A content-style homogeneity loss function is used to supervise the encoder of the font generation model to further increase the heterogeneity between the text structure features and the text style features. The content-style homogeneity loss function is obtained based on the distance between the vectors corresponding to the text structure features and the text style features. The larger the distance, the smaller the content-style homogeneity loss function.

[0012] Another aspect of the embodiments of the present application provides a device for generating fonts of a minority language, the device comprising: An acquisition module, used for acquiring a reference character and a character to be processed, wherein the reference character is a non-minority language character of a reference font, and the character to be processed is a minority language character; A generation module is used to input the reference character and the character to be processed into a decoder of a font generation model, extract text style features for the reference character, and extract text structure features for the character to be processed; the text style features and the text structure features are fused through an encoder so that the character to be processed learns the font style of the reference character, and generates a small language font corresponding to the character to be processed, wherein the font generation model is obtained based on a heterogeneous aggregation expert model.

[0013] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0014] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0015] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0016] The above technical solution adopted in the embodiment of the present application may have the following advantages: Cross-language font generation is achieved through a font generation model based on a heterogeneous aggregation expert model, which improves the efficiency of font generation for small languages. In addition, an effective model can be trained without the need for low-resource small language sample data. This can solve the problem of limited or non-existent font libraries for small languages, allowing small languages ​​to have more fonts and saving the manpower cost of designing small language fonts. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0018] Figure 1 A diagram schematically shows an operating environment of a method for generating fonts of a small language according to the first embodiment of the present application; Figure 2 A flowchart of a method for generating a font for a small language according to the first embodiment of the present application is schematically shown; Figure 3 A schematic diagram of the structure of a heterogeneous aggregation expert model according to the first embodiment of the present application is shown; Figure 4 Schematically shows Figure 2 A detailed flow chart of step S202; Figure 5 Schematically shows Figure 4 A detailed flow chart of step S2024; Figure 6 A flowchart schematically shows a training process of a font generation model according to the first embodiment of the present application; Figure 7 A schematic diagram schematically shows an exemplary application according to the first embodiment of the present application; Figure 8 A block diagram schematically shows a device for generating fonts for a small language according to the second embodiment of the present application; and Fig. 9 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0020] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0021] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0022] First, the following terms are explained: Minority languages: Languages ​​that are used only in a few countries or regions, compared to Chinese and English, which are widely used and have many users. In this application, the minority languages ​​mainly refer to languages ​​with fewer data sources and fewer existing fonts, including foreign languages ​​and domestic minority languages, such as Hindi, Uyghur, Kazakh, etc. It is worth noting that although the number of users of foreign languages ​​such as Korean and Japanese is not as large as that of Chinese and English, they are not considered in this application because of the large number of data sources and existing fonts.

[0023] Mixture of Experts (MoE): It is composed of multiple specialized sub-models (i.e. "experts"), each of which contributes in its field of expertise. The mechanism that determines which "expert" participates in answering a specific question is called a "gating network". The mixed expert model mainly consists of two key parts: a sparse MoE layer, which contains several "experts", each of which is an independent neural network. In practical applications, these "experts" are usually feedforward neural networks, but they can also be more complex network structures, or even the MoE layer itself, forming a hierarchical MoE structure. The gating network or routing is used to decide which tokens are sent to which "expert". Sometimes, a token can even be sent to multiple "experts".

[0024] Multilayer Perceptron (MLP): is a feedforward neural network that consists of an input layer, one or more hidden layers, and an output layer. MLP is a fully connected neural network where neurons in each layer are connected to every neuron in the next layer. This structure enables MLP to handle complex nonlinear relationships and thus solve many complex pattern recognition problems.

[0025] Heterogeneous Aggregation Experts (HAE): A hybrid expert model with a heterogeneous aggregation attention (HAA) module that aggregates channel information and spatial information, allowing the model to better separate content features and style features.

[0026] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: The role of font generation is to generate a specified style from multilingual character data, thereby improving the efficiency of font design. Current font generation methods are either based on global information, which makes it difficult to capture local information and produces unsatisfactory results; or they are component-based methods that ignore the spatial and channel information of images, have limited accuracy, and cannot achieve the best results for resource-poor languages.

[0027] To this end, the embodiment of the present application provides a technical solution for generating fonts for small languages. In this technical solution, cross-language font generation can be achieved through a font generation model based on a heterogeneous aggregation expert model, which improves the efficiency of generating fonts for small languages. In addition, an effective model can be trained without low-resource small language sample data, which can solve the problem of limited or non-existent font libraries for small languages. See below for details.

[0028] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0029] See also Figure 1 In this operating environment diagram, the present application can be applied to application environments including, but not limited to, a service platform 2, a client 4, and a network 6.

[0030] The client 4 is used to display the interface of the current application to the user and receive the user's text input and other operations. The client 4 can be a terminal device such as a PC (Personal Computer), a mobile phone, a tablet computer, a portable computer, a wearable device, etc.

[0031] The service platform 2 is used to provide data and technical support for the client 4. The service platform 2 can be a computing device such as a rack server, a blade server, a tower server or a cabinet server, and can be an independent server or a server cluster composed of multiple servers.

[0032] The network 6 can be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), the Wideband Code Division Multiple Access (WCDMA), the 4G network, the 5G network, Bluetooth, Wi-Fi, etc. The service platform 2 and one or more of the clients 4 are connected to each other through the network 6 for data transmission and interaction.

[0033] Embodiment 1 Figure 2The flowchart of the method for generating fonts in a small language according to the first embodiment of the present application is schematically shown. It can be understood that the flowchart in the embodiment of the present method is not used to limit the order of executing the steps. Some steps in the flowchart can be added or deleted as needed. The execution subject of the method for generating fonts in a small language can be the service platform or the client.

[0034] like Figure 2 As shown, the method for generating a font for a minority language may include steps S200 to S202, wherein: Step S200, obtaining a reference character and a character to be processed.

[0035] In the present embodiment, the reference characters are non-minority language characters of the reference font, such as characters in languages ​​such as Chinese and English. The characters to be processed are minority language characters, such as characters in languages ​​such as Hindi, Uighur, and Kazakh. Since there are few data sources for minority language characters and few existing fonts, the present embodiment can generate the corresponding font of the minority language character by simulating the font of the non-minority language character by taking the font of the non-minority language character as a reference. First, it is necessary to obtain the non-minority language character and the minority language character, that is, the reference character and the character to be processed. Here, the image of the reference character and the image of the character to be processed can be obtained respectively, and then feature analysis can be performed based on these images.

[0036] In other optional embodiments, the characters to be processed may also be non-minority language characters, which will not be described in detail here.

[0037] Step S202, input the reference character and the character to be processed into the decoder of the font generation model, extract the text style features of the reference character, extract the text structure features of the character to be processed, and then fuse the text style features and the text structure features through the encoder to enable the character to be processed to learn the font style of the reference character, and generate a small language font corresponding to the character to be processed.

[0038] Currently, existing font generation methods are generally divided into two categories: global information-based methods and component-based methods. Global-based methods use content encoders combined with supervisory signals, but due to the complex glyphs of some characters, it is difficult to capture local information, so the results are unsatisfactory. Component-based methods decompose character images into multiple parts and train each part using content and style signals, which can more effectively capture the implicit glyph information inherent in multilingual fonts. Some schemes have also proposed the use of expert mixture models to capture implicit component information in characters, which is very important for texts that lack specific component annotations. However, the expert mixture model uses a relatively simple MLP structure and ignores the spatial and channel information of the image. Therefore, the accuracy of these methods is limited, resulting in the inability to obtain optimal output results for resource-poor small language font generation, which in turn affects the ability of experts to capture implicit component information.

[0039] In view of this, this embodiment adopts a font generation model based on a heterogeneous aggregation expert model. The overall framework of the font generation model consists of an encoder and a decoder structure. The encoder part adopts a hybrid expert strategy to encode characters at the component level (such as stroke dimension) and analyze character features. The decoder part is used to decode character features and generate new fonts. The hybrid expert strategy of this embodiment adopts k heterogeneous aggregation expert models as the basic encoder, where k is a positive integer. The attention part of the encoder architecture of the heterogeneous aggregation expert model introduces a heterogeneous aggregation attention mechanism. The heterogeneous aggregation attention mechanism is the base of the encoder, which can aggregate channel information and spatial information, so that the model can better separate text structure features and text style features, which can also be called content features and style features, which is conducive to the decoupling of character content and style.

[0040] Specifically, the heterogeneous aggregation expert model adopts a dual-branch structure, where one branch processes channel information and the other processes spatial information. Finally, the outputs of the two branches are fused: , , in, It is the feature processed by the previous convolutional neural network. is the separation feature along the channel, and They are spatial features and channel features, and They are spatial modeling function and channel modeling function respectively.

[0041] like Figure 3 The figure shows a schematic diagram of the structure of the heterogeneous aggregation expert model (heterogeneous aggregation attention mechanism). In channel modeling, the self-attention mechanism is used to create a channel attention map. , indicating the importance of the channel, where C is the number of channels. For spatial modeling, the K vector and V vector are pooled, but the Q vector is not pooled to maintain high-resolution details. Spatial Attention Map , indicates the importance of the region, where H is the length, W is the width, and s is the pooling hyperparameter to reduce the computational complexity. The heterogeneous aggregation expert model brings together channel information and spatial information, enabling the encoder to extract more details from the character image.

[0042] In addition, in order to further ensure the decoupling of content and style, a new loss function is proposed, called content-style homogeneity loss. The content-style homogeneity loss function is obtained based on the distance between the vector corresponding to the text structure feature and the text style feature. The larger the distance, the smaller the content-style homogeneity loss function. The calculation formula is the content-style homogeneity loss function .in, and Represent the content features and style features of the encoder output respectively. This loss function is used to supervise the encoder and increase the heterogeneity between content features (text structure features) and style features (text style features).

[0043] Further reading Figure 4 , is a detailed flow chart of the above step S202. In this embodiment, the step S202 specifically includes: Step S2020, decomposing the reference character and the character to be processed into stroke dimension codes.

[0044] In order to make the character to be processed better simulate the font of the reference character, the encoder in the font generation model will first decompose the reference character and the character to be processed into stroke dimension codes. The specific function of decomposing into stroke dimension codes is to enable the character to be processed to simulate the style (font) of the reference character through similar strokes.

[0045] Step S2022: analyzing the text structure features of the character to be processed and the text style features of the reference character based on the stroke dimension code.

[0046] In this embodiment, the encoder in the font generation model can analyze the text structure features and text style features of the character to be processed, as well as the text structure features and text style features of the reference character based on the stroke dimension encoding.

[0047] The text structure features of the character to be processed are used to guide the text content of the output result of the font generation model, and the text style features of the reference character are used to guide the font of the output result of the font generation model. The font generation model can achieve font simulation between the character to be processed and the reference character by analyzing and learning the text structure features of the character to be processed and the text style features of the reference character, that is, the text content of the character to be processed simulates the font of the reference character.

[0048] Step S2024, combining the text structure feature and the text style feature to generate a small language font corresponding to the character to be processed.

[0049] The decoder in the font generation model decodes the text structure features of the character to be processed and the text style features of the reference character, and then combines the two to generate a font of a minority language corresponding to the character to be processed. In other words, the text content of the output result of the font generation model is the text content of the character to be processed, but the font of the output result is a new font generated by simulating the font of the reference character.

[0050] Further reading Figure 5 , is a detailed flow chart of the above step S2024. In this embodiment, the step S2024 specifically includes: Step S300: Find the strokes of the reference character corresponding to the character structure feature according to the character structure feature of the character to be processed.

[0051] That is, for each stroke in the character to be processed, a similar stroke in the reference character is found.

[0052] Step S302, obtaining the text style features corresponding to the strokes, and generating a small language font corresponding to the character to be processed by simulating the text style features.

[0053] Each stroke in the character to be processed will simulate the style features of the corresponding similar strokes in the reference character, and the strokes are combined to form a new font for the character to be processed.

[0054] It is worth noting that the font generation model is trained based on multilingual sample data, and the multilingual sample data generally needs to include the language corresponding to the reference character, but may not include the minority language corresponding to the character to be processed. For example, due to the small number of data sources for Uyghur and the lack of sufficient training sample data, this embodiment can use characters of languages ​​such as Chinese and English, which have many data sources and many existing fonts, as training sample data to train the font generation model. The trained model can be directly used for cross-language font generation of minority language characters such as Uyghur, and can also obtain better output results.

[0055] Since the font generation model decomposes the characters into strokes and realizes the font learning of the character to be processed from the reference character by analyzing the text structure features of the character to be processed and the text style features of the reference character, in this case, even if there are no minority language training samples during model training, the font generation model can generate a minority language font corresponding to the character to be processed by allowing the character to be processed to simulate the style features of similar strokes in the reference character.

[0056] In addition, in an optional embodiment, if the reference character is a character in an alphabetic language such as an English character, it is not necessary to decompose the reference character into strokes.

[0057] like Figure 6 As shown, it is a flow chart of the training process of the font generation model. It can be understood that the flow chart is not used to limit the order of executing the steps. As needed, some steps in the flow chart can be added or deleted. In this embodiment, the training process of the font generation model specifically includes: Step S400: obtaining multilingual sample data as a training set and a test set, wherein the multilingual sample data is a character sample of a non-minority language.

[0058] Since there are few sources of character data in many minority languages, character samples of non-minority languages ​​can be used for model training in this embodiment. The multilingual sample data includes content images corresponding to several characters providing content and style images corresponding to several characters providing style.

[0059] Since the font generation model needs to realize the function of decomposing characters into stroke dimension codes, the strokes in the character samples are already marked, which can be marked manually or by other large models, and there is no limitation here.

[0060] Step S402: training the font generation model according to the multilingual sample data.

[0061] A portion of the sample data is used as a training set to input into the model for training, while another portion of the sample data is used as a test set to test the model's performance. The model combines the style features from the style image and the content features from the content image, decodes them, and generates a new character image. Moreover, a heterogeneous aggregation attention mechanism is adopted in the model to aggregate channel information and spatial information, enabling the model to better separate the text structure features and text style features, which is conducive to the decoupling of character content and style.

[0062] Step S404, evaluate the model's performance through at least one loss function to obtain the finally trained font generation model.

[0063] In this embodiment, loss functions such as adversarial loss (Ladv), feature matching loss (Lfm), content-style homogeneity loss (Lcsh), and pixel-level reconstruction loss (Lrecon) are combined to supervise the model. After repeated model training and adjustment, when the overall objective function meets the conditions, the trained font generation model is obtained. Using the font generation model, the above-mentioned small language font generation method can be realized.

[0064] To make this application easier to understand, the following is combined with Figure 7 Provide an exemplary application.

[0065] In Figure 7 , the "Luo" on the upper left represents a reference character for providing the text style; the Uyghur character below is a character to be processed. Through the font generation model based on the heterogeneous aggregation expert model to process the reference character and the character to be processed, encode and decode the reference character and the character to be processed, and transfer the style of the reference character to the character to be processed, that is, transfer the font of "Luo" to the Uyghur character below, and successfully generate the font style as shown on the Figure 7 right side.

[0066] The small language font generation method provided in this embodiment realizes cross-language font generation through the font generation model based on the heterogeneous aggregation expert model, and uses the newly proposed loss function (i.e., content-style homogeneity loss) to further distinguish content and style, achieving excellent results in terms of visual quality and numerical indicators. This method improves the efficiency of small language font generation, and an effective model can be trained without small language sample data with low resources, which can solve the problem of limited or non-existent font libraries for small languages, enabling more fonts for small languages and saving the labor cost of designing small language fonts.

[0067] Embodiment 2 Figure 8The block diagram of the device for generating fonts for small languages ​​according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 8 As shown, the device 60 may include: an acquisition module 600 and a generation module 602, wherein: The acquisition module 600 is used to acquire reference characters and characters to be processed, wherein the reference characters are non-minority language characters of a reference font, and the characters to be processed are minority language characters.

[0068] The generation module 602 is used to input the reference character and the character to be processed into the decoder of the font generation model, extract the text style features of the reference character, extract the text structure features of the character to be processed, and then fuse the text style features and the text structure features through the encoder so that the character to be processed learns the font style of the reference character, and generates a small language font corresponding to the character to be processed. The font generation model is obtained based on a heterogeneous aggregation expert model.

[0069] Specifically, the extracting of text style features from the reference characters and the extracting of text structure features from the characters to be processed include: Decomposing the reference character and the character to be processed into stroke dimension codes; Based on the stroke dimension code, the text structure features of the character to be processed and the text style features of the reference character are analyzed.

[0070] The step of fusing the text style feature and the text structure feature so that the character to be processed learns the font style of the reference character and generates a small language font corresponding to the character to be processed includes: According to the character structure feature of the character to be processed, finding the strokes of the reference character corresponding to the character structure feature; The text style feature corresponding to the stroke is obtained, and a small language font corresponding to the character to be processed is generated by simulating the text style feature.

[0071] In an optional embodiment, in the font generation model, the text structure features and text style features of each character are decoupled through a heterogeneous aggregation expert model. The heterogeneous aggregation expert model adopts a dual-branch structure, wherein the first branch processes the channel information of the character, and the second branch processes the spatial information of the character, and aggregates the channel information output by the first branch and the spatial information output by the second branch.

[0072] The font generation model is trained based on multilingual sample data, and the multilingual sample data includes the language corresponding to the reference character, but does not include the minority language corresponding to the character to be processed.

[0073] In addition, a content-style homogeneity loss function is used to supervise the encoder of the font generation model to further increase the heterogeneity between the text structure features and the text style features. The content-style homogeneity loss function is obtained based on the distance between the vectors corresponding to the text structure features and the text style features. The larger the distance, the smaller the content-style homogeneity loss function.

[0074] The specific implementation process of the above-mentioned functions of each module can be found in the description of the above-mentioned embodiment 1, and will not be repeated here.

[0075] The small language font generation system provided in this embodiment realizes cross-language font generation through a font generation model based on a heterogeneous aggregation expert model, and uses a newly proposed loss function (i.e., content-style homogeneity loss) to further distinguish content and style, and has achieved excellent results in visual quality and numerical indicators. The system improves the efficiency of small language font generation, and can train an effective model without low-resource small language sample data, which can solve the problem of limited or non-existent font libraries for small languages, so that small languages ​​have more fonts and can also save the labor cost of designing small language fonts.

[0076] Embodiment 3 Fig. 9 The hardware architecture diagram of a computer device 10000 suitable for implementing a method for generating fonts in a minority language according to the third embodiment of the present application is schematically shown. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig. 9 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed in the computer device 10000, such as the program code of the method for generating fonts in a minority language, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or will be output.

[0077] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0078] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0079] It should be pointed out that Fig. 9 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0080] In this embodiment, the method for generating fonts for a small language stored in the memory 10010 may also be divided into one or more program modules and executed by one or more processors (such as the processor 10020) to complete the embodiment of the present application.

[0081] Embodiment 4 The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for generating fonts in a small language in the embodiment are implemented.

[0082] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device, such as the program code of the method for generating fonts of small languages ​​in the embodiment, etc. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0083] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0084] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0085] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for generating fonts of a minority language, characterized in that: The method comprises: Obtain a reference character and a character to be processed, wherein the reference character is a non-minority language character of a reference font, and the character to be processed is a minority language character; Input the reference character and the character to be processed into a decoder of a font generation model, extract text style features from the reference character, and extract text structure features from the character to be processed; The text style features and the text structure features are fused through an encoder so that the character to be processed learns the font style of the reference character and generates a small language font corresponding to the character to be processed. The font generation model is obtained based on a heterogeneous aggregation expert model.

2. The method according to claim 1, characterized in that The step of extracting text style features from the reference characters and extracting text structure features from the characters to be processed includes: Decomposing the reference character and the character to be processed into stroke dimension codes; Based on the stroke dimension code, the text structure features of the character to be processed and the text style features of the reference character are analyzed.

3. The method according to claim 2, characterized in that The step of fusing the text style feature and the text structure feature so that the character to be processed learns the font style of the reference character and generates a small language font corresponding to the character to be processed includes: According to the character structure feature of the character to be processed, finding the strokes of the reference character corresponding to the character structure feature; The text style feature corresponding to the stroke is obtained, and a small language font corresponding to the character to be processed is generated by simulating the text style feature.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: In the font generation model, the text structure features and text style features of each character are decoupled through a heterogeneous aggregation expert model. The heterogeneous aggregation expert model adopts a dual-branch structure, wherein the first branch processes the channel information of the character, and the second branch processes the spatial information of the character, and aggregates the channel information output by the first branch and the spatial information output by the second branch.

5. The method according to claim 1, characterized in that The font generation model is trained based on multilingual sample data, and the multilingual sample data includes the language corresponding to the reference character, but does not include the minority language corresponding to the character to be processed.

6. The method according to claim 5, characterized in that The training process of the font generation model includes: Acquire multilingual sample data, where the multilingual sample data is character samples of non-minority languages; Training the font generation model according to the multilingual sample data; The model effect is evaluated by at least one loss function to obtain the finally trained font generation model.

7. The method according to claim 6, characterized in that The evaluating the model effect by at least one loss function comprises: A content-style homogeneity loss function is used to supervise the encoder of the font generation model to further increase the heterogeneity between the text structure features and the text style features. The content-style homogeneity loss function is obtained based on the distance between the vectors corresponding to the text structure features and the text style features. The larger the distance, the smaller the content-style homogeneity loss function.

8. A device for generating fonts of a minority language, characterized in that: The device comprises: An acquisition module, used for acquiring a reference character and a character to be processed, wherein the reference character is a non-minority language character of a reference font, and the character to be processed is a minority language character; A generation module is used to input the reference character and the character to be processed into a decoder of a font generation model, extract text style features for the reference character, and extract text structure features for the character to be processed; the text style features and the text structure features are fused through an encoder so that the character to be processed learns the font style of the reference character, and generates a small language font corresponding to the character to be processed, wherein the font generation model is obtained based on a heterogeneous aggregation expert model.

9. A computer device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 7 are implemented.

Images