Method, system, device and medium for generating font based on neural transform field

Abstract

The application relates to the technical field of computer vision, in particular to a font generation method, system, device and medium based on a neural transform field, which comprises the following steps: constructing a neural transform field based on structure information of a source font; positions in the neural transform field correspond to font styles one by one, and each position represents a specific transform; constructing a font style estimator, and estimating the position of each font style in the neural transform field based on the font style estimator; generating a plurality of sampling points based on the estimated position and a font transform path from an origin to the estimated position; calculating the generation intensity and dissipation rate of font pixels of the sampling points through the neural transform field, and obtaining an intermediate transform result; and accumulating the intermediate transform result through a font rendering formula to obtain a target font image. The application effectively solves the problem that the existing method lacks space transform modeling when the font style is transformed, and improves the performance of the font generation method.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer vision, in particular to a font generation method and system based on neural transformation field, a device and a medium. BACKGROUND

[0002] The purpose of the font generation task is to generate a font image with the same style as the reference sample based on a small amount of reference samples. This task has high academic, commercial and artistic value, and therefore has attracted widespread attention, especially for some rich character systems such as Chinese and Korean.

[0003] In recent years, the style-content decoupling method has become the most popular solution to the font generation task. This method decouples the font image into the style representation of the font and the content representation of the character. Therefore, by using a carefully designed decoder, the text image with the reference sample font style is generated by combining the style representation from the reference sample and the content representation from the standard font.

[0004] Existing methods usually try to improve the font generation capability by exploring powerful style representation methods, but this is a suboptimal solution to the font generation task because the difference between font styles mainly comes from the shape deformation and transformation of the source character. The existing methods lack modeling of spatial transformation when the font style is transformed. SUMMARY

[0005] Embodiments of the present application provide a font generation method and system based on neural transformation field, which models the font transformation process in the shape space for the differences between different fonts in the font style transformation process, effectively solves the problem that the existing methods lack modeling of spatial transformation when the font style is transformed, and improves the performance of the font generation method.

[0006] To solve the above technical problems, in a first aspect, embodiments of the present application provide a font generation method based on neural transformation field, comprising the following steps: first, constructing a neural transformation field based on the structural information of the source font; the positions in the neural transformation field correspond one-to-one to the font styles, and each position represents a specific transformation. The path from the origin to the target position corresponds to the transformation process from the source font to the target font; next, constructing a font style estimator and estimating the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; then, generating a plurality of sampling points based on the estimated position and the font transformation path from the origin to the estimated position; and calculating the generation intensity and dissipation rate of the font pixels of the sampling points through the neural transformation field; finally, obtaining an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels of the sampling points; and accumulating the intermediate transformation result through a font rendering formula to obtain a target font image.

[0007] In some example embodiments, based on the estimated position and the font transformation path from the origin to the estimated position, a plurality of sampling points are generated; and the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field, including: dividing the font transformation path from the origin to the estimated position into a plurality of segments, and sampling at the end of each segment; and stacking the estimated position and the structure information of the source font with each other, and inputting into the neural transformation field to calculate the generation intensity and dissipation rate of the font pixels of the sampling points.

[0008] In some example embodiments, the font rendering formula is as follows:

[0009]

[0010] wherein, θ represents the font transformation path from the origin to the estimated position; ξ represents the segments into which the font transformation path from the origin to the estimated position is divided, τ i represents the dissipation rate of the font pixels of the sampling points.

[0011] In some example embodiments, the above-mentioned font generation method based on the neural transformation field further includes: evaluating the target font image.

[0012] In some example embodiments, evaluating the target font image includes: refining the font style to obtain a local style representation; and evaluating the target font image based on the local style representation.

[0013] In the second aspect, the embodiments of the present application also provide a font generation system based on a neural transformation field, including: a neural transformation field construction module, a font style estimator construction module and a data processing module connected in sequence; the neural transformation field construction module is used to construct a neural transformation field according to the structure information of a source font; the positions in the neural transformation field correspond one-to-one to font styles, and each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; the font style estimator construction module is used to construct a font style estimator, and estimate the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; the data processing module is used to generate a plurality of sampling points according to the estimated position and the font transformation path from the origin to the estimated position; and calculate the generation intensity and dissipation rate of the font pixels of the sampling points through the neural transformation field; and obtain an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels of the sampling points; and accumulate the intermediate transformation result through a font rendering formula to obtain a target font image.

[0014] In some example embodiments, the above-mentioned font generation system based on neural transformation field further comprises an evaluation module connected to the data processing module; the evaluation module is configured to evaluate the target font image.

[0015] In addition, the present application also provides an electronic device, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned font generation method based on neural transformation field.

[0016] In addition, the present application also provides a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the above-mentioned font generation method based on neural transformation field.

[0017] The technical scheme provided by the embodiments of the present application has at least the following advantages:

[0018] The embodiments of the present application provide a font generation method, system, device and medium based on neural transformation field, which comprises the following steps: first, constructing a neural transformation field based on the structural information of a source font; the positions in the neural transformation field correspond to font styles one by one, and each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process of the source font to the target font; next, constructing a font style estimator and estimating the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; then, generating a plurality of sampling points based on the estimated position and the font transformation path from the origin to the estimated position; and calculating the generation intensity and dissipation rate of the font pixels of the sampling points through the neural transformation field; finally, obtaining an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels of the sampling points; and accumulating the intermediate transformation result through a font rendering formula to obtain a target font image.

[0019] According to the generation intensity and dissipation rate of the font pixels, the present application models the font generation process as a continuous conversion process, and constructs a neural transformation field to encode the parameters required for the above-mentioned conversion process. The present application models the font transformation process in the shape space in view of the difference between different fonts in the font style transformation process, effectively solves the problem that the existing method lacks modeling of spatial transformation when transforming the font style, and improves the performance of the font generation method. BRIEF DESCRIPTION OF DRAWINGS

[0020] One or more embodiments are illustrated by way of example in the figures that are part of this disclosure and which illustrate the principles of the embodiments. Unless otherwise specifically noted, the drawings represent one or more embodiments, and are not necessarily used in conjunction with one another. The drawings are not necessarily to scale, except as specifically noted.

[0021] Figure 1 A flowchart of a font generation method based on a neural transformation field is provided for an embodiment of the present application.

[0022] Figure 2 A flowchart of a font generation method based on a neural transformation field is provided for another embodiment of the present application.

[0023] Figure 3 A comparison diagram of a target font image generated by a font generation method based on a neural transformation field provided for an embodiment of the present application and other methods is provided.

[0024] Figure 4 A structural diagram of a font generation system based on a neural transformation field provided for an embodiment of the present application is provided.

[0025] Figure 5 A structural diagram of an electronic device provided for an embodiment of the present application is provided. DETAILED DESCRIPTION

[0026] As can be known from the background, the existing methods usually try to improve the font generation capability by exploring powerful style representation methods, but this is a suboptimal solution for the font generation task because the differences between font styles mainly come from shape deformation and transformation on the source glyphs, and the existing methods lack modeling of spatial transformation when font style transformation is performed.

[0027] To solve the above technical problems, an embodiment of the present application provides a font generation method based on a neural transformation field, which comprises the following steps: first, constructing a neural transformation field based on the structural information of a source font; the positions in the neural transformation field correspond one-to-one to font styles, and each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; next, constructing a font style estimator and estimating the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; then, generating a plurality of sampling points based on the estimated position and the font transformation path from the origin to the estimated position; and calculating the generation intensity and dissipation rate of the font pixels of the sampling points through the neural transformation field; finally, obtaining an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels of the sampling points; and accumulating the intermediate transformation result through a font rendering formula to obtain a target font image. The present application models the font transformation process in the shape space in view of the differences between different fonts in the font style transformation process, effectively solves the problem that the existing methods lack modeling of spatial transformation when font style transformation is performed, and improves the performance of the font generation method.

[0028] The embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that, in the embodiments of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and various changes and modifications based on the following embodiments.

[0029] Referring to Figure 1 The embodiments of the present application provide a font generation method based on a neural transformation field, comprising the following steps:

[0030] Step S1, constructing a neural transformation field based on the structure information of a source font; the positions in the neural transformation field correspond to font styles one by one, and each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process of the source font to the target font.

[0031] Step S2, constructing a font style estimator, and estimating the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position.

[0032] Step S3, generating a plurality of sampling points based on the estimated position and the font transformation path from the origin to the estimated position; and calculating the generation intensity and dissipation rate of font pixels of the sampling points through the neural transformation field.

[0033] Step S4, obtaining an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels of the sampling points; and accumulating the intermediate transformation result through a font rendering formula to obtain a target font image.

[0034] In view of the technical problem that the existing method lacks modeling of spatial transformation when the font style is transformed, the present application models the font generation process as a continuous conversion process through the generation intensity and dissipation rate of font pixels, and constructs a neural transformation field to encode the parameters required for the above conversion process. Specifically, the present application constructs a neural transformation field (NTF) based on the structure information of a source character. Each position in the NTF represents a specific transformation, and the path from the origin to this position corresponds to the transformation process of the source font (standard font) to the target font. Therefore, each font style has a specific position, which is related to the transformation process required for generating a font image. The present application uses a font style estimator to estimate this position. Using the estimated position and the corresponding transformation path, the NTF generates a series of intermediate transformations through a sampling process, and then accumulates these intermediate transformation results through the font rendering formula proposed by the present application to finally obtain a target font image.

[0035] It should be noted that the source font (source character) in step S1 is a standard font. Step S2 is to model the font generation process as a continuous font conversion process, and to construct a neural transformation field to encode the parameters required by the above conversion process. Therefore, the font style estimator is used to estimate the position of a certain style in the neural transformation field.

[0036] In some embodiments, based on the estimated position and the font transformation path from the origin to the estimated position, a plurality of sampling points are generated in step S3; and the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field, including:

[0037] Step S301, the font transformation path from the origin to the estimated position is divided into a plurality of segments, and sampling is performed at the end of each segment.

[0038] Step S302, the estimated position and the structure information of the source font are stacked with each other, and the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field.

[0039] Each position in the NTF represents a specific transformation, and the path from the origin to this position corresponds to the transformation process of the source font (standard font) to the target font. Therefore, each font style has a specific position, which is related to the transformation process required for generating a font image. After the font style estimator estimates the style position θ, the path from the origin to the position θ is the transformation process required for font generation. The present application calculates a series of intermediate transformations corresponding to the generation process by sampling.

[0040] Specifically, the present application divides the transformation path into N segments and samples at the end of each segment (θ i = ξ). The present application stacks θ i and the structure information of the character with each other, and inputs them into the neural transformation field to calculate the generation intensity and dissipation rate τ i of the font pixels of the sampling points.

[0041] The font rendering formula is obtained by solving the font intensity formula of the positions in the NTF. Specifically, the font intensity at position ω can be represented as:

[0042]

[0043] In the above formula, the first term represents the generation process of the character pixels, and the second term represents the dissipation process.

[0044] The solution of the above formula is:

[0045]

[0046] The above formula is a font rendering formula, wherein,

[0047] In practical applications, the above formula is solved by numerical methods. Specifically, the integral path is divided into N segments and sampling is performed at the end of each segment (θ i = ξ). Therefore, the font rendering formula can be approximated as follows:

[0048]

[0049] wherein, θ represents the font transformation path from the origin to the estimated position; ξ represents the segments into which the font transformation path from the origin to the estimated position is divided, τ i represents the dissipation rate of the font pixel of the sampling point.

[0050] Referring to Figure 2 , in some embodiments, the above font generation method based on neural transformation field further comprises:

[0051] Step S5, evaluating the target font image.

[0052] In some embodiments, step S5 of evaluating the target font image comprises:

[0053] Step S501, refining the font style to obtain a local style representation.

[0054] Step S502, evaluating the target font image based on the local style representation.

[0055] Since the font style contains many fine-grained structures and local correlations, the font style is further refined into a series of local style representations, and a large number of experiments are performed to evaluate the model of the present application. The experimental results show that, as Figure 3 shown in the table, the model of the present application exhibits excellent performance in the font generation task.

[0056] The font generation method based on the neural transformation field provided in the application is characterized in that: the application models the font generation process as a continuous conversion process. After researchers try various theoretical modeling methods and different network structures, it is found that modeling the font generation process by using the generation intensity and dissipation rate of font pixels is a relatively optimal choice, and high-performance font generation is achieved through a finely designed font style estimator and a font pixel generation intensity and dissipation rate prediction layer. Therefore, after multiple experiments and analysis, the application provides a font generation method based on the neural transformation field. In the model training, conventional training data and training methods are used, and the best results can still be achieved after controlling the variables in the font generation process, which is sufficient to prove that the model itself is effective and can be evaluated and verified through various data.

[0057] Referring to Figure 4 The embodiment of the application further provides a font generation system based on the neural transformation field, comprising: a neural transformation field construction module 101, a font style estimator construction module 102 and a data processing module 103 connected in sequence; the neural transformation field construction module 101 is used to construct a neural transformation field according to the structure information of a source font; the positions in the neural transformation field correspond one-to-one to font styles, and each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process of the source font to the target font; the font style estimator construction module 102 is used to construct a font style estimator and estimate the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; the data processing module 103 is used to generate a plurality of sampling points according to the estimated position and the font transformation path from the origin to the estimated position; calculate the generation intensity and dissipation rate of the font pixels of the sampling points through the neural transformation field; and obtain an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels of the sampling points; and accumulate the intermediate transformation result through a font rendering formula to obtain a target font image.

[0058] In some exemplary embodiments, the font generation system based on the neural transformation field described above further comprises: an evaluation module 104 connected to the data processing module 103; the evaluation module 104 is used to evaluate the target font image.

[0059] The font generation method and system based on the neural transform field provided by the embodiments of the present application have the advantages of high accuracy and strong universality compared with the prior art. On the one hand, the model of the present application can generate high-performance font images, and the visualization results and performance indicators on the data set are better than those of the prior art, so it is proved that the model of the present application is fully capable of being applied to real scenarios. On the other hand, the model of the present application has strong universality and portability, and for any font generation model, after a simple modification, the method of the present application can be used to generate high-performance font images.

[0060] Referring to Figure 5 Another embodiment of the present application provides an electronic device, comprising: at least one processor 110; and a memory 111 in communication connection with the at least one processor; wherein the memory 111 stores instructions executable by the at least one processor 110, and the instructions are executed by the at least one processor 110 to enable the at least one processor 110 to perform any of the above method embodiments.

[0061] The memory 111 and the processor 110 are connected in a bus manner, the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of the one or more processors 110 and the memory 111 together. The bus can also connect various other circuits such as peripheral devices, voltage stabilizers and power management circuits together, which are well known in the art, and therefore, they will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements such as multiple receivers and transmitters, which provide units for communicating with various other devices on the transmission medium. The data processed by the processor 110 is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor 110.

[0062] The processor 110 is responsible for managing the bus and general processing, and can also provide various functions including timing, peripheral interface, voltage regulation, power management and other control functions. And the memory 111 can be used to store the data used by the processor 110 in the execution of the operation.

[0063] Another embodiment of the present application relates to a computer readable storage medium storing a computer program. The computer program is executed by the processor to implement the above method embodiments.

[0064] From the above technical solutions, the embodiments of the present application provide a font generation method, system, device and medium based on a neural transform field, which comprises the following steps:

[0065] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a program stored in a storage medium, including a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the above-mentioned methods of various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0066] According to the above technical solution, the embodiment of the present application provides a font generation method, system, device and medium based on a neural transformation field. The method comprises the following steps: first, constructing a neural transformation field based on the structural information of a source font; the positions in the neural transformation field correspond one-to-one to font styles, and each position represents a specific transformation. The path from the origin to the target position corresponds to the transformation process of the source font to the target font. Next, a font style estimator is constructed, and the position of each font style in the neural transformation field is estimated based on the font style estimator to obtain an estimated position. Then, a plurality of sampling points are generated based on the estimated position and the font transformation path from the origin to the estimated position. The generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field. Finally, the intermediate transformation result is obtained based on the generation intensity and dissipation rate of the font pixels of the sampling points, and the intermediate transformation result is accumulated through a font rendering formula to obtain a target font image.

[0067] According to the generation intensity and dissipation rate of the font pixels, the present application models the font generation process as a continuous conversion process, and constructs a neural transformation field to encode the parameters required for the above-mentioned conversion process. The present application models the font transformation process in the shape space in view of the differences between different fonts in the font style transformation process, effectively solving the problem that existing methods lack modeling of spatial transformation when transforming font styles, and improving the performance of the font generation method.

[0068] Those skilled in the art can understand that the above-mentioned embodiments are specific embodiments for implementing the present application, and in actual applications, various changes can be made in form and detail without departing from the spirit and scope of the present application. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, therefore the protection scope of the present application should be limited by the scope defined in the claims.

Claims

1. A font generation method based on neural transform field, characterized by, The method comprises the following steps: Based on the structure information of the source font, a neural transformation field is constructed; The positions in the neural transformation field correspond to the font styles one by one, and each position represents a specific transformation. The path from the origin to the target position corresponds to the transformation process from the source font to the target font; A font style estimator is constructed, and the position of each font style in the neural transformation field is estimated based on the font style estimator to obtain an estimated position; Based on the estimated position and the font transformation path from the origin to the estimated position, a plurality of sampling points are generated; And the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field; Based on the generation intensity and dissipation rate of the font pixels of the sampling points, an intermediate transformation result is obtained; And the intermediate transformation result is accumulated through a font rendering formula to obtain a target font image.

2. The method of claim 1, wherein, The method comprises the following steps: Based on the estimated position and the font transformation path from the origin to the estimated position, a plurality of sampling points are generated; And the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field, comprising: Divide the font transformation path from the origin to the estimated position into multiple segments, and sample at the end of each segment; 3. The method of claim 1, wherein the neural transform field-based font generation method is characterized by, Stack the estimated position and the structure information of the source font with each other, and input them into the neural transformation field to calculate the generation intensity and dissipation rate of the font pixels of the sampling points. wherein Θ represents the font transform path from the origin to the estimated position; ξ represents a segment into which the font transform path from the origin to the estimated position is divided, τ i represents the dissipation rate of the font pixel of the sampling point.

4. The method of claim 1, wherein the neural transform field-based font generation method is characterized by, The font rendering formula is as follows: The method further comprises the following steps:

5. The method of claim 4, wherein, Evaluate the target font image. The evaluation of the target font image comprises the following steps: Refine the font style to obtain a local style representation; 6. A font generation system based on neural transform fields, characterized by, Based on the local style representation, evaluate the target font image. The method comprises the following steps: The neural transformation field construction module, the font style estimator construction module and the data processing module are connected in sequence; The neural transformation field construction module is used to construct a neural transformation field according to the structure information of the source font; the positions in the neural transformation field correspond to the font styles one by one, and each position represents a specific transformation. The path from the origin to the target position corresponds to the transformation process from the source font to the target font; The font style estimator construction module is used to construct a font style estimator, and estimate the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; The data processing module is used to generate a plurality of sampling points according to the estimated position and the font transformation path from the origin to the estimated position; And the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field; and based on the generation intensity and dissipation rate of the font pixels of the sampling points, an intermediate transformation result is obtained; 7. The neural transform field-based font generation system of claim 6, wherein, And the intermediate transformation result is accumulated through a font rendering formula to obtain a target font image. The method further comprises the following steps: The evaluation module connected with the data processing module; 8. An electronic device, comprising: The evaluation module is used to evaluate the target font image. The method comprises the following steps: At least one processor; And, A memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method for generating a font based on a neural transfer field according to any one of claims 1 to 5.

9. A computer readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the method for generating a font based on a neural transfer field according to any one of claims 1 to 5.

Images