Font generation method and system based on neural transformation field, device, and medium
By constructing a neural transformation field and a font style estimator, and utilizing the generation intensity and dissipation rate of font pixels, the spatial transformation modeling problem during font style transformation is solved, thereby improving the performance and quality of font generation.
Patent Information
- Application Number
- PCT/CN2023/097781
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-06-01
- Publication Date
- 2024-12-05
AI Technical Summary
Existing font generation methods lack modeling of spatial transformations when changing font styles, resulting in insufficient generation capabilities.
By constructing a neural transformation field, the font generation process is modeled using the generation intensity and dissipation rate of font pixels. A font style estimator is then established, generating multiple sampling points and accumulating them through a rendering formula to obtain the target font image.
The performance of the font generation method has been improved, and effective modeling of spatial transformations has been achieved, resulting in the generation of high-quality target font images.
Smart Images

Figure CN2023097781_05122024_PF_FP_ABST
Abstract
Description
Font generation method, system, device and medium based on neural transformation field Technical Field
[0001] The embodiments of the present application relate to the field of computer vision technology, and in particular to a font generation method, system, device and medium based on neural transformation fields. Background Art
[0002] The goal of font generation is to generate font images with the same style as a small number of reference samples. This task has high academic, commercial, and artistic value, and has therefore attracted widespread attention, especially for scripts with rich glyphs, such as Chinese and Korean.
[0003] In recent years, style-content decoupling methods have become the most popular solution for font generation tasks. This method decouples font images into the font's style representation and the character content representation. Therefore, a carefully designed decoder is used to combine the style representation of a reference sample with the content representation of a standard font to generate a text image with the style of the reference sample font.
[0004] Existing methods generally attempt to improve font generation capabilities by exploring powerful style representations. However, this is a suboptimal solution for the font generation task because the differences between font styles mainly come from shape deformation and transformation of the source glyphs. Existing methods lack modeling of spatial transformations when transforming font styles.
[0005] Summary of the Invention
[0006] The embodiments of the present application provide a font generation method, system, device and medium based on a neural transformation field. In view of the differences between different fonts during the font style transformation process, the font transformation process is modeled in the shape space, which effectively solves the problem of the existing methods lacking spatial transformation modeling during font style transformation and improves the performance of the font generation method.
[0007] To solve the above technical problems, in the first aspect, an embodiment of the present application provides a font generation method based on a neural transformation field, comprising the following steps: first, based on the structural information of the source font, a neural transformation field is constructed; the positions in the neural transformation field correspond one-to-one to the font styles, each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; next, a font style estimator is constructed, and based on the font style estimator, the position of each font style in the neural transformation field is estimated to obtain an estimated position; then, based on the estimated position and the font transformation path from the origin to the estimated position, a plurality of sampling points are generated; and the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field; finally, based on the generation intensity and dissipation rate of the font pixels of the sampling points, an intermediate transformation result is obtained; and the intermediate transformation results are accumulated through the font rendering formula to obtain the target font image.
[0008] In some exemplary embodiments, a plurality of sampling points are generated based on the estimated position and the font transformation path from the origin to the estimated position; and the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field, including: dividing the font transformation path from the origin to the estimated position into a plurality of segments, and sampling at the end of each segment; stacking the estimated position and the structural information of the source font, and feeding them into the neural transformation field to calculate the generation intensity and dissipation rate of the font pixels of the sampling points.
[0009] In some exemplary embodiments, the font rendering formula is as follows:
[0010] in, θ represents the font transformation path from the origin to the estimated position; ξ represents the segments into which the font transformation path from the origin to the estimated position is divided, τ i Indicates the dissipation rate of the font pixel at the sampling point.
[0011] In some exemplary embodiments, the above-mentioned font generation method based on neural transformation field further includes: evaluating the target font image.
[0012] In some exemplary embodiments, evaluating the target font image includes: refining the font style to obtain a local style representation; and evaluating the target font image based on the local style representation.
[0013] In the second aspect, an embodiment of the present application also provides a font generation system based on a neural transformation field, comprising: a neural transformation field construction module, a font style estimator construction module and a data processing module connected in sequence; the neural transformation field construction module is used to construct a neural transformation field according to the structural information of the source font; the position in the neural transformation field corresponds one-to-one to the font style, each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; the font style estimator construction module is used to construct a font style estimator, and based on the font style estimator, estimate the position of each font style in the neural transformation field to obtain an estimated position; the data processing module is used to generate multiple sampling points according to the estimated position and the font transformation path from the origin to the estimated position; and calculate the generation intensity and dissipation rate of the font pixels of the sampling points through the neural transformation field; and obtain an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels of the sampling points; and accumulate the intermediate transformation results through the font rendering formula to obtain a target font image.
[0014] In some exemplary embodiments, the neural transformation field-based font generation system further includes: an evaluation module connected to the data processing module; the evaluation module is used to evaluate the target font image.
[0015] In addition, the present application also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned font generation method based on the neural transformation field.
[0016] In addition, the present application also provides a computer-readable storage medium storing a computer program, which implements the above-mentioned font generation method based on neural transformation field when executed by a processor.
[0017] The technical solution provided by the embodiments of the present application has at least the following advantages:
[0018] An embodiment of the present application provides a font generation method, system, device and medium based on a neural transformation field, the method comprising the following steps: first, constructing a neural transformation field based on the structural information of a source font; the positions in the neural transformation field correspond one-to-one to the font styles, each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; next, constructing a font style estimator, and estimating the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; then, generating multiple sampling points based on the estimated position and the font transformation path from the origin to the estimated position; and calculating the generation intensity and dissipation rate of the font pixels of the sampling points through the neural transformation field; finally, obtaining an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels of the sampling points; and accumulating the intermediate transformation results through a font rendering formula to obtain a target font image.
[0019] This application models the font generation process as a continuous transformation process based on the generation intensity and dissipation rate of font pixels, and constructs a neural transformation field to encode the parameters required for this transformation process. This application models the font transformation process in shape space, addressing the differences between different fonts during the font style transformation process. This effectively addresses the problem of existing methods lacking spatial transformation modeling during font style transformation, and improves the performance of font generation methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] One or more embodiments are exemplarily described by the pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Unless otherwise stated, the pictures in the drawings do not constitute proportional limitations.
[0021] FIG1 is a flow chart of a font generation method based on a neural transformation field according to an embodiment of the present application;
[0022] FIG2 is a flow chart of a font generation method based on a neural transformation field according to another embodiment of the present application;
[0023] FIG3 is a comparison diagram of a target font image generated by a font generation method based on a neural transformation field and other methods provided by an embodiment of the present application;
[0024] FIG4 is a schematic structural diagram of a font generation system based on neural transformation fields according to an embodiment of the present application;
[0025] FIG5 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0026] As can be seen from the background technology, existing methods generally attempt to improve font generation capabilities by exploring powerful style representation methods. However, this is a suboptimal solution for the font generation task because the differences between font styles mainly come from shape deformation and transformation of the source glyphs. Existing methods lack modeling of spatial transformations when transforming font styles.
[0027] In order to solve the above technical problems, an embodiment of the present application provides a font generation method based on a neural transformation field, comprising the following steps: first, based on the structural information of the source font, a neural transformation field is constructed; the position in the neural transformation field corresponds one-to-one to the font style, each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; next, a font style estimator is constructed, and based on the font style estimator, the position of each font style in the neural transformation field is estimated to obtain an estimated position; then, based on the estimated position and the font transformation path from the origin to the estimated position, a plurality of sampling points are generated; and the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field; finally, based on the generation intensity and dissipation rate of the font pixels of the sampling points, an intermediate transformation result is obtained; and the intermediate transformation results are accumulated through the font rendering formula to obtain the target font image. This application models the font transformation process in the shape space in response to the differences between different fonts in the font style transformation process, effectively solving the problem that the existing methods lack spatial transformation modeling during font style transformation, and improving the performance of the font generation method.
[0028] The following detailed description of the various embodiments of the present application is provided in conjunction with the accompanying drawings. However, those skilled in the art will appreciate that many technical details are provided in the various embodiments of the present application to facilitate a better understanding of the present application. However, even without these technical details and the various variations and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.
[0029] 1 , an embodiment of the present application provides a font generation method based on a neural transformation field, comprising the following steps:
[0030] Step S1: Construct a neural transformation field based on the structural information of the source font; the positions in the neural transformation field correspond one-to-one to the font style, each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font.
[0031] Step S2: construct a font style estimator, and estimate the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position.
[0032] Step S3: Generate multiple sampling points based on the estimated position and the font transformation path from the origin to the estimated position; and calculate the generation intensity and dissipation rate of the font pixels of the sampling points through the neural transformation field.
[0033] Step S4: obtaining an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels at the sampling points; and accumulating the intermediate transformation results through a font rendering formula to obtain a target font image.
[0034] In response to the technical problem of the lack of modeling of spatial transformation when converting font styles in existing methods, the present application models the font generation process as a continuous conversion process through the generation intensity and dissipation rate of font pixels, and constructs a neural transformation field to encode the parameters required for the above transformation process. Specifically, the present application constructs a neural transformation field (NTF) based on the structural information of the source characters. Each position in the NTF represents a specific transformation, and the path from the origin to this position corresponds to the transformation process from the source font (standard font) to the target font. Therefore, each font style has a specific position, which is related to the transformation process required to generate the font image. The present application uses a font style estimator to estimate this position. Using the estimated position and the corresponding transformation path, the NTF generates a series of intermediate transformations through a sampling process, and then accumulates these intermediate transformation results through the font rendering formula proposed in this application to finally obtain the target font image.
[0035] It should be noted that the source font (source character) in step S1 is a standard font. Step S2 models the font generation process as a continuous font conversion process and constructs a neural transformation field to encode the parameters required for the above transformation process. Therefore, this application uses a font style estimator to estimate the position of a certain style in the neural transformation field.
[0036] In some embodiments, step S3 generates a plurality of sampling points based on the estimated position and the font transformation path from the origin to the estimated position; and calculates the generation intensity and dissipation rate of the font pixels at the sampling points through the neural transformation field, including:
[0037] Step S301: Divide the font transformation path from the origin to the estimated position into multiple segments, and perform sampling at the end of each segment.
[0038] Step S302: stack the estimated position and the structural information of the source font, and send them into the neural transformation field to calculate the generation intensity and dissipation rate of the font pixel at the sampling point.
[0039] Each position in the NTF represents a specific transformation, and the path from the origin to this position corresponds to the transformation process from the source font (standard font) to the target font. Therefore, each font style has a specific position that is related to the transformation process required to generate the font image. After the font style estimator estimates the style position θ, the path from the origin to this position θ is the transformation process required for font generation. This application calculates a series of intermediate transformations corresponding to the generation process by sampling.
[0040] Specifically, this application divides the transformation path into N segments At the end of each segment (θ i =iξ) for sampling. This application will i The structural information of the characters is stacked together and sent to the neural transformation field to calculate the font pixel generation intensity of the sampling point and dissipation rate τ i .
[0041] By solving the formula for the font intensity at the position in the NTF, we can get the font rendering formula. Specifically, the font intensity at position ω can be expressed as:
[0042] Among them, the first term in the above formula represents the generation process of text pixels, and the second term represents the dissipation process.
[0043] The solution of the above formula is:
[0044] The above formula is the font rendering formula, where
[0045] In practical applications, the above equation is solved by numerical methods. Specifically, this application divides the integral path into N segments At the end of each segment (θ i =iξ) for sampling. Therefore, the font rendering formula can be approximated as follows:
[0046] in, θ represents the font transformation path from the origin to the estimated position; ξ represents the segments into which the font transformation path from the origin to the estimated position is divided, τ i Indicates the dissipation rate of the font pixel at the sampling point.
[0047] 2 , in some embodiments, the font generation method based on the neural transformation field further includes:
[0048] Step S5: Evaluate the target font image.
[0049] In some embodiments, step S5 of evaluating the target font image includes:
[0050] Step S501: Refine the font style to obtain a local style representation.
[0051] Step S502: Evaluate the target font image based on the local style representation.
[0052] Because font styles contain many fine-grained structures and local correlations, this application further refines font styles into a series of local style representations and conducts extensive experiments to evaluate the proposed model. The experimental results, as shown in Figure 3, show that the proposed model demonstrates excellent performance in the font generation task.
[0053] The key to the font generation method based on neural transformation field provided by this application is that this application models the font generation process as a continuous conversion process. After trying a variety of theoretical modeling methods and different network structures, the R&D personnel found that modeling the font generation process using the generation intensity and dissipation rate of font pixels is a better choice, and achieved high-performance font generation through a carefully designed font style estimator and a font pixel generation intensity and dissipation rate prediction layer. Therefore, after multiple experiments and analyses, this application proposes a font generation method based on neural transformation field, which uses conventional training data and training methods in model training, and can still achieve the best results after controlling the variables in the font generation process. This is enough to show that the model of this application itself is effective and can be evaluated and verified by a variety of data.
[0054] 4 , an embodiment of the present application further provides a font generation system based on a neural transformation field, comprising: a neural transformation field construction module 101, a font style estimator construction module 102, and a data processing module 103 connected in sequence; the neural transformation field construction module 101 is used to construct a neural transformation field based on the structural information of the source font; the position in the neural transformation field corresponds one-to-one to the font style, each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; the font style estimator construction module 102 is used to construct a font style estimator, and estimate the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; the data processing module 103 is used to generate multiple sampling points based on the estimated position and the font transformation path from the origin to the estimated position; and calculate the generation intensity and dissipation rate of the font pixels of the sampling points through the neural transformation field; and obtain an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels of the sampling points; and accumulate the intermediate transformation results through the font rendering formula to obtain a target font image.
[0055] In some exemplary embodiments, the neural transformation field-based font generation system further includes: an evaluation module 104 connected to the data processing module 103; the evaluation module 104 is configured to evaluate the target font image.
[0056] The neural transformation field-based font generation method and system provided in the embodiments of the present application have the advantages of high accuracy and strong universality compared to the existing technology. On the one hand, the model of the present application is able to generate high-performance font images, and the visualization results and performance indicators on the data set are better than those of the existing methods. Therefore, this shows that the model of the present application is fully capable of being applied to real-world scenarios. On the other hand, the model of the present application has strong universality and portability. For any font generation model, after simple modification, the method of the present application can be used to generate high-performance font images.
[0057] Referring to Figure 5, another embodiment of the present application provides an electronic device, including: at least one processor 110; and a memory 111 communicatively connected to the at least one processor; wherein the memory 111 stores instructions that can be executed by the at least one processor 110, and the instructions are executed by the at least one processor 110 to enable the at least one processor 110 to execute any of the above method embodiments.
[0058] The memory 111 and the processor 110 are connected using a bus. The bus may include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors 110 and the memory 111. The bus may also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor 110 is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits the data to the processor 110.
[0059] The processor 110 is responsible for managing the bus and general processing, and may also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 111 may be used to store data used by the processor 110 when performing operations.
[0060] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.
[0061] Based on the above technical solutions, the embodiments of the present application provide a method, system, device, and medium for font generation based on neural transformation fields. The method includes the following steps:
[0062] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps in the above-mentioned methods of each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.
[0063] Based on the above technical solution, the embodiment of the present application provides a font generation method, system, device and medium based on a neural transformation field, which includes the following steps: first, based on the structural information of the source font, a neural transformation field is constructed; the position in the neural transformation field corresponds one-to-one to the font style, each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; next, a font style estimator is constructed, and based on the font style estimator, the position of each font style in the neural transformation field is estimated to obtain an estimated position; then, based on the estimated position and the font transformation path from the origin to the estimated position, multiple sampling points are generated; and the generation intensity and dissipation rate of the font pixels of the sampling points are calculated through the neural transformation field; finally, based on the generation intensity and dissipation rate of the font pixels of the sampling points, an intermediate transformation result is obtained; and the intermediate transformation results are accumulated through the font rendering formula to obtain the target font image.
[0064] This application models the font generation process as a continuous transformation process based on the generation intensity and dissipation rate of font pixels, and constructs a neural transformation field to encode the parameters required for this transformation process. This application models the font transformation process in shape space, addressing the differences between different fonts during the font style transformation process. This effectively addresses the problem of existing methods lacking spatial transformation modeling during font style transformation, and improves the performance of font generation methods.
[0065] Those skilled in the art will appreciate that the above-described embodiments are specific examples for implementing the present application, and that in actual applications, various changes in form and detail may be made thereto without departing from the spirit and scope of the present application. Any person skilled in the art may make changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the scope defined in the claims.
Claims
1. A font generation method based on neural transformation field, characterized in that: include: Based on the structural information of the source font, a neural transformation field is constructed; The positions in the neural transformation field correspond one-to-one to the font styles, each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; Constructing a font style estimator, and estimating the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; generating a plurality of sampling points based on the estimated position and a font transformation path from an origin to the estimated position; and calculating the generation intensity and dissipation rate of the font pixels at the sampling points through the neural transformation field; Based on the generation intensity and dissipation rate of the font pixels at the sampling points, an intermediate transformation result is obtained; The intermediate transformation results are accumulated through a font rendering formula to obtain a target font image.
2. The font generation method based on neural transformation field according to claim 1 is characterized in that: generating a plurality of sampling points based on the estimated position and a font transformation path from an origin to the estimated position; The generation intensity and dissipation rate of the font pixels at the sampling points are calculated by using the neural transformation field, including: Dividing the font transformation path from the origin to the estimated position into a plurality of segments, and sampling at the end of each segment; The estimated position and the structural information of the source font are stacked together and sent to the neural transformation field to calculate the generation intensity and dissipation rate of the font pixel of the sampling point.
3. The font generation method based on neural transformation field according to claim 1, characterized in that: The font rendering formula is as follows: in, θ represents the font transformation path from the origin to the estimated position; ξ represents the segments into which the font transformation path from the origin to the estimated position is divided, τ i Indicates the dissipation rate of the font pixel at the sampling point.
4. The font generation method based on neural transformation field according to claim 1, characterized in that: Also includes: The target font image is evaluated.
5. The font generation method based on neural transformation field according to claim 4 is characterized in that: The target font image is evaluated, including: Refine the font style to obtain local style representation; The target font image is evaluated based on the local style representation.
6. A font generation system based on neural transformation field, characterized in that: include: A neural transformation field building module, a font style estimator building module, and a data processing module connected in sequence; The neural transformation field construction module is used to construct a neural transformation field according to the structural information of the source font; the positions in the neural transformation field correspond to the font styles one by one, each position represents a specific transformation, and the path from the origin to the target position corresponds to the transformation process from the source font to the target font; The font style estimator construction module is used to construct a font style estimator, and estimate the position of each font style in the neural transformation field based on the font style estimator to obtain an estimated position; The data processing module is used to generate a plurality of sampling points according to the estimated position and a font transformation path from an origin to the estimated position; and calculating the generation intensity and dissipation rate of the font pixels at the sampling points through the neural transformation field; and obtaining an intermediate transformation result based on the generation intensity and dissipation rate of the font pixels at the sampling points; The intermediate transformation results are accumulated through a font rendering formula to obtain a target font image.
7. The font generation system based on neural transformation field according to claim 6, characterized in that: Also includes: an evaluation module connected to the data processing module; The evaluation module is used to evaluate the target font image.
8. An electronic device, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so as to enable the at least one processor to execute the font generation method based on the neural transformation field as described in any one of claims 1 to 5.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the font generation method based on the neural transformation field described in any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Method and system for converting Chinese character fonts in image, computer equipment and medium
CN110135530A
Generation method of Chinese character font generation model and Chinese character font generation method and device
CN113962192A
Font style sampling method, electronic equipment, storage medium and system
CN115512163A
Font creation apparatus, font creation method, and font creation program
US20210201548A1
