Artificial intelligence-based character library processing method, device, equipment and storage medium

Abstract

This application provides an artificial intelligence-based font processing method, device, electronic device, and computer-readable storage medium; it relates to artificial intelligence technology and includes: determining a first style of a font and a target character of the font; performing character encoding processing on a character image of the target character in the second style to obtain target character features of the character image; fusing the first style and the target character features to obtain a character fusion feature; decoding the character fusion feature to obtain a character image of the target character in the first style, and updating the font using the character image of the target character in the first style. This application enables efficient and automated processing of updating fonts.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to an artificial intelligence-based character library processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] Artificial Intelligence (AI) is a comprehensive field of computer science. By studying the design principles and implementation methods of various intelligent machines, AI enables them to possess the capabilities of perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing a wide range of fields, including natural language processing and machine learning / deep learning. With technological advancements, AI will be applied in even more areas and play an increasingly important role.

[0003] Fonts are a way to display characters in computers. In related technologies, font management (such as generating fonts, modifying fonts, etc.) requires manual operation by designers, which has low processing efficiency and cannot meet the diverse and rapid demand for fonts in the Internet era.

[0004] Related technologies lack solutions for efficient processing of character libraries based on artificial intelligence. Summary of the Invention

[0005] The embodiments of the present application provide an artificial intelligence-based character library processing method, device, electronic device, and computer-readable storage medium to achieve efficient and automated processing of updating character libraries.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] The present invention provides an image object recognition method, comprising:

[0008] determining a first style of a character library and a target character of the character library;

[0009] performing character encoding processing on the character image of the target character in the second style to obtain target character features of the character image;

[0010] fusing the first style and the target character feature to obtain a character fusion feature;

[0011] The character fusion feature is decoded to obtain a character image of the target character in a first style, and the character library is updated using the character image of the target character in the first style.

[0012] In the above technical solution, the fusion processing of the first style and the target character feature to obtain the character fusion feature includes:

[0013] Performing mapping processing on the first style and the target character features to obtain an intermediate feature vector including the first style and the target character;

[0014] An affine transformation is performed on the intermediate feature vector including the first style and the target character to obtain the character fusion feature.

[0015] In the above technical solution, determining the first style of the font library includes:

[0016] Determining the number of character images in the candidate character library;

[0017] When the number of the character images is less than a threshold value of the number of images, the candidate character library is used as the character library, and the style of the character library is used as the first style.

[0018] In the above technical solution, determining the target character of the character library includes:

[0019] determining a candidate character library different from the first style;

[0020] Screening the character images in the candidate character library to obtain a character image containing the target character;

[0021] The character image containing the target character does not exist in the character library.

[0022] In the above technical solution, the character images in the candidate character library are screened to obtain a character image containing the target character, including:

[0023] determining a frequency at which candidate character images in the candidate character library are selected;

[0024] When the selected frequency is greater than a selection frequency threshold, the candidate character image is used as the character image containing the target character.

[0025] The present invention provides an image object recognition method, comprising:

[0026] presenting a character library, the character library including at least one character of a first style;

[0027] In response to an update operation on the character library, a generation model is called to generate a character image of the first style based on a character image of the second style of the target character, and

[0028] presenting the updated character library based on the character image of the first style of the target character;

[0029] The target character includes at least one of a missing character and a damaged character in the character library.

[0030] The present invention provides a character library processing device, comprising:

[0031] a determination module, configured to determine a first style of a character library and a target character of the character library;

[0032] an encoding module, configured to perform character encoding processing on the character image of the target character in the second style to obtain target character features of the character image;

[0033] a fusion module, configured to fuse the first style and the target character feature to obtain a character fusion feature;

[0034] A decoding module is used to decode the character fusion feature to obtain a character image of the first style of the target character, and update the character library through the character image of the first style of the target character.

[0035] In the above technical solution, the encoding module is further used to perform feature extraction processing on the character image of the second style of the target character to obtain a feature map of the character image;

[0036] The style space information in the feature map is removed, and the feature map with the style space information removed is used as the target character feature of the character image.

[0037] In the above technical solution, the fusion module is further used to perform encoding processing on the first style to obtain encoding features of the first style;

[0038] The encoding feature of the first style and the target character feature are concatenated to obtain the character fusion feature.

[0039] In the above technical solution, the fusion module is further used to perform mapping processing on the first style and the target character features to obtain an intermediate feature vector containing the first style and the target character;

[0040] An affine transformation is performed on the intermediate feature vector including the first style and the target character to obtain the character fusion feature.

[0041] In the above technical solution, the decoding process is implemented by a decoding model, which includes multiple cascaded decoding layers, and the multiple cascaded decoding layers are obtained through progressive training; the decoding module is also used to perform cascaded upsampling decoding processing on the character fusion features through the multiple cascaded decoding layers, and

[0042] The decoding result output by the last decoding layer is used as the character image of the first style of the target character.

[0043] In the above technical solution, the progressive training is achieved through the following process: performing cascade upsampling decoding processing on the character fusion features through the multiple cascaded decoding layers;

[0044] Based on the weights of the plurality of cascaded decoding layers, performing weighted summation processing on the decoding results respectively output by the plurality of cascaded decoding layers, and using the result of the weighted summation processing as the false character image sample;

[0045] The weight of the last decoding layer gradually increases as the number of times the generation model is updated increases, and the weights of other decoding layers gradually decrease as the number of times the generation model is updated increases;

[0046] The character fusion feature is obtained by performing character encoding processing on a real character image sample by the encoding model.

[0047] In the above technical solution, the character encoding process and the character decoding process are implemented by a generative model in a generative adversarial network, and the generative adversarial network also includes a discriminative model; the device also includes:

[0048] A training module, configured to train the generative adversarial network by:

[0049] Initialize the generative model and the discriminative model, and perform the following training tasks alternately:

[0050] Based on a real first character image sample and a real second character image sample, training the generation model so that the generation network generates a false character image sample based on the real first character image sample;

[0051] The real first character image sample includes a target character sample of a first style sample, and the real second image sample and the false character image sample include a target character sample of a second style sample;

[0052] Based on the real second character image sample and the false character image sample, training the discriminant model so that the discriminant model can distinguish between the real character image sample and the false character image sample;

[0053] When the generative model is trained, the parameters of the discriminative model are fixed; when the discriminative model is trained, the parameters of the generative model are fixed.

[0054] In the above technical solution, the generation model includes an encoding model and a decoding model; the training module is further used to perform character encoding processing on the real first character image sample through the encoding model to obtain the target character sample features of the real first character image sample;

[0055] Fusing the first style sample and the target character sample features to obtain a character fusion feature;

[0056] Decoding the character fusion feature through the decoding model to generate the false character image sample;

[0057] Constructing a loss function of the generation model based on the real second character image sample and the false character image sample;

[0058] Parameters of the generative model are updated based on the loss function of the generative model.

[0059] In the above technical solution, when the weight of the last decoding layer is less than 1, the training module is further used to obtain an image distance loss function based on the distance between the real second character image sample and the false character image sample.

[0060] The image distance loss function is used as the loss function of the generative model.

[0061] In the above technical solution, when the weight of the last decoding layer is equal to 1, the training module is further used to perform character encoding processing on the false character image sample through the encoding model to obtain the target character sample feature of the false character image sample;

[0062] Performing character encoding processing on the real second character image sample by using the encoding model to obtain target character sample features of the real second character image sample;

[0063] constructing a character encoding loss function based on the target character sample features of the false character image sample and the target character sample features of the real second character image sample;

[0064] A weighted summation process is performed on the image distance loss function and the character encoding loss function, and a result of the weighted summation process is used as the loss function of the generation model.

[0065] In the above technical solution, the training module is further used to generate the false character image samples through the generation model;

[0066] Using the real second character image sample and the false character image sample as training samples, and determining the predicted probability that the training sample belongs to a real character image through the discriminant model;

[0067] Constructing a loss function of the discriminant model based on the difference between the predicted probability and the true probability of the training sample;

[0068] The parameters of the discriminant model are updated based on the loss function of the discriminant model.

[0069] In the above technical solution, the training module is further configured to, when the predicted probability indicates that the training sample is a real character image, perform style classification processing on the training sample using the discriminant model to obtain a predicted style of the training sample;

[0070] Constructing a style classification loss function based on the predicted style and the true style of the training sample;

[0071] constructing a authenticity loss function based on the difference between the predicted probability and the true probability of the training sample;

[0072] A weighted summation process is performed on the authenticity loss function and the style classification loss function, and a result of the weighted summation process is used as the loss function of the discriminant model.

[0073] In the above technical solution, the determination module is further used to determine the number of character images in the candidate character library;

[0074] When the number of the character images is less than a threshold value of the number of images, the candidate character library is used as the character library, and the style of the character library is used as the first style.

[0075] In the above technical solution, the determining module is further used to determine a candidate character library that is different from the first style;

[0076] Screening the character images in the candidate character library to obtain a character image containing the target character;

[0077] The character image containing the target character does not exist in the character library.

[0078] In the above technical solution, the determination module is further used to determine the frequency with which the candidate character images in the candidate character library are selected;

[0079] When the selected frequency is greater than a selection frequency threshold, the candidate character image is used as the character image containing the target character.

[0080] The present invention provides a character library processing device, comprising:

[0081] a presentation module, configured to present a character library, wherein the character library includes at least one character of a first style;

[0082] a generating module, configured to, in response to an update operation on the character library, call a generating model to generate a character image of the first style based on a character image of the second style of a target character;

[0083] The presentation module is further configured to present the character library updated based on the character image of the first style of the target character; wherein the target character includes at least one of a missing character and a damaged character in the character library.

[0084] An embodiment of the present application provides an electronic device for character library processing, the electronic device comprising:

[0085] a memory for storing executable instructions;

[0086] The processor is configured to implement the image target recognition method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0087] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the artificial intelligence-based character library processing method provided in the embodiment of the present application.

[0088] The embodiments of the present application have the following beneficial effects:

[0089] By combining the second style character image of the target character with the first style of the font library, a first style character image of the target character is generated, and the font library is updated based on the first style character image of the target character, thereby efficiently completing the font library or generating a new font library. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 This is a schematic diagram of an application scenario of the character library processing system provided in an embodiment of the present application;

[0091] Figure 2-Figure 3 is a structural diagram of an electronic device for character library processing provided by an embodiment of the present application;

[0092] Figures 4A-4C 1 is a flow chart of a method for processing character libraries based on artificial intelligence provided by an embodiment of the present application;

[0093] Figure 5 Schematic diagram of the process of progressive training provided in the embodiment of the present application;

[0094] Figure 6 1 is a flow chart of a method for processing character libraries based on artificial intelligence provided by an embodiment of the present application;

[0095] Figure 7 This is a schematic diagram of the network framework provided by the embodiment of the present application;

[0096] Figure 8A Schematic diagram of a low-resolution plateau phase provided in an embodiment of the present application;

[0097] Figure 8BSchematic diagram of the resolution growth transition period provided by the embodiment of the present application;

[0098] Figure 8C Schematic diagram of a high-resolution plateau phase provided in an embodiment of the present application;

[0099] Figures 9A-9C 2 is a comparative diagram of the font character image results provided in the embodiment of the present application. DETAILED DESCRIPTION

[0100] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0101] In the following description, the terms "first\second" are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0102] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0103] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0104] 1) Generative Adversarial Networks (GAN): A deep learning model that has emerged as one of the most promising unsupervised learning methods for complex distributions in recent years. It produces highly accurate outputs through a game of learning between a generator and a discriminator. Discriminator models require input variables for prediction, while generative models require implicit information to randomly generate observational data (generated fake data). For example, given a character image, a discriminator can determine whether it is a real character image or a generated fake one. A generative model can generate a new character image (not in the dataset) given a character image of the same style (a dataset).

[0105] 2) Style: The representative visual art form presented by the character image as a whole. The style of a character image is at least manifested in the font type used. Character images of different styles have different font types. For example, Chinese fonts such as Songti, Kaiti, Handwriting, and Artistic Fonts all belong to different styles, and English fonts such as Times New Roma and Arial also belong to different styles. The style of a character image can also be manifested in a combination of fonts and visual elements (such as decorative static or dynamic special effects).

[0106] The embodiments of the present application provide an artificial intelligence-based font processing method, device, electronic device, and computer-readable storage medium, which can automatically update the font based on the generated character image.

[0107] The artificial intelligence-based font processing method provided in the embodiments of the present application can be implemented by the terminal / server alone; it can also be implemented by the terminal and the server in collaboration. For example, the terminal alone undertakes the artificial intelligence-based font processing method described below, or the terminal sends an update request for the font to the server (the font includes multiple characters of a first style (i.e., the style of the character image in the font, such as the font type presented by the character image)), and the server executes the artificial intelligence-based font processing method based on the received update request for the font, generates a character image of the first style of the target character in response to the update request for the font, and automatically completes the font based on the generated character image of the first style of the target character.

[0108] The electronic device for font library processing provided in the embodiments of the present application can be various types of terminal devices or servers, wherein the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services; the terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0109] Taking servers as an example, it can be a server cluster deployed in the cloud, opening artificial intelligence cloud services (AI as a Service, AIaaS) to users. The AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to an AI theme mall. All users can access and use one or more artificial intelligence services provided by the AIaaS platform through the application programming interface.

[0110] For example, one of the artificial intelligence cloud services can be a font processing service, that is, the server in the cloud is encapsulated with the font processing program provided by the embodiment of the present application. The user calls the font processing service in the cloud service through a terminal (running a client, such as a font completion client, etc.), so that the server deployed in the cloud calls the encapsulated font processing program, and based on the character image of the second style (different from the first style) of the target character and the first style of the font, generates a character image of the first style of the target character, and updates the font through the character image of the first style of the target character. For example, for a font completion application, the font is updated through the character image of the first style of the target character to complete the missing or damaged characters in the font, providing more data support for projects involving the use of fonts to generate data, enriching data distribution, and improving project effects.

[0111] See also Figure 1 , Figure 1 This is a schematic diagram of an application scenario of the font processing system 10 provided in an embodiment of the present application. The terminal 200 is connected to the server 100 via a network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0112] Terminal 200 (running a client, such as a font completion client, etc.) can be used to obtain an update request for the font library. For example, when the user determines that there are missing or damaged characters in the font library, the user inputs an instruction to update the font library to the terminal 200, that is, inputs the identifier of the first style of the font library (for example, the identifier of "Kaiti"), and the terminal automatically obtains the update request for the font library (including the first style of the font library).

[0113] In some embodiments, a font processing plug-in may be implanted in the client running in the terminal to implement an artificial intelligence-based font processing method locally on the client. For example, after the terminal 200 obtains an update request for the font (including the first style of the font), it calls the font processing plug-in to implement an artificial intelligence-based font processing method, generates a first-style character image of the target character based on the second-style character image and the first style of the font, updates the font through the first-style character image of the target character, and completes the missing or damaged characters in the font, for example, completing the missing or damaged characters in the regular font, providing more data support for projects involving the use of fonts to generate data, enriching data distribution, and improving project effects.

[0114] In some embodiments, after the terminal 200 obtains an update request for the font library, it calls the font processing interface of the server 100 (which can be provided in the form of a cloud service, i.e., a font processing service). The server 100 calls the machine learning model and generates a character image of the first style of the target character based on the character image of the second style of the target character and the first style of the font library. For example, for a font completion application, the terminal 200 automatically generates an update request for the font library through user input, and sends the update request for the font library to the server 100. The server 100 parses the update request for the font library, obtains the first style of the font library, and determines the target character of the font library. Based on the character image of the second style of the target character and the first style of the font library, it generates a character image of the first style of the target character, and sends the character image of the first style of the target character to the terminal 200. The terminal 200 updates the font library through the character image of the first style of the target character to complete the missing or damaged characters in the font library, for example, to complete the missing or damaged characters in the regular font library, thereby providing more data support for projects involving the use of fonts to generate data, enriching data distribution, and improving project effects.

[0115] The following describes the structure of the electronic device for character library processing provided by the embodiment of the present application. Figure 2 , Figure 2 1 is a structural diagram of an electronic device 500 for font library processing provided in an embodiment of the present application, and takes the electronic device 500 as a server as an example. Figure 2 The electronic device 500 for character library processing shown includes: at least one processor 510, a memory 550, at least one network interface 520 and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 540 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 540 is not described in detail. Figure 2 Various buses are labeled as bus system 540 .

[0116] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0117] The memory 550 includes a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory. The memory 550 may optionally include one or more storage devices physically remote from the processor 510.

[0118] In some embodiments, the memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0119] Operating system 551, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0120] A network communication module 552 for reaching other computing devices via one or more (wired or wireless) network interfaces 520 , exemplary network interfaces 520 including Bluetooth, WiFi, and USB;

[0121] In some embodiments, the font processing device provided in the embodiments of the present application can be implemented in software. For example, it can be a font processing plug-in in the terminal described above, or it can be a font processing service in the server described above. Of course, this is not limited to this. The font processing device provided in the embodiments of the present application can be provided as various software embodiments, including various forms such as application programs, software, software modules, scripts, or code.

[0122] Figure 2 A font processing device 555 stored in a memory 550 is shown, which can be software in the form of a program and a plug-in, such as a font processing plug-in, and includes a series of modules, including a determination module 5551, an encoding module 5552, a fusion module 5553, a decoding module 5554 and a training module 5555; wherein the determination module 5551, the encoding module 5552, the fusion module 5553, and the decoding module 5554 are used to implement the font processing function provided in the embodiment of the present application, and the training module 5555 is used to train a generative adversarial network.

[0123] The following describes an electronic device for character library processing provided by an embodiment of the present application (eg Figure 1 The structure of the terminal 200 in Figure 3 , Figure 3 : is a structural diagram of an electronic device 600 for character library processing provided in an embodiment of the present application. Figure 3 The electronic device 600 for character library processing shown includes at least one processor 610, a memory 650, at least one network interface 620, and a user interface 630. The various components in the electronic device 600 are coupled together via a bus system 640. The functions of the processor, memory, network interface, and user interface in the electronic device 600 are similar to those in the electronic device 500 and will not be further described here.

[0124] Figure 3 A font processing device 655 stored in a memory 650 is shown, which can be software in the form of a program and a plug-in, and includes a series of modules, including a presentation module 6551 and a generation module 6552; wherein, the presentation module 6551 and the generation module 6552 are used to implement the artificial intelligence-based font processing method provided in an embodiment of the present application.

[0125] As mentioned above, the AI-based character library processing method provided in the embodiments of the present application can be implemented by various types of electronic devices. Figure 4A , Figure 4A This is a flow chart of the method for processing character database based on artificial intelligence provided by the embodiment of the present application, combined with Figure 4A The steps shown are explained.

[0126] In the following steps, the target character is a missing or damaged character in the font library. Style refers to the overall representative visual artistic expression of the character image. The style of a character image is at least manifested in the font type used. Character images of different styles have different font types, such as Songti and Kaiti. The style of a character image can also be manifested as a combination of fonts and visual elements (such as decorative static or dynamic effects).

[0127] In step 101 , a first style of a character library and a target character of the character library are determined.

[0128] As an example of obtaining the first style and the target character, the user can input the font library of the first style that needs to be supplemented or updated and the target character to be generated on the input interface of the terminal. For example, input the identifier of "KaiTi" on the input interface of the terminal, that is, the terminal takes "KaiTi" as the first style of the font library that needs to be supplemented or updated. Input the character "Xing" on the input interface of the terminal, then the terminal takes "Xing" as the target character to be generated. After the input is completed, the terminal automatically obtains an update request for the font library (including the first style and the target character, that is, the "KaiTi" style and the character "Xing"), and sends the update request to the server. After receiving the update request for the font library, the server analyzes the update request for the font library to obtain the first style of the font library and the target character of the font library, so as to generate character images based on the first style of the font library and the target character of the font library subsequently.

[0129] In some embodiments, determining the first style of the font library includes: determining the number of character images in the candidate font library; when the number of character images is less than the image number threshold, taking the candidate font library as the font library and taking the style of the font library as the first style.

[0130] For example, there are various candidate font libraries in the database, and the styles of different candidate font libraries are different (that is, the font types of the characters in different candidate font libraries are different). For example, the character images in the first font library are all in the KaiTi form, and the character images in the second font library are all in the SongTi form. For any candidate font library, when the number of character images in the candidate font library is less than the image number threshold, it means that the character images in this candidate font library are incomplete, and the character images in this candidate font library need to be supplemented to complete this candidate font library and implement the function of automatically completing the font library.

[0131] In some embodiments, determining the target character of the font library includes: determining a candidate font library different from the first style; performing a screening process on the character images in the candidate font library to obtain a character image containing the target character; wherein the character image containing the target character does not exist in the font library.

[0132] For example, traverse the character images in the candidate font library other than this font library (the style of the candidate font library is different from the first style). When it is determined that the character image including a certain character does not exist in this font library, it means that this font library does not include this kind of character, then this character is determined as the target character. For example, the first style is KaiTi, traverse the candidate font libraries other than the KaiTi font library. When it is determined that the character "Xing" in the candidate font library does not exist in the KaiTi font library, then the character "Xing" is determined as the target character to be generated.

[0133] Continuing with the above example, the character images in the candidate character library are screened to obtain character images containing the target character, including: determining the frequency with which the candidate character images in the candidate character library are selected; when the frequency of selection is greater than the selection frequency threshold, the candidate character image is used as the character image containing the target character.

[0134] For example, each font library needs to have popular characters, common characters, etc., and the frequency of selection of candidate character images in the candidate font library can be used to determine whether they are popular characters or common characters, so as to complete the popular characters and common characters in each font library.

[0135] In step 102, character encoding processing is performed on the character image of the target character in the second style to obtain target character features of the character image.

[0136] After obtaining the first style and the target character, a second style different from the first style is determined, and target character features of the character image of the target character in the second style are extracted, so as to subsequently perform a decoding operation based on the target character features.

[0137] See also Figure 4B , Figure 4B This is an optional flow chart of the AI-based character library processing method provided in the embodiment of the present application. Figure 4B Show Figure 4A Step 102 in the above example can be achieved by Figure 4B Steps 1021-1022 shown are implemented as follows: in step 1021, feature extraction processing is performed on the character image of the second style of the target character to obtain a feature map of the character image; in step 1022, the style space information in the feature map is removed, and the feature map with the style space information removed is used as the target character feature of the character image.

[0138] For example, character encoding processing is implemented through an encoding model (character encoder), and the character recognition model includes a convolution layer and a pooling layer. The convolution layer in the encoding model performs feature extraction processing on the character image of the second style of the target character to obtain a feature map of the character image. The pooling layer in the encoding model removes the style space information in the feature map, and the feature map without the style space information is used as the target character feature of the character image, thereby stripping the style information (such as font space information) of the input character image, extracting the character-specific information in the character image, avoiding the influence of the style information, and improving the representation of the target character feature.

[0139] In step 103, the first style and the target character features are fused to obtain character fusion features.

[0140] After obtaining the first style and the target character features, it is necessary to first fuse the first style and the target character features so that subsequent decoding processing can be performed based on the character fusion features to obtain the character image of the target character in the first style.

[0141] In some embodiments, the first style and the target character features are fused to obtain character fusion features, including: encoding the first style to obtain the encoded features of the first style; concatenating the encoded features of the first style and the target character features to obtain character fusion features.

[0142] For example, the first style is encoded by an autoencoder to obtain the encoded features of the first style, and the encoded features of the first style and the target character features are concatenated to obtain character fusion features.

[0143] In some embodiments, the first style and the target character features are fused to obtain character fusion features, including: performing a mapping process on the first style and the target character features to obtain an intermediate feature vector containing the first style and the target character; performing an affine transformation on the intermediate feature vector containing the first style and the target character to obtain character fusion features.

[0144] As an example, a mapping process is performed on the first style and the target character features to obtain an intermediate feature vector containing the first style and the target character, and an affine transformation is performed on the intermediate feature vector to perform a linear transformation on the intermediate feature vector and perform a translation to transform it into another spatial vector, generating a character fusion feature containing the first style and the target character. The formula for the affine transformation is represents the intermediate feature vector, represents the character fusion feature, A, represents the learning parameters. Through mapping and affine transformation, that is, multi-layer transformation, the accuracy of the character fusion feature is improved.

[0145] In step 104, the character fusion features are decoded to obtain the character image of the target character in the first style, and the font library is updated with the character image of the target character in the first style.

[0146] After obtaining the character fusion features containing the first style and the target character, the character fusion features are decoded to generate the character image of the target character in the first style. The font library is updated with the character image of the target character in the first style. For example, the first style is regular script, the second style is Song typeface, and the target character is "幸". Based on the character image of "幸" with Song typeface style, a character image of "幸" with regular script style is generated, and the regular script font library is updated with the character image of "幸" with regular script style to realize the function of automatically completing the font library.

[0147] In some embodiments, the decoding process is implemented through a decoding model, which includes multiple cascaded decoding layers, and the multiple cascaded decoding layers are obtained through progressive training; the character fusion features are decoded to obtain a first-style character image of the target character, including: cascaded upsampling decoding processing of the character fusion features through multiple cascaded decoding layers, and the decoding result output by the last decoding layer is used as the first-style character image of the target character.

[0148] For example, the character fusion features are upsampled and decoded through the first decoding layer in multiple cascaded decoding layers, and the decoding results of the first decoding layer are output to the subsequent cascaded decoding layers, so as to continue the upsampled decoding processing and decoding result output in the subsequent cascaded decoding layers until it is output to the last decoding layer, and the decoding result output by the last decoding layer is used as the first style character image of the target character.

[0149] In some embodiments, progressive training is achieved through the following process: performing cascaded upsampling decoding processing on the character fusion features through multiple cascaded decoding layers; based on the weights of the multiple cascaded decoding layers, performing weighted summation processing on the decoding results output by the multiple cascaded decoding layers respectively, and using the result of the weighted summation processing as a false character image sample.

[0150] Among them, the weight of the last decoding layer gradually increases as the number of times the generation model is updated increases, and the weights of other decoding layers gradually decrease as the number of times the generation model is updated increases. Among them, the character fusion feature is obtained by the encoding model performing character encoding processing on real character image samples.

[0151] like Figure 5 As shown, in the process of progressive training, the character fusion feature is upsampled and decoded by the first decoding layer in multiple cascaded decoding layers to obtain the decoding result y1 of the first decoding layer, and y1 is output to the second decoding layer. The second decoding layer upsamples and decodes y1 to obtain the decoding result y2 of the second decoding layer. The above steps are iterated to obtain the decoding result y of the Nth decoding layer. N , where N represents the number of decoding layers. Based on the weights of multiple cascaded decoding layers, the decoding results output by the multiple cascaded decoding layers are weighted and summed, and the result of the weighted summation is used as a false character image sample. The weight α of the Nth decoding layer is N As the number of times the generation model is updated increases, the weights of other decoding layers (α1, α2, ..., α N-1 ) gradually decreases as the number of times the model is updated, where α1+α2,…,α N =1, until the end of progressive training, αN is 1. The progressive training strategy can ensure stable convergence of the network.

[0152] See also Figure 4C , Figure 4C This is an optional flow chart of the AI-based character library processing method provided in the embodiment of the present application. Figure 4C Show Figure 4A It also includes steps 105 and 106: character encoding processing and character decoding processing are implemented by a generative model in a generative adversarial network, and the generative adversarial network also includes a discriminant model; the generative adversarial network is trained in the following manner: the generative model and the discriminant model are initialized, and the following training tasks are performed alternately: in step 105, the generative model is trained based on the real first character image sample and the real second character image sample, so that the generative network generates a false character image sample based on the real first character image sample; in step 106, the discriminant model is trained based on the real second character image sample and the false character image sample, so that the discriminant model distinguishes between the real character image sample and the false character image sample.

[0153] The real first character image samples include target character samples of the first style sample, and the real second image samples and the fake character image samples include target character samples of the second style sample. When training the generative model, the parameters of the discriminative model are fixed; when training the discriminative model, the parameters of the generative model are fixed.

[0154] In some embodiments, the generation model includes an encoding model and a decoding model; based on a real first character image sample and a real second character image sample, the generation model is trained, including: performing character encoding processing on the real first character image sample through the encoding model to obtain the target character sample features of the real first character image sample; fusing the first style sample and the target character sample features to obtain character fusion features; decoding the character fusion features through the decoding model to generate a false character image sample; constructing a loss function of the generation model based on the real second character image sample and the false character image sample; and updating the parameters of the generation model based on the loss function of the generation model.

[0155] For example, the character fusion features are upsampled and decoded through the first decoding layer in multiple cascaded decoding layers, and the decoding results of the first decoding layer are output to the subsequent cascaded decoding layers, so as to continue the upsampled decoding processing and decoding result output in the subsequent cascaded decoding layers until it is output to the last decoding layer. Based on the weights of the multiple cascaded decoding layers, the decoding results output by the multiple cascaded decoding layers are weighted summed, and the result of the weighted summation is used as a false character image sample; wherein the weight of the last decoding layer gradually increases with the number of times the character generation model is updated.

[0156] In some embodiments, when the weight of the last decoding layer is less than 1, a loss function of the generation model is constructed based on the real second character image sample and the false character image sample, including: obtaining an image distance loss function based on the distance between the real second character image sample and the false character image sample, and using the image distance loss function as the loss function of the generation model.

[0157] For example, when progressive training is not completed, the decoder only uses the image distance loss function without receiving the style features (style features) of the input character image, which simplifies the search space of network parameters and helps the network converge stably to the ideal state to solve the problem that generative adversarial networks are difficult to converge.

[0158] In some embodiments, when the weight of the last decoding layer is equal to 1, a loss function of the generation model is constructed based on the real second character image sample and the false character image sample, including: performing character encoding processing on the false character image sample through the encoding model to obtain the target character sample features of the false character image sample; performing character encoding processing on the real second character image sample through the encoding model to obtain the target character sample features of the real second character image sample; constructing a character encoding loss function based on the target character sample features of the false character image sample and the target character sample features of the real second character image sample; performing weighted summation processing on the image distance loss function and the character encoding loss function, and using the result of the weighted summation processing as the loss function of the generation model.

[0159] For example, after progressive training is completed, the generative adversarial network can already output high-quality, high-resolution character images. At this time, the early parameter training of the encoding model will be released, allowing the generative adversarial network to learn character features that are more suitable for image generation, further optimizing the generation effect of character images.

[0160] In some embodiments, a discriminant model is trained based on real second character image samples and fake character image samples, including: generating fake character image samples through a generative model; using the real second character image samples and the fake character image samples as training samples, and determining the prediction probability that the training samples belong to real character images through the discriminant model; constructing a loss function of the discriminant model based on the difference between the prediction probability and the real probability of the training samples; and updating the parameters of the discriminant model based on the loss function of the discriminant model.

[0161] For example, in the discriminant model optimization stage, try to make the discriminant model distinguish real images from the fake images generated by the generative model. In the generative model optimization stage, try to make the generative model obtain images that can deceive the discriminant model. Through the mutual game between the discriminant model and the generative model, the generative model can generate character images that are as real as possible.

[0162] Among them, constructing a loss function of the discriminant model based on the difference between the prediction probability and the real probability of the training samples includes: when the prediction probability represents that the training sample is a real character image, performing style classification processing on the training sample through the discriminant model to obtain the predicted style of the training sample; constructing a style classification loss function based on the predicted style and the real style of the training sample; constructing a authenticity loss function based on the difference between the prediction probability and the real probability of the training sample; performing weighted summation processing on the authenticity loss function and the style classification loss function, and using the result of the weighted summation processing as the loss function of the discriminant model.

[0163] For example, the style classification loss is a multi-class cross-entropy loss. In the discriminant model optimization stage, the discriminant model's good style classification ability is trained through real images, so that the generative model can correctly classify images, thereby helping the generative model obtain character images that meet the target style.

[0164] The following combines the exemplary applications and implementations of the terminal provided in the embodiments of the present application to illustrate the artificial intelligence-based font library processing method provided in the embodiments of the present application. Refer to Figure 6 , Figure 6 is a schematic flowchart of the artificial intelligence-based font library processing method provided in the embodiments of the present application, and will be described in combination with the steps shown in Figure 6 .

[0165] In step 201, a font library is presented, and the font library includes at least one character of a first style.

[0166] For example, after the user opens the client for font library completion in the terminal, the client will present the font library, and may also present at least one character of the first style in the font library. For example, the character "Xing" in the regular script font library. When the user confirms that the font library is missing characters or there are damaged characters, the user can determine that the font library needs to be updated through operations such as clicking and swiping.

[0167] Among them, for the case of generating a new font library, there must be at least one character in the font library, that is, the character of the first style included in the font library is a seed, which is used to provide style information; for the case of completing the font library, the font library includes multiple characters of the first style, but some characters are still missing, and it is necessary to generate missing characters or damaged characters of the first style based on characters of other styles to complete the font library.

[0168] In step 202 , in response to an update operation on the character library, a generation model is called to generate a character image of the first style based on a character image of the second style of the target character.

[0169] For example, after receiving the update operation for the font library, the client calls the generation model to generate a character image of the first style based on a character image of the second style of a target character, wherein the target character includes at least one of a missing character and a damaged character in the font library.

[0170] For example, character encoding processing is performed on the second style character image of the target character to obtain the target character feature of the character image, the first style and the target character feature are fused to obtain the character fusion feature, the character fusion feature is decoded to obtain the first style character image of the target character, and the font library is updated through the first style character image of the target character.

[0171] In step 203 , a character library updated based on the character image of the first style of the target character is presented.

[0172] Below, an exemplary application of the embodiment of the present application in a practical application scenario will be described.

[0173] Generating a specific style of Chinese character font usually requires artists to design a unified corresponding writing style for all characters, which is very time-consuming and labor-intensive. This is especially true when generating fonts in the specific handwriting style of public figures, where the time cost issue is even more significant. In addition, due to the huge size of the Chinese character system, many fonts usually only contain 6,000 to 7,000 commonly used characters, and the Chinese characters contained in different fonts are not exactly the same. In some deep learning projects (such as optical character recognition (OCR), Chinese handwriting recognition, and auxiliary font design), a large amount of training data is generated from existing handwritten fonts. The problem of incomplete fonts will have a certain impact on the project. The huge labor cost of generating a complete font also makes it almost impossible to collect small amounts of data on specific people in actual projects to optimize the model in a targeted manner.

[0174] The applicant discovered the following problems during the implementation of the embodiments of the present application: 1) Asking artists or designated public figures to design each character, or asking a large amount of manual handwriting data to complete data for projects such as OCR, is time-consuming and labor-intensive, and the labor cost is very high; 2) Using a GAN structure to convert one font image to another target font image, because the font learned by the GAN structure is relatively simple, it performs well on characters that have been seen (i.e., training set), but performs poorly for characters that have not been seen, and cannot generate a completely new font library; 3) Using a GAN structure to perform multi-font conversion and using a U-shaped network structure to ensure network convergence, the U-shaped network structure provides shallow features for the generation model. Although it can help the network converge, it also limits the transformation space of the font image. As a result, although this solution works well when the font shape changes slightly (the overall shape does not change but the stroke thickness changes), it does not work well when the font shape changes significantly (for example, Songti becomes cursive). However, if the existing network structure does not adopt a U-shaped structure in multi-font conversion, it cannot ensure network convergence.

[0175] To address the above issues, the present invention provides a solution for automatic completion of Chinese character fonts based on deep adversarial learning (GAN). When a Chinese character font library is incomplete, this solution can learn the style of the font based on the existing Chinese characters in the font library, and automatically complete the missing Chinese characters in the font library of the same style. This greatly shortens the creative time of artists or designers, and eliminates the need to design a large number of Chinese fonts one by one, thereby improving efficiency and reducing labor costs. It also provides more data support for projects that use fonts to generate data, enriches data distribution, and improves project effectiveness.

[0176] like Figure 7 As shown, the GAN network framework of the embodiment of the present application consists of a generative model and a discriminative model. By alternately optimizing the discriminative model and the generative model, the generative model is prompted to output high-quality images of characters in a specified font. The generative model includes three parts: a character encoder (encoding model, used for character feature extraction), a font encoder, and a font character decoder (decoding model).

[0177] In terms of training strategy, the embodiment of the present application uses multi-font mixed-style adversarial training to fully utilize existing data to allow the network to fully learn the essential characteristics of fonts and characters, thereby enabling the network to better generate target font character images that it has never seen before, even font images that the network has never seen before. At the same time, the embodiment of the present application also adopts a progressive training strategy, training font character decoders from small resolution to large resolution, ensuring that the network can stably converge and output high-quality target font images.

[0178] Regarding the character feature extraction part, the embodiment of the present application adopts a pre-trained handwriting recognition model (character encoder). The handwriting recognition network is used to extract the basic morphological features of the same characters in different fonts, which can provide a good initial state for subsequent multi-font mixed style adversarial training, so that the generation network can focus on the encoding and reconstruction of character features in the early stage of progressive training.

[0179] Among them, the embodiment of the present application uses a pre-trained handwriting recognition model to extract character features in the image. The handwriting recognition model is a classification model, and the training goal of the classification model is to classify the input character images of various fonts into corresponding character categories. The trained handwriting recognition model can well strip the font information of the input character image and extract the glyph information unique to the character in the character image. This part abstracts the character information into high-dimensional feature information, so that the subsequent generation model can learn the basic framework information of the target character, that is, the strokes and arrangement of the target character. The handwriting recognition model fixes the parameters in the early stage of adversarial training, reduces the parameters that the network needs to optimize, thereby reducing the difficulty of optimizing the generation model and ensuring the convergence of the entire model.

[0180] Regarding the font encoding part, the embodiment of the present application uses an autoencoder to automatically learn a suitable font vector for all style images. The font information is first encoded using the autoencoder method. The font character decoder then determines which target font should be selected based on the input of the autoencoder. The font character decoder then restores the font style information, namely the target font's stroke weight, slope, connection method, etc. The font encoding part can also use a font encoding network to provide more detailed font information.

[0181] Regarding the font character decoder part, the embodiment of the present application adopts a progressive training method to train the font character feature deconstruction method layer by layer from small resolution to large resolution, so that the decoder can still converge stably without receiving the shallow features (font features) of the input character image (that is, without using the U-shaped network structure), thereby eliminating the influence of the font features contained in the shallow features of the input character image and generating a high-quality target font character image.

[0182] Regarding the discriminant model, the present embodiment adopts a multi-task discrimination scheme. It takes a font image as input and outputs whether the font image is a real image or a fake image generated by the generative model. Simultaneously, the discriminant model makes a font classification judgment based on the font in the input image. Multiple discriminators can also be used to correspond to the multi-task discrimination work.

[0183] In the discriminant model optimization stage, the optimization goal of the discriminant model is to correctly classify whether the input image is a real image, and to correctly judge which font the input is a real image. For the fake image generated by the generative model, there is no need to judge its font at this stage.

[0184] To fully utilize the different features missing from different fonts, this embodiment of the application uses multi-font mixed-style adversarial training. After multi-font mixed-style adversarial training, although the network in this embodiment of the application does not know what the target character should look like in the target font, the network can create an image of the target character in the target font by using the structural features of the target character in other fonts and the stylistic features of other characters in the target font.

[0185] Regarding progressive training, in order to eliminate the influence of font features brought by shallow features, the embodiment of the present application does not adopt a U-shaped network, so the input of the character font decoder is all features that have been abstracted through high-dimensionality. However, the target image should be a character image containing sufficient details, which is a great challenge for the convergence of the generative model. Therefore, in order to ensure stable convergence of the network, the embodiment of the present application adopts a progressive training strategy. Among them, the progressive training strategy is specifically as follows:

[0186] When training the network, first train the small-resolution character images, and then use the model weights trained at small resolution to assist the next stage of large-resolution model weight training. Figure 8A Shows the plateau period of small resolution (16*16 resolution), Figure 8B Showing the resolution growth transition period, Figure 8C Shows the plateau period of large resolution (32*32 resolution), through Figure 8B The growth transition period shown will be assisted by Figure 8A The 16*16 resolution stage shown transitions to Figure 8C In the 32*32 resolution stage shown in the figure, the weight of α is gradually increased during the continuous iteration process, so that the model can smoothly transition from a highly abstract stage to a more detailed stage.

[0187] This training approach gives the network a relatively simple task in the early stages of training, then gradually refines the details of the generated image. This simplifies the search space for network parameters and helps the network converge stably to the ideal state. After the growth transition period, when the network can output high-quality, high-resolution character images, the initial parameter training of the character encoder is relaxed, allowing the network to learn character features more suitable for image generation, further optimizing the generated character images.

[0188] Regarding the loss function, the loss function of the embodiment of the present application includes four parts: authenticity loss function, font classification loss function (style classification loss function), character encoding loss function and image L1 loss function (image distance loss function). en The formula is shown in formula (1):

[0189] L en =||En(T)-En(De(En(S)+s))||2 (1)

[0190] Among them, T represents the real target image, S represents the input image, s represents the input font, En represents the character encoder, and De represents the glyph font decoder.

[0191] Among them, the image L1 loss function L l1 The formula is shown in formula (2):

[0192] L l1 =||T-De(En(S)+s)||1 (2)

[0193] Among them, the authenticity loss function and the font classification loss function are both provided by the discriminant model. The authenticity loss is an adversarial loss. During the discriminant model optimization stage, the discriminant model is optimized to distinguish between real images and fake images generated by the generative model. During the generative model optimization stage, the generative model is optimized to obtain images that can confuse the discriminant model. The font classification loss is a multi-classification cross-entropy loss. During the discriminant model optimization stage, the discriminant model is trained with real data to have good font (style) classification capabilities. During the generative model optimization stage, the network needs to generate images that can be correctly classified by the generative model, thereby helping the generative model obtain images that meet the target font requirements. Among them, the character encoding loss is added to the training after the progressive training completes all growth transition periods to help the character encoder obtain character features that are more suitable for image generation.

[0194] Figures 9A-9C As shown, an example of the font character image result generated by the embodiment of the present application is given. Figure 8A The character image 901 is the character image input to the network, and the character image 902 is the character image after being converted into a specified font. Figure 9B The character image 903 is the character image input to the network, and the character image 904 is the character image after being converted into a specified font. Figure 9C The character image 905 is the character image input to the network, and the character image 906 is the character image after being converted into a prescribed font.

[0195] In summary, the font image generation solution based on deep adversarial learning proposed in the embodiments of the present application has the following beneficial effects:

[0196] 1) Use the pre-trained handwriting recognition network to extract character features, ensuring that the network can obtain high-quality and relatively fixed character features, helping the network converge;

[0197] 2) We effectively leverage the fact that different Chinese characters in different fonts are not identical to each other by designing multi-font mixed-style adversarial training to generate target font images. This allows the network to learn the appearance of target characters in other reference fonts, even though it has never seen the target characters in the target font. This helps the network generate higher-quality target character images in the target font more efficiently.

[0198] 3) Multi-font mixed-style adversarial training also provides a font encoding component, so that after training is completed, users can manually adjust the font encoding to generate a new font library;

[0199] 4) Using a progressive training method, the network does not need the help of shallow features in the early training stage. While ensuring network convergence, it also eliminates the influence of spatial information (font features) of shallow features, ensuring that the generated font images can have large changes in font shape, not just changes in stroke thickness;

[0200] 5) It can reduce the cost of completing the font library, assist designers in completing the design work of specific fonts, improve the designer's work efficiency and thus reduce labor costs;

[0201] 6) Reduce the cost of generating fonts and realize the function of helping ordinary users generate their own handwritten fonts;

[0202] 7) It can also provide data completion and enrich data distribution for other Chinese character recognition related projects. Especially for projects such as handwriting recognition where data collection costs are extremely high, it can quickly generate a large amount of training data to help projects improve their results.

[0203] So far, the AI-based character processing method provided by the embodiment of the present application has been described in combination with the exemplary application and implementation of the character processing system provided by the embodiment of the present application. The embodiment of the present application also provides a character processing device. In actual applications, the various functional modules in the AI-based character processing device can be collaboratively implemented by hardware resources of electronic devices (such as servers or server clusters), such as computing resources such as processors, communication resources (such as those used to support various communication methods such as optical cables and cellular networks), and memories. Among them, the character processing device ( Figure 2 shows a character library processing device 555 stored in the memory 550, Figure 3The font processing device 655 stored in the memory 650 is shown to be software in the form of programs and plug-ins, for example, software modules designed in programming languages ​​such as C / C++ and Java, application software designed in programming languages ​​such as C / C++ and Java, or dedicated software modules in large software systems, application programming interfaces, plug-ins, cloud services, etc. Different implementation methods are explained below with examples.

[0204] Example 1: The font processing device is a mobile application and module

[0205] The font processing device in the embodiment of the present application can be provided as a software module designed using programming languages ​​such as software C / C++, Java, etc., and embedded in various mobile applications based on systems such as Android or iOS (stored in the storage medium of the mobile terminal as executable instructions and executed by the processor of the mobile terminal), so as to directly use the computing resources of the mobile terminal itself to complete the relevant information recommendation tasks, and transmit the processing results to a remote server regularly or irregularly through various network communication methods, or save them locally on the mobile terminal.

[0206] Example 2: The font processing device is a server application and platform

[0207] The font processing device in the embodiment of the present application can be provided as an application software designed using programming languages ​​such as C / C++ and Java, or a dedicated software module in a large software system, running on the server side (stored in the storage medium on the server side in the form of executable instructions and run by the processor on the server side), and the server uses its own computing resources to complete related font processing tasks.

[0208] The embodiments of the present application can also be provided as a distributed, parallel computing platform composed of multiple servers, equipped with a customized, easy-to-interact network (Web) interface or other user interfaces (UI, User Interface), to form an information recommendation platform (for recommendation lists) for use by individuals, groups or units.

[0209] Example 3: The font processing device is a server-side application program interface (API) and plug-in

[0210] The font processing device in the embodiment of the present application can be provided as a server-side API or plug-in for users to call to execute the artificial intelligence-based font processing method of the embodiment of the present application, and embedded in various applications.

[0211] Example 4: The font processing device is a mobile device client API and plug-in

[0212] The font processing device in the embodiment of the present application can be provided as an API or plug-in on the mobile device for users to call to execute the artificial intelligence-based font processing method in the embodiment of the present application.

[0213] Example 5: The font processing device is a cloud-based open service

[0214] The font processing device in the embodiment of the present application can provide a font processing cloud service developed for users, allowing individuals, groups or units to process fonts.

[0215] The character library processing device 555 includes a series of modules, including a determination module 5551, an encoding module 5552, a fusion module 5553, a decoding module 5554, and a training module 5555. The following further describes how the various modules in the character library processing device 555 provided in the embodiment of the present application cooperate to implement the character library processing solution.

[0216] The determination module 5551 is used to determine the first style of the font library and the target character of the font library; the encoding module 5552 is used to perform character encoding processing on the character image of the second style of the target character to obtain the target character feature of the character image; the fusion module 5553 is used to perform fusion processing on the first style and the target character feature to obtain a character fusion feature; the decoding module 5554 is used to decode the character fusion feature to obtain the character image of the first style of the target character, and update the font library through the character image of the first style of the target character.

[0217] In some embodiments, the encoding module 5552 is also used to perform feature extraction processing on the second style character image of the target character to obtain a feature map of the character image; remove the style space information in the feature map, and use the feature map with the style space information removed as the target character feature of the character image.

[0218] In some embodiments, the fusion module 5553 is further configured to perform encoding processing on the first style to obtain encoding features of the first style; and perform splicing processing on the encoding features of the first style and the target character features to obtain the character fusion features.

[0219] In some embodiments, the fusion module 5553 is also used to map the first style and the target character features to obtain an intermediate feature vector containing the first style and the target character; and perform affine transformation on the intermediate feature vector containing the first style and the target character to obtain the character fusion feature.

[0220] In some embodiments, the decoding process is implemented through a decoding model, which includes multiple cascaded decoding layers, and the multiple cascaded decoding layers are obtained through progressive training; the decoding module 5554 is also used to perform cascaded upsampling decoding processing on the character fusion features through the multiple cascaded decoding layers, and use the decoding result output by the last decoding layer as the first style character image of the target character.

[0221] In some embodiments, the progressive training is achieved through the following process: performing cascaded upsampling decoding processing on the character fusion features through the multiple cascaded decoding layers; based on the weights of the multiple cascaded decoding layers, performing weighted summation processing on the decoding results output by the multiple cascaded decoding layers respectively, and using the result of the weighted summation processing as the false character image sample; wherein, the weight of the last decoding layer gradually increases as the number of times the generation model is updated increases, and the weights of other decoding layers gradually decrease as the number of times the generation model is updated increases; wherein, the character fusion feature is obtained by the encoding model performing character encoding processing on the real character image sample.

[0222] In some embodiments, the character encoding processing and the character decoding processing are implemented by a generative model in a generative adversarial network, and the generative adversarial network also includes a discriminant model; the device 555 also includes: a training module 5555, for training the generative adversarial network in the following manner: initializing the generative model and the discriminant model, and alternately performing the following training tasks: based on a real first character image sample and a real second character image sample, training the generative model so that the generative network generates a false character image sample based on the real first character image sample; wherein the real first character image sample includes a target character sample of a first style sample, and the real second image sample and the false character image sample include a target character sample of a second style sample; based on the real second character image sample and the false character image sample, training the discriminant model so that the discriminant model distinguishes between real character image samples and false character image samples; wherein, when training the generative model, the parameters of the discriminant model are fixed unchanged; when training the discriminant model, the parameters of the generative model are fixed unchanged.

[0223] In some embodiments, the generation model includes an encoding model and a decoding model; the training module 5555 is also used to perform character encoding processing on the real first character image sample through the encoding model to obtain the target character sample feature of the real first character image sample; fuse the first style sample and the target character sample feature to obtain a character fusion feature; decode the character fusion feature through the decoding model to generate the false character image sample; construct the loss function of the generation model based on the real second character image sample and the false character image sample; and update the parameters of the generation model based on the loss function of the generation model.

[0224] In some embodiments, when the weight of the last decoding layer is less than 1, the training module 5555 is also used to obtain an image distance loss function based on the distance between the real second character image sample and the false character image sample, and use the image distance loss function as the loss function of the generation model.

[0225] In some embodiments, when the weight of the last decoding layer is equal to 1, the training module 5555 is also used to perform character encoding processing on the false character image sample through the encoding model to obtain the target character sample features of the false character image sample; perform character encoding processing on the real second character image sample through the encoding model to obtain the target character sample features of the real second character image sample; construct a character encoding loss function based on the target character sample features of the false character image sample and the target character sample features of the real second character image sample; perform weighted summation processing on the image distance loss function and the character encoding loss function, and use the result of the weighted summation processing as the loss function of the generation model.

[0226] In some embodiments, the training module 5555 is also used to generate the false character image sample through the generation model; use the real second character image sample and the false character image sample as training samples, and determine the predicted probability that the training sample belongs to the real character image through the discriminant model; construct the loss function of the discriminant model based on the difference between the predicted probability and the true probability of the training sample; and update the parameters of the discriminant model based on the loss function of the discriminant model.

[0227] In some embodiments, the training module 5555 is also used to, when the predicted probability represents that the training sample is a real character image, perform style classification processing on the training sample through the discriminant model to obtain the predicted style of the training sample; construct a style classification loss function based on the predicted style and the true style of the training sample; construct an authenticity loss function based on the difference between the predicted probability and the true probability of the training sample; perform weighted summation processing on the authenticity loss function and the style classification loss function, and use the result of the weighted summation processing as the loss function of the discriminant model.

[0228] In some embodiments, the determination module 5551 is further used to determine the number of character images in the candidate character library; when the number of character images is less than the image number threshold, the candidate character library is used as the character library, and the style of the character library is used as the first style.

[0229] In some embodiments, the determination module 5551 is further used to determine a candidate character library that is different from the first style; screen the character images in the candidate character library to obtain a character image containing the target character; wherein the character image containing the target character does not exist in the character library.

[0230] In some embodiments, the determination module 5551 is further used to determine the frequency of selection of candidate character images in the candidate character library; when the selection frequency is greater than a selection frequency threshold, the candidate character image is used as the character image containing the target character.

[0231] The font processing device 655 includes a series of modules, including a presentation module 6551 and a generation module 6552. The following further describes how the various modules in the font processing device 655 provided in the embodiment of the present application cooperate to implement the font processing solution.

[0232] A presentation module 6551 is used to present a font library, which includes at least one character in a first style; a generation module 6552 is used to call a generation model in response to an update operation on the font library to generate a character image in the first style based on a character image in the second style of a target character; the presentation module 6551 is also used to present the font library updated based on the character image in the first style of the target character; wherein the target character includes at least one of a missing character and a damaged character in the font library.

[0233] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the artificial intelligence-based character library processing method described in the present invention.

[0234] The embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the artificial intelligence-based character library processing method provided by the embodiment of the present application, for example, Figures 4A-4C The artificial intelligence-based character library processing method shown, or Figure 6 The character library processing method based on artificial intelligence is shown.

[0235] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0236] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0237] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (for example, files storing one or more modules, subroutines, or code portions).

[0238] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0239] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A character library processing method based on artificial intelligence, characterized in that: include: determining a first style of a character library and a target character of the character library; performing character encoding processing on the character image of the target character in the second style to obtain target character features of the character image; fusing the first style and the target character feature to obtain a character fusion feature; Decoding the character fusion feature to obtain a character image of the target character in a first style, and updating the character library using the character image of the target character in the first style; The decoding process is implemented by a decoding model, which includes multiple cascaded decoding layers, and the multiple cascaded decoding layers are obtained by progressive training, and the progressive training is implemented by the following process: Performing cascade upsampling decoding processing on the character fusion features obtained by performing character encoding processing on the real character image samples through the multiple cascaded decoding layers; Based on the weights of the plurality of cascaded decoding layers, performing weighted summation processing on the decoding results respectively output by the plurality of cascaded decoding layers, and using the result of the weighted summation processing as a false character image sample; Among them, the weight of the last decoding layer gradually increases as the number of times the generation model is updated increases, and the weights of other decoding layers gradually decrease as the number of times the generation model is updated increases. The generation model is used to implement the character encoding processing and the decoding processing.

2. The method according to claim 1, characterized in that The performing character encoding processing on the character image of the target character in the second style to obtain target character features of the character image includes: performing feature extraction processing on the character image of the second style of the target character to obtain a feature map of the character image; The style space information in the feature map is removed, and the feature map with the style space information removed is used as the target character feature of the character image.

3. The method according to claim 1, characterized in that The fusing the first style and the target character feature to obtain a character fusion feature includes: performing encoding processing on the first style to obtain encoding features of the first style; The encoding feature of the first style and the target character feature are concatenated to obtain the character fusion feature.

4. The method according to claim 1, wherein The decoding process of the character fusion feature to obtain a character image of the target character in the first style includes: The character fusion feature is subjected to cascade upsampling decoding processing through the multiple cascaded decoding layers, and The decoding result output by the last decoding layer is used as the character image of the first style of the target character.

5. The method according to any one of claims 1 to 4, characterized in that The character encoding process and the decoding process are implemented by a generative model in a generative adversarial network, and the generative adversarial network also includes a discriminative model; The method further comprises: The generative adversarial network is trained as follows: Initialize the generative model and the discriminative model, and perform the following training tasks alternately: Based on a real first character image sample and a real second character image sample, training the generation model so that the generation network generates a false character image sample based on the real first character image sample; The real first character image sample includes a target character sample of a first style sample, and the real second image sample and the false character image sample include a target character sample of a second style sample; Based on the real second character image sample and the false character image sample, training the discriminant model so that the discriminant model can distinguish between the real character image sample and the false character image sample; When the generative model is trained, the parameters of the discriminative model are fixed; when the discriminative model is trained, the parameters of the generative model are fixed.

6. The method according to claim 5, characterized in that The generation model includes an encoding model and a decoding model; The training of the generative model based on the real first character image sample and the real second character image sample includes: Performing character encoding processing on the real first character image sample by using the encoding model to obtain target character sample features of the real first character image sample; Fusing the first style sample and the target character sample features to obtain a character fusion feature; Decoding the character fusion feature through the decoding model to generate the false character image sample; Constructing a loss function of the generation model based on the real second character image sample and the false character image sample; Parameters of the generative model are updated based on the loss function of the generative model.

7. The method according to claim 6, characterized in that When the weight of the last decoding layer is less than 1, constructing the loss function of the generation model based on the real second character image sample and the false character image sample includes: Based on the distance between the real second character image sample and the false character image sample, an image distance loss function is obtained. The image distance loss function is used as the loss function of the generative model.

8. The method according to claim 7, characterized in that When the weight of the last decoding layer is equal to 1, constructing the loss function of the generation model based on the real second character image sample and the false character image sample includes: Performing character encoding processing on the false character image sample by using the encoding model to obtain target character sample features of the false character image sample; Performing character encoding processing on the real second character image sample by using the encoding model to obtain target character sample features of the real second character image sample; constructing a character encoding loss function based on the target character sample features of the false character image sample and the target character sample features of the real second character image sample; A weighted summation process is performed on the image distance loss function and the character encoding loss function, and a result of the weighted summation process is used as the loss function of the generation model.

9. The method according to claim 5, characterized in that The training of the discrimination model based on the real second character image sample and the false character image sample includes: generating the false character image sample by using the generation model; Using the real second character image sample and the false character image sample as training samples, and determining the predicted probability that the training sample belongs to a real character image through the discriminant model; Constructing a loss function of the discriminant model based on the difference between the predicted probability and the true probability of the training sample; The parameters of the discriminant model are updated based on the loss function of the discriminant model.

10. The method according to claim 9, characterized in that The constructing of the loss function of the discriminant model based on the difference between the predicted probability and the true probability of the training sample includes: When the predicted probability indicates that the training sample is a real character image, performing style classification processing on the training sample using the discriminant model to obtain a predicted style of the training sample; Constructing a style classification loss function based on the predicted style and the true style of the training sample; constructing a authenticity loss function based on the difference between the predicted probability and the true probability of the training sample; A weighted summation process is performed on the authenticity loss function and the style classification loss function, and a result of the weighted summation process is used as the loss function of the discriminant model.

11. A character library processing method based on artificial intelligence, characterized in that: The method comprises: presenting a character library, the character library including at least one character of a first style; In response to an update operation on the character library, a generation model is called to generate a character image of the first style based on a character image of the second style of the target character, and presenting the updated character library based on the character image of the first style of the target character; The target character includes at least one of a missing character and a damaged character in the character library, and the character image of the first style is obtained by the artificial intelligence-based character library processing method according to any one of claims 1 to 10.

12. A character library processing device based on artificial intelligence, characterized in that: The device comprises: a determination module, configured to determine a first style of a character library and a target character of the character library; an encoding module, configured to perform character encoding processing on the character image of the target character in the second style to obtain target character features of the character image; a fusion module, configured to fuse the first style and the target character feature to obtain a character fusion feature; a decoding module, configured to decode the character fusion feature to obtain a character image of the target character in a first style, and update the character library using the character image of the target character in the first style; The decoding process is implemented by a decoding model, which includes multiple cascaded decoding layers, and the multiple cascaded decoding layers are obtained by progressive training, and the progressive training is implemented by the following process: Performing cascade upsampling decoding processing on the character fusion features obtained by performing character encoding processing on the real character image samples through the multiple cascaded decoding layers; Based on the weights of the plurality of cascaded decoding layers, performing weighted summation processing on the decoding results respectively output by the plurality of cascaded decoding layers, and using the result of the weighted summation processing as a false character image sample; Among them, the weight of the last decoding layer gradually increases as the number of times the generation model is updated increases, and the weights of other decoding layers gradually decrease as the number of times the generation model is updated increases. The generation model is used to implement the character encoding processing and the decoding processing.

13. An electronic device, characterized in that: The electronic device comprises: a memory for storing executable instructions; The processor is configured to implement the artificial intelligence-based character library processing method according to any one of claims 1 to 11 when executing the executable instructions stored in the memory.

14. A computer-readable storage medium, characterized in that Executable instructions are stored for implementing the artificial intelligence-based character library processing method described in any one of claims 1 to 11 when executed by a processor.

15. A computer program product, characterized in that The computer program product includes computer instructions, and the computer instructions are used to implement the artificial intelligence-based character library processing method according to any one of claims 1 to 11 when executed by a processor.

Images